Introduction
Finite mixture models combine flexible distributional modeling with an interpretable description of population heterogeneity ([19 , 24 ] ). With suitable kernels and sufficiently many components, they can approximate arbitrary densities in L 1 L^1 L 1 ([26 ] ). Their generative interpretation assigns each observation to a latent component, providing a natural basis for clustering. Concretely, a finite mixture has density
p G ( x ) : = ∫ Θ f ( x ∣ θ ) d G ( θ ) = ∑ j = 1 p π j f ( x ∣ θ j ) , x ∈ R d , p_G(x) := \int_{\Theta} f(x \mid\theta)\,dG(\theta) = \sum_{j=1}^{p} \pi_j f(x \mid\theta_j), \qquad x \in\mathbb{R}^{d}, p G ( x ) := ∫ Θ f ( x ∣ θ ) d G ( θ ) = j = 1 ∑ p π j f ( x ∣ θ j ) , x ∈ R d ,
where { f ( ⋅ ∣ θ ) : θ ∈ Θ } \{f(\cdot\mid\theta) : \theta\in\Theta\} { f ( ⋅ ∣ θ ) : θ ∈ Θ } is a family of probability density kernels and G = ∑ j = 1 p π j δ θ j G = \sum_{j=1}^{p}\pi_j\delta_{\theta_j} G = ∑ j = 1 p π j δ θ j is the mixing measure , with distinct atoms θ 1 , … , θ p ∈ Θ \theta_1,\ldots,\theta_p \in\Theta θ 1 , … , θ p ∈ Θ , positive weights π 1 , … , π p \pi_1,\ldots,\pi_p π 1 , … , π p , and ∑ j = 1 p π j = 1 \sum_{j=1}^{p}\pi_j = 1 ∑ j = 1 p π j = 1 . Equivalently, an observation is generated by first selecting a latent component Z Z Z with P ( Z = j ) = π j \mathbb{P}(Z=j)=\pi_j P ( Z = j ) = π j and then drawing X ∣ Z = j X \mid Z=j X ∣ Z = j from f ( ⋅ ∣ θ j ) f(\cdot\mid\theta_j) f ( ⋅ ∣ θ j ) . The mixing measure thus encodes both the component parameters and their relative prevalence in the population.
Given independent observations X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n from p G ∗ p_{G_*} p G ∗ , two related but distinct estimation problems arise. The first is density estimation , which seeks to recover the observable distribution p G ∗ p_{G_*} p G ∗ . The second is mixing-measure estimation , which seeks to recover the latent distribution G ∗ G_* G ∗ , including the component parameters and their mixing proportions. The former describes the distribution of the observed data, whereas the latter describes the underlying population heterogeneity.
Related literature
Maximum likelihood estimation is a standard approach to estimating both mixture densities and their mixing measures [17 , 19 , 24 ] . For a class G \mathcal{G} G of mixing measures, the maximum likelihood estimator is defined by
G ^ n ∈ arg max G ∈ G ∑ ℓ = 1 n log p G ( X ℓ ) . \widehat{G}_n \in\underset{G\in\mathcal{G}}{\operatorname{arg\,max}}\sum_{\ell=1}^{n}\log p_G(X_\ell). G n ∈ G ∈ G arg max ℓ = 1 ∑ n log p G ( X ℓ ) .
When the model is restricted to a single component, estimation reduces to ordinary parametric maximum likelihood. Under standard regularity conditions, the estimator θ ^ n \widehat{\theta}_n θ n satisfies
n ( θ ^ n − θ ∗ ) ⇝ N ( 0 , I ( θ ∗ ) − 1 ) . \sqrt{n}(\widehat{\theta}_n-\theta_*) \rightsquigarrow N(0,I(\theta_*)^{-1}). n ( θ n − θ ∗ ) ⇝ N ( 0 , I ( θ ∗ ) − 1 ) .
where I ( θ ∗ ) I(\theta_*) I ( θ ∗ ) is the Fisher information matrix [31 ] . For example, in a Gaussian location model with known positive definite covariance Σ \Sigma Σ , the maximum likelihood estimator is the sample mean, and n ( μ ^ n − μ ∗ ) ⇝ N d ( 0 , Σ ) \sqrt{n}(\widehat{\mu}_n-\mu_*) \rightsquigarrow N_d(0,\Sigma) n ( μ n − μ ∗ ) ⇝ N d ( 0 , Σ ) . Asymptotic normality also holds for exactly specified finite mixtures, after aligning component labels, under suitable regularity conditions and nonsingular Fisher information [28 Theorem 3.1] .
A prominent framework for studying mixing-measure estimation, developed by Nguyen [27 ] and extended by Ho and Nguyen [13 ] , connects density estimation to the recovery of latent component parameters through inverse inequalities. The density-estimation step uses empirical-process arguments and bracketing entropy to control the complexity of the mixture density class [32 , 30 ] . These arguments yield bounds for Hellinger distance of the form d H ( p G ^ n , p G ∗ ) = O P ( ε n ) d_H(p_{\widehat{G}_n},p_{G_*})=O_P(\varepsilon_n) d H ( p G n , p G ∗ ) = O P ( ε n ) , where ε n \varepsilon_n ε n is determined by the relevant entropy bounds. To measure discrepancy between mixing measures, this framework uses Wasserstein distances [33 ] . For r ≥ 1 r\geq1 r ≥ 1 , define
W r ( G , G ′ ) : = { inf γ ∈ Γ ( G , G ′ ) ∫ Θ × Θ ∥ θ − θ ′ ∥ r d γ ( θ , θ ′ ) } 1 / r . W_r(G,G'):=\left\{\inf_{\gamma\in\Gamma(G,G')}\int_{\Theta\times\Theta}\lVert\theta-\theta'\rVert^r\,\mathrm{d}\gamma(\theta,\theta')\right\}^{1/r}. W r ( G , G ′ ) := { γ ∈ Γ ( G , G ′ ) inf ∫ Θ × Θ ∥ θ − θ ′ ∥ r d γ ( θ , θ ′ ) } 1/ r .
where Γ ( G , G ′ ) \Gamma(G,G') Γ ( G , G ′ ) denotes the set of probability measures with marginals G G G and G ′ G' G ′ . Inverse inequalities then relate density discrepancies to transportation costs. In particular, under suitable smoothness and strong identifiability conditions, one obtains locally around a fixed G ∗ G_* G ∗
d H ( p G , p G ∗ ) ≥ c W 1 ( G , G ∗ ) in the exactly specified setting, d_H(p_G,p_{G_*})\geq cW_1(G,G_*) \qquad\text{in the exactly specified setting,} d H ( p G , p G ∗ ) ≥ c W 1 ( G , G ∗ ) in the exactly specified setting,
and
d H ( p G , p G ∗ ) ≥ c W 2 2 ( G , G ∗ ) in the over-specified setting, d_H(p_G,p_{G_*})\geq cW_2^2(G,G_*) \qquad\text{in the over-specified setting,} d H ( p G , p G ∗ ) ≥ c W 2 2 ( G , G ∗ ) in the over-specified setting,
for some constant c > 0 c>0 c > 0 [27 , 13 ] . The first inequality applies when the fitted and true numbers of components agree, under first-order strong identifiability. The second applies when additional components are permitted, under second-order strong identifiability.
These inequalities translate a density rate ε n \varepsilon_n ε n into mixing-measure rates ε n \varepsilon_n ε n under W 1 W_1 W 1 and ε n 1 / 2 \varepsilon_n^{1/2} ε n 1/2 under W 2 W_2 W 2 , respectively. Thus, a root-n n n density rate yields the familiar n − 1 / 2 n^{-1/2} n − 1/2 and n − 1 / 4 n^{-1/4} n − 1/4 rates; logarithmic factors in the density bound propagate accordingly. The change in transportation order reflects the possibility of first-order cancellation: several fitted components near a true atom may have weighted displacements that sum to zero, leaving a second-order density discrepancy.
Subsequent work refines this framework in several directions. Manole and Ho [21 ] develop stronger loss functions that distinguish the convergence rates of individual fitted components, revealing heterogeneity that a single Wasserstein loss can obscure. Heinrich and Kahn [12 ] use coarse-graining trees to organize component configurations across multiple scales and establish uniform minimax rates. More recently, Nguyen et al. [25 ] quantify how separation between component parameters affects inverse inequalities and estimation rates in finite Gaussian location mixtures. Together, these developments clarify how component configurations govern estimation accuracy.
Another prominent line of work studies mixing-measure estimation through moment-based methods [34 , 6 ] . By relating moment discrepancies to Wasserstein distance, these methods yield minimax-optimal estimation rates for Gaussian location mixtures in both univariate and high-dimensional settings without requiring component separation.
Contributions
Existing work establishes asymptotic normality of likelihood estimators for exactly specified mixtures under suitable regularity conditions [28 ] , together with extensive convergence-rate theory for overspecified mixtures [2 , 27 , 13 , 21 ] . These results leave a distinct question: what is the limiting distribution of the mixing-measure estimation error when redundant components are permitted? Our goal is to answer this question for the global maximum likelihood estimator
G ^ n ∈ arg max G ∈ O k ( Θ ) ℓ n ( G ) , ℓ n ( G ) : = ∑ r = 1 n log p G ( X r ) , \widehat{G}_n \in\underset{G \in\mathcal{O}_k(\Theta)}{\operatorname{arg\,max}} \ell_n(G), \qquad\ell_n(G) := \sum_{r=1}^{n} \log p_G(X_r), G n ∈ G ∈ O k ( Θ ) arg max ℓ n ( G ) , ℓ n ( G ) := r = 1 ∑ n log p G ( X r ) ,
where O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) denotes the mixing measures with at most k k k atoms and the true mixing measure G ∗ G_\ast G ∗ has p ≤ k p \le k p ≤ k distinct atoms. In the second-order strongly identifiable regime, existing inverse inequalities translate root-n n n density convergence into the fourth-root mixing-measure rate W 2 ( G ^ n , G ∗ ) = O P ( n − 1 / 4 ) W_2(\widehat{G}_n,G_\ast)=O_{\mathbb{P}}(n^{-1/4}) W 2 ( G n , G ∗ ) = O P ( n − 1/4 ) [27 , 13 ] , but do not determine its limiting distribution. Under second-order strong identifiability and the stated regularity conditions, we establish a weak limit for n 1 / 4 W 2 ( G ^ n , G ∗ ) n^{1/4}W_2(\widehat{G}_n,G_\ast) n 1/4 W 2 ( G n , G ∗ ) . Our analysis connects the local likelihood geometry to the leading transportation cost through two geometric results.
Our starting point is the scaled relative-density error
h n : = n p G ^ n − p ∗ p ∗ , p ∗ : = p G ∗ . h_n := \sqrt{n}\,\frac{p_{\widehat{G}_n}-p_\ast}{p_\ast}, \qquad p_\ast:= p_{G_\ast}. h n := n p ∗ p G n − p ∗ , p ∗ := p G ∗ .
To describe its possible limits, we characterize the attainable tangent set
T k : = { h ∈ L 2 ( P ∗ ) : ∃ t m ↓ 0 and G m ∈ O k ( Θ ) : p G m − p ∗ t m p ∗ ⟶ h in L 2 ( P ∗ ) } . \mathcal{T}_k := \left\{ h \in L^2(P_*) : \exists t_m \downarrow0 \text{ and } G_m \in\mathcal{O}_k(\Theta) : \frac{p_{G_m}-p_*}{t_m p_*} \longrightarrow h \text{ in } L^2(P_*) \right\}. T k := { h ∈ L 2 ( P ∗ ) : ∃ t m ↓ 0 and G m ∈ O k ( Θ ) : t m p ∗ p G m − p ∗ ⟶ h in L 2 ( P ∗ ) } .
Writing G ∗ = ∑ i = 1 p π i ∗ δ θ i ∗ G_* = \sum_{i=1}^{p} \pi_i^* \delta_{\theta_i^*} G ∗ = ∑ i = 1 p π i ∗ δ θ i ∗ , we show that every h ∈ T k h \in\mathcal{T}_k h ∈ T k admits a unique representation
h ( x ) = 1 p ∗ ( x ) [ ∑ i = 1 p { α i f θ i ∗ ( x ) + β i T ∇ θ f θ i ∗ ( x ) + 1 2 tr ( C i ∇ θ 2 f θ i ∗ ( x ) ) } + ∫ Θ f θ ( x ) d ν ( θ ) ] , (1) h(x) = \frac{1}{p_*(x)}\left[\sum_{i=1}^{p}\left\{\alpha_i f_{\theta_i^*}(x) + \beta_i^{\mathsf T}\nabla_{\theta}f_{\theta_i^*}(x) + \frac{1}{2}\operatorname{tr}\left(C_i\nabla_{\theta}^2 f_{\theta_i^*}(x)\right)\right\} + \int_{\Theta} f_{\theta}(x)\,d\nu(\theta)\right], \tag*{(1)} h ( x ) = p ∗ ( x ) 1 [ i = 1 ∑ p { α i f θ i ∗ ( x ) + β i T ∇ θ f θ i ∗ ( x ) + 2 1 tr ( C i ∇ θ 2 f θ i ∗ ( x ) ) } + ∫ Θ f θ ( x ) d ν ( θ ) ] , (1)
where f θ : = f ( ⋅ ∣ θ ) f_{\theta} := f(\cdot\mid\theta) f θ := f ( ⋅ ∣ θ ) . Here, α i \alpha_i α i and β i \beta_i β i describe changes in mass and location, the positive-semidefinite matrices C i C_i C i describe local splitting, and the positive atomic measure ν \nu ν , supported away from the true atoms, describes nonlocal components with vanishing weights. These quantities satisfy
∑ i = 1 p α i + ν ( Θ ) = 0 , q + ∑ i = 1 p rank ( C i ) ≤ k − p , q : = ∣ supp ( ν ) ∣ . \sum_{i=1}^{p}\alpha_i+\nu(\Theta)=0,\qquad q+\sum_{i=1}^{p}\operatorname{rank}(C_i)\leq k-p,\qquad q:=|\operatorname{supp}(\nu)|. i = 1 ∑ p α i + ν ( Θ ) = 0 , q + i = 1 ∑ p rank ( C i ) ≤ k − p , q := ∣ supp ( ν ) ∣.
The latter constraint is necessary and sufficient: it specifies how the additional fitted components can be allocated between local splitting and nonlocal perturbations.
This characterization leads to the distributional limit of the relative-density estimator. For h n , G : = n p G − p ∗ p ∗ h_{n,G}:=\sqrt{n}\frac{p_G-p_*}{p_*} h n , G := n p ∗ p G − p ∗ , a local expansion of the log-likelihood gives
ℓ n ( G ) − ℓ n ( G ∗ ) = 1 n ∑ r = 1 n h n , G ( X r ) − 1 2 n ∑ r = 1 n h n , G ( X r ) 2 + R n , G . \ell_n(G)-\ell_n(G_*)=\frac{1}{\sqrt{n}}\sum_{r=1}^{n}h_{n,G}(X_r)-\frac{1}{2n}\sum_{r=1}^{n}h_{n,G}(X_r)^2+R_{n,G}. ℓ n ( G ) − ℓ n ( G ∗ ) = n 1 r = 1 ∑ n h n , G ( X r ) − 2 n 1 r = 1 ∑ n h n , G ( X r ) 2 + R n , G .
Since P ∗ h n , G = 0 P_*h_{n,G}=0 P ∗ h n , G = 0 , this becomes
ℓ n ( G ) − ℓ n ( G ∗ ) = G n h n , G − 1 2 ∥ h n , G ∥ ∗ 2 + R ~ n , G , \ell_n(G)-\ell_n(G_*)=\mathbb{G}_n h_{n,G}-\frac{1}{2}\|h_{n,G}\|_*^2+\widetilde{R}_{n,G}, ℓ n ( G ) − ℓ n ( G ∗ ) = G n h n , G − 2 1 ∥ h n , G ∥ ∗ 2 + R n , G ,
where G n : = n ( n − 1 ∑ r = 1 n δ X r − P ∗ ) \mathbb{G}_n:=\sqrt{n}(n^{-1}\sum_{r=1}^{n}\delta_{X_r}-P_*) G n := n ( n − 1 ∑ r = 1 n δ X r − P ∗ ) and ∥ h ∥ ∗ 2 : = ∫ h 2 d P ∗ \|h\|_*^2:=\int h^2\,dP_* ∥ h ∥ ∗ 2 := ∫ h 2 d P ∗ . We establish uniform control of the remainder on the relevant localized classes and a functional central limit theorem for the empirical process. The limiting likelihood criterion is therefore
Q ( h ) : = W ( h ) − 1 2 ∥ h ∥ ∗ 2 , h ∈ T k , Q(h):=\mathbb{W}(h)-\frac{1}{2}\|h\|_*^2,\qquad h\in\mathcal{T}_k, Q ( h ) := W ( h ) − 2 1 ∥ h ∥ ∗ 2 , h ∈ T k ,
where W \mathbb{W} W is a centered Gaussian process with covariance Cov ( W ( h ) , W ( g ) ) = ∫ h g d P ∗ \operatorname{Cov}(\mathbb{W}(h),\mathbb{W}(g))=\int hg\,dP_* Cov ( W ( h ) , W ( g )) = ∫ h g d P ∗ . Together with convergence of the attainable sets, tightness, and almost sure uniqueness of the limiting maximizer, a likelihood argmax theorem yields
h ^ n ⇝ H † , H † : = arg max h ∈ T k { W ( h ) − 1 2 ∥ h ∥ ∗ 2 } . (2) \widehat{h}_n \rightsquigarrow H^\dagger,\qquad H^\dagger:=\underset{h\in\mathcal{T}_k}{\operatorname{arg\,max}}\left\{\mathbb{W}(h)-\frac{1}{2}\|h\|_*^2\right\}. \tag*{(2)} h n ⇝ H † , H † := h ∈ T k arg max { W ( h ) − 2 1 ∥ h ∥ ∗ 2 } . (2)
Our second geometric result connects this density limit to mixing-measure loss. Consider any sequence satisfying
p G m − p ∗ t m p ∗ ⟶ h ∈ T k in L 2 ( P ∗ ) , t m ↓ 0. \frac{p_{G_m}-p_*}{t_m p_*}\longrightarrow h\in\mathcal{T}_k\quad\text{in }L^2(P_*),\qquad t_m\downarrow0. t m p ∗ p G m − p ∗ ⟶ h ∈ T k in L 2 ( P ∗ ) , t m ↓ 0.
After passing to a subsequence and labeling the fitted atoms, let L i L_i L i contain those converging to θ i ∗ \theta_i^* θ i ∗ , for i = 1 , … , p i=1,\ldots,p i = 1 , … , p , and let L 0 L_0 L 0 contain the remaining, nonlocal atoms. Define the intermediate mixing measure
G ‾ m : = ∑ i = 1 p ( ∑ j ∈ L i π j , m ) δ θ i ∗ + ∑ j ∈ L 0 π j , m δ θ j , m , \overline{G}_m := \sum_{i=1}^{p}\left(\sum_{j\in L_i}\pi_{j,m}\right)\delta_{\theta_i^*}+\sum_{j\in L_0}\pi_{j,m}\delta_{\theta_{j,m}}, G m := i = 1 ∑ p j ∈ L i ∑ π j , m δ θ i ∗ + j ∈ L 0 ∑ π j , m δ θ j , m ,
which moves each local atom to its corresponding true center while retaining the nonlocal atoms. We prove the decomposition
W 2 2 ( G m , G ∗ ) = A m + W 2 2 ( G ‾ m , G ∗ ) + o ( t m ) , A m : = ∑ i = 1 p ∑ j ∈ L i π j , m ∥ θ j , m − θ i ∗ ∥ 2 . W_2^2(G_m,G_*)=A_m+W_2^2(\overline{G}_m,G_*)+o(t_m),\qquad A_m:=\sum_{i=1}^{p}\sum_{j\in L_i}\pi_{j,m}\lVert\theta_{j,m}-\theta_i^*\rVert^2. W 2 2 ( G m , G ∗ ) = A m + W 2 2 ( G m , G ∗ ) + o ( t m ) , A m := i = 1 ∑ p j ∈ L i ∑ π j , m ∥ θ j , m − θ i ∗ ∥ 2 .
The tangent representation determines both leading terms:
A m t m → ∑ i = 1 p tr ( C i ) , W 2 2 ( G ‾ m , G ∗ ) t m → M ∗ ( α , ν ) , \frac{A_m}{t_m}\to\sum_{i=1}^{p}\operatorname{tr}(C_i),\qquad\frac{W_2^2(\overline{G}_m,G_*)}{t_m}\to\mathcal{M}_*(\alpha,\nu), t m A m → i = 1 ∑ p tr ( C i ) , t m W 2 2 ( G m , G ∗ ) → M ∗ ( α , ν ) ,
where C ( α , ν ) \mathcal{C}(\alpha,\nu) C ( α , ν ) is the value of a finite optimal-transport problem for the residual mass. Consequently,
W 2 2 ( G m , G ∗ ) t m → Ω ( h ) : = ∑ i = 1 p tr ( C i ) + M ∗ ( α , ν ) . (3) \frac{W_2^2(G_m,G_*)}{t_m}\to\Omega(h):=\sum_{i=1}^{p}\operatorname{tr}(C_i)+\mathcal{M}_*(\alpha,\nu). \tag*{(3)} t m W 2 2 ( G m , G ∗ ) → Ω ( h ) := i = 1 ∑ p tr ( C i ) + M ∗ ( α , ν ) . (3)
Crucially, this limit depends only on the relative-density direction h h h , not on the particular sequence of atoms and weights that realizes it. We call this a sequential first-order expansion . Combining the likelihood argmax theorem (2) with this sequential transportation expansion (3) gives the joint weak convergence
( h ^ n , n W 2 2 ( G ^ n , G ∗ ) ) ⇝ ( H † , Ω ( H † ) ) . \left(\widehat{h}_n,\sqrt{n}W_2^2(\widehat{G}_n,G_*)\right)\rightsquigarrow\left(H^\dagger,\Omega(H^\dagger)\right). ( h n , n W 2 2 ( G n , G ∗ ) ) ⇝ ( H † , Ω ( H † ) ) .
In particular,
n 1 / 4 W 2 ( G ^ n , G ∗ ) ⇝ { Ω ( H † ) } 1 / 2 . n^{1/4}W_2(\widehat{G}_n,G_*)\rightsquigarrow\left\{\Omega(H^\dagger)\right\}^{1/2}. n 1/4 W 2 ( G n , G ∗ ) ⇝ { Ω ( H † ) } 1/2 .
This result identifies the generally non-Gaussian leading distribution of the mixing-measure error, extending the analysis beyond its convergence rate.
Although this weak limit provides a distributional characterization of the estimation error, its dependence on an implicit Gaussian optimization problem and the underlying tangent geometry makes direct implementation difficult. We therefore develop an m m m -out-of-n n n parametric bootstrap and prove its consistency when m → ∞ m\to\infty m → ∞ and m / n → 0 m/n\to0 m / n → 0 , enabling inference without explicitly constructing the limiting law. We assess its finite-sample performance in a toy example and illustrate its use with flow-cytometry data.
Organization of the article
Section 2 introduces the model, assumptions, and Gaussian location example. Section 3 characterizes the attainable likelihood directions and their Gaussian limit. Section 4 establishes the sequential first-order Wasserstein expansion and estimator-level weak law. Section 5 develops bootstrap inference. Sections 6 and present numerical studies and a flow-cytometry illustration, and Section 7 concludes. Detailed proofs appear in the appendices.
Notations and preliminaries
Notations and preliminaries
This section sets out the notation and assumptions used throughout the paper.
For a random element Z Z Z , let L ( Z ) \mathcal{L}(Z) L ( Z ) denote its probability law. We write ⇝ \rightsquigarrow ⇝ and → d \overset{d}{\to} → d interchangeably for convergence in distribution, and → P \overset{P}{\to} → P for convergence in probability. The notation O P ( 1 ) O_P(1) O P ( 1 ) denotes boundedness in probability, while o P ( 1 ) o_P(1) o P ( 1 ) denotes convergence in probability to zero. Let N = { 1 , 2 , … } \mathbb{N}=\{1,2,\ldots\} N = { 1 , 2 , … } and N 0 = N ∪ { 0 } \mathbb{N}_0=\mathbb{N}\cup\{0\} N 0 = N ∪ { 0 } . For a ∈ R a\in\mathbb{R} a ∈ R , write a + = max { a , 0 } a^+=\max\{a,0\} a + = max { a , 0 } and a − = max { − a , 0 } a^-=\max\{-a,0\} a − = max { − a , 0 } . The indicator of a set A A A is denoted by 1 A \mathbf{1}_A 1 A . For vectors, ∥ ⋅ ∥ \|\cdot\| ∥ ⋅ ∥ denotes the Euclidean norm, and v ⊤ v^\top v ⊤ denotes the transpose of v v v . For a matrix A A A , tr ( A ) \operatorname{tr}(A) tr ( A ) and rank ( A ) \operatorname{rank}(A) rank ( A ) denote its trace and rank. We write S + d \mathbb{S}_+^d S + d for the set of symmetric positive-semidefinite d × d d\times d d × d matrices. For nonnegative quantities a a a and b b b , the notation a ≲ b a\lesssim b a ≲ b means a ≤ C b a\le Cb a ≤ C b for a constant C > 0 C>0 C > 0 independent of the relevant asymptotic index. We write a ≍ b a\asymp b a ≍ b when both a ≲ b a\lesssim b a ≲ b and b ≲ a b\lesssim a b ≲ a hold.
Mixing measures and likelihood
In this paper, we consider a compact and convex parameter space Θ ⊂ R d \Theta\subset\mathbb{R}^d Θ ⊂ R d and fix an integer k ≥ 1 k\ge1 k ≥ 1 . Let
O k ( Θ ) = { G = ∑ j = 1 m π j δ θ j : 1 ≤ m ≤ k , π j ≥ 0 , ∑ j = 1 m π j = 1 , θ j ∈ Θ } . \mathcal{O}_k(\Theta)=\left\{G=\sum_{j=1}^{m}\pi_j\delta_{\theta_j}:1\le m\le k,\ \pi_j\ge0,\ \sum_{j=1}^{m}\pi_j=1,\ \theta_j\in\Theta\right\}. O k ( Θ ) = { G = j = 1 ∑ m π j δ θ j : 1 ≤ m ≤ k , π j ≥ 0 , j = 1 ∑ m π j = 1 , θ j ∈ Θ } .
denote the space of mixing measures with at most k k k atoms. For compactness arguments, we may represent measures with fewer than k k k atoms by adding zero-weight labels without changing the underlying measure. Let f θ = f ( ⋅ ∣ θ ) f_\theta=f(\cdot\mid\theta) f θ = f ( ⋅ ∣ θ ) be a density relative to a dominating measure μ \mu μ on an observation space X ⊂ R d ′ \mathcal{X}\subset\mathbb{R}^{d'} X ⊂ R d ′ . For each G ∈ O k ( Θ ) G\in\mathcal{O}_k(\Theta) G ∈ O k ( Θ ) , define
p G ( x ) = ∫ Θ f θ ( x ) d G ( θ ) , P k ( Θ ) = { p G : G ∈ O k ( Θ ) } . p_G(x)=\int_\Theta f_\theta(x)\,dG(\theta),\qquad\mathcal{P}_k(\Theta)=\{p_G:G\in\mathcal{O}_k(\Theta)\}. p G ( x ) = ∫ Θ f θ ( x ) d G ( θ ) , P k ( Θ ) = { p G : G ∈ O k ( Θ )} .
Throughout the paper, unless otherwise stated, we assume that X 1 , … , X n ∼ i i d P ∗ X_1,\ldots,X_n\overset{\mathrm{iid}}{\sim}P_* X 1 , … , X n ∼ iid P ∗ , where P ∗ P_* P ∗ has density p ∗ = p G ∗ p_*=p_{G_*} p ∗ = p G ∗ and
G ∗ = ∑ i = 1 p π i ∗ δ θ i ∗ ∈ O p ( Θ ) , 1 ≤ p < k . G_*=\sum_{i=1}^{p}\pi_i^*\delta_{\theta_i^*}\in\mathcal{O}_p(\Theta),\qquad1\le p<k. G ∗ = i = 1 ∑ p π i ∗ δ θ i ∗ ∈ O p ( Θ ) , 1 ≤ p < k .
The true atoms θ i ∗ \theta_i^* θ i ∗ are pairwise distinct and belong to int ( Θ ) \operatorname{int}(\Theta) int ( Θ ) , and their weights π i ∗ \pi_i^* π i ∗ are strictly positive. We focus on overspecified models, hence the restriction k > p k>p k > p . For measurable g g g , write P ∗ g = ∫ X g d P ∗ P_*g=\int_{\mathcal{X}}g\,dP_* P ∗ g = ∫ X g d P ∗ whenever the integral exists. For g , h ∈ L 2 ( P ∗ ) g,h\in L^2(P_*) g , h ∈ L 2 ( P ∗ ) , define
⟨ g , h ⟩ ∗ = P ∗ ( g h ) , ∥ g ∥ ∗ = ( P ∗ g 2 ) 1 / 2 . \langle g,h\rangle_* = P_*(gh),\qquad\|g\|_*=(P_*g^2)^{1/2}. ⟨ g , h ⟩ ∗ = P ∗ ( g h ) , ∥ g ∥ ∗ = ( P ∗ g 2 ) 1/2 .
Let P n = n − 1 ∑ r = 1 n δ X r P_n=n^{-1}\sum_{r=1}^{n}\delta_{X_r} P n = n − 1 ∑ r = 1 n δ X r be the empirical measure, so that P n g = n − 1 ∑ r = 1 n g ( X r ) P_ng=n^{-1}\sum_{r=1}^{n}g(X_r) P n g = n − 1 ∑ r = 1 n g ( X r ) . For measurable g g g with P ∗ ∣ g ∣ < ∞ P_*|g|<\infty P ∗ ∣ g ∣ < ∞ , define the empirical process by
G n g : = n ( P n − P ∗ ) g = 1 n ∑ r = 1 n { g ( X r ) − P ∗ g } . \mathbb{G}_ng:=\sqrt{n}(P_n-P_*)g=\frac{1}{\sqrt{n}}\sum_{r=1}^{n}\{g(X_r)-P_*g\}. G n g := n ( P n − P ∗ ) g = n 1 r = 1 ∑ n { g ( X r ) − P ∗ g } .
Our main object of study is the maximum likelihood estimator, defined as a measurable global maximizer
G ^ n ∈ a r g m a x G ∈ O k ( Θ ) ℓ n ( G ) , ℓ n ( G ) = ∑ r = 1 n log p G ( X r ) . \widehat{G}_n \in\mathop{\rm arg\,max}_{G\in\mathcal{O}_k(\Theta)} \ell_n(G), \qquad\ell_n(G)=\sum_{r=1}^{n}\log p_G(X_r). G n ∈ arg max G ∈ O k ( Θ ) ℓ n ( G ) , ℓ n ( G ) = r = 1 ∑ n log p G ( X r ) .
The maximizer need not be unique. Under the conditions of the measurable selection theorem in Rockafellar and Wets [29 ] , we may choose a measurable maximizer. To quantify the local density error, define ρ ( G ) = ∥ ( p G − p ∗ ) / p ∗ ∥ ∗ \rho(G)=\left\|(p_G-p_*)/p_*\right\|_* ρ ( G ) = ∥ ( p G − p ∗ ) / p ∗ ∥ ∗ .
Assumptions
We state the assumptions used to establish consistency, expand the likelihood, and characterize the local density directions.
Assumption 2.1 (Relative C 3 C^3 C 3 smoothness and a common envelope). For μ \mu μ -almost every x x x , the map θ ↦ f θ ( x ) \theta\mapsto f_\theta(x) θ ↦ f θ ( x ) is three times continuously differentiable on an open neighborhood of Θ \Theta Θ , and f θ ( x ) > 0 f_\theta(x)>0 f θ ( x ) > 0 for every θ ∈ Θ \theta\in\Theta θ ∈ Θ . Furthermore, there exist η > 0 \eta>0 η > 0 and a measurable function F F F such that P ∗ F 2 + η < ∞ P_*F^{2+\eta}<\infty P ∗ F 2 + η < ∞ and, for r = 0 , 1 , 2 , 3 , r=0,1,2,3, r = 0 , 1 , 2 , 3 ,
sup θ ∈ Θ ∥ ∇ θ r f θ ( x ) ∥ p ∗ ( x ) ≤ F ( x ) for P ∗ -almost every x . \sup_{\theta\in\Theta}\frac{\left\|\nabla_\theta^r f_\theta(x)\right\|}{p_*(x)}\leq F(x)\qquad\text{for }P_*\text{-almost every }x. θ ∈ Θ sup p ∗ ( x ) ∥ ∇ θ r f θ ( x ) ∥ ≤ F ( x ) for P ∗ -almost every x . where tensor norms may be any fixed equivalent norms.
Assumption 2.1 controls the kernel and its first three derivatives relative to p ∗ p_* p ∗ through a common L 2 + η ( P ∗ ) L^{2+\eta}(P_*) L 2 + η ( P ∗ ) envelope. The first two derivative bounds control the linear and quadratic terms in the Taylor expansion of f θ f_\theta f θ around each true atom, while the third-derivative bound controls the remainder after normalization by p ∗ p_* p ∗ . This integrability condition allows the relative quantities to be unbounded in x x x , accommodating the tail behavior of mixture kernels without requiring a uniformly bounded density ratio.
Assumption 2.2 (Likelihood consistency and ordinary identifiability). The class { log p G : G ∈ O k ( Θ ) } \{\log p_G:G\in\mathcal{O}_k(\Theta)\} { log p G : G ∈ O k ( Θ )} is P ∗ P_* P ∗ -Glivenko–Cantelli with an integrable envelope. In addition,
p G = p ∗ μ -almost everywhere ⟹ G = G ∗ ( G ∈ O k ( Θ ) ) . p_G=p_*\quad\mu\text{-almost everywhere}\quad\Longrightarrow\quad G=G_*\qquad(G\in\mathcal{O}_k(\Theta)). p G = p ∗ μ -almost everywhere ⟹ G = G ∗ ( G ∈ O k ( Θ )) . Equivalently, P ∗ log p G < P ∗ log p ∗ P_*\log p_G<P_*\log p_* P ∗ log p G < P ∗ log p ∗ for every G ≠ G ∗ . G\ne G_*. G = G ∗ .
The Glivenko–Cantelli condition ensures uniform convergence of the normalized log-likelihood 1 n ℓ n ( G ) \frac{1}{n}\ell_n(G) n 1 ℓ n ( G ) to its population criterion G ↦ P ∗ log p G G\mapsto P_*\log p_G G ↦ P ∗ log p G , while identifiability makes G ∗ G_* G ∗ its unique maximizer through the Kullback–Leibler inequality. Together with compactness of O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) and continuity of the population criterion, these properties yield W 2 ( G ^ n , G ∗ ) → 0 W_2(\widehat{G}_n,G_*)\to0 W 2 ( G n , G ∗ ) → 0 in probability by the standard consistency theorem for M M M -estimators (van der Vaart [31 ] ).
Assumption 2.3 (Second-order strong identifiability up to the fitted order). For every collection of pairwise distinct points ξ 1 , … , ξ m ∈ Θ \xi_1,\ldots,\xi_m\in\Theta ξ 1 , … , ξ m ∈ Θ with m ≤ 2 k m\leq2k m ≤ 2 k , the identity
∑ ℓ = 1 m [ a ℓ f ξ ℓ + b ℓ T ∇ f ξ ℓ + 1 2 tr ( C ℓ ∇ 2 f ξ ℓ ) ] = 0 μ -almost everywhere \sum_{\ell=1}^{m}\left[a_\ell f_{\xi_\ell}+b_\ell^{\mathsf T}\nabla f_{\xi_\ell}+\frac{1}{2}\operatorname{tr}\left(C_\ell\nabla^2 f_{\xi_\ell}\right)\right]=0\qquad\mu\text{-almost everywhere} ℓ = 1 ∑ m [ a ℓ f ξ ℓ + b ℓ T ∇ f ξ ℓ + 2 1 tr ( C ℓ ∇ 2 f ξ ℓ ) ] = 0 μ -almost everywhere
with a ℓ ∈ R a_\ell\in\mathbb{R} a ℓ ∈ R , b ℓ ∈ R d b_\ell\in\mathbb{R}^{d} b ℓ ∈ R d , and symmetric C ℓ ∈ R d × d C_\ell\in\mathbb{R}^{d \times d} C ℓ ∈ R d × d implies
a ℓ = 0 , b ℓ = 0 , C ℓ = 0 for every ℓ . a_\ell= 0,\qquad b_\ell= 0,\qquad C_\ell= 0 \quad\text{for every } \ell. a ℓ = 0 , b ℓ = 0 , C ℓ = 0 for every ℓ .
Assumption 2.3 rules out cancellations among kernels and their first two derivatives, providing a standard basis for establishing inverse inequalities in mixture estimation [13 , 12 ] . It also ensures that each h ∈ T k h \in\mathcal{T}_k h ∈ T k has a unique representation in (1) : subtracting two such representations yields an identity involving at most 2 k 2k 2 k distinct locations, so the assumption forces the corresponding coefficients to agree.
A representative example: Gaussian location mixtures with known covariance
The preceding assumptions are standard in the analysis of strongly identifiable finite mixtures and hold for the Gaussian location family with known covariance. The following corollary, whose proof is deferred to Appendix E, verifies them for f θ ( x ) = φ Σ ( x − θ ) f_\theta(x) = \varphi_\Sigma(x - \theta) f θ ( x ) = φ Σ ( x − θ ) , where φ Σ \varphi_\Sigma φ Σ denotes the density of zero-mean Gaussian vector with covariance matrix Σ \Sigma Σ , Σ ≻ 0 \Sigma\succ0 Σ ≻ 0 is known and Θ ⊂ R d \Theta\subset\mathbb{R}^{d} Θ ⊂ R d is compact and convex.
Corollary 2.4. For the Gaussian location family with known positive-definite covariance matrix Σ \Sigma Σ , Assumptions 2.1–2.3 hold.
Likelihood geometry and the density weak limit
In this section, we characterize the attainable tangent set T k \mathcal{T}_k T k and illustrate how the finite component budget constrains local splitting and vanishing nonlocal components. We then establish the weak limit of the relative-density maximum likelihood estimator n p ^ G ^ n − p ∗ p ∗ \sqrt{n}\frac{\widehat{p}_{\widehat{G}_n}-p_*}{p_*} n p ∗ p G n − p ∗ as the maximizer of a Gaussian quadratic criterion over T k \mathcal{T}_k T k .
The attainable tangent set and the finite component budget
We begin by defining the attainable tangent set of relative-density directions attainable by mixing measures with at most k k k atoms:
T k : = { h ∈ L 2 ( P ∗ ) : ∃ t m ↓ 0 , G m ∈ O k ( Θ ) such that ∥ p G m − p ∗ t m p ∗ − h ∥ ∗ → 0 } . (4) \mathcal{T}_k := \left\{ h \in L^2(P_*) : \exists\, t_m \downarrow0,\ G_m \in\mathcal{O}_k(\Theta)\ \text{such that}\ \left\| \frac{p_{G_m} - p_*}{t_m p_*} - h \right\|_* \to0 \right\}. \tag*{(4)} T k := { h ∈ L 2 ( P ∗ ) : ∃ t m ↓ 0 , G m ∈ O k ( Θ ) such that t m p ∗ p G m − p ∗ − h ∗ → 0 } . (4)
Taking t m = n − 1 / 2 t_m = n^{-1/2} t m = n − 1/2 gives the scaling relevant to the relative-density maximum likelihood estimator. Every element of T k \mathcal{T}_k T k has mean zero under P ∗ P_* P ∗ , since
P ∗ ( p G m − p ∗ t m p ∗ ) = 0. P_*\left(\frac{p_{G_m}-p_*}{t_m p_*}\right)=0. P ∗ ( t m p ∗ p G m − p ∗ ) = 0.
The following proposition characterizes the elements of T k \mathcal{T}_k T k through changes in cluster mass, local first and second moments, and vanishing nonlocal components.
Proposition 3.1 (Characteristics of attainable tangent set). Under Assumptions 2.1–2.3, a function h ∈ L 2 ( P ∗ ) h \in L^2(P_*) h ∈ L 2 ( P ∗ ) belongs to T k \mathcal{T}_k T k if and only if it admits the representation
h ( x ) = 1 p ∗ ( x ) [ ∑ i = 1 p { α i f θ i ∗ ( x ) + β i T ∇ f θ i ∗ ( x ) + 1 2 tr ( C i ∇ 2 f θ i ∗ ( x ) ) } + ∫ Θ f θ ( x ) d ν ( θ ) ] , (5) h(x) = \frac{1}{p_*(x)}\left[\sum_{i=1}^{p}\left\{\alpha_i f_{\theta_i^*}(x) + \beta_i^{\mathsf{T}}\nabla f_{\theta_i^*}(x) + \frac{1}{2}\operatorname{tr}\left(C_i\nabla^2 f_{\theta_i^*}(x)\right)\right\} + \int_{\Theta} f_\theta(x)\,d\nu(\theta)\right], \tag*{(5)} h ( x ) = p ∗ ( x ) 1 [ i = 1 ∑ p { α i f θ i ∗ ( x ) + β i T ∇ f θ i ∗ ( x ) + 2 1 tr ( C i ∇ 2 f θ i ∗ ( x ) ) } + ∫ Θ f θ ( x ) d ν ( θ ) ] , (5) where α i ∈ R \alpha_i \in\mathbb{R} α i ∈ R , β i ∈ R d \beta_i \in\mathbb{R}^d β i ∈ R d , C i ∈ S + d C_i \in\mathbb{S}_+^d C i ∈ S + d , and ν = ∑ ℓ = 1 q c ℓ δ η ℓ \nu= \sum_{\ell=1}^{q} c_\ell\delta_{\eta_\ell} ν = ∑ ℓ = 1 q c ℓ δ η ℓ , here c ℓ > 0 c_\ell> 0 c ℓ > 0 , η ℓ ∈ Θ ∖ { θ 1 ∗ , … , θ p ∗ } \eta_\ell\in\Theta\setminus\{\theta_1^*,\ldots,\theta_p^*\} η ℓ ∈ Θ ∖ { θ 1 ∗ , … , θ p ∗ } , with distinct η ℓ \eta_\ell η ℓ . The case q = 0 q=0 q = 0 corresponds to ν = 0 \nu=0 ν = 0 . These coordinates satisfy mass conservation,
∑ i = 1 p α i + ν ( Θ ) = 0 , (6) \sum_{i=1}^{p}\alpha_i+\nu(\Theta)=0, \tag*{(6)} i = 1 ∑ p α i + ν ( Θ ) = 0 , (6) and the finite component constraint,
q + ∑ i = 1 p rank ( C i ) ≤ k − p . (7) q+\sum_{i=1}^{p}\operatorname{rank}(C_i)\leq k-p. \tag*{(7)} q + i = 1 ∑ p rank ( C i ) ≤ k − p . (7) The representation is unique up to permutation of the atoms of ν \nu ν .
We provide the proof of this result in Appendix § 11 .§ 1 . We call γ = ( α , β , C , ν ) \gamma=(\alpha,\beta,C,\nu) γ = ( α , β , C , ν ) the tangent configuration associated with h h h . The coefficient α i \alpha_i α i describes the signed first-order change in the mass near θ i ∗ \theta_i^* θ i ∗ , while β i \beta_i β i and C i C_i C i describe its scaled local first and second moments. The measure ν \nu ν records fitted atoms whose weights vanish but whose limiting locations lie outside the true support. These mechanisms may occur within the same sequence, subject to (6) and (7) .
Equation (6) expresses conservation of total probability mass: the signed changes in the masses of the true clusters must offset the mass assigned to nonlocal components. In particular, a nonzero ν \nu ν requires a corresponding net decrease in the total mass of the true clusters. Meanwhile, the component constraint (7) reflects the number of support points needed to realize each configuration. A local second-moment matrix of rank r r r requires at least r + 1 r+1 r + 1 nearby fitted atoms, consuming r r r additional atoms beyond the one needed for the true cluster. Each support point of ν \nu ν consumes one further atom. Their total cannot exceed the k − p k-p k − p additional components available. Conversely, local simplex constructions and nonlocal atoms with weights proportional to t m t_m t m realize every configuration satisfying this constraint. Thus, together with mass conservation, (7) is necessary and sufficient for a configuration γ \gamma γ such that h γ ∈ T k h_\gamma\in\mathcal{T}_k h γ ∈ T k .
Example: Tangent and in a one-to-two-component
To illustrate how the finite component budget constrains the attainable tangent directions, we consider a one-to-two-component Gaussian location mixture. Let f θ ( x ) = φ ( x − θ ) f_\theta(x) = \varphi(x-\theta) f θ ( x ) = φ ( x − θ ) , where φ \varphi φ is the standard normal density, and let Θ ⊂ R \Theta\subset\mathbb{R} Θ ⊂ R be compact with 0 0 0 in its interior. Take G ∗ = δ 0 G_* = \delta_0 G ∗ = δ 0 and fitted upper order k = 2 k=2 k = 2 . Define
u ( x ) : = x , v ( x ) : = 1 2 ( x 2 − 1 ) , r η ( x ) : = exp { η x − η 2 / 2 } − 1. u(x) := x,\qquad v(x) := \frac{1}{2}(x^2-1),\qquad r_\eta(x) := \exp\{\eta x-\eta^2/2\}-1. u ( x ) := x , v ( x ) := 2 1 ( x 2 − 1 ) , r η ( x ) := exp { η x − η 2 /2 } − 1.
These functions satisfy u = f 0 ′ f 0 u = \frac{f'_0}{f_0} u = f 0 f 0 ′ , v = f 0 ′ ′ 2 f 0 v = \frac{f''_0}{2f_0} v = 2 f 0 f 0 ′′ , and r η = f η − f 0 f 0 r_\eta= \frac{f_\eta-f_0}{f_0} r η = f 0 f η − f 0 , where derivatives are taken with respect to the location parameter. The attainable tangent set is
T 2 = { b u + c v : b ∈ R , c ≥ 0 } ∪ { b u + a r η : b ∈ R , a ≥ 0 , η ∈ Θ ∖ { 0 } } . (8) \mathcal{T}_2 = \{bu+cv:b\in\mathbb{R},\ c\ge0\}\ \cup\ \{bu+ar_\eta:b\in\mathbb{R},\ a\ge0,\ \eta\in\Theta\setminus\{0\}\}. \tag*{(8)} T 2 = { b u + c v : b ∈ R , c ≥ 0 } ∪ { b u + a r η : b ∈ R , a ≥ 0 , η ∈ Θ ∖ { 0 }} . (8)
The term b u bu b u represents a first-order location shift and may occur in either family. The term c v cv c v arises from local splitting of the true atom, whereas a r η ar_\eta a r η represents the relative-density perturbation obtained by transferring mass of order t t t from the true atom at 0 0 0 to a nonlocal atom at η \eta η . Since only one additional fitted atom is available, the constraint q + rank ( C ) ≤ 1 q+\operatorname{rank}(C)\le1 q + rank ( C ) ≤ 1 prevents nontrivial local splitting and a nonlocal component from occurring simultaneously.
The likelihood-selected Gaussian limit
To characterize the direction selected by the likelihood, define H : = span ( T k ) ‾ L 2 ( P ∗ ) \mathcal{H}:=\overline{\operatorname{span}(\mathcal{T}_k)}^{L^2(P_*)} H := span ( T k ) L 2 ( P ∗ ) , and let W \mathbb{W} W be an isonormal Gaussian process indexed by H \mathcal{H} H , with covariance E { W ( h ) W ( g ) } = ⟨ h , g ⟩ ∗ \mathbb{E}\{\mathbb{W}(h)\mathbb{W}(g)\}=\langle h,g\rangle_* E { W ( h ) W ( g )} = ⟨ h , g ⟩ ∗ , where h , g ∈ H h,g\in\mathcal{H} h , g ∈ H . The limiting local log-likelihood criterion takes the form Q ( h ) : = W ( h ) − 1 2 ∥ h ∥ ∗ 2 Q(h):=\mathbb{W}(h)-\frac{1}{2}\lVert h\rVert_*^2 Q ( h ) := W ( h ) − 2 1 ∥ h ∥ ∗ 2 . Since the fitted model permits only directions in T k \mathcal{T}_k T k , we define
H † : = arg max h ∈ T k { W ( h ) − 1 2 ∥ h ∥ ∗ 2 } . (9) \mathcal{H}^{\dagger}:=\underset{h\in\mathcal{T}_k}{\arg\max}\left\{\mathbb{W}(h)-\frac{1}{2}\lVert h\rVert_*^2\right\}. \tag*{(9)} H † := h ∈ T k arg max { W ( h ) − 2 1 ∥ h ∥ ∗ 2 } . (9)
Lemma B.3 establishes existence and almost sure uniqueness of this maximizer. The argument uses a separable version of W \mathbb{W} W on bounded sections of T k \mathcal{T}_k T k and does not require bounded sample paths on the full Hilbert-space unit ball.
Theorem 3.4 (Root-n n n weak limit of the relative-density MLE). Under Assumptions 2.1 2.1 2.1 –2.3 2.3 2.3 ,
ρ ( G ^ n ) = ∥ p G ^ n − p ∗ ∥ ∥ p ∗ ∥ ∗ = O P ( n − 1 / 2 ) , (10) \rho(\widehat{G}_n)=\frac{\lVert p_{\widehat{G}_n}-p_*\rVert}{\lVert p_*\rVert_*}=O_P(n^{-1/2}), \tag*{(10)} ρ ( G n ) = ∥ p ∗ ∥ ∗ ∥ p G n − p ∗ ∥ = O P ( n − 1/2 ) , (10) and
h ^ n : = n p G ^ n − p ∗ p ∗ ⇝ H † in L 2 ( P ∗ ) . (11) \widehat{h}_n:=\sqrt{n}\frac{p_{\widehat{G}_n}-p_*}{p_*}\rightsquigarrow\mathcal{H}^{\dagger}\quad\text{in }L^2(P_*). \tag*{(11)} h n := n p ∗ p G n − p ∗ ⇝ H † in L 2 ( P ∗ ) . (11)
The proof of this theorem is discussed in Appendix § 11 .§ 3 . Despite the root-n n n normalization, the limiting relative-density error in (11) need not be Gaussian. Its distribution is determined by maximizing a Gaussian quadratic criterion over the generally nonconvex set T k \mathcal{T}_k T k .
Proof sketch of Theorem 3.4. The starting point is a Taylor expansion of the log-likelihood difference. Writing h n , G : = n p G − p ∗ p ∗ h_{n,G} := \sqrt{n}\frac{p_G-p_*}{p_*} h n , G := n p ∗ p G − p ∗ , we have
ℓ n ( G ) − ℓ n ( G ∗ ) = ∑ r = 1 n log ( 1 + h n , G ( X r ) n ) . \ell_n(G)-\ell_n(G_*)=\sum_{r=1}^{n}\log\left(1+\frac{h_{n,G}(X_r)}{\sqrt{n}}\right). ℓ n ( G ) − ℓ n ( G ∗ ) = r = 1 ∑ n log ( 1 + n h n , G ( X r ) ) .
Whenever the G G G is sufficiently close to true mixing measure G ∗ G_* G ∗ , expansion of the logarithm log ( 1 + x ) = x − 1 2 x 2 + O ( x 3 ) \log(1+x)=x-\frac{1}{2}x^2+O(x^3) log ( 1 + x ) = x − 2 1 x 2 + O ( x 3 ) for x > − 1 x>-1 x > − 1 gives
ℓ n ( G ) − ℓ n ( G ∗ ) = G n h n , G − 1 2 P n h n , G 2 + R n , G , (12) \ell_n(G)-\ell_n(G_*)=\mathbb{G}_n h_{n,G}-\frac{1}{2}P_nh_{n,G}^2+R_{n,G}, \tag*{(12)} ℓ n ( G ) − ℓ n ( G ∗ ) = G n h n , G − 2 1 P n h n , G 2 + R n , G , (12)
where we use P ∗ h n , G = 0 P_*h_{n,G}=0 P ∗ h n , G = 0 . We first obtain a rate bound from a one-sided likelihood inequality, then establish a uniform version of (12) to identify the weak limit.
Part I: Consistency and root-n n n localization.
Step I.1. Using Lemma A.1, we have W 2 ( G ^ n , G ∗ ) → 0 W_2(\widehat{G}_n,G_*)\to0 W 2 ( G n , G ∗ ) → 0 . Thus when n n n sufficiently large, there exists a small neighborhood U \mathcal{U} U of G ∗ G_* G ∗ such that G ^ n ∈ U \widehat{G}_n\in\mathcal{U} G n ∈ U with probability tending to one. Consider the normalized class
S : = { s G = p G − p ∗ ρ ( G ) p ∗ : G ∈ U , G ≠ G ∗ } , \mathcal{S}:=\left\{s_G=\frac{p_G-p_*}{\rho(G)p_*}:G\in\mathcal{U},\ G\ne G_*\right\}, S := { s G = ρ ( G ) p ∗ p G − p ∗ : G ∈ U , G = G ∗ } ,
whose elements satisfy P ∗ s G = 0 P_*s_G=0 P ∗ s G = 0 and P ∗ s G 2 = 1 P_*s_G^2=1 P ∗ s G 2 = 1 . As it is possible to construct a common envelop F 0 F_0 F 0 (Lemma A.5), we choose K < ∞ K<\infty K < ∞ such that P ∗ { F 0 2 1 ( F 0 > K ) } < 1 4 P_*\{F_0^2\mathbf{1}(F_0>K)\}<\frac{1}{4} P ∗ { F 0 2 1 ( F 0 > K )} < 4 1 , then
inf s ∈ S P ∗ { s 2 1 ( F 0 ≤ K ) } ≥ 3 4 . \inf_{s\in\mathcal{S}}P_*\{s^2\mathbf{1}(F_0\le K)\}\ge\frac{3}{4}. s ∈ S inf P ∗ { s 2 1 ( F 0 ≤ K )} ≥ 4 3 .
In addition, the Glivenko–Cantelli property of the truncated squared class gives
sup s ∈ S ∣ ( P n − P ∗ ) { s 2 1 ( F 0 ≤ K ) } ∣ → P 0. \sup_{s\in\mathcal{S}}\left|(P_n-P_*)\{s^2\mathbf{1}(F_0\le K)\}\right|\xrightarrow{P}0. s ∈ S sup ( P n − P ∗ ) { s 2 1 ( F 0 ≤ K )} P 0.
Consequently, the event E n : = { inf s ∈ S P n { s 2 1 ( F 0 ≤ K ) } ≥ 1 2 } \mathcal{E}_n:=\{\inf_{s\in\mathcal{S}}P_n\{s^2\mathbf{1}(F_0\le K)\}\ge\frac{1}{2}\} E n := { inf s ∈ S P n { s 2 1 ( F 0 ≤ K )} ≥ 2 1 } has probability tending to one.
Shrink U \mathcal{U} U so that ρ ( G ) K ≤ 1 / 2 \rho(G)K\le1/2 ρ ( G ) K ≤ 1/2 for G ∈ U G\in\mathcal{U} G ∈ U . Using log ( 1 + y ) ≤ y − 1 4 y 2 1 ( y ≤ 1 / 2 ) \log(1+y)\le y-\frac{1}{4}y^2\mathbf{1}(y\le1/2) log ( 1 + y ) ≤ y − 4 1 y 2 1 ( y ≤ 1/2 ) for y > − 1 y>-1 y > − 1 , and ∣ s G ∣ ≤ F 0 |s_G|\le F_0 ∣ s G ∣ ≤ F 0 , we obtain
ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − 1 4 n ρ ( G ) 2 P n { s G 2 1 ( F 0 ≤ K ) } . \ell_n(G)-\ell_n(G_*)\le\sqrt{n}\rho(G)\mathbb{G}_ns_G-\frac{1}{4}n\rho(G)^2P_n\{s_G^2\mathbf{1}(F_0\le K)\}. ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − 4 1 n ρ ( G ) 2 P n { s G 2 1 ( F 0 ≤ K )} .
On E n \mathcal{E}_n E n , this yields ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − c n ρ ( G ) 2 \ell_n(G)-\ell_n(G_*)\le\sqrt{n}\rho(G)\mathbb{G}_ns_G-cn\rho(G)^2 ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − c n ρ ( G ) 2 for c = 1 8 c=\frac{1}{8} c = 8 1 .
Step I.2. Define Z n : = sup s ∈ S ∣ G n s ∣ Z_n:=\sup_{s\in\mathcal{S}}|\mathbb{G}_ns| Z n := sup s ∈ S ∣ G n s ∣ . Since ℓ n ( G ^ n ) ≥ ℓ n ( G ∗ ) \ell_n(\widehat{G}_n)\ge\ell_n(G_*) ℓ n ( G n ) ≥ ℓ n ( G ∗ ) , the preceding inequality implies, on an event whose probability tends to one, n ρ ( G ^ n ) ≤ c − 1 Z n \sqrt{n}\rho(\widehat{G}_n)\le c^{-1}Z_n n ρ ( G n ) ≤ c − 1 Z n . In addition, the Donsker property of S \mathcal{S} S gives Z n = O P ( 1 ) Z_n=O_P(1) Z n = O P ( 1 ) , hence ρ ( G ^ n ) = O P ( n − 1 / 2 ) \rho(\widehat{G}_n)=O_P(n^{-1/2}) ρ ( G n ) = O P ( n − 1/2 ) .
Part II: Weak convergence of the relative-density MLE.
Step II.1. Fix M < ∞ M < \infty M < ∞ and restrict attention to
H n , M : = { h n , G : G ∈ O k ( Θ ) , n ρ ( G ) ≤ M } . \mathcal{H}_{n,M} := \{h_{n,G}: G \in\mathcal{O}_k(\Theta),\ \sqrt{n}\rho(G) \le M\}. H n , M := { h n , G : G ∈ O k ( Θ ) , n ρ ( G ) ≤ M } .
For sufficiently large n n n , the corresponding mixing measures belong to U \mathcal{U} U , and h n , G = a G s G h_{n,G} = a_Gs_G h n , G = a G s G with 0 ≤ a G ≤ M 0 \le a_G \le M 0 ≤ a G ≤ M and s G ∈ S s_G \in\mathcal{S} s G ∈ S . The Glivenko–Cantelli property of the squared class gives
sup n ρ ( G ) ≤ M ∣ ( P n − P ∗ ) h n , G 2 ∣ ≤ M 2 sup s ∈ S ∣ ( P n − P ∗ ) s 2 ∣ = o P ( 1 ) . \sup_{\sqrt{n}\rho(G)\le M} |(P_n-P_*)h_{n,G}^2| \le M^2 \sup_{s\in\mathcal{S}} |(P_n-P_*)s^2| = o_P(1). n ρ ( G ) ≤ M sup ∣ ( P n − P ∗ ) h n , G 2 ∣ ≤ M 2 s ∈ S sup ∣ ( P n − P ∗ ) s 2 ∣ = o P ( 1 ) .
Furthermore, we can control the remainder R n , G R_{n,G} R n , G in equation (12) as sup n ρ ( G ) ≤ M ∣ R n , G ∣ = o P ( 1 ) \sup_{\sqrt{n}\rho(G)\le M}|R_{n,G}| = o_P(1) sup n ρ ( G ) ≤ M ∣ R n , G ∣ = o P ( 1 ) using the common envelop F 0 F_0 F 0 . Together, these bounds control the Taylor remainder uniformly and give
sup n ρ ( G ) ≤ M ∣ ℓ n ( G ) − ℓ n ( G ∗ ) − G n h n , G + 1 2 ∥ h n , G ∥ ∗ 2 ∣ ⟶ P 0. \sup_{\sqrt{n}\rho(G)\le M}\left|\ell_n(G)-\ell_n(G_*)-\mathbb{G}_nh_{n,G}+\frac{1}{2}\|h_{n,G}\|_*^2\right|\mathop{\longrightarrow}^{P}0. n ρ ( G ) ≤ M sup ℓ n ( G ) − ℓ n ( G ∗ ) − G n h n , G + 2 1 ∥ h n , G ∥ ∗ 2 ⟶ P 0.
Thus, for G G G such that n ρ ( G ) ≤ M \sqrt{n}\rho(G)\le M n ρ ( G ) ≤ M , the likelihood discrepancy ℓ n ( G ) − ℓ n ( G ∗ ) \ell_n(G)-\ell_n(G_*) ℓ n ( G ) − ℓ n ( G ∗ ) can be written as a function of h ∈ H n , M h\in\mathcal{H}_{n,M} h ∈ H n , M :
Q n ( h ) = G n h − 1 2 ∥ h ∥ ∗ 2 + r n ( h ) , sup h ∈ H n , M ∣ r n ( h ) ∣ = o P ( 1 ) . Q_n(h)=\mathbb{G}_nh-\frac{1}{2}\|h\|_*^2+r_n(h),\qquad\sup_{h\in\mathcal{H}_{n,M}}|r_n(h)|=o_P(1). Q n ( h ) = G n h − 2 1 ∥ h ∥ ∗ 2 + r n ( h ) , h ∈ H n , M sup ∣ r n ( h ) ∣ = o P ( 1 ) .
Step II.2. The compact sets H n , M \mathcal{H}_{n,M} H n , M and A M : = T k ∩ { ∥ h ∥ ∗ ≤ M } A_M:=\mathcal{T}_k\cap\{\|h\|_*\le M\} A M := T k ∩ { ∥ h ∥ ∗ ≤ M } lie in a common compact set K M K_M K M . As established in the detailed proof, d H ( H n , M , A M ) → 0 d_H(\mathcal{H}_{n,M},A_M)\to0 d H ( H n , M , A M ) → 0 . Define Z n ( h ) : = G n h − 1 2 ∥ h ∥ ∗ 2 Z_n(h):=\mathbb{G}_nh-\frac{1}{2}\|h\|_*^2 Z n ( h ) := G n h − 2 1 ∥ h ∥ ∗ 2 , for h ∈ K M h\in K_M h ∈ K M . The Donsker lemma gives Z n ⇝ Q Z_n\rightsquigarrow Q Z n ⇝ Q in ℓ ∞ ( K M ) \ell^\infty(K_M) ℓ ∞ ( K M ) , where Q Q Q has almost surely continuous paths on K M K_M K M . The uniform likelihood expansion yields
Q n ( h ) = Z n ( h ) + r n ( h ) , Δ n , M : = sup h ∈ H n , M ∣ r n ( h ) ∣ = o P ( 1 ) . Q_n(h)=Z_n(h)+r_n(h),\qquad\Delta_{n,M}:=\sup_{h\in\mathcal{H}_{n,M}}|r_n(h)|=o_P(1). Q n ( h ) = Z n ( h ) + r n ( h ) , Δ n , M := h ∈ H n , M sup ∣ r n ( h ) ∣ = o P ( 1 ) .
Consequently, any restricted likelihood maximizer h ^ n , M ∈ a r g m a x h ∈ H n , M Q n ( h ) \widehat{h}_{n,M}\in\mathop{\rm arg\,max}_{h\in\mathcal{H}_{n,M}}Q_n(h) h n , M ∈ arg max h ∈ H n , M Q n ( h ) satisfies
Z n ( h ^ n , M ) ≥ sup h ∈ H n , M Z n ( h ) − 2 Δ n , M . Z_n(\widehat{h}_{n,M})\ge\sup_{h\in\mathcal{H}_{n,M}}Z_n(h)-2\Delta_{n,M}. Z n ( h n , M ) ≥ h ∈ H n , M sup Z n ( h ) − 2 Δ n , M .
By Lemma B.3 , the restriction of Q Q Q to A M A_M A M admits an almost surely unique maximizer H M † H_M^\dagger H M † . The argmax theorem for varying domains (Cox, [3 ] , Theorem 1) therefore gives h ^ n , M ⇝ H M † \widehat{h}_{n,M}\rightsquigarrow H_M^\dagger h n , M ⇝ H M † in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) .
Step II.3. Equation (10) gives ∥ h ^ n ∥ ∗ = O P ( 1 ) \|\widehat{h}_n\|_*=O_P(1) ∥ h n ∥ ∗ = O P ( 1 ) , and hence lim M → ∞ lim sup n → ∞ P { ∥ h ^ n ∥ ∗ > M } = 0 \lim_{M\to\infty}\limsup_{n\to\infty}P\{\|\widehat{h}_n\|_*>M\}=0 lim M → ∞ lim sup n → ∞ P { ∥ h n ∥ ∗ > M } = 0 . Choose h ^ n , M \widehat{h}_{n,M} h n , M to agree with h ^ n \widehat{h}_n h n whenever ∥ h ^ n ∥ ∗ ≤ M \|\widehat{h}_n\|_*\le M ∥ h n ∥ ∗ ≤ M . Moreover, H M † = H † H_M^\dagger=H^\dagger H M † = H † whenever ∥ H † ∥ ∗ ≤ M \|H^\dagger\|_*\le M ∥ H † ∥ ∗ ≤ M , by uniqueness of the global maximizer. Since ∥ H † ∥ ∗ < ∞ \|H^\dagger\|_*<\infty ∥ H † ∥ ∗ < ∞ almost surely, a bounded-Lipschitz comparison, first letting n → ∞ n\to\infty n → ∞ and then M → ∞ M\to\infty M → ∞ , yields
h ^ n ⇝ H † in L 2 ( P ∗ ) . \widehat{h}_n\rightsquigarrow H^\dagger\quad\text{in }L^2(P_*). h n ⇝ H † in L 2 ( P ∗ ) .
The density limit alone does not determine the asymptotic behavior of the fitted component parameters or their Wasserstein loss. For example, local splitting at distance n − 1 / 4 n^{-1/4} n − 1/4 and mass changes of order n − 1 / 2 n^{-1/2} n − 1/2 both contribute at the root-n n n density scale, but affect transportation costs differently.
Connecting Theorem 3.4 to mixing-measure loss therefore requires an additional deterministic expansion.
Sequential expansion and weak convergence of Wasserstein error
To connect the density limit established in § 4 to the Wasserstein estimation error W 2 ( G ^ n , G ∗ ) W_2(\widehat{G}_n,G_\ast) W 2 ( G n , G ∗ ) , we establish a sequential expansion of the squared Wasserstein distance. Specifically, whenever t m ↓ 0 t_m \downarrow0 t m ↓ 0 and p G m − p ∗ t m p ∗ → h ∈ T k \frac{p_{G_m}-p_\ast}{t_m p_\ast} \to h \in\mathcal{T}_k t m p ∗ p G m − p ∗ → h ∈ T k in L 2 ( P ∗ ) L^2(P_\ast) L 2 ( P ∗ ) , we show that
W 2 2 ( G m , G ∗ ) = t m Ω ( h ) + o ( t m ) . W_2^2(G_m,G_\ast)=t_m\Omega(h)+o(t_m). W 2 2 ( G m , G ∗ ) = t m Ω ( h ) + o ( t m ) .
The coefficient Ω ( h ) \Omega(h) Ω ( h ) depends only on the limiting relative-density direction h h h , not on the sequence realizing it. It combines the contribution of local second moments around the true atoms with the transportation cost of residual mass between separated locations. This expansion applies to every sequence attaining h h h , rather than only to a prescribed parameterized path, and yields the Wasserstein weak limit when combined with Theorem 3.4 .
First-order transportation functional and sequential expansion
We construct the transportation functional by separating local spreading from residual mass transport. Let γ = ( α , β , C , ν ) \gamma=(\alpha,\beta,C,\nu) γ = ( α , β , C , ν ) be the unique tangent configuration associated with h γ ∈ T k h_\gamma\in\mathcal{T}_k h γ ∈ T k , where ν = ∑ ℓ = 1 q c ℓ δ η ℓ \nu=\sum_{\ell=1}^{q}c_\ell\delta_{\eta_\ell} ν = ∑ ℓ = 1 q c ℓ δ η ℓ . Consider a sequence satisfying
p G m − p ∗ t m p ∗ ⟶ h γ in L 2 ( P ∗ ) , t m ↓ 0. \frac{p_{G_m}-p_\ast}{t_m p_\ast}\longrightarrow h_\gamma\quad\text{in }L^2(P_\ast),\qquad t_m\downarrow0. t m p ∗ p G m − p ∗ ⟶ h γ in L 2 ( P ∗ ) , t m ↓ 0.
Our goal is to determine the limit of the normalized squared Wasserstein distance W 2 2 ( G m , G ∗ ) / t m W_2^2(G_m,G_\ast)/t_m W 2 2 ( G m , G ∗ ) / t m . We proceed in two steps.
Step 1: Decomposition of W 2 2 ( G m , G ∗ ) W_2^2(G_m,G_\ast) W 2 2 ( G m , G ∗ ) . After passing to a subsequence and labeling the fitted atoms, let L i L_i L i contain those approaching θ i ∗ \theta_i^\ast θ i ∗ , and let L 0 L_0 L 0 contain the remaining atoms. Define the intermediate measure G ‾ m \overline{G}_m G m by moving each local cluster of G m G_m G m onto its true support point while retaining the nonlocal atoms:
M i , m : = ∑ j ∈ L i π j , m , G ‾ m : = ∑ i = 1 p M i , m δ θ i ∗ + ∑ j ∈ L 0 π j , m δ θ j , m ⏟ : = ν m M_{i,m}:=\sum_{j\in L_i}\pi_{j,m},\qquad\overline{G}_m:=\sum_{i=1}^{p}M_{i,m}\delta_{\theta_i^\ast}+\underbrace{\sum_{j\in L_0}\pi_{j,m}\delta_{\theta_{j,m}}}_{:=\nu_m} M i , m := j ∈ L i ∑ π j , m , G m := i = 1 ∑ p M i , m δ θ i ∗ + := ν m j ∈ L 0 ∑ π j , m δ θ j , m
The cost of this local collapse is A m : = ∑ i = 1 p ∑ j ∈ L i π j , m ∥ θ j , m − θ i ∗ ∥ 2 A_m:=\sum_{i=1}^{p}\sum_{j\in L_i}\pi_{j,m}\lVert\theta_{j,m}-\theta_i^\ast\rVert^2 A m := ∑ i = 1 p ∑ j ∈ L i π j , m ∥ θ j , m − θ i ∗ ∥ 2 . Using the intermediate measure G ‾ m \overline{G}_m G m , we can show in Lemma C.1 the decomposition of W 2 2 ( G m , G ∗ ) W_2^2(G_m,G_\ast) W 2 2 ( G m , G ∗ ) into quadratic local collapse A m A_m A m and the transportation from G ‾ m \overline{G}_m G m to G ∗ G_\ast G ∗ :
W 2 2 ( G m , G ∗ ) = A m + W 2 2 ( G ‾ m , G ∗ ) + o ( t m ) . (13) W_2^2(G_m,G_\ast)=A_m+W_2^2(\overline{G}_m,G_\ast)+o(t_m). \tag*{(13)} W 2 2 ( G m , G ∗ ) = A m + W 2 2 ( G m , G ∗ ) + o ( t m ) . (13)
The term A m / t m A_m/t_m A m / t m converges directly to ∑ i = 1 p tr ( C i ) \sum_{i=1}^{p}\operatorname{tr}(C_i) ∑ i = 1 p tr ( C i ) , the sum of the traces of the local second-moment limits.
Step 2: Asymptotic behavior of W 2 2 ( G ‾ m , G ∗ ) W_2^2(\overline{G}_m,G_*) W 2 2 ( G m , G ∗ ) . It remains to determine the limit of W 2 2 ( G ‾ m , G ∗ ) / t m W_2^2(\overline{G}_m,G_*)/t_m W 2 2 ( G m , G ∗ ) / t m . For sufficiently large m m m , this cost accounts for cluster mass discrepancies and nonlocal atoms. Put d i j 2 : = ∥ θ i ∗ − θ j ∗ ∥ 2 d_{ij}^2 := \lVert\theta_i^*-\theta_j^*\rVert^2 d ij 2 := ∥ θ i ∗ − θ j ∗ ∥ 2 , let x i j ≥ 0 x_{ij} \ge0 x ij ≥ 0 denote the first-order mass transported from true support point i i i to true support point j j j , and let y ℓ i ≥ 0 y_{\ell i} \ge0 y ℓ i ≥ 0 denote the first-order mass transported from η ℓ \eta_\ell η ℓ to θ i ∗ \theta_i^* θ i ∗ . Define M ∗ ( α , ν ) \mathcal{M}_*(\alpha,\nu) M ∗ ( α , ν ) as the optimal value of
min x i j , y ℓ i ≥ 0 i ≠ j { ∑ i ≠ j d i j 2 x i j + ∑ ℓ = 1 q ∑ i = 1 p ∥ η ℓ − θ i ∗ ∥ 2 y ℓ i } . (14) \min_{\substack{x_{ij},y_{\ell i}\ge0\\i\ne j}}\left\{\sum_{i\ne j}d_{ij}^2x_{ij}+\sum_{\ell=1}^{q}\sum_{i=1}^{p}\lVert\eta_\ell-\theta_i^*\rVert^2y_{\ell i}\right\}. \tag*{(14)} x ij , y ℓ i ≥ 0 i = j min ⎩ ⎨ ⎧ i = j ∑ d ij 2 x ij + ℓ = 1 ∑ q i = 1 ∑ p ∥ η ℓ − θ i ∗ ∥ 2 y ℓ i ⎭ ⎬ ⎫ . (14)
subject to
∑ i = 1 p y ℓ i = c ℓ , ℓ = 1 , … , q , (15) \sum_{i=1}^{p}y_{\ell i}=c_\ell,\qquad\ell=1,\ldots,q, \tag*{(15)} i = 1 ∑ p y ℓ i = c ℓ , ℓ = 1 , … , q , (15)
and
∑ j ≠ i ( x i j − x j i ) = α i + ∑ ℓ = 1 q y ℓ i , i = 1 , … , p . (16) \sum_{j\ne i}(x_{ij}-x_{ji})=\alpha_i+\sum_{\ell=1}^{q}y_{\ell i},\qquad i=1,\ldots,p. \tag*{(16)} j = i ∑ ( x ij − x j i ) = α i + ℓ = 1 ∑ q y ℓ i , i = 1 , … , p . (16)
Constraint (15) assigns all nonlocal mass to the true support. Constraint (16) balances mass at each true atom: its net outgoing mass equals its initial surplus α i \alpha_i α i plus the mass received from nonlocal atoms.
The tangent configuration describes precisely the limiting inputs to this finite transport problem. In particular, the scaled local mass discrepancies satisfy ( M i , m − π i ∗ ) / t m → α i (M_{i,m}-\pi_i^*)/t_m\to\alpha_i ( M i , m − π i ∗ ) / t m → α i for i = 1 , … , p i=1,\ldots,p i = 1 , … , p . The nonlocal mass is represented by ν m : = ∑ j ∈ L 0 π j , m δ θ j , m \nu_m:=\sum_{j\in L_0}\pi_{j,m}\delta_{\theta_{j,m}} ν m := ∑ j ∈ L 0 π j , m δ θ j , m , and its scaled measure satisfies t m − 1 ν m ⇒ ν t_m^{-1}\nu_m\Rightarrow\nu t m − 1 ν m ⇒ ν , where the convergence is weak convergence of finite measures. Stability of the finite transportation problem (Lemma C.2) therefore yields
W 2 2 ( G ‾ m , G ∗ ) t m ⟶ M ∗ ( α , ν ) . \frac{W_2^2(\overline{G}_m,G_*)}{t_m}\longrightarrow\mathcal{M}_*(\alpha,\nu). t m W 2 2 ( G m , G ∗ ) ⟶ M ∗ ( α , ν ) .
Combining these two limits A m / t m A_m/t_m A m / t m and W 2 2 ( G ‾ m , G ∗ ) / t m W_2^2(\overline{G}_m,G_*)/t_m W 2 2 ( G m , G ∗ ) / t m into (13) , we define the first-order transportation functional by
Ω ( h γ ) : = ∑ i = 1 p tr ( C i ) + M ∗ ( α , ν ) , (17) \Omega(h_\gamma):=\sum_{i=1}^{p}\operatorname{tr}(C_i)+\mathcal{M}_*(\alpha,\nu), \tag*{(17)} Ω ( h γ ) := i = 1 ∑ p tr ( C i ) + M ∗ ( α , ν ) , (17)
where we denote by convention Ω ( 0 ) = 0 \Omega(0)=0 Ω ( 0 ) = 0 . Uniqueness of the tangent coordinates makes Ω \Omega Ω a well-defined functional on T k \mathcal{T}_k T k . The first-moment coordinates β i \beta_i β i do not contribute directly: a displacement of order t m t_m t m of an atom with positive limiting mass has squared transportation cost of order t m 2 t_m^2 t m 2 . They nevertheless enter the likelihood criterion and may affect the distribution of Ω ( H † ) \Omega(H^\dagger) Ω ( H † ) through the selected direction H † H^\dagger H † .
The following proposition states the resulting expansion. Although the construction uses classified subsequences, uniqueness of the tangent coordinates ensures that every such subsequence yields the same limit.
Proposition 4.1 (Sequential first-order expansion of the squared Wasserstein distance). Let t m ↓ 0 t_m\downarrow0 t m ↓ 0 and G m ∈ O k ( Θ ) G_m\in\mathcal{O}_k(\Theta) G m ∈ O k ( Θ ) satisfy
p G m − p ∗ t m p ∗ ⟶ h ∈ T k in L 2 ( P ∗ ) . \frac{p_{G_m}-p_*}{t_mp_*}\longrightarrow h\in\mathcal{T}_k\qquad\text{in }L^2(P_*). t m p ∗ p G m − p ∗ ⟶ h ∈ T k in L 2 ( P ∗ ) .
Then
W 2 2 ( G m , G ∗ ) t m ⟶ Ω ( h ) . (18) \frac{W_2^2(G_m,G_\ast)}{t_m} \longrightarrow\Omega(h). \tag*{(18)} t m W 2 2 ( G m , G ∗ ) ⟶ Ω ( h ) . (18)
We show the proof of Proposition 4.1 in Appendix § 12 .§ 1 . Proposition 4.1 shows that every sequence attaining the same relative-density direction h h h satisfies
W 2 2 ( G m , G ∗ ) = t m Ω ( h ) + o ( t m ) . W_2^2(G_m,G_\ast)=t_m\Omega(h)+o(t_m). W 2 2 ( G m , G ∗ ) = t m Ω ( h ) + o ( t m ) .
We call this a sequential first-order expansion : it is first-order because its leading term is linear in the density-perturbation scale t m t_m t m , and sequential because it holds for every sequence realizing h h h , rather than only along a prescribed parameterized path. The coefficient Ω ( h ) \Omega(h) Ω ( h ) combines local spreading, described by C i C_i C i , with residual mass transport, determined by α \alpha α and ν \nu ν . The latter cost is the optimal value of a finite linear program in transported masses, with fixed squared distances as coefficients.
Joint weak convergence and the fourth-root law
Combining the density limit in Theorem 3.4 with the sequential expansion in Proposition 4.1 , we obtain joint weak convergence of the relative-density estimator and its squared Wasserstein error.
Theorem 4.3 (Joint relative-density and Wasserstein weak limit). Under Assumptions 2.1–2.3,
( h ^ n , n W 2 2 ( G ^ n , G ∗ ) ) ⇝ ( H † , Ω ( H † ) ) in L 2 ( P ∗ ) × R . (19) \left(\widehat{h}_n,\sqrt{n}W_2^2(\widehat{G}_n,G_\ast)\right)\rightsquigarrow\left(H^\dagger,\Omega(H^\dagger)\right)\quad\text{in }L^2(P_\ast)\times\mathbb{R}. \tag*{(19)} ( h n , n W 2 2 ( G n , G ∗ ) ) ⇝ ( H † , Ω ( H † ) ) in L 2 ( P ∗ ) × R . (19) Consequently,
n 1 / 4 W 2 ( G ^ n , G ∗ ) ⇝ { Ω ( H † ) } 1 / 2 , (20) n^{1/4}W_2(\widehat{G}_n,G_\ast)\rightsquigarrow\{\Omega(H^\dagger)\}^{1/2}, \tag*{(20)} n 1/4 W 2 ( G n , G ∗ ) ⇝ { Ω ( H † ) } 1/2 , (20) and
W 2 ( G ^ n , G ∗ ) = O P ( n − 1 / 4 ) . (21) W_2(\widehat{G}_n,G_\ast)=O_P(n^{-1/4}). \tag*{(21)} W 2 ( G n , G ∗ ) = O P ( n − 1/4 ) . (21) The limiting distribution in (20) is generally non-Gaussian.
The proof this theorem is given in Appendix § 12 .§ 3 . Theorem 4.3 identifies how the density and mixing-measure errors fluctuate jointly. The likelihood criterion selects the random density direction H † H^\dagger H † , and the functional Ω \Omega Ω translates this direction into the leading squared transportation cost. Thus, the Wasserstein limit is determined by the same random direction that governs the density limit.
A few remarks clarify the relation to existing rate results and the nature of the limiting distribution. First, inverse inequalities provide a route from density estimation to mixing-measure convergence. In particular, the bound W 2 2 ( G , G ∗ ) ≲ ρ ( G ) W_2^2(G,G_\ast)\lesssim\rho(G) W 2 2 ( G , G ∗ ) ≲ ρ ( G ) in Lemma A.4 , combined with ρ ( G ^ n ) = O P ( n − 1 / 2 ) \rho(\widehat{G}_n)=O_P(n^{-1/2}) ρ ( G n ) = O P ( n − 1/2 ) , yields the fourth-root upper rate in (21) . This bound alone, however, does not identify a limiting distribution. Theorem 4.3 follows the same density-to-mixing-measure route but replaces the inequality with a sequential expansion, thereby identifying the random leading term Ω ( H † ) \Omega(H^\dagger) Ω ( H † ) .
Second, the Wasserstein limit in (20) is generally non-Gaussian, as it is obtained by applying the nonlinear map h ↦ { Ω ( h ) } 1 / 2 h \mapsto\{\Omega(h)\}^{1/2} h ↦ { Ω ( h ) } 1/2 to the likelihood-selected direction H † H^\dagger H † . Non-Gaussian distance limits also arise in regular models: for a one-dimensional Gaussian location model with p = k = 1 p=k=1 p = k = 1 and known variance σ 2 \sigma^2 σ 2 ,
n W 1 ( G ^ n , G ∗ ) ⇝ ∣ Z ∣ , Z ∼ N ( 0 , σ 2 ) . \sqrt{n}W_1(\widehat{G}_n,G_\ast)\rightsquigarrow|Z|,\qquad Z\sim N(0,\sigma^2). n W 1 ( G n , G ∗ ) ⇝ ∣ Z ∣ , Z ∼ N ( 0 , σ 2 ) .
Here the root-n n n distance limit is non-Gaussian, while the fourth-root limit in (20) degenerates to zero.
Example continued: Wasserstein expansion in a one-to-two-component mixture
We return to the Gaussian location mixture in § 4.2 and compute the transportation functional for its two tangent families. Using the notation introduced there, consider
G t s p l i t = 1 2 δ b t + c t + 1 2 δ b t − c t , G t r e m o t e = ( 1 − a t ) δ b t + a t δ η , G_t^{\mathrm{split}}=\frac{1}{2}\delta_{b t+\sqrt{c}t}+\frac{1}{2}\delta_{b t-\sqrt{c}t},\qquad G_t^{\mathrm{remote}}=(1-at)\delta_{bt}+at\delta_\eta, G t split = 2 1 δ b t + c t + 2 1 δ b t − c t , G t remote = ( 1 − a t ) δ b t + a t δ η ,
where b ∈ R b\in\mathbb{R} b ∈ R , c , a ≥ 0 c,a\geq0 c , a ≥ 0 , and η ∈ Θ ∖ { 0 } \eta\in\Theta\setminus\{0\} η ∈ Θ ∖ { 0 } . Both paths belong to O 2 ( Θ ) O_2(\Theta) O 2 ( Θ ) for sufficiently small t > 0 t>0 t > 0 and satisfy
p G t s p l i t − p ∗ t p ∗ ⟶ b u + c v , p G t r e m o t e − p ∗ t p ∗ ⟶ b u + a r η in L 2 ( P ∗ ) . \frac{p_{G_t^{\mathrm{split}}}-p_\ast}{t p_\ast}\longrightarrow bu+cv,\qquad\frac{p_{G_t^{\mathrm{remote}}}-p_\ast}{t p_\ast}\longrightarrow bu+ar_\eta\quad\text{in }L^2(P_\ast). t p ∗ p G t split − p ∗ ⟶ b u + c v , t p ∗ p G t remote − p ∗ ⟶ b u + a r η in L 2 ( P ∗ ) .
Since G ∗ = δ 0 G_\ast=\delta_0 G ∗ = δ 0 , every fitted atom must transport its mass to 0. The squared Wasserstein costs are therefore
W 2 2 ( G t s p l i t , G ∗ ) = t c + b 2 t 2 , W 2 2 ( G t r e m o t e , G ∗ ) = a t η 2 + ( 1 − a t ) b 2 t 2 . (22) W_2^2(G_t^{\mathrm{split}},G_\ast)=tc+b^2t^2,\qquad W_2^2(G_t^{\mathrm{remote}},G_\ast)=at\eta^2+(1-at)b^2t^2. \tag*{(22)} W 2 2 ( G t split , G ∗ ) = t c + b 2 t 2 , W 2 2 ( G t remote , G ∗ ) = a t η 2 + ( 1 − a t ) b 2 t 2 . (22)
Dividing by t t t and taking limits gives
Ω ( b u + c v ) = c , Ω ( b u + a r η ) = a η 2 . (23) \Omega(bu+cv)=c,\qquad\Omega(bu+ar_\eta)=a\eta^2. \tag*{(23)} Ω ( b u + c v ) = c , Ω ( b u + a r η ) = a η 2 . (23)
The local splitting cost is thus determined by the second-moment coefficient c c c , while the nonlocal cost is the transferred mass coefficient a a a multiplied by the squared distance η 2 \eta^2 η 2 . The location-shift coefficient b b b contributes only at order t 2 t^2 t 2 and disappears from the first-order expansion. Both formulas give zero on the family { b u : b ∈ R } \{bu:b\in\mathbb{R}\} { b u : b ∈ R } .
Although these calculations use explicit paths, Proposition 4.1 ensures that (23) holds for every sequence attaining the corresponding density direction. The formulas therefore determine the Wasserstein limit after the likelihood selects H † H^\dagger H † :
n 1 / 4 W 2 ( G ^ n , G ∗ ) ⇝ { c , if H † = b u + c v , a ∣ η ∣ , if H † = b u + a r η . n^{1/4}W_2(\widehat{G}_n,G_\ast)\rightsquigarrow
\begin{cases}
\sqrt{c}, & \text{if }H^\dagger=bu+cv,\\
\sqrt{a}\,|\eta|, & \text{if }H^\dagger=bu+ar_\eta.
\end{cases} n 1/4 W 2 ( G n , G ∗ ) ⇝ { c , a ∣ η ∣ , if H † = b u + c v , if H † = b u + a r η .
The two expressions agree and exactly equal to 0 wherever the families overlap.
The likelihood optimization takes place over the union of the two families in (8) . It compares their optimized Gaussian quadratic criteria, including optimization over the nonlocal location η \eta η . Consequently, the selected direction depends on the Gaussian process indexed by these locations, and its transportation cost cannot be recovered from the optimized likelihood value alone. This example makes the transportation functional explicit, while the unrestricted Gaussian argmax and its induced Wasserstein distribution remain implicitly characterized.
Bootstrap inference for Wasserstein uncertainty
Although Theorem 4.3 characterizes the limiting distribution of the Wasserstein estimation error, direct simulation is difficult because the tangent set T k T_k T k has a complex, generally nonconvex structure, and the optimization criterion involves a Gaussian process indexed by this set. In this section, we develop an m m m -out-of-n n n parametric bootstrap that approximates this distribution without explicitly constructing the limiting Gaussian process or transportation functional. We prove its consistency when m → ∞ m \to\infty m → ∞ and m / n → 0 m/n \to0 m / n → 0 , and establish convergence of bootstrap quantiles and asymptotically correct coverage of Wasserstein confidence regions under a quantile regularity condition. We also show that these regions can be implemented using Monte Carlo bootstrap replicates while retaining their asymptotic coverage as the number of replicates increases.
Consistency of the m m m -out-of-n n n parametric bootstrap
Let F n = σ ( X 1 , … , X n ) \mathcal{F}_n = \sigma(X_1,\ldots,X_n) F n = σ ( X 1 , … , X n ) be the σ \sigma σ -field generated by the original data, and let Q n = P G ^ n Q_n = P_{\widehat{G}_n} Q n = P G n be the fitted sampling distribution. For each sample size r r r , fix a measurable map A r A_r A r selecting a global maximum likelihood estimator over O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) . Let m = m n m = m_n m = m n be a deterministic bootstrap sample size satisfying
m → ∞ , m / n → 0. (24) m \to\infty, \qquad m/n \to0. \tag*{(24)} m → ∞ , m / n → 0. (24)
Conditional on F n \mathcal{F}_n F n , draw X 1 ∗ , … , X m ∗ ∼ i i d Q n X_1^*,\ldots,X_m^* \mathrel{\overset{\mathrm{iid}}{\sim}} Q_n X 1 ∗ , … , X m ∗ ∼ iid Q n and define G ^ m ∗ = A m ( X 1 ∗ , … , X m ∗ ) \widehat{G}_m^* = A_m(X_1^*,\ldots,X_m^*) G m ∗ = A m ( X 1 ∗ , … , X m ∗ ) as the bootstrap maximum likelihood estimator. The statistic
T n , m ∗ : = m 1 / 4 W 2 ( G ^ m ∗ , G ^ n ) (25) T_{n,m}^* := m^{1/4}W_2(\widehat{G}_m^*,\widehat{G}_n) \tag*{(25)} T n , m ∗ := m 1/4 W 2 ( G m ∗ , G n ) (25)
measures the scaled bootstrap estimation error around the original estimate. For comparison, let
T n : = n 1 / 4 W 2 ( G ^ n , G ∗ ) , T ∞ : = { Ω ( H † ) } 1 / 2 (26) T_n := n^{1/4}W_2(\widehat{G}_n,G_*), \qquad T_\infty:= \{\Omega(H^\dagger)\}^{1/2} \tag*{(26)} T n := n 1/4 W 2 ( G n , G ∗ ) , T ∞ := { Ω ( H † ) } 1/2 (26)
be the original estimation-error statistic and its weak limit. Write L ∗ \mathcal{L}^* L ∗ for conditional law given F n \mathcal{F}_n F n . For probability laws P P P and Q Q Q on R \mathbb{R} R , define the bounded-Lipschitz distance by
d B L ( P , Q ) : = sup f ∈ B L 1 ∣ ∫ f d P − ∫ f d Q ∣ , d_{\mathrm{BL}}(P,Q) := \sup_{f\in\mathrm{BL}_1}\left|\int f\,dP-\int f\,dQ\right|, d BL ( P , Q ) := f ∈ BL 1 sup ∫ f d P − ∫ f d Q ,
where B L 1 = { f : R → R : ∥ f ∥ ∞ ≤ 1 , ∣ f ( x ) − f ( y ) ∣ ≤ ∣ x − y ∣ , ∀ x , y ∈ R } \mathrm{BL}_1 = \{f:\mathbb{R}\to\mathbb{R}:\|f\|_\infty\le1,\ |f(x)-f(y)|\le|x-y|,\ \forall x,y\in\mathbb{R}\} BL 1 = { f : R → R : ∥ f ∥ ∞ ≤ 1 , ∣ f ( x ) − f ( y ) ∣ ≤ ∣ x − y ∣ , ∀ x , y ∈ R } . Although Θ \Theta Θ is compact, the scaled Wasserstein statistic T n = n 1 / 4 W 2 ( G ^ n , G ∗ ) T_n = n^{1/4}W_2(\widehat{G}_n,G_*) T n = n 1/4 W 2 ( G n , G ∗ ) need not be uniformly bounded in n n n . We therefore use d B L d_{\mathrm{BL}} d BL to express convergence of its distribution without imposing additional moment conditions.
Theorem 5.1 (Consistency of the m m m -out-of-n n n parametric bootstrap). Under Assumptions 2.1 –2.3 and (24),
d B L ( L ∗ ( T n , m ∗ ) , L ( T ∞ ) ) → P 0. (27) d_{\mathrm{BL}}\left(\mathcal{L}^*(T_{n,m}^*),\mathcal{L}(T_\infty)\right)\xrightarrow{P}0. \tag*{(27)} d BL ( L ∗ ( T n , m ∗ ) , L ( T ∞ ) ) P 0. (27) Consequently,
d B L ( L ∗ ( T n , m ∗ ) , L ( T n ) ) → P 0. (28) d_{\mathrm{BL}}\left(\mathcal{L}^*(T_{n,m}^*),\mathcal{L}(T_n)\right)\xrightarrow{P}0. \tag*{(28)} d BL ( L ∗ ( T n , m ∗ ) , L ( T n ) ) P 0. (28)
The proof of Theorem 5.1 is provided in Appendix § 13 . It establishes bootstrap consistency by comparing the fitted and true sampling distributions at sample size m m m , then controlling the change of center from G ∗ G_\ast G ∗ to G ^ n \widehat{G}_n G n .
Specifically, let Y 1 , … , Y m ∼ i i d P ∗ Y_1,\ldots,Y_m \mathrel{\overset{\mathrm{iid}}{\sim}} P_\ast Y 1 , … , Y m ∼ iid P ∗ be independent of the original data, and define G ~ m : = A m ( Y 1 , … , Y m ) \widetilde{G}_m := \mathcal{A}_m(Y_1,\ldots,Y_m) G m := A m ( Y 1 , … , Y m ) and T m 0 : = m 1 / 4 W 2 ( G ~ m , G ∗ ) T_m^0 := m^{1/4}W_2(\widetilde{G}_m,G_\ast) T m 0 := m 1/4 W 2 ( G m , G ∗ ) . Using the same measurable MLE selection for both samples gives
d B L ( L ∗ ( T n , m ∗ ) , L ( T m 0 ) ) ≤ m 1 / 4 W 2 ( G ^ n , G ∗ ) + 2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V (29) d_{\mathrm{BL}}\left(\mathcal{L}^\ast(T_{n,m}^\ast),\mathcal{L}(T_m^0)\right) \le m^{1/4}W_2(\widehat{G}_n,G_\ast)+2\left\|Q_n^{\otimes m}-P_\ast^{\otimes m}\right\|_{\mathrm{TV}} \tag*{(29)} d BL ( L ∗ ( T n , m ∗ ) , L ( T m 0 ) ) ≤ m 1/4 W 2 ( G n , G ∗ ) + 2 Q n ⊗ m − P ∗ ⊗ m TV (29)
The first term controls the centering error through the triangle inequality and is ( m / n ) 1 / 4 T n = o P ( 1 ) (m/n)^{1/4}T_n=o_P(1) ( m / n ) 1/4 T n = o P ( 1 ) , while the second controls the total variation distance between Q n ⊗ m Q_n^{\otimes m} Q n ⊗ m and P ∗ ⊗ m P_\ast^{\otimes m} P ∗ ⊗ m . It also vanishes because ρ ( G ^ n ) 2 = O P ( n − 1 ) \rho(\widehat{G}_n)^2=O_P(n^{-1}) ρ ( G n ) 2 = O P ( n − 1 ) , m / n → 0 m/n\to0 m / n → 0 , and the inequality
2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V ≤ [ { 1 + ρ ( G ^ n ) 2 } m − 1 ] 1 / 2 . 2\left\|Q_n^{\otimes m}-P_\ast^{\otimes m}\right\|_{\mathrm{TV}}\le\left[\{1+\rho(\widehat{G}_n)^2\}^{m}-1\right]^{1/2}. 2 Q n ⊗ m − P ∗ ⊗ m TV ≤ [ { 1 + ρ ( G n ) 2 } m − 1 ] 1/2 .
Finally, T m 0 ⇝ T ∞ T_m^0\rightsquigarrow T_\infty T m 0 ⇝ T ∞ as m → ∞ m\to\infty m → ∞ , which yields (27) ; comparison with T n ⇝ T ∞ T_n\rightsquigarrow T_\infty T n ⇝ T ∞ gives (28) .
The condition m / n → 0 m/n\to0 m / n → 0 provides a sufficient asymptotic rule, rather than an optimal finite-sample choice of m m m . Smaller bootstrap samples reduce the two comparison terms but may increase the finite-m m m error in approximating the limiting distribution. § 8 examines this tradeoff.
Critical values, confidence regions, and Monte Carlo implementation
The bootstrap approximation yields inference for both the Wasserstein error and its square. Since m W 2 2 ( G ^ m ∗ , G ^ n ) = ( T n , m ∗ ) 2 \sqrt{m}W_2^2(\widehat{G}_m^\ast,\widehat{G}_n)=(T_{n,m}^\ast)^2 m W 2 2 ( G m ∗ , G n ) = ( T n , m ∗ ) 2 , the continuous mapping theorem gives the following consequence.
Corollary 5.2 (Bootstrap validity for the squared Wasserstein statistic). Under the conditions of Theorem 5.1 ,
L ∗ { m W 2 2 ( G ^ m ∗ , G ^ n ) } ⇝ P L { Ω ( H † ) } . (30) \mathcal{L}^\ast\left\{\sqrt{m}W_2^2(\widehat{G}_m^\ast,\widehat{G}_n)\right\}\rightsquigarrow_P\mathcal{L}\{\Omega(H^\dagger)\}. \tag*{(30)} L ∗ { m W 2 2 ( G m ∗ , G n ) } ⇝ P L { Ω ( H † )} . (30)
To construct confidence regions, let F ∞ ( t ) : = P ( T ∞ ≤ t ) F_\infty(t):=P(T_\infty\le t) F ∞ ( t ) := P ( T ∞ ≤ t ) be the limiting distribution function and F n , m ∗ ( t ) : = P ∗ ( T n , m ∗ ≤ t ) F_{n,m}^\ast(t):=P^\ast(T_{n,m}^\ast\le t) F n , m ∗ ( t ) := P ∗ ( T n , m ∗ ≤ t ) its conditional bootstrap counterpart. For p 0 ∈ ( 0 , 1 ) p_0\in(0,1) p 0 ∈ ( 0 , 1 ) , define
q p 0 : = inf { t : F ∞ ( t ) ≥ p 0 } , q n , m ∗ ( p 0 ) : = inf { t : F n , m ∗ ( t ) ≥ p 0 } . q_{p_0}:=\inf\{t:F_\infty(t)\ge p_0\},\qquad q_{n,m}^\ast(p_0):=\inf\{t:F_{n,m}^\ast(t)\ge p_0\}. q p 0 := inf { t : F ∞ ( t ) ≥ p 0 } , q n , m ∗ ( p 0 ) := inf { t : F n , m ∗ ( t ) ≥ p 0 } .
as their generalized inverse quantiles. Convergence of the bootstrap quantile and exact asymptotic coverage require regularity of the limiting distribution at the chosen quantile.
Corollary 5.3 (Bootstrap quantiles and Wasserstein confidence regions). Under the conditions of Theorem 5.1 , fix α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) and put p 0 = 1 − α p_0=1-\alpha p 0 = 1 − α . Suppose that q p 0 : = inf { t : F ∞ ( t ) ≥ p 0 } q_{p_0}:=\inf\{t:F_\infty(t)\ge p_0\} q p 0 := inf { t : F ∞ ( t ) ≥ p 0 } is a regular quantile, meaning that
P ( T ∞ = q p 0 ) = 0 , F ∞ ( q p 0 ) = p 0 , F ∞ ( q p 0 + ε ) > p 0 for every ε > 0. (31) P(T_\infty=q_{p_0})=0,\qquad F_\infty(q_{p_0})=p_0,\qquad F_\infty(q_{p_0}+\varepsilon)>p_0\quad\text{for every }\varepsilon>0. \tag*{(31)} P ( T ∞ = q p 0 ) = 0 , F ∞ ( q p 0 ) = p 0 , F ∞ ( q p 0 + ε ) > p 0 for every ε > 0. (31) Then the bootstrap quantile is consistent, q n , m ∗ ( p 0 ) → P q p 0 q_{n,m}^\ast(p_0)\xrightarrow{P}q_{p_0} q n , m ∗ ( p 0 ) P q p 0 , and its associated critical value has asymptotically exact coverage:
P { n 1 / 4 W 2 ( G ^ n , G ∗ ) ≤ q n , m ∗ ( 1 − α ) } ⟶ 1 − α . (32) P\left\{n^{1/4}W_2(\widehat{G}_n,G_\ast)\le q_{n,m}^\ast(1-\alpha)\right\}\longrightarrow1-\alpha. \tag*{(32)} P { n 1/4 W 2 ( G n , G ∗ ) ≤ q n , m ∗ ( 1 − α ) } ⟶ 1 − α . (32)
Consequently, the Wasserstein confidence region
C n , 1 − α : = { G : W 2 ( G , G ^ n ) ≤ n − 1 / 4 q n , m ∗ ( 1 − α ) } (33) C_{n,1-\alpha}:=\left\{G:W_2(G,\widehat{G}_n)\le n^{-1/4}q_{n,m}^*(1-\alpha)\right\} \tag*{(33)} C n , 1 − α := { G : W 2 ( G , G n ) ≤ n − 1/4 q n , m ∗ ( 1 − α ) } (33)
satisfies P { G ∗ ∈ C n , 1 − α } → 1 − α P\{G^*\in C_{n,1-\alpha}\}\to1-\alpha P { G ∗ ∈ C n , 1 − α } → 1 − α .
Generalized inverse quantiles and their empirical approximations are standard in statistical practice and can be computed without assuming that the underlying distribution is continuous ([9 , 15 ] ). Condition (31) is needed for the quantile-consistency and exact-coverage conclusions, rather than for defining or computing the bootstrap quantile.
For implementation, generate B = B n B=B_n B = B n conditionally independent bootstrap replicates T n , m ∗ ( 1 ) , … , T n , m ∗ ( B ) T_{n,m}^{*(1)},\ldots,T_{n,m}^{*(B)} T n , m ∗ ( 1 ) , … , T n , m ∗ ( B ) and define
F ^ n , m , B ∗ ( t ) : = 1 B ∑ b = 1 B 1 { T n , m ∗ ( b ) ≤ t } \widehat{F}_{n,m,B}^*(t):=\frac{1}{B}\sum_{b=1}^{B}\mathbb{1}\{T_{n,m}^{*(b)}\le t\} F n , m , B ∗ ( t ) := B 1 b = 1 ∑ B 1 { T n , m ∗ ( b ) ≤ t }
as their empirical distribution function. Its generalized inverse,
q ^ n , m , B ∗ ( p 0 ) : = inf { t : F ^ n , m , B ∗ ( t ) ≥ p 0 } , \widehat{q}_{n,m,B}^*(p_0):=\inf\{t:\widehat{F}_{n,m,B}^*(t)\ge p_0\}, q n , m , B ∗ ( p 0 ) := inf { t : F n , m , B ∗ ( t ) ≥ p 0 } ,
provides a Monte Carlo approximation to the conditional bootstrap quantile.
Proposition 5.4 (Monte Carlo approximation of the conditional bootstrap). Under the conditions of Theorem 5.1 , if B = B n → ∞ B=B_n\to\infty B = B n → ∞ , then
sup t ∣ F ^ n , m , B ∗ ( t ) − F n , m ∗ ( t ) ∣ → P 0. (34) \sup_t\left|\widehat{F}_{n,m,B}^*(t)-F_{n,m}^*(t)\right|\xrightarrow{P}0. \tag*{(34)} t sup F n , m , B ∗ ( t ) − F n , m ∗ ( t ) P 0. (34) Under the quantile conditions of Corollary 5.3 ,
q ^ n , m , B ∗ ( 1 − α ) → P q 1 − α , (35) \widehat{q}_{n,m,B}^*(1-\alpha)\xrightarrow{P}q_{1-\alpha}, \tag*{(35)} q n , m , B ∗ ( 1 − α ) P q 1 − α , (35) and the implementable confidence region
C ^ n , 1 − α : = { G : W 2 ( G , G ^ n ) ≤ n − 1 / 4 q ^ n , m , B ∗ ( 1 − α ) } (36) \widehat{C}_{n,1-\alpha}:=\left\{G:W_2(G,\widehat{G}_n)\le n^{-1/4}\widehat{q}_{n,m,B}^*(1-\alpha)\right\} \tag*{(36)} C n , 1 − α := { G : W 2 ( G , G n ) ≤ n − 1/4 q n , m , B ∗ ( 1 − α ) } (36) has asymptotic coverage 1 − α 1-\alpha 1 − α .
Thus, replacing the conditional bootstrap quantile by its Monte Carlo estimate preserves asymptotic coverage as B → ∞ B\to\infty B → ∞ . The proofs of both corollaries and Proposition 5.4 are given in Appendix § 13 .
For nominal coverage 1 − α 1-\alpha 1 − α , Algorithm 1 constructs a Wasserstein confidence region without explicitly constructing the tangent set or the limiting Gaussian process. The coverage guarantee holds when m → ∞ m\to\infty m → ∞ , m / n → 0 m/n\to0 m / n → 0 , and B → ∞ B\to\infty B → ∞ , at a regular target quantile. Numerical approximation of the global maximizer and the finite-sample choice of m m m remain implementation choices, which we assess in the following experiments.
Experimental Results
Algorithm 1 Constructing a Wasserstein confidence region
Require: Observations X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n ; fitted upper order k k k ; bootstrap sample size m < n m<n m < n ; number of replicates B B B ; significance level α ∈ ( 0 , 1 ) \alpha\in(0,1) α ∈ ( 0 , 1 ) .
Ensure: Wasserstein confidence region C ^ n , 1 − α \widehat{\mathcal{C}}_{n,1-\alpha} C n , 1 − α .
Fit the global mixture likelihood over O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) to obtain G ^ n \widehat{G}_n G n .
for b = 1 , … , B b=1,\ldots,B b = 1 , … , B do
Generate X 1 ∗ ( b ) , … , X m ∗ ( b ) ∼ i i d P G ^ n X_1^{*(b)},\ldots,X_m^{*(b)}\stackrel{\mathrm{iid}}{\sim}P_{\widehat{G}_n} X 1 ∗ ( b ) , … , X m ∗ ( b ) ∼ iid P G n , independently across replicates.
Refit the global likelihood over O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) to obtain G ^ m ∗ ( b ) \widehat{G}_m^{*(b)} G m ∗ ( b ) .
Compute T n , m ∗ ( b ) = m 1 / 4 W 2 ( G ^ m ∗ ( b ) , G ^ n ) T_{n,m}^{*(b)}=m^{1/4}W_2(\widehat{G}_m^{*(b)},\widehat{G}_n) T n , m ∗ ( b ) = m 1/4 W 2 ( G m ∗ ( b ) , G n ) .
end for
Compute the empirical generalized inverse quantile
q ^ n , m , B ∗ ( 1 − α ) : = inf { t : 1 B ∑ b = 1 B 1 { T n , m ∗ ( b ) ≤ t } ≥ 1 − α } . \widehat{q}_{n,m,B}^{*}(1-\alpha):=\inf\left\{t:\frac{1}{B}\sum_{b=1}^{B}\mathbf{1}\left\{T_{n,m}^{*(b)}\leq t\right\}\geq1-\alpha\right\}. q n , m , B ∗ ( 1 − α ) := inf { t : B 1 b = 1 ∑ B 1 { T n , m ∗ ( b ) ≤ t } ≥ 1 − α } .
Set r n , α = n − 1 / 4 q ^ n , m , B ∗ ( 1 − α ) r_{n,\alpha}=n^{-1/4}\widehat{q}_{n,m,B}^{*}(1-\alpha) r n , α = n − 1/4 q n , m , B ∗ ( 1 − α ) .
return C ^ n , 1 − α = { G ∈ O k ( Θ ) : W 2 ( G , G ^ n ) ≤ r n , α } \widehat{\mathcal{C}}_{n,1-\alpha}=\left\{G\in\mathcal{O}_k(\Theta):W_2(G,\widehat{G}_n)\leq r_{n,\alpha}\right\} C n , 1 − α = { G ∈ O k ( Θ ) : W 2 ( G , G n ) ≤ r n , α } .
Experimental Results
In this section, we empirically assess the finite-sample implications of our theory through Gaussian location-mixture simulations with known covariance and a flow-cytometry application. Our evaluation focuses on two aspects: (i) the fourth-root scale of mixing-measure error and the influence of transportation geometry and the fitted component budget on its distribution; and (ii) the use of these regions to obtain model-based uncertainty bounds for population-level summaries that are invariant to component labeling.
Simulation Studies
We conduct numerical studies in Gaussian location mixtures with known covariance, beginning by investigating whether the fourth-root normalization stabilizes error distributions across sample sizes. To interpret the errors on this scale, we separate local displacement from mass reallocation and quantify their contributions to Wasserstein loss. Because the component budget constrains the attainable perturbations, we further study how changing this budget affects the error distribution and multivariate splitting geometry. This dependence of the error distribution on the fitted component budget also matters for inference, since upper quantiles of the normalized error determine the radii of Wasserstein confidence regions.
Throughout the simulated experiments, we keep the covariance fixed at I d I_d I d so that fitting varies only the component locations and mixing weights. For each study, we generate observations from a specified true mixing measure G ⋆ G^\star G ⋆ and fit a model allowing at most k k k components. All centers are restricted to Θ = [ − 5 , 5 ] d \Theta=[-5,5]^d Θ = [ − 5 , 5 ] d , unless stated otherwise. For each configuration, we repeat the experiment on independently generated datasets. The number of repetitions is reported in the corresponding subsection.
To approximate the global MLE considered in the theory, we use the expectation–maximization (EM) algorithm [5 ] with multiple initializations based on data quantiles, nearby component centers, small component weights, and random starts. At each EM iteration, we restrict the component centers to Θ \Theta Θ and keep the mixing weights positive and normalized to sum to one. Finally, we evaluate the error of the fitted mixing measures using Wasserstein distances, computed by monotone matching in one dimension and by solving a finite transport linear program in two dimensions.
Scaling of Mixing-Measure Error
We first examine how mixing-measure estimation error varies with sample size in an over-specified Gaussian mixture.
Setup. We consider a one-dimensional Gaussian location mixture with G ⋆ = δ 0 G^\star= \delta_0 G ⋆ = δ 0 and fitted upper order k = 2 k = 2 k = 2 . Although the observations come from a single Gaussian, the additional fitted component permits local splitting around the true center. This provides a simple over-specified setting in which to study the estimation error while keeping the sampling distribution fixed. We generate observations from N ( 0 , 1 ) N(0, 1) N ( 0 , 1 ) and repeat the experiment on 100 independent datasets for each n ∈ { 250 , 500 , 1000 , 2000 , 5000 , 10000 } n \in\{250, 500, 1000, 2000, 5000, 10000\} n ∈ { 250 , 500 , 1000 , 2000 , 5000 , 10000 } . For each fit, we record
T n = n 1 / 4 W 2 ( G ^ n , G ⋆ ) , V n = n W 2 2 ( G ^ n , G ⋆ ) . T_n = n^{1/4}W_2(\widehat{G}_n, G^\star), \qquad V_n = \sqrt{n}W_2^2(\widehat{G}_n, G^\star). T n = n 1/4 W 2 ( G n , G ⋆ ) , V n = n W 2 2 ( G n , G ⋆ ) .
The normalized errors allow us to compare distributions across sample sizes, while the corresponding unnormalized errors reveal how the magnitude of the estimation error decreases with n n n .
Results. In Figure 1 (a), the empirical distributions of T n T_{n} T n remain broadly similar across sample sizes after fourth-root normalization, suggesting that this normalization captures a common scale for the estimation error. By comparing the error distributions, we also consider their upper quantiles, which are not captured by the average error alone.
Figure 1. Wasserstein estimation error for a one-component Gaussian mixture fitted with at most two components. (a) Empirical CDFs of n 1 / 4 W 2 ( G ^ n , G ⋆ ) n^{1/4}W_2(\widehat{G}_n, G^\star) n 1/4 W 2 ( G n , G ⋆ ) across sample sizes. (b) Mean W 2 W_2 W 2 and W 2 2 W_2^2 W 2 2 errors on logarithmic axes, with reference lines proportional to n − 1 / 4 n^{-1/4} n − 1/4 and n − 1 / 2 n^{-1/2} n − 1/2 , respectively.
To evaluate the rate of decay, in Figure 1 (b) we plot the mean unnormalized errors against sample size on logarithmic axes and compare these estimates with − 1 / 4 -1/4 − 1/4 and − 1 / 2 -1/2 − 1/2 , which is the exponents associated with the theoretical scalings n − 1 / 4 n^{-1/4} n − 1/4 for W 2 W_{2} W 2 and n − 1 / 2 n^{-1/2} n − 1/2 for W 2 2 W_{2}^{2} W 2 2 , respectively. The mean errors decrease and generally follow the reference trends. Both theoretical exponents lie within the corresponding 95% Monte Carlo intervals in Table 1 , which indicates that the observed mean errors are compatible with these scalings over the simulated sample sizes.
Table 1: Estimated slopes of log mean Wasserstein error against log sample size. Each mean is computed from 100 independent datasets. The 95% Monte Carlo intervals are obtained by resampling the replicate errors within each sample size and refitting the slope, using 2,000 bootstrap resamples.
Table 1.
For comparison, fitting a single Gaussian to the same data gives a center estimate equal to the sample mean with probability tending to one, and its Wasserstein error has scale n − 1 / 2 n^{-1/2} n − 1/2 . This error decreases faster than the n − 1 / 4 n^{-1/4} n − 1/4 scale observed when two components are allowed. Consequently, multiplying the one-component error by n 1 / 4 n^{1/4} n 1/4 would make it converge to zero, whereas the normalized errors from the two-component fits remain on a comparable scale in Figure 1 (a). This comparison illustrates how allowing a redundant component can slow mixing-measure recovery, even though the data still come from the same single Gaussian.
Local Displacement and Mass Reallocation
In preceding experiments, we investigate how quickly mixing-measure error decreases with sample size. We continue by studying the contributions to this error when the true mixture contains more than one component. In this setting, the fitted measure can differ from the truth both in the locations of its atoms and in the mass assigned to each true component. Motivated by the transportation expansion in Proposition 4.1 , we separate these contributions to assess how much of the Wasserstein error comes from local displacement and how much comes from mass reallocation between true components.
Setup. We consider two-point true mixing measures with separations Δ ∈ { 1 , 2 , 4 } \Delta\in\{1,2,4\} Δ ∈ { 1 , 2 , 4 } , using both balanced and unequal mixing weights, and fit mixtures with at most three components. This allows the fitted measure to split a true component while also assigning different total masses to the two true components. For each configuration, we generate 60 independent datasets at sample sizes ranging from 500 to 5000.
Error decomposition. To measure local displacement, we assign each fitted center θ ^ j \hat{\theta}_j θ ^ j to a nearest true center θ ι ( j ) ∗ \theta^*_{\iota(j)} θ ι ( j ) ∗ and compute L n = ∑ j π ^ j ( θ ^ j − θ ι ( j ) ∗ ) 2 L_n = \sum_j \hat{\pi}_j(\hat{\theta}_j-\theta^*_{\iota(j)})^2 L n = ∑ j π ^ j ( θ ^ j − θ ι ( j ) ∗ ) 2 . This sum gives the total transportation cost of moving the fitted atoms onto their assigned true centers. After this movement, the resulting measure is
G ^ n c = ∑ i M ^ i δ θ i ∗ , M ^ i = ∑ j : ι ( j ) = i π ^ j . \widehat{G}^{c}_n = \sum_i \widehat{M}_i\delta_{\theta_i^*}, \qquad\widehat{M}_i = \sum_{j:\iota(j)=i}\widehat{\pi}_j. G n c = i ∑ M i δ θ i ∗ , M i = j : ι ( j ) = i ∑ π j .
The centers of G ^ n c \widehat{G}^{c}_n G n c now agree with those of G ∗ G^* G ∗ , but their weights may still differ. To quantify this difference in weights, we compute M n = W 2 2 ( G ^ n c , G ∗ ) M_n = W_2^2(\widehat{G}^{c}_n,G^*) M n = W 2 2 ( G n c , G ∗ ) . For a two-point truth, this cost equals the mass that must be transferred between the two true centers multiplied by their squared separation. From this perspective, M n M_n M n measures the cost of reallocating excess mass from one true component to the other.
The costs L n L_n L n and M n M_n M n describe two steps, namely moving fitted atoms onto their nearest true centers and then reallocating mass between the true centers. The optimal coupling can instead transport mass directly from the fitted centers to the true centers, so its cost need not equal L n + M n L_n+M_n L n + M n . We measure this discrepancy by the signed transport residual
R n O T = W 2 2 ( G ^ n , G ∗ ) ⏟ optimal transport cost − L n ⏟ local displacement cost − M n ⏟ mass-reallocation cost . R_n^{\mathrm{OT}} =
\underbrace{W_2^2(\widehat{G}_n,G^*)}_{\text{optimal transport cost}}
-
\underbrace{L_n}_{\text{local displacement cost}}
-
\underbrace{M_n}_{\text{mass-reallocation cost}}. R n OT = optimal transport cost W 2 2 ( G n , G ∗ ) − local displacement cost L n − mass-reallocation cost M n .
The residual measures how closely the sum of the two contributions approximates the actual Wasserstein loss. Since it can be positive or negative, we use its absolute value to quantify the size of this discrepancy. For a direct comparison with the scaled Wasserstein loss n W 2 2 ( G ^ n , G ∗ ) \sqrt{n}W_2^2(\widehat{G}_n,G^*) n W 2 2 ( G n , G ∗ ) , we report the local displacement cost, mass-reallocation cost, and absolute residual as n L n \sqrt{n}L_n n L n , n M n \sqrt{n}M_n n M n , and n ∣ R n O T ∣ \sqrt{n}|R_n^{\mathrm{OT}}| n ∣ R n OT ∣ , respectively.
Results. Figure 2 shows how these quantities vary with sample size at separation Δ = 2 \Delta=2 Δ = 2 , for both balanced and unequal mixing weights. Table 2 complements this comparison by reporting the median quantities at n = 5000 n=5000 n = 5000 for balanced truths with separations Δ = 2 \Delta=2 Δ = 2 and Δ = 4 \Delta=4 Δ = 4 . In both configurations, the median mass-reallocation cost is substantially larger than the median local displacement cost. For the balanced truth at Δ = 4 \Delta=4 Δ = 4 , the median scaled mass-reallocation cost is 4.549, compared with 0.875 for local displacement and 0.054 for the absolute residual. These results show that discrepancies in component masses can make a substantial contribution to Wasserstein error, even after the fitted atoms are moved onto the true centers.
Table 2: Median Wasserstein error and transport costs at n = 5000 n=5000 n = 5000 for a balanced two-component truth, fitted with at most three components. Each separation uses 60 independent datasets. Here T n = n 1 / 4 W 2 ( G ^ n , G ⋆ ) T_{n}=n^{1/4}W_{2}(\widehat{G}_{n},G^{\star}) T n = n 1/4 W 2 ( G n , G ⋆ ) , while the transport costs and absolute residual are scaled by n \sqrt{n} n . Medians are computed separately for each column and do not provide an additive decomposition.
Table 2.
Figure 2. Local displacement, mass reallocation, and absolute transport residual for a two-component Gaussian mixture fitted with at most three components. The true centers have separation Δ = 2 \Delta= 2 Δ = 2 , with mixing weights ( 0.5 , 0.5 ) (0.5, 0.5) ( 0.5 , 0.5 ) on the left and ( 0.8 , 0.2 ) (0.8, 0.2) ( 0.8 , 0.2 ) on the right. Curves show the median of each quantity after scaling by n \sqrt{n} n .
The decomposition also reveals differences that are less apparent from the total error. Although the median normalized Wasserstein errors T n T_n T n are similar at the two separations, the median local contribution increases from 0.344 at Δ = 2 \Delta= 2 Δ = 2 to 0.875 at Δ = 4 \Delta= 4 Δ = 4 . Meanwhile, the median mass-reallocation contributions remain close, at 4.363 and 4.549, respectively. Despite their similar total Wasserstein errors, the two configurations differ in the contributions from local displacement and mass reallocation. By separating these contributions, we can distinguish errors in the fitted locations from errors in the mass assigned to each true component.
Effect of Component Budget on Wasserstein Error
The fourth-root scale presented in § 8.1.1 describes the magnitude of mixing-measure error, but it does not by itself determine the error distribution. Our theory further characterizes this distribution through the local perturbations available to the fitted model, which depend on the permitted number of components. More specifically, increasing the component budget expands the set of attainable perturbations and may change the direction selected by likelihood maximization. To clarify the resulting effect on estimation error, we compare different fitted component budgets while keeping the true mixing measure fixed.
Setup. We generate observations from the balanced two-point mixing measure G ⋆ = 1 2 δ − 1 + 1 2 δ 1 G^\star= \frac{1}{2}\delta_{-1} + \frac{1}{2}\delta_1 G ⋆ = 2 1 δ − 1 + 2 1 δ 1 . To extend the analysis beyond the lower-fluorescence group, we compare the complete fitted mixing measures using Wasserstein distance. The case k = 2 k = 2 k = 2 matches the true number of components, whereas k = 3 k = 3 k = 3 and k = 4 k = 4 k = 4 allow one and two additional components, respectively. These are upper limits on the fitted component count, so the fit is not required to use every available component. For each configuration, we use 100 independently generated datasets at sample sizes ranging from 500 to 5000. We compare the resulting distributions of T n = n 1 / 4 W 2 ( G ^ n , G ⋆ ) T_n = n^{1/4}W_2(\widehat{G}_n, G^\star) T n = n 1/4 W 2 ( G n , G ⋆ ) through their medians and 90th percentiles. The medians summarize typical estimation error, whereas the upper percentiles describe larger errors and are relevant to confidence-region calibration.
Results. For the three fitted component budgets, Figure 3 compares the medians and 90th percentiles of T n T_n T n across sample sizes. In particular, at n = 5000 n = 5000 n = 5000 , the 90th percentile increases when the component budget changes from k = 2 k = 2 k = 2 to k = 3 k = 3 k = 3 , but decreases when the budget is further increased to k = 4 k = 4 k = 4 . These differences indicate that the estimated upper quantiles depend on the fitted component budget, even when the true mixing measure and sample size are held fixed.
Figure 3. Wasserstein estimation error n 1 / 4 W 2 ( G ^ n , G ⋆ ) n^{1/4}W_{2}(\widehat{G}_{n}, G^\star) n 1/4 W 2 ( G n , G ⋆ ) across sample sizes for fitted component budgets k = 2 , 3 , 4 k = 2, 3, 4 k = 2 , 3 , 4 , with G ⋆ = 1 2 δ − 1 + 1 2 δ 1 G^\star= \frac{1}{2}\delta_{-1} + \frac{1}{2}\delta_{1} G ⋆ = 2 1 δ − 1 + 2 1 δ 1 . Solid and dashed curves show the medians and 90th percentiles, respectively.
The dependence on the fitted component budget is consistent with its role in our weak-limit theory. The attainable tangent sets satisfy T 2 ⊂ T 3 ⊂ T 4 \mathcal{T}_2 \subset\mathcal{T}_3 \subset\mathcal{T}_4 T 2 ⊂ T 3 ⊂ T 4 , so increasing k k k gives likelihood maximization additional local perturbations to choose from. Because the selected perturbation maximizes the likelihood objective rather than minimizes transportation cost, this inclusion does not imply that Wasserstein error or its upper quantiles must increase with k k k . This distinction explains why the nonmonotone ordering observed at n = 5000 n = 5000 n = 5000 is compatible with the nested tangent sets, although the theory does not predict the relative sizes of these quantiles.
This experiment implies that the fourth-root scale alone is insufficient to determine a confidence-region radius. The radius also depends on an upper quantile of the error distribution, whose theoretical characterization involves the fitted component budget. To account for this dependence, bootstrap refits should use the same component budget as the original fit, so that the resulting critical values reflect the estimation procedure being assessed.
Component Budget and Multivariate Splitting Geometry
In § 8.1.3 , we show that changing the fitted component budget can affect the distribution of Wasserstein error. In multiple dimensions, additional components also allow different spatial arrangements around a true center. For example, fitted centers may spread mainly along a line or extend along a second independent direction. These arrangements need not be distinguishable from the total Wasserstein error alone, which motivates us to examine how the fitted component budget affects splitting geometry and how the observed configurations relate to the rank constraint in (7) .
Setup. We generate observations from N ( 0 , I 2 ) \mathcal{N}(0, I_{2}) N ( 0 , I 2 ) , corresponding to the true mixing measure G ⋆ = δ ( 0 , 0 ) G^{\star} = \delta_{(0,0)} G ⋆ = δ ( 0 , 0 ) , and fit Gaussian location mixtures with at most k = 2 k = 2 k = 2 or k = 3 k = 3 k = 3 components. By considering a single true component, we exclude mass reallocation between distinct true centers and focus on how the fitted atoms are arranged around the origin. Both budgets permit over-specification, and comparing them isolates the effect of allowing one additional fitted atom while leaving the sampling distribution unchanged. For each budget, we use 100 independently generated datasets at each sample size n ∈ { 500 , 1000 , 2000 , 5000 } n \in\{500, 1000, 2000, 5000\} n ∈ { 500 , 1000 , 2000 , 5000 } .
Measuring splitting geometry. To describe the arrangement of the fitted atoms, we compute their rescaled second-moment matrix about the true center, C ^ n = n ∑ j π ^ j θ ^ j θ ^ j ⊤ \widehat{C}_{n} = \sqrt{n} \sum_{j} \widehat{\pi}_{j} \widehat{\theta}_{j} \widehat{\theta}_{j}^{\top} C n = n ∑ j π j θ j θ j ⊤ . For any unit vector u u u , the quantity u ⊤ C ^ n u = n ∑ j π ^ j ( u ⊤ θ ^ j ) 2 u^{\top}\widehat{C}_{n}u = \sqrt{n} \sum_{j} \widehat{\pi}_{j}(u^{\top}\widehat{\theta}_{j})^{2} u ⊤ C n u = n ∑ j π j ( u ⊤ θ j ) 2 measures the weighted squared displacement of the fitted centers along direction u u u . Through these directional second moments, C ^ n \widehat{C}_{n} C n describes both how far the fitted atoms lie from the origin and how their displacements are distributed across directions. The factor n \sqrt{n} n places these squared displacements on the scale used for second-order local perturbations in our theory.
Let λ 1 ( C ^ n ) ≥ λ 2 ( C ^ n ) ≥ 0 \lambda_{1}(\widehat{C}_{n}) \ge\lambda_{2}(\widehat{C}_{n}) \ge0 λ 1 ( C n ) ≥ λ 2 ( C n ) ≥ 0 denote its eigenvalues. The larger eigenvalue measures the second moment along the dominant direction, while the smaller eigenvalue measures the second moment along the perpendicular direction. Whenever λ 1 ( C ^ n ) > 0 \lambda_{1}(\widehat{C}_{n}) > 0 λ 1 ( C n ) > 0 , we summarize their relative contributions by λ 2 ( C ^ n ) λ 1 ( C ^ n ) \frac{\lambda_{2}(\widehat{C}_{n})}{\lambda_{1}(\widehat{C}_{n})} λ 1 ( C n ) λ 2 ( C n ) . A ratio close to zero indicates that the fitted second moment is concentrated mainly along one direction. A larger ratio indicates a more substantial contribution from the second direction, with a ratio of one corresponding to equal contributions along the two principal directions. Because the common scaling factor cancels, the ratio describes the relative directional contributions independently of their overall magnitude.
This distinction matters because Wasserstein error summarizes the total displacement without specifying how it is distributed across directions. To be precise, for the single-atom truth considered here,
n W 2 2 ( G ^ n , G ⋆ ) = tr ( C ^ n ) = λ 1 ( C ^ n ) + λ 2 ( C ^ n ) . \sqrt{n} W_{2}^{2}(\widehat{G}_{n}, G^{\star}) = \operatorname{tr}(\widehat{C}_{n}) = \lambda_{1}(\widehat{C}_{n}) + \lambda_{2}(\widehat{C}_{n}). n W 2 2 ( G n , G ⋆ ) = tr ( C n ) = λ 1 ( C n ) + λ 2 ( C n ) .
Consequently, two fits can have the same scaled Wasserstein error even if one spreads mainly along a line and the other has a substantial contribution from a second direction. The eigenvalue ratio distinguishes these configurations. We report its median and 90th percentile across datasets to describe both typical fitted arrangements and those with a larger relative contribution from the second direction.
Our theory explains why these arrangements can depend on the component budget. Under (7) , a local second-moment matrix of rank r r r requires at least r + 1 r + 1 r + 1 nearby fitted atoms. With k = 2 k = 2 k = 2 , a nonzero limiting local splitting matrix therefore has rank at most one. Increasing the budget to k = 3 k = 3 k = 3 makes rank-two local splitting attainable. We investigate whether this additional splitting direction appears in finite-sample fits by comparing the eigenvalue ratios under k = 2 k = 2 k = 2 and k = 3 k = 3 k = 3 .
Results. Figure 4 reports the median and 90th percentile of the eigenvalue ratio across sample sizes. At n = 5000 n = 5000 n = 5000 , the two component budgets give the following results.
Figure 4. Medians and 90th percentiles of λ 2 ( C ^ n ) / λ 1 ( C ^ n ) \lambda_2(\widehat{C}_n)/\lambda_1(\widehat{C}_n) λ 2 ( C n ) / λ 1 ( C n ) across sample sizes for observations from N ( 0 , I 2 ) \mathcal{N}(0,I_2) N ( 0 , I 2 ) , fitted with at most k = 2 k=2 k = 2 or k = 3 k=3 k = 3 components. The quantities λ 1 ( C ^ n ) \lambda_1(\widehat{C}_n) λ 1 ( C n ) and λ 2 ( C ^ n ) \lambda_2(\widehat{C}_n) λ 2 ( C n ) are the largest and second-largest eigenvalues of the fitted second-moment matrix about the origin. Solid and dashed curves represent the medians and 90th percentiles, respectively.
With k = 2 k = 2 k = 2 , the median ratio is close to zero, and the 90th percentile is below 0.1. The second principal direction contributes little relative to the first, both at the median and at the upper percentile.
(ii) With k = 3 k=3 k = 3 , the 90th percentile is around 0.3, compared with less than 0.1 under k = 2 k=2 k = 2 . This higher upper percentile suggests that the additional component can accommodate more substantial spreading along the second direction.
The observed difference is consistent with the additional splitting directions permitted by (7) . This constraint concerns limiting local configurations, so it does not require the second eigenvalue of every finite-sample matrix under k = 2 k=2 k = 2 to be exactly zero. In particular, C ^ n \widehat{C}_n C n is measured about the true center, and finite-sample shifts of the fitted centers can contribute to its second eigenvalue. Similarly, making rank-two splitting attainable under k = 3 k=3 k = 3 does not require every dataset to produce a fit with substantial spreading in both directions. The experiment illustrates how the component budget restricts the geometry available to likelihood maximization, while the eigenvalue ratios reveal which directional contributions appear in the resulting fits. The eigenvalue ratios complement Wasserstein error by describing how the fitted atoms spread across directions, beyond the total magnitude of their displacements.
Flow-Cytometry Analysis with Wasserstein Uncertainty
Our theory provides uncertainty regions for an entire mixing measure, from which we derive bounds for population-level summaries without matching individual component labels across fits. We apply this approach to flow-cytometry measurements, where a broad fluorescence group may be represented by different numbers of mixture components. First, we examine how the fitted representation changes with the component budget. Furthermore, we construct bootstrap Wasserstein regions within each specified Gaussian model and use them to bound summaries of the mixing mass near a selected fluorescence profile.
Data and Gaussian Mixture Model
Data and Gaussian Mixture Model
Data and preprocessing. We use the public Rituximab dataset from the Bioconductor flowClust package [10 , 20 ] , containing n = 1,545 n = 1{,}545 n = 1 , 545 cell events and eight recorded channels. We analyze FL1.H, which measures anti-BrdU FITC fluorescence related to DNA synthesis, and FL3.H, which measures 7-AAD fluorescence related to DNA content. These two channels describe complementary aspects of the fluorescence profile of each event. We retain all events and transform their recorded intensities by Z i = ( asinh ( F i 1 / 150 ) , asinh ( F i 3 / 150 ) ) T Z_i = (\operatorname{asinh}(F_{i1}/150), \operatorname{asinh}(F_{i3}/150))^{\mathsf{T}} Z i = ( asinh ( F i 1 /150 ) , asinh ( F i 3 /150 ) ) T , where F i 1 F_{i1} F i 1 and F i 3 F_{i3} F i 3 denote the two channel intensities. The transformation accommodates zero measurements and compresses large values, and all subsequent fitting and transport calculations use these transformed coordinates.
Model and fitting procedure. We fit Gaussian location mixtures of the form
p G ( z ) = ∑ j = 1 k π j ϕ 2 ( z ; μ j , σ 2 I 2 ) , G = ∑ j = 1 k π j δ μ j , μ j ∈ [ − 1 , 5 ] 2 , (37) p_G(z) = \sum_{j=1}^{k} \pi_j \phi_2(z; \mu_j, \sigma^2 I_2), \qquad G = \sum_{j=1}^{k} \pi_j \delta_{\mu_j}, \qquad\mu_j \in[-1,5]^2, \tag*{(37)} p G ( z ) = j = 1 ∑ k π j ϕ 2 ( z ; μ j , σ 2 I 2 ) , G = j = 1 ∑ k π j δ μ j , μ j ∈ [ − 1 , 5 ] 2 , (37)
using fixed σ = 0.30 \sigma= 0.30 σ = 0.30 and component budgets k ∈ { 2 , 3 , 4 } k \in\{2,3,4\} k ∈ { 2 , 3 , 4 } . Each atom of G G G represents a latent fluorescence-profile center, with its weight giving the mixing mass assigned to that profile. Fixing the covariance leaves the centers and weights as the quantities to be estimated and gives the Gaussian location family considered in Corollary 2.4 . We treat this specification as a working model: its fitted centers describe the fluorescence distribution under the chosen kernel, and their biological interpretation requires additional validation.
For each budget, we fit the full dataset by multistart EM with 35 initializations and retain the fit with the highest likelihood. We also assess density prediction using a common fixed 80–20 training/test split. For this comparison, each model is fitted on the training observations and evaluated on the held-out observations. We use these held-out likelihoods to assess predictive fit and the full-data estimates to obtain the population summaries and bootstrap regions below.
Component Splitting and Mixing-Mass Stability
We report the fitted weights and likelihood summaries for the three component budgets in Table 3 . For component budgets k = 2 , 3 , 4 k = 2, 3, 4 k = 2 , 3 , 4 , the held-out log likelihood per event is − 1.382 -1.382 − 1.382 , − 1.263 -1.263 − 1.263 , and − 1.187 -1.187 − 1.187 , respectively. The increase across these budgets indicates better density prediction on the common test set. To understand what changes in the latent representation accompany this improvement, we examine the fitted centers and the mass assigned to the main fluorescence groups. Figure 5 shows the three- and four-component fits over the transformed observations. With three components, the principal lower-fluorescence group is represented by an atom near ( 1.2 , 1.0 ) (1.2, 1.0) ( 1.2 , 1.0 ) with weight 0.726. As for the four-component fit, two nearby atoms describe this region, carrying weights 0.098 and 0.643. Their combined weight is about 0.74, close to that of the single atom in the three-component fit. Although the representation changes from one atom to two, the total mass assigned to the broad fluorescence region remains similar. This change makes a componentwise comparison ambiguous. The weight 0.726 in the three-component fit has no single counterpart in the four-component fit because two atoms contribute to the same broad region. Combining their contributions gives a more direct description of the mass in that region. To compare this region across different component representations, we use summaries based on the locations and weights of all contributing atoms.
Table 3. Rituximab mixture fits with fixed σ = 0.30 \sigma= 0.30 σ = 0.30 . Log likelihood, BIC, and component weights use the full-data fits; held-out log likelihood uses a common 80–20 training/test split. Within each fit, components are ordered by the first coordinate of their centers. Component indices do not imply matching across budgets, and a dash denotes an absent component.
Figure 5. Transformed Rituximab observations and fitted mixing atoms, with their weights, for (a) three and (b) four components. The principal lower-fluorescence region receives about 73% and 74% of the total mixing mass, respectively, while being represented by different numbers of atoms.
We also compare the complete fitted mixing measures using Wasserstein distance. Exact discrete transport gives W 2 ( G ^ 2 , G ^ 3 ) = 0.245 W_2(\hat{G}_2,\hat{G}_3) = 0.245 W 2 ( G ^ 2 , G ^ 3 ) = 0.245 and W 2 ( G ^ 3 , G ^ 4 ) = 0.237 W_2(\hat{G}_3,\hat{G}_4) = 0.237 W 2 ( G ^ 3 , G ^ 4 ) = 0.237 . These distances account jointly for changes in locations and weights without requiring a one-to-one correspondence between atoms. They describe sensitivity to the component budget. To assess sampling uncertainty for a given budget, we next keep that budget fixed and use the bootstrap to construct a region around its fitted measure.
Bootstrap Wasserstein Regions
Bootstrap construction. For each k ∈ { 3 , 4 } k \in\{3,4\} k ∈ { 3 , 4 } , we generate B = 250 B = 250 B = 250 parametric bootstrap datasets from the corresponding fitted observation distribution. Each bootstrap dataset is refitted using eight EM initializations, with the component budget, covariance, and center constraints held fixed. Through these refits, we quantify sampling uncertainty separately for each component budget. We use two bootstrap sample sizes, m ∈ { 118 , 355 } = { ⌊ n 0.65 ⌋ , ⌊ n 0.80 ⌋ } m \in\{118,355\} = \{\lfloor n^{0.65}\rfloor,\lfloor n^{0.80}\rfloor\} m ∈ { 118 , 355 } = {⌊ n 0.65 ⌋ , ⌊ n 0.80 ⌋} , and record T n , m ∗ ( b ) = m 1 / 4 W 2 ( G ^ m ∗ ( b ) , G ^ n ) T_{n,m}^{*(b)} = m^{1/4}W_2(\hat{G}_m^{*(b)},\hat{G}_n) T n , m ∗ ( b ) = m 1/4 W 2 ( G ^ m ∗ ( b ) , G ^ n ) for b = 1 , … , B b = 1,\ldots,B b = 1 , … , B . We use these normalized bootstrap errors to estimate a critical value on the fourth-root scale. Following Corollary 5.3 , we convert this critical value to a radius for the original sample size by
r ^ .95 ( m ) = n − 1 / 4 Quantile .95 { T n , m ∗ ( 1 ) , … , T n , m ∗ ( B ) } . (38) \hat{r}_{.95}(m) = n^{-1/4}\operatorname{Quantile}_{.95}\left\{T_{n,m}^{*(1)},\ldots,T_{n,m}^{*(B)}\right\}. \tag*{(38)} r ^ .95 ( m ) = n − 1/4 Quantile .95 { T n , m ∗ ( 1 ) , … , T n , m ∗ ( B ) } . (38)
The resulting ball consists of mixing measures within distance r ^ .95 ( m ) \hat{r}_{.95}(m) r ^ .95 ( m ) of G ^ n \hat{G}_n G ^ n in the specified model class. Its nominal 95% coverage interpretation follows from the assumptions of the corollary, with the likelihood maximizers approximated numerically by multistart EM.
Results. The estimated critical values and radii are given in Table 4 . Figure 6 (a) shows that all four radii are close to 0.2. At m = 355 m = 355 m = 355 , the radii under k = 3 k = 3 k = 3 and k = 4 k = 4 k = 4 are nearly equal, although the corresponding uncertainty regions are centered at different fitted mixing measures. The between-budget distance and the bootstrap radii address separate questions. The distance W 2 ( G ^ 3 , G ^ 4 ) = 0.237 W_2(\hat{G}_3,\hat{G}_4) = 0.237 W 2 ( G ^ 3 , G ^ 4 ) = 0.237 measures the change produced by allowing an additional component. Each bootstrap radius instead measures uncertainty within one fixed model specification. We retain both comparisons because a population summary can be sensitive to the permitted representation as well as to sampling variability.
Table 4. Bootstrap critical values and radii of nominal 95% Wasserstein confidence regions for the Rituximab mixing measure (n = 1545 n = 1545 n = 1545 , fixed σ = 0.30 \sigma= 0.30 σ = 0.30 ). Each ( k , m ) (k,m) ( k , m ) configuration uses B = 250 B = 250 B = 250 parametric bootstrap samples of size m m m , with at most k k k components in both the original fit and bootstrap refits. Here q ^ .95 ∗ \widehat{q}^{*}_{.95} q .95 ∗ is the empirical 95th percentile of m 1 / 4 W 2 ( G ^ m ∗ , G ^ n ) m^{1/4}W_{2}(\widehat{G}^{*}_{m}, \widehat{G}_{n}) m 1/4 W 2 ( G m ∗ , G n ) , and r ^ .95 ( m ) = n − 1 / 4 q ^ .95 ∗ \widehat{r}_{.95}(m) = n^{-1/4}\widehat{q}^{*}_{.95} r .95 ( m ) = n − 1/4 q .95 ∗ is the radius of the ball centered at G ^ n \widehat{G}_{n} G n .
Figure 6. Bootstrap uncertainty estimates for the Rituximab mixing measures with fitted component budgets k = 3 , 4 k = 3, 4 k = 3 , 4 . (a) Radii of nominal 95% Wasserstein regions for bootstrap sample sizes m = 118 , 355 m = 118, 355 m = 118 , 355 . (b) Smooth-gate mass estimates and intervals using m = 355 m = 355 m = 355 , c = ( 1.16 , 1.05 ) c = (1.16, 1.05) c = ( 1.16 , 1.05 ) , and h = 1.20 h = 1.20 h = 1.20 . Points denote the estimates, and horizontal bars show the intervals obtained by adding and subtracting the corresponding Wasserstein radius divided by h h h .
Bounds for Population-Level Summaries
We now use the Wasserstein regions to bound the mixing mass near a selected fluorescence profile. A direct sum of nearby component weights requires deciding which fitted atoms belong to the region. Such a sum can change abruptly when an atom crosses the chosen boundary. To obtain a summary that varies gradually as fitted centers move, we use the smooth gate
ψ c , h ( μ ) = ( 1 − ∥ μ − c ∥ h ) + , M c , h ( G ) = ∫ ψ c , h ( μ ) d G ( μ ) , \psi_{c,h}(\mu) = \left(1-\frac{\lVert\mu-c\rVert}{h}\right)_{+}, \qquad M_{c,h}(G)=\int\psi_{c,h}(\mu)\,dG(\mu), ψ c , h ( μ ) = ( 1 − h ∥ μ − c ∥ ) + , M c , h ( G ) = ∫ ψ c , h ( μ ) d G ( μ ) ,
where c ∈ R 2 c \in\mathbb{R}^{2} c ∈ R 2 specifies the profile of interest and h > 0 h>0 h > 0 specifies the neighborhood width. The gate assigns value one at c c c , decreases linearly with distance, and assigns zero beyond distance h h h . For a fitted discrete measure, this gives M c , h ( G ^ n ) = ∑ j π ^ j ψ c , h ( μ ^ j ) M_{c,h}(\widehat{G}_n)=\sum_j \widehat{\pi}_j\psi_{c,h}(\widehat{\mu}_j) M c , h ( G n ) = ∑ j π j ψ c , h ( μ j ) . Every fitted atom contributes according to both its weight and its distance from c c c . Whether the region is represented by one atom or several, this definition combines their contributions without depending on component labels. The quantity describes the latent fluorescence-profile centers under the mixture model; it differs from the fraction of observed events falling inside a hard gate.
Uncertainty bounds. To quantify uncertainty in this summary, we bound its deviation from the fitted value in terms of the Wasserstein distance. Since ψ c , h \psi_{c,h} ψ c , h has Lipschitz constant 1 / h 1/h 1/ h , the Kantorovich–Rubinstein bound and W 1 ≤ W 2 W_{1}\le W_{2} W 1 ≤ W 2 imply
∣ M c , h ( G ) − M c , h ( G ^ n ) ∣ ≤ 1 h W 1 ( G , G ^ n ) ≤ 1 h W 2 ( G , G ^ n ) . (39) \left|M_{c,h}(G)-M_{c,h}(\widehat{G}_n)\right|\le\frac{1}{h}W_{1}(G,\widehat{G}_n)\le\frac{1}{h}W_{2}(G,\widehat{G}_n). \tag*{(39)} M c , h ( G ) − M c , h ( G n ) ≤ h 1 W 1 ( G , G n ) ≤ h 1 W 2 ( G , G n ) . (39)
Hence, every mixing measure in a Wasserstein ball of radius r r r has smooth-gate mass within r / h r/h r / h of the fitted value. Because the mass lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] , the induced interval is
[ max { 0 , M c , h ( G ^ n ) − r / h } , min { 1 , M c , h ( G ^ n ) + r / h } ] . \left[\max\{0, M_{c,h}(\widehat{G}_{n}) - r/h\},\ \min\{1, M_{c,h}(\widehat{G}_{n}) + r/h\}\right]. [ max { 0 , M c , h ( G n ) − r / h } , min { 1 , M c , h ( G n ) + r / h } ] .
This inequality holds for every c c c and h > 0 h > 0 h > 0 . On the event that the target mixing measure belongs to the confidence ball, all these bounds hold simultaneously, so the same region can be used to study several fluorescence profiles without constructing a separate bootstrap region for each summary.
For a fixed Wasserstein radius r r r , the bandwidth h h h determines both the size of the neighborhood around c c c and the uncertainty bound r / h r/h r / h . In particular, a smaller h h h focuses the summary on a narrower neighborhood, but increases the uncertainty allowance r / h r/h r / h for a fixed Wasserstein radius. By contrast, a larger h h h averages contributions over a broader neighborhood and gives a smaller allowance. This dependence follows directly from how quickly the gate changes with the location of an atom.
Results. For the lower-fluorescence region, we take c = ( 1.16 , 1.05 ) c = (1.16, 1.05) c = ( 1.16 , 1.05 ) and h = 1.20 h = 1.20 h = 1.20 , using the bootstrap radii obtained with m = 355 m = 355 m = 355 . The estimated smooth-gate mass is 0.710 under k = 3 k = 3 k = 3 and 0.681 under k = 4 k = 4 k = 4 . As shown in Table 5 and Figure 6 (b), the corresponding confidence intervals have similar widths and overlap substantially. To interpret these estimates, recall that the smooth gate weights each atom’s contribution by its distance from c c c , whereas the combined weights reported in § 8.2.3 simply sum the weights of the atoms assigned to the group. Although the lower-fluorescence group is represented by different numbers of atoms, the same summary remains well defined under both budgets, and each fitted model supplies an associated uncertainty interval through its Wasserstein region. This suggests a practical use of inference for the complete mixing measure, as a bound on its estimation error yields uncertainty bounds for population summaries that remain meaningful as the atomic representation changes.
Table 5. Estimates and conservative confidence intervals for the lower-fluorescence smooth-gate mass in the Rituximab data. The mass estimate is M c , h ( G ^ n ) = ∑ j π ^ j ( 1 − ∥ μ ^ j − c ∥ / h ) + M_{c,h}(\widehat{G}_{n}) = \sum_j \widehat{\pi}_j(1 - \lVert\widehat{\mu}_j - c\rVert/h)_{+} M c , h ( G n ) = ∑ j π j ( 1 − ∥ μ j − c ∥ / h ) + , with c = ( 1.16 , 1.05 ) c = (1.16, 1.05) c = ( 1.16 , 1.05 ) and h = 1.20 h = 1.20 h = 1.20 . Each interval is the estimate plus or minus the error bound r ^ .95 ( m ) / h \widehat{r}_{.95}(m)/h r .95 ( m ) / h , truncated to [ 0 , 1 ] [0,1] [ 0 , 1 ] . The radii are from the nominal 95% Wasserstein regions in Table 4 , using m = 355 m = 355 m = 355 , B = 250 B = 250 B = 250 , and fixed σ = 0.30 \sigma= 0.30 σ = 0.30 . Endpoints use unrounded values.
Discussion
We studied the weak convergence of the Wasserstein estimation error for global maximum likelihood estimation in finite mixtures with a fixed upper bound on the number of components. Our analysis follows the density-to-mixing-measure approach used in convergence-rate theory, but develops a distributional counterpart to the inverse inequalities underlying that approach. We first derive a uniform quadratic expansion of the log-likelihood difference and characterize the attainable relative-density directions under the finite component budget. The density weak limit is then determined by maximizing a Gaussian quadratic functional over a generally nonconvex set. A sequential expansion of the squared Wasserstein distance connects this selected direction to the leading transportation cost, yielding joint weak convergence of the relative-density estimator and its Wasserstein error. We also establish an m m m -out-of-n n n parametric bootstrap that approximates the resulting distribution and provides Wasserstein confidence regions at regular target quantiles.
Our approach opens several directions for further investigation. On the theoretical side, a natural extension is to heteroscedastic location-scale mixtures, where scale parameters vary across components. As discussed in Remark 2.5 , the derivative identities of Gaussian location-scale kernels lead to higher-order cancellations and require a different characterization of attainable directions.
Related progress for infinite mixtures with an unknown shared scale is provided by [1 ] , who establish inverse inequalities connecting density discrepancies to errors in the location mixing measure and scale parameter. Extending the present weak-limit theory to component-specific scales would require identifying the appropriate higher-order density directions and the corresponding transportation expansion. The interaction between derivative cancellations and the number of redundant components would be central to this analysis.
On the methodological and experimental side, a natural direction is to connect our likelihood geometry with mixture-order selection and penalized estimation [16 , 22 ] , accounting for the singular structure that also motivates information criteria beyond regular likelihood theory [7 ] . Increasing the fitted order enlarges the model and can improve the maximized log-likelihood, even when the additional components are redundant. A joint weak-limit analysis of these improvements across candidate orders could clarify how penalties affect the probabilities of selecting different component counts. Numerical studies could then examine penalty calibration and bootstrap approximation of selection probabilities under varying component separation and optimization accuracy.
Background and Preliminary Results
Consistency and local geometry
Before presenting the detailed proofs, we recall several facts used repeatedly below. First, Cauchy–Schwarz inequality gives
∫ ∣ p G − p ∗ ∣ d μ = P ∗ ∣ p G − p ∗ p ∗ ∣ ≤ ∥ p G − p ∗ p ∗ ∥ ∗ = ρ ( G ) , \int|p_G-p_*|\,d\mu= P_* \left|\frac{p_G-p_*}{p_*}\right| \le\left\|\frac{p_G-p_*}{p_*}\right\|_* = \rho(G), ∫ ∣ p G − p ∗ ∣ d μ = P ∗ p ∗ p G − p ∗ ≤ p ∗ p G − p ∗ ∗ = ρ ( G ) ,
so convergence in relative L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) norm implies L 1 ( μ ) L^1(\mu) L 1 ( μ ) convergence of the mixture densities. Throughout the proofs, unless otherwise stated, ∇ f θ \nabla f_\theta ∇ f θ , ∇ 2 f θ \nabla^2 f_\theta ∇ 2 f θ , and ∇ r f θ \nabla^r f_\theta ∇ r f θ denote derivatives with respect to the parameter θ \theta θ . Second, Assumption 2.1 implies, uniformly in θ ∈ Θ \theta\in\Theta θ ∈ Θ ,
∣ f θ p ∗ ∣ ≤ F , ∥ ∇ r f θ p ∗ ∥ ≤ F , r = 1 , 2 , 3 , \left|\frac{f_\theta}{p_*}\right| \le F,\qquad\left\|\frac{\nabla^r f_\theta}{p_*}\right\| \le F,\quad r=1,2,3, p ∗ f θ ≤ F , p ∗ ∇ r f θ ≤ F , r = 1 , 2 , 3 ,
with F ∈ L 2 + η ( P ∗ ) F \in L^{2+\eta}(P_*) F ∈ L 2 + η ( P ∗ ) . In particular, all displayed relative derivatives belong to L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . The dominated convergence theorem also gives L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) -continuity of θ ↦ f θ / p ∗ \theta\mapsto f_\theta/p_* θ ↦ f θ / p ∗ and of its derivatives through order two. Whenever we pass a kernel, gradient, Hessian, or exact secant to a parameter limit below, this is the domination being used. We also use the subsequence characterization of convergence in probability: Z n → 0 Z_n \to0 Z n → 0 in probability if and only if every subsequence contains a further subsequence along which Z n → 0 Z_n \to0 Z n → 0 almost surely. For deterministic sequences, compactness and the bounds established below allow us to extract subsequences along which the relevant coefficients and atom locations converge.
Consistency and local geometry
We establish consistency, an exact density decomposition, and local comparisons between relative-density ρ ( G ) \rho(G) ρ ( G ) and Wasserstein discrepancies W 2 ( G , G ∗ ) W_2(G,G_*) W 2 ( G , G ∗ ) . These results provide the geometric ingredients for the convergence-rate and weak-limit arguments developed below, after establishing the required empirical-process properties.
Lemma A.1 (Consistency). Under Assumptions 2.1 2.1 2.1 -2.3 2.3 2.3 , we have
W 1 ( G ^ n , G ∗ ) → P 0 , ρ ( G ^ n ) : = ∥ p G ^ n − p ∗ p ∗ ∥ ∗ → P 0. W_1(\widehat{G}_n,G_*) \xrightarrow{P} 0,\qquad\rho(\widehat{G}_n):=\left\|\frac{p_{\widehat{G}_n}-p_*}{p_*}\right\|_* \xrightarrow{P} 0. W 1 ( G n , G ∗ ) P 0 , ρ ( G n ) := p ∗ p G n − p ∗ ∗ P 0.
Proof. This is a standard compact M M M -estimation argument once the parameter is viewed as a mixing measure, see Kiefer and Wolfowitz [17 ] ; Leroux [18 ] ; van der Vaart [31 ] for comparison. For completeness, we provide a proof below.
Consistency of G ^ n \widehat{G}_n G n . We prove that W 1 ( G ^ n , G ∗ ) → P 0 W_1(\widehat{G}_n,G_*) \xrightarrow{P} 0 W 1 ( G n , G ∗ ) P 0 . As Θ \Theta Θ is a compact set, we can verify that O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) is a weakly compact set. This weak compactness can similarly be understood that O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) is compact under any Wasserstein metric, as these distances can metrize the weak convergence in compact support.
Consider the functional M : O k ( Θ ) → R M:\mathcal{O}_k(\Theta)\to\mathbb{R} M : O k ( Θ ) → R , G ↦ P ∗ log p G G\mapsto P_*\log p_G G ↦ P ∗ log p G , then M M M is continuous. Indeed, under Assumption 2.2 2.2 2.2 about integrable envelope for the log-likelihood class, for G m ⇒ G G_m\Rightarrow G G m ⇒ G , using dominated convergence theorem, we have M ( G m ) → M ( G ) M(G_m)\to M(G) M ( G m ) → M ( G ) . Furthermore, for any G ∈ O k ( Θ ) G\in\mathcal{O}_k(\Theta) G ∈ O k ( Θ ) ,
M ( G ) − M ( G ∗ ) = P ∗ log p G p ∗ = − D K L ( P ∗ ∥ P G ) ≤ 0 , M(G)-M(G_*)=P_*\log\frac{p_G}{p_*}=-D_{\mathrm{KL}}(P_*\|P_G)\leq0, M ( G ) − M ( G ∗ ) = P ∗ log p ∗ p G = − D KL ( P ∗ ∥ P G ) ≤ 0 ,
where the equality holds only if p G = p ∗ p_G=p_* p G = p ∗ , μ \mu μ -a.e., which implies that G = G ∗ G=G_* G = G ∗ by Assumption 2.2 2.2 2.2 . Hence G ∗ G_* G ∗ is the unique maximizer of M M M .
Fix ε > 0 \varepsilon>0 ε > 0 and let K ε = { G ∈ O k ( Θ ) : W 1 ( G , G ∗ ) ≥ ε } K_\varepsilon=\{G\in\mathcal{O}_k(\Theta):W_1(G,G_*)\geq\varepsilon\} K ε = { G ∈ O k ( Θ ) : W 1 ( G , G ∗ ) ≥ ε } , which is a compact set. Since M M M is continuous and G ∗ G_* G ∗ is its unique maximizer, there exists a positive gap δ ε \delta_\varepsilon δ ε between M ( G ∗ ) M(G_*) M ( G ∗ ) and M ( G ) M(G) M ( G ) for G ∈ K ε G\in K_\varepsilon G ∈ K ε , in other words,
δ ε : = M ( G ∗ ) − sup G ∈ K ε M ( G ) > 0. \delta_\varepsilon:=M(G_*)-\sup_{G\in K_\varepsilon}M(G)>0. δ ε := M ( G ∗ ) − G ∈ K ε sup M ( G ) > 0.
Assumption 2.2 2.2 2.2 that the class { log p G : G ∈ O k ( Θ ) } \{\log p_G:G\in\mathcal{O}_k(\Theta)\} { log p G : G ∈ O k ( Θ )} is P ∗ P_* P ∗ -Glivenko-Cantelli with integrable envelop implies that
sup G ∈ O k ( Θ ) ∣ P n log p G − P ∗ log p G ∣ → P 0. \sup_{G\in\mathcal{O}_k(\Theta)}\left|P_n\log p_G-P_*\log p_G\right|\xrightarrow{P}0. G ∈ O k ( Θ ) sup ∣ P n log p G − P ∗ log p G ∣ P 0.
On the event that the preceding supremum is smaller than δ ε / 3 \delta_\varepsilon/3 δ ε /3 , every G ∈ K ε G\in K_\varepsilon G ∈ K ε satisfies
P n log p G ≤ M ( G ) + δ ε 3 ≤ M ( G ∗ ) − 2 δ ε 3 ≤ P n log p ∗ − δ ε 3 . P_n\log p_G\leq M(G)+\frac{\delta_\varepsilon}{3}\leq M(G_*)-\frac{2\delta_\varepsilon}{3}\leq P_n\log p_*-\frac{\delta_\varepsilon}{3}. P n log p G ≤ M ( G ) + 3 δ ε ≤ M ( G ∗ ) − 3 2 δ ε ≤ P n log p ∗ − 3 δ ε .
Such a G G G cannot maximize the empirical likelihood. Therefore P { W 1 ( G ^ n , G ∗ ) ≥ ε } → 0 \mathbb{P}\{W_1(\widehat{G}_n,G_*)\geq\varepsilon\}\to0 P { W 1 ( G n , G ∗ ) ≥ ε } → 0 .
Continuity of the ρ ( G ^ n ) \rho(\widehat{G}_n) ρ ( G n ) . We prove that ρ ( G ^ n ) → P 0 \rho(\widehat{G}_n)\xrightarrow{P}0 ρ ( G n ) P 0 . Consider a sequence { G m } m ≥ 0 ⊂ O k ( Θ ) \{G_m\}_{m\geq0}\subset\mathcal{O}_k(\Theta) { G m } m ≥ 0 ⊂ O k ( Θ ) such that G m ⇒ G ∗ G_m\Rightarrow G_* G m ⇒ G ∗ . For P ∗ P_* P ∗ -a.e. x x x , we have p G m ( x ) / p ∗ ( x ) → 1 p_{G_m}(x)/p_*(x)\to1 p G m ( x ) / p ∗ ( x ) → 1 . Moreover, Assumption 2.1 2.1 2.1 with r = 0 r=0 r = 0 gives
0 ≤ p G m ( x ) p ∗ ( x ) = ∫ f θ ( x ) p ∗ ( x ) d G m ( θ ) ≤ F ( x ) , 0\leq\frac{p_{G_m}(x)}{p_*(x)}=\int\frac{f_\theta(x)}{p_*(x)}\,dG_m(\theta)\leq F(x), 0 ≤ p ∗ ( x ) p G m ( x ) = ∫ p ∗ ( x ) f θ ( x ) d G m ( θ ) ≤ F ( x ) ,
which implies
∣ p G ^ m ( x ) − p ∗ ( x ) p ∗ ( x ) ∣ 2 ≤ ( F ( x ) + 1 ) 2 . \left|\frac{p_{\widehat{G}_m}(x)-p_*(x)}{p_*(x)}\right|^2 \le(F(x)+1)^2. p ∗ ( x ) p G m ( x ) − p ∗ ( x ) 2 ≤ ( F ( x ) + 1 ) 2 .
Because F ∈ L 2 + η ( P ∗ ) F \in L^{2+\eta}(P_*) F ∈ L 2 + η ( P ∗ ) , the dominating function ( F + 1 ) 2 (F+1)^2 ( F + 1 ) 2 is integrable. Applying Dominated Convergence Theorem, we have
∥ p G ^ m − p ∗ p ∗ ∥ ∗ → 0. (40) \left\|\frac{p_{\widehat{G}_m}-p_*}{p_*}\right\|_* \to0. \tag*{(40)} p ∗ p G m − p ∗ ∗ → 0. (40)
We now pass from W 2 W_2 W 2 consistency of G ^ n \widehat{G}_n G n to the limit ρ ( G ^ n ) → 0 \rho(\widehat{G}_n) \to0 ρ ( G n ) → 0 . Suppose that ρ ( G ^ n ) \rho(\widehat{G}_n) ρ ( G n ) does not converge in probability to zero, then there would exist a subsequence and ε > 0 \varepsilon> 0 ε > 0 on which P { ρ ( G ^ n ) > ε } \mathbb{P}\{\rho(\widehat{G}_n)>\varepsilon\} P { ρ ( G n ) > ε } stays bounded away from zero. A further subsequence, denote as { G ^ n k } k ≥ 1 \{\widehat{G}_{n_k}\}_{k\geq1} { G n k } k ≥ 1 , can be chosen along which G ^ n k → G ∗ \widehat{G}_{n_k} \to G_* G n k → G ∗ almost surely; then the result in equation (40) above would then imply ρ ( G ^ n k ) → 0 \rho(\widehat{G}_{n_k}) \to0 ρ ( G n k ) → 0 almost surely, a contradiction. Therefore
ρ ( G ^ n ) = ∥ p G ^ n − p ∗ p ∗ ∥ ∗ → P 0. \rho(\widehat{G}_n)=\left\|\frac{p_{\widehat{G}_n}-p_*}{p_*}\right\|_* \xrightarrow{P} 0. ρ ( G n ) = p ∗ p G n − p ∗ ∗ P 0.
The following lemma expresses the density discrepancy p G − p ∗ p_G-p_* p G − p ∗ through second-order Taylor expansions around the true atom nearest to each fitted atom, called exact second-order secant decomposition . Such decompositions are standard in the analysis of overfitted mixtures [2 , 13 , 12 ] . We state this representation as an intermediate step toward establishing the convergence rate of W 2 2 ( G ^ n , G ∗ ) W_2^2(\widehat{G}_n,G_*) W 2 2 ( G n , G ∗ ) , and, more importantly, to explain the local moment structure underlying the sequential first-order transportation expansion in § 5.1 .
Lemma A.2 (Exact second-order secant decomposition). Fix a measurable nearest-true-atom map ι ( θ ) ∈ arg min 1 ≤ i ≤ p ∥ θ − θ i ∗ ∥ \iota(\theta) \in\arg\min_{1\leq i\leq p}\|\theta-\theta_i^*\| ι ( θ ) ∈ arg min 1 ≤ i ≤ p ∥ θ − θ i ∗ ∥ , with deterministic tie breaking. For G = ∑ j = 1 m π j δ θ j ∈ O k ( Θ ) G=\sum_{j=1}^{m}\pi_j\delta_{\theta_j}\in\mathcal{O}_k(\Theta) G = ∑ j = 1 m π j δ θ j ∈ O k ( Θ ) , define i j = ι ( θ j ) i_j=\iota(\theta_j) i j = ι ( θ j ) , Δ j = θ j − θ i j ∗ \Delta_j=\theta_j-\theta_{i_j}^* Δ j = θ j − θ i j ∗ , and r j = ∥ Δ j ∥ r_j=\|\Delta_j\| r j = ∥ Δ j ∥ be the distance from θ j \theta_j θ j to its nearest true center. For r > 0 r>0 r > 0 , and unit vector u ∈ S d − 1 u\in S^{d-1} u ∈ S d − 1 , set
Q i , r , u = f θ i ∗ + r u − f θ i ∗ − r u T ∇ f θ i ∗ r 2 , Q_{i,r,u}=\frac{f_{\theta_i^*+ru}-f_{\theta_i^*}-ru^{\mathsf T}\nabla f_{\theta_i^*}}{r^2}, Q i , r , u = r 2 f θ i ∗ + r u − f θ i ∗ − r u T ∇ f θ i ∗ , with continuous extension in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) when r = 0 r=0 r = 0 as Q i , 0 , u = 1 2 u T ∇ 2 f θ i ∗ u Q_{i,0,u}=\frac{1}{2}u^{\mathsf T}\nabla^2f_{\theta_i^*}u Q i , 0 , u = 2 1 u T ∇ 2 f θ i ∗ u . Let
a i ( G ) = ( ∑ j : i j = i π j ) − π i ∗ , b i ( G ) = ∑ j : i j = i π j Δ j , λ j ( G ) = π j r j 2 . a_i(G)=\left(\sum_{j:i_j=i}\pi_j\right)-\pi_i^*,\qquad b_i(G)=\sum_{j:i_j=i}\pi_j\Delta_j,\qquad\lambda_j(G)=\pi_jr_j^2. a i ( G ) = j : i j = i ∑ π j − π i ∗ , b i ( G ) = j : i j = i ∑ π j Δ j , λ j ( G ) = π j r j 2 . The following representation is exact:
p G − p ∗ = ∑ i = 1 p a i ( G ) f θ i ∗ + ∑ i = 1 p b i ( G ) T ∇ f θ i ∗ + ∑ j : r j > 0 λ j ( G ) Q i j , r j , u j , (41) p_G-p_*=\sum_{i=1}^{p}a_i(G)f_{\theta_i^*}+\sum_{i=1}^{p}b_i(G)^{\mathsf T}\nabla f_{\theta_i^*}+\sum_{j:r_j>0}\lambda_j(G)Q_{i_j,r_j,u_j}, \tag*{(41)} p G − p ∗ = i = 1 ∑ p a i ( G ) f θ i ∗ + i = 1 ∑ p b i ( G ) T ∇ f θ i ∗ + j : r j > 0 ∑ λ j ( G ) Q i j , r j , u j , (41) where u j = Δ j / r j u_j=\Delta_j/r_j u j = Δ j / r j . Moreover, the zero-order parameters a i ( G ) a_i(G) a i ( G ) have vanishing sum ∑ i = 1 p a i ( G ) = 0 \sum_{i=1}^{p}a_i(G)=0 ∑ i = 1 p a i ( G ) = 0 .
Proof. Taylor expansion for each true atom. We develop the Taylor expansion for density function f θ f_\theta f θ at each true atom θ i ∗ \theta_i^* θ i ∗ . Fix i i i , r > 0 r>0 r > 0 , and u ∈ S d − 1 u\in S^{d-1} u ∈ S d − 1 such that θ i ∗ + r u ∈ Θ \theta_i^*+ru\in\Theta θ i ∗ + r u ∈ Θ . Define g ( s ) = f θ i ∗ + s r u g(s)=f_{\theta_i^*+sru} g ( s ) = f θ i ∗ + sr u for 0 ≤ s ≤ 1 0 \le s \le1 0 ≤ s ≤ 1 . By Assumption 2.1 , g g g is twice continuously differentiable with respect to s s s and
g ′ ( s ) = r u ⊤ ∇ f θ i ∗ + s r u , g ′ ′ ( s ) = r 2 u ⊤ ∇ 2 f θ i ∗ + s r u u . g'(s)=ru^\top\nabla f_{\theta_i^*+sru},\qquad g''(s)=r^2u^\top\nabla^2 f_{\theta_i^*+sru}u. g ′ ( s ) = r u ⊤ ∇ f θ i ∗ + sr u , g ′′ ( s ) = r 2 u ⊤ ∇ 2 f θ i ∗ + sr u u .
Taylor’s formula with integral remainder yields
g ( 1 ) = g ( 0 ) + g ′ ( 0 ) + ∫ 0 1 ( 1 − s ) g ′ ′ ( s ) d s . g(1)=g(0)+g'(0)+\int_0^1(1-s)g''(s)\,ds. g ( 1 ) = g ( 0 ) + g ′ ( 0 ) + ∫ 0 1 ( 1 − s ) g ′′ ( s ) d s .
Consequently, we have
Q i , r , u = ∫ 0 1 ( 1 − s ) u ⊤ ∇ 2 f θ i ∗ + s r u u d s . (42) Q_{i,r,u}=\int_0^1(1-s)u^\top\nabla^2f_{\theta_i^*+sru}u\,ds. \tag*{(42)} Q i , r , u = ∫ 0 1 ( 1 − s ) u ⊤ ∇ 2 f θ i ∗ + sr u u d s . (42)
By continuity of ∇ 2 f θ \nabla^2f_\theta ∇ 2 f θ in θ \theta θ , we can naturally extend Q i , r , u ( x ) Q_{i,r,u}(x) Q i , r , u ( x ) to the case when r = 0 r=0 r = 0 as
Q i , r , u ( x ) → 1 2 u ⊤ ∇ 2 f θ i ∗ ( x ) u = : Q i , 0 , u ( x ) Q_{i,r,u}(x)\to\frac{1}{2}u^\top\nabla^2f_{\theta_i^*}(x)u=:Q_{i,0,u}(x) Q i , r , u ( x ) → 2 1 u ⊤ ∇ 2 f θ i ∗ ( x ) u =: Q i , 0 , u ( x )
for P ∗ P_* P ∗ -a.e. x x x . Moreover, from (42) and Assumption 2.1 , ∣ Q i , r , u ( x ) / p ∗ ( x ) ∣ ≤ 1 2 F ( x ) |Q_{i,r,u}(x)/p_*(x)|\le\frac{1}{2}F(x) ∣ Q i , r , u ( x ) / p ∗ ( x ) ∣ ≤ 2 1 F ( x ) . Hence the extension is also continuous in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) by dominated convergence.
Aggregation over fitted atoms. Now we apply this expansion to the fitted atom f θ j f_{\theta_j} f θ j at the true atom θ i j ∗ \theta_{i_j}^* θ i j ∗ before summing up. For each fitted atom θ j \theta_j θ j with r j > 0 r_j>0 r j > 0 , the above expansion gives
f θ j = f θ i j ∗ + Δ j ⊤ ∇ f θ i j ∗ + r j 2 Q i j , r j , u j . f_{\theta_j}=f_{\theta_{i_j}^*}+\Delta_j^\top\nabla f_{\theta_{i_j}^*}+r_j^2Q_{i_j,r_j,u_j}. f θ j = f θ i j ∗ + Δ j ⊤ ∇ f θ i j ∗ + r j 2 Q i j , r j , u j .
If r j = 0 r_j=0 r j = 0 , the last two terms are zero and the same formula is valid after interpreting u j u_j u j arbitrarily. Multiplying this identity by π j \pi_j π j and summing over j j j , we have
p G = ∑ i = 1 p ( ∑ j : i j = i π j ) f θ i ∗ + ∑ i = 1 p ( ∑ j : ι ( j ) = i π j Δ j ) ⊤ ∇ f θ i ∗ + ∑ j : r j > 0 π j r j 2 Q i j , r j , u j . p_G=\sum_{i=1}^{p}\left(\sum_{j:\,i_j=i}\pi_j\right)f_{\theta_i^*}+\sum_{i=1}^{p}\left(\sum_{j:\,\iota(j)=i}\pi_j\Delta_j\right)^\top\nabla f_{\theta_i^*}+\sum_{j:\,r_j>0}\pi_jr_j^2Q_{i_j,r_j,u_j}. p G = i = 1 ∑ p j : i j = i ∑ π j f θ i ∗ + i = 1 ∑ p j : ι ( j ) = i ∑ π j Δ j ⊤ ∇ f θ i ∗ + j : r j > 0 ∑ π j r j 2 Q i j , r j , u j .
Subtracting the equation above to p ∗ = ∑ i = 1 p π i ∗ f θ i ∗ p_*=\sum_{i=1}^{p}\pi_i^*f_{\theta_i^*} p ∗ = ∑ i = 1 p π i ∗ f θ i ∗ gives (41) . This completes our proof.
Next, we define the following geometric local size, which incorporates the coefficients of secant decomposition in Lemma A.2 .
D ( G ) = ∑ i = 1 p ∣ a i ( G ) ∣ + ∑ i = 1 p ∥ b i ( G ) ∥ + ∑ j = 1 m π j r j 2 . (43) D(G)=\sum_{i=1}^{p}|a_i(G)|+\sum_{i=1}^{p}\lVert b_i(G)\rVert+\sum_{j=1}^{m}\pi_jr_j^2. \tag*{(43)} D ( G ) = i = 1 ∑ p ∣ a i ( G ) ∣ + i = 1 ∑ p ∥ b i ( G )∥ + j = 1 ∑ m π j r j 2 . (43)
The next result establishes local equivalence between the relative-density distance ρ ( G ) \rho(G) ρ ( G ) and D ( G ) D(G) D ( G ) under second-order strong identifiability. The lower bound is a local inverse inequality, while the upper bound follows from smoothness of the kernel. Related inverse inequalities use Hellinger or L 1 L^1 L 1 discrepancies between mixture densities to control mixing-measure errors [27 , 13 , 12 , 21 ] .
Lemma A.3 (Local equivalence of relative-density and mixing-measure discrepancies). There exist
a W 2 W_2 W 2 neighborhood U \mathcal{U} U of G ∗ G_\ast G ∗ in O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) and constants 0 < c < C < ∞ 0<c<C<\infty 0 < c < C < ∞ such that
c D ( G ) ≤ ρ ( G ) ≤ C D ( G ) , G ∈ U . (44) cD(G)\leq\rho(G)\leq CD(G), \qquad G\in\mathcal{U}. \tag*{(44)} cD ( G ) ≤ ρ ( G ) ≤ C D ( G ) , G ∈ U . (44)
Proof. The upper bound is immediate from (41) and the relative derivative envelope. Indeed, with Q i , r , u Q_{i,r,u} Q i , r , u as in Lemma A.2 , the integral Taylor formula gives ∥ Q i , r , u / p ∗ ∥ ∗ ≤ ∥ F ∥ 2 / 2 \lVert Q_{i,r,u}/p_\ast\rVert_\ast\leq\lVert F\rVert_2/2 ∥ Q i , r , u / p ∗ ∥ ∗ ≤ ∥ F ∥ 2 /2 , uniformly in its feasible arguments. Thus ρ ( G ) ≤ C D ( G ) \rho(G)\leq CD(G) ρ ( G ) ≤ C D ( G ) .
Suppose the lower bound does not hold locally. Then for each m m m there exists G m ≠ G ∗ G_m\ne G_\ast G m = G ∗ with W 2 ( G m , G ∗ ) ≤ m − 1 W_2(G_m,G_\ast)\leq m^{-1} W 2 ( G m , G ∗ ) ≤ m − 1 , D m : = D ( G m ) > 0 D_m:=D(G_m)>0 D m := D ( G m ) > 0 , and ρ ( G m ) / D m → 0 \rho(G_m)/D_m\to0 ρ ( G m ) / D m → 0 . Pad every measure to k k k labeled atoms (allowing zero weights) and use the exact secant decomposition. Set
A i , m = a i ( G m ) / D m , B i , m = b i ( G m ) / D m , L j , m = π j , m r j , m 2 / D m . A_{i,m}=a_i(G_m)/D_m,\qquad B_{i,m}=b_i(G_m)/D_m,\qquad L_{j,m}=\pi_{j,m}r_{j,m}^{2}/D_m. A i , m = a i ( G m ) / D m , B i , m = b i ( G m ) / D m , L j , m = π j , m r j , m 2 / D m .
By definition,
∑ i ∣ A i , m ∣ + ∑ i ∥ B i , m ∥ + ∑ j L j , m = 1 , L j , m ≥ 0. (45) \sum_i |A_{i,m}|+\sum_i\lVert B_{i,m}\rVert+\sum_j L_{j,m}=1,\qquad L_{j,m}\geq0. \tag*{(45)} i ∑ ∣ A i , m ∣ + i ∑ ∥ B i , m ∥ + j ∑ L j , m = 1 , L j , m ≥ 0. (45)
After passing to a subsequence, we may assume that the nearest-center indices i j i_j i j are fixed and that all bounded coefficients converge to limits denoted by A ‾ i \overline{A}_i A i , B ‾ i \overline{B}_i B i , and L ‾ j \overline{L}_j L j . Write r j , m = ∥ θ j , m − θ i j ∗ ∥ r_{j,m}=\lVert\theta_{j,m}-\theta_{i_j}^\ast\rVert r j , m = ∥ θ j , m − θ i j ∗ ∥ and choose unit directions u j , m u_{j,m} u j , m (arbitrarily when r j , m = 0 r_{j,m}=0 r j , m = 0 ) such that θ j , m − θ i j ∗ = r j , m u j , m \theta_{j,m}-\theta_{i_j}^\ast=r_{j,m}u_{j,m} θ j , m − θ i j ∗ = r j , m u j , m . Using the compactness of [ 0 , diam ( Θ ) ] × S d − 1 [0,\operatorname{diam}(\Theta)]\times S^{d-1} [ 0 , diam ( Θ )] × S d − 1 , we can suppose that r j , m → r ‾ j r_{j,m}\to\overline{r}_j r j , m → r j and u j , m → u ‾ j u_{j,m}\to\overline{u}_j u j , m → u j . The L 2 ( P ∗ ) L^2(P_\ast) L 2 ( P ∗ ) continuity of Q i , r , u / p ∗ Q_{i,r,u}/p_\ast Q i , r , u / p ∗ follows from its integral Hessian representation and the common L 2 ( P ∗ ) L^2(P_\ast) L 2 ( P ∗ ) envelope. Therefore ρ ( G m ) / D m → 0 \rho(G_m)/D_m\to0 ρ ( G m ) / D m → 0 yields, in L 2 ( P ∗ ) L^2(P_\ast) L 2 ( P ∗ ) and hence μ \mu μ -almost everywhere,
∑ i A ‾ i f θ i ∗ + ∑ i B ‾ i T ∇ f θ i ∗ + ∑ j L ‾ j Q i j , r ‾ j , u ‾ j = 0. (46) \sum_i \overline{A}_i f_{\theta_i^\ast}+\sum_i \overline{B}_i^\mathsf{T}\nabla f_{\theta_i^\ast}+\sum_j \overline{L}_jQ_{i_j,\overline{r}_j,\overline{u}_j}=0. \tag*{(46)} i ∑ A i f θ i ∗ + i ∑ B i T ∇ f θ i ∗ + j ∑ L j Q i j , r j , u j = 0. (46)
If r ‾ j > 0 \overline{r}_j>0 r j > 0 , set η j = θ i j ∗ + r ‾ j u ‾ j \eta_j=\theta_{i_j}^\ast+\overline{r}_j\overline{u}_j η j = θ i j ∗ + r j u j . Nearest-center assignment ensures that η j \eta_j η j cannot equal a different true center. Expanding Q i j , r ‾ j , u ‾ j Q_{i_j,\overline{r}_j,\overline{u}_j} Q i j , r j , u j , the coefficient of f η f_\eta f η at each distinct nontrue location η \eta η is the nonnegative sum ∑ j : η j = η , r ‾ j > 0 L ‾ j / r ‾ j 2 \sum_{j:\eta_j=\eta,\ \overline{r}_j>0}\overline{L}_j/\overline{r}_j^2 ∑ j : η j = η , r j > 0 L j / r j 2 . At r ‾ j = 0 \overline{r}_j=0 r j = 0 , we have the contribution of the Hessian can be expressed as
∑ j : i j = i , r ‾ j = 0 L ‾ j u ‾ j T ∇ 2 f θ i ∗ u ‾ j = ∑ j : i j = i , r ‾ j = 0 L ‾ j tr ( u ‾ j u ‾ j T ∇ 2 f θ i ∗ ) = tr ( C ‾ i ∇ 2 f θ i ∗ ) , \sum_{j:i_j=i,\ \overline{r}_j=0}\overline{L}_j\overline{u}_j^\mathsf{T}\nabla^2 f_{\theta_i^\ast}\overline{u}_j
=
\sum_{j:i_j=i,\ \overline{r}_j=0}\overline{L}_j\operatorname{tr}\!\left(\overline{u}_j\overline{u}_j^\mathsf{T}\nabla^2 f_{\theta_i^\ast}\right)
=
\operatorname{tr}\!\left(\overline{C}_i\nabla^2 f_{\theta_i^\ast}\right), j : i j = i , r j = 0 ∑ L j u j T ∇ 2 f θ i ∗ u j = j : i j = i , r j = 0 ∑ L j tr ( u j u j T ∇ 2 f θ i ∗ ) = tr ( C i ∇ 2 f θ i ∗ ) ,
where C i = ∑ j : i j = i , r ‾ j = 0 L j u j u j T C_i=\sum_{j:i_j=i,\ \overline{r}_j=0}L_j u_j u_j^\mathsf{T} C i = ∑ j : i j = i , r j = 0 L j u j u j T . Therefore, by rewriting the identity (46) based on above observation, we receive a linear combination of f θ i ∗ f_{\theta_i^\ast} f θ i ∗ , ∇ f θ i ∗ \nabla f_{\theta_i^\ast} ∇ f θ i ∗ , ∇ 2 f θ i ∗ \nabla^2 f_{\theta_i^\ast} ∇ 2 f θ i ∗ , and the f η j f_{\eta_j} f η j for each r ‾ j > 0 \overline{r}_j>0 r j > 0 such that this combination contains at most p + k ≤ 2 k p+k\leq2k p + k ≤ 2 k distinct locations of θ i ∗ \theta_i^\ast θ i ∗ and η j \eta_j η j . Assumption 2.3 implies that every kernel, gradient and Hessian coefficient vanishes. By considering the coefficient of f η j f_{\eta_j} f η j , as L ‾ j ≥ 0 \overline{L}_j\geq0 L j ≥ 0 , we have L ‾ j = 0 \overline{L}_j=0 L j = 0 for every r ‾ j > 0 \overline{r}_j>0 r j > 0 . By plugging in back to (46) and considering the contribution of Hessian matrix ∇ 2 f θ i ∗ \nabla^2 f_{\theta_i^\ast} ∇ 2 f θ i ∗ , we have tr ( C ‾ i ) = ∑ j : i j = i , r ‾ j = 0 L ‾ j = 0 \operatorname{tr}(\overline{C}_i)=\sum_{j:i_j=i,\overline{r}_j=0}\overline{L}_j=0 tr ( C i ) = ∑ j : i j = i , r j = 0 L j = 0 forces L ‾ j = 0 \overline{L}_j=0 L j = 0 . The true-location coefficients then imply A ‾ i = 0 \overline{A}_i=0 A i = 0 and B ‾ i = 0 \overline{B}_i=0 B i = 0 . This contradicts the limit of (45) , whose left-hand side equals one. The local lower bound follows. □
The preceding lemma compares ρ ( G ) \rho(G) ρ ( G ) with the coefficient discrepancy D ( G ) D(G) D ( G ) . We next relate these quantities to the squared Wasserstein distance. The first bound controls ρ ( G ) 2 \rho(G)^2 ρ ( G ) 2 by W 2 2 ( G , G ∗ ) W_2^2(G,G_\ast) W 2 2 ( G , G ∗ ) , while the second bounds the transportation cost through cluster mass discrepancies and local second moments. Combined with the local equivalence in Lemma A.3 , the latter yields W 2 2 ( G , G ∗ ) ≲ ρ ( G ) W_2^2(G,G_\ast)\lesssim\rho(G) W 2 2 ( G , G ∗ ) ≲ ρ ( G ) .
Lemma A.4. (i) There exists c < ∞ c < \infty c < ∞ such that c ρ 2 ( G ) ≤ W 2 2 ( G , G ∗ ) c\rho^2(G) \le W_2^2(G,G_\ast) c ρ 2 ( G ) ≤ W 2 2 ( G , G ∗ ) .
(ii) There exists c , C < ∞ c,C < \infty c , C < ∞ such that for all G G G sufficiently close to G ∗ G_\ast G ∗ in Wasserstein metric,
W 2 2 ( G , G ∗ ) ≤ C { ∑ i ∣ a i ( G ) ∣ + ∑ j π j r j 2 } ≤ C D ( G ) ≤ C ρ ( G ) . (47) W_2^2(G,G_\ast) \le C\left\{\sum_i |a_i(G)|+\sum_j \pi_j r_j^2\right\} \le CD(G) \le C\rho(G). \tag*{(47)} W 2 2 ( G , G ∗ ) ≤ C { i ∑ ∣ a i ( G ) ∣ + j ∑ π j r j 2 } ≤ C D ( G ) ≤ C ρ ( G ) . (47)
Proof. For the part (i), let π \pi π be a coupling between G G G and G ∗ G_\ast G ∗ . We observe that for any couple of parameter θ \theta θ and θ ′ \theta' θ ′ , from Assumption 2.1 ,
∣ f θ ( x ) − f θ ′ ( x ) p ∗ ( x ) ∣ ≤ F ( x ) ∥ θ − θ ′ ∥ . \left|\frac{f_\theta(x)-f_{\theta'}(x)}{p_\ast(x)}\right| \le F(x)\|\theta-\theta'\|. p ∗ ( x ) f θ ( x ) − f θ ′ ( x ) ≤ F ( x ) ∥ θ − θ ′ ∥.
Therefore, we have
∣ p G ( x ) − p G ′ ( x ) p ∗ ( x ) ∣ = ∣ ∫ f θ ( x ) − f θ ′ ( x ) p ∗ ( x ) d π ( θ , θ ′ ) ∣ ≤ F ( x ) ∫ ∥ θ − θ ′ ∥ d π ( θ , θ ′ ) . \left|\frac{p_G(x)-p_{G'}(x)}{p_\ast(x)}\right| = \left|\int\frac{f_\theta(x)-f_{\theta'}(x)}{p_\ast(x)}\,d\pi(\theta,\theta')\right| \le F(x)\int\|\theta-\theta'\|\,d\pi(\theta,\theta'). p ∗ ( x ) p G ( x ) − p G ′ ( x ) = ∫ p ∗ ( x ) f θ ( x ) − f θ ′ ( x ) d π ( θ , θ ′ ) ≤ F ( x ) ∫ ∥ θ − θ ′ ∥ d π ( θ , θ ′ ) .
Thus, by plugging in the optimal coupling with respect to 1-Wasserstein distance and using the integrability of F F F , we have
ρ ( G ) ≲ W 1 ( G , G ∗ ) ≤ W 2 ( G , G ∗ ) . \rho(G) \lesssim W_1(G,G_\ast) \le W_2(G,G_\ast). ρ ( G ) ≲ W 1 ( G , G ∗ ) ≤ W 2 ( G , G ∗ ) .
For part (ii), from Lemma A.3 , it is sufficient to prove that there exists a constant C C C satisfying
W 2 2 ( G , G ∗ ) ≤ C { ∑ i ∣ a i ( G ) ∣ + ∑ j π j r j 2 } . (48) W_2^2(G,G_\ast) \le C\left\{\sum_i |a_i(G)|+\sum_j \pi_j r_j^2\right\}. \tag*{(48)} W 2 2 ( G , G ∗ ) ≤ C { i ∑ ∣ a i ( G ) ∣ + j ∑ π j r j 2 } . (48)
To prove this, we consider the following intermediate measure G c = ∑ i = 1 p ( π i ∗ + a i ) δ θ i ∗ G^c=\sum_{i=1}^p(\pi_i^\ast+a_i)\delta_{\theta_i^\ast} G c = ∑ i = 1 p ( π i ∗ + a i ) δ θ i ∗ . We can interprete the mass of each center θ i ∗ \theta_i^\ast θ i ∗ to be ∑ j : j j = i π j \sum_{j:j_j=i}\pi_j ∑ j : j j = i π j . Using triangle inequality and Cauchy-Schwartz inequality, we have
W 2 2 ( G , G ∗ ) ≤ 2 ( W 2 2 ( G , G c ) + W 2 2 ( G c , G ∗ ) ) . W_2^2(G,G_\ast) \le2\left(W_2^2(G,G^c)+W_2^2(G^c,G_\ast)\right). W 2 2 ( G , G ∗ ) ≤ 2 ( W 2 2 ( G , G c ) + W 2 2 ( G c , G ∗ ) ) .
Estimate W 2 2 ( G , G c ) W_2^2(G,G^c) W 2 2 ( G , G c ) . Consider the transportation plan in which each fitted centers θ j \theta_j θ j are transferred to their closet true center θ i j ∗ \theta_{i_j}^\ast θ i j ∗ . Then, we can bound W 2 2 ( G , G c ) W_2^2(G,G^c) W 2 2 ( G , G c ) by the squared cost of this transportation
W 2 2 ( G , G c ) ≤ ∑ j = 1 m π j ∥ θ j − θ i j ∗ ∥ 2 . (49) W_2^2(G,G^c) \le\sum_{j=1}^m \pi_j\|\theta_j-\theta_{i_j}^\ast\|^2. \tag*{(49)} W 2 2 ( G , G c ) ≤ j = 1 ∑ m π j ∥ θ j − θ i j ∗ ∥ 2 . (49)
Estimate W 2 2 ( G c , G ∗ ) W_2^2(G^c,G_\ast) W 2 2 ( G c , G ∗ ) . We take into account the transportation plan from G c G^c G c to G ∗ G_\ast G ∗ which keep a mass of min { π i ∗ + a i , π i ∗ } \min\{\pi_i^\ast+a_i,\pi_i^\ast\} min { π i ∗ + a i , π i ∗ } for the center θ i \theta_i θ i , while the remaining mass can be transferred to any other centers. Then, we can show that the total remaining mass is equal to 1 2 ∑ i ∣ a i ∣ \frac{1}{2}\sum_i|a_i| 2 1 ∑ i ∣ a i ∣ , and as the cost to move the mass from one center to another one is no more than diam ( Θ ) \operatorname{diam}(\Theta) diam ( Θ ) , we have the following estimation
W 2 2 ( G c , G ∗ ) ≤ 1 2 diam ( Θ ) 2 ∑ i ∣ a i ∣ . (50) W_2^2(G^c,G_\ast) \le\frac{1}{2}\operatorname{diam}(\Theta)^2\sum_i|a_i|. \tag*{(50)} W 2 2 ( G c , G ∗ ) ≤ 2 1 diam ( Θ ) 2 i ∑ ∣ a i ∣. (50)
Putting everything together. Combining the estimations in (48) , (49) , and (50) , we achieve the estimation (47) , completing the proof.
Once the inverse inequality in Lemma A.4 (ii) is established, obtaining a convergence rate for W 2 2 ( G ^ n , G ∗ ) W_2^2(\widehat{G}_n,G_*) W 2 2 ( G n , G ∗ ) reduces to controlling the relative-density discrepancy ρ ( G ^ n ) \rho(\widehat{G}_n) ρ ( G n ) . The following lemma establishes its root-n n n rate, which immediately yields the same rate for the squared Wasserstein error.
Empirical-process properties and convergence rates
The local comparisons established above reduce control of the Wasserstein error to that of the relative-density discrepancy ρ ( G ) \rho(G) ρ ( G ) . We now establish envelope and bracketing bounds for the normalized relative-density class, yielding the Donsker and Glivenko–Cantelli properties needed for likelihood localization [32 , 30 ] . These properties give ρ ( G ^ n ) = O P ( n − 1 / 2 ) \rho(\widehat{G}_n)=O_{P}(n^{-1/2}) ρ ( G n ) = O P ( n − 1/2 ) , and Lemma A.4 (ii) then yields W 2 ( G ^ n , G ∗ ) = O P ( n − 1 / 4 ) W_2(\widehat{G}_n,G_*)=O_{P}(n^{-1/4}) W 2 ( G n , G ∗ ) = O P ( n − 1/4 ) . Although this Wasserstein rate is known under suitable conditions [27 , 13 ] , we derive it within our framework to support the subsequent likelihood expansion and weak-limit analysis.
Lemma A.5 (Compact normalized secants and Donsker property). Let
S = { s G = ( p G − p ∗ ) / p ∗ ρ ( G ) : G ∈ U , G ≠ G ∗ } . \mathcal{S}=\left\{s_G=\frac{(p_G-p_*)/p_*}{\rho(G)}:G\in\mathcal{U},\ G\ne G_*\right\}. S = { s G = ρ ( G ) ( p G − p ∗ ) / p ∗ : G ∈ U , G = G ∗ } . where U \mathcal{U} U is the neighborhood in Lemma A.3 . Then:
(i) S \mathcal{S} S admits an envelope F 0 ∈ L 2 + η ( P ∗ ) F_0\in L^{2+\eta}(P_*) F 0 ∈ L 2 + η ( P ∗ ) ;
(ii) S \mathcal{S} S is totally bounded in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) and for sufficiently small ε \varepsilon ε ,
log N [ ] ( ε , S , L 2 ( P ∗ ) ) ≤ C log ( 1 / ε ) , \log N_{[]}(\varepsilon,\mathcal{S},L^2(P_*))\le C\log(1/\varepsilon), log N [ ] ( ε , S , L 2 ( P ∗ )) ≤ C log ( 1/ ε ) , where N [ ] N_{[]} N [ ] denotes the bracketing number. Consequently, S \mathcal{S} S is a relatively compact subset in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) ;
(iii) S \mathcal{S} S is P ∗ P_* P ∗ -Donsker and S 2 = { s 2 : s ∈ S } \mathcal{S}^2=\{s^2:s\in\mathcal{S}\} S 2 = { s 2 : s ∈ S } is P ∗ P_* P ∗ -Glivenko–Cantelli. In addition, the truncated class of S 2 \mathcal{S}^2 S 2 , defined as { s 2 1 ( F 0 ≤ K ) : s ∈ S } \{s^2\mathbf{1}(F_0\le K):s\in\mathcal{S}\} { s 2 1 ( F 0 ≤ K ) : s ∈ S } , is also P ∗ P_* P ∗ -Glivenko–Cantelli for any K ∈ R K\in\mathbb{R} K ∈ R
Proof. Firstly, we prove part (i). For each G ∈ U G\in\mathcal{U} G ∈ U , by dividing secant identity (41) by ρ ( G ) p ∗ \rho(G)p_* ρ ( G ) p ∗ , we have
s G = p G − p ∗ ρ ( G ) p ∗ = ∑ i = 1 p a i ( G ) ρ ( G ) f θ i ∗ p ∗ + ∑ i = 1 p b i ( G ) ρ ( G ) ∇ f θ i ∗ T p ∗ + ∑ j : r j > 0 λ j ( G ) ρ ( G ) Q i j , r j , u j p ∗ . s_G=\frac{p_G-p_*}{\rho(G)p_*}=\sum_{i=1}^{p}\frac{a_i(G)}{\rho(G)}\frac{f_{\theta_i^*}}{p_*}+\sum_{i=1}^{p}\frac{b_i(G)}{\rho(G)}\frac{\nabla f_{\theta_i^*}^{\mathsf{T}}}{p_*}+\sum_{j:r_j>0}\frac{\lambda_j(G)}{\rho(G)}\frac{Q_{i_j,r_j,u_j}}{p_*}. s G = ρ ( G ) p ∗ p G − p ∗ = i = 1 ∑ p ρ ( G ) a i ( G ) p ∗ f θ i ∗ + i = 1 ∑ p ρ ( G ) b i ( G ) p ∗ ∇ f θ i ∗ T + j : r j > 0 ∑ ρ ( G ) λ j ( G ) p ∗ Q i j , r j , u j .
Noting that Lemma A.3 gives D ( G ) / ρ ( G ) ≤ c − 1 D(G)/\rho(G)\le c^{-1} D ( G ) / ρ ( G ) ≤ c − 1 , we have each element s ( G ) s(G) s ( G ) can be expressed as a linear combination of f θ i ∗ / p ∗ f_{\theta_i^*}/p_* f θ i ∗ / p ∗ , ∇ f θ i ∗ / p ∗ \nabla f_{\theta_i^*}/p_* ∇ f θ i ∗ / p ∗ , and Q i j , r j , u j / p ∗ Q_{i_j,r_j,u_j}/p_* Q i j , r j , u j / p ∗ with coefficients bounded by c − 1 c^{-1} c − 1 . The fixed kernel and gradient terms satisfy ∣ f θ i ∗ / p ∗ ∣ ≤ F \lvert f_{\theta_i^*}/p_*\rvert\le F ∣ f θ i ∗ / p ∗ ∣ ≤ F and ∥ ∇ f θ i ∗ / p ∗ ∥ ≤ F \lVert\nabla f_{\theta_i^*}/p_*\rVert\le F ∥ ∇ f θ i ∗ / p ∗ ∥ ≤ F thanks to Assumption 2.1 . To study the boundedness of quadratic remainder Q i , r , u Q_{i,r,u} Q i , r , u , we employ the integral representation
Q i , r , u = ∫ 0 1 ( 1 − s ) u T ∇ 2 f θ i ∗ + s r u u d s . (51) Q_{i,r,u}=\int_0^1(1-s)u^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sru}\,u\,\mathrm{d}s. \tag*{(51)} Q i , r , u = ∫ 0 1 ( 1 − s ) u T ∇ 2 f θ i ∗ + sr u u d s . (51)
Assumption 2.1 similarly gives us ∣ ∇ 2 f θ / p ∗ ∣ ≤ F \lvert\nabla^2 f_{\theta}/p_*\rvert\le F ∣ ∇ 2 f θ / p ∗ ∣ ≤ F , hence the integral representation implies ∣ Q i , r , u / p ∗ ∣ ≤ F / 2 \lvert Q_{i,r,u}/p_*\rvert\le F/2 ∣ Q i , r , u / p ∗ ∣ ≤ F /2 . Together with the fact that all coefficients in s G s_G s G are bounded, we have
∣ s G ( x ) ∣ ≤ F 0 ( x ) : = C 0 F ( x ) , s G ∈ S . (52) \lvert s_G(x)\rvert\le F_0(x):=C_0F(x),\qquad s_G\in\mathcal{S}. \tag*{(52)} ∣ s G ( x )∣ ≤ F 0 ( x ) := C 0 F ( x ) , s G ∈ S . (52)
for a fixed C 0 < ∞ C_0 < \infty C 0 < ∞ , hence F 0 ∈ L 2 + η ( P ∗ ) F_0 \in L^{2+\eta}(P_*) F 0 ∈ L 2 + η ( P ∗ ) . This means that S \mathcal{S} S admits an envelope F 0 ∈ L 2 + η ( P ∗ ) F_0 \in L^{2+\eta}(P_*) F 0 ∈ L 2 + η ( P ∗ ) .
Next, we prove part (ii). To establish the bound for the bracketing number, we first prove the Lipschitz property of quadratic remainder Q i , r , u Q_{i,r,u} Q i , r , u with respect to r r r and u u u . We observe that for any r r r , u u u , r ′ r' r ′ , u ′ u' u ′ and s ∈ ( 0 , 1 ) s \in(0,1) s ∈ ( 0 , 1 ) , we have
∣ u T ∇ 2 f θ i ∗ + s r u u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ ∣ ≤ ∣ u T ( ∇ 2 f θ i ∗ + s r u − ∇ 2 f θ i ∗ + s r ′ u ′ ) u ∣ + ∣ u T ∇ 2 f θ i ∗ + s r ′ u ′ u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ ∣ . \left|u^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sru}u-(u')^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sr'u'}u'\right|
\leq\left|u^{\mathsf{T}}\left(\nabla^2 f_{\theta_i^*+sru}-\nabla^2 f_{\theta_i^*+sr'u'}\right)u\right|
+\left|u^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sr'u'}u-(u')^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sr'u'}u'\right|. u T ∇ 2 f θ i ∗ + sr u u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ ≤ u T ( ∇ 2 f θ i ∗ + sr u − ∇ 2 f θ i ∗ + s r ′ u ′ ) u + u T ∇ 2 f θ i ∗ + s r ′ u ′ u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ .
For the first term, the mean-value theorem and the third-derivative envelope give
∣ u T ( ∇ 2 f θ i ∗ + s r u − ∇ 2 f θ i ∗ + s r ′ u ′ ) u ∣ ≤ p ∗ F s ( ∣ r − r ′ ∣ + R ∥ u − u ′ ∥ ) . \left|u^{\mathsf{T}}\left(\nabla^2 f_{\theta_i^*+sru}-\nabla^2 f_{\theta_i^*+sr'u'}\right)u\right|
\leq p_*F s\left(|r-r'|+R\lVert u-u'\rVert\right). u T ( ∇ 2 f θ i ∗ + sr u − ∇ 2 f θ i ∗ + s r ′ u ′ ) u ≤ p ∗ F s ( ∣ r − r ′ ∣ + R ∥ u − u ′ ∥ ) .
For the second term, using the fact that ∥ ∇ 2 f θ / p ∗ ∥ ≤ F \lVert\nabla^2 f_{\theta}/p_*\rVert\leq F ∥ ∇ 2 f θ / p ∗ ∥ ≤ F and ∥ u ∥ = ∥ u ′ ∥ = 1 \lVert u\rVert=\lVert u'\rVert=1 ∥ u ∥ = ∥ u ′ ∥ = 1 give
∣ u T ∇ 2 f θ i ∗ + s r ′ u ′ u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ ∣ ≤ 2 p ∗ F ∥ u − u ′ ∥ . \left|u^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sr'u'}u-(u')^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sr'u'}u'\right|
\leq2p_*F\lVert u-u'\rVert. u T ∇ 2 f θ i ∗ + s r ′ u ′ u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ ≤ 2 p ∗ F ∥ u − u ′ ∥ .
Consequently, using the integral representation in equation (51) and Assumption 2.1, we have
∣ Q i , r , u − Q i , r ′ , u ′ p ∗ ∣ ≤ 1 p ∗ ∫ 0 1 ( 1 − s ) ∣ u T ∇ 2 f θ i ∗ + s r u u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ ∣ d s ≤ F [ ∥ u − u ′ ∥ ∫ 0 1 2 ( 1 − s ) d s + ( ∣ r − r ′ ∣ + R ∥ u − u ′ ∥ ) ∫ 0 1 s d s ] , \begin{aligned}
\left|\frac{Q_{i,r,u}-Q_{i,r',u'}}{p_*}\right|
\leq\frac{1}{p_*}\int_0^1(1-s)\left|u^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sru}u-(u')^{\mathsf{T}}\nabla^2 f_{\theta_i^*+sr'u'}u'\right|\,ds \\
&\leq F\left[\lVert u-u'\rVert\int_0^1 2(1-s)\,ds+\left(|r-r'|+R\lVert u-u'\rVert\right)\int_0^1s\,ds\right],
\end{aligned} p ∗ Q i , r , u − Q i , r ′ , u ′ ≤ p ∗ 1 ∫ 0 1 ( 1 − s ) u T ∇ 2 f θ i ∗ + sr u u − ( u ′ ) T ∇ 2 f θ i ∗ + s r ′ u ′ u ′ d s ≤ F [ ∥ u − u ′ ∥ ∫ 0 1 2 ( 1 − s ) d s + ( ∣ r − r ′ ∣ + R ∥ u − u ′ ∥ ) ∫ 0 1 s d s ] ,
therefore, there is a constant C C C such that the following inequality holds point-wise
∣ Q i , r , u − Q i , r ′ , u ′ p ∗ ∣ ≤ C F { ∣ r − r ′ ∣ + ∥ u − u ′ ∥ } . (53) \left|\frac{Q_{i,r,u}-Q_{i,r',u'}}{p_*}\right|\leq CF\left\{|r-r'|+\lVert u-u'\rVert\right\}. \tag*{(53)} p ∗ Q i , r , u − Q i , r ′ , u ′ ≤ C F { ∣ r − r ′ ∣ + ∥ u − u ′ ∥ } . (53)
We continue by proving the total boundedness and estimation for bracket number of S \mathcal{S} S . Consider the following compact set
R = { ( r , u ) : r > 0 , u ∈ S d − 1 , ∃ i : θ i ∗ + r u ∈ Θ } ⊂ [ 0 , R ] × S d − 1 . \mathcal{R}=\left\{(r,u):r>0,\ u\in\mathbb{S}^{d-1},\ \exists i:\theta_i^*+ru\in\Theta\right\}\subset[0,R]\times\mathbb{S}^{d-1}. R = { ( r , u ) : r > 0 , u ∈ S d − 1 , ∃ i : θ i ∗ + r u ∈ Θ } ⊂ [ 0 , R ] × S d − 1 .
After padding to k k k labels, for each second-order secant decomposition (cf. equation (41) ) of an element s G ∈ S s_G\in\mathcal{S} s G ∈ S , the scalar coefficients are in [ − c − 1 , c − 1 ] p [-c^{-1},c^{-1}]^p [ − c − 1 , c − 1 ] p , the gradient coefficients are in [ − c − 1 , c − 1 ] p d [-c^{-1},c^{-1}]^{pd} [ − c − 1 , c − 1 ] p d , and the k k k nonnegative secant coefficients are bounded by c − 1 c^{-1} c − 1 . Therefore, the resulting parameter set is contained in a bounded subset Θ ~ \widetilde{\Theta} Θ of a fixed Euclidean space of dimension D = D ( p , k , d ) D=D(p,k,d) D = D ( p , k , d ) , and the associated functions may be written s ϑ s_{\vartheta} s ϑ , for ϑ ∈ Θ ~ \vartheta\in\widetilde{\Theta} ϑ ∈ Θ . Then, using linearity in coefficient, and the pointwise Lipschitz property of f θ i ∗ / p ∗ f_{\theta_i^*}/p_* f θ i ∗ / p ∗ , ∇ f θ i ∗ / p ∗ \nabla f_{\theta_i^*}/p_* ∇ f θ i ∗ / p ∗ and quadratic remainder Q i , r , u / p ∗ Q_{i,r,u}/p_* Q i , r , u / p ∗ in (53), there is an envelope L ∈ L 2 + η ( P ∗ ) L\in L^{2+\eta}(P_*) L ∈ L 2 + η ( P ∗ ) such that
∣ s ϑ ( x ) − s ϑ ′ ( x ) ∣ ≤ L ( x ) ∥ ϑ − ϑ ′ ∥ E . (54) |s_{\vartheta}(x)-s_{\vartheta'}(x)|\leq L(x)\lVert\vartheta-\vartheta'\rVert_{\mathrm{E}}. \tag*{(54)} ∣ s ϑ ( x ) − s ϑ ′ ( x ) ∣ ≤ L ( x ) ∥ ϑ − ϑ ′ ∥ E . (54)
For each assignment pattern, take a Euclidean δ \delta δ -net C \mathcal{C} C of the compact parameter set Θ ~ \widetilde{\Theta} Θ , with at most C δ − D C\delta^{-D} C δ − D centers. Then, the union of brackets { [ s ϑ − δ L , s ϑ + δ L ] , ϑ ∈ C } \{[s_{\vartheta}-\delta L,s_{\vartheta}+\delta L],\vartheta\in\mathcal{C}\} {[ s ϑ − δ L , s ϑ + δ L ] , ϑ ∈ C } , such that the L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) sizes of each bracket is exactly 2 δ ∥ L ∥ 2 2\delta\lVert L\rVert_2 2 δ ∥ L ∥ 2 , contains all elements of S \mathcal{S} S . By choosing δ = ε / ( 2 ∥ L ∥ 2 ) \delta=\varepsilon/(2\lVert L\rVert_2) δ = ε / ( 2 ∥ L ∥ 2 ) , we achieve that
N [ ] ( ε , S , L 2 ( P ∗ ) ) ≤ C ε − D , log N [ ] ( ε , S , L 2 ( P ∗ ) ) ≤ C 1 + C 2 log ( 1 / ε ) . (55) N_{[]}(\varepsilon,\mathcal{S},L^2(P_*)) \le C\varepsilon^{-D}, \qquad\log N_{[]}(\varepsilon,\mathcal{S},L^2(P_*)) \le C_1+C_2\log(1/\varepsilon). \tag*{(55)} N [ ] ( ε , S , L 2 ( P ∗ )) ≤ C ε − D , log N [ ] ( ε , S , L 2 ( P ∗ )) ≤ C 1 + C 2 log ( 1/ ε ) . (55)
This proves total boundedness of S \mathcal{S} S in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) as well as upper bound for bracket number. As L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) is a complete space, S \mathcal{S} S is relatively compact. Using similar argument, we also have all the result above also holds for L 1 ( P ∗ ) L^1(P_*) L 1 ( P ∗ ) space with L 1 L^1 L 1 norm.
Lastly, we prove part ( i i i ) (iii) ( iii ) . To prove that S \mathcal{S} S is P ∗ P_* P ∗ -Donsker, we start with the observation that thanks to equation (55) , we have
∫ 0 1 log N [ ] ( ε ∥ F 0 ∥ 2 , S , L 2 ( P ∗ ) ) d ε ≲ ∫ 0 1 log ( 1 / ε ) d ε < ∞ . \int_0^1 \sqrt{\log N_{[]}(\varepsilon\lVert F_0\rVert_2,\mathcal{S},L^2(P_*))}\,d\varepsilon\lesssim\int_0^1 \sqrt{\log(1/\varepsilon)}\,d\varepsilon< \infty. ∫ 0 1 log N [ ] ( ε ∥ F 0 ∥ 2 , S , L 2 ( P ∗ )) d ε ≲ ∫ 0 1 log ( 1/ ε ) d ε < ∞.
Together with the square-integrable envelope F 0 F_0 F 0 , the bracketing central limit theorem yields that S \mathcal{S} S is P ∗ P_* P ∗ -Donsker, see Chapter 2.5 of van der Vaart and Wellner [32 ] . In particular,
sup s ∈ S ∣ G n s ∣ = O P ( 1 ) , \sup_{s\in\mathcal{S}}|\mathbb{G}_n s|=O_P(1), s ∈ S sup ∣ G n s ∣ = O P ( 1 ) ,
and the empirical process is asymptotically uniformly continuous in the L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) semimetric on the closure of S \mathcal{S} S .
We show that S 2 \mathcal{S}^2 S 2 and its truncated class are P ∗ P_* P ∗ -Glivenko-Cantelli. For two parameter ϑ , ϑ ′ ∈ Θ ~ \vartheta,\vartheta'\in\widetilde{\Theta} ϑ , ϑ ′ ∈ Θ , we have
∣ s ϑ 2 − s ϑ ′ 2 ∣ ≤ ( ∣ s ϑ ∣ + ∣ s ϑ ′ ∣ ) ∣ s ϑ − s ϑ ′ ∣ ≤ 2 F 0 L ∥ ϑ − ϑ ′ ∥ E . |s_\vartheta^2-s_{\vartheta'}^2|\le(|s_\vartheta|+|s_{\vartheta'}|)|s_\vartheta-s_{\vartheta'}|\le2F_0L\lVert\vartheta-\vartheta'\rVert_{\mathrm{E}}. ∣ s ϑ 2 − s ϑ ′ 2 ∣ ≤ ( ∣ s ϑ ∣ + ∣ s ϑ ′ ∣ ) ∣ s ϑ − s ϑ ′ ∣ ≤ 2 F 0 L ∥ ϑ − ϑ ′ ∥ E .
For every center s ϑ 0 s_{\vartheta_0} s ϑ 0 in the finite Euclidean δ \delta δ -net C \mathcal{C} C , the preceding inequality gives valid pointwise brackets [ s ϑ 0 2 − 2 δ F 0 L , s ϑ 0 2 + 2 δ F 0 L ] [s_{\vartheta_0}^2-2\delta F_0L,s_{\vartheta_0}^2+2\delta F_0L] [ s ϑ 0 2 − 2 δ F 0 L , s ϑ 0 2 + 2 δ F 0 L ] for the squares of functions indexed by its δ \delta δ -ball. Their L 1 ( P ∗ ) L^1(P_*) L 1 ( P ∗ ) width is at most 4 δ P ∗ ( F 0 L ) 4\delta P_*(F_0L) 4 δ P ∗ ( F 0 L ) , which is finite due to the fact that F 0 , L ∈ L 2 ( P ∗ ) F_0,L\in L^2(P_*) F 0 , L ∈ L 2 ( P ∗ ) . Hence S 2 \mathcal{S}^2 S 2 has finite L 1 ( P ∗ ) L^1(P_*) L 1 ( P ∗ ) bracketing numbers at every positive accuracy. In addition, its envelope F 0 2 F_0^2 F 0 2 is integrable, thus, using the bracketing Glivenko–Cantelli theorem, we have
sup s ∈ S ∣ ( P n − P ∗ ) s 2 ∣ → P 0. \sup_{s\in\mathcal{S}}|(P_n-P_*)s^2|\xrightarrow{P}0. s ∈ S sup ∣ ( P n − P ∗ ) s 2 ∣ P 0.
Multiplying the functions by a fixed indicator such as 1 ( F 0 ≤ K ) \mathbb{1}(F_0\le K) 1 ( F 0 ≤ K ) only decreases the bracket sizes, so the same conclusion holds for the truncated squared class used in Lemma A.6 . This establishes the stated envelope, bracketing, Donsker, and Glivenko–Cantelli properties.
Lemma A.6 (Root-n n n density localization). Under the standing assumptions, the likelihood-selected relative density satisfies ρ ( G ^ n ) = O P ( n − 1 / 2 ) \rho(\widehat{G}_n)=O_P(n^{-1/2}) ρ ( G n ) = O P ( n − 1/2 ) . Consequently, the squared Wasserstein loss satisfies
W 2 2 ( G ^ n , G ∗ ) = O P ( n − 1 / 2 ) . W_2^2(\widehat{G}_n,G_*)=O_P(n^{-1/2}). W 2 2 ( G n , G ∗ ) = O P ( n − 1/2 ) .
Proof. Step 1: A local quadratic upper bound for the log-likelihood. In this step, we establish an upper bound for ℓ n ( G ) − ℓ n ( G ∗ ) \ell_n(G)-\ell_n(G_*) ℓ n ( G ) − ℓ n ( G ∗ ) based on ρ ( G ) \rho(G) ρ ( G ) for G ∈ U G\in\mathcal{U} G ∈ U , where U \mathcal{U} U is the neighbor in Lemma A.3 , shrinking it as needed. By definition, P ∗ s G 2 = 1 P_*s_G^2=1 P ∗ s G 2 = 1 . Additionally, since both p G p_G p G and p ∗ p_* p ∗ integrate to one,
P ∗ s G = ρ ( G ) − 1 ∫ ( p G − p ∗ ) d μ = 0 , (56) P_*s_G=\rho(G)^{-1}\int(p_G-p_*)\,d\mu=0, \tag*{(56)} P ∗ s G = ρ ( G ) − 1 ∫ ( p G − p ∗ ) d μ = 0 , (56)
Consider F 0 F_0 F 0 be the envelope from Lemma A.5 . As F 0 2 F_0^2 F 0 2 is integrable, we can choose a positive K < ∞ K < \infty K < ∞ such that P ∗ { F 0 2 1 ( F 0 > K ) } < 1 4 P_*\{F_0^2\mathbb{1}(F_0 > K)\} < \frac{1}{4} P ∗ { F 0 2 1 ( F 0 > K )} < 4 1 . Then, for every s ∈ S s \in S s ∈ S , s 2 ≤ F 0 2 s^2 \le F_0^2 s 2 ≤ F 0 2 , we have
P ∗ { s 2 1 ( F 0 ≤ K ) } = 1 − P ∗ { s 2 1 ( F 0 > K ) } ≥ 3 4 . P_*\{s^2\mathbb{1}(F_0 \le K)\} = 1 - P_*\{s^2\mathbb{1}(F_0 > K)\} \ge\frac{3}{4}. P ∗ { s 2 1 ( F 0 ≤ K )} = 1 − P ∗ { s 2 1 ( F 0 > K )} ≥ 4 3 .
From part (iii) of Lemma A.5 about the Glivenko–Cantelli property of the truncated squared class,
sup s ∈ S ∣ ( P n − P ∗ ) { s 2 1 ( F 0 ≤ K ) } ∣ → P 0. \sup_{s\in S}\left|(P_n-P_*)\{s^2\mathbb{1}(F_0\le K)\}\right|\xrightarrow{P}0. s ∈ S sup ( P n − P ∗ ) { s 2 1 ( F 0 ≤ K )} P 0.
Therefore the event E n : = { inf s ∈ S P n { s 2 1 ( F 0 ≤ K ) } ≥ 1 2 } \mathcal{E}_n := \{\inf_{s\in S}P_n\{s^2\mathbb{1}(F_0\le K)\}\ge\frac{1}{2}\} E n := { inf s ∈ S P n { s 2 1 ( F 0 ≤ K )} ≥ 2 1 } has probability tending to one.
Using the fact that c ρ 2 ( G ) ≤ W 2 2 ( G , G ∗ ) c\rho^2(G)\le W_2^2(G,G_*) c ρ 2 ( G ) ≤ W 2 2 ( G , G ∗ ) proved in Lemma A.4 , it is possible to shrink the neighbor U \mathcal{U} U such that in U \mathcal{U} U , as W 2 ( G , G ∗ ) ≤ m W_2(G,G_*)\le m W 2 ( G , G ∗ ) ≤ m , then ρ ( G ) ≲ m \rho(G)\lesssim\sqrt{m} ρ ( G ) ≲ m . By shrinking neighbor U \mathcal{U} U if necessary, we can suppose that ρ ( G ) K ≤ 1 / 2 \rho(G)K\le1/2 ρ ( G ) K ≤ 1/2 for all G ∈ U G\in\mathcal{U} G ∈ U . On { F 0 ≤ K } \{F_0\le K\} { F 0 ≤ K } , we have
∣ p G − p ∗ p ∗ ∣ = ρ ( G ) ∣ s G ∣ ≤ ρ ( G ) K ≤ 1 2 . \left|\frac{p_G-p_*}{p_*}\right|=\rho(G)|s_G|\le\rho(G)K\le\frac{1}{2}. p ∗ p G − p ∗ = ρ ( G ) ∣ s G ∣ ≤ ρ ( G ) K ≤ 2 1 .
Applying the inequality log ( 1 + y ) ≤ y − 1 4 y 2 1 ( y ≤ 1 2 ) \log(1+y)\le y-\frac{1}{4}y^2\mathbb{1}(y\le\frac{1}{2}) log ( 1 + y ) ≤ y − 4 1 y 2 1 ( y ≤ 2 1 ) , we have
log ( 1 + p G − p ∗ p ∗ ) = log ( 1 + ρ ( G ) s ( G ) ) ≤ ρ ( G ) s ( G ) − 1 4 ρ ( G ) 2 s ( G ) 2 1 ( F 0 ≤ K ) . \log\left(1+\frac{p_G-p_*}{p_*}\right)=\log(1+\rho(G)s(G))\le\rho(G)s(G)-\frac{1}{4}\rho(G)^2s(G)^2\mathbb{1}(F_0\le K). log ( 1 + p ∗ p G − p ∗ ) = log ( 1 + ρ ( G ) s ( G )) ≤ ρ ( G ) s ( G ) − 4 1 ρ ( G ) 2 s ( G ) 2 1 ( F 0 ≤ K ) .
By taking the expectation with respect to empirical measure P n P_n P n , we have
ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − 1 4 n ρ ( G ) 2 P n { s G 2 1 ( F 0 ≤ K ) } . \ell_n(G)-\ell_n(G_*)\le\sqrt{n}\rho(G)\mathbb{G}_ns_G-\frac{1}{4}n\rho(G)^2P_n\{s_G^2\mathbb{1}(F_0\le K)\}. ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − 4 1 n ρ ( G ) 2 P n { s G 2 1 ( F 0 ≤ K )} .
On the event E n \mathcal{E}_n E n , by letting c = 1 / 8 c=1/8 c = 1/8 , this inequality becomes
ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − c n ρ ( G ) 2 . (57) \ell_n(G)-\ell_n(G_*)\le\sqrt{n}\rho(G)\mathbb{G}_ns_G-cn\rho(G)^2. \tag*{(57)} ℓ n ( G ) − ℓ n ( G ∗ ) ≤ n ρ ( G ) G n s G − c n ρ ( G ) 2 . (57)
Step 2: We prove that n ρ ( G ^ n ) = O P ( n − 1 / 2 ) \sqrt{n}\rho(\widehat{G}_n)=O_P(n^{-1/2}) n ρ ( G n ) = O P ( n − 1/2 ) . Lemma A.1 gives ρ ( G ^ n ) → P 0 \rho(\widehat{G}_n)\xrightarrow{P}0 ρ ( G n ) P 0 . Since K K K is fixed, P { ρ ( G ^ n ) K ≤ 1 / 2 } → 1 \mathbb{P}\{\rho(\widehat{G}_n)K\le1/2\}\to1 P { ρ ( G n ) K ≤ 1/2 } → 1 . Moreover, G ^ n \widehat{G}_n G n belongs to the neighborhood U \mathcal{U} U with probability tending to one. Hence using the estimation (57) , on an event whose probability tends to one, we have
ℓ n ( G ^ n ) − ℓ n ( G ∗ ) ≤ n ρ ( G ^ n ) G n s G ^ n − c n ρ ( G ^ n ) 2 . \ell_n(\widehat{G}_n)-\ell_n(G_*)\le\sqrt{n}\rho(\widehat{G}_n)\mathbb{G}_ns_{\widehat{G}_n}-cn\rho(\widehat{G}_n)^2. ℓ n ( G n ) − ℓ n ( G ∗ ) ≤ n ρ ( G n ) G n s G n − c n ρ ( G n ) 2 .
Moreover, by the definition of the maximum likelihood estimator, we have 0 ≤ ℓ n ( G ^ n ) − ℓ n ( G ∗ ) 0\le\ell_n(\widehat{G}_n)-\ell_n(G_*) 0 ≤ ℓ n ( G n ) − ℓ n ( G ∗ ) . Therefore, by defining Z n : = sup s ∈ S ∣ G n s ∣ Z_n:=\sup_{s\in S}|\mathbb{G}_ns| Z n := sup s ∈ S ∣ G n s ∣ , on an event whose probability tends to one,
0 ≤ n ρ ( G ^ n ) Z n − c n ρ ( G ^ n ) 2 , 0\le\sqrt{n}\rho(\widehat{G}_n)Z_n-cn\rho(\widehat{G}_n)^2, 0 ≤ n ρ ( G n ) Z n − c n ρ ( G n ) 2 ,
which implies n ρ ( G ^ n ) ≤ c − 1 Z n \sqrt{n}\rho(\widehat{G}_n)\le c^{-1}Z_n n ρ ( G n ) ≤ c − 1 Z n . Furthermore, Lemma A.5 implies Z n = O P ( 1 ) Z_n=O_P(1) Z n = O P ( 1 ) , which leads to ρ ( G ^ n ) = O P ( n − 1 / 2 ) \rho(\widehat{G}_n)=O_P(n^{-1/2}) ρ ( G n ) = O P ( n − 1/2 ) .
For the rate of W 2 2 ( G ^ n , G ∗ ) W_2^2(\widehat{G}_n,G_*) W 2 2 ( G n , G ∗ ) , Lemma A.4 gives
W 2 2 ( G ^ n , G ∗ ) ≤ C ρ ( G ^ n ) = O P ( n − 1 / 2 ) . W_2^2(\widehat{G}_n,G_*)\le C\rho(\widehat{G}_n)=O_P(n^{-1/2}). W 2 2 ( G n , G ∗ ) ≤ C ρ ( G n ) = O P ( n − 1/2 ) .
This proves both assertions.
Proofs for Section 3
Properties of attainable tangent set T k T_k T k
In this section, we prove the geometric properties of the attainable tangent set T k T_k T k . Proposition 3.1 characterizes its elements and establishes uniqueness of their canonical representations, while Lemma B.1 shows that each bounded section T k , M T_{k,M} T k , M is compact in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . These results describe the structure of T k T_k T k and provide key ingredients for proving the density weak limit in Theorem 3.4 .
Proof of Proposition 3.1 . The proof has three parts: necessity, sufficiency, and uniqueness of the canonical representation. The geometric approach is related to tangent-cone methods for constrained estimation ([11 ] ) and locally conic parametrizations of mixture models ([4 ] ). Here, we explicitly track the number of fitted atoms to derive the component-budget constraint (7) .
Part 1: Necessity. Consider a sequence of measures { G m } m ⊂ O k ( Θ ) \{G_m\}_m \subset O_k(\Theta) { G m } m ⊂ O k ( Θ ) along with t m ↓ 0 t_m \downarrow0 t m ↓ 0 satisfying ∥ ( p G m − p ∗ ) / ( t m p ∗ ) − h ∥ ∗ → 0 \left\|(p_{G_m}-p_*)/(t_m p_*)-h\right\|_* \to0 ∥ ( p G m − p ∗ ) / ( t m p ∗ ) − h ∥ ∗ → 0 for some h h h . We construct a quadruple γ = ( α , β , C , ν ) \gamma=(\alpha,\beta,C,\nu) γ = ( α , β , C , ν ) such that
h ( x ) = h γ ( x ) = 1 p ∗ ( x ) [ ∑ i = 1 p { α i f θ i ∗ ( x ) + β i T ∇ f θ i ∗ ( x ) + 1 2 tr ( C i ∇ 2 f θ i ∗ ( x ) ) } + ∫ f θ ( x ) d ν ( θ ) ] h(x)=h_\gamma(x)=\frac{1}{p_*(x)}\left[\sum_{i=1}^{p}\left\{\alpha_i f_{\theta_i^*}(x)+\beta_i^{\mathsf T}\nabla f_{\theta_i^*}(x)+\frac{1}{2}\operatorname{tr}\left(C_i\nabla^2 f_{\theta_i^*}(x)\right)\right\}+\int f_\theta(x)\,\mathrm{d}\nu(\theta)\right] h ( x ) = h γ ( x ) = p ∗ ( x ) 1 [ i = 1 ∑ p { α i f θ i ∗ ( x ) + β i T ∇ f θ i ∗ ( x ) + 2 1 tr ( C i ∇ 2 f θ i ∗ ( x ) ) } + ∫ f θ ( x ) d ν ( θ ) ]
and show that the quadruple γ \gamma γ satisfies the conditions in (6) and (7) .
Step 1: Construction of quadruple γ \gamma γ . From the assumption about G m G_m G m , we have
ρ ( G m ) = ∥ p G m − p ∗ p ∗ ∥ ∗ = t m ∥ h + o L 2 ( 1 ) ∥ ∗ = O ( t m ) . \rho(G_m)=\left\|\frac{p_{G_m}-p_*}{p_*}\right\|_* =t_m\left\|h+o_{L^2}(1)\right\|_*=O(t_m). ρ ( G m ) = p ∗ p G m − p ∗ ∗ = t m ∥ h + o L 2 ( 1 ) ∥ ∗ = O ( t m ) .
Therefore, ρ ( G m ) → 0 \rho(G_m)\to0 ρ ( G m ) → 0 , which leads to G m ⇒ G ∗ G_m\Rightarrow G_* G m ⇒ G ∗ . For all sufficiently large m m m , G m G_m G m lies in the local neighborhood U \mathcal{U} U in Lemma A.3 , so that D ( G m ) = O ( t m ) D(G_m)=O(t_m) D ( G m ) = O ( t m ) . In particular,
∑ i ∣ a i ( G m ) ∣ = O ( t m ) , ∑ i ∥ b i ( G m ) ∥ = O ( t m ) , ∑ j π j , m r j , m 2 = O ( t m ) . (58) \sum_i\lvert a_i(G_m)\rvert=O(t_m),\qquad\sum_i\lVert b_i(G_m)\rVert=O(t_m),\qquad\sum_j\pi_{j,m}r_{j,m}^2=O(t_m). \tag*{(58)} i ∑ ∣ a i ( G m )∣ = O ( t m ) , i ∑ ∥ b i ( G m )∥ = O ( t m ) , j ∑ π j , m r j , m 2 = O ( t m ) . (58)
By adding zero-weight atoms, we can pad to exactly k k k labels. Since Θ k \Theta^k Θ k is compact, every subsequence has a further subsequence for which θ j , m → θ ˉ j ∈ Θ \theta_{j,m}\to\bar{\theta}_j\in\Theta θ j , m → θ ˉ j ∈ Θ for j = 1 , … , k j=1,\ldots,k j = 1 , … , k . We call a label j j j to be local to i i i if θ ˉ j = θ i ∗ \bar{\theta}_j=\theta_i^* θ ˉ j = θ i ∗ , and call it non-local if θ ˉ j \bar{\theta}_j θ ˉ j is not a true atom. Because the true support is finite and separated, an atom local to i i i is eventually assigned by the nearest-center map to i i i . Conversely, an atom whose limit is another true center cannot remain assigned to θ i ∗ \theta_i^* θ i ∗ for large m m m .
Now we construct the measure ν \nu ν , which stands for the non-local component of G m G_m G m . Fix a nonlocal label j j j , define δ j : = min i ∥ θ ˉ j − θ i ∗ ∥ > 0 \delta_j:=\min_i\lVert\bar{\theta}_j-\theta_i^*\rVert>0 δ j := min i ∥ θ ˉ j − θ i ∗ ∥ > 0 as its limiting distance from the true support. For large m m m , r j , m ≥ δ j / 2 r_{j,m}\ge\delta_j/2 r j , m ≥ δ j /2 . Since every term in the nonnegative sum ∑ j π j , m r j , m 2 \sum_j\pi_{j,m}r_{j,m}^2 ∑ j π j , m r j , m 2 is bounded by D ( G m ) D(G_m) D ( G m ) , it follows that
π j , m δ j 2 4 ≤ π j , m r j , m 2 = O ( t m ) , \pi_{j,m}\frac{\delta_j^2}{4}\le\pi_{j,m}r_{j,m}^2=O(t_m), π j , m 4 δ j 2 ≤ π j , m r j , m 2 = O ( t m ) ,
so π j , m = O ( t m ) \pi_{j,m}=O(t_m) π j , m = O ( t m ) . After another subsequence, π j , m / t m → c j ≥ 0 \pi_{j,m}/t_m\to c_j\ge0 π j , m / t m → c j ≥ 0 . By merging labels having the same nontrue limiting location and retaining only the positive limiting coefficients, we obtain a finite positive measure ν = ∑ ℓ = 1 q c ℓ δ η ℓ \nu= \sum_{\ell=1}^{q} c_{\ell}\delta_{\eta_{\ell}} ν = ∑ ℓ = 1 q c ℓ δ η ℓ , where c ℓ > 0 c_{\ell} > 0 c ℓ > 0 , with pairwise distinct nontrue η ℓ \eta_{\ell} η ℓ .
The next step is to construct the component α \alpha α , β \beta β , C C C of the quadruple γ \gamma γ . Let L i L_i L i be the set of labels local to θ i ∗ \theta_i^* θ i ∗ and define M i , m = ∑ j ∈ L i π j , m M_{i,m} = \sum_{j \in L_i} \pi_{j,m} M i , m = ∑ j ∈ L i π j , m . Because G m ⇒ G ∗ G_m \Rightarrow G_* G m ⇒ G ∗ and the true atoms are separated, choose disjoint balls B i B_i B i centered at θ i ∗ \theta_i^* θ i ∗ containing no other true atom. Eventually all local labels in L i L_i L i lie in B i B_i B i , all labels local to another true atom lie outside B i B_i B i , and all genuinely nonlocal labels have total mass O ( t m ) = o ( 1 ) O(t_m) = o(1) O ( t m ) = o ( 1 ) . Weak convergence therefore gives
M i , m → G ∗ ( { θ i ∗ } ) = π i ∗ > 0. M_{i,m} \to G_*(\{\theta_i^*\}) = \pi_i^* > 0. M i , m → G ∗ ({ θ i ∗ }) = π i ∗ > 0.
In particular, every L i L_i L i is nonempty for m m m sufficiently large. Define
α i , m = 1 t m ( M i , m − π i ∗ ) , \alpha_{i,m} = \frac{1}{t_m}(M_{i,m} - \pi_i^*), α i , m = t m 1 ( M i , m − π i ∗ ) ,
β i , m = 1 t m ∑ j ∈ L i π j , m ( θ j , m − θ i ∗ ) , \beta_{i,m} = \frac{1}{t_m}\sum_{j \in L_i} \pi_{j,m}(\theta_{j,m} - \theta_i^*), β i , m = t m 1 j ∈ L i ∑ π j , m ( θ j , m − θ i ∗ ) ,
C i , m = 1 t m ∑ j ∈ L i π j , m ( θ j , m − θ i ∗ ) ( θ j , m − θ i ∗ ) T . C_{i,m} = \frac{1}{t_m}\sum_{j \in L_i} \pi_{j,m}(\theta_{j,m} - \theta_i^*)(\theta_{j,m} - \theta_i^*)^{\mathsf{T}}. C i , m = t m 1 j ∈ L i ∑ π j , m ( θ j , m − θ i ∗ ) ( θ j , m − θ i ∗ ) T .
We show that { α i , m } m \{\alpha_{i,m}\}_m { α i , m } m , { β i , m } m \{\beta_{i,m}\}_m { β i , m } m , and { C i , m } m \{C_{i,m}\}_m { C i , m } m are bounded. For { α i , m } m \{\alpha_{i,m}\}_m { α i , m } m , we have
a i ( G m ) = ∑ j : j i = j π j , m − π i ∗ = t m α i , m ⏟ local component + ∑ j : j i = j j ∉ L i π j , m ⏟ non-local component . a_i(G_m) = \sum_{j:j_i=j} \pi_{j,m} - \pi_i^* = \underbrace{t_m\alpha_{i,m}}_{\text{local component}} + \underbrace{\sum_{\substack{j:j_i=j\\j\notin L_i}} \pi_{j,m}}_{\text{non-local component}}. a i ( G m ) = j : j i = j ∑ π j , m − π i ∗ = local component t m α i , m + non-local component j : j i = j j ∈ / L i ∑ π j , m .
From the construction of ν \nu ν , we have π j , m = O ( t m ) \pi_{j,m} = O(t_m) π j , m = O ( t m ) for non-local index j j j . Therefore, as a i ( G m ) = O ( t m ) a_i(G_m) = O(t_m) a i ( G m ) = O ( t m ) , we have α i , m = O ( 1 ) \alpha_{i,m} = O(1) α i , m = O ( 1 ) , which is bounded. Using similar argument, we also have that β i , m \beta_{i,m} β i , m is bounded. For the second-moment components C i , m C_{i,m} C i , m are positive semidefinite matrices whose traces are bounded because
∑ i tr ( C i , m ) ≤ 1 t m ∑ j π j , m r j , m 2 = O ( 1 ) . \sum_i \operatorname{tr}(C_{i,m}) \le\frac{1}{t_m}\sum_j \pi_{j,m}r_{j,m}^{2} = O(1). i ∑ tr ( C i , m ) ≤ t m 1 j ∑ π j , m r j , m 2 = O ( 1 ) .
As a consequence, C i , m C_{i,m} C i , m are bounded. By passing to a subsequence, there exist a triple ( α , β , C ) (\alpha,\beta,C) ( α , β , C ) such that
α i , m → α i , β i , m → β i , C i , m → C i ⪰ 0. \alpha_{i,m} \to\alpha_i,\qquad\beta_{i,m} \to\beta_i,\qquad C_{i,m} \to C_i \succeq0. α i , m → α i , β i , m → β i , C i , m → C i ⪰ 0.
Step 2: Prove that h = h γ h = h_\gamma h = h γ . For γ \gamma γ established in previous step, we show that h γ h_\gamma h γ is exactly the h h h such that ∥ ( p G m − p ∗ ) / ( t m p ∗ ) − h ∥ ∗ → 0 \|(p_{G_m} - p_*)/(t_m p_*) - h\|_* \to0 ∥ ( p G m − p ∗ ) / ( t m p ∗ ) − h ∥ ∗ → 0 . For j ∈ L i j \in L_i j ∈ L i , write Δ j , m = θ j , m − θ i ∗ \Delta_{j,m} = \theta_{j,m} - \theta_i^* Δ j , m = θ j , m − θ i ∗ , then max j ∈ L i ∥ Δ j , m ∥ → 0 \max_{j \in L_i}\|\Delta_{j,m}\| \to0 max j ∈ L i ∥ Δ j , m ∥ → 0 . Second-order Taylor’s formula along the segment lying θ i ∗ \theta_i^* θ i ∗ and θ j , m \theta_{j,m} θ j , m gives
f θ i ∗ + Δ j , m = f θ i ∗ + Δ j , m T ∇ f θ i ∗ + 1 2 tr ( Δ j , m Δ j , m T ∇ 2 f θ i ∗ ) + R j , m . (59) f_{\theta_i^*+\Delta_{j,m}} = f_{\theta_i^*} + \Delta_{j,m}^{\mathsf{T}}\nabla f_{\theta_i^*} + \frac{1}{2}\operatorname{tr}(\Delta_{j,m}\Delta_{j,m}^{\mathsf{T}}\nabla^2 f_{\theta_i^*}) + R_{j,m}. \tag*{(59)} f θ i ∗ + Δ j , m = f θ i ∗ + Δ j , m T ∇ f θ i ∗ + 2 1 tr ( Δ j , m Δ j , m T ∇ 2 f θ i ∗ ) + R j , m . (59)
Here, by using Assumption 2.1 , the pointwise relative bound for the remainder R j , m R_{j,m} R j , m can be expressed as
∣ R j , m ( x ) p ∗ ( x ) ∣ ≤ C F ( x ) ∥ Δ j , m ∥ 3 . \left|\frac{R_{j,m}(x)}{p_*(x)}\right| \le C F(x)\|\Delta_{j,m}\|^3. p ∗ ( x ) R j , m ( x ) ≤ C F ( x ) ∥ Δ j , m ∥ 3 .
Therefore, after weighting and summing, and combining with estimation (58) , we have
∥ ∑ j ∈ L i π j , m R j , m t m p ∗ ∥ ∗ ≤ C ∥ F ∥ 2 ( max j ∈ L i ∥ Δ j , m ∥ ) ∑ j ∈ L i π j , m ∥ Δ j , m ∥ 2 t m ⟶ 0 , \left\|\frac{\sum_{j\in L_i}\pi_{j,m}R_{j,m}}{t_m p_*}\right\|_* \le C\|F\|_2\left(\max_{j\in L_i}\|\Delta_{j,m}\|\right)\frac{\sum_{j\in L_i}\pi_{j,m}\|\Delta_{j,m}\|^2}{t_m}\longrightarrow0, t m p ∗ ∑ j ∈ L i π j , m R j , m ∗ ≤ C ∥ F ∥ 2 ( j ∈ L i max ∥ Δ j , m ∥ ) t m ∑ j ∈ L i π j , m ∥ Δ j , m ∥ 2 ⟶ 0 ,
Consequently, the local component sum converges in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) :
1 t m p ∗ { ∑ j ∈ L i π j , m f θ j , m − π i ∗ f θ i ∗ } ⟶ α i f i + β i T ∇ f θ i ∗ + 1 2 tr ( C i ∇ 2 f θ i ∗ ) p ∗ \frac{1}{t_m p_*}\left\{\sum_{j\in L_i}\pi_{j,m}f_{\theta_{j,m}}-\pi_i^*f_{\theta_i^*}\right\}\longrightarrow\frac{\alpha_i f_i+\beta_i^{\mathsf T}\nabla f_{\theta_i^*}+\frac{1}{2}\operatorname{tr}(C_i\nabla^2 f_{\theta_i^*})}{p_*} t m p ∗ 1 ⎩ ⎨ ⎧ j ∈ L i ∑ π j , m f θ j , m − π i ∗ f θ i ∗ ⎭ ⎬ ⎫ ⟶ p ∗ α i f i + β i T ∇ f θ i ∗ + 2 1 tr ( C i ∇ 2 f θ i ∗ )
For nonlocal labels, consider the center θ j , m → η \theta_{j,m}\to\eta θ j , m → η and π j , m / t m → c \pi_{j,m}/t_m\to c π j , m / t m → c , then we have
∥ π j , m t m f θ j , m p ∗ − c f η p ∗ ∥ ∗ ⟶ 0. \left\|\frac{\pi_{j,m}}{t_m}\frac{f_{\theta_{j,m}}}{p_*}-c\frac{f_\eta}{p_*}\right\|_*\longrightarrow0. t m π j , m p ∗ f θ j , m − c p ∗ f η ∗ ⟶ 0.
Indeed, the coefficient converges and θ ↦ f θ / p ∗ \theta\mapsto f_\theta/p_* θ ↦ f θ / p ∗ is L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) continuous by the derivative envelope. Summing the finitely many nonlocal labels gives the term ∫ f θ d ν ( θ ) / p ∗ \int f_\theta\,d\nu(\theta)/p_* ∫ f θ d ν ( θ ) / p ∗ . Combining the local and nonlocal limits proves h = h γ h=h_\gamma h = h γ with the stated canonical form.
Step 3: Verify the conditions (6) and (7) for γ \gamma γ . For the mass conservation condition in (6) , noting that for m m m sufficiently large,
∑ i M i , m + ∑ j ∈ L 0 π j , m = 1 , \sum_i M_{i,m}+\sum_{j\in L_0}\pi_{j,m}=1, i ∑ M i , m + j ∈ L 0 ∑ π j , m = 1 ,
where L 0 L_0 L 0 is the set of nonlocal labels. Subtracting ∑ i π i ∗ = 1 \sum_i\pi_i^*=1 ∑ i π i ∗ = 1 from the preceding identity, dividing by t m t_m t m , and passing to the limit, noting that the non-local labels produce the sum of mass of ν \nu ν and local label ( M i , m − π i ∗ ) / t m → α i (M_{i,m}-\pi_i^*)/t_m\to\alpha_i ( M i , m − π i ∗ ) / t m → α i , we obtain mass conservation in (6) :
∑ i α i + ∑ ℓ = 1 q c ℓ = 0 , \sum_i\alpha_i+\sum_{\ell=1}^{q}c_\ell=0, i ∑ α i + ℓ = 1 ∑ q c ℓ = 0 ,
For rank condition (7) , the definition
C i , m = 1 t m ∑ j ∈ L i π j , m ( θ j , m − θ i ∗ ) ( θ j , m − θ i ∗ ) T . C_{i,m}=\frac{1}{t_m}\sum_{j\in L_i}\pi_{j,m}(\theta_{j,m}-\theta_i^*)(\theta_{j,m}-\theta_i^*)^{\mathsf T}. C i , m = t m 1 j ∈ L i ∑ π j , m ( θ j , m − θ i ∗ ) ( θ j , m − θ i ∗ ) T .
directly yields rank ( C i , m ) ≤ r i \operatorname{rank}(C_{i,m})\le r_i rank ( C i , m ) ≤ r i for r i = ∣ L i ∣ r_i=|L_i| r i = ∣ L i ∣ , which leads to rank ( C i ) ≤ r i \operatorname{rank}(C_i)\le r_i rank ( C i ) ≤ r i . We establish the sharper rank bound that rank ( C i ) ≤ r i − 1 \operatorname{rank}(C_i)\le r_i-1 rank ( C i ) ≤ r i − 1 . Define the weighted local barycenter
Δ ‾ i , m = ∑ j ∈ L i π j , m Δ j , m M i , m = t m β i , m M i , m \overline{\Delta}_{i,m}=\frac{\sum_{j\in L_i}\pi_{j,m}\Delta_{j,m}}{M_{i,m}}=\frac{t_m\beta_{i,m}}{M_{i,m}} Δ i , m = M i , m ∑ j ∈ L i π j , m Δ j , m = M i , m t m β i , m
and the weighted centered covariance
V i , m = ∑ j ∈ L i π j , m ( Δ j , m − Δ ‾ i , m ) ( Δ j , m − Δ ‾ i , m ) T . V_{i,m}=\sum_{j\in L_i}\pi_{j,m}(\Delta_{j,m}-\overline{\Delta}_{i,m})(\Delta_{j,m}-\overline{\Delta}_{i,m})^{\mathsf T}. V i , m = j ∈ L i ∑ π j , m ( Δ j , m − Δ i , m ) ( Δ j , m − Δ i , m ) T .
Let r i , m + r_{i,m}^{+} r i , m + denote the number of labels in L i L_i L i having positive weight at sample size m m m such that they are only the labels contributing to V i , m V_{i,m} V i , m . Their centered displacement vectors satisfy the weighted relation
∑ j ∈ L i : π j , m > 0 π j , m ( Δ j , m − Δ ‾ i , m ) = 0. \sum_{\substack{j\in L_i:\\ \pi_{j,m}>0}} \pi_{j,m}\left(\Delta_{j,m}-\overline{\Delta}_{i,m}\right)=0. j ∈ L i : π j , m > 0 ∑ π j , m ( Δ j , m − Δ i , m ) = 0.
Hence their span has dimension at most r i , m + − 1 r_{i,m}^{+}-1 r i , m + − 1 , and therefore rank ( V i , m ) ≤ r i , m + − 1 ≤ r i − 1 \operatorname{rank}(V_{i,m})\leq r_{i,m}^{+}-1\leq r_i-1 rank ( V i , m ) ≤ r i , m + − 1 ≤ r i − 1 . Direct expansion gives
C i , m = V i , m t m + t m β i , m β i , m T M i , m . C_{i,m}=\frac{V_{i,m}}{t_m}+\frac{t_m\beta_{i,m}\beta_{i,m}^{\mathsf T}}{M_{i,m}}. C i , m = t m V i , m + M i , m t m β i , m β i , m T .
Since M i , m → π i ∗ > 0 M_{i,m}\to\pi_i^*>0 M i , m → π i ∗ > 0 and β i , m \beta_{i,m} β i , m is bounded, the second term tends to zero, therefore V i , m / t m → C i V_{i,m}/t_m\to C_i V i , m / t m → C i . In addition, the set { A : rank ( A ) ≤ r i − 1 } \{A:\operatorname{rank}(A)\leq r_i-1\} { A : rank ( A ) ≤ r i − 1 } is closed, thus, we obtain the rank bound rank ( C i ) ≤ r i − 1 \operatorname{rank}(C_i)\leq r_i-1 rank ( C i ) ≤ r i − 1 .
The sets L 1 , … , L p L_1,\ldots,L_p L 1 , … , L p and the nonlocal labels are disjoint subsets of the k k k labels. Each distinct positive-mass nonlocal limit η ℓ \eta_\ell η ℓ requires at least one nonlocal label. Hence, q + ∑ i = 1 p r i ≤ k q+\sum_{i=1}^{p}r_i\leq k q + ∑ i = 1 p r i ≤ k . Using the rank bound above, we obtain the budget restriction (7) :
q + ∑ i rank ( C i ) ≤ q + ∑ i ( r i − 1 ) ≤ k − p . q+\sum_i\operatorname{rank}(C_i)\leq q+\sum_i(r_i-1)\leq k-p. q + i ∑ rank ( C i ) ≤ q + i ∑ ( r i − 1 ) ≤ k − p .
This completes the necessity argument.
Part 2: Sufficiency. Now fix a canonical configuration γ = ( α , β , C , ν ) \gamma=(\alpha,\beta,C,\nu) γ = ( α , β , C , ν ) satisfying the conditions (6) and (7) , and denote q q q the number of support point of ν \nu ν . We construct a family of discrete measures ( G t ) t ↓ 0 ⊂ O k ( Θ ) (G_t)_{t\downarrow0}\subset\mathcal{O}_k(\Theta) ( G t ) t ↓ 0 ⊂ O k ( Θ ) such that ∥ ( p G t − p G ∗ ) / ( t p G ∗ ) − h γ ∥ ∗ → 0 \lVert(p_{G_t}-p_{G^*})/(tp_{G^*})-h_\gamma\rVert_*\to0 ∥( p G t − p G ∗ ) / ( t p G ∗ ) − h γ ∥ ∗ → 0 .
Let r i = rank ( C i ) r_i=\operatorname{rank}(C_i) r i = rank ( C i ) . If r i = 0 r_i=0 r i = 0 , take one point v i , 1 = 0 v_{i,1}=0 v i , 1 = 0 with weight 1. Suppose r i ≥ 1 r_i\geq1 r i ≥ 1 . Choose a factorization C i / π i ∗ = A i A i T C_i/\pi_i^*=A_iA_i^{\mathsf T} C i / π i ∗ = A i A i T , for A i ∈ R d × r i A_i\in\mathbb{R}^{d\times r_i} A i ∈ R d × r i with full column rank. Choose z i , 1 , … , z i , r i + 1 z_{i,1},\ldots,z_{i,r_i+1} z i , 1 , … , z i , r i + 1 as the vertices of a regular simplex in R r i \mathbb{R}^{r_i} R r i , centered at the origin and scaled to have squared norm r i r_i r i . With equal weights ω i , h = 1 / ( r i + 1 ) \omega_{i,h}=1/(r_i+1) ω i , h = 1/ ( r i + 1 ) , these vectors satisfy
∑ h = 1 r i + 1 ω i , h z i , h = 0 , ∑ h = 1 r i + 1 ω i , h z i , h z i , h T = I r i . \sum_{h=1}^{r_i+1}\omega_{i,h}z_{i,h}=0,\qquad
\sum_{h=1}^{r_i+1}\omega_{i,h}z_{i,h}z_{i,h}^{\mathsf T}=I_{r_i}. h = 1 ∑ r i + 1 ω i , h z i , h = 0 , h = 1 ∑ r i + 1 ω i , h z i , h z i , h T = I r i .
By setting v i , h = A i z i , h v_{i,h}=A_i z_{i,h} v i , h = A i z i , h , we have
∑ h = 1 r i + 1 ω i , h v i , h = 0 , ∑ h = 1 r i + 1 ω i , h v i , h v i , h T = C i / π i ∗ . (60) \sum_{h=1}^{r_i+1}\omega_{i,h}v_{i,h}=0,\qquad
\sum_{h=1}^{r_i+1}\omega_{i,h}v_{i,h}v_{i,h}^{\mathsf T}=C_i/\pi_i^*. \tag*{(60)} h = 1 ∑ r i + 1 ω i , h v i , h = 0 , h = 1 ∑ r i + 1 ω i , h v i , h v i , h T = C i / π i ∗ . (60)
For sufficiently small t > 0 t>0 t > 0 , define the local cluster mass M i ( t ) = π i ∗ + t α i > 0 M_i(t)=\pi_i^*+t\alpha_i>0 M i ( t ) = π i ∗ + t α i > 0 and locations
θ i , h ( t ) = θ i ∗ + t v i , h + t β i π i ∗ . (61) \theta_{i,h}(t)=\theta_i^*+\sqrt{t}\,v_{i,h}+t\frac{\beta_i}{\pi_i^*}. \tag*{(61)} θ i , h ( t ) = θ i ∗ + t v i , h + t π i ∗ β i . (61)
When t t t is small enough, as θ i ∗ \theta_i^* θ i ∗ lies in the interior of Θ \Theta Θ and there are finitely many v i , h v_{i,h} v i , h , all these points θ i , h ( t ) \theta_{i,h}(t) θ i , h ( t ) belong to Θ \Theta Θ . Thus, the measures G t G_t G t can be constructed by giving θ i , h ( t ) \theta_{i,h}(t) θ i , h ( t ) mass M i ( t ) ω i , h M_i(t)\omega_{i,h} M i ( t ) ω i , h and for each nonlocal atom of ν \nu ν , putting mass t c ℓ tc_\ell t c ℓ at η ℓ \eta_\ell η ℓ :
G t = ∑ i = 1 p ( ∑ h = 1 r i + 1 M i ( t ) ω i , h δ θ i , h ( t ) ) + t ν . G_t=\sum_{i=1}^{p}\left(\sum_{h=1}^{r_i+1}M_i(t)\omega_{i,h}\delta_{\theta_{i,h}(t)}\right)+t\nu. G t = i = 1 ∑ p ( h = 1 ∑ r i + 1 M i ( t ) ω i , h δ θ i , h ( t ) ) + t ν .
By (6) , the total mass of G t G_t G t is exactly equal to
∑ i M i ( t ) + t ∑ ℓ c ℓ = 1 + t ( ∑ i α i + ∑ ℓ c ℓ ) = 1. \sum_i M_i(t) + t\sum_\ell c_\ell= 1 + t\left(\sum_i \alpha_i + \sum_\ell c_\ell\right) = 1. i ∑ M i ( t ) + t ℓ ∑ c ℓ = 1 + t ( i ∑ α i + ℓ ∑ c ℓ ) = 1.
In addition, the number of local support points is ∑ i ( r i + 1 ) = p + ∑ i rank ( C i ) \sum_i(r_i+1)=p+\sum_i\operatorname{rank}(C_i) ∑ i ( r i + 1 ) = p + ∑ i rank ( C i ) , so the total number of support points is at most
p + ∑ i rank ( C i ) + q ≤ k . p+\sum_i\operatorname{rank}(C_i)+q\le k. p + i ∑ rank ( C i ) + q ≤ k .
Therefore, we have G t ∈ O k ( Θ ) G_t\in\mathcal{O}_k(\Theta) G t ∈ O k ( Θ ) .
Now we show that ∥ p G t − p G ∗ t p G ∗ − h γ ∥ ∗ → 0 \left\|\frac{p_{G_t}-p_{G_*}}{tp_{G_*}}-h_\gamma\right\|_*\to0 t p G ∗ p G t − p G ∗ − h γ ∗ → 0 . Let Δ i , h ( t ) = θ i , h ( t ) − θ i ∗ \Delta_{i,h}(t)=\theta_{i,h}(t)-\theta_i^* Δ i , h ( t ) = θ i , h ( t ) − θ i ∗ . We apply the Taylor expansion for the local component as in the equation (59) and evaluate the contribution of weighted zero-, first-, second-order components and remainders. Firstly, we observe that the local displacements satisfy max h ∥ Δ i , h ( t ) ∥ = O ( t ) \max_h\|\Delta_{i,h}(t)\|=O(\sqrt{t}) max h ∥ Δ i , h ( t ) ∥ = O ( t ) . For the first-order displacement, it follows from the equation (60) and the definition of θ i , h \theta_{i,h} θ i , h that as t ↓ 0 t\downarrow0 t ↓ 0 ,
∑ h M i ( t ) ω i , h Δ i , h ( t ) = M i ( t ) { t ∑ h ω i , h v i , h + t β i π i ∗ } = t β i + t 2 α i π i ∗ β i = t β i + o ( t ) . \sum_h M_i(t)\omega_{i,h}\Delta_{i,h}(t)=M_i(t)\left\{\sqrt{t}\sum_h\omega_{i,h}v_{i,h}+t\frac{\beta_i}{\pi_i^*}\right\}=t\beta_i+t^2\frac{\alpha_i}{\pi_i^*}\beta_i=t\beta_i+o(t). h ∑ M i ( t ) ω i , h Δ i , h ( t ) = M i ( t ) { t h ∑ ω i , h v i , h + t π i ∗ β i } = t β i + t 2 π i ∗ α i β i = t β i + o ( t ) .
Again using the centering in (60) , noting that all t 3 / 2 t^{3/2} t 3/2 cross terms vanish after summation, we have
∑ h = 1 r i + 1 M i ( t ) ω i , h Δ i , h ( t ) Δ i , h ( t ) ⊤ = M i ( t ) t ∑ h ω i , h v i , h v i , h ⊤ + O ( t 2 ) = t M i ( t ) π i ∗ C i + O ( t 2 ) = t C i + o ( t ) . \sum_{h=1}^{r_i+1}M_i(t)\omega_{i,h}\Delta_{i,h}(t)\Delta_{i,h}(t)^\top=M_i(t)t\sum_h\omega_{i,h}v_{i,h}v_{i,h}^\top+O(t^2)=t\frac{M_i(t)}{\pi_i^*}C_i+O(t^2)=tC_i+o(t). h = 1 ∑ r i + 1 M i ( t ) ω i , h Δ i , h ( t ) Δ i , h ( t ) ⊤ = M i ( t ) t h ∑ ω i , h v i , h v i , h ⊤ + O ( t 2 ) = t π i ∗ M i ( t ) C i + O ( t 2 ) = t C i + o ( t ) .
For the weighted third-order remainder, we have
∥ 1 t p ∗ ∑ h M i ( t ) ω i , h R i , h ( t ) ∥ ∗ ≤ C ∥ F ∥ 2 O ( t 3 / 2 ) t = O ( t ) → 0. \left\|\frac{1}{tp_*}\sum_h M_i(t)\omega_{i,h}R_{i,h}(t)\right\|_*\le C\|F\|_2\frac{O(t^{3/2})}{t}=O(\sqrt{t})\to0. t p ∗ 1 h ∑ M i ( t ) ω i , h R i , h ( t ) ∗ ≤ C ∥ F ∥ 2 t O ( t 3/2 ) = O ( t ) → 0.
The nonlocal atoms contribute exactly t ∫ f θ d ν ( θ ) t\int f_\theta\,d\nu(\theta) t ∫ f θ d ν ( θ ) . Combining the zeroth-, first-, and second-order calculations gives
∥ p G t − p ∗ t p ∗ − h γ ∥ ∗ → 0. \left\|\frac{p_{G_t}-p_*}{tp_*}-h_\gamma\right\|_*\to0. t p ∗ p G t − p ∗ − h γ ∗ → 0.
Uniqueness of the canonical representation
Suppose that there exist two tangent configurations γ = ( α , β , C , ν ) \gamma=(\alpha,\beta,C,\nu) γ = ( α , β , C , ν ) and γ ~ = ( α ~ , β ~ , C ~ , ν ~ ) \widetilde{\gamma}=(\widetilde{\alpha},\widetilde{\beta},\widetilde{C},\widetilde{\nu}) γ = ( α , β , C , ν ) such that h γ = h γ ~ h_\gamma=h_{\widetilde{\gamma}} h γ = h γ . By multiplying by p ∗ p_* p ∗ and regrouping the term, we have
∑ i = 1 p [ ( α i − α ~ i ) f θ i ∗ + ( β i − β ~ i ) ⊤ ∇ f θ i ∗ + 1 2 tr ( ( C i − C ~ i ) ∇ 2 f θ i ∗ ) ] + ∫ f θ ( x ) d ( ν − ν ~ ) ( θ ) = 0. (62) \sum_{i=1}^{p}\left[(\alpha_i-\widetilde{\alpha}_i)f_{\theta_i^*}+(\beta_i-\widetilde{\beta}_i)^\top\nabla f_{\theta_i^*}+\frac{1}{2}\operatorname{tr}\left((C_i-\widetilde{C}_i)\nabla^2f_{\theta_i^*}\right)\right]+\int f_\theta(x)d(\nu-\widetilde{\nu})(\theta)=0. \tag*{(62)} i = 1 ∑ p [ ( α i − α i ) f θ i ∗ + ( β i − β i ) ⊤ ∇ f θ i ∗ + 2 1 tr ( ( C i − C i ) ∇ 2 f θ i ∗ ) ] + ∫ f θ ( x ) d ( ν − ν ) ( θ ) = 0. (62)
Let q q q and q ~ \widetilde{q} q denote the numbers of support points of ν \nu ν and ν ~ \widetilde{\nu} ν , respectively. The budget constraint implies q , q ~ ≤ k − p q,\widetilde{q}\le k-p q , q ≤ k − p . Therefore, the number of distinct locations in the identity (62) is at most
p + q + q ~ ≤ p + 2 ( k − p ) = 2 k − p ≤ 2 k . p+q+\widetilde{q}\le p+2(k-p)=2k-p\le2k. p + q + q ≤ p + 2 ( k − p ) = 2 k − p ≤ 2 k .
Assumption 2.3 forces every coefficient in the difference identity to vanish. Hence
α i = α ~ i , β i = β ~ i , C i = C ~ i \alpha_i=\widetilde{\alpha}_i,\qquad\beta_i=\widetilde{\beta}_i,\qquad C_i=\widetilde{C}_i α i = α i , β i = β i , C i = C i
for every true atom, and the signed difference of the two nonlocal finite measures ν − ν ~ \nu-\widetilde{\nu} ν − ν is zero. Therefore ν = ν ~ \nu=\widetilde{\nu} ν = ν as measures, which means equality up to permutation of their listed atoms. This proves uniqueness.
Lemma B.1 (Compactness of bounded attainable tangent set). For every M < ∞ M<\infty M < ∞ , the set
T k , M : = T k ∩ { h : ∥ h ∥ ∗ ≤ M } \mathcal{T}_{k,M}:=\mathcal{T}_k\cap\{h:\|h\|_*\le M\} T k , M := T k ∩ { h : ∥ h ∥ ∗ ≤ M } is compact in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) .
Proof. We first show that the unit attainable tangent section S k = { h ∈ T k : ∥ h ∥ ∗ = 1 } \mathcal{S}_k=\{h\in\mathcal{T}_k:\|h\|_*=1\} S k = { h ∈ T k : ∥ h ∥ ∗ = 1 } is relatively compact. Let h ∈ T k h\in\mathcal{T}_k h ∈ T k with ∥ h ∥ ∗ = 1 \|h\|_*=1 ∥ h ∥ ∗ = 1 . Then there exist t m ↓ 0 t_m\downarrow0 t m ↓ 0 and G m ∈ O k ( Θ ) G_m\in\mathcal{O}_k(\Theta) G m ∈ O k ( Θ ) with
p G m − p ∗ t m p ∗ → h in L 2 ( P ∗ ) . \frac{p_{G_m}-p_*}{t_m p_*}\to h\quad\text{in }L^2(P_*). t m p ∗ p G m − p ∗ → h in L 2 ( P ∗ ) .
Then we have
ρ ( G m ) t m = ∥ p G m − p ∗ t m p ∗ ∥ ∗ → 1. \frac{\rho(G_m)}{t_m}=\left\|\frac{p_{G_m}-p_*}{t_m p_*}\right\|_*\to1. t m ρ ( G m ) = t m p ∗ p G m − p ∗ ∗ → 1.
Consequently the normalized secants satisfy
s G m = t m ρ ( G m ) p G m − p ∗ t m p ∗ → h in L 2 ( P ∗ ) . s_{G_m}=\frac{t_m}{\rho(G_m)}\frac{p_{G_m}-p_*}{t_m p_*}\to h\quad\text{in }L^2(P_*). s G m = ρ ( G m ) t m t m p ∗ p G m − p ∗ → h in L 2 ( P ∗ ) .
Thus the unit attainable tangent section S k \mathcal{S}_k S k lies in the closure of the normalized secant class S \mathcal{S} S , and therefore this closure is compact in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) due to Lemma Lemma A.5 .
We prove that T k \mathcal{T}_k T k is closed. Indeed, if h m ∈ T k h_m\in\mathcal{T}_k h m ∈ T k and h m → h h_m\to h h m → h in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) , then from the definition of T k , m \mathcal{T}_{k,m} T k , m , we can choose positive numbers t m < 1 / m t_m<1/m t m < 1/ m and measures G m G_m G m such that
∥ p G m − p ∗ t m p ∗ − h m ∥ ∗ ≤ m − 1 . \left\|\frac{p_{G_m}-p_*}{t_m p_*}-h_m\right\|_*\le m^{-1}. t m p ∗ p G m − p ∗ − h m ∗ ≤ m − 1 .
The triangle inequality yields convergence of p G m − p ∗ t m p ∗ \frac{p_{G_m}-p_*}{t_m p_*} t m p ∗ p G m − p ∗ to h h h in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) , consequently h ∈ T k h\in\mathcal{T}_k h ∈ T k . An implication of this result is that S k \mathcal{S}_k S k is compact in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) .
Finally, we show that T k , M \mathcal{T}_{k,M} T k , M is compact in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . For every a > 0 a>0 a > 0 , if h = h ( α , β , C , ν ) ∈ T k h=h_{(\alpha,\beta,C,\nu)}\in\mathcal{T}_k h = h ( α , β , C , ν ) ∈ T k then a h = h ( a α , a β , a C , a ν ) ∈ T k ah=h_{(a\alpha,a\beta,aC,a\nu)}\in\mathcal{T}_k ah = h ( a α , a β , a C , a ν ) ∈ T k : positive scaling preserves mass conservation, matrix ranks and support counts. Hence the bounded section T k , M \mathcal{T}_{k,M} T k , M is the continuous image of [ 0 , M ] × S k → T k [0,M]\times\mathcal{S}_k\to\mathcal{T}_k [ 0 , M ] × S k → T k , ( a , s ) ↦ a s (a,s)\mapsto as ( a , s ) ↦ a s , and as [ 0 , M ] × S k [0,M]\times\mathcal{S}_k [ 0 , M ] × S k is a compact set, T k , M \mathcal{T}_{k,M} T k , M is a compact set in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) too.
Likelihood expansion and convergence of the maximizer
We now use the tangent-set geometry established above to prove Theorem 3.4 . Lemma Lemma B.2 provides a uniform quadratic expansion of the local log-likelihood, and Lemma Lemma B.3 establishes existence and almost-sure uniqueness of the limiting Gaussian maximizer. Together with root-n n n localization and compactness of bounded tangent sections, these results yield weak convergence of the relative-density MLE through an argmax argument.
Lemma B.2 (Uniform local quadratic likelihood expansion). For every fixed M < ∞ M < \infty M < ∞ ,
sup n ρ ( G ) ≤ M ∣ ℓ n ( G ) − ℓ n ( G ∗ ) − G n h n , G + 1 2 ∥ h n , G ∥ ∗ 2 ∣ → P 0 (63) \sup_{\sqrt{n}\rho(G)\le M}\left|\ell_n(G)-\ell_n(G_\ast)-\mathbb{G}_n h_{n,G}+\frac{1}{2}\lVert h_{n,G}\rVert_\ast^2\right|\xrightarrow{P}0 \tag*{(63)} n ρ ( G ) ≤ M sup ℓ n ( G ) − ℓ n ( G ∗ ) − G n h n , G + 2 1 ∥ h n , G ∥ ∗ 2 P 0 (63) where
h n , G = n p G − p ∗ p ∗ . h_{n,G}=\sqrt{n}\frac{p_G-p_\ast}{p_\ast}. h n , G = n p ∗ p G − p ∗ .
Proof. Step 1: We precise the neighbors of G ∗ G_\ast G ∗ in O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) containing measure G G G that we work through out the proof. Consider the neighborhood U \mathcal{U} U in Lemma A.3 . By identifiability, the unique zero point of ρ \rho ρ is G ∗ G_\ast G ∗ . By compactness of O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) and continuity of G ↦ ρ ( G ) G\mapsto\rho(G) G ↦ ρ ( G ) , there is δ > 0 \delta>0 δ > 0 such that ρ ( G ) < δ \rho(G)<\delta ρ ( G ) < δ implies G ∈ U G\in\mathcal{U} G ∈ U . For M M M fixed, consider H n , M a t t = { h n , G : n ρ ( G ) ≤ M } H^{\mathrm{att}}_{n,M}=\{h_{n,G}:\sqrt{n}\rho(G)\le M\} H n , M att = { h n , G : n ρ ( G ) ≤ M } , then for n ≥ ( M / δ ) 2 n\ge(M/\delta)^2 n ≥ ( M / δ ) 2 , we have H n , M a t t ⊂ U H^{\mathrm{att}}_{n,M}\subset\mathcal{U} H n , M att ⊂ U . In addition, every function in the H n , M a t t H^{\mathrm{att}}_{n,M} H n , M att can be written as
h n , G = a G s G , 0 ≤ a G ≤ M , s G ∈ S . h_{n,G}=a_Gs_G,\qquad0\le a_G\le M,\qquad s_G\in\mathcal{S}. h n , G = a G s G , 0 ≤ a G ≤ M , s G ∈ S .
where the case ρ ( G ) = 0 \rho(G)=0 ρ ( G ) = 0 gives G = G ∗ G=G_\ast G = G ∗ by identifiability and hence h n , G = 0 h_{n,G}=0 h n , G = 0 .
Step 2: We give an estimation for the process max r ≤ n ∣ h n , G ( X r ) ∣ \max_{r\le n}|h_{n,G}(X_r)| max r ≤ n ∣ h n , G ( X r ) ∣ so that we use later in the proof. From Lemma A.5 , ∣ s G ∣ ≤ F 0 |s_G|\le F_0 ∣ s G ∣ ≤ F 0 , hence ∣ h n , G ( x ) ∣ ≤ M F 0 ( x ) |h_{n,G}(x)|\le MF_0(x) ∣ h n , G ( x ) ∣ ≤ M F 0 ( x ) uniformly over the local set. In particular, we have the bound
P ∗ h n , G 2 ≤ M 2 ∥ F 0 ∥ ∗ 2 . P_\ast h_{n,G}^2\le M^2\lVert F_0\rVert_\ast^2. P ∗ h n , G 2 ≤ M 2 ∥ F 0 ∥ ∗ 2 .
For ϵ > 0 \epsilon>0 ϵ > 0 , the union bound and Markov’s inequality of order 2 + η 2+\eta 2 + η give
P { max 1 ≤ r ≤ n F 0 ( X r ) > ϵ n } ≤ n P X ∼ P ∗ { F 0 ( X ) > ϵ n } ≤ n P ∗ F 0 2 + η ϵ 2 + η n 1 + η / 2 = P ∗ F 0 2 + η ϵ 2 + η n η / 2 ⟶ 0. \mathbb{P}\left\{\max_{1\le r\le n}F_0(X_r)>\epsilon\sqrt{n}\right\}
\le n\mathbb{P}_{X\sim P_\ast}\{F_0(X)>\epsilon\sqrt{n}\}
\le\frac{nP_\ast F_0^{2+\eta}}{\epsilon^{2+\eta}n^{1+\eta/2}}
=\frac{P_\ast F_0^{2+\eta}}{\epsilon^{2+\eta}n^{\eta/2}}
\longrightarrow0. P { 1 ≤ r ≤ n max F 0 ( X r ) > ϵ n } ≤ n P X ∼ P ∗ { F 0 ( X ) > ϵ n } ≤ ϵ 2 + η n 1 + η /2 n P ∗ F 0 2 + η = ϵ 2 + η n η /2 P ∗ F 0 2 + η ⟶ 0.
Consequently,
sup n ρ ( G ) ≤ M max r ≤ n ∣ h n , G ( X r ) n ∣ ≤ M n max r ≤ n F 0 ( X r ) = o P ( 1 ) . (64) \sup_{\sqrt{n}\rho(G)\le M}\max_{r\le n}\left|\frac{h_{n,G}(X_r)}{\sqrt{n}}\right|
\le\frac{M}{\sqrt{n}}\max_{r\le n}F_0(X_r)=o_P(1). \tag*{(64)} n ρ ( G ) ≤ M sup r ≤ n max n h n , G ( X r ) ≤ n M r ≤ n max F 0 ( X r ) = o P ( 1 ) . (64)
Step 3: We expand the likelihood difference ℓ n ( G ) − ℓ n ( G ∗ ) \ell_n(G)-\ell_n(G_\ast) ℓ n ( G ) − ℓ n ( G ∗ ) using Taylor expansion and estimate the first, second, and remainder components. Consider the event
Ξ n = { sup n ρ ( G ) ≤ M max r ≤ n ∣ h n , G ( X r ) n ∣ < 1 2 } , \Xi_n=\left\{\sup_{\sqrt{n}\rho(G)\le M}\max_{r\le n}\left|\frac{h_{n,G}(X_r)}{\sqrt{n}}\right|<\frac{1}{2}\right\}, Ξ n = { n ρ ( G ) ≤ M sup r ≤ n max n h n , G ( X r ) < 2 1 } ,
whose probability tends to 1 1 1 as n → ∞ n\to\infty n → ∞ by (64) . In this event, by writing p G ( X r ) p ∗ ( X r ) = 1 + h n , G ( X r ) n \frac{p_G(X_r)}{p_\ast(X_r)}=1+\frac{h_{n,G}(X_r)}{\sqrt{n}} p ∗ ( X r ) p G ( X r ) = 1 + n h n , G ( X r ) , and applying the following estimation for ∣ x ∣ < 1 / 2 |x|<1/2 ∣ x ∣ < 1/2
log ( 1 + x ) = x − 1 2 x 2 + R ( x ) , ∣ R ( x ) ∣ ≤ C ∣ x ∣ 3 , \log(1+x)=x-\frac{1}{2}x^2+R(x),\qquad|R(x)|\le C|x|^3, log ( 1 + x ) = x − 2 1 x 2 + R ( x ) , ∣ R ( x ) ∣ ≤ C ∣ x ∣ 3 ,
we have from estimation (64) that in the set whose probability tending to one when n → ∞ n \to\infty n → ∞ ,
ℓ n ( G ) − ℓ n ( G ∗ ) = 1 n ∑ r = 1 n h n , G ( X r ) − 1 2 n ∑ r = 1 n h n , G ( X r ) 2 + R n , G = n P n h n , G − 1 2 P n h n , G 2 + R n , G . (65) \begin{aligned}
\ell_n(G)-\ell_n(G_*)=\frac{1}{\sqrt{n}}\sum_{r=1}^{n}h_{n,G}(X_r)-\frac{1}{2n}\sum_{r=1}^{n}h_{n,G}(X_r)^2+R_{n,G} \\
&=\sqrt{n}P_nh_{n,G}-\frac{1}{2}P_nh_{n,G}^2+R_{n,G}. \tag*{(65)}
\end{aligned} ℓ n ( G ) − ℓ n ( G ∗ ) = n 1 r = 1 ∑ n h n , G ( X r ) − 2 n 1 r = 1 ∑ n h n , G ( X r ) 2 + R n , G = n P n h n , G − 2 1 P n h n , G 2 + R n , G . (65)
where the remainder R n , G R_{n,G} R n , G can be estimated as
∣ R n , G ∣ ≤ C ∑ r = 1 n ∣ h n , G ( X r ) n ∣ 3 . \lvert R_{n,G}\rvert\leq C\sum_{r=1}^{n}\left\lvert\frac{h_{n,G}(X_r)}{\sqrt{n}}\right\rvert^3. ∣ R n , G ∣ ≤ C r = 1 ∑ n n h n , G ( X r ) 3 .
The next step is to give a representation or estimation for each first, second-order components and remainder R n , G R_{n,G} R n , G .
Step 3.1: Representation for first-order component n P n h n , G \sqrt{n}P_nh_{n,G} n P n h n , G . For every attainable local function, we have P ∗ h n , G = n ∫ ( p G − p ∗ ) d μ = 0 P_*h_{n,G}=\sqrt{n}\int(p_G-p_*)\,d\mu=0 P ∗ h n , G = n ∫ ( p G − p ∗ ) d μ = 0 . Therefore, the first-order term in equation (65) can be written as
n P n h n , G = n ( P n h n , G − P ∗ h n , G ) = G n h n , G . (66) \sqrt{n}P_nh_{n,G}=\sqrt{n}(P_nh_{n,G}-P_*h_{n,G})=\mathbb{G}_nh_{n,G}. \tag*{(66)} n P n h n , G = n ( P n h n , G − P ∗ h n , G ) = G n h n , G . (66)
Step 3.2: Estimation for second-order component 1 2 P n h n , G 2 \frac{1}{2}P_nh_{n,G}^2 2 1 P n h n , G 2 . Taking into account that h n , G = a G s G h_{n,G}=a_Gs_G h n , G = a G s G with a G ≤ M a_G\leq M a G ≤ M , using the Glivenko-Cantelli of the class of function { s 2 : s ∈ S } \{s^2:s\in\mathcal{S}\} { s 2 : s ∈ S } in Lemma A.5, we have
sup n ρ ( G ) ≤ M ∣ ( P n − P ∗ ) h n , G 2 ∣ ≤ M 2 sup s ∈ S ∣ ( P n − P ∗ ) s 2 ∣ = o P ( 1 ) . (67) \sup_{\sqrt{n}\rho(G)\leq M}\left\lvert(P_n-P_*)h_{n,G}^2\right\rvert\leq M^2\sup_{s\in\mathcal{S}}\left\lvert(P_n-P_*)s^2\right\rvert=o_P(1). \tag*{(67)} n ρ ( G ) ≤ M sup ( P n − P ∗ ) h n , G 2 ≤ M 2 s ∈ S sup ( P n − P ∗ ) s 2 = o P ( 1 ) . (67)
Step 3.3: Estimation for remainder component R n , G R_{n,G} R n , G . Since P ∗ h n , G 2 ≤ M 2 ∥ F 0 ∥ ∗ 2 P_*h_{n,G}^2\leq M^2\lVert F_0\rVert_*^2 P ∗ h n , G 2 ≤ M 2 ∥ F 0 ∥ ∗ 2 , estimation (67) yields
sup n ρ ( G ) ≤ M ∣ P n h n , G 2 ∣ = O P ( 1 ) . (68) \sup_{\sqrt{n}\rho(G)\leq M}\left\lvert P_nh_{n,G}^2\right\rvert=O_P(1). \tag*{(68)} n ρ ( G ) ≤ M sup P n h n , G 2 = O P ( 1 ) . (68)
Using ∑ ∣ x r ∣ 3 ≤ ( max ∣ x r ∣ ) ∑ x r 2 \sum\lvert x_r\rvert^3\leq(\max\lvert x_r\rvert)\sum x_r^2 ∑ ∣ x r ∣ 3 ≤ ( max ∣ x r ∣) ∑ x r 2 with x r = h n , G ( X r ) / n x_r=h_{n,G}(X_r)/\sqrt{n} x r = h n , G ( X r ) / n ,
∣ R n , G ∣ ≤ C { max r ≤ n ∣ h n , G ( X r ) n ∣ } P n h n , G 2 . \lvert R_{n,G}\rvert\leq C\left\{\max_{r\leq n}\left\lvert\frac{h_{n,G}(X_r)}{\sqrt{n}}\right\rvert\right\}P_nh_{n,G}^2. ∣ R n , G ∣ ≤ C { r ≤ n max n h n , G ( X r ) } P n h n , G 2 .
By plugging in the estimation in (64) and (68) to the estimation above, we have
sup n ρ ( G ) ≤ M ∣ R n , G ∣ = o P ( 1 ) . (69) \sup_{\sqrt{n}\rho(G)\leq M}\lvert R_{n,G}\rvert=o_P(1). \tag*{(69)} n ρ ( G ) ≤ M sup ∣ R n , G ∣ = o P ( 1 ) . (69)
Step 3.4: Putting everything together. Substituting the estimations in (66) , (67) , and (69) into (65) , we obtain
sup n ρ ( G ) ≤ M ∣ ℓ n ( G ) − ℓ n ( G ∗ ) − G n h n , G + 1 2 ∥ h n , G ∥ ∗ 2 ∣ → P 0 , \sup_{\sqrt{n}\rho(G)\leq M}\left\lvert\ell_n(G)-\ell_n(G_*)-\mathbb{G}_nh_{n,G}+\frac{1}{2}\lVert h_{n,G}\rVert_*^2\right\rvert\xrightarrow{P}0, n ρ ( G ) ≤ M sup ℓ n ( G ) − ℓ n ( G ∗ ) − G n h n , G + 2 1 ∥ h n , G ∥ ∗ 2 P 0 ,
which is (63) . This completes our proof.
Lemma B.3 (Existence and almost-sure uniqueness of the Gaussian argmax). *The maximization problem (9) admits an almost surely unique maximizer. In other words, there exists an almost surely
unique H † ∈ T k H^\dagger\in\mathcal{T}_k H † ∈ T k such that
Q ( H † ) = sup h ∈ T k Q ( h ) . Q(H^\dagger) = \sup_{h \in\mathcal{T}_k} Q(h). Q ( H † ) = h ∈ T k sup Q ( h ) .
Moreover, for every fixed M > 0 M > 0 M > 0 , the restriction of Q Q Q to T k , M : = T k ∩ { h : ∥ h ∥ ∗ ≤ M } \mathcal{T}_{k,M} := \mathcal{T}_k \cap\{h : \lVert h \rVert_\ast\le M\} T k , M := T k ∩ { h : ∥ h ∥ ∗ ≤ M } also admits an almost surely unique maximizer H M † H^\dagger_M H M † .
Proof. Step 1: Existence. We prove the almost sure existence of maximizer of Q Q Q over T k \mathcal{T}_k T k . By Lemma A.5 , the class S \mathcal{S} S is P ∗ P_\ast P ∗ -Donsker. Therefore, there exists a version of Gaussian process W \mathbb{W} W in S \mathcal{S} S with bounded and uniformly continuous sample path. This version extends continuously to the L 2 ( P ∗ ) L^2(P_\ast) L 2 ( P ∗ ) -closure S ‾ \overline{\mathcal{S}} S . Since { h ∈ T k : ∥ h ∥ ∗ = 1 } ⊆ S ‾ \{h \in\mathcal{T}_k : \lVert h \rVert_\ast= 1\} \subseteq\overline{\mathcal{S}} { h ∈ T k : ∥ h ∥ ∗ = 1 } ⊆ S , we obtain
B : = sup h ∈ T k ∥ h ∥ ∗ = 1 ∣ W ( h ) ∣ < ∞ almost surely. B := \sup_{\substack{h \in\mathcal{T}_k\\ \lVert h \rVert_\ast= 1}} |\mathbb{W}(h)| < \infty
\qquad\text{almost surely.} B := h ∈ T k ∥ h ∥ ∗ = 1 sup ∣ W ( h ) ∣ < ∞ almost surely.
Then ∣ W ( h ) ∣ ≤ B ∥ h ∥ ∗ |\mathbb{W}(h)| \le B\lVert h \rVert_\ast ∣ W ( h ) ∣ ≤ B ∥ h ∥ ∗ for every h ∈ T k h \in\mathcal{T}_k h ∈ T k . Since Q ( h ) = W ( h ) − ∥ h ∥ ∗ 2 / 2 ≤ B ∥ h ∥ ∗ − ∥ h ∥ ∗ 2 / 2 → − ∞ Q(h) = \mathbb{W}(h) - \lVert h \rVert_\ast^2/2 \le B\lVert h \rVert_\ast- \lVert h \rVert_\ast^2/2 \to-\infty Q ( h ) = W ( h ) − ∥ h ∥ ∗ 2 /2 ≤ B ∥ h ∥ ∗ − ∥ h ∥ ∗ 2 /2 → − ∞ as ∥ h ∥ ∗ → ∞ \lVert h \rVert_\ast\to\infty ∥ h ∥ ∗ → ∞ , the supremum of Q ( h ) Q(h) Q ( h ) is attained.
Step 2: Uniqueness. For any two almost surely measurable maximizer h 1 h_1 h 1 and h 2 h_2 h 2 of Q Q Q over T k \mathcal{T}_k T k , we show that h 1 = h 2 h_1 = h_2 h 1 = h 2 almost surely. Let ( e r ) r ≥ 1 (e_r)_{r \ge1} ( e r ) r ≥ 1 be an orthonormal basis of H = span ( T k ) \mathcal{H} = \operatorname{span}(\mathcal{T}_k) H = span ( T k ) , put Z r = W ( e r ) Z_r = \mathbb{W}(e_r) Z r = W ( e r ) , and set W − r ( h ) = W ( h ) − Z r ⟨ h , e r ⟩ ∗ \mathbb{W}_{-r}(h) = \mathbb{W}(h) - Z_r\langle h,e_r\rangle_\ast W − r ( h ) = W ( h ) − Z r ⟨ h , e r ⟩ ∗ . For fixed r r r , Z r ∼ N ( 0 , 1 ) Z_r \sim N(0,1) Z r ∼ N ( 0 , 1 ) is independent of the separable remainder process W − r \mathbb{W}_{-r} W − r . Conditional on this remainder, the function
V r ( z ) = sup h ∈ T k { z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 1 2 ∥ h ∥ ∗ 2 } . V_r(z) = \sup_{h \in\mathcal{T}_k} \left\{z\langle h,e_r\rangle_\ast+ \mathbb{W}_{-r}(h) - \frac{1}{2}\lVert h\rVert_\ast^2\right\}. V r ( z ) = h ∈ T k sup { z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 2 1 ∥ h ∥ ∗ 2 } .
Joint Gaussianity and orthogonality imply that Z r ∼ N ( 0 , 1 ) Z_r \sim N(0,1) Z r ∼ N ( 0 , 1 ) is independent of the entire process W − r \mathbb{W}_{-r} W − r . Fix a realization on which ∣ W ( h ) ∣ ≤ B ∥ h ∥ ∗ |\mathbb{W}(h)| \le B\lVert h\rVert_\ast ∣ W ( h ) ∣ ≤ B ∥ h ∥ ∗ for every h ∈ T k h \in\mathcal{T}_k h ∈ T k . For z ∈ R z \in\mathbb{R} z ∈ R , define
V r ( z ) = sup h ∈ T k { z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 1 2 ∥ h ∥ ∗ 2 } . V_r(z) = \sup_{h \in\mathcal{T}_k} \left\{z\langle h,e_r\rangle_\ast+ \mathbb{W}_{-r}(h) - \frac{1}{2}\lVert h\rVert_\ast^2\right\}. V r ( z ) = h ∈ T k sup { z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 2 1 ∥ h ∥ ∗ 2 } .
For each fixed h h h , the expression inside the supremum is affine in z z z . Hence V r V_r V r is convex. Moreover,
z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 1 2 ∥ h ∥ ∗ 2 ≤ ∣ z − Z r ∣ ∣ ⟨ h , e r ⟩ ∗ ∣ + B ∥ h ∥ ∗ − 1 2 ∥ h ∥ ∗ 2 ≤ ( ∣ z − Z r ∣ + B ) ∥ h ∥ ∗ − 1 2 ∥ h ∥ ∗ 2 → − ∞ \begin{aligned}
z\langle h,e_r\rangle_\ast+ \mathbb{W}_{-r}(h) - \frac{1}{2}\lVert h\rVert_\ast^2
\le|z-Z_r|\,|\langle h,e_r\rangle_\ast| + B\lVert h\rVert_\ast- \frac{1}{2}\lVert h\rVert_\ast^2 \\
&\le(|z-Z_r|+B)\lVert h\rVert_\ast- \frac{1}{2}\lVert h\rVert_\ast^2 \to-\infty
\end{aligned} z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 2 1 ∥ h ∥ ∗ 2 ≤ ∣ z − Z r ∣ ∣ ⟨ h , e r ⟩ ∗ ∣ + B ∥ h ∥ ∗ − 2 1 ∥ h ∥ ∗ 2 ≤ ( ∣ z − Z r ∣ + B ) ∥ h ∥ ∗ − 2 1 ∥ h ∥ ∗ 2 → − ∞
as ∥ h ∥ ∗ → ∞ \lVert h\rVert_\ast\to\infty ∥ h ∥ ∗ → ∞ . Since bounded tangent sections are compact and the criterion is continuous on these sections, V r ( z ) V_r(z) V r ( z ) is finite and its supremum is attained.
A finite convex function on R \mathbb{R} R is differentiable outside a Lebesgue-null set. At a differentiability point z z z , suppose that h 1 h_1 h 1 and h 2 h_2 h 2 both attain the supremum. Their slopes a j = ⟨ h j , e r ⟩ ∗ a_j = \langle h_j,e_r\rangle_\ast a j = ⟨ h j , e r ⟩ ∗ , for j = 1 , 2 j = 1,2 j = 1 , 2 , belong to the subdifferential ∂ V r ( z ) \partial V_r(z) ∂ V r ( z ) , because the corresponding affine functions support V r V_r V r at z z z . Differentiability implies ∂ V r ( z ) = { V r ′ ( z ) } \partial V_r(z) = \{V_r'(z)\} ∂ V r ( z ) = { V r ′ ( z )} , so a 1 = a 2 a_1 = a_2 a 1 = a 2 . Thus, at every differentiability point, all maximizers have the same e r e_r e r coordinate.
The exceptional nondifferentiability set may depend on the realization of W − r \mathbb{W}_{-r} W − r , but has Lebesgue measure zero for almost every such realization. Conditional on W − r \mathbb{W}_{-r} W − r , the independent variable Z r Z_r Z r has a standard normal density. Consequently,
P { Z r belongs to the exceptional set ∣ W − r } = 0 almost surely . \mathbb{P}\{Z_r\text{ belongs to the exceptional set}\mid W_{-r}\}=0 \qquad\text{almost surely}. P { Z r belongs to the exceptional set ∣ W − r } = 0 almost surely .
Taking expectations shows that, with probability one, all maximizers h h h of the original random criterion have the same coordinate ⟨ h , e r ⟩ ∗ \langle h,e_r\rangle_* ⟨ h , e r ⟩ ∗ . Taking the countable intersection of these probability-one events over the orthonormal basis, we have ⟨ h 1 − h 2 , e r ⟩ ∗ = 0 \langle h_1-h_2,e_r\rangle_*=0 ⟨ h 1 − h 2 , e r ⟩ ∗ = 0 for every r r r . Therefore h 1 = h 2 h_1=h_2 h 1 = h 2 in H \mathcal{H} H .
Step 4: Restricted problem. Lastly, we prove the existence and uniqueness for the solution of Q Q Q in T k , M \mathcal{T}_{k,M} T k , M . Existence follows from compactness of T k , M \mathcal{T}_{k,M} T k , M and almost sure continuity of Q Q Q on this set.
For uniqueness, we use the same argument as in Step 2. Indeed, we use the same orthonormal basis ( e r ) r ≥ 1 (e_r)_{r\geq1} ( e r ) r ≥ 1 and decomposition
W ( h ) = Z r ⟨ h , e r ⟩ ∗ + W − r ( h ) W(h)=Z_r\langle h,e_r\rangle_*+W_{-r}(h) W ( h ) = Z r ⟨ h , e r ⟩ ∗ + W − r ( h )
as above. Conditional on W − r W_{-r} W − r , define
V r , M ( z ) : = sup h ∈ T k , M { z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 1 2 ∥ h ∥ ∗ 2 } . V_{r,M}(z):=\sup_{h\in\mathcal{T}_{k,M}}\left\{z\langle h,e_r\rangle_*+W_{-r}(h)-\frac{1}{2}\lVert h\rVert_*^2\right\}. V r , M ( z ) := h ∈ T k , M sup { z ⟨ h , e r ⟩ ∗ + W − r ( h ) − 2 1 ∥ h ∥ ∗ 2 } .
This function is finite and convex on R \mathbb{R} R , and the supremum is attained. At each differentiability point of V r , M V_{r,M} V r , M , all maximizing points have the same e r e_r e r coordinate, because their coordinates belong to the subdifferential of V r , M V_{r,M} V r , M . Since Z r Z_r Z r is independent of W − r W_{-r} W − r and has a density, it almost surely avoids the Lebesgue-null set of nondifferentiability points. Taking a countable intersection over r r r shows that all maximizers have identical coordinates in the orthonormal basis, and hence coincide.
Proof of Theorem 3.4. We have established the convergence rate (10) of ρ ( G ^ n ) \rho(\widehat{G}_n) ρ ( G n ) in Lemma A.6 . For the weak limit of scaled relative density discrepancy h ^ n \widehat{h}_n h n , we divide our proof into following steps.
Step 1. For M > 0 M>0 M > 0 , define
H n , M = { h n , G : = n p G − p ∗ p ∗ : G ∈ O k ( Θ ) , n ρ ( G ) ≤ M } . \mathcal{H}_{n,M}=\left\{h_{n,G}:=\sqrt{n}\frac{p_G-p_*}{p_*}:G\in\mathcal{O}_k(\Theta),\ \sqrt{n}\,\rho(G)\leq M\right\}. H n , M = { h n , G := n p ∗ p G − p ∗ : G ∈ O k ( Θ ) , n ρ ( G ) ≤ M } .
We can verify that H n , M \mathcal{H}_{n,M} H n , M is a compact subset of L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . Using Lemma E.1, we show that H n , M \mathcal{H}_{n,M} H n , M converges in Hausdorff distance to T k , M \mathcal{T}_{k,M} T k , M in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) , defined in Lemma B.1. Let S ‾ \overline{\mathcal{S}} S denote the closure of S \mathcal{S} S in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . By Lemma A.5 (ii), S \mathcal{S} S is totally bounded, so S ‾ \overline{\mathcal{S}} S is compact. Define
K M = { a s : 0 ≤ a ≤ M , s ∈ S ‾ } . K_M=\{as:0\leq a\leq M,\ s\in\overline{\mathcal{S}}\}. K M = { a s : 0 ≤ a ≤ M , s ∈ S } .
The map ( a , s ) ↦ a s (a,s)\mapsto as ( a , s ) ↦ a s is continuous, so K M K_M K M is compact. In addition, for n n n sufficiently large, both H n , M \mathcal{H}_{n,M} H n , M and T k , M \mathcal{T}_{k,M} T k , M are contained in K M K_M K M .
Suppose that h n j ∈ H n j , M h_{n_j}\in\mathcal{H}_{n_j,M} h n j ∈ H n j , M and h n j → h h_{n_j}\to h h n j → h in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . There exist G n j ∈ O k ( Θ ) G_{n_j}\in\mathcal{O}_k(\Theta) G n j ∈ O k ( Θ ) such that
h n j = p G n j − p ∗ t n j p ∗ , t n j = n j − 1 / 2 . h_{n_j}=\frac{p_{G_{n_j}}-p_*}{t_{n_j}p_*},\qquad t_{n_j}=n_j^{-1/2}. h n j = t n j p ∗ p G n j − p ∗ , t n j = n j − 1/2 .
Proposition 3.1 gives h ∈ T k h\in\mathcal{T}_k h ∈ T k . Norm continuity also gives ∥ h ∥ ∗ ≤ M \lVert h\rVert_*\leq M ∥ h ∥ ∗ ≤ M , hence h ∈ T k , M h\in\mathcal{T}_{k,M} h ∈ T k , M .
Since T k \mathcal{T}_k T k is a cone, the subset { h ∈ T k : ∥ h ∥ ∗ < M } \{h\in\mathcal{T}_k:\lVert h\rVert_*<M\} { h ∈ T k : ∥ h ∥ ∗ < M } is dense in T k , M \mathcal{T}_{k,M} T k , M : for every h ∈ T k , M h\in\mathcal{T}_{k,M} h ∈ T k , M , the points ( 1 − j − 1 ) h (1-j^{-1})h ( 1 − j − 1 ) h belong to this subset and converge to h h h . Fix h ∈ T k , M h \in T_{k,M} h ∈ T k , M with ∥ h ∥ ∗ < M \lVert h\rVert_* < M ∥ h ∥ ∗ < M . The recovery construction in Proposition 3.1 provides mixing measures G t G_t G t for all sufficiently small t > 0 t>0 t > 0 such that ( p G t − p ∗ ) / ( t p ∗ ) → h (p_{G_t}-p_*)/(t p_*) \to h ( p G t − p ∗ ) / ( t p ∗ ) → h in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . For n n n sufficiently large, taking t = t n = n − 1 / 2 t=t_n=n^{-1/2} t = t n = n − 1/2 and setting
h n = n p G t n − p ∗ p ∗ h_n=\sqrt{n}\frac{p_{G_{t_n}}-p_*}{p_*} h n = n p ∗ p G t n − p ∗
gives h n → h h_n \to h h n → h . Since ∥ h ∥ ∗ < M \lVert h\rVert_*<M ∥ h ∥ ∗ < M , we have h n ∈ H n , M h_n \in\mathcal{H}_{n,M} h n ∈ H n , M for all sufficiently large n n n . Combining all the above results with Lemma E.1, we have d H ( H n , M , T k , M ) → 0 d_H(\mathcal{H}_{n,M},T_{k,M})\to0 d H ( H n , M , T k , M ) → 0 , where the closures and Hausdorff distance are taken in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) .
Step 2. Fix M > 0 M>0 M > 0 . On the common compact index set K M K_M K M , define
Z n ( h ) : = G n h − 1 2 ∥ h ∥ ∗ 2 , h ∈ K M . Z_n(h):=\mathbb{G}_n h-\frac{1}{2}\lVert h\rVert_*^2,\qquad h\in K_M. Z n ( h ) := G n h − 2 1 ∥ h ∥ ∗ 2 , h ∈ K M .
The Donsker property in Lemma A.5 extends to K M K_M K M by taking the closure of S \mathcal{S} S and then allowing bounded scalar multiplication. Consequently, Z n ⇝ Q Z_n\rightsquigarrow Q Z n ⇝ Q in ℓ ∞ ( K M ) \ell^\infty(K_M) ℓ ∞ ( K M ) , where Q ( h ) = W ( h ) − 1 2 ∥ h ∥ ∗ 2 Q(h)=W(h)-\frac{1}{2}\lVert h\rVert_*^2 Q ( h ) = W ( h ) − 2 1 ∥ h ∥ ∗ 2 has almost surely continuous sample paths under the L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) metric.
By Lemma B.2 , the local likelihood criterion satisfies Q n ( h ) = Z n ( h ) + r n ( h ) Q_n(h)=Z_n(h)+r_n(h) Q n ( h ) = Z n ( h ) + r n ( h ) for h ∈ H n , M h\in\mathcal{H}_{n,M} h ∈ H n , M , where Δ n , M : = sup h ∈ H n , M ∣ r n ( h ) ∣ = o P ( 1 ) \Delta_{n,M}:=\sup_{h\in\mathcal{H}_{n,M}}|r_n(h)|=o_P(1) Δ n , M := sup h ∈ H n , M ∣ r n ( h ) ∣ = o P ( 1 ) . Let h ^ n , M \widehat{h}_{n,M} h n , M be a measurable maximizer of Q n Q_n Q n over H n , M \mathcal{H}_{n,M} H n , M . Then
Z n ( h ^ n , M ) ≥ Q n ( h ^ n , M ) − Δ n , M = sup h ∈ H n , M Q n ( h ) − Δ n , M ≥ sup h ∈ H n , M Z n ( h ) − 2 Δ n , M . Z_n(\widehat{h}_{n,M})\geq Q_n(\widehat{h}_{n,M})-\Delta_{n,M}
=\sup_{h\in\mathcal{H}_{n,M}}Q_n(h)-\Delta_{n,M}
\geq\sup_{h\in\mathcal{H}_{n,M}}Z_n(h)-2\Delta_{n,M}. Z n ( h n , M ) ≥ Q n ( h n , M ) − Δ n , M = h ∈ H n , M sup Q n ( h ) − Δ n , M ≥ h ∈ H n , M sup Z n ( h ) − 2 Δ n , M .
In addition, by Step 1, d H ( H n , M , T k , M ) → 0 d_H(\mathcal{H}_{n,M},T_{k,M})\to0 d H ( H n , M , T k , M ) → 0 , and both sets lie in K M K_M K M for all sufficiently large n n n . The existence and uniqueness argument in Lemma B.3 applies also to the compact set T k , M T_{k,M} T k , M , so
H M † = arg max h ∈ T k , M Q ( h ) H_M^\dagger=\arg\max_{h\in T_{k,M}}Q(h) H M † = arg h ∈ T k , M max Q ( h )
exists and is almost surely unique. Theorem 1 of Cox [3 ] yields
h ^ n , M ⇝ H M † in L 2 ( P ∗ ) . \widehat{h}_{n,M}\rightsquigarrow H_M^\dagger
\quad\text{in }L^2(P_*). h n , M ⇝ H M † in L 2 ( P ∗ ) .
Step 3. Lastly, we prove that h ^ n ⇝ H † \widehat{h}_n\rightsquigarrow H^\dagger h n ⇝ H † . It is possible to choose the restricted maximizer h ^ n , M \widehat{h}_{n,M} h n , M to agree with h ^ n \widehat{h}_n h n whenever ∥ h ^ n ∥ ∗ ≤ M \lVert\widehat{h}_n\rVert_*\leq M ∥ h n ∥ ∗ ≤ M as h ^ n \widehat{h}_n h n is still a maximizer for restrained problem in H n , M \mathcal{H}_{n,M} H n , M . Hence
P { h ^ n , M ≠ h ^ n } ≤ P { ∥ h ^ n ∥ ∗ > M } . P\{\widehat{h}_{n,M}\ne\widehat{h}_n\}\leq P\{\lVert\widehat{h}_n\rVert_*>M\}. P { h n , M = h n } ≤ P {∥ h n ∥ ∗ > M } .
Similarly, almost sure uniqueness of the global maximizer implies H M † = H † H_M^\dagger=H^\dagger H M † = H † on { ∥ H † ∥ ∗ ≤ M } \{\lVert H^\dagger\rVert_*\leq M\} {∥ H † ∥ ∗ ≤ M } .
Let f : L 2 ( P ∗ ) → R f:L^2(P_*)\to\mathbb{R} f : L 2 ( P ∗ ) → R be bounded and Lipschitz, with ∥ f ∥ ∞ ≤ 1 \lVert f\rVert_\infty\leq1 ∥ f ∥ ∞ ≤ 1 . The preceding observations give
∣ E f ( h ^ n ) − E f ( H † ) ∣ ≤ 2 P { ∥ h ^ n ∥ ∗ > M } + ∣ E f ( h ^ n , M ) − E f ( H M † ) ∣ + 2 P { ∥ H † ∥ ∗ > M } . \left|Ef(\widehat{h}_n)-Ef(H^\dagger)\right|
\leq2P\{\lVert\widehat{h}_n\rVert_*>M\}
+\left|Ef(\widehat{h}_{n,M})-Ef(H_M^\dagger)\right|
+2P\{\lVert H^\dagger\rVert_*>M\}. E f ( h n ) − E f ( H † ) ≤ 2 P {∥ h n ∥ ∗ > M } + E f ( h n , M ) − E f ( H M † ) + 2 P {∥ H † ∥ ∗ > M } .
For each fixed M M M , the middle term converges to zero as n → ∞ n\to\infty n → ∞ by Step 2. Moreover, Lemma A.6 gives ∥ h ^ n ∥ ∗ = O P ( 1 ) \lVert\widehat{h}_n\rVert_*=O_P(1) ∥ h n ∥ ∗ = O P ( 1 ) , and Lemma B.3 gives ∥ H † ∥ ∗ < ∞ \lVert H^\dagger\rVert_*<\infty ∥ H † ∥ ∗ < ∞ almost surely. Therefore, taking n → ∞ n\to\infty n → ∞ and then M → ∞ M\to\infty M → ∞ yields E f ( h ^ n ) → E f ( H † ) Ef(\widehat{h}_n)\to Ef(H^\dagger) E f ( h n ) → E f ( H † ) . Since bounded Lipschitz functions determine weak convergence on L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) , we conclude that
h ^ n ⇝ H † in L 2 ( P ∗ ) . \widehat{h}_n\rightsquigarrow H^\dagger
\quad\text{in }L^2(P_*). h n ⇝ H † in L 2 ( P ∗ ) .
This completes the proof.
Proofs for Section 4
Sequential expansion and continuity of the transportation functional
In this subsection, we prove the sequential first-order expansion in Proposition 4.1 and the continuity of Ω \Omega Ω . Lemma C.1 separates the squared Wasserstein cost into local collapse and residual mass transport, while Lemma C.2 establishes stability of the finite transportation problem governing the residual cost. Together, these results yield the sequential expansion. We then prove continuity of Ω \Omega Ω in Lemma C.4 , providing the additional ingredient needed to combine this expansion with the density weak limit.
Lemma C.1 (Local collapse and first-order squared transport). Let t m ↓ 0 t_m \downarrow0 t m ↓ 0 and G m ∈ O k ( Θ ) G_m \in\mathcal{O}_k(\Theta) G m ∈ O k ( Θ ) satisfy
p G m − p ∗ t m p ∗ ⟶ h γ ∈ T k in L 2 ( P ∗ ) . \frac{p_{G_m}-p_{\ast}}{t_m p_{\ast}} \longrightarrow h_{\gamma} \in\mathcal{T}_k \quad\text{in } L^2(P_{\ast}). t m p ∗ p G m − p ∗ ⟶ h γ ∈ T k in L 2 ( P ∗ ) . On a classified subsequence as in proof of Proposition 3.1 , write L i L_i L i for labels approaching θ i ∗ \theta_i^{\ast} θ i ∗ and L 0 L_0 L 0 for the other labels, and set
Δ j , m = θ j , m − θ i ∗ ( j ∈ L i ) , A m = ∑ i = 1 p ∑ j ∈ L i π j , m ∥ Δ j , m ∥ 2 . \Delta_{j,m}=\theta_{j,m}-\theta_i^{\ast}\quad(j\in L_i), \qquad A_m=\sum_{i=1}^{p}\sum_{j\in L_i}\pi_{j,m}\lVert\Delta_{j,m}\rVert^2. Δ j , m = θ j , m − θ i ∗ ( j ∈ L i ) , A m = i = 1 ∑ p j ∈ L i ∑ π j , m ∥ Δ j , m ∥ 2 . Let M i , m = ∑ j ∈ L i π j , m M_{i,m}=\sum_{j\in L_i}\pi_{j,m} M i , m = ∑ j ∈ L i π j , m and define the collapsed measure
G ‾ m = ∑ i = 1 p M i , m δ θ i ∗ + ∑ j ∈ L 0 π j , m δ θ j , m . \overline{G}_m=\sum_{i=1}^{p}M_{i,m}\delta_{\theta_i^{\ast}}+\sum_{j\in L_0}\pi_{j,m}\delta_{\theta_{j,m}}. G m = i = 1 ∑ p M i , m δ θ i ∗ + j ∈ L 0 ∑ π j , m δ θ j , m . Then
A m / t m ⟶ ∑ i = 1 p tr ( C i ) , (70) A_m/t_m \longrightarrow\sum_{i=1}^{p}\operatorname{tr}(C_i), \tag*{(70)} A m / t m ⟶ i = 1 ∑ p tr ( C i ) , (70) W 2 2 ( G m , G ∗ ) = A m + W 2 2 ( G ‾ m , G ∗ ) + o ( t m ) . (71) W_2^2(G_m,G_{\ast})=A_m+W_2^2(\overline{G}_m,G_{\ast})+o(t_m). \tag*{(71)} W 2 2 ( G m , G ∗ ) = A m + W 2 2 ( G m , G ∗ ) + o ( t m ) . (71)
Proof. The trace limit follows immediately from the local second moments in the of necessity part, proof of Proposition 3.1 :
1 t m ∑ j ∈ L i π j , m Δ j , m Δ j , m T ⟶ C i . \frac{1}{t_m}\sum_{j\in L_i}\pi_{j,m}\Delta_{j,m}\Delta_{j,m}^{\mathsf{T}}\longrightarrow C_i. t m 1 j ∈ L i ∑ π j , m Δ j , m Δ j , m T ⟶ C i .
Taking traces and summing over i i i gives (70) . It remains to estimate W 2 2 ( G m , G ∗ ) W_2^2(G_m,G_{\ast}) W 2 2 ( G m , G ∗ ) through W 2 2 ( G ‾ m , G ∗ ) W_2^2(\overline{G}_m,G_{\ast}) W 2 2 ( G m , G ∗ ) and A m A_m A m .
Step 1. We prove that W 2 2 ( G m , G ∗ ) = O ( t m ) W_2^2(G_m,G_{\ast})=O(t_m) W 2 2 ( G m , G ∗ ) = O ( t m ) . Using the same argument as in Step 1, Part 1 in the proof of Proposition 3.1 , we have ρ ( G m ) = O ( t m ) \rho(G_m)=O(t_m) ρ ( G m ) = O ( t m ) and G m G_m G m belongs to the neighborhood U \mathcal{U} U defined in Lemma A.3 such that we can apply the Lemma A.4 to achieve W 2 2 ( G m , G ∗ ) = O ( t m ) W_2^2(G_m,G_{\ast})=O(t_m) W 2 2 ( G m , G ∗ ) = O ( t m ) .
Step 2. We first relate transportation plans from G m G_m G m to G ∗ G_{\ast} G ∗ to those from G ‾ m \overline{G}_m G m to G ∗ G_{\ast} G ∗ , and conversely. Comparing their costs then allows us to estimate W 2 2 ( G m , G ∗ ) W_2^2(G_m,G_{\ast}) W 2 2 ( G m , G ∗ ) in terms of the local collapse cost A m A_m A m and the residual transportation cost W 2 2 ( G ‾ m , G ∗ ) W_2^2(\overline{G}_m,G_*) W 2 2 ( G m , G ∗ ) in the next step. Let Γ = ( γ j r ) j , r \Gamma=(\gamma_{jr})_{j,r} Γ = ( γ j r ) j , r be a coupling from G m G_m G m to G ∗ G_* G ∗ . We can check that the transportation plan Γ \Gamma Γ induces a transportation plan Γ ‾ = ( γ ‾ i r ) i , r \overline{\Gamma}=(\overline{\gamma}_{ir})_{i,r} Γ = ( γ i r ) i , r between G ‾ m \overline{G}_m G m and G G G as
γ ‾ i , r = ∑ j ∈ L i γ j r . \overline{\gamma}_{i,r}=\sum_{j\in L_i}\gamma_{jr}. γ i , r = j ∈ L i ∑ γ j r .
Conversely, let Γ ‾ = ( γ ‾ i r ) i , r \overline{\Gamma}=(\overline{\gamma}_{ir})_{i,r} Γ = ( γ i r ) i , r be a coupling between G ‾ m \overline{G}_m G m and G G G . We construct a transportation plan Γ = ( γ j r ) j , r \Gamma=(\gamma_{jr})_{j,r} Γ = ( γ j r ) j , r between G m G_m G m to G ∗ G_* G ∗ satisfying relation γ ‾ i , r = ∑ j ∈ L i γ j r \overline{\gamma}_{i,r}=\sum_{j\in L_i}\gamma_{jr} γ i , r = ∑ j ∈ L i γ j r . In fact, this transportation plan can be constructed as
γ j r = { γ ‾ j r , if j ∈ L 0 , π j , m ∑ k ∈ L i π k , m γ ‾ i r , if j ∈ L i and i ≠ 0. \gamma_{jr}=
\begin{cases}
\overline{\gamma}_{jr}, & \text{if }j\in L_0,\\
\frac{\pi_{j,m}}{\sum_{k\in L_i}\pi_{k,m}}\overline{\gamma}_{ir}, & \text{if }j\in L_i\text{ and }i\ne0.
\end{cases} γ j r = { γ j r , ∑ k ∈ L i π k , m π j , m γ i r , if j ∈ L 0 , if j ∈ L i and i = 0.
and we can check the condition (75) for this transportation plan. Henceforth, unless otherwise stated, given a transportation plan Γ \Gamma Γ from G m G_m G m to G ∗ G_* G ∗ , we write Γ ‾ \overline{\Gamma} Γ for the transportation plan from G ‾ m \overline{G}_m G m to G G G constructed via (75). Conversely, given a transportation plan Γ ‾ \overline{\Gamma} Γ from G ‾ m \overline{G}_m G m to G G G , we write Γ \Gamma Γ for the transportation plan from G m G_m G m to G ∗ G_* G ∗ constructed via (76). We also write Π Γ ( G m , G ∗ ) \Pi_{\Gamma}(G_m,G_*) Π Γ ( G m , G ∗ ) and Π Γ ‾ ( G ‾ m , G ∗ ) \Pi_{\overline{\Gamma}}(\overline{G}_m,G_*) Π Γ ( G m , G ∗ ) be the total squared costs corresponding to transportation plan Γ \Gamma Γ and Γ ‾ \overline{\Gamma} Γ as
Π Γ ( G m , G ∗ ) = ∑ j , r γ j r ∥ θ j , m − θ r ∗ ∥ 2 , Π Γ ‾ ( G ‾ m , G ∗ ) = ∑ i , r γ ‾ i r ∥ θ i ∗ − θ r ∗ ∥ 2 + ∑ i ∈ L 0 ∑ r γ ‾ i r ∥ θ i , m − θ r ∗ ∥ 2 . \Pi_{\Gamma}(G_m,G_*)=\sum_{j,r}\gamma_{jr}\lVert\theta_{j,m}-\theta_r^*\rVert^2,\qquad
\Pi_{\overline{\Gamma}}(\overline{G}_m,G_*)=\sum_{i,r}\overline{\gamma}_{ir}\lVert\theta_i^*-\theta_r^*\rVert^2+\sum_{i\in L_0}\sum_r\overline{\gamma}_{ir}\lVert\theta_{i,m}-\theta_r^*\rVert^2. Π Γ ( G m , G ∗ ) = j , r ∑ γ j r ∥ θ j , m − θ r ∗ ∥ 2 , Π Γ ( G m , G ∗ ) = i , r ∑ γ i r ∥ θ i ∗ − θ r ∗ ∥ 2 + i ∈ L 0 ∑ r ∑ γ i r ∥ θ i , m − θ r ∗ ∥ 2 .
For j ∈ L i j\in L_i j ∈ L i , we consider the expansion
∥ θ j , m − θ r ∗ ∥ 2 = ∥ Δ j , m ∥ 2 + ∥ θ i ∗ − θ r ∗ ∥ 2 + 2 Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) . \lVert\theta_{j,m}-\theta_r^*\rVert^2=\lVert\Delta_{j,m}\rVert^2+\lVert\theta_i^*-\theta_r^*\rVert^2+2\Delta_{j,m}^{\top}(\theta_i^*-\theta_r^*). ∥ θ j , m − θ r ∗ ∥ 2 = ∥ Δ j , m ∥ 2 + ∥ θ i ∗ − θ r ∗ ∥ 2 + 2 Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) .
For a transportation plan Γ = ( γ j r ) j , r \Gamma=(\gamma_{jr})_{j,r} Γ = ( γ j r ) j , r , by multiplying with weight γ j r \gamma_{jr} γ j r , we have
Π Γ ( G m , G ∗ ) = ∑ i = 1 k ∑ j ∈ L i ∑ r γ j r ( ∥ Δ j , m ∥ 2 + ∥ θ i ∗ − θ r ∗ ∥ 2 + 2 Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) ) + ∑ j ∈ L 0 ∑ r γ j r ∥ θ j , m − θ r ∗ ∥ 2 = ( ∑ i = 1 k ∑ j ∈ L i ∑ r γ j r ∥ Δ j , m ∥ 2 ) ⏟ = A m + ( ∑ i = 1 k ∑ j ∈ L i ∑ r γ j r ∥ θ i ∗ − θ r ∗ ∥ 2 + ∑ j ∈ L 0 ∑ r γ j r ∥ θ j , m − θ r ∗ ∥ 2 ) ⏟ Π Γ ‾ ( G ‾ m , G ∗ ) + 2 ∑ j ∈ L i ∑ r γ j r Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) . \begin{aligned}
\Pi_{\Gamma}(G_m,G_*)&=\sum_{i=1}^{k}\sum_{j\in L_i}\sum_r\gamma_{jr}\left(\lVert\Delta_{j,m}\rVert^2+\lVert\theta_i^*-\theta_r^*\rVert^2+2\Delta_{j,m}^{\top}(\theta_i^*-\theta_r^*)\right)+\sum_{j\in L_0}\sum_r\gamma_{jr}\lVert\theta_{j,m}-\theta_r^*\rVert^2\\
&=\underbrace{\left(\sum_{i=1}^{k}\sum_{j\in L_i}\sum_r\gamma_{jr}\lVert\Delta_{j,m}\rVert^2\right)}_{=A_m}+\underbrace{\left(\sum_{i=1}^{k}\sum_{j\in L_i}\sum_r\gamma_{jr}\lVert\theta_i^*-\theta_r^*\rVert^2+\sum_{j\in L_0}\sum_r\gamma_{jr}\lVert\theta_{j,m}-\theta_r^*\rVert^2\right)}_{\Pi_{\overline{\Gamma}}(\overline{G}_m,G_*)}\\
&\quad+2\sum_{j\in L_i}\sum_r\gamma_{jr}\Delta_{j,m}^{\top}(\theta_i^*-\theta_r^*).
\end{aligned} Π Γ ( G m , G ∗ ) = i = 1 ∑ k j ∈ L i ∑ r ∑ γ j r ( ∥ Δ j , m ∥ 2 + ∥ θ i ∗ − θ r ∗ ∥ 2 + 2 Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) ) + j ∈ L 0 ∑ r ∑ γ j r ∥ θ j , m − θ r ∗ ∥ 2 = = A m i = 1 ∑ k j ∈ L i ∑ r ∑ γ j r ∥ Δ j , m ∥ 2 + Π Γ ( G m , G ∗ ) i = 1 ∑ k j ∈ L i ∑ r ∑ γ j r ∥ θ i ∗ − θ r ∗ ∥ 2 + j ∈ L 0 ∑ r ∑ γ j r ∥ θ j , m − θ r ∗ ∥ 2 + 2 j ∈ L i ∑ r ∑ γ j r Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) .
Therefore, we have
Π Γ ( G m , G ∗ ) = A m + Π Γ ‾ ( G ‾ m , G ∗ ) + 2 ∑ j ∈ L i ∑ r ≠ i γ j r Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) ⏟ = : I m . (72) \Pi_{\Gamma}(G_m,G_*)=A_m+\Pi_{\overline{\Gamma}}(\overline{G}_m,G_*)+2\underbrace{\sum_{j\in L_i}\sum_{r\ne i}\gamma_{jr}\Delta_{j,m}^{\top}(\theta_i^*-\theta_r^*)}_{=:I_m}. \tag*{(72)} Π Γ ( G m , G ∗ ) = A m + Π Γ ( G m , G ∗ ) + 2 =: I m j ∈ L i ∑ r = i ∑ γ j r Δ j , m ⊤ ( θ i ∗ − θ r ∗ ) . (72)
Step 3. We prove the estimation (71). Let Γ = ( γ j r ) j , r \Gamma=(\gamma_{jr})_{j,r} Γ = ( γ j r ) j , r be the optimal coupling between G m G_m G m and G ∗ G_* G ∗ . For s ∗ : = min i ≠ r ∥ θ i ∗ − θ r ∗ ∥ > 0 s_*:=\min_{i\ne r}\lVert\theta_i^*-\theta_r^*\rVert>0 s ∗ := min i = r ∥ θ i ∗ − θ r ∗ ∥ > 0 be the minimal separation among true center, since the local displacements vanish, for all large m m m , we have max i max j ∈ L i ∥ Δ j , m ∥ ≤ s ∗ / 4 \max_i\max_{j\in L_i}\lVert\Delta_{j,m}\rVert\le s_*/4 max i max j ∈ L i ∥ Δ j , m ∥ ≤ s ∗ /4 . Hence a local atom from cluster i i i transported to a different true atom θ r ∗ \theta_r^* θ r ∗ travels at least 3 s ∗ / 4 3s_*/4 3 s ∗ /4 . In the optimal plan Γ \Gamma Γ , we therefore achieve that γ j r = O ( t m ) \gamma_{jr}=O(t_m) γ j r = O ( t m ) for j ∈ L i j\in L_i j ∈ L i and i ≠ r i\ne r i = r . Consequently, we have the following estimation for I m I_m I m as
∣ I m ∣ ≤ ∑ j ∈ L i ∑ r ≠ i γ j r ⏟ O ( t m ) ∥ Δ j , m ∥ ⏟ o ( 1 ) ∥ θ i ∗ − θ r ∗ ∥ ⏟ ≤ Diam ( Θ ) = o ( t m ) . (73) |I_m|\le\sum_{j\in L_i}\sum_{r\ne i}\underbrace{\gamma_{jr}}_{O(t_m)}\underbrace{\|\Delta_{j,m}\|}_{o(1)}\underbrace{\|\theta_i^*-\theta_r^*\|}_{\le\operatorname{Diam}(\Theta)}=o(t_m). \tag*{(73)} ∣ I m ∣ ≤ j ∈ L i ∑ r = i ∑ O ( t m ) γ j r o ( 1 ) ∥ Δ j , m ∥ ≤ Diam ( Θ ) ∥ θ i ∗ − θ r ∗ ∥ = o ( t m ) . (73)
Thus, by plugging in the estimation (73) for I m I_m I m and the inequality Π Γ ‾ ( G ‾ m , G ∗ ) ≥ W 2 2 ( G ‾ m , G ∗ ) \Pi_{\overline{\Gamma}}(\overline{G}_m,G_*)\ge W_2^2(\overline{G}_m,G_*) Π Γ ( G m , G ∗ ) ≥ W 2 2 ( G m , G ∗ ) to (72) , we achieve
W 2 2 ( G m , G ∗ ) ≥ A m + W 2 2 ( G ‾ m , G ∗ ) + o ( t m ) . (74) W_2^2(G_m,G_*)\ge A_m+W_2^2(\overline{G}_m,G_*)+o(t_m). \tag*{(74)} W 2 2 ( G m , G ∗ ) ≥ A m + W 2 2 ( G m , G ∗ ) + o ( t m ) . (74)
For the inverse direction, let Γ ‾ = ( γ ‾ i r ) i , r \overline{\Gamma}=(\overline{\gamma}_{ir})_{i,r} Γ = ( γ i r ) i , r be the optimal transportation plan from G ‾ m \overline{G}_m G m to G ∗ G_* G ∗ , which corresponds to the transportation plan Γ \Gamma Γ from G m G_m G m to G ∗ G_* G ∗ . As W 2 2 ( G m , G ∗ ) = O ( t m ) W_2^2(G_m,G_*)=O(t_m) W 2 2 ( G m , G ∗ ) = O ( t m ) , from the estimation (74) and the fact that A m ≥ 0 A_m\ge0 A m ≥ 0 , we have W 2 2 ( G ‾ m , G ∗ ) = O ( t m ) W_2^2(\overline{G}_m,G_*)=O(t_m) W 2 2 ( G m , G ∗ ) = O ( t m ) . Therefore, the off-diagonal mass γ ‾ i r \overline{\gamma}_{ir} γ i r transported between distinct true atoms i , r i,r i , r is of order O ( t m ) O(t_m) O ( t m ) , since every such unit of mass costs at least s ∗ 2 s_*^2 s ∗ 2 . As a consequence, we also achieve the estimation ∣ I m ∣ = o ( t m ) |I_m|=o(t_m) ∣ I m ∣ = o ( t m ) in (73) . Moreover, Π Γ ‾ ( G m , G ∗ ) ≥ W 2 2 ( G m , G ∗ ) \Pi_{\overline{\Gamma}}(G_m,G_*)\ge W_2^2(G_m,G_*) Π Γ ( G m , G ∗ ) ≥ W 2 2 ( G m , G ∗ ) , we plug in these estimations above to (72) and achieve
W 2 2 ( G m , G ∗ ) ≤ A m + W 2 2 ( G ‾ m , G ∗ ) + o ( t m ) . W_2^2(G_m,G_*)\le A_m+W_2^2(\overline{G}_m,G_*)+o(t_m). W 2 2 ( G m , G ∗ ) ≤ A m + W 2 2 ( G m , G ∗ ) + o ( t m ) .
This completes our proof.
Lemma C.2 (Stability of a finite transportation problem). Fix distinct locations θ 1 ∗ , … , θ p ∗ \theta_1^*,\ldots,\theta_p^* θ 1 ∗ , … , θ p ∗ and a nonnegative integer q q q . Suppose α m → α \alpha_m\to\alpha α m → α in R p \mathbb{R}^p R p , c m → c c_m\to c c m → c in [ 0 , ∞ ) q [0,\infty)^q [ 0 , ∞ ) q , and η ℓ , m → η ℓ \eta_{\ell,m}\to\eta_\ell η ℓ , m → η ℓ in Θ \Theta Θ such that η ℓ ≠ θ i ∗ \eta_\ell\ne\theta_i^* η ℓ = θ i ∗ for 1 ≤ i ≤ p 1\le i\le p 1 ≤ i ≤ p and
∑ i α i , m + ∑ ℓ = 1 q c ℓ , m = 0. \sum_i\alpha_{i,m}+\sum_{\ell=1}^{q}c_{\ell,m}=0. i ∑ α i , m + ℓ = 1 ∑ q c ℓ , m = 0. For each m m m define V m V_m V m as the minimum of
Ψ m ( x , y ) = ∑ i ≠ j ∥ θ i ∗ − θ j ∗ ∥ 2 x i j + ∑ ℓ = 1 q ∑ i = 1 p ∥ η ℓ , m − θ i ∗ ∥ 2 y ℓ i \Psi_m(x,y)=\sum_{i\ne j}\|\theta_i^*-\theta_j^*\|^2x_{ij}+\sum_{\ell=1}^{q}\sum_{i=1}^{p}\|\eta_{\ell,m}-\theta_i^*\|^2y_{\ell i} Ψ m ( x , y ) = i = j ∑ ∥ θ i ∗ − θ j ∗ ∥ 2 x ij + ℓ = 1 ∑ q i = 1 ∑ p ∥ η ℓ , m − θ i ∗ ∥ 2 y ℓ i over x i j x_{ij} x ij and y ℓ i y_{\ell i} y ℓ i belong to the constraint set
P m = { x , y ≥ 0 , ∑ i y ℓ i = c ℓ , m , ∑ j ≠ i ( x i j − x j i ) = α i , m + ∑ ℓ y ℓ i } . (75) \mathcal{P}_m=\left\{x,y\ge0,\quad\sum_i y_{\ell i}=c_{\ell,m},\quad\sum_{j\ne i}(x_{ij}-x_{ji})=\alpha_{i,m}+\sum_\ell y_{\ell i}\right\}. \tag*{(75)} P m = ⎩ ⎨ ⎧ x , y ≥ 0 , i ∑ y ℓ i = c ℓ , m , j = i ∑ ( x ij − x j i ) = α i , m + ℓ ∑ y ℓ i ⎭ ⎬ ⎫ . (75) Let V V V denote the analogous minimum at the limiting masses and locations:
V = min ( x , y ) ∈ P Ψ ( x , y ) : = ∑ i ≠ j ∥ θ i ∗ − θ j ∗ ∥ 2 x i j + ∑ ℓ = 1 q ∑ i = 1 p ∥ η ℓ − θ i ∗ ∥ 2 y ℓ i V=\min_{(x,y)\in\mathcal{P}}\Psi(x,y):=\sum_{i\ne j}\|\theta_i^*-\theta_j^*\|^2x_{ij}+\sum_{\ell=1}^{q}\sum_{i=1}^{p}\|\eta_\ell-\theta_i^*\|^2y_{\ell i} V = ( x , y ) ∈ P min Ψ ( x , y ) := i = j ∑ ∥ θ i ∗ − θ j ∗ ∥ 2 x ij + ℓ = 1 ∑ q i = 1 ∑ p ∥ η ℓ − θ i ∗ ∥ 2 y ℓ i where
P = { x , y ≥ 0 , ∑ i y ℓ i = c ℓ , ∑ j ≠ i ( x i j − x j i ) = α i + ∑ ℓ y ℓ i } . \mathcal{P}=\left\{x,y\ge0,\quad\sum_i y_{\ell i}=c_\ell,\quad\sum_{j\ne i}(x_{ij}-x_{ji})=\alpha_i+\sum_\ell y_{\ell i}\right\}. P = ⎩ ⎨ ⎧ x , y ≥ 0 , i ∑ y ℓ i = c ℓ , j = i ∑ ( x ij − x j i ) = α i + ℓ ∑ y ℓ i ⎭ ⎬ ⎫ .
Then V m → V V_m \to V V m → V . This conclusion allows c ℓ = 0 c_\ell= 0 c ℓ = 0 and arbitrary changes in the active edges of the minimizing flows.
Proof. Step 1. First, we show that sup m V m < ∞ \sup_m V_m < \infty sup m V m < ∞ and there exists a constant M > 0 M > 0 M > 0 such that for m m m sufficiently large, there exists an optimal solution ( x m , y m ) (x^m,y^m) ( x m , y m ) of Ψ m \Psi_m Ψ m over P m \mathcal{P}_m P m such that ∥ x m ∥ ∞ , ∥ y m ∥ ∞ ≤ M \lVert x^m\rVert_\infty,\lVert y^m\rVert_\infty\le M ∥ x m ∥ ∞ , ∥ y m ∥ ∞ ≤ M . In fact, as θ 1 ∗ , … , θ p ∗ \theta_1^*,\ldots,\theta_p^* θ 1 ∗ , … , θ p ∗ are distinct and η ℓ ≠ θ i ∗ \eta_\ell\ne\theta_i^* η ℓ = θ i ∗ for all 1 ≤ i ≤ p 1 \le i \le p 1 ≤ i ≤ p , there exists a positive constant s ∗ s_* s ∗ such that for m m m sufficiently large,
∥ θ i ∗ − θ j ∗ ∥ ≥ s ∗ , ∥ η ℓ , m − θ i ∗ ∥ ≥ s ∗ . \lVert\theta_i^*-\theta_j^*\rVert\ge s_*, \qquad\lVert\eta_{\ell,m}-\theta_i^*\rVert\ge s_*. ∥ θ i ∗ − θ j ∗ ∥ ≥ s ∗ , ∥ η ℓ , m − θ i ∗ ∥ ≥ s ∗ .
Therefore, for m m m sufficiently large, Ψ m ( x , y ) ≥ s ∗ 2 ( ∑ i ≠ j x i j + ∑ ℓ = 1 q ∑ i = 1 p y ℓ i ) \Psi_m(x,y) \ge s_*^2\left(\sum_{i\ne j}x_{ij}+\sum_{\ell=1}^{q}\sum_{i=1}^{p}y_{\ell i}\right) Ψ m ( x , y ) ≥ s ∗ 2 ( ∑ i = j x ij + ∑ ℓ = 1 q ∑ i = 1 p y ℓ i ) . Meanwhile, we can see that as c ℓ , m → c c_{\ell,m}\to c c ℓ , m → c and α m → α \alpha_m\to\alpha α m → α and bounded, there exists a constant C C C such that P m ∩ { ∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } \mathcal{P}_m\cap\{\lVert x\rVert_\infty,\lVert y\rVert_\infty\le C\} P m ∩ {∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } and P ∩ { ∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } \mathcal{P}\cap\{\lVert x\rVert_\infty,\lVert y\rVert_\infty\le C\} P ∩ {∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } are non-empty. Moreover, Ψ m ≤ C p ( p + q − 1 ) ⋅ Diam ( Θ ) \Psi_m\le Cp(p+q-1)\cdot\operatorname{Diam}(\Theta) Ψ m ≤ C p ( p + q − 1 ) ⋅ Diam ( Θ ) in P m ∩ { ∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } \mathcal{P}_m\cap\{\lVert x\rVert_\infty,\lVert y\rVert_\infty\le C\} P m ∩ {∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } and Ψ ≤ C p ( p + q − 1 ) ⋅ Diam 2 ( Θ ) \Psi\le Cp(p+q-1)\cdot\operatorname{Diam}^{2}(\Theta) Ψ ≤ C p ( p + q − 1 ) ⋅ Diam 2 ( Θ ) in P ∩ { ∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } \mathcal{P}\cap\{\lVert x\rVert_\infty,\lVert y\rVert_\infty\le C\} P ∩ {∥ x ∥ ∞ , ∥ y ∥ ∞ ≤ C } . Combining these above observations, we have sup m V m ≤ C p ( p + q − 1 ) ⋅ Diam 2 ( Θ ) < ∞ \sup_m V_m\le Cp(p+q-1)\cdot\operatorname{Diam}^{2}(\Theta)<\infty sup m V m ≤ C p ( p + q − 1 ) ⋅ Diam 2 ( Θ ) < ∞ and there exists M > 0 M>0 M > 0 such that some optimal solution ( x m , y m ) (x^m,y^m) ( x m , y m ) of Ψ m \Psi_m Ψ m over P m \mathcal{P}_m P m satisfying ∥ x m ∥ ∞ , ∥ y m ∥ ∞ ≤ M \lVert x^m\rVert_\infty,\lVert y^m\rVert_\infty\le M ∥ x m ∥ ∞ , ∥ y m ∥ ∞ ≤ M .
Step 2. We prove that lim inf m V m ≥ V \liminf_m V_m\ge V lim inf m V m ≥ V . Consider a sequence { m j } j \{m_j\}_j { m j } j such that V m j → lim inf m V m V_{m_j}\to\liminf_m V_m V m j → lim inf m V m . By extracting a subsequence, we can suppose that the optimal solution ( x m j , y m j ) (x^{m_j},y^{m_j}) ( x m j , y m j ) constructed in Step 1 converges to ( x ∗ , y ∗ ) ∈ P (x^*,y^*)\in\mathcal{P} ( x ∗ , y ∗ ) ∈ P . Therefore,
lim inf m V m = lim j → ∞ V m j = lim j → ∞ Ψ m j ( x m j , y m j ) ≥ lim j → ∞ Ψ m j ( x ∗ , y ∗ ) = Ψ ( x ∗ , y ∗ ) ≥ V . \liminf_m V_m=\lim_{j\to\infty}V_{m_j}=\lim_{j\to\infty}\Psi_{m_j}(x^{m_j},y^{m_j})\ge\lim_{j\to\infty}\Psi_{m_j}(x^*,y^*)=\Psi(x^*,y^*)\ge V. m lim inf V m = j → ∞ lim V m j = j → ∞ lim Ψ m j ( x m j , y m j ) ≥ j → ∞ lim Ψ m j ( x ∗ , y ∗ ) = Ψ ( x ∗ , y ∗ ) ≥ V .
Step 3. We prove that lim sup m V m ≤ V \limsup_m V_m\le V lim sup m V m ≤ V . Fix an optimal solution ( x , y ) ∈ P (x,y)\in\mathcal{P} ( x , y ) ∈ P , we construct a sequence ( x ( m ) , y ( m ) ) ∈ P m (x^{(m)},y^{(m)})\in\mathcal{P}_m ( x ( m ) , y ( m ) ) ∈ P m based on ( x , y ) (x,y) ( x , y ) . For every ℓ = 1 , … , q \ell=1,\ldots,q ℓ = 1 , … , q and i = 1 , … , p i=1,\ldots,p i = 1 , … , p , define
y ℓ i ( m ) = { c ℓ , m c ℓ y ℓ i , c ℓ > 0 , c ℓ , m 1 { i = 1 } , c ℓ = 0. y_{\ell i}^{(m)}=
\begin{cases}
\dfrac{c_{\ell,m}}{c_\ell}y_{\ell i}, & c_\ell>0,\\
c_{\ell,m}\mathbf{1}_{\{i=1\}}, & c_\ell=0.
\end{cases} y ℓ i ( m ) = ⎩ ⎨ ⎧ c ℓ c ℓ , m y ℓ i , c ℓ , m 1 { i = 1 } , c ℓ > 0 , c ℓ = 0.
Then y ( m ) ≥ 0 y^{(m)}\ge0 y ( m ) ≥ 0 and ∑ i y ℓ i ( m ) = c ℓ , m \sum_i y_{\ell i}^{(m)}=c_{\ell,m} ∑ i y ℓ i ( m ) = c ℓ , m . Moreover, y ( m ) → y y^{(m)}\to y y ( m ) → y : when c ℓ = 0 c_\ell=0 c ℓ = 0 , nonnegativity and ∑ i y ℓ i = 0 \sum_i y_{\ell i}=0 ∑ i y ℓ i = 0 imply y ℓ i = 0 y_{\ell i}=0 y ℓ i = 0 for every i i i . For x ( m ) x^{(m)} x ( m ) , we define the balance discrepancies
δ i , m = α i , m + ∑ ℓ = 1 q y ℓ i ( m ) − ∑ j ≠ i ( x i j − x j i ) , i = 1 , … , p . \delta_{i,m}=\alpha_{i,m}+\sum_{\ell=1}^{q}y_{\ell i}^{(m)}-\sum_{j\ne i}(x_{ij}-x_{ji}),\qquad i=1,\ldots,p. δ i , m = α i , m + ℓ = 1 ∑ q y ℓ i ( m ) − j = i ∑ ( x ij − x j i ) , i = 1 , … , p .
Since ( x , y ) ∈ P (x,y)\in\mathcal{P} ( x , y ) ∈ P and y ( m ) → y y^{(m)}\to y y ( m ) → y , we have δ i , m → 0 \delta_{i,m}\to0 δ i , m → 0 for every i i i . Furthermore, mass conservation gives
∑ i = 1 p δ i , m = ∑ i = 1 p α i , m + ∑ ℓ = 1 q c ℓ , m = 0. \sum_{i=1}^{p}\delta_{i,m}=\sum_{i=1}^{p}\alpha_{i,m}+\sum_{\ell=1}^{q}c_{\ell,m}=0. i = 1 ∑ p δ i , m = i = 1 ∑ p α i , m + ℓ = 1 ∑ q c ℓ , m = 0.
Write δ i , m + = max { δ i , m , 0 } \delta_{i,m}^{+}=\max\{\delta_{i,m},0\} δ i , m + = max { δ i , m , 0 } and δ i , m − = max { − δ i , m , 0 } \delta_{i,m}^{-}=\max\{-\delta_{i,m},0\} δ i , m − = max { − δ i , m , 0 } , and put D m : = ∑ i = 1 p δ i , m + = ∑ i = 1 p δ i , m − = 1 2 ∑ i = 1 p ∣ δ i , m ∣ D_m:=\sum_{i=1}^{p}\delta_{i,m}^{+}=\sum_{i=1}^{p}\delta_{i,m}^{-}=\frac{1}{2}\sum_{i=1}^{p}|\delta_{i,m}| D m := ∑ i = 1 p δ i , m + = ∑ i = 1 p δ i , m − = 2 1 ∑ i = 1 p ∣ δ i , m ∣ . For i ≠ j i\ne j i = j , define the correcting flows explicitly by
z i j ( m ) = { δ i , m + δ j , m − D m , D m > 0 , 0 , D m = 0. z_{ij}^{(m)}=
\begin{cases}
\dfrac{\delta_{i,m}^{+}\delta_{j,m}^{-}}{D_m}, & D_m>0,\\
0, & D_m=0.
\end{cases} z ij ( m ) = ⎩ ⎨ ⎧ D m δ i , m + δ j , m − , 0 , D m > 0 , D m = 0.
These flows are nonnegative. If D m > 0 D_m > 0 D m > 0 , the identity δ i , m + δ i , m − = 0 \delta_{i,m}^{+}\delta_{i,m}^{-}=0 δ i , m + δ i , m − = 0 implies ∑ j ≠ i z i j ( m ) = δ i , m + \sum_{j\ne i} z_{ij}^{(m)}=\delta_{i,m}^{+} ∑ j = i z ij ( m ) = δ i , m + and ∑ j ≠ i z j i ( m ) = δ i , m − \sum_{j\ne i} z_{ji}^{(m)}=\delta_{i,m}^{-} ∑ j = i z j i ( m ) = δ i , m − . The same identities hold when D m = 0 D_m=0 D m = 0 , because all δ i , m \delta_{i,m} δ i , m then vanish. Consequently, ∑ j ≠ i ( z i j ( m ) − z j i ( m ) ) = δ i , m \sum_{j\ne i}(z_{ij}^{(m)}-z_{ji}^{(m)})=\delta_{i,m} ∑ j = i ( z ij ( m ) − z j i ( m ) ) = δ i , m . Define x i j ( m ) : = x i j + z i j ( m ) x_{ij}^{(m)}:=x_{ij}+z_{ij}^{(m)} x ij ( m ) := x ij + z ij ( m ) . Then
∑ j ≠ i ( x i j ( m ) − x j i ( m ) ) = α i , m + ∑ ℓ = 1 q y ℓ i ( m ) , \sum_{j\ne i}\left(x_{ij}^{(m)}-x_{ji}^{(m)}\right)=\alpha_{i,m}+\sum_{\ell=1}^{q}y_{\ell i}^{(m)}, j = i ∑ ( x ij ( m ) − x j i ( m ) ) = α i , m + ℓ = 1 ∑ q y ℓ i ( m ) ,
so ( x ( m ) , y ( m ) ) ∈ P m (x^{(m)},y^{(m)})\in\mathcal{P}_m ( x ( m ) , y ( m ) ) ∈ P m . Finally, ∑ i ≠ j z i j ( m ) = D m → 0 \sum_{i\ne j}z_{ij}^{(m)}=D_m\to0 ∑ i = j z ij ( m ) = D m → 0 , and hence x ( m ) → x x^{(m)}\to x x ( m ) → x . Together with y ( m ) → y y^{(m)}\to y y ( m ) → y and η ℓ , m → η ℓ \eta_{\ell,m}\to\eta_\ell η ℓ , m → η ℓ , this gives
Ψ m ( x ( m ) , y ( m ) ) ⟶ Ψ ( x , y ) = V . \Psi_m(x^{(m)},y^{(m)})\longrightarrow\Psi(x,y)=V. Ψ m ( x ( m ) , y ( m ) ) ⟶ Ψ ( x , y ) = V .
Since V m ≤ Ψ m ( x ( m ) , y ( m ) ) V_m\le\Psi_m(x^{(m)},y^{(m)}) V m ≤ Ψ m ( x ( m ) , y ( m ) ) , we conclude that lim sup m V m ≤ V \limsup_m V_m\le V lim sup m V m ≤ V . Combined with Step 2, this proves V m → V V_m\to V V m → V .
□ \square □
Proof of Proposition 4.1. We show that every subsequence { G m i } i \{G_{m_i}\}_i { G m i } i of { G m } m \{G_m\}_m { G m } m admits a further subsequence along which the convergence in (18) holds. For notational simplicity, we relabel the subsequence { G m i } i \{G_{m_i}\}_i { G m i } i as { G m } m \{G_m\}_m { G m } m .
Using Proposition 3.1, we extract a further subsequence of G m G_m G m on which all fitted labels have limiting locations and are classified into local sets L i L_i L i and a nonlocal set L 0 L_0 L 0 . For notational convenience, we also relabel this subsequence as G m G_m G m . Along this classified subsequence the scaled local moments converge to the canonical coefficients ( α , β , C ) (\alpha,\beta,C) ( α , β , C ) and the positive scaled nonlocal masses converge to the atoms of ν \nu ν , which contributes to the tangent configuration γ ‾ = ( α , β , C , ν ) \overline{\gamma}=(\alpha,\beta,C,\nu) γ = ( α , β , C , ν ) . By uniqueness of the canonical tangent representation, every classified subsequence gives the same γ ‾ = γ = ( α , β , C , ν ) \overline{\gamma}=\gamma=(\alpha,\beta,C,\nu) γ = γ = ( α , β , C , ν ) .
In this proof, we use the same notation as in Lemma C.1. Using this lemma, we reduce the transportation problem W 2 2 ( G m , G ∗ ) W_2^2(G_m,G_*) W 2 2 ( G m , G ∗ ) to the transportation problem W 2 2 ( G ‾ m , G ∗ ) W_2^2(\overline{G}_m,G_*) W 2 2 ( G m , G ∗ ) , which we study in the next step. In addition, consider the constant C 0 C_0 C 0 such that
W 2 2 ( G m , G ∗ ) ≤ C 0 t m , W 2 2 ( G ‾ m , G ∗ ) ≤ C 0 t m . W_2^2(G_m,G_*)\le C_0t_m,\qquad W_2^2(\overline{G}_m,G_*)\le C_0t_m. W 2 2 ( G m , G ∗ ) ≤ C 0 t m , W 2 2 ( G m , G ∗ ) ≤ C 0 t m .
*Step 1: Solution of the transportation problem between G ‾ m \overline{G}_m G m and G ∗ G_* G ∗ . We retain all nonlocal labels in the classified subsequence, while including labels whose scaled weights converge to zero. Denote these nonlocal labels as θ ℓ , m \theta_{\ell,m} θ ℓ , m , where 1 ≤ ℓ ≤ q 0 1\le\ell\le q_0 1 ≤ ℓ ≤ q 0 . Set
c ℓ , m = π ℓ , m t m , η ℓ , m = θ ℓ , m , α i , m = M i , m − π i ∗ t m . c_{\ell,m}=\frac{\pi_{\ell,m}}{t_m},\qquad\eta_{\ell,m}=\theta_{\ell,m},\qquad\alpha_{i,m}=\frac{M_{i,m}-\pi_i^*}{t_m}. c ℓ , m = t m π ℓ , m , η ℓ , m = θ ℓ , m , α i , m = t m M i , m − π i ∗ .
After the chosen subsequence,
c ℓ , m → c ℓ ≥ 0 , η ℓ , m → η ℓ , α i , m → α i . c_{\ell,m}\to c_\ell\ge0,\qquad\eta_{\ell,m}\to\eta_\ell,\qquad\alpha_{i,m}\to\alpha_i. c ℓ , m → c ℓ ≥ 0 , η ℓ , m → η ℓ , α i , m → α i .
The labels with c ℓ > 0 c_\ell>0 c ℓ > 0 are exactly the atoms of ν \nu ν . Exact mass conservation gives
∑ i α i , m + ∑ ℓ = 1 q 0 c ℓ , m = 0. (76) \sum_i\alpha_{i,m}+\sum_{\ell=1}^{q_0}c_{\ell,m}=0. \tag*{(76)} i ∑ α i , m + ℓ = 1 ∑ q 0 c ℓ , m = 0. (76)
Consider the following function
Ψ m ( x , y ) = ∑ i ≠ j d i j 2 x i j + ∑ ℓ , i ∥ η ℓ , m − θ i ∗ ∥ 2 y ℓ i . \Psi_m(x,y)=\sum_{i\ne j}d_{ij}^2x_{ij}+\sum_{\ell,i}\lVert\eta_{\ell,m}-\theta_i^*\rVert^2y_{\ell i}. Ψ m ( x , y ) = i = j ∑ d ij 2 x ij + ℓ , i ∑ ∥ η ℓ , m − θ i ∗ ∥ 2 y ℓ i .
along with the constraint
P m = { x , y ≥ 0 , ∑ i y ℓ i = c ℓ , m , ∑ j ≠ i ( x i j − x j i ) = α i , m + ∑ ℓ y ℓ i } , (77) \mathcal{P}_m=\left\{x,y\geq0,\quad\sum_i y_{\ell i}=c_{\ell,m},\quad\sum_{j\ne i}(x_{ij}-x_{ji})=\alpha_{i,m}+\sum_\ell y_{\ell i}\right\}, \tag*{(77)} P m = ⎩ ⎨ ⎧ x , y ≥ 0 , i ∑ y ℓ i = c ℓ , m , j = i ∑ ( x ij − x j i ) = α i , m + ℓ ∑ y ℓ i ⎭ ⎬ ⎫ , (77)
where the constraint x , y ≥ 0 x,y\geq0 x , y ≥ 0 is understood as x , y x,y x , y are element-wise non-negative. We prove that for m m m sufficiently large, the optimal transportation problem W 2 2 ( G ‾ m , G ) W_2^2(\overline{G}_m,G) W 2 2 ( G m , G ) and the problem min ( x , y ) ∈ P m Ψ m ( x , y ) \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y) min ( x , y ) ∈ P m Ψ m ( x , y ) under constraint (77) are equivalent in the sense that
min ( x , y ) ∈ P m Ψ m ( x , y ) = W 2 2 ( G ‾ m , G ) t m , (78) \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y)=\frac{W_2^2(\overline{G}_m,G)}{t_m}, \tag*{(78)} ( x , y ) ∈ P m min Ψ m ( x , y ) = t m W 2 2 ( G m , G ) , (78)
and it is possible to construct the optimal solution for each problem from the other problem. Noting that min ( x , y ) ∈ P m Ψ m ( x , y ) \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y) min ( x , y ) ∈ P m Ψ m ( x , y ) is well-defined, as Ψ m ( x , y ) \Psi_m(x,y) Ψ m ( x , y ) is a continuous function satisfying lim x , y → ∞ Ψ m ( x , y ) = ∞ \lim_{x,y\to\infty}\Psi_m(x,y)=\infty lim x , y → ∞ Ψ m ( x , y ) = ∞ , and P m \mathcal{P}_m P m is a closed set.
Consider the optimal coupling from G ‾ m \overline{G}_m G m to G ∗ G_* G ∗ , let γ i j ( m ) \gamma_{ij}^{(m)} γ ij ( m ) be the flow from true source i i i to true target j j j and let ζ ℓ i ( m ) \zeta_{\ell i}^{(m)} ζ ℓ i ( m ) be the flow from nonlocal source ℓ \ell ℓ to true target i i i . Since W 2 2 ( G ‾ m , G ∗ ) = O ( t m ) W_2^2(\overline{G}_m,G_*)=O(t_m) W 2 2 ( G m , G ∗ ) = O ( t m ) and distinct true atoms have positive separation s ∗ > 0 s_*>0 s ∗ > 0 , each γ i j ( m ) \gamma_{ij}^{(m)} γ ij ( m ) for i ≠ j i\ne j i = j or ζ ℓ i ( m ) \zeta_{\ell i}^{(m)} ζ ℓ i ( m ) are O ( t m ) O(t_m) O ( t m ) . Define
x i j ( m ) = γ i j ( m ) t m , i ≠ j , y ℓ i ( m ) = ζ ℓ i ( m ) t m . x_{ij}^{(m)}=\frac{\gamma_{ij}^{(m)}}{t_m},\qquad i\ne j,\qquad y_{\ell i}^{(m)}=\frac{\zeta_{\ell i}^{(m)}}{t_m}. x ij ( m ) = t m γ ij ( m ) , i = j , y ℓ i ( m ) = t m ζ ℓ i ( m ) .
The nonlocal row constraints are ∑ i y ℓ i ( m ) = c ℓ , m \sum_i y_{\ell i}^{(m)}=c_{\ell,m} ∑ i y ℓ i ( m ) = c ℓ , m for all 1 ≤ ℓ ≤ q 0 1\leq\ell\leq q_0 1 ≤ ℓ ≤ q 0 . In addition, the flow constraint for each center θ i ∗ \theta_i^* θ i ∗ as source and target can be expressed as
∑ 1 ≤ j ≤ k γ j i ( m ) + ∑ 1 ≤ ℓ ≤ q 0 ζ ℓ i ( m ) = π i ∗ , ∑ 1 ≤ j ≤ k γ i j ( m ) = M i , m . \sum_{1\leq j\leq k}\gamma_{ji}^{(m)}+\sum_{1\leq\ell\leq q_0}\zeta_{\ell i}^{(m)}=\pi_i^*,\qquad\sum_{1\leq j\leq k}\gamma_{ij}^{(m)}=M_{i,m}. 1 ≤ j ≤ k ∑ γ j i ( m ) + 1 ≤ ℓ ≤ q 0 ∑ ζ ℓ i ( m ) = π i ∗ , 1 ≤ j ≤ k ∑ γ ij ( m ) = M i , m .
By subtracting these two constraints, we have
∑ j ≠ i ( x i j ( m ) − x j i ( m ) ) = α i , m + ∑ ℓ y ℓ i ( m ) . \sum_{j\ne i}\left(x_{ij}^{(m)}-x_{ji}^{(m)}\right)=\alpha_{i,m}+\sum_\ell y_{\ell i}^{(m)}. j = i ∑ ( x ij ( m ) − x j i ( m ) ) = α i , m + ℓ ∑ y ℓ i ( m ) .
Thus, the optimal transportation problem W 2 2 ( G ‾ m , G ) W_2^2(\overline{G}_m,G) W 2 2 ( G m , G ) is equivalent to minimize Ψ m \Psi_m Ψ m over a subset of P m \mathcal{P}_m P m , which implies
min ( x , y ) ∈ P m Ψ m ( x , y ) ≤ W 2 2 ( G ‾ m , G ) t m . \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y)\leq\frac{W_2^2(\overline{G}_m,G)}{t_m}. ( x , y ) ∈ P m min Ψ m ( x , y ) ≤ t m W 2 2 ( G m , G ) .
Furthermore, as W 2 2 ( G ‾ m , G ) ≤ C 0 t m W_2^2(\overline{G}_m,G)\leq C_0t_m W 2 2 ( G m , G ) ≤ C 0 t m , we have min ( x , y ) ∈ P m Ψ m ( x , y ) ≤ C 0 \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y)\leq C_0 min ( x , y ) ∈ P m Ψ m ( x , y ) ≤ C 0 , and for m m m sufficiently large, there exists a constant s 0 s_0 s 0 such that ∥ η ℓ , m − θ i ∗ ∥ ≥ s 0 \lVert\eta_{\ell,m}-\theta_i^*\rVert\geq s_0 ∥ η ℓ , m − θ i ∗ ∥ ≥ s 0 for all ℓ , i \ell,i ℓ , i . It follows that the optimal solution ( x ( m ) , y ( m ) ) (x^{(m)},y^{(m)}) ( x ( m ) , y ( m ) ) of min ( x , y ) ∈ P m Ψ m ( x , y ) \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y) min ( x , y ) ∈ P m Ψ m ( x , y ) satisfies
x i j ( m ) ≤ C 0 min i ≠ j d i j 2 , y ℓ i ( m ) ≤ C 0 s 0 2 . x_{ij}^{(m)} \le\frac{C_0}{\min_{i\ne j}d_{ij}^2}, \qquad y_{\ell i}^{(m)} \le\frac{C_0}{s_0^2}. x ij ( m ) ≤ min i = j d ij 2 C 0 , y ℓ i ( m ) ≤ s 0 2 C 0 .
We construct the coupling between G ‾ m \overline{G}_m G m and G ∗ G_\ast G ∗ from ( x ( m ) , y ( m ) ) (x^{(m)},y^{(m)}) ( x ( m ) , y ( m ) ) as following. For each i ≠ j i\ne j i = j denote the different indices of true centers, let γ i j ( m ) = t m x i j ( m ) \gamma_{ij}^{(m)}=t_m x_{ij}^{(m)} γ ij ( m ) = t m x ij ( m ) denote the total mass that moves from θ i ∗ \theta_i^\ast θ i ∗ of G ‾ m \overline{G}_m G m to θ j ∗ \theta_j^\ast θ j ∗ of G ∗ G_\ast G ∗ , and ζ ℓ i ( m ) = t m y ℓ i ( m ) \zeta_{\ell i}^{(m)}=t_m y_{\ell i}^{(m)} ζ ℓ i ( m ) = t m y ℓ i ( m ) denote the total mass that moves from η ℓ , m \eta_{\ell,m} η ℓ , m of G ‾ m \overline{G}_m G m to θ i ∗ \theta_i^\ast θ i ∗ of G ∗ G_\ast G ∗ . The total mass that remains in θ i ∗ \theta_i^\ast θ i ∗ is defined as γ i i ( m ) = π i ∗ + t m α i , m − t m ∑ j ≠ i x i j \gamma_{ii}^{(m)}=\pi_i^\ast+t_m\alpha_{i,m}-t_m\sum_{j\ne i}x_{ij} γ ii ( m ) = π i ∗ + t m α i , m − t m ∑ j = i x ij . As x i j ( m ) x_{ij}^{(m)} x ij ( m ) is bounded for m m m sufficiently large and t m ↓ 0 t_m\downarrow0 t m ↓ 0 , we have γ i i ( m ) ≥ 0 \gamma_{ii}^{(m)}\ge0 γ ii ( m ) ≥ 0 for m m m large enough. We can check that the transportation plan consisting of γ i j ( m ) \gamma_{ij}^{(m)} γ ij ( m ) and ζ ℓ i ( m ) \zeta_{\ell i}^{(m)} ζ ℓ i ( m ) is actually a coupling between G ‾ m \overline{G}_m G m and G ∗ G_\ast G ∗ . Therefore, we have
min ( x , y ) ∈ P m Ψ m ( x , y ) ≥ W 2 2 ( G ‾ m , G ) t m . \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y)\ge\frac{W_2^2(\overline{G}_m,G)}{t_m}. ( x , y ) ∈ P m min Ψ m ( x , y ) ≥ t m W 2 2 ( G m , G ) .
Thus, we establish the equivalence in (78) and show how to construct an optimal solution to either problem from an optimal solution to the other.
Step 2: Stability of the finite transportation problem. We retain the finitely the nonlocal rows whose limiting masses may be zero. The parameter sequences ( α i , m , c ℓ , m , η ℓ , m ) (\alpha_{i,m},c_{\ell,m},\eta_{\ell,m}) ( α i , m , c ℓ , m , η ℓ , m ) satisfy the exact mass balance (76) and converge to ( α i , c ℓ , η ℓ ) (\alpha_i,c_\ell,\eta_\ell) ( α i , c ℓ , η ℓ ) . Lemma C.2 therefore applies to the feasible sets (77) , and yields
min ( x , y ) ∈ P m Ψ m ( x , y ) ⟶ min ( x , y ) ∈ P Ψ ( x , y ) = M ∗ ( α , ν ) . (79) \min_{(x,y)\in\mathcal{P}_m}\Psi_m(x,y)\longrightarrow\min_{(x,y)\in\mathcal{P}}\Psi(x,y)=\mathcal{M}_\ast(\alpha,\nu). \tag*{(79)} ( x , y ) ∈ P m min Ψ m ( x , y ) ⟶ ( x , y ) ∈ P min Ψ ( x , y ) = M ∗ ( α , ν ) . (79)
Finally, Step 1, Lemma C.1 and (79) give
W 2 2 ( G m , G ∗ ) t m ⟶ ∑ i = 1 p tr ( C i ) + M ∗ ( α , ν ) = Ω ( h γ ) . \frac{W_2^2(G_m,G_\ast)}{t_m}\longrightarrow\sum_{i=1}^{p}\operatorname{tr}(C_i)+\mathcal{M}_\ast(\alpha,\nu)=\Omega(h_\gamma). t m W 2 2 ( G m , G ∗ ) ⟶ i = 1 ∑ p tr ( C i ) + M ∗ ( α , ν ) = Ω ( h γ ) .
The same argument applies to a further subsequence of every original subsequence; uniqueness of the canonical coordinates identifies the same limit throughout. Thus the full sequential assertion holds. □ \square □
Lemma C.4 (Continuity of the transportation functional). The map Ω : T k → R ≥ 0 \Omega:\mathcal{T}_k\to\mathbb{R}_{\ge0} Ω : T k → R ≥ 0 is continuous. In other words, if h m , h ∈ T k h_m,h\in\mathcal{T}_k h m , h ∈ T k and h m → h h_m\to h h m → h in L 2 ( P ∗ ) L^2(P_\ast) L 2 ( P ∗ ) , then
Ω ( h m ) → Ω ( h ) . \Omega(h_m)\to\Omega(h). Ω ( h m ) → Ω ( h ) .
Proof. For every fixed m m m , from Proposition 3.1 we have a family G m , t ∈ O k ( Θ ) G_{m,t}\in\mathcal{O}_k(\Theta) G m , t ∈ O k ( Θ ) , t ↓ 0 t\downarrow0 t ↓ 0 , such that
p G m , t − p ∗ t ρ ∗ ⟶ h m in L 2 ( P ∗ ) . \frac{p_{G_{m,t}}-p_\ast}{t\rho_\ast}\longrightarrow h_m \quad\text{in }L^2(P_\ast). t ρ ∗ p G m , t − p ∗ ⟶ h m in L 2 ( P ∗ ) .
Applying Proposition 4.1 to G m , t G_{m,t} G m , t , we have
W 2 2 ( G m , t , G ∗ ) t → Ω ( h m ) . \frac{W_2^2(G_{m,t},G_*)}{t} \to\Omega(h_m). t W 2 2 ( G m , t , G ∗ ) → Ω ( h m ) .
For each m m m , choose t m ∈ ( 0 , 1 / m ) t_m \in(0,1/m) t m ∈ ( 0 , 1/ m ) sufficiently small that
∥ p G m , t m − p ∗ t m p ∗ − h m ∥ ∗ ≤ m − 1 , ∣ W 2 2 ( G m , t m , G ∗ ) t m − Ω ( h m ) ∣ ≤ m − 1 . (80) \left\|\frac{p_{G_{m,t_m}}-p_*}{t_m p_*}-h_m\right\|_* \le m^{-1}, \qquad\left|\frac{W_2^2(G_{m,t_m},G_*)}{t_m}-\Omega(h_m)\right| \le m^{-1}. \tag*{(80)} t m p ∗ p G m , t m − p ∗ − h m ∗ ≤ m − 1 , t m W 2 2 ( G m , t m , G ∗ ) − Ω ( h m ) ≤ m − 1 . (80)
Given that h m → h h_m \to h h m → h ,
∥ p G m , t m − p ∗ t m p ∗ − h ∥ ∗ ≤ m − 1 + ∥ h m − h ∥ ∗ → 0. \left\|\frac{p_{G_{m,t_m}}-p_*}{t_m p_*}-h\right\|_* \le m^{-1}+\|h_m-h\|_* \to0. t m p ∗ p G m , t m − p ∗ − h ∗ ≤ m − 1 + ∥ h m − h ∥ ∗ → 0.
Proposition 4.1 therefore gives
W 2 2 ( G m , t m , G ∗ ) t m → Ω ( h ) . \frac{W_2^2(G_{m,t_m},G_*)}{t_m} \to\Omega(h). t m W 2 2 ( G m , t m , G ∗ ) → Ω ( h ) .
Combining this convergence with the second part of (80) yields
∣ Ω ( h m ) − Ω ( h ) ∣ ≤ m − 1 + ∣ W 2 2 ( G m , t m , G ∗ ) t m − Ω ( h ) ∣ → 0. |\Omega(h_m)-\Omega(h)| \le m^{-1}+\left|\frac{W_2^2(G_{m,t_m},G_*)}{t_m}-\Omega(h)\right| \to0. ∣Ω ( h m ) − Ω ( h ) ∣ ≤ m − 1 + t m W 2 2 ( G m , t m , G ∗ ) − Ω ( h ) → 0.
Consequently, Ω \Omega Ω is continuous on T k \mathcal{T}_k T k in the L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) topology.
Proof of the joint density and Wasserstein weak limit
Once we understand the geometry of the sequential first-order expansion, we prove Theorem 4.3 on the joint density and Wasserstein weak limit. The argument combines the density convergence in Theorem 3.4 with the sequential expansion and continuity of Ω \Omega Ω through the extended continuous mapping theorem.
Proof of Theorem 4.3 . By Theorem 3.4 , h ^ n ⇝ H † \hat h_n \rightsquigarrow H^\dagger h ^ n ⇝ H † in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) . For fixed n n n , write
H n = { h n , G = n ( p G − p ∗ ) / p ∗ : G ∈ O k ( Θ ) } . \mathcal{H}_n=\left\{h_{n,G}=\sqrt{n}(p_G-p_*)/p_*:G\in\mathcal{O}_k(\Theta)\right\}. H n = { h n , G = n ( p G − p ∗ ) / p ∗ : G ∈ O k ( Θ ) } .
Assumption 2.3 about ordinary mixture identifiability makes the map
( O k ( Θ ) , W 2 ) → ( H n , L 2 ( P ∗ ) ) , G ↦ h n , G (\mathcal{O}_k(\Theta),W_2)\to(\mathcal{H}_n,L^2(P_*)),\qquad G\mapsto h_{n,G} ( O k ( Θ ) , W 2 ) → ( H n , L 2 ( P ∗ )) , G ↦ h n , G
injective. In addition, we also have the inverse map h n , G ↦ G h_{n,G}\mapsto G h n , G ↦ G is continuous. Indeed, if h n , G m → h n , G h_{n,G_m}\to h_{n,G} h n , G m → h n , G in L 2 ( P ∗ ) L^2(P_*) L 2 ( P ∗ ) , then p G m → p G p_{G_m}\to p_G p G m → p G in L 1 ( μ ) L^1(\mu) L 1 ( μ ) by Cauchy–Schwarz with respect to P ∗ P_* P ∗ . Every subsequence of G m G_m G m has a further W 2 W_2 W 2 -convergent subsequence by compactness of O k ( Θ ) \mathcal{O}_k(\Theta) O k ( Θ ) . Continuity of the kernel mixture map identifies its density limit as p G p_G p G , and ordinary identifiability identifies the measure limit as G G G . Therefore G m → G G_m\to G G m → G in W 2 W_2 W 2 . In particular, the finite-n n n statistic below is Borel measurable on H n \mathcal{H}_n H n .
Define F n : H n → L 2 ( P ∗ ) × R F_n:\mathcal{H}_n\to L^2(P_*)\times\mathbb{R} F n : H n → L 2 ( P ∗ ) × R by
F n ( h n , G ) = ( h n , G , n W 2 2 ( G , G ∗ ) ) F_n(h_{n,G})=(h_{n,G},\sqrt{n}W_2^2(G,G_*)) F n ( h n , G ) = ( h n , G , n W 2 2 ( G , G ∗ ))
and define F : T k → L 2 ( P ∗ ) × R F:\mathcal{T}_k\to L^2(P_*)\times\mathbb{R} F : T k → L 2 ( P ∗ ) × R by F ( h ) = ( h , Ω ( h ) ) F(h)=(h,\Omega(h)) F ( h ) = ( h , Ω ( h )) . For any deterministic h n ∈ H n h_n\in\mathcal{H}_n h n ∈ H n converging to h ∈ T k h \in\mathcal{T}_k h ∈ T k , write h n = ( p G n − p ∗ ) / ( t n p ∗ ) h_n = (p_{G_n} - p_*)/(t_n p_*) h n = ( p G n − p ∗ ) / ( t n p ∗ ) with t n = n − 1 / 2 t_n = n^{-1/2} t n = n − 1/2 . Proposition 4.1 gives
n W 2 2 ( G n , G ∗ ) = W 2 2 ( G n , G ∗ ) t n ⟶ Ω ( h ) , \sqrt{n}W_2^2(G_n,G_*) = \frac{W_2^2(G_n,G_*)}{t_n} \longrightarrow\Omega(h), n W 2 2 ( G n , G ∗ ) = t n W 2 2 ( G n , G ∗ ) ⟶ Ω ( h ) ,
so F n ( h n ) → F ( h ) F_n(h_n) \to F(h) F n ( h n ) → F ( h ) . The support of H † H^\dagger H † is contained in T k \mathcal{T}_k T k and F F F is continuous there by Lemma C.4 . The extended continuous mapping theorem for maps on moving domains therefore implies
F n ( h ^ n ) ⇝ F ( H † ) , F_n(\widehat{h}_n) \rightsquigarrow F(H^\dagger), F n ( h n ) ⇝ F ( H † ) ,
which is precisely the joint convergence in the theorem. Applying the continuous square-root map on [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) , we have
n 1 / 4 W 2 ( G ^ n , G ∗ ) ⇝ { Ω ( H † ) } 1 / 2 . n^{1/4}W_2(\widehat{G}_n,G_*) \rightsquigarrow\{\Omega(H^\dagger)\}^{1/2}. n 1/4 W 2 ( G n , G ∗ ) ⇝ { Ω ( H † ) } 1/2 .
As a result, W 2 ( G ^ n , G ∗ ) = O P ( n − 1 / 4 ) W_2(\widehat{G}_n,G_*) = O_P(n^{-1/4}) W 2 ( G n , G ∗ ) = O P ( n − 1/4 ) . This completes the proof.
Proofs for Section 5
Proof of Theorem 5.1. We divide the proof into following steps.
Step 1: Consistency of Q n ⊗ m Q_n^{\otimes m} Q n ⊗ m . Recall that Q n = P G ^ n Q_n = P_{\widehat{G}_n} Q n = P G n , we prove that under the assumption m = o ( n ) m = o(n) m = o ( n ) , we have ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V → P 0 \lVert Q_n^{\otimes m} - P_*^{\otimes m}\rVert_{\mathrm{TV}} \xrightarrow{P} 0 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ TV P 0 . Assumption 2.1 includes positivity of p ∗ p_* p ∗ , so Q n = P G ^ n Q_n = P_{\widehat{G}_n} Q n = P G n is absolutely continuous with respect to P ∗ P_* P ∗ . Therefore, the following density ration is well-defined
R n ( x ) : = d Q n d P ∗ ( x ) = p G ^ n ( x ) p ∗ ( x ) . R_n(x) := \frac{dQ_n}{dP_*}(x) = \frac{p_{\widehat{G}_n}(x)}{p_*(x)}. R n ( x ) := d P ∗ d Q n ( x ) = p ∗ ( x ) p G n ( x ) .
As Q n Q_n Q n is a probability measure, P ∗ R n = 1 P_*R_n = 1 P ∗ R n = 1 . Pearson’s χ 2 \chi^2 χ 2 divergence therefore can be expressed as
χ 2 ( Q n ∥ P ∗ ) = P ∗ ( R n − 1 ) 2 = ∫ ( p G ^ n − p ∗ p ∗ ) 2 p ∗ d μ = ρ ( G ^ n ) 2 . \chi^2(Q_n\Vert P_*) = P_*(R_n - 1)^2 = \int\left(\frac{p_{\widehat{G}_n} - p_*}{p_*}\right)^2 p_*\,\mathrm{d}\mu= \rho(\widehat{G}_n)^2. χ 2 ( Q n ∥ P ∗ ) = P ∗ ( R n − 1 ) 2 = ∫ ( p ∗ p G n − p ∗ ) 2 p ∗ d μ = ρ ( G n ) 2 .
The tensor-product identity for chi-square divergence gives
1 + χ 2 ( Q n ⊗ m ∥ P ∗ ⊗ m ) = { 1 + ρ ( G ^ n ) 2 } m . 1 + \chi^2(Q_n^{\otimes m}\Vert P_*^{\otimes m}) = \{1 + \rho(\widehat{G}_n)^2\}^m. 1 + χ 2 ( Q n ⊗ m ∥ P ∗ ⊗ m ) = { 1 + ρ ( G n ) 2 } m .
Therefore, we have the following comparison
2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V ≤ { χ 2 ( Q n ⊗ m ∥ P ∗ ⊗ m ) } 1 / 2 ≤ [ { 1 + ρ ( G ^ n ) 2 } m − 1 ] 1 / 2 . (81) 2\lVert Q_n^{\otimes m} - P_*^{\otimes m}\rVert_{\mathrm{TV}} \le\{\chi^2(Q_n^{\otimes m}\Vert P_*^{\otimes m})\}^{1/2} \le\left[\{1+\rho(\widehat{G}_n)^2\}^m-1\right]^{1/2}. \tag*{(81)} 2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ TV ≤ { χ 2 ( Q n ⊗ m ∥ P ∗ ⊗ m ) } 1/2 ≤ [ { 1 + ρ ( G n ) 2 } m − 1 ] 1/2 . (81)
The root-n n n density bound in Lemma A.6 yields n ρ ( G ^ n ) 2 = O P ( 1 ) n\rho(\widehat{G}_n)^2 = O_P(1) n ρ ( G n ) 2 = O P ( 1 ) . Thus m ρ ( G ^ n ) 2 = o P ( 1 ) m\rho(\widehat{G}_n)^2 = o_P(1) m ρ ( G n ) 2 = o P ( 1 ) whenever m / n → 0 m/n \to0 m / n → 0 , and
{ 1 + ρ ( G ^ n ) 2 } m − 1 ≤ exp { m ρ ( G ^ n ) 2 } − 1 = o P ( 1 ) . (82) \{1+\rho(\widehat{G}_n)^2\}^m-1 \le\exp\{m\rho(\widehat{G}_n)^2\}-1 = o_P(1). \tag*{(82)} { 1 + ρ ( G n ) 2 } m − 1 ≤ exp { m ρ ( G n ) 2 } − 1 = o P ( 1 ) . (82)
Combining this result with equation (81) , we have ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V → P 0 \lVert Q_n^{\otimes m} - P_*^{\otimes m}\rVert_{\mathrm{TV}} \xrightarrow{P} 0 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ TV P 0 .
Step 2: Prove equation (29) . Put
S n , m ∗ = m 1 / 4 W 2 ( G ^ m ∗ , G ∗ ) , T m 0 = m 1 / 4 W 2 ( G ~ m , G ∗ ) , S_{n,m}^* = m^{1/4}W_2(\widehat{G}_m^*,G_*), \qquad T_m^0 = m^{1/4}W_2(\widetilde{G}_m,G_*), S n , m ∗ = m 1/4 W 2 ( G m ∗ , G ∗ ) , T m 0 = m 1/4 W 2 ( G m , G ∗ ) ,
We estimate T n , m ∗ T_{n,m}^{*} T n , m ∗ through S n , m ∗ S_{n,m}^{*} S n , m ∗ . Using the triangle inequality, we have
∣ T n , m ∗ − S n , m ∗ ∣ ≤ m 1 / 4 W 2 ( G ^ n , G ∗ ) . \lvert T_{n,m}^{*}-S_{n,m}^{*}\rvert\le m^{1/4}W_{2}(\widehat{G}_{n},G_{*}). ∣ T n , m ∗ − S n , m ∗ ∣ ≤ m 1/4 W 2 ( G n , G ∗ ) .
Denote E ∗ f : = E [ f ∣ X 1 , … , X n ] E^{*}f:=E[f\mid X_{1},\ldots,X_{n}] E ∗ f := E [ f ∣ X 1 , … , X n ] , P ∗ P^{*} P ∗ and d B L ∗ d_{\mathrm{BL}}^{*} d BL ∗ be the conditional expectation of f f f , conditional probability, and the conditional d B L d_{\mathrm{BL}} d BL -distance given sample X 1 , … , X n X_{1},\ldots,X_{n} X 1 , … , X n . For any f ∈ B L 1 f\in\mathrm{BL}_{1} f ∈ BL 1 , from estimation above, we have
∣ E ∗ f ( T n , m ∗ ) − E ∗ f ( S n , m ∗ ) ∣ ≤ E ∗ [ ∣ T n , m ∗ − S n , m ∗ ∣ ] ≤ m 1 / 4 W 2 ( G ^ n , G ∗ ) . (83) \lvert E^{*}f(T_{n,m}^{*})-E^{*}f(S_{n,m}^{*})\rvert\le E^{*}[\lvert T_{n,m}^{*}-S_{n,m}^{*}\rvert]\le m^{1/4}W_{2}(\widehat{G}_{n},G_{*}). \tag*{(83)} ∣ E ∗ f ( T n , m ∗ ) − E ∗ f ( S n , m ∗ )∣ ≤ E ∗ [∣ T n , m ∗ − S n , m ∗ ∣] ≤ m 1/4 W 2 ( G n , G ∗ ) . (83)
Consider the measurable statistic on X m \mathcal{X}^{m} X m
Φ m ( x 1 , … , x m ) = m 1 / 4 W 2 { A m ( x 1 , … , x m ) , G ∗ } . \Phi_{m}(x_{1},\ldots,x_{m})=m^{1/4}W_{2}\{\mathcal{A}_{m}(x_{1},\ldots,x_{m}),G_{*}\}. Φ m ( x 1 , … , x m ) = m 1/4 W 2 { A m ( x 1 , … , x m ) , G ∗ } .
Measurability follows from the assumed measurable MLE selection and continuity of W 2 W_{2} W 2 on the compact mixing-measure space. Under Q n ⊗ m Q_{n}^{\otimes m} Q n ⊗ m , Φ m \Phi_{m} Φ m has conditional law equal to the law of S n , m ∗ S_{n,m}^{*} S n , m ∗ ; under P ∗ ⊗ m P_{*}^{\otimes m} P ∗ ⊗ m it has the law of T m 0 T_{m}^{0} T m 0 . By applying following estimation for any measurable g g g with ∥ g ∥ ∞ ≤ 1 \lVert g\rVert_{\infty}\le1 ∥ g ∥ ∞ ≤ 1 ,
∣ ∫ g d Q − ∫ g d P ∣ ≤ 2 ∥ Q − P ∥ T V , \left\lvert\int g\,dQ-\int g\,dP\right\rvert\le2\lVert Q-P\rVert_{\mathrm{TV}}, ∫ g d Q − ∫ g d P ≤ 2 ∥ Q − P ∥ TV ,
to g = f ∘ Φ m g=f\circ\Phi_{m} g = f ∘ Φ m for f ∈ B L 1 f\in\mathrm{BL}_{1} f ∈ BL 1 , we obtain
∣ E ∗ f ( S n , m ∗ ) − E f ( T m 0 ) ∣ ≤ 2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V . (84) \lvert E^{*}f(S_{n,m}^{*})-Ef(T_{m}^{0})\rvert\le2\lVert Q_{n}^{\otimes m}-P_{*}^{\otimes m}\rVert_{\mathrm{TV}}. \tag*{(84)} ∣ E ∗ f ( S n , m ∗ ) − E f ( T m 0 )∣ ≤ 2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ TV . (84)
By combining the estimations from (83) and (84) , we have for all f ∈ B L 1 f\in\mathrm{BL}_{1} f ∈ BL 1 :
∣ E ∗ f ( T n , m ∗ ) − E f ( T m 0 ) ∣ ≤ m 1 / 4 W 2 ( G ^ n , G ∗ ) + 2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ T V . \lvert E^{*}f(T_{n,m}^{*})-Ef(T_{m}^{0})\rvert\le m^{1/4}W_{2}(\widehat{G}_{n},G_{*})+2\lVert Q_{n}^{\otimes m}-P_{*}^{\otimes m}\rVert_{\mathrm{TV}}. ∣ E ∗ f ( T n , m ∗ ) − E f ( T m 0 )∣ ≤ m 1/4 W 2 ( G n , G ∗ ) + 2 ∥ Q n ⊗ m − P ∗ ⊗ m ∥ TV .
Therefore we achieve estimation (29) .
Step 3: Conclusion. We give the proof of the bounds (27) and (28) . Firstly, we observe that Theorem 4.3 gives T n = n 1 / 4 W 2 ( G ^ n , G ∗ ) = O P ( 1 ) T_{n}=n^{1/4}W_{2}(\widehat{G}_{n},G_{*})=O_{P}(1) T n = n 1/4 W 2 ( G n , G ∗ ) = O P ( 1 ) , and from the hypothesis m / n → 0 m/n\to0 m / n → 0 , we have
m 1 / 4 W 2 ( G ^ n , G ∗ ) = ( m n ) 1 / 4 T n = o P ( 1 ) . m^{1/4}W_{2}(\widehat{G}_{n},G_{*})=\left(\frac{m}{n}\right)^{1/4}T_{n}=o_{P}(1). m 1/4 W 2 ( G n , G ∗ ) = ( n m ) 1/4 T n = o P ( 1 ) .
Combining this estimation with Step 1, we have two terms in the RHS of (29) converge in probability to 0. Hence d B L { L ∗ ( T n , m ∗ ) , L ( T m 0 ) } → P 0 d_{\mathrm{BL}}\{\mathcal{L}^{*}(T_{n,m}^{*}),\mathcal{L}(T_{m}^{0})\}\xrightarrow{P}0 d BL { L ∗ ( T n , m ∗ ) , L ( T m 0 )} P 0 .
Theorem 4.3 applied to an independent size-m m m sample gives T m 0 ⇝ T ∞ T_{m}^{0}\rightsquigarrow T_{\infty} T m 0 ⇝ T ∞ . Since bounded-Lipschitz distance metrizes weak convergence on R \mathbb{R} R , d B L { L ( T m 0 ) , L ( T ∞ ) } → 0 d_{\mathrm{BL}}\{\mathcal{L}(T_{m}^{0}),\mathcal{L}(T_{\infty})\}\to0 d BL { L ( T m 0 ) , L ( T ∞ )} → 0 . Using triangle inequality for d B L d_{\mathrm{BL}} d BL , we achieve immediately d B L { L ∗ ( T n , m ∗ ) , L ( T ∞ ) } → P 0 d_{\mathrm{BL}}\{\mathcal{L}^{*}(T_{n,m}^{*}),\mathcal{L}(T_{\infty})\}\xrightarrow{P}0 d BL { L ∗ ( T n , m ∗ ) , L ( T ∞ )} P 0 , which proves the inequality (27) . In addition, a second application of Theorem 4.3 gives d B L { L ( T n ) , L ( T ∞ ) } → 0 d_{\mathrm{BL}}\{\mathcal{L}(T_{n}),\mathcal{L}(T_{\infty})\}\to0 d BL { L ( T n ) , L ( T ∞ )} → 0 , which leads to d B L { L ∗ ( T n , m ∗ ) , L ( T n ) } → P 0 d_{\mathrm{BL}}\{\mathcal{L}^{*}(T_{n,m}^{*}),\mathcal{L}(T_{n})\}\xrightarrow{P}0 d BL { L ∗ ( T n , m ∗ ) , L ( T n )} P 0 and therefore establishes (28) .
□ \square □
Proof of Corollary 5.2. Recall from equation (27) that d B L { L ∗ ( T n , m ∗ ) , L ( T ∞ ) } → P 0 d_{\mathrm{BL}}\{\mathcal{L}^{*}(T_{n,m}^{*}),\mathcal{L}(T_{\infty})\}\xrightarrow{P}0 d BL { L ∗ ( T n , m ∗ ) , L ( T ∞ )} P 0 . As L ( T ∞ ) \mathcal{L}(T_{\infty}) L ( T ∞ ) and H † H^{\dagger} H † follow the same distribution, with the choose of H † H^{\dagger} H † such that H † H^{\dagger} H † is independent of sample X 1 , … , X n X_1,\ldots,X_n X 1 , … , X n . Using Lemma E.2, we have
d B L ∗ ( ( T n , m ∗ ) 2 , Ω ( H † ) ) ≤ ( 2 R + 4 ) d B L { L ∗ ( T n , m ∗ ) , L ( T ∞ ) } + 4 P ( Ω ( H † ) > R 2 ) . d_{\mathrm{BL}}^*\left((T_{n,m}^*)^2,\Omega(H^\dagger)\right) \leq(2R+4)d_{\mathrm{BL}}\left\{\mathcal{L}^*(T_{n,m}^*),\mathcal{L}(T_\infty)\right\}+4\mathbb{P}\left(\Omega(H^\dagger)>R^2\right). d BL ∗ ( ( T n , m ∗ ) 2 , Ω ( H † ) ) ≤ ( 2 R + 4 ) d BL { L ∗ ( T n , m ∗ ) , L ( T ∞ ) } + 4 P ( Ω ( H † ) > R 2 ) .
By first letting n → ∞ n\to\infty n → ∞ , then R → ∞ R\to\infty R → ∞ , we achieve that d B L ∗ ( ( T n , m ∗ ) 2 , Ω ( H † ) ) → P 0 d_{\mathrm{BL}}^*\left((T_{n,m}^*)^2,\Omega(H^\dagger)\right)\xrightarrow{P}0 d BL ∗ ( ( T n , m ∗ ) 2 , Ω ( H † ) ) P 0 . As d B L d_{\mathrm{BL}} d BL metrizes the weak convergence, we achieve that
L ∗ { m W 2 2 ( G ^ m ∗ , G ^ n ) } ⇝ P L { Ω ( H † ) } . \mathcal{L}^*\left\{\sqrt{m}W_2^2(\widehat{G}_m^*,\widehat{G}_n)\right\}\rightsquigarrow_P\mathcal{L}\{\Omega(H^\dagger)\}. L ∗ { m W 2 2 ( G m ∗ , G n ) } ⇝ P L { Ω ( H † )} .
This proves equation (30) .
Proof of Corollary 5.3. We divide the proof into following steps.
Step 1: Convergence of quantile q n , m ∗ ( p 0 ) q_{n,m}^*(p_0) q n , m ∗ ( p 0 ) . * We prove that q n , m ∗ ( p 0 ) q_{n,m}^*(p_0) q n , m ∗ ( p 0 ) converges in probability to q p 0 q_{p_0} q p 0 . According to Theorem 5.1 , we have d B L { T n , m ∗ , T ∞ } → P 0 d_{\mathrm{BL}}\{T_{n,m}^*,T_\infty\}\xrightarrow{P}0 d BL { T n , m ∗ , T ∞ } P 0 . Let t t t be any continuity point of F ∞ F_\infty F ∞ , we show that F n , m ∗ ( t ) → P F ∞ ( t ) F_{n,m}^*(t)\xrightarrow{P}F_\infty(t) F n , m ∗ ( t ) P F ∞ ( t ) . For ϵ ∈ ( 0 , 1 ) \epsilon\in(0,1) ϵ ∈ ( 0 , 1 ) , using Lemma E.3 , we have
∣ F n , m ∗ ( t ) − F ∞ ( t ) ∣ ≤ ϵ − 1 d B L ∗ ( T n , m ∗ , T ∞ ) + ( F Q ( t + ϵ ) − F Q ( t − ϵ ) ) . \left|F_{n,m}^*(t)-F_\infty(t)\right|\leq\epsilon^{-1}d_{\mathrm{BL}}^*(T_{n,m}^*,T_\infty)+\left(F_Q(t+\epsilon)-F_Q(t-\epsilon)\right). F n , m ∗ ( t ) − F ∞ ( t ) ≤ ϵ − 1 d BL ∗ ( T n , m ∗ , T ∞ ) + ( F Q ( t + ϵ ) − F Q ( t − ϵ ) ) .
By first letting n → ∞ n\to\infty n → ∞ , then ϵ → 0 \epsilon\to0 ϵ → 0 , we achieve that ∣ F n , m ∗ ( t ) − F ∞ ( t ) ∣ → P 0 \left|F_{n,m}^*(t)-F_\infty(t)\right|\xrightarrow{P}0 F n , m ∗ ( t ) − F ∞ ( t ) P 0 .
Fix ϵ > 0 \epsilon>0 ϵ > 0 and write p 0 = 1 − α p_0=1-\alpha p 0 = 1 − α . By hypothesis (31) , F ∞ ( q p 0 − ϵ ) < p 0 < F ∞ ( q p 0 + ϵ ) F_\infty(q_{p_0}-\epsilon)<p_0<F_\infty(q_{p_0}+\epsilon) F ∞ ( q p 0 − ϵ ) < p 0 < F ∞ ( q p 0 + ϵ ) . As distribution function has at most countably many discontinuities, we can choose continuity points a , b a,b a , b satisfying q p 0 − ϵ < a < q p 0 < b < q p 0 + ϵ q_{p_0}-\epsilon<a<q_{p_0}<b<q_{p_0}+\epsilon q p 0 − ϵ < a < q p 0 < b < q p 0 + ϵ and still F ∞ ( a ) < p 0 < F ∞ ( b ) F_\infty(a)<p_0<F_\infty(b) F ∞ ( a ) < p 0 < F ∞ ( b ) . Let δ = 1 2 min { p 0 − F ∞ ( a ) , F ∞ ( b ) − p 0 } > 0 \delta=\frac{1}{2}\min\{p_0-F_\infty(a),F_\infty(b)-p_0\}>0 δ = 2 1 min { p 0 − F ∞ ( a ) , F ∞ ( b ) − p 0 } > 0 , by the convergence of CDF at continuous point of F ∞ F_\infty F ∞ just established, we have
F n , m ∗ ( a ) → P F ∞ ( a ) , F n , m ∗ ( b ) → P F ∞ ( b ) . F_{n,m}^*(a)\xrightarrow{P}F_\infty(a),\qquad F_{n,m}^*(b)\xrightarrow{P}F_\infty(b). F n , m ∗ ( a ) P F ∞ ( a ) , F n , m ∗ ( b ) P F ∞ ( b ) .
Hence with probability tending to one,
F n , m ∗ ( a ) ≤ F ∞ ( a ) + δ < p 0 and F n , m ∗ ( b ) ≥ F ∞ ( b ) − δ > p 0 . F_{n,m}^*(a)\leq F_\infty(a)+\delta<p_0\qquad\text{and}\qquad F_{n,m}^*(b)\geq F_\infty(b)-\delta>p_0. F n , m ∗ ( a ) ≤ F ∞ ( a ) + δ < p 0 and F n , m ∗ ( b ) ≥ F ∞ ( b ) − δ > p 0 .
By the definition of the generalized inverse, F n , m ∗ ( a ) < p 0 F_{n,m}^*(a)<p_0 F n , m ∗ ( a ) < p 0 implies q n , m ∗ ( p 0 ) > a q_{n,m}^*(p_0)>a q n , m ∗ ( p 0 ) > a , while F n , m ∗ ( b ) > p 0 F_{n,m}^*(b)>p_0 F n , m ∗ ( b ) > p 0 leads to q n , m ∗ ( p 0 ) ≤ b q_{n,m}^*(p_0)\leq b q n , m ∗ ( p 0 ) ≤ b . Thus, with probability tending to one,
q p 0 − ϵ < a < q n , m ∗ ( p 0 ) ≤ b < q p 0 + ϵ . q_{p_0}-\epsilon<a<q_{n,m}^*(p_0)\leq b<q_{p_0}+\epsilon. q p 0 − ϵ < a < q n , m ∗ ( p 0 ) ≤ b < q p 0 + ϵ .
Since ϵ \epsilon ϵ is arbitrary, this proves q n , m ∗ ( p 0 ) → P q p 0 q_{n,m}^*(p_0)\xrightarrow{P}q_{p_0} q n , m ∗ ( p 0 ) P q p 0 .
Step 2: Consistency of random Wasserstein ball. * We prove the equation (32) and (33) . Theorem 4.3 gives T n ⇝ T ∞ T_n\rightsquigarrow T_\infty T n ⇝ T ∞ , and the quantile consistency established above gives q n , m ∗ ( p 0 ) → P q p 0 q_{n,m}^*(p_0)\xrightarrow{P}q_{p_0} q n , m ∗ ( p 0 ) P q p 0 . Noting that q p 0 q_{p_0} q p 0 is a constant, we employ Slutsky’s theorem to achieve
( T n , q n , m ∗ ( p 0 ) ) ⇝ ( T ∞ , q p 0 ) . (T_n,q_{n,m}^*(p_0))\rightsquigarrow(T_\infty,q_{p_0}). ( T n , q n , m ∗ ( p 0 )) ⇝ ( T ∞ , q p 0 ) .
Consider the closed half-space A = { ( x , y ) ∈ R 2 : x ≤ y } A=\{(x,y)\in\mathbb{R}^2:x\leq y\} A = {( x , y ) ∈ R 2 : x ≤ y } , whose boundary is { x = y } \{x=y\} { x = y } . Under the limiting law, the boundary event ( T ∞ , q p 0 ) ∈ A (T_\infty,q_{p_0})\in A ( T ∞ , q p 0 ) ∈ A is exactly { T ∞ = q p 0 } \{T_\infty=q_{p_0}\} { T ∞ = q p 0 } , which has probability zero by hypothesis (31) . The Portmanteau theorem therefore gives
P { T n ≤ q n , m ∗ ( p 0 ) } → P { T ∞ ≤ q p 0 } = F ∞ ( q p 0 ) = p 0 . \mathbb{P}\{T_n\leq q_{n,m}^*(p_0)\}\to\mathbb{P}\{T_\infty\leq q_{p_0}\}=F_\infty(q_{p_0})=p_0. P { T n ≤ q n , m ∗ ( p 0 )} → P { T ∞ ≤ q p 0 } = F ∞ ( q p 0 ) = p 0 .
This proves equation (32) . To prove equation (33) , from the definition of T n T_n T n , we have
G ∗ ∈ C n , 1 − α ⟺ W 2 ( G ∗ , G ^ n ) ≤ n − 1 / 4 q n , m ∗ ( 1 − α ) ⟺ T n ≤ q n , m ∗ ( 1 − α ) . G_\ast\in C_{n,1-\alpha} \Longleftrightarrow W_2(G_\ast,\widehat{G}_n) \le n^{-1/4}q^\ast_{n,m}(1-\alpha) \Longleftrightarrow T_n \le q^\ast_{n,m}(1-\alpha). G ∗ ∈ C n , 1 − α ⟺ W 2 ( G ∗ , G n ) ≤ n − 1/4 q n , m ∗ ( 1 − α ) ⟺ T n ≤ q n , m ∗ ( 1 − α ) .
Therefore, equation (33) follows from
P ( G ∗ ∈ C n , 1 − α ) = P ( T n ≤ q n , m ∗ ( 1 − α ) ) → p 0 . \mathbb{P}(G_\ast\in C_{n,1-\alpha}) = \mathbb{P}(T_n \le q^\ast_{n,m}(1-\alpha)) \to p_0. P ( G ∗ ∈ C n , 1 − α ) = P ( T n ≤ q n , m ∗ ( 1 − α )) → p 0 .
This completes our proof.
Proof of Proposition 5.4. We divide the proof into following steps.
Step 1: Consistency of Monte Carlo approximation. We prove equation (34) . Given F n \mathcal{F}_n F n , the B = B n B=B_n B = B n bootstrap repetitions are independent and identically distributed, and each statistic T n , m ∗ ( b ) T^{\ast(b)}_{n,m} T n , m ∗ ( b ) has CDF F n , m ∗ F^\ast_{n,m} F n , m ∗ . Thus, conditional on F n \mathcal{F}_n F n , F ^ n , m , B ∗ \widehat{F}^\ast_{n,m,B} F n , m , B ∗ is the empirical CDF based on B B B i.i.d. observations from F n , m ∗ F^\ast_{n,m} F n , m ∗ . The DKW inequality (see Dvoretzky et al. [8 ] ; Massart [23 ] ) gives
P ∗ { sup t ∣ F ^ n , m , B ∗ ( t ) − F n , m ∗ ( t ) ∣ > ε } ≤ 2 e − 2 B ε 2 , (85) \mathbb{P}^\ast\left\{\sup_t\left|\widehat{F}^\ast_{n,m,B}(t)-F^\ast_{n,m}(t)\right|>\varepsilon\right\}\le2e^{-2B\varepsilon^2}, \tag*{(85)} P ∗ { t sup F n , m , B ∗ ( t ) − F n , m ∗ ( t ) > ε } ≤ 2 e − 2 B ε 2 , (85)
where the right side is deterministic and tends to zero because B n → ∞ B_n\to\infty B n → ∞ . Taking unconditional expectations gives
P { sup t ∣ F ^ n , m , B ∗ ( t ) − F n , m ∗ ( t ) ∣ > ε } ≤ 2 e − 2 B n ε 2 → 0 , \mathbb{P}\left\{\sup_t\left|\widehat{F}^\ast_{n,m,B}(t)-F^\ast_{n,m}(t)\right|>\varepsilon\right\}\le2e^{-2B_n\varepsilon^2}\to0, P { t sup F n , m , B ∗ ( t ) − F n , m ∗ ( t ) > ε } ≤ 2 e − 2 B n ε 2 → 0 ,
which proves equation (34) .
Step 2: Asymptotic behavior of quantiles and coverage. We study the asymptotic behavior of q ^ n , m , B ∗ ( p 0 ) \widehat{q}^\ast_{n,m,B}(p_0) q n , m , B ∗ ( p 0 ) and C ^ n , 1 − α \widehat{C}_{n,1-\alpha} C n , 1 − α . Fix p 0 = 1 − α p_0=1-\alpha p 0 = 1 − α and ε > 0 \varepsilon>0 ε > 0 . Choose continuity points a , b a,b a , b exactly as in the proof of Corollary 5.3 so that
q p 0 − ε < a < q p 0 < b < q p 0 + ε , F ∞ ( a ) < p 0 < F ∞ ( b ) . q_{p_0}-\varepsilon<a<q_{p_0}<b<q_{p_0}+\varepsilon,\qquad F_\infty(a)<p_0<F_\infty(b). q p 0 − ε < a < q p 0 < b < q p 0 + ε , F ∞ ( a ) < p 0 < F ∞ ( b ) .
Theorem 5.1 yields F n , m ∗ ( a ) → P F ∞ ( a ) F^\ast_{n,m}(a)\xrightarrow{P}F_\infty(a) F n , m ∗ ( a ) P F ∞ ( a ) and F n , m ∗ ( b ) → P F ∞ ( b ) F^\ast_{n,m}(b)\xrightarrow{P}F_\infty(b) F n , m ∗ ( b ) P F ∞ ( b ) , while the DKW bound (85) gives
F ^ n , m , B ∗ ( a ) − F n , m ∗ ( a ) → P 0 , F ^ n , m , B ∗ ( b ) − F n , m ∗ ( b ) → P 0. \widehat{F}^\ast_{n,m,B}(a)-F^\ast_{n,m}(a)\xrightarrow{P}0,\qquad\widehat{F}^\ast_{n,m,B}(b)-F^\ast_{n,m}(b)\xrightarrow{P}0. F n , m , B ∗ ( a ) − F n , m ∗ ( a ) P 0 , F n , m , B ∗ ( b ) − F n , m ∗ ( b ) P 0.
Therefore F ^ n , m , B ∗ ( a ) < p 0 < F ^ n , m , B ∗ ( b ) \widehat{F}^\ast_{n,m,B}(a)<p_0<\widehat{F}^\ast_{n,m,B}(b) F n , m , B ∗ ( a ) < p 0 < F n , m , B ∗ ( b ) with probability tending to one. Moreover, the generalized-inverse definition then forces a < q ^ n , m , B ∗ ( p 0 ) ≤ b a<\widehat{q}^\ast_{n,m,B}(p_0)\le b a < q n , m , B ∗ ( p 0 ) ≤ b with probability tending to one. Since a , b a,b a , b lie within ε \varepsilon ε of q p 0 q_{p_0} q p 0 and ε \varepsilon ε is arbitrary, we achieve the limit (35) as
q ^ n , m , B ∗ ( p 0 ) → P q p 0 . \widehat{q}^\ast_{n,m,B}(p_0)\xrightarrow{P}q_{p_0}. q n , m , B ∗ ( p 0 ) P q p 0 .
Using Slutsky theorem for T n ⇝ T ∞ T_n\rightsquigarrow T_\infty T n ⇝ T ∞ and the equation (35) , we have ( T n , q ^ n , m , B ∗ ( p 0 ) ) ⇝ ( T ∞ , q p 0 ) (T_n,\widehat{q}^\ast_{n,m,B}(p_0))\rightsquigarrow(T_\infty,q_{p_0}) ( T n , q n , m , B ∗ ( p 0 )) ⇝ ( T ∞ , q p 0 ) . By considering A = { ( x , y ) ∈ R 2 : x ≤ y } A=\{(x,y)\in\mathbb{R}^2:x\le y\} A = {( x , y ) ∈ R 2 : x ≤ y } and using the same argument as in Step 2 in the proof of Corollary 5.3 , we have
P { T n ≤ q ^ n , m , B ∗ ( 1 − α ) } → P { T ∞ ≤ q p 0 } = 1 − α . \mathbb{P}\{T_n\le\widehat{q}^\ast_{n,m,B}(1-\alpha)\}\to\mathbb{P}\{T_\infty\le q_{p_0}\}=1-\alpha. P { T n ≤ q n , m , B ∗ ( 1 − α )} → P { T ∞ ≤ q p 0 } = 1 − α .
This event is exactly { G ∗ ∈ C ^ n , 1 − α } \{G_* \in\widehat{\mathcal{C}}_{n,1-\alpha}\} { G ∗ ∈ C n , 1 − α } by the definition of the ball, proving the equation (36) .
Auxiliary Results
Proof of Corollary 2.4. In this section, we denote
f θ ( x ) = ( 2 π ) − d / 2 ∣ Σ ∣ − 1 / 2 exp { − 1 2 ( x − θ ) ⊤ Σ − 1 ( x − θ ) } . f_\theta(x) = (2\pi)^{-d/2}|\Sigma|^{-1/2}\exp\left\{-\frac{1}{2}(x-\theta)^\top\Sigma^{-1}(x-\theta)\right\}. f θ ( x ) = ( 2 π ) − d /2 ∣Σ ∣ − 1/2 exp { − 2 1 ( x − θ ) ⊤ Σ − 1 ( x − θ ) } .
Step 1: Verify Assumption 2.1. It is obvious to check that the gradient of Gaussian density with respect to θ \theta θ is
∇ θ f θ ( x ) = Σ − 1 ( x − θ ) f θ ( x ) , \nabla_\theta f_\theta(x) = \Sigma^{-1}(x-\theta)f_\theta(x), ∇ θ f θ ( x ) = Σ − 1 ( x − θ ) f θ ( x ) ,
and every higher derivative has the form
∇ θ r f θ ( x ) = P r ( x − θ ) f θ ( x ) , r = 0 , 1 , 2 , 3 , \nabla_\theta^r f_\theta(x) = P_r(x-\theta)f_\theta(x), \qquad r=0,1,2,3, ∇ θ r f θ ( x ) = P r ( x − θ ) f θ ( x ) , r = 0 , 1 , 2 , 3 ,
where P r P_r P r is a tensor-valued polynomial of degree at most r r r whose coefficients depend only on Σ − 1 \Sigma^{-1} Σ − 1 . Fix any true atom, without loss of generality, we can choose θ 1 ∗ \theta_1^* θ 1 ∗ . Since p ∗ ( x ) ≥ π min ∗ f θ 1 ∗ ( x ) p_*(x) \ge\pi_{\min}^* f_{\theta_1^*}(x) p ∗ ( x ) ≥ π m i n ∗ f θ 1 ∗ ( x ) ,
f θ ( x ) p ∗ ( x ) ≤ ( π min ∗ ) − 1 f θ ( x ) f θ 1 ∗ ( x ) . \frac{f_\theta(x)}{p_*(x)} \le(\pi_{\min}^*)^{-1}\frac{f_\theta(x)}{f_{\theta_1^*}(x)}. p ∗ ( x ) f θ ( x ) ≤ ( π m i n ∗ ) − 1 f θ 1 ∗ ( x ) f θ ( x ) .
A direct calculation gives
log f θ ( x ) f θ 1 ∗ ( x ) = x ⊤ Σ − 1 ( θ − θ 1 ∗ ) − 1 2 { θ ⊤ Σ − 1 θ − ( θ 1 ∗ ) ⊤ Σ − 1 θ 1 ∗ } . \log\frac{f_\theta(x)}{f_{\theta_1^*}(x)}
= x^\top\Sigma^{-1}(\theta-\theta_1^*)-\frac{1}{2}\left\{\theta^\top\Sigma^{-1}\theta-(\theta_1^*)^\top\Sigma^{-1}\theta_1^*\right\}. log f θ 1 ∗ ( x ) f θ ( x ) = x ⊤ Σ − 1 ( θ − θ 1 ∗ ) − 2 1 { θ ⊤ Σ − 1 θ − ( θ 1 ∗ ) ⊤ Σ − 1 θ 1 ∗ } .
As Θ \Theta Θ is compact, there exist constants C 1 , C 2 C_1,C_2 C 1 , C 2 such that uniformly in θ ∈ Θ \theta\in\Theta θ ∈ Θ , f θ ( x ) / p ∗ ( x ) ≤ C 1 e C 2 ∥ x ∥ f_\theta(x)/p_*(x)\le C_1e^{C_2\|x\|} f θ ( x ) / p ∗ ( x ) ≤ C 1 e C 2 ∥ x ∥ . Combining this with the polynomial derivative representation and compactness of Θ \Theta Θ , there exists constant C > 0 C>0 C > 0 such that
sup θ ∈ Θ ∥ ∇ θ r f θ ( x ) ∥ p ∗ ( x ) ≤ C ( 1 + ∥ x ∥ 3 ) e C ∥ x ∥ , r ≤ 3. (86) \sup_{\theta\in\Theta}\frac{\|\nabla_\theta^r f_\theta(x)\|}{p_*(x)}
\le C(1+\|x\|^3)e^{C\|x\|}, \qquad r\le3. \tag*{(86)} θ ∈ Θ sup p ∗ ( x ) ∥ ∇ θ r f θ ( x ) ∥ ≤ C ( 1 + ∥ x ∥ 3 ) e C ∥ x ∥ , r ≤ 3. (86)
Fix finite a > 0 a>0 a > 0 and an integer m ≥ 1 m\ge1 m ≥ 1 , and let h ( x ) = ( 1 + ∥ x ∥ 3 ) m e a ∥ x ∥ h(x)=(1+\|x\|^3)^m e^{a\|x\|} h ( x ) = ( 1 + ∥ x ∥ 3 ) m e a ∥ x ∥ . Compactness of Θ \Theta Θ and the Gaussian moment-generating bound imply
sup θ ∈ Θ ∫ R d h ( x ) f θ ( x ) d x < ∞ , P ∗ h = ∑ i = 1 p π i ∗ ∫ R d h ( x ) f θ i ∗ ( x ) d x < ∞ . \sup_{\theta\in\Theta}\int_{\mathbb{R}^d}h(x)f_\theta(x)\,dx<\infty,\qquad
P_*h=\sum_{i=1}^{p}\pi_i^*\int_{\mathbb{R}^d}h(x)f_{\theta_i^*}(x)\,dx<\infty. θ ∈ Θ sup ∫ R d h ( x ) f θ ( x ) d x < ∞ , P ∗ h = i = 1 ∑ p π i ∗ ∫ R d h ( x ) f θ i ∗ ( x ) d x < ∞.
Consequently the right side of (86) belongs to L q ( P ∗ ) L^q(P_*) L q ( P ∗ ) for every finite q q q . Assumption 2.1 follows for any desired η > 0 \eta>0 η > 0 with F ( x ) = C ( 1 + ∥ x ∥ 3 ) e C ∥ x ∣ F(x)=C(1+\|x\|^3)e^{C\|x|} F ( x ) = C ( 1 + ∥ x ∥ 3 ) e C ∥ x ∣ .
Step 2: Verify Assumption 2.2. First we construct an integrable envelope for the log-likelihood class for the class { log p G : G ∈ O k ( Θ ) } \{\log p_G:G\in\mathcal{O}_k(\Theta)\} { log p G : G ∈ O k ( Θ )} . For the upper bound ℓ \ell ℓ , every Gaussian density with covariance Σ \Sigma Σ is bounded by the same finite constant ( 2 π ) − d / 2 ( det Σ ) − 1 / 2 (2\pi)^{-d/2}(\det\Sigma)^{-1/2} ( 2 π ) − d /2 ( det Σ ) − 1/2 , so
log p G ( x ) ≤ − d 2 log ( 2 π ) − 1 2 log ( det Σ ) . \log p_G(x)\le-\frac{d}{2}\log(2\pi)-\frac{1}{2}\log(\det\Sigma). log p G ( x ) ≤ − 2 d log ( 2 π ) − 2 1 log ( det Σ ) .
uniformly in G ∈ O k ( Θ ) G \in\mathcal{O}_k(\Theta) G ∈ O k ( Θ ) . For the lower bound, in any k k k -component representation, there exists one weight, whose mass is at least 1 / k 1/k 1/ k . If that component has location θ ∈ Θ \theta\in\Theta θ ∈ Θ , then p G ( x ) ≥ k − 1 f θ ( x ) p_G(x) \ge k^{-1}f_\theta(x) p G ( x ) ≥ k − 1 f θ ( x ) . Furthermore, compactness of Θ \Theta Θ gives inf θ ∈ Θ log f θ ( x ) ≥ − C ( 1 + ∥ x ∥ 2 ) \inf_{\theta\in\Theta} \log f_\theta(x) \ge-C(1+\lVert x\rVert^2) inf θ ∈ Θ log f θ ( x ) ≥ − C ( 1 + ∥ x ∥ 2 ) . Therefore, there exists a constant C C C depending only on Θ \Theta Θ such that
− C ( 1 + ∥ x ∥ 2 ) ≤ log p G ( x ) ≤ C -C(1+\lVert x\rVert^2) \le\log p_G(x) \le C − C ( 1 + ∥ x ∥ 2 ) ≤ log p G ( x ) ≤ C
uniformly in G ∈ O k ( Θ ) G \in\mathcal{O}_k(\Theta) G ∈ O k ( Θ ) . This envelope is integrable under the finite Gaussian mixture P ∗ P_* P ∗ . Moreover, the parameter space of G G G , given as Δ k − 1 × Θ k \Delta_{k-1} \times\Theta^k Δ k − 1 × Θ k , is a compact set. For each x x x , the map
( π , θ 1 , … , θ k ) ↦ log { ∑ j = 1 k π j f θ j ( x ) } (\pi,\theta_1,\ldots,\theta_k) \mapsto\log\left\{\sum_{j=1}^k \pi_j f_{\theta_j}(x)\right\} ( π , θ 1 , … , θ k ) ↦ log { j = 1 ∑ k π j f θ j ( x ) }
is continuous for every x x x . The compact parametrization, pointwise continuity, and integrable envelope establish the P ∗ P_* P ∗ -Glivenko–Cantelli property.
Now suppose that G , H G,H G , H produce the same Gaussian location-mixture density. In this case, Fourier transformation gives
e − 1 2 t ⊤ Σ t ∫ e − i t ⊤ θ d G ( θ ) = e − 1 2 t ⊤ Σ t ∫ e − i t ⊤ θ d H ( θ ) . e^{-\frac{1}{2}t^\top\Sigma t}\int e^{-it^\top\theta}\,dG(\theta)
=
e^{-\frac{1}{2}t^\top\Sigma t}\int e^{-it^\top\theta}\,dH(\theta). e − 2 1 t ⊤ Σ t ∫ e − i t ⊤ θ d G ( θ ) = e − 2 1 t ⊤ Σ t ∫ e − i t ⊤ θ d H ( θ ) .
The common Gaussian factor is never zero, so the characteristic functions of G G G and H H H coincide
∫ e − i t ⊤ θ d G ( θ ) = ∫ e − i t ⊤ θ d H ( θ ) . \int e^{-it^\top\theta}\,dG(\theta)
=
\int e^{-it^\top\theta}\,dH(\theta). ∫ e − i t ⊤ θ d G ( θ ) = ∫ e − i t ⊤ θ d H ( θ ) .
Uniqueness of characteristic functions gives G = H G=H G = H . Thus the population likelihood has the unique maximizer required in Assumption 2.2 .
Step 3: Verify Assumption 2.3 . Suppose that for pairwise distinct ξ 1 , … , ξ m \xi_1,\ldots,\xi_m ξ 1 , … , ξ m ,
∑ j = 1 m [ a j f ξ j + b j ⊤ ∇ θ f ξ j + 1 2 tr ( C j ∇ θ 2 f ξ j ) ] = 0 , (87) \sum_{j=1}^{m}\left[a_jf_{\xi_j}+b_j^\top\nabla_\theta f_{\xi_j}+\frac{1}{2}\operatorname{tr}(C_j\nabla_\theta^2f_{\xi_j})\right]=0, \tag*{(87)} j = 1 ∑ m [ a j f ξ j + b j ⊤ ∇ θ f ξ j + 2 1 tr ( C j ∇ θ 2 f ξ j ) ] = 0 , (87)
where C 1 , … , C M C_1,\ldots,C_M C 1 , … , C M are symmetric matrices. We show that all the coefficients a j a_j a j , b j b_j b j , and C j C_j C j are equal to 0. In this proof, we use the Fourier transform convention g ^ ( t ) = ∫ e − i t ⊤ x g ( x ) d x \widehat{g}(t)=\int e^{-it^\top x}g(x)\,dx g ( t ) = ∫ e − i t ⊤ x g ( x ) d x . For a Gaussian location kernel f θ ( x ) f_\theta(x) f θ ( x ) , the Fourier transformation with respect to x x x of f f f and its derivatives are
f ξ j ^ ( t ) = e − i t ⊤ ξ j e − 1 2 t ⊤ Σ t , ∇ θ f ξ j ^ ( t ) = − i t f ξ j ^ ( t ) , ∇ θ 2 f ξ j ^ ( t ) = − t t ⊤ f ξ j ^ ( t ) . \widehat{f_{\xi_j}}(t)=e^{-it^\top\xi_j}e^{-\frac{1}{2}t^\top\Sigma t},\qquad
\widehat{\nabla_\theta f_{\xi_j}}(t)=-it\widehat{f_{\xi_j}}(t),\qquad
\widehat{\nabla_\theta^2 f_{\xi_j}}(t)=-tt^\top\widehat{f_{\xi_j}}(t). f ξ j ( t ) = e − i t ⊤ ξ j e − 2 1 t ⊤ Σ t , ∇ θ f ξ j ( t ) = − i t f ξ j ( t ) , ∇ θ 2 f ξ j ( t ) = − t t ⊤ f ξ j ( t ) .
By taking the Fourier transformation to both side of identity (87) and dividing by the everywhere nonzero Gaussian factor e − 1 2 t ⊤ Σ t e^{-\frac{1}{2}t^\top\Sigma t} e − 2 1 t ⊤ Σ t , we have
∑ j = 1 m e − i t ⊤ ξ j P j ( t ) = 0 , t ∈ R d , (88) \sum_{j=1}^{m}e^{-it^\top\xi_j}P_j(t)=0,\qquad t\in\mathbb{R}^d, \tag*{(88)} j = 1 ∑ m e − i t ⊤ ξ j P j ( t ) = 0 , t ∈ R d , (88)
with P j ( t ) = a j − i b j ⊤ t − 1 2 t ⊤ C j t P_j(t)=a_j-ib_j^\top t-\frac{1}{2}t^\top C_jt P j ( t ) = a j − i b j ⊤ t − 2 1 t ⊤ C j t . Now we consider V = R d ∖ ⋃ j ≠ ℓ { v : v ⊤ ( ξ j − ξ ℓ ) = 0 } \mathcal{V}=\mathbb{R}^d\setminus\bigcup_{j\ne\ell}\{v:v^\top(\xi_j-\xi_\ell)=0\} V = R d ∖ ⋃ j = ℓ { v : v ⊤ ( ξ j − ξ ℓ ) = 0 } . The complement is a finite union of proper hyperplanes, so V \mathcal{V} V is open and dense. Fix an arbitrary v ∈ V v\in\mathcal{V} v ∈ V and put λ j ( v ) = v ⊤ ξ j \lambda_j(v)=v^\top\xi_j λ j ( v ) = v ⊤ ξ j . These numbers are pairwise distinct. Restricting (88) to t = s v t=sv t = s v gives
∑ j = 1 m e − i s λ j ( v ) q j , v ( s ) = 0 , (89) \sum_{j=1}^{m} e^{-is\lambda_j(v)}q_{j,v}(s)=0, \tag*{(89)} j = 1 ∑ m e − i s λ j ( v ) q j , v ( s ) = 0 , (89)
where every q j , v : = P j ( s v ) q_{j,v}:=P_j(sv) q j , v := P j ( s v ) is a polynomial of degree at most two. For the fixed v ∈ V v\in\mathcal{V} v ∈ V and a fixed index j j j , apply the differential operator L j , v = ∏ ℓ ≠ j ( d d s + i λ ℓ ( v ) ) 3 \mathcal{L}_{j,v}=\prod_{\ell\ne j}\left(\frac{d}{ds}+i\lambda_\ell(v)\right)^3 L j , v = ∏ ℓ = j ( d s d + i λ ℓ ( v ) ) 3 , to the preceding identity. Note that we have the following identity
( d d s + i b ) ( e i a f ) = e i a ( ( d d s + i ( a + b ) ) f ) . \left(\frac{d}{ds}+ib\right)\left(e^{ia}f\right)=e^{ia}\left(\left(\frac{d}{ds}+i(a+b)\right)f\right). ( d s d + ib ) ( e ia f ) = e ia ( ( d s d + i ( a + b ) ) f ) .
therefore for ℓ ≠ j \ell\ne j ℓ = j ,
( d d s + i λ ℓ ( v ) ) 3 { e − i s λ ℓ ( v ) q ℓ , v ( s ) } = e − i s λ ℓ ( v ) q ℓ , v ( 3 ) ( s ) = 0 , \left(\frac{d}{ds}+i\lambda_\ell(v)\right)^3\left\{e^{-is\lambda_\ell(v)}q_{\ell,v}(s)\right\}=e^{-is\lambda_\ell(v)}q_{\ell,v}^{(3)}(s)=0, ( d s d + i λ ℓ ( v ) ) 3 { e − i s λ ℓ ( v ) q ℓ , v ( s ) } = e − i s λ ℓ ( v ) q ℓ , v ( 3 ) ( s ) = 0 ,
because deg ( q ℓ , v ) ≤ 2 \deg(q_{\ell,v})\le2 deg ( q ℓ , v ) ≤ 2 . Therefore,
e − i s λ j ( v ) ∏ ℓ ≠ j ( d d s + i ( λ ℓ ( v ) − λ j ( v ) ) ) 3 q j , v ( s ) = 0. e^{-is\lambda_j(v)}\prod_{\ell\ne j}\left(\frac{d}{ds}+i\left(\lambda_\ell(v)-\lambda_j(v)\right)\right)^3q_{j,v}(s)=0. e − i s λ j ( v ) ℓ = j ∏ ( d s d + i ( λ ℓ ( v ) − λ j ( v ) ) ) 3 q j , v ( s ) = 0.
Furthermore, if c ≠ 0 c\ne0 c = 0 , the linear operator d / d s + i c d/ds+ic d / d s + i c is injective on the set of polynomials: the highest-degree term of i c q icq i c q cannot be cancelled by q ′ q' q ′ . Hence the product operator is injective and q j , v ≡ 0 q_{j,v}\equiv0 q j , v ≡ 0 . As a result, the identity q j , v ≡ 0 q_{j,v}\equiv0 q j , v ≡ 0 holds for j j j and for every v ∈ V v\in\mathcal{V} v ∈ V .
Taking s = 1 s=1 s = 1 shows that P j ( v ) = 0 P_j(v)=0 P j ( v ) = 0 for every v ∈ V v\in\mathcal{V} v ∈ V . Since V \mathcal{V} V is dense and P j P_j P j is a polynomial, P j P_j P j vanishes identically on R d \mathbb{R}^d R d . Consequently, the formula P j ( t ) = a j − i b j ⊤ t − 1 2 t ⊤ C j t P_j(t)=a_j-i b_j^\top t-\frac{1}{2}t^\top C_jt P j ( t ) = a j − i b j ⊤ t − 2 1 t ⊤ C j t and the fact that C j C_j C j is symmetric then gives a j = 0 a_j=0 a j = 0 , b j = 0 b_j=0 b j = 0 , and C j = 0 C_j=0 C j = 0 for each j j j (the matrices C j C_j C j are symmetric). This establishes second-order strong identifiability.
Lemma E.1 (Criterion for Hausdorff convergence). Let ( X , d ) (X,d) ( X , d ) be a metric space, and let A , A n ⊂ X A,A_n\subset X A , A n ⊂ X , n ≥ 1 n\ge1 n ≥ 1 , be nonempty compact sets. Suppose that:
There exists a compact set K ⊂ X K\subset X K ⊂ X such that A n ⊂ K A_n\subset K A n ⊂ K for every n n n .
For every strictly increasing sequence ( n m ) (n_m) ( n m ) and every sequence y m ∈ A n m y_m\in A_{n_m} y m ∈ A n m , if y m → y y_m\to y y m → y in X X X , then y ∈ A y\in A y ∈ A .
There exists A ′ ⊂ A A'\subset A A ′ ⊂ A with A ′ ‾ = A \overline{A'}=A A ′ = A such that, for every x ∈ A ′ x\in A' x ∈ A ′ , there is a sequence x n ∈ A n x_n\in A_n x n ∈ A n satisfying x n → x x_n\to x x n → x .
Then
d H ( A n , A ) ⟶ 0 , d_H(A_n,A)\longrightarrow0, d H ( A n , A ) ⟶ 0 , where
d H ( B , C ) : = max { sup b ∈ B inf c ∈ C d ( b , c ) , sup c ∈ C inf b ∈ B d ( c , b ) } . d_H(B,C):=\max\left\{\sup_{b\in B}\inf_{c\in C}d(b,c),\sup_{c\in C}\inf_{b\in B}d(c,b)\right\}. d H ( B , C ) := max { b ∈ B sup c ∈ C inf d ( b , c ) , c ∈ C sup b ∈ B inf d ( c , b ) } .
Proof. Write d ( x , B ) : = inf b ∈ B d ( x , b ) d(x,B):=\inf_{b\in B}d(x,b) d ( x , B ) := inf b ∈ B d ( x , b ) . We first show that
sup y ∈ A n d ( y , A ) ⟶ 0. \sup_{y\in A_n}d(y,A)\longrightarrow0. y ∈ A n sup d ( y , A ) ⟶ 0.
Otherwise, there exist ε > 0 \varepsilon>0 ε > 0 , a strictly increasing sequence ( n m ) (n_m) ( n m ) , and points y m ∈ A n m y_m\in A_{n_m} y m ∈ A n m such that d ( y m , A ) ≥ ε d(y_m,A)\ge\varepsilon d ( y m , A ) ≥ ε for every m m m . Since y m ∈ K y_m\in K y m ∈ K and K K K is compact, a subsequence of ( y m ) (y_m) ( y m ) converges to some y ∈ K y\in K y ∈ K . By the second assumption, y ∈ A y\in A y ∈ A . Along this subsequence,
d ( y m , A ) ≤ d ( y m , y ) ⟶ 0 , d(y_m,A)\le d(y_m,y)\longrightarrow0, d ( y m , A ) ≤ d ( y m , y ) ⟶ 0 ,
which is a contradiction.
Next, fix ε > 0 \varepsilon>0 ε > 0 . Since A ′ A' A ′ is dense in A A A and A A A is compact, there exist z 1 , … , z r ∈ A ′ z_1,\ldots,z_r\in A' z 1 , … , z r ∈ A ′ such that
A ⊂ ⋃ i = 1 r B ( z i , ε / 2 ) . A\subset\bigcup_{i=1}^{r}B(z_i,\varepsilon/2). A ⊂ i = 1 ⋃ r B ( z i , ε /2 ) .
For each i i i , choose z i , n ∈ A n z_{i,n}\in A_n z i , n ∈ A n such that z i , n → z i z_{i,n}\to z_i z i , n → z i . Since there are finitely many such sequences, for all sufficiently large n n n ,
max 1 ≤ i ≤ r d ( z i , z i , n ) < ε / 2. \max_{1\le i\le r}d(z_i,z_{i,n})<\varepsilon/2. 1 ≤ i ≤ r max d ( z i , z i , n ) < ε /2.
For any x ∈ A x\in A x ∈ A , choose i i i with d ( x , z i ) < ε / 2 d(x,z_i)<\varepsilon/2 d ( x , z i ) < ε /2 . Then
d ( x , A n ) ≤ d ( x , z i , n ) ≤ d ( x , z i ) + d ( z i , z i , n ) < ε . d(x,A_n)\le d(x,z_{i,n})\le d(x,z_i)+d(z_i,z_{i,n})<\varepsilon. d ( x , A n ) ≤ d ( x , z i , n ) ≤ d ( x , z i ) + d ( z i , z i , n ) < ε .
Thus sup x ∈ A d ( x , A n ) → 0 \sup_{x\in A}d(x,A_n)\to0 sup x ∈ A d ( x , A n ) → 0 . Combining the two directed distance bounds gives d H ( A n , A ) → 0 d_H(A_n,A)\to0 d H ( A n , A ) → 0 . □
Lemma E.2. Let X X X and Y Y Y be real-valued random variables. Then, for any R > 0 R>0 R > 0 , we have
d B L ( X 2 , Y 2 ) ≤ ( 2 R + 2 ) d B L ( X , Y ) + 4 P ( ∣ X ∣ > R − 1 ) . d_{\mathrm{BL}}(X^2,Y^2)\le(2R+2)d_{\mathrm{BL}}(X,Y)+4\mathbb{P}(|X|>R-1). d BL ( X 2 , Y 2 ) ≤ ( 2 R + 2 ) d BL ( X , Y ) + 4 P ( ∣ X ∣ > R − 1 ) .
Proof. Write Δ : = d B L ( X , Y ) \Delta:=d_{\mathrm{BL}}(X,Y) Δ := d BL ( X , Y ) . For R ≥ 1 R\ge1 R ≥ 1 , define g R ( x ) = min { x 2 , R 2 } g_R(x)=\min\{x^2,R^2\} g R ( x ) = min { x 2 , R 2 } . Since g R g_R g R is 2 R 2R 2 R -Lipschitz, for every f ∈ B L 1 ( R ) f\in\mathrm{BL}_1(\mathbb{R}) f ∈ BL 1 ( R ) the function ( f ∘ g R ) / ( 2 R ) (f\circ g_R)/(2R) ( f ∘ g R ) / ( 2 R ) belongs to B L 1 ( R ) \mathrm{BL}_1(\mathbb{R}) BL 1 ( R ) . Hence
∣ E [ f ( g R ( X ) ) ] − E [ f ( g R ( Y ) ) ] ∣ ≤ 2 R Δ . \left|\mathbb{E}[f(g_R(X))]-\mathbb{E}[f(g_R(Y))]\right|\le2R\Delta. ∣ E [ f ( g R ( X ))] − E [ f ( g R ( Y ))] ∣ ≤ 2 R Δ.
Moreover, g R ( x ) = x 2 g_R(x)=x^2 g R ( x ) = x 2 whenever ∣ x ∣ ≤ R |x|\le R ∣ x ∣ ≤ R , so
∣ E [ f ( X 2 ) ] − E [ f ( Y 2 ) ] ∣ ≤ 2 R Δ + 2 P ( ∣ X ∣ > R ) + 2 P ( ∣ Y ∣ > R ) . \left|\mathbb{E}[f(X^2)]-\mathbb{E}[f(Y^2)]\right|\le2R\Delta+2\mathbb{P}(|X|>R)+2\mathbb{P}(|Y|>R). E [ f ( X 2 )] − E [ f ( Y 2 )] ≤ 2 R Δ + 2 P ( ∣ X ∣ > R ) + 2 P ( ∣ Y ∣ > R ) .
To bound the tail probability P ( ∣ Y ∣ > R ) \mathbb{P}(|Y|>R) P ( ∣ Y ∣ > R ) , define q R ( x ) = min { 1 , ( ∣ x ∣ − R + 1 ) + } q_R(x)=\min\{1,(|x|-R+1)_+\} q R ( x ) = min { 1 , ( ∣ x ∣ − R + 1 ) + } . Then q R ∈ B L 1 ( R ) q_R\in\mathrm{BL}_1(\mathbb{R}) q R ∈ BL 1 ( R ) and
1 { ∣ x ∣ > R } ≤ q R ( x ) ≤ 1 { ∣ x ∣ > R − 1 } . \mathbf{1}_{\{|x|>R\}}\le q_R(x)\le\mathbf{1}_{\{|x|>R-1\}}. 1 { ∣ x ∣ > R } ≤ q R ( x ) ≤ 1 { ∣ x ∣ > R − 1 } .
It follows that
P ( ∣ Y ∣ > R ) ≤ E [ q R ( Y ) ] ≤ E [ q R ( X ) ] + Δ ≤ P ( ∣ X ∣ > R − 1 ) + Δ . \mathbb{P}(|Y|>R)\le\mathbb{E}[q_R(Y)]\le\mathbb{E}[q_R(X)]+\Delta\le\mathbb{P}(|X|>R-1)+\Delta. P ( ∣ Y ∣ > R ) ≤ E [ q R ( Y )] ≤ E [ q R ( X )] + Δ ≤ P ( ∣ X ∣ > R − 1 ) + Δ.
Taking the supremum over f ∈ B L 1 ( R ) f \in\mathrm{BL}_1(\mathbb{R}) f ∈ BL 1 ( R ) gives
d B L ( X n 2 , X 2 ) ≤ ( 2 R + 2 ) Δ + 4 P ( ∣ X ∣ > R − 1 ) . d_{\mathrm{BL}}(X_n^2,X^2) \le(2R+2)\Delta+ 4\mathbb{P}(|X|>R-1). d BL ( X n 2 , X 2 ) ≤ ( 2 R + 2 ) Δ + 4 P ( ∣ X ∣ > R − 1 ) .
Lemma E.3. Let P , Q P,Q P , Q be probability measures on R \mathbb{R} R , with distribution functions F P , F Q F_P,F_Q F P , F Q , then, for every t ∈ R t \in\mathbb{R} t ∈ R and ε > 0 \varepsilon> 0 ε > 0 ,
∣ F P ( t ) − F Q ( t ) ∣ ≤ max { 1 , ε − 1 } d B L ( P , Q ) + ( F Q ( t + ε ) − F Q ( t − ε ) ) . |F_P(t)-F_Q(t)| \le\max\{1,\varepsilon^{-1}\}d_{\mathrm{BL}}(P,Q) + (F_Q(t+\varepsilon)-F_Q(t-\varepsilon)). ∣ F P ( t ) − F Q ( t ) ∣ ≤ max { 1 , ε − 1 } d BL ( P , Q ) + ( F Q ( t + ε ) − F Q ( t − ε )) . Proof. Fix ε > 0 \varepsilon>0 ε > 0 and define
u ( x ) = { 1 , x ≤ t , 1 − ( x − t ) / ε , t < x < t + ε , 0 , x ≥ t + ε , ℓ ( x ) = u ( x + ε ) . u(x)=
\begin{cases}
1, & x\le t,\\
1-(x-t)/\varepsilon, & t<x<t+\varepsilon,\\
0, & x\ge t+\varepsilon,
\end{cases}
\qquad\ell(x)=u(x+\varepsilon). u ( x ) = ⎩ ⎨ ⎧ 1 , 1 − ( x − t ) / ε , 0 , x ≤ t , t < x < t + ε , x ≥ t + ε , ℓ ( x ) = u ( x + ε ) . Both functions have sup norm at most 1 and Lipschitz constant at most ε − 1 \varepsilon^{-1} ε − 1 . Thus, with c ε = max { 1 , ε − 1 } c_\varepsilon=\max\{1,\varepsilon^{-1}\} c ε = max { 1 , ε − 1 } ,
∣ ∫ u d P − ∫ u d Q ∣ ≤ c ε d B L ( P , Q ) , ∣ ∫ ℓ d P − ∫ ℓ d Q ∣ ≤ c ε d B L ( P , Q ) . \left|\int u\,dP-\int u\,dQ\right|\le c_\varepsilon d_{\mathrm{BL}}(P,Q),\qquad
\left|\int\ell\,dP-\int\ell\,dQ\right|\le c_\varepsilon d_{\mathrm{BL}}(P,Q). ∫ u d P − ∫ u d Q ≤ c ε d BL ( P , Q ) , ∫ ℓ d P − ∫ ℓ d Q ≤ c ε d BL ( P , Q ) . The pointwise inequalities 1 ( − ∞ , t − ε ] ≤ ℓ ≤ 1 ( − ∞ , t ] ≤ u ≤ 1 ( − ∞ , t + ε ] \mathbf{1}_{(-\infty,t-\varepsilon]}\le\ell\le\mathbf{1}_{(-\infty,t]}\le u\le\mathbf{1}_{(-\infty,t+\varepsilon]} 1 ( − ∞ , t − ε ] ≤ ℓ ≤ 1 ( − ∞ , t ] ≤ u ≤ 1 ( − ∞ , t + ε ] therefore imply
F Q ( t − ε ) − c ε d B L ( P , Q ) ≤ F P ( t ) ≤ F Q ( t + ε ) + c ε d B L ( P , Q ) . F_Q(t-\varepsilon)-c_\varepsilon d_{\mathrm{BL}}(P,Q)\le F_P(t)\le F_Q(t+\varepsilon)+c_\varepsilon d_{\mathrm{BL}}(P,Q). F Q ( t − ε ) − c ε d BL ( P , Q ) ≤ F P ( t ) ≤ F Q ( t + ε ) + c ε d BL ( P , Q ) . Subtracting F Q ( t ) F_Q(t) F Q ( t ) from both sides gives
∣ F P ( t ) − F Q ( t ) ∣ ≤ max { 1 , ε − 1 } d B L ( P , Q ) + max { F Q ( t + ε ) − F Q ( t ) , F Q ( t ) − F Q ( t − ε ) } , |F_P(t)-F_Q(t)|\le\max\{1,\varepsilon^{-1}\}d_{\mathrm{BL}}(P,Q)+\max\{F_Q(t+\varepsilon)-F_Q(t),F_Q(t)-F_Q(t-\varepsilon)\}, ∣ F P ( t ) − F Q ( t ) ∣ ≤ max { 1 , ε − 1 } d BL ( P , Q ) + max { F Q ( t + ε ) − F Q ( t ) , F Q ( t ) − F Q ( t − ε )} , completing the proof.