betaFeedback

WorksMathematics

Weak Limits of Parameter Estimation in Strongly Identifiable Mixture Models

Abstract

We study the asymptotic behavior of global maximum likelihood estimation in finite mixtures with an unknown number of components bounded by kk. Although convergence rates are well understood under strong identifiability, they do not determine the limiting distribution of mixing-measure loss. Under second-order strong identifiability, we establish two geometric results that connect density asymptotics to uncertainty about the mixing measure. First, we characterize the exact attainable density tangent set, accounting for the component budget shared by nonlocal vanishing components and local splitting directions. Second, we derive a sequential first-order expansion of squared Wasserstein loss, showing that its leading transportation cost is uniquely determined by the relative-density direction. Together with a Gaussian likelihood argmax theorem, these results yield joint weak convergence of the relative-density maximum likelihood estimator and nW22(G^n,G∗)\sqrt n W_2^2(\widehat G_n,G_*). The density limit maximizes a Gaussian likelihood process over the attainable tangent set, and the induced limit of n1/4W2(G^n,G∗)n^{1/4}W_2(\widehat G_n,G_*) is generally non-Gaussian. We establish consistency of an mm-out-of-nn parametric bootstrap when m→∞m\to\infty and m/n→0m/n\to0. Numerical studies assess the approximation and distinguish bootstrap and optimization errors; a flow-cytometry application illustrates uncertainty summaries invariant to component labeling under a working mixture model.

Introduction

Finite mixture models combine flexible distributional modeling with an interpretable description of population heterogeneity ([19, 24]). With suitable kernels and sufficiently many components, they can approximate arbitrary densities in L1L^1 ([26]). Their generative interpretation assigns each observation to a latent component, providing a natural basis for clustering. Concretely, a finite mixture has density

pG(x):=∫Θf(x∣θ) dG(θ)=∑j=1pπjf(x∣θj),x∈Rd,p_G(x) := \int_{\Theta} f(x \mid\theta)\,dG(\theta) = \sum_{j=1}^{p} \pi_j f(x \mid\theta_j), \qquad x \in\mathbb{R}^{d},

where {f(⋅∣θ):θ∈Θ}\{f(\cdot\mid\theta) : \theta\in\Theta\} is a family of probability density kernels and G=∑j=1pπjδθjG = \sum_{j=1}^{p}\pi_j\delta_{\theta_j} is the mixing measure, with distinct atoms θ1,…,θp∈Θ\theta_1,\ldots,\theta_p \in\Theta, positive weights π1,…,πp\pi_1,\ldots,\pi_p, and ∑j=1pπj=1\sum_{j=1}^{p}\pi_j = 1. Equivalently, an observation is generated by first selecting a latent component ZZ with P(Z=j)=πj\mathbb{P}(Z=j)=\pi_j and then drawing X∣Z=jX \mid Z=j from f(⋅∣θj)f(\cdot\mid\theta_j). The mixing measure thus encodes both the component parameters and their relative prevalence in the population.

Given independent observations X1,…,XnX_1,\ldots,X_n from pG∗p_{G_*}, two related but distinct estimation problems arise. The first is density estimation, which seeks to recover the observable distribution pG∗p_{G_*}. The second is mixing-measure estimation, which seeks to recover the latent distribution G∗G_*, including the component parameters and their mixing proportions. The former describes the distribution of the observed data, whereas the latter describes the underlying population heterogeneity.

Related literature

Maximum likelihood estimation is a standard approach to estimating both mixture densities and their mixing measures [17, 19, 24]. For a class G\mathcal{G} of mixing measures, the maximum likelihood estimator is defined by

G^n∈arg max⁡G∈G∑ℓ=1nlog⁡pG(Xℓ).\widehat{G}_n \in\underset{G\in\mathcal{G}}{\operatorname{arg\,max}}\sum_{\ell=1}^{n}\log p_G(X_\ell).

When the model is restricted to a single component, estimation reduces to ordinary parametric maximum likelihood. Under standard regularity conditions, the estimator θ^n\widehat{\theta}_n satisfies

n(θ^n−θ∗)⇝N(0,I(θ∗)−1).\sqrt{n}(\widehat{\theta}_n-\theta_*) \rightsquigarrow N(0,I(\theta_*)^{-1}).

where I(θ∗)I(\theta_*) is the Fisher information matrix [31]. For example, in a Gaussian location model with known positive definite covariance Σ\Sigma, the maximum likelihood estimator is the sample mean, and n(μ^n−μ∗)⇝Nd(0,Σ)\sqrt{n}(\widehat{\mu}_n-\mu_*) \rightsquigarrow N_d(0,\Sigma). Asymptotic normality also holds for exactly specified finite mixtures, after aligning component labels, under suitable regularity conditions and nonsingular Fisher information [28 Theorem 3.1].

A prominent framework for studying mixing-measure estimation, developed by Nguyen [27] and extended by Ho and Nguyen [13], connects density estimation to the recovery of latent component parameters through inverse inequalities. The density-estimation step uses empirical-process arguments and bracketing entropy to control the complexity of the mixture density class [32, 30]. These arguments yield bounds for Hellinger distance of the form dH(pG^n,pG∗)=OP(εn)d_H(p_{\widehat{G}_n},p_{G_*})=O_P(\varepsilon_n), where εn\varepsilon_n is determined by the relevant entropy bounds. To measure discrepancy between mixing measures, this framework uses Wasserstein distances [33]. For r≥1r\geq1, define

Wr(G,G′):={inf⁡γ∈Γ(G,G′)∫Θ×Θ∥θ−θ′∥r dγ(θ,θ′)}1/r.W_r(G,G'):=\left\{\inf_{\gamma\in\Gamma(G,G')}\int_{\Theta\times\Theta}\lVert\theta-\theta'\rVert^r\,\mathrm{d}\gamma(\theta,\theta')\right\}^{1/r}.

where Γ(G,G′)\Gamma(G,G') denotes the set of probability measures with marginals GG and G′G'. Inverse inequalities then relate density discrepancies to transportation costs. In particular, under suitable smoothness and strong identifiability conditions, one obtains locally around a fixed G∗G_*

dH(pG,pG∗)≥cW1(G,G∗)in the exactly specified setting,d_H(p_G,p_{G_*})\geq cW_1(G,G_*) \qquad\text{in the exactly specified setting,}

and

dH(pG,pG∗)≥cW22(G,G∗)in the over-specified setting,d_H(p_G,p_{G_*})\geq cW_2^2(G,G_*) \qquad\text{in the over-specified setting,}

for some constant c>0c>0 [27, 13]. The first inequality applies when the fitted and true numbers of components agree, under first-order strong identifiability. The second applies when additional components are permitted, under second-order strong identifiability.

These inequalities translate a density rate εn\varepsilon_n into mixing-measure rates εn\varepsilon_n under W1W_1 and εn1/2\varepsilon_n^{1/2} under W2W_2, respectively. Thus, a root-nn density rate yields the familiar n−1/2n^{-1/2} and n−1/4n^{-1/4} rates; logarithmic factors in the density bound propagate accordingly. The change in transportation order reflects the possibility of first-order cancellation: several fitted components near a true atom may have weighted displacements that sum to zero, leaving a second-order density discrepancy.

Subsequent work refines this framework in several directions. Manole and Ho [21] develop stronger loss functions that distinguish the convergence rates of individual fitted components, revealing heterogeneity that a single Wasserstein loss can obscure. Heinrich and Kahn [12] use coarse-graining trees to organize component configurations across multiple scales and establish uniform minimax rates. More recently, Nguyen et al. [25] quantify how separation between component parameters affects inverse inequalities and estimation rates in finite Gaussian location mixtures. Together, these developments clarify how component configurations govern estimation accuracy.

Another prominent line of work studies mixing-measure estimation through moment-based methods [34, 6]. By relating moment discrepancies to Wasserstein distance, these methods yield minimax-optimal estimation rates for Gaussian location mixtures in both univariate and high-dimensional settings without requiring component separation.

Contributions

Existing work establishes asymptotic normality of likelihood estimators for exactly specified mixtures under suitable regularity conditions [28], together with extensive convergence-rate theory for overspecified mixtures [2, 27, 13, 21]. These results leave a distinct question: what is the limiting distribution of the mixing-measure estimation error when redundant components are permitted? Our goal is to answer this question for the global maximum likelihood estimator

G^n∈arg max⁡G∈Ok(Θ)ℓn(G),ℓn(G):=∑r=1nlog⁡pG(Xr),\widehat{G}_n \in\underset{G \in\mathcal{O}_k(\Theta)}{\operatorname{arg\,max}} \ell_n(G), \qquad\ell_n(G) := \sum_{r=1}^{n} \log p_G(X_r),

where Ok(Θ)\mathcal{O}_k(\Theta) denotes the mixing measures with at most kk atoms and the true mixing measure G∗G_\ast has p≤kp \le k distinct atoms. In the second-order strongly identifiable regime, existing inverse inequalities translate root-nn density convergence into the fourth-root mixing-measure rate W2(G^n,G∗)=OP(n−1/4)W_2(\widehat{G}_n,G_\ast)=O_{\mathbb{P}}(n^{-1/4}) [27, 13], but do not determine its limiting distribution. Under second-order strong identifiability and the stated regularity conditions, we establish a weak limit for n1/4W2(G^n,G∗)n^{1/4}W_2(\widehat{G}_n,G_\ast). Our analysis connects the local likelihood geometry to the leading transportation cost through two geometric results.

Our starting point is the scaled relative-density error

hn:=n pG^n−p∗p∗,p∗:=pG∗.h_n := \sqrt{n}\,\frac{p_{\widehat{G}_n}-p_\ast}{p_\ast}, \qquad p_\ast:= p_{G_\ast}.

To describe its possible limits, we characterize the attainable tangent set

Tk:={h∈L2(P∗):∃tm↓0 and Gm∈Ok(Θ):pGm−p∗tmp∗⟶h in L2(P∗)}.\mathcal{T}_k := \left\{ h \in L^2(P_*) : \exists t_m \downarrow0 \text{ and } G_m \in\mathcal{O}_k(\Theta) : \frac{p_{G_m}-p_*}{t_m p_*} \longrightarrow h \text{ in } L^2(P_*) \right\}.

Writing G∗=∑i=1pπi∗δθi∗G_* = \sum_{i=1}^{p} \pi_i^* \delta_{\theta_i^*}, we show that every h∈Tkh \in\mathcal{T}_k admits a unique representation

h(x)=1p∗(x)[∑i=1p{αifθi∗(x)+βiT∇θfθi∗(x)+12tr⁡(Ci∇θ2fθi∗(x))}+∫Θfθ(x) dν(θ)],(1)h(x) = \frac{1}{p_*(x)}\left[\sum_{i=1}^{p}\left\{\alpha_i f_{\theta_i^*}(x) + \beta_i^{\mathsf T}\nabla_{\theta}f_{\theta_i^*}(x) + \frac{1}{2}\operatorname{tr}\left(C_i\nabla_{\theta}^2 f_{\theta_i^*}(x)\right)\right\} + \int_{\Theta} f_{\theta}(x)\,d\nu(\theta)\right], \tag*{(1)}

References

  1. [1]Nicola Bariletto, Dung Le, Alessandro Rinaldo, and Nhat Ho. Convergence rates for latent mixing measures in infinite homoscedastic location-scale mixture models. arXiv preprint arXiv:2605.06845, 2026. doi: 10.48550/arXiv.2605.06845. (Cited on page 33.)DOI
  2. [2]Jiahua Chen. Optimal rate of convergence for finite mixture models. Annals of Statistics, 23(1): 221–233, 1995. doi: 10.1214/aos/1176324464. (Cited on pages 3 and 36.)DOI
  3. [3]Gregory Cox. A generalized argmax theorem with applications. arXiv preprint arXiv:2209.08793, 2022. doi: 10.48550/arXiv.2209.08793. (Cited on pages 12 and 54.)DOI
  4. [4]Didier Dacunha-Castelle and Elisabeth Gassiat. Testing the order of a model using locally conic parametrization: Population mixtures and stationary ARMA processes. Annals of Statistics, 27 (4):1178–1209, 1999. doi: 10.1214/aos/1017938921. (Cited on page 44.)
  5. [5]Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39(1):1–38, 1977. (Cited on page 21.)DOI
  6. [6]Natalie Doss, Yihong Wu, Pengkun Yang, and Harrison H. Zhou. Optimal estimation of high-dimensional Gaussian location mixtures. The Annals of Statistics, 51(1):62–95, 2023. doi: 10.1214/22-AOS2207. (Cited on page 3.)DOI
  7. [7]Mathias Drton and Martyn Plummer. A Bayesian information criterion for singular models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(2):323–380, 2017. doi: 10.1111/rssb.12187. (Cited on page 33.)DOI
  8. [8]Aryeh Dvoretzky, Jack Kiefer, and Jacob Wolfowitz. Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. The Annals of Mathematical Statistics, 27(3):642–669, 1956. (Cited on page 66.)
  9. [9]Paul Embrechts and Marius Hofert. A note on generalized inverses. Mathematical Methods of Operations Research, 77(3):423–432, 2013. doi: 10.1007/s00186-013-0436-7. (Cited on page 19.)DOI
  10. [10]Maura Gasparetto, Tracy Gentry, Said Sebti, Erica O’Bryan, Ramadevi Nimmanapalli, Michelle A. Blaskovich, Kapil Bhalla, David Rizzieri, Perry Haaland, Jack Dunne, and Clay Smith. Identification of compounds that enhance the anti-lymphoma activity of rituximab using flow cytometric high-content screening. Journal of Immunological Methods, 292(1–2):59–71, 2004. doi: 10.1016/j.jim.2004.06.003. (Cited on page 28.)DOI
  11. [11]Charles J. Geyer. On the asymptotics of constrained M-estimation. Annals of Statistics, 22(4):1993–2010, 1994. doi: 10.1214/aos/1176325768. (Cited on page 44.)DOI
  12. [12]Philippe Heinrich and Jonas Kahn. Strong identifiability and optimal minimax rates for finite mixture estimation. Annals of Statistics, 46(6A):2844–2870, 2018. doi: 10.1214/17-AOS1641. (Cited on pages 3, 8, 36, and 37.)DOI
  13. [13]Nhat Ho and XuanLong Nguyen. On strong identifiability and convergence rates of parameter estimation in finite mixtures. Electronic Journal of Statistics, 10(1):271–307, 2016a. doi: 10.1214/16-EJS1105. (Cited on pages 2, 3, 8, 36, 37, and 40.)DOI
  14. [14]Nhat Ho and XuanLong Nguyen. Convergence rates of parameter estimation for some weakly identifiable finite mixtures. Annals of Statistics, 44(6):2726–2755, 2016b. doi: 10.1214/16-AOS1444. (Cited on pages 2 and 8.)DOI
  15. [15]Rob J. Hyndman and Yanan Fan. Sample quantiles in statistical packages. The American Statistician, 50(4):361–365, 1996. doi: 10.1080/00031305.1996.10473566. (Cited on page 19.)DOI
  16. [16]Christine Keribin. Consistent estimation of the order of mixture models. Sankhya: The Indian Journal of Statistics, Series A, 62(1):49–66, 2000. (Cited on page 33.)
  17. [17]Jack Kiefer and Jacob Wolfowitz. Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Annals of Mathematical Statistics, 27(4):887–906, 1956. (Cited on pages 2 and 35.)
  18. [18]Brian G. Leroux. Consistent estimation of a mixing distribution. Annals of Statistics, 20(3): 1350–1360, 1992. (Cited on page 35.)DOI
  19. [19]Bruce G. Lindsay. Mixture Models: Theory, Geometry and Applications, volume 5 of NSF-CBMS Regional Conference Series in Probability and Statistics. Institute of Mathematical Statistics, 1995. (Cited on pages 1 and 2.)
  20. [20]Kenneth Lo, Florian Hahne, Ryan R. Brinkman, and Raphael Gottardo. flowClust: a Bioconductor package for automated gating of flow cytometry data. BMC Bioinformatics, 10:145, 2009. doi: 10.1186/1471-2105-10-145. (Cited on page 28.)DOI
  21. [21]Tudor Manole and Nhat Ho. Refined convergence rates for maximum likelihood estimation under finite mixture models. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 14979–15006, 2022. (Cited on pages 3 and 37.)arxiv.org/abs/2202.08786
  22. [22]Tudor Manole and Abbas Khalili. Estimating the number of components in finite mixture models via the Group-Sort-Fuse procedure. The Annals of Statistics, 49(6):3043–3069, 2021. doi: 10.1214/21-AOS2072. (Cited on page 33.)DOI
  23. [23]Pascal Massart. The tight constant in the Dvoretzky–Kiefer–Wolfowitz inequality. The Annals of Probability, 18(3):1269–1283, 1990. (Cited on page 66.)DOI
  24. [24]Geoffrey J. McLachlan and David Peel. Finite Mixture Models. Wiley, New York, 2000. (Cited on pages 1 and 2.)DOI
  25. [25]Huy Nguyen, Dung Le, Alessandro Rinaldo, and Nhat Ho. On the geometry of separation in finite Gaussian mixtures. arXiv preprint arXiv:2606.16179, 2026. doi: 10.48550/arXiv.2606.16179. (Cited on page 3.)DOI
  26. [26]TrungTin Nguyen, Faicel Chamroukhi, Hien D. Nguyen, and Geoffrey J. McLachlan. Approximation of probability density functions via location-scale finite mixtures in Lebesgue spaces. Communications in Statistics—Theory and Methods, 52(14):5048–5059, 2023. doi: 10.1080/03610926.2021.2002360. (Cited on page 1.)DOI
  27. [27]XuanLong Nguyen. Convergence of latent mixing measures in finite and infinite mixture models. Annals of Statistics, 41(1):370–400, 2013. doi: 10.1214/12-AOS1065. (Cited on pages 2, 3, 37, and 40.)DOI
  28. [28]Richard A. Redner and Homer F. Walker. Mixture densities, maximum likelihood and the EM algorithm. SIAM Review, 26(2):195–239, 1984. doi: 10.1137/1026034. (Cited on pages 2 and 3.)DOI
  29. [29]R. Tyrrell Rockafellar and Roger J.-B. Wets. Variational Analysis, volume 317 of Grundlehren der mathematischen Wissenschaften. Springer, Berlin, Heidelberg, 1998. ISBN 978-3-540-62772-2. doi: 10.1007/978-3-642-02431-3. (Cited on page 7.)DOI
  30. [30]Sara A. van de Geer. Empirical Processes in M-Estimation. Cambridge University Press, Cambridge, 2000. (Cited on pages 2 and 40.)
  31. [31]Aad W. van der Vaart. Asymptotic Statistics. Cambridge University Press, Cambridge, 1998. (Cited on pages 2, 7, and 35.)DOI
  32. [32]Aad W. van der Vaart and Jon A. Wellner. Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York, 1996. (Cited on pages 2, 40, and 42.)
  33. [33]Cédric Villani. Optimal Transport: Old and New. Springer, Berlin, 2009. (Cited on page 2.)
  34. [34]Yihong Wu and Pengkun Yang. Optimal estimation of Gaussian mixtures via denoised method of moments. Annals of Statistics, 48(4):1981–2007, 2020. doi: 10.1214/19-AOS1873. (Cited on page 3.)DOI

Paper details

vlnc:W_4afskdsv
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 13

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents