WorksMathematics
Comments on: `CliPS: How to identify cluster distributions in Bayesian mixture models'
Abstract
I congratulate the authors on their contribution, which addresses important challenges in understanding posterior distributions in Bayesian mixture models. As underlined by the authors, the Bayesian framework has several advantages, yet label-switching, identifiability issues, and the trans-dimensional nature of parameter spaces make exploiting the Bayesian approach and understanding the resulting posterior difficult in practice. The paper offers clear explanations and a simple, intuitive procedure to address this by clustering the Markov chain Monte Carlo (MCMC) draws of component-specific parameters. It is clearly illustrated on several examples.
Main contribution
The authors [1] develop a procedure to make inference about the underlying sub-populations present in the population based on Markov chain Monte Carlo (MCMC) draws of component-specific parameters from Bayesian mixture models. For data , the mixture model assumes:
where the component weights are non-negative and sum to one and the -th component-distribution is parametrized by the -dimensional vector . This formulation is very general and applied in various domains, such as biology, business, economics, and medicine. The Bayesian model is completed with prior distributions:
where is the prior for the component-specific parameters with hyperparameter and denotes a symmetric -dimensional Dirichlet distribution. In practice the number of clusters is rarely known, and the authors consider and review various Bayesian approaches to address this. In all approaches, MCMC is commonly used to estimate the Bayesian mixture model, producing a large number of posterior draws of the component specific parameters
where denotes the -th draw of the number of clusters in the data, that is, the number of occupied components, which may vary across iterations1.
To address the trans-dimensionality of the parameter space, in addition to well-known identifiability issues and label-switching, the authors propose Clustering in the Parameter Space (CliPS) [1]. The main idea is to:
Filter the MCMC draws by eliminating all iterations with , for a suitable estimate , such as the posterior mode. Let denote the remaining number of iterations.
Apply clustering, namely -means, to group the component-specific parameter draws for and . This produces group assignments for the -th component of the -th iteration.
Lastly, the iterations are filtered once again and reordered, by first removing all those whose group assignments do not define a permutation of . This ensures that every component-specific parameter in the -th draw is associated to a unique cluster. The relative number of retained partitions provides a diagnostic indicating how well-separated the clusters are. A unique labeling of the final set of retained draws is achieved by reordering the components through the inverse of ; these reordered draws characterize the distribution of the component-specific parameters given clusters.
The approach is elegantly simple and demonstrated through several examples.
Extensions and alternatives to clustering in the parameter space
A particularly appealing feature of CliPS is that it transforms a difficult relabeling problem into a geometrically intuitive clustering problem in parameter space. This perspective naturally suggests several possible extensions, including alternative clustering strategies that reduce the need for filtering and discarding information and accommodate more flexible cluster shapes. Motivated by the structure visible in Figure 8b of the paper [1], one possible alternative is DBSCAN [Density Based Spatial Clustering of Applications with Noise, [2]]. DBSCAN could be applied directly to cluster all draws corresponding to non-empty components. Since DBSCAN does not force every draw to belong to a group, it can automatically filter low-density draws (e.g. corresponding to a small cluster that is occasionally split off). In this setting, the group assignment is , with 0 denoting noise and denoting the number of clusters identified by DBSCAN. Thus, defines a sub-partition. Filtering draws that consist of duplicated non-noise cluster labels, e.g. , is still sensible, but draws that consist of unique subsets of non-noise labels can be retained, e.g. . This provides a unique ordering across draws of potentially varying numbers of clusters. Moreover, DBSCAN allows more general shapes in the cluster distributions.
As a simple illustration, I reran the diabetes data set example, available through the R package mclust [3]. Prior specification follows [1], except the use of a static mixture2 within the telescope package to obtain MCMC draws [4]. In this case, the posterior mode of is 4, and after filtering for , the number of MCMC draws retained is 553 out of 1,000. Applying CliPS, the non-permutation is 0.0633, and the cluster distributions, visualized in Figure 1(a), are from 518 draws. In contrast, I applied DBSCAN to all MCMC draws (with minPts = 10 and eps = 30 chosen based on the kNN distance plot). Cluster distributions colored by DBSCAN group assignments are shown in Figure 1(b), with noise assignments marked with a gray cross. This is based on 999 of the initial 1,000 MCMC draws (only one is removed after filtering draws which contain duplicated clusters). Moreover, for each DBSCAN group, we can assess its reliability by the proportion of retained draws in which it is contained. High values for groups 1, 2, and 3 indicate that they are present in (almost) all iterations, whereas there is more uncertainty in the presence of group 4.
Figure 1. Diabetes example: summary of the cluster distributions based on MCMC draws of the component-specific means from a static Bayesian mixture. Top left: CliPS with at posterior mode retains 518 of the 1,000 initial MCMC draws (the non-permutation rate is 0.0633 and ). Top right: DBSCAN retains 999 of the 1,000 initial MCMC draws (the non-permutation rate is 0.032). Draws are colored by the DBSCAN cluster labels, with those identified as noise marked with a gray cross. Bottom left: posterior on the number of clusters. Bottom right: Proportion of draws in DBSCAN in which the cluster is present. This reflects the reliability of the cluster, since a value of 1 indicates it is present in all MCMC draws.
Comparison with other approaches
The paper [1] provides an overview in the Introduction of other approaches to summarize posterior distributions in Bayesian mixture models, but does not provide empirical comparisons in the examples. Thus, to further contextualize the contribution of CliPS, I compare summaries of related approaches on selected examples. In particular, while the paper [1] emphasizes the ‘use the data to infer a statistical model which allows to determine the cluster distributions of the data generating process and thus characterizes the population of interest, beyond focusing only on the partition of the sample at hand’, the subsequent summarization step yields an estimated partition induced by the identified cluster distributions. In contrast, other approaches first estimate the partition [e.g. [5, 6, 7, 8, 9]] and subsequently, obtain inference on the sub-population parameters conditioned on the partition estimate. Therefore, it is natural to compare such approaches. In addition, while not directly compared here, we also highlight the elegant work of [10], which is more directly related to CliPS. Using a decision-theoretic framework, [10] identify a point estimate of the underlying mixing measure by minimizing posterior expected sliced Wasserstein distance, and subsequently obtain a partition estimate given the estimated mixing measure.
We illustrate these comparisons using the diabetes example, comparing the CliPS summary with that obtained from [7] and [11]. In [7], an estimate of partition is obtained by minimizing the posterior expected variation of information [VI, 12] and credible balls are constructed around this estimate. Figure 2(a) shows the partition estimate containing 3 clusters, along with the fitted mean and covariance given the partition. The 95% credible ball around this estimate can be summarized by the upper vertical bound (most distance among those in the credible ball with the fewest number of clusters), the lower vertical bound (most distance among those in the credible ball with the greatest number of clusters), and the horizontal bound (most distance among those in the credible ball). For conciseness, we show only the lower vertical bound in Figure 2(b) containing five clusters, one of which is a singleton and another with only 6 observations, and note that the upper vertical bound has three clusters, while the horizontal bound has four. Comparing with CliPS in Figure 1(a), the main conclusions are similar. However, CliPS places greater emphasis on the possibility of a small fourth sub-population. Instead, CliPS–DBSCAN (Figure 1(b)) estimates a reliability probability of less than 0.5 for this fourth, small cluster, and this cluster is not present in the partition estimate in Figure 2(a), although is present in the 95% credible ball.
Figure 2. Diabetes example: summary of the posterior partition structure based on MCMC draws from a static Bayesian mixture. Pair plots of the observed data colored by cluster, with the cross and ellipse illustrating the fitted mean and covariance conditioned on the estimated partition. Kernel density estimates are shown on the diagonal. Left: the partition minimizing the posterior expected VI lower bound [7] contains 3 clusters. Right: the lower bound in the 95% credible ball around the estimate contains five clusters.
[11] propose to compress the posterior on the partition space with a small number of weighted partition estimates reflecting the main clustering patterns in the posterior. This compressed posterior places the highest approximate posterior weight of 0.605 on the partition with three clusters in Figure 3(a), and a smaller, yet still substantial weight on the partition with four clusters Figure 3(b). Interestingly, this closely resembles the posterior summary provided by CliPS-DBSCAN in Figure 1(b).
Figure 3. Diabetes example: summary of the posterior partition structure based on MCMC draws from a static Bayesian mixture. Pair plots of the observed data colored by cluster, with the cross and ellipse illustrating the fitted mean and covariance conditioned on the estimated partition. Kernel density estimates are shown on the diagonal. WASABI [11] summarizes the posterior with a small number of particles (i.e. partitions) capturing different modes in the posterior. Left: the first particle contains 3 clusters and has a weight of 0.605. Right: the second particle contains 4 clusters and has a weight of 0.395.
Next, we consider the simulated example in [11], consisting of a two-dimensional Gaussian mixture with four components, each with mean located in one of the four quadrants and identity covariance matrix. Here , so that the clusters are only mildly separated. The authors mention that CliPS is appropriate for well separated clusters. Since the degree of cluster separation is rarely known in advance, it is also interesting to investigate how CliPS behaves in more challenging settings where the underlying structure is only weakly present.
MCMC draws are obtained by fitting a diagonal location-scale Dirichlet process mixture (DPM) [13]. The posterior on the number of clusters (Figure 4(a)) is concentrated around the range of 3 to 10 clusters. In CliPS, we set to the posterior mode. However, the non-permutation rate is high, and the CliPS summary provided in Figure 4(b), which is based on only 93 of the original 10,000 MCMC draws, provides a less informative summary of the true component-specific means at . As the authors suggest, when the non-permutation rate is high (as in this example), CliPS is not suitable and other summarization methods may be more appropriate (such as the WASABI summary in Figure 4(c)). But, this raises an interesting question: can we quantify how high the non-permutation rate should be for CliPS to provide an effective summary? Moreover, while not shown here for brevity, increasing the separation (e.g. ) substantially reduces the ambiguity in the posterior summary, increasing the non-permutation rate and making CliPS more effective. This raises a related interesting question for future work: can one characterize the degree of cluster separation required for CliPS to provide an effective summary?
Figure 4. Slightly multi-modal example: observations are generated from a two-dimensional Gaussian mixture with four components, each with mean located in one of the four quadrants and identity covariance matrix. MCMC draws are obtained by fitting a DPM of Gaussian. Top left: posterior on the number of clusters. Top right: CliPS with at the posterior mode retains only 93 of the initial 10,000 MCMC draws (the non-permutation rate is 0.964 and ). Bottom: WASABI summary with two particles, the first consists of a single cluster with a weight of 0.65, while the second contains 3 clusters with a weight of 0.35.
Further comments
I end with a few further comments.
What distribution do the retained, reordered draws characterize? The approach and resulting interpretations could be further strengthened by more formally characterizing the distribution that is approximated by the retained draws. The first filtering clearly retains draws that approximate the posterior distribution of parameters conditioned on clusters, but it is not as clear to understand how the final filtering step affects the target posterior distribution that is being approximated. A more formal characterization would help clarify the inferential target of the procedure and facilitate interpretation of the resulting cluster distributions.
The authors further propose to use a lower-dimensional functional of the component-specific parameters, and apply -means in this lower-dimensional space. They provide examples, such as clustering only the means and not the covariance matrix , which may be considered nuisance parameters in mixtures of multivariate Gaussians. In general, choosing an appropriate function is not straightforward, but it may also help to combat the curse of dimensionality in high-dimensional parameter spaces. In genomics, and other fields where high-dimensions are a concern, standard pipelines [14, 15] first apply
principal component analysis (PCA) and then cluster in the lower-dimensional space spanned by the selected number of principal components. Similar dimensionality-reduction strategies may prove useful within the CliPS framework, particularly when clustering is performed in high-dimensional parameter spaces where distance-based methods can become less reliable.
Declarations
Funding Statement
S.W. acknowledges funding by the Engineering and Physical Sciences Research Council (EP/Y028783/1) and by a Leverhulme Trust Research Fellowship (RF-2026-207\9).
Competing Interests Statement
The author declares no competing interests.
References
- [1]Malsiner-Walli, G., Frühwirth-Schnatter, S., Grün, B.: CliPS: How to identify cluster distributions in Bayesian mixture models. TEST, 1–36 (2026)
- [2]Ester, M., Kriegel, H.-P., Sander, J., Xu, X., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, vol. 96, pp. 226–231 (1996)
- [3]Scrucca, L., Fraley, C., Murphy, T.B., Raftery, A.E.: Model-based Clustering, Classification, and Density Estimation Using Mclust in R. Chapman & Hall/CRC Press, New York (2023)DOI
- [4]Frühwirth-Schnatter, S., Malsiner-Walli, G., Grün, B.: Generalized mixtures of finite mixtures and telescoping sampling. Bayesian Analysis 16(4), 1279–1307 (2021)
- [5]Lau, J.W., Green, P.J.: Bayesian model-based clustering procedures. Journal of Computational and Graphical Statistics 16, 526–558 (2007)DOI
- [6]Fritsch, A., Ickstadt, K.: Improved criteria for clustering based on the posterior similarity matrix. Bayesian Analysis 4, 367–392 (2009)DOI
- [7]Wade, S., Gharamani, Z.: Bayesian cluster analysis: Point estimation and credible balls (with discussion). Bayesian Analysis 13(2), 559–626 (2018)DOI
- [8]Rastelli, R., Friel, N.: Optimal Bayesian estimators for latent variable cluster models. Statistics and Computing 28(6), 1169–1186 (2018)DOI
- [9]Dahl, D.B., Johnson, D.J., Müller, P.: Search algorithms and loss functions for Bayesian clustering. Journal of Computational and Graphical Statistics 31(4), 1189–1201 (2022)arxiv.org/abs/2105.04451
- [10]Nguyen, K., Mueller, P.: Summarizing nonparametric Bayesian mixture posteriors—Sliced optimal transport metrics for Gaussian mixtures. Journal of Computational and Graphical Statistics, 1–12 (2026)DOI
- [11]Balocchi, C., Wade, S.: Understanding uncertainty in Bayesian cluster analysis. arXiv preprint arXiv:2506.16295 (2025)
- [13]Corradin, R., Canale, A., Nipoti, B.: BNPrmix: An R package for Bayesian nonparametric modeling via Pitman-Yor mixtures. Journal of Statistical Software 100(15), 1–33 (2021)DOI
- [14]Gribov, A., Sill, M., Lück, S., Rücker, F., Döhner, K., Bullinger, L., Benner, A., Unwin, A.: SEURAT: visual analytics for the integrated analysis of microarray data. BMC Medical Genomics 3(1), 21 (2010)DOI
- [15]Wolf, F.A., Angerer, P., Theis, F.J.: SCANPY: large-scale single-cell gene expression data analysis. Genome Biology 19(1), 15 (2018)DOI



