betaFeedback

WorksMathematics

On the discretization of the object space in inverse problems with application to cryo-electron microscopy

Abstract

In many inverse problems, the aim is to recover a probability distribution on a latent (or object state) space from indirect, noisy observations. When the observations can be modelled as noisy samples from the mixing latent distribution, the recovery problem is a deconvolution problem on the space of probability measures. A common strategy is to fix a finite set of candidate points and estimate a weight for each, turning the problem into a finite-dimensional concave maximum likelihood problem on the simplex. We study the combined effect of this discretization and of the noise in the context of cryo-electron microscopy (cryo-EM), where the candidate points are biomolecular conformations and the weights describe the relative frequency of each conformation. Our results pertain to both statistical and algorithmic aspects of the estimator. We analyze the weight-recovery problem in which the candidate states and their likelihoods are known. A nearby pair of candidates forces a near-null direction. More generally, the grid and the noise level impose a uniform lower bound on the achievable Kullback–Leibler divergence between observation densities, even with infinite data. The finite-grid estimator is asymptotically normal when its population target is in the interior of the simplex; at a boundary target, its limit is a cone-projected Gaussian. Finally, the exact proximal form of Expectation–Maximization leads to a global high-noise comparison between an early iterate and a KL-penalized likelihood, without a basin assumption or a linearization of the recursion. Tests on synthetic images of the Hsp90 molecule illustrate the theoretical findings and translate them into practical guidelines for interpreting reweighted ensembles.

Introduction

An objective of science is describing accurately how and why certain quantities or objects fluctuate. The prototypical example of interest for this paper is understanding different conformations of a molecule, i.e., the different states in which that molecule can be found in the nature and their probabilities. There usually is a continuum of such states and, if one had direct observations, the problem would be a distribution-estimation problem. We call this distribution the ensemble distribution; the standard usage in the mixture-model and empirical-Bayes literature would be mixture distribution, and the space on which it lives the latent space. For our purposes, the space is rather called the conformational space as latent space bears a different meaning in the cryo-EM community.

In the setting of interest, one does not have direct observations from the conformational space as data is usually stained by noise or only indirectly observed due to the imaging projection. Simplifying a bit, each observation is a noisy sample from the mixing distribution, with Gaussian noise of known variance. The density of the observations is therefore the convolution of the mixing distribution with a Gaussian kernel, and recovering the mixing distribution from the observations is a deconvolution problem on the space of probability measures. As no parametric form for the mixing distribution usually exists, various approaches (see Section 3.4 below for a review) approximate it by a discrete distribution, i.e., a weighted sum of Dirac masses supported on a finite set of candidate points, as done by Tang et al. (2023) [43] in cryo-EM to cite but one. The weights are then estimated from the data, usually by maximum likelihood. The candidate points can come from domain knowledge, from numerical simulations such as molecular dynamics (MD), or from a regular grid. In Section 3.4, we further discuss the importance of discretization in a variety of schemes.

This paper is about the combined effect of the Gaussian noise and of the discretization step. Once the support is fixed, the maximum likelihood problem becomes a finite-dimensional concave optimization on the simplex of weights, which is computationally convenient.

We argue that the noise level and the geometry of the candidate points interact in ways a practitioner should know about. How many candidate points there are, how far apart they sit, and how well they cover the conformational space, together with the noise level σ\sigma, control three things at once: the conditioning of the optimization problem, the best KL approximation of the observation density that can be achieved even with infinite data, and the way reconstructed weights relate to the empirical frequencies (“populations” in the cryo-EM literature) one would estimate from the data. In a cryo-EM context this means, concretely, that two candidate structures which are close compared to the noise level can exchange weight almost freely without changing the likelihood, that an MD run which fails to explore a region of the landscape produces a population observation-density KL floor that more data cannot lower, and that large reconstructed weights should not be read directly as large conformational probabilities. Quantifying these statements is the goal of the paper.

More precisely, we explain

  • why a nearby pair of candidates forces a nearly flat direction for the optimization, at a scale controlled by their spacing relative to the noise;

  • why the candidate points (obtained for instance from MD), together with the noise level, influence the best achievable KL divergence, even with infinite data;

  • why sparse fitted weights need not correspond to large conformational probabilities;

  • what the statistical properties of the estimator are.

The main theoretical results include the uniform fixed-grid bound of Theorem 1, the global early-EM comparison of Proposition 6, the additional-noise stability result of Theorem 2, and the constrained limit distribution of Theorem 3.

Besides these statistical aspects, the solver used can subtly influence the outcome. The Expectation-Maximization (EM) algorithm is a popular way to solve this optimization problem, even though off-the-shelf convex solvers could also provide a solution. The EM algorithm is easy to implement and has a small computational cost per iteration.

The maximum-likelihood objective is concave. The fitted vector Kα^K\hat{\alpha} is unique, and the weights are unique under the tangent-injectivity condition introduced below. The estimate can nevertheless be sparse or spiky; see [27] (2020, Section 3) or panel f. of Figure 1.

Figure 1. The estimation of the weights as a function of the EM iteration, illustrated on the cryo-EM Hsp90 example from Section 7, with n=100,000n = 100{,}000 images and M=20M = 20 candidates, where the distribution is shown as a function of the chain opening collective variable θ\theta. The blue curve is the displayed truncated target density; normalized grid weights are used when a weight divergence is reported. a. Molecule Hsp90 of interest for the cryo-EM simulations. The motion simulated is the opening and closing of the angle between the two chains, illustrated by the transparent models and collective variable θ\theta. b. Example of simulated cryo-EM images. c. Evolution of a scaled negative log-likelihood as a function of the number of iterations (log scale). d. Estimation of the frequency (population) of each structure based on the number of nearest neighbors. e. Estimated weights at iteration number 5000, 50,000 and 150,000, respectively, of the EM algorithm. The sparsity pattern evolves towards the SCS fit in panel f. f. Convex-optimization fit produced by CVXPY/SCS with absolute and relative tolerances 10−510^{-5}.

We explain why EM can become slow when components are similar or the optimum lies on the boundary. From a positive initialization it stays in the relative interior, and in the high-noise regime its proximal interpretation makes precise how early stopping can regularize the weights. We also study practical stopping diagnostics.

Organization. We start with a more formal description of the problem that we tackle in Section 2. Then, Section 3 provides additional background: we introduce cryo-EM (Sec. 3.1), the statistical-physics view of conformational distributions (Sec. 3.2), the ensemble-reweighting scheme of [43] (2023) and its connection with nonparametric maximum likelihood (Sec. 3.3), and related work in cryo-EM (Sec. 3.4). Section 4 analyses the objective function: we compute its gradient and Hessian and quantify the ill-conditioning caused by close structures (Sec. 4.1), establish self-concordance, establish a fundamental KL lower bound when the grid fails to cover the conformational space (Sec. 4.3), and close with an exact clusterwise description of how empirical frequencies enter the objective (Sec. 4.4). Section 5 is devoted to the Expectation–Maximization algorithm used in practice: we explain when early stopping can induce an implicit regularization, give a proximal-point interpretation, and discuss stopping criteria. Section 6 studies stability under an additional noise component and a central limit theorem for the estimated weights α^\hat{\alpha}. Section 7 illustrates the theoretical findings through simulations on the Hsp90 molecule. We finalize with a conclusions Section 8. The remaining proofs are deferred to Appendix A. An additional example illustrating theoretical findings, the IgG molecule, is provided in Appendix B.

Problem formulation

In this section, we formally introduce the problem and fix notation for the rest of the paper. A mixing distribution with density ρ:Rd→R\rho:\mathbb{R}^{d} \to\mathbb{R} is given, and n∈Nn \in\mathbb{N} i.i.d. sample points x1,…,xn\boldsymbol{x}_1,\ldots,\boldsymbol{x}_n are drawn from it. One does not observe the xi\boldsymbol{x}_i directly, but rather yi=xi+ϵiy_i = \boldsymbol{x}_i + \epsilon_i, where the ϵi\epsilon_i are independent Gaussian random vectors with covariance σ2Id\sigma^2 I_d. This is the weight-recovery subproblem obtained when the forward operators and candidate structures are treated as known; we expect the same mechanisms to remain relevant in the full cryo-EM problem.

Write pσp_{\sigma} for the density of N(x,σ2Id)\mathcal{N}(x,\sigma^{2}I_d). In the simplified model we consider, each yiy_i has density

∫Rdpσ(y∣x) ρ(x) dx,\int_{\mathbb{R}^{d}} p_{\sigma}(y\mid x)\,\rho(x)\,\mathrm{d}x,

the convolution of ρ\rho with the Gaussian kernel.

The measure ρ\rho will be estimated by a discrete measure of the form ∑m=1Mαmδxm\sum_{m=1}^{M}\alpha_m\delta_{x_m}, where the points x1,…,xMx_1,\ldots,x_M are given and α\alpha lies in the simplex ΔM−1:={α∈RM:αm≥0, ∑mαm=1}\Delta_{M-1}:=\{\alpha\in\mathbb{R}^{M}:\alpha_m\ge0,\ \sum_m\alpha_m=1\}. The associated maximum likelihood problem is

max⁡α∈ΔM−1F(α),F(α):=1n∑i=1nlog⁡(∑m=1Mpσ(yi∣xm)αm).(1)\max_{\alpha\in\Delta_{M-1}} F(\alpha),\qquad F(\alpha):=\frac{1}{n}\sum_{i=1}^{n}\log\left(\sum_{m=1}^{M}p_{\sigma}(y_i\mid x_m)\alpha_m\right). \tag*{(1)}

Fixing the points xmx_m rather than optimizing over them is common practice in integrative structural biology methods that rely on reweighting structural ensembles against experimental data [6]. Finding the optimal α\alpha then amounts to reweighting the structures. In the running example of this work, each xmx_m represents a conformation of Hsp90. The latter are indexed by an opening angle θ\theta and the αm\alpha_m’s would be the relative proportions. General background on this application is provided in Section 3 and the particular running example is shown on Figure 1 and developed in Section 7.1.

Background

The goal of this paper is to analyze the properties and limitations of a previously introduced denoising method [43, 15] for cryo-EM data. This method leverages noisy experimental data as well as output of numerical simulations, such as MD, and combines both sources with the objective of retrieving the most accurate representation of the underlying probability density of the molecules. Though our focus is on the reweighting scheme, our results extend beyond the method analyzed here, and we give related work in Section 3.4.

Cryo-electron microscopy

Biomolecules undergo changes between various conformations to perform different functions. An important research problem is to understand the various conformations a biomolecule can take and subsequently infer the underlying biophysical mechanisms. These conformations can be studied through cryo-EM, a leading technique for imaging biomolecules at near atomic resolution [34]. This method has become a fundamental scientific tool: the 2017 Nobel prize in chemistry was awarded to Dubochet, Frank and Henderson for the development of cryo-EM, and the experiment was labeled a “Method to Watch” by Nature Methods in 2022 [14]. In cryo-EM, a solution of many copies of biomolecule is flash frozen and imaged with a transmission electron microscope. Due to fast freezing, the biomolecules remain in a near-native state and the resulting images capture noisy snapshots of various conformations. This contrasts with crystallization-based techniques that restrict the conformational variability.

Analyzing the acquired images is very challenging. As the molecules are flash-frozen, each is in a random unknown orientation. Also, one only observes a two-dimensional view of the 3D volume. Finally, the noise level is extremely high, which makes the problem of recovering the conformation landscape, i.e., the set of possible conformations, more intricate. Usually, the image formation process is modeled as in (2).

Our analysis is motivated by the forward model for heterogeneous cryo-EM datasets. In this forward model, one considers a dataset of nn images, where each image yiy_i is a projection of an (unknown) volume ViV_i corresponding to the electrostatic potential of the molecule of interest in the ii-th conformation. The collection of acquired images {yi}i=1n\{y_i\}_{i=1}^{n} is then modeled as

yi=(PSFi∘Ti∘P∘Ri)Vi+ηi,i∈{1,…,n},(2)y_i = (PSF_i \circ T_i \circ P \circ R_i)V_i + \eta_i,\qquad i \in\{1,\ldots,n\}, \tag*{(2)}

where RiR_i is a 3D rotation operator corresponding to the random orientation of the volume ViV_i, PP is the 2D projection operator and TiT_i is a 2D translation operator corresponding to the offset of the projected volume with respect to the center of the image. Both RiR_i and TiT_i are specific to each particle image yiy_i and are unknown. PSFiPSF_i is the Point Spread Function operator applied to the projected image and can be thought of as a convolution coming from the fact that the microscope is imperfect and distorts the signal. Finally, ηi\eta_i is some random noise, often assumed to be Gaussian, the covariance matrix of which we do not specify for now. For initial processing of an image dataset, PSFiPSF_i, TiT_i, RiR_i are all unknown in addition to ViV_i for i=1,…,ni = 1,\ldots,n, and must be estimated, often through alternating minimization using a likelihood similar to (3) [42].

The composition of the various functions appearing in (2) is called the forward operator, which we denote Img⁡Ti,Ri(⋅)\operatorname{Img}_{T_i,R_i}(\cdot). In practice, the conditional density pσ(yi∣Img⁡Ti,Ri(V))p_{\sigma}(y_i \mid\operatorname{Img}_{T_i,R_i}(V)) for a volume VV, usually is taken to be Gaussian, i.e.,

pσ(yi∣Img⁡Ti,Ri(V)):=(2πσ2)−N/2exp⁡(−∥yi−Img⁡Ti,Ri(V)∥22σ2),(3)p_{\sigma}(y_i \mid\operatorname{Img}_{T_i,R_i}(V)) := (2\pi\sigma^2)^{-N/2}\exp\left(-\frac{\lVert y_i-\operatorname{Img}_{T_i,R_i}(V)\rVert^2}{2\sigma^2}\right), \tag*{(3)}

where NN is the number of pixels, and σ\sigma is the fixed noise level, which we assume is known. We note that we take the cryo-EM forward operator (2) and likelihood (3) as our inspiration, but for the analysis we will consider an identity forward operator and Gaussian likelihood with large variance, as explained in Section 3.3.

Modeling the conformational ensemble distribution

A fundamental idea of statistical physics is that the variations of the configuration of a molecule, i.e., the different conformations should depend on the energy level of the configuration and the temperature. That is, one expects to observe only configurations that are stable and thus have a low energy level. At higher temperatures, as the system gets more excited, more variability can be expected. The probability distribution underlying the configurations is modeled as the Boltzmann distribution

ρ(x)=1Z0e−βH(x),\rho(x) = \frac{1}{Z_0}e^{-\beta\mathcal{H}(x)},

where H\mathcal{H} is the molecular Hamiltonian characterizing the energy, Z0Z_0 is the normalizing constant and β\beta the inverse temperature parameter. Unfortunately, the Hamiltonian is not known exactly, which is why understanding of molecular conformations is still an active topic of research.

In parallel to the experiments and the data they deliver, computational advances from MD ([24]) or AlphaFold ([1]) for instance, enable one to predict such conformations. A natural direction is then to combine the strength of those numerical methods with the datasets available to obtain the most accurate reconstructions possible and eventually improve the understanding of the conformational landscape. This strand of ideas is known as integrative biology, see [45] and [7]. This combination is important, as the computational methods are only providing approximations of the unknown truth and might exhibit biases.

Ensemble reweighting as nonparametric maximum likelihood

An integrative approach as the one above is put forward in [43] and subsequent works ([41]). Their procedure is the one that we focus on in this paper, even though some of the phenomena we highlight are relevant for other types of analyses. Let us stress that we get rid of the issue of the rotations and projections for the analyses of the present work.

[43] propose the approach below. For the analysis conducted in this paper, we work with a simplified version of their setting: the cryo-EM forward operator of equation (2) is replaced by the identity, the random rotations, translations and PSF are discarded, and the noise is assumed to be isotropic Gaussian with known variance σ2\sigma^2. This simplification lets us isolate the statistical properties of the reweighting step itself, independently of the pose-estimation and PSF calibration problems, and study ill-conditioning, early-stopping regularization, and spike formation within this subproblem. We assume that an i.i.d. sample of images can be produced, call it Y:={yi}i=1nY := \{y_i\}_{i=1}^n. For simplicity, assume again that the images are vectorized and live on RN\mathbb{R}^N where NN is the total number of pixels.

Let us introduce the initial candidate probability density of the conformations ρ0(x)\rho_0(x), which could come from a model or any educated guess. Then, to incorporate the information contained in the data, a class of functions h(x;α)h(x;\alpha) depending on α\alpha is introduced. The α\alpha parameter will be learnt so that h(x;α)ρ0(x)h(x;\alpha)\rho_0(x) describes more accurately the conformational space. One can then write the conditional likelihood of the data given the reweighting function as

pσ(Y∣α)=∏i=1n(∫pσ(yi∣x)h(x;α)ρ0(x) dx).p_{\sigma}(Y \mid\alpha) = \prod_{i=1}^{n} \left(\int p_{\sigma}(y_i \mid x)h(x;\alpha)\rho_0(x)\,\mathrm{d}x\right).

As a parametric form for h(x;α)h(x;\alpha) is not to be expected, the authors propose to consider a discrete counterpart of the likelihood, which reads

p~σ(Y∣α)=∏i=1n(∑m=1Mpσ(yi∣xm)αm),(4)\widetilde{p}_{\sigma}(Y \mid\alpha) = \prod_{i=1}^{n} \left(\sum_{m=1}^{M} p_{\sigma}(y_i \mid x_m)\alpha_m\right), \tag*{(4)}

where the vector α\alpha lies in the (M−1)(M-1)-simplex ΔM−1\Delta_{M-1} and {xm}m=1M\{x_m\}_{m=1}^{M} is a representative subset of the possible conformations, for instance a discrete conformational path, which will be obtained from MD simulations. The weight vector α\alpha now plays the role of the reweighting density h(x;α)h(x;\alpha) above. Finally note that

α^n∈arg max⁡α∈ΔM−1 p~σ(Y∣α)=arg max⁡α∈ΔM−1 1n∑i=1nlog⁡(∑m=1Mpσ(yi∣xm)αm),(5)\widehat{\alpha}_n \in\underset{\alpha\in\Delta_{M-1}}{\operatorname{arg\,max}}\,\widetilde{p}_{\sigma}(Y \mid\alpha) = \underset{\alpha\in\Delta_{M-1}}{\operatorname{arg\,max}}\,\frac{1}{n}\sum_{i=1}^{n}\log\left(\sum_{m=1}^{M}p_{\sigma}(y_i \mid x_m)\alpha_m\right), \tag*{(5)}

which is convenient in many cases as noted by the authors.

Let us take a step back and start again from the fact that no parametric form for h(x,α)h(x,\alpha) could be postulated. Indeed, no explicit form of an α\alpha-parametrized Hamiltonian Hα(x)\mathcal{H}_{\alpha}(x) is usually available or sufficiently accurate1, so that a parametrization of the form h(x;α)ρ0(x)∝e−Hα(x)h(x;\alpha)\rho_0(x) \propto e^{-\mathcal{H}_{\alpha}(x)} is not feasible. Still, following the principle of maximum likelihood, one could then maximize the log-likelihood over all possible distributions, i.e.,

max⁡q∈P 1n∑i=1nlog⁡∫pσ(yi∣u) dq(u),(6)\underset{q \in\mathcal{P}}{\operatorname{max}}\ \frac{1}{n}\sum_{i=1}^{n}\log\int p_{\sigma}(y_i \mid u)\,\mathrm{d}q(u), \tag*{(6)}

where P\mathcal{P} is the set of probability distributions supported on RN\mathbb{R}^{N}. This problem is classical in statistics, known as the nonparametric maximum likelihood estimation problem, which at least dates back to [38]. A book-long exposition of the subject is [30] and the interested reader might profit from Chapters 1, 5 and 6 in particular.

This problem has been studied much over the past decades and keeps on being an active research field. We refer to the lecture notes by [20]. An important fact is that the objective functional is concave in qq. As it is an infinite dimensional problem, however, modifications have to be performed to make the problem computationally-friendly. Following [29], [22] considered the idea of restricting the set of measures in the optimization problem above by fixing the support of qq to a (large) discrete set of points, say {xm}m=1M\{x_m\}_{m=1}^{M}. With the set of points fixed, maximizing over all measures is then equivalent to optimizing the measures weights as is suggested above.

This is the first takeaway: fixing the support to a discrete grid is a classical and principled way of making the NPMLE problem tractable. In our setting, this choice is not made freely: the grid {xm}\{x_m\} is determined by MD simulations, which is precisely what makes the reweighting problem an instance of integrative biology instead of a purely statistical question.

Having to optimize over the weights only is a great advantage: the log-likelihood problem (5) is concave and finite-dimensional; classical optimization tools can thus be used. To that end, [28] proposed to solve that problem using duality and an interior point solver, for instance. The more recent algorithm by [27] reports better performance in the case n≫Mn \gg M. Finally, [46] further improved on the computations.

Within cryo-EM workflows, the reweighting and discretization updates analyzed here serve as foundational building blocks for several existing algorithms and pipelines. Both in cryoSPARC ([36]) and RELION ([39]), two dominant software tools in the field, the conformational space is discretized, although structures are not given. In the Flatiron Challenge for Cryo-EM Conformational Heterogeneity ([2]), the participants were required to submit a discrete set of structures, i.e., provide a discrete estimate of the conformational space. In multi-class expectation maximization such as in FREALIGN ([32]), the scheme alternates between the reweighting update and an optimization step for the volumes of 3D classes and corresponds to soft assignment of particles to structures. The relationship to hard assignment is discussed in [15]. Similarly, the reweighting scheme mentioned above corresponds to an E step in another EM algorithm used to learn both structures and denoise at the same time, see for instance [4]. Thus, our results inform that scheme as well, even though capturing only part of the subtleties recently put forward. We further note that other empirical Bayes problems arise in cryo-EM: [18] apply a PCA approach which induces a hierarchical linear model with heteroscedastic noise on latent cryo-EM volumes.

Related work also considers Bayesian approaches to compare MD structures with cryo-EM images using the same likelihood as (4) ([43, 19]) or focusing on calculating (3) more efficiently when the pose is unknown [13]. More recently, [33] introduce an information-theoretic framework to determine the optimal discretization of conformational space in cryo-EM ensemble reweighting. By maximizing mutual information between structural weights and noisy simulated images, it is shown that measurement noise sets a limit on the ensemble resolution, enabling the selection of representative structures that capture molecular heterogeneity without spatial redundancy.

From the inverse-problems viewpoint, the fixed candidate set is a discretization of the unknown measure, and the EM iteration count can act as a smoothing or iterativeregularization parameter; Proposition 6 quantifies one high-noise sense in which this occurs. Discretization effects have long been studied in statistical inverse problems [23, ?]bertero2009_image_deblurring_with``. Our focus is the interaction of these two effects in the finite-grid weight problem.

Properties of the MLE objective function

As discussed above, the reweighting problem is an optimization problem. It is therefore fundamental to study the properties of the objective function.

Gradient, Hessian and optimality condition.

The objective function F(α)F(\alpha) has a specific structure that determines computational tractability. We now characterize its curvature properties, which govern both optimization difficulty and statistical efficiency. Set Kij:=pσ(yi∣xj)K_{ij} := p_{\sigma}(y_i \mid x_j), s(α):=Kαs(\alpha) := K\alpha, and T:=1M⊥\mathcal{T} := \mathbf{1}_M^{\perp}.

Proposition 1. For every α∈ΔM−1\alpha\in\Delta_{M-1},

∇F(α)=1nKTs(α)−1,\nabla F(\alpha) = \frac{1}{n}K^{\mathsf{T}}s(\alpha)^{-1},

and

∇2F(α)=−1nKTdiag⁡(si(α)−2)K,(7)\nabla^2 F(\alpha) = -\frac{1}{n}K^{\mathsf{T}}\operatorname{diag}\left(s_i(\alpha)^{-2}\right)K, \tag*{(7)}

where the inverse is understood componentwise. Thus FF is concave, and it is strictly concave on the simplex when

ker⁡(K)∩T={0}.(8)\operatorname{ker}(K) \cap\mathcal{T} = \{0\}. \tag*{(8)}

In that case the maximizer is unique; otherwise, the fitted vector Kα^K\hat{\alpha} is still unique.

Note that condition (8) can hold only if M−1≤nM - 1 \le n. Moreover, it holds that

⟨∇F(α),α⟩=1.(9)\langle\nabla F(\alpha), \alpha\rangle= 1. \tag*{(9)}

Hence the KKT conditions are

∇F(α^)−1M+μ^=0,μ^≥0,α^≥0,μ^jα^j=0,1MTα^=1.(10)\nabla F(\hat{\alpha}) - \mathbf{1}_M + \hat{\mu} = 0,\qquad\hat{\mu} \ge0,\qquad\hat{\alpha} \ge0,\qquad\hat{\mu}_j\hat{\alpha}_j = 0,\qquad\mathbf{1}_M^{\mathsf{T}}\hat{\alpha} = 1. \tag*{(10)}

Equivalently, [∇F(α^)]j=1[\nabla F(\hat{\alpha})]_j = 1 if α^j>0\hat{\alpha}_j > 0, while [∇F(α^)]j≤1[\nabla F(\hat{\alpha})]_j \le1 if α^j=0\hat{\alpha}_j = 0. The following proposition quantifies the loss of curvature when two structures are close. Define the smallest tangent curvature by

λmin⁡T(−∇2F(α)):=min⁡v∈T∥v∥=1vT[−∇2F(α)]v.\lambda_{\min}^{\mathcal{T}}\left(-\nabla^2 F(\alpha)\right) := \min_{\substack{v \in\mathcal{T}\\ \lVert v\rVert= 1}} v^{\mathsf{T}}\left[-\nabla^2 F(\alpha)\right]v.

Proposition 2. For distinct candidate indices kk and ℓ\ell, let FhF_h, h∈Rdh \in\mathbb{R}^d, denote the objective in (1) obtained by replacing xkx_k with xℓ+hx_\ell+ h, while keeping the data, σ\sigma, and the other candidate points fixed. For the Gaussian kernel,

sup⁡α∈ΔM−1λmin⁡T(−∇2Fh(α))=O(∥h∥2),h→0.\sup_{\alpha\in\Delta_{M-1}} \lambda_{\min}^{\mathcal{T}}\left(-\nabla^2 F_h(\alpha)\right) = O\left(\lVert h\rVert^2\right),\qquad h \to0.

This implies that weight can be shifted between two close structures with negligible effect on the objective.

Under the high-noise scaling Yi=μi+σZiY_i = \mu_i + \sigma Z_i, with Zi∼iidN(0,Id)Z_i \overset{\mathrm{iid}}{\sim} \mathcal{N}(0,I_d) and μi\mu_i, xℓx_\ell, and the remaining candidate points fixed, the same left-hand side is Op(∥h∥2/σ2)O_p(\lVert h\rVert^2/\sigma^2) as σ→∞\sigma\to\infty, uniformly for hh in a fixed neighborhood of zero.

Self-concordance of the objective

Importantly, the objective function exhibits a special structure called self-concordance. Self-concordance is a particularly convenient property for the analysis of the Newton algorithm, see for example [8].

Definition 1. Let Ω⊆RM\Omega\subseteq\mathbb{R}^M be open and convex. A convex, three-times differentiable function f:Ω→Rf:\Omega\to\mathbb{R} is called MfM_f-self-concordant if

∣D3f(x)[u,u,u]∣≤Mf(D2f(x)[u,u])3/2\left|D^3f(x)[u,u,u]\right| \le M_f\left(D^2f(x)[u,u]\right)^{3/2}

for every x∈Ωx \in\Omega and u∈RMu \in\mathbb{R}^M. The case Mf=2M_f = 2 is the standard normalization.

Proposition 3. Let

Ω:={α∈RM:Ki,∙Tα>0, i=1,…,n}.\Omega:= \left\{\alpha\in\mathbb{R}^M : K_{i,\bullet}^{\mathsf{T}}\alpha> 0,\ i = 1,\ldots,n\right\}.

The negative objective f(α):=−F(α)f(\alpha) := -F(\alpha) is 2n2\sqrt{n}-self-concordant on Ω\Omega. Equivalently, −nF-nF is standard self-concordant. Its Hessian is positive definite on the simplex tangent space exactly under (8).

Fundamental limits of grid-based estimation

We first quantify the population likelihood gap created when the candidate points cannot match the target moments.

For α∈ΔM−1\alpha\in\Delta_{M-1}, set PMα:=∑mαmδxmP_M^\alpha:= \sum_m \alpha_m\delta_{x_m}. At the population level,

∫(pσ∗ρ)(y)log⁡(pσ∗PMα)(y) dy=∫(pσ∗ρ)(y)log⁡(pσ∗ρ)(y) dy−KL⁡(pσ∗ρ∥pσ∗PMα).\int(p_\sigma* \rho)(y)\log(p_\sigma* P_M^\alpha)(y)\,\mathrm{d}y = \int(p_\sigma* \rho)(y)\log(p_\sigma* \rho)(y)\,\mathrm{d}y - \operatorname{KL}(p_\sigma* \rho\mathbin{\|} p_\sigma* P_M^\alpha).

Thus, the population maximum likelihood minimizes this KL divergence.

For κ=(κ1,…,κd)∈N0d\kappa= (\kappa_1,\ldots,\kappa_d) \in\mathbb{N}_0^d, put ∣κ∣=∑i=1dκi|\kappa| = \sum_{i=1}^d \kappa_i and xκ=∏i=1dxiκix^\kappa= \prod_{i=1}^d x_i^{\kappa_i}. We further place ourselves in the high noise case, i.e., one should think of σ2\sigma^2 large with respect to the signal in the images, which is what one expects in cryo-EM.

As we have seen, maximizing the objective function is equivalent to minimizing the Kullback–Leibler divergence. Even in the case of infinite data, the latter cannot be zero unless

ρ∈{∑m=1Mαmδxm:α∈ΔM−1}.\rho\in\left\{\sum_{m=1}^{M}\alpha_m\delta_{x_m}:\alpha\in\Delta_{M-1}\right\}.

Let us also define

m:=sup⁡{K∈N0:∃α∈ΔM−1 such that ∑m=1Mαmxmκ=∫xκ dρ(x) for every ∣κ∣≤K}∈N0∪{+∞}.(11)\mathbf{m} := \sup\left\{ \mathcal{K} \in\mathbb{N}_0 : \exists\alpha\in\Delta_{M-1} \text{ such that } \sum_{m=1}^{M} \alpha_m \mathbf{x}_m^{\boldsymbol{\kappa}} = \int x^{\boldsymbol{\kappa}} \,d\rho(x) \text{ for every } \lvert\boldsymbol{\kappa} \rvert\le\mathcal{K} \right\} \in\mathbb{N}_0 \cup\{+\infty\}. \tag*{(11)}

More intuitively, m\mathbf{m} is the largest degree up to which all moments of ρ\rho can be matched simultaneously by PMαP^\alpha_M for a suitable choice of α\alpha. Under the bounded-support assumption below, m=+∞\mathbf{m} = +\infty if and only if ρ=PMα\rho= P^\alpha_M for some α\alpha; this follows from compactness of the simplex and moment determinacy on compact sets. The theorem concerns the remaining case.

Theorem 1. Suppose that ρ\rho has bounded support and let m<∞\mathbf{m} < \infty be defined by (11). Then, for constants c>0c > 0 and σ0<∞\sigma_0 < \infty depending only on ρ\rho and the candidate points,

inf⁡α∈ΔM−1KL⁡(pσ∗ρ∥pσ∗PMα)≥cσ−2(m+1),σ≥σ0.(12)\inf_{\alpha\in\Delta_{M-1}} \operatorname{KL}(p_\sigma* \rho\mathbin\Vert p_\sigma* P^\alpha_M) \ge c \sigma^{-2(m+1)}, \qquad\sigma\ge\sigma_0. \tag*{(12)}

The theorem shows that an incomplete grid leaves a nonzero approximation error, even with infinite data, in the forward KL divergence between the true observation density and the best convolved candidate mixture.

(12) also reveals that a higher noise level σ\sigma reduces the lower bound. This does not mean that the estimate gets better with higher noise level. As σ→∞\sigma\to\infty, the signal is progressively lost and different weight vectors become increasingly difficult to distinguish. One could say that “if you squint your eyes enough, everything looks the same”. Seen differently, the objective function gets flat.

The moment-matching picture in Theorem 1 is closely related to the low-SNR expansions of [25] and [16], who study Gaussian mixture log-likelihoods with unknown centers. Our setting differs: the candidate points are fixed, and the inference is over the simplex of weights. The same moment-matching obstruction reappears, now in terms of which moments of ρ\rho the fixed grid can reproduce. For a fixed pair of Gaussian-smoothed measures, [9] obtain exact high-noise KL asymptotics. The point of Theorem 1 is different: its one-sided bound is uniform over the simplex, so even weights allowed to depend on σ\sigma cannot remove the moment obstruction imposed by the fixed grid.

The practical implication is the following. As there are (d+md)−1\binom{d+m}{d} - 1 nonconstant coordinate moments through order m\mathbf{m} and since an MM-point measure has M−1M - 1 free weights, a generic moment vector requires, for fixed dd, MM of order mdm^d. This necessary dimension-counting condition might not be a sufficient condition for a fixed grid. It further holds in the case of infinite data. However, as the conditioning deteriorates with structures close to one another, a smaller number of structures might deliver better results in finite sample applications. Initial work has confirmed this in the integrative structural biology setting, where [33] sub-sample structures through clustering procedures that stabilize quantities related to the Hessian.

Empirical frequencies and candidate geometry

The optimizer can lie on the boundary and be spiky. We next consider a correctly specified on-grid model and describe how empirical frequencies and candidate geometry enter the objective.

To understand the role of empirical frequencies, condition on labels zi∈{1,…,M}z_i \in\{1,\ldots,M\} and suppose that the observations are independent with Yi∣zi=k∼N(xk,σ2Id)Y_i \mid z_i = k \sim\mathcal{N}(x_k,\sigma^2 I_d). Set Nk:=#{i:zi=k}N_k := \#\{i : z_i = k\} and Rx:=max⁡k,j∥xj−xk∥R_x := \max_{k,j} \lVert x_j - x_k \rVert.

Proposition 4 (Clusterwise representation). Assume that MM, dd and the candidate centers are fixed, and fix σ>0\sigma> 0. If, as n→∞n \to\infty, Nk/n→πk>0N_k/n \to\pi_k > 0, then, conditionally on the labels, uniformly in α∈ΔM−1\alpha\in\Delta_{M-1},

F(α)=−d2{1+log⁡(2πσ2)}−∑k=1MNknKL⁡(pσ(⋅∣xk)∥pσ∗PMα)+Op(n−1/2).(13)F(\alpha) = -\frac{d}{2}\{1+\log(2\pi\sigma^2)\}-\sum_{k=1}^{M}\frac{N_k}{n}\operatorname{KL}(p_\sigma(\cdot\mid x_k)\Vert p_\sigma* P_M^\alpha)+O_p(n^{-1/2}). \tag*{(13)}

Remark 1 (High-noise consequence). The first-order case of [25] (2023, Theorem 2.1), applied with truth δxk\delta_{x_k} and candidate measure PMαP_M^\alpha, gives, uniformly in kk and α\alpha,

KL⁡(pσ(⋅∣xk)∥pσ∗PMα)=∥∑jαjxj−xk∥22σ2+OM,d(Rx4σ−4),σ≥Rx.(14)\operatorname{KL}(p_\sigma(\cdot\mid x_k)\Vert p_\sigma* P_M^\alpha)=\frac{\left\lVert\sum_j\alpha_jx_j-x_k\right\rVert^2}{2\sigma^2}+O_{M,d}(R_x^4\sigma^{-4}),\qquad\sigma\ge R_x. \tag*{(14)}

Writing xˉn:=∑k(Nk/n)xk\bar{x}_n := \sum_k(N_k/n)x_k, combining the two displays gives, for fixed σ≥Rx\sigma\ge R_x and up to terms independent of α\alpha,

F(α)=−12σ2∥∑jαjxj−xˉn∥2+OM,d(Rx4σ−4)+Op(n−1/2).F(\alpha)=-\frac{1}{2\sigma^2}\left\lVert\sum_j\alpha_jx_j-\bar{x}_n\right\rVert^2+O_{M,d}(R_x^4\sigma^{-4})+O_p(n^{-1/2}).

Thus the σ−2\sigma^{-2} term is maximized by the weights whose barycenter is xˉn\bar{x}_n. Empirical frequencies determine this barycenter; the candidate geometry determines which weights share it.

Expectation-Maximization and early-stopping regularization

There is a long tradition of solving mixture problems with the expectation-maximization algorithm. The history goes back at least 50 years, with application in fields such as astrophysics, medical imaging, finance, and information theory, see for example [31], [44], and [12]. We refer to [37] and [3] for a complete introduction, the underlying motivation and further background.

The EM algorithm updates the weight vectors

αj(t+1)=1n∑i=1nKijαj(t)∑m=1MKimαm(t),t=0,1,….(15)\alpha_j^{(t+1)}=\frac{1}{n}\sum_{i=1}^{n}\frac{K_{ij}\alpha_j^{(t)}}{\sum_{m=1}^{M}K_{im}\alpha_m^{(t)}},\qquad t=0,1,\ldots. \tag*{(15)}

Write u:=1M/Mu := 1_M/M and initialize at α(0)=u\alpha^{(0)} = u. We use TT for the number of iterations. Define

rm(y;α):=αmpσ(y∣xm)∑ℓ=1Mαℓpσ(y∣xℓ),Φn(α):=1n∑i=1nr(yi;α),(16)r_m(y;\alpha) := \frac{\alpha_m p_\sigma(y \mid x_m)}{\sum_{\ell=1}^{M} \alpha_\ell p_\sigma(y \mid x_\ell)}, \qquad\Phi_n(\alpha) := \frac{1}{n}\sum_{i=1}^{n} r(y_i;\alpha), \tag*{(16)}

so that the recursion reads α(t+1)=Φn(α(t))\alpha^{(t+1)} = \Phi_n(\alpha^{(t)}). This nonlinear recursion remains particularly challenging to study. Also note that rr are usually called responsibilities.

This algorithm, in spite of its great popularity in practice, can be slow in some settings. Indeed, [30] stated: “However, one key feature of the algorithm is that it commonly displays a very slow linear rate of convergence, where the rate constant is related to the amount of information in the missing portion of the data. If the components are similar in their densities, then the convergence is extremely slow. The convergence will also be slow when the maximum likelihood solution requires some of the weight parameters to be zero, because the algorithm can never reach such a boundary point.”

As hinted at in the Introduction, we aim at raising awareness in the community that there can be a regularization phenomenon through early stopping, as the EM might not be run long enough in the case of very small or zero optimal weights. This might however be a feature for the applications of interest. Recall that other fast convex-optimization methods exist and were discussed in Section 3.3; they can produce the same spiky boundary solutions in our examples (see Figure 1 panel f.). The stopping time along the EM path matters, as the spikiness of the converged fit is both a blessing and a curse. The converged fit indeed is adaptive, but might also introduce non-physical spikes. We believe that a properly early-stopped EM algorithm might provide the best of both worlds.

As detailed in Section 7.1, we use the Hsp90 system and a simplified cryo-EM simulation setup as an example to recover a distribution over the chain opening angle θ\theta (shown in blue in Figure 1). That function is discretized and its atoms are used as the candidates to reweight. The figure follows EM towards the convex-optimization fit produced by SCS. The latter is sparser than the displayed target, which is not desirable in this example.

Remark that the update can be rewritten

α(t+1)=α(t)+α(t)⊙(∇F(α(t))−1M),(17)\alpha^{(t+1)} = \alpha^{(t)} + \alpha^{(t)} \odot\left(\nabla F(\alpha^{(t)}) - 1_M\right), \tag*{(17)}

which is a diagonally scaled multiplicative gradient step. At the first step, as α=1M/M\alpha= 1_M/M, this is an ordinary gradient step of size 1/M1/M in the simplex tangent direction ∇F(1M/M)−1M\nabla F(1_M/M) - 1_M. Remark further that once an entry of the vector α\alpha is zero, it cannot become positive again.

The proximal point algorithm view on early stopping regularization

Another way of seeing the algorithm is as a proximal-point algorithm, as in [11] ; see also [26]. With the responsibilities defined in

(16), set

Q(α∣β):=1n∑i,jrj(yi;β)log⁡(αjKij),D(α,β):=1n∑iKL⁡(r(yi;β)∥r(yi;α)).Q(\alpha\mid\beta) := \frac{1}{n}\sum_{i,j} r_j(y_i;\beta)\log(\alpha_j K_{ij}), \qquad D(\alpha,\beta) := \frac{1}{n}\sum_i \operatorname{KL}\left(r(y_i;\beta)\mathbin{\|}r(y_i;\alpha)\right).

We use the standard extended-value conventions 0log⁡0=00\log0 = 0, alog⁡0=−∞a\log0 = -\infty for a>0a > 0, and the corresponding infinite-KL convention.

Proposition 5. The EM update is the unique maximizer of Q(α∣α(k))Q(\alpha\mid\alpha^{(k)}) and, for a constant CkC_k independent of α\alpha,

Q(α∣α(k))=F(α)−D(α,α(k))+Ck.(18)Q(\alpha\mid\alpha^{(k)}) = F(\alpha) - D(\alpha,\alpha^{(k)}) + C_k. \tag*{(18)}

Thus EM is an exact proximal-point iteration with a KL penalty between the responsibilities at two successive weight vectors.

Observe that the curvature of QQ in coordinate jj is −αj(k+1)/αj2-\alpha_j^{(k+1)}/\alpha_j^2; it therefore becomes large near the boundary when αj(k+1)>0\alpha_j^{(k+1)} > 0. This explains why the EM algorithm slows down.

Figure 2 illustrates this proximal update on a three-dimensional simplex. On that figure, the red broken line represents the iteration path, the blue dot the starting vector of weights, the green dot the optimizer and the two black arrows represent the two competing terms in equation (18).

Figure 2. Visualization of the trajectory of the EM algorithm (red) on the simplex (transparent blue), when initialized at the equal weight vector (blue point). The solution of the optimization problem is the point in green. At each step, the algorithm balances the two terms in Equation (18), which are represented by the black arrows.

There is no tunable step size: the nonnegative penalty changes with the iterate and vanishes at the current one. This makes the implicit regularization of the scheme hard to parse in full generality. The high-noise comparison in Proposition 6 below makes the role of the iteration count as a regularization parameter precise. To establish that result, let Yi(σ)=μi+σZiY_i^{(\sigma)} = \mu_i + \sigma Z_i, with Zi∼iidN(0,Id)Z_i \stackrel{\mathrm{iid}}{\sim} \mathcal{N}(0,I_d). Recall RxR_x and set

Ax:=max⁡i,j∥xj−μi∥,aσ:=Rxσ+AxRxσ2.A_x := \max_{i,j} \lVert x_j-\mu_i\rVert,\qquad a_\sigma:= \frac{R_x}{\sigma}+\frac{A_xR_x}{\sigma^2}.

For an integer T≥0T \ge0, let α~σ,T\widetilde{\alpha}_{\sigma,T} be the unique maximizer of

F(α)−1T+1KL⁡(u∥α)F(\alpha)-\frac{1}{T+1}\operatorname{KL}(u\Vert\alpha)

over ΔM−1\Delta_{M-1}.

Proposition 6 (Early EM as KL regularization). Fix nn, MM, dd, the means μi\mu_i, and the candidate grid, with Rx>0R_x > 0. If TσT_\sigma is deterministic and (Tσ+1)aσ→0(T_\sigma+1)a_\sigma\to0, then

∥α(Tσ+1)−α~σ,Tσ∥2∥α(Tσ+1)−u∥2=Op((Tσ+1)aσ)=op(1).(19)\frac{\lVert\alpha^{(T_\sigma+1)}-\widetilde{\alpha}_{\sigma,T_\sigma}\rVert_2}{\lVert\alpha^{(T_\sigma+1)}-u\rVert_2} = O_p\left((T_\sigma+1)a_\sigma\right) = o_p(1). \tag*{(19)}

Thus, (Tσ+1)(T_{\sigma}+1) EM updates are, to first order, equivalent to maximizing the regularized objective F(α)−(Tσ+1)−1KL⁡(u∥α)F(\alpha) - (T_{\sigma}+1)^{-1}\operatorname{KL}(u\|\alpha).

Early stopping and variability of the estimates

Theorem 3 describes the local fluctuations of the finite-grid MLE, while Proposition 6 compares an early-stopped EM iterate with a penalized optimizer. The two effects are distinct: sampling changes the estimator, whereas early stopping changes how well the objective is optimized. Intuitively, with less iterates, the iterates must remain closer to the uniform initialization and are therefore less variable but biased. On the other hand, a fully solved optimization problem comes with a high variability because of the flatness of the objective when candidate structures are close. Completely understanding and quantifying this phenomenon rigorously remains an interesting challenge.

Stopping criteria

In view of the KKT condition, the gap

G(α):=max⁡1≤j≤M[∇F(α)]j−1.(20)G(\alpha) := \max_{1 \le j \le M} [\nabla F(\alpha)]_j - 1. \tag*{(20)}

vanishes exactly at a maximizer, and concavity gives 0≤F(α^n)−F(α)≤G(α)0 \le F(\hat{\alpha}_n) - F(\alpha) \le G(\alpha). Indeed, concavity bounds the difference by max⁡β∈ΔM−1⟨∇F(α),β−α⟩\max_{\beta\in\Delta_{M-1}}\langle\nabla F(\alpha), \beta- \alpha\rangle, which equals (20) by (9). The infinity norm may still be useful empirically, but need not vanish at a sparse solution.

For the EM step, write dt:=α(t+1)−α(t)d_t := \alpha^{(t+1)} - \alpha^{(t)} and define

Vt:=⟨∇F(α(t)),dt⟩.(21)\mathcal{V}_t := \langle\nabla F(\alpha^{(t)}), d_t \rangle. \tag*{(21)}

(17) gives the useful identity

Vt=∑jαj(t)([∇F(α(t))]j−1)2.\mathcal{V}_t = \sum_j \alpha_j^{(t)} \left([\nabla F(\alpha^{(t)})]_j - 1\right)^2 .

Thus Vt\mathcal{V}_t is the variance gap of Chok and Vasil (2025) [10], a weighted squared norm that downweights coordinates approaching zero. For β∈ΔM−1\beta\in\Delta_{M-1}, define the local seminorm ∥v∥β2:=v⊤[−∇2F(β)]v\lVert v\rVert_\beta^2 := v^\top[-\nabla^2 F(\beta)]v.

Proposition 7. Let α,β∈ΔM−1\alpha,\beta\in\Delta_{M-1}, put d=α−βd=\alpha-\beta, and assume

τ:=n∥d∥β<1.\tau:= \sqrt{n}\lVert d\rVert_\beta< 1.

Then

(1+τ)−1∥d∥β2≤⟨∇F(β)−∇F(α),d⟩≤(1−τ)−1∥d∥β2.(22)(1+\tau)^{-1}\lVert d\rVert_\beta^2 \le\langle\nabla F(\beta)-\nabla F(\alpha),d\rangle\le(1-\tau)^{-1}\lVert d\rVert_\beta^2 . \tag*{(22)}

Corollary 1. Let τt:=n∥dt∥α(t)<1\tau_t := \sqrt{n}\lVert d_t\rVert_{\alpha^{(t)}} < 1. Then

⟨∇F(α(t+1)),dt⟩+(1+τt)−1∥dt∥α(t)2≤Vt≤⟨∇F(α(t+1)),dt⟩+(1−τt)−1∥dt∥α(t)2.\langle\nabla F(\alpha^{(t+1)}),d_t\rangle+(1+\tau_t)^{-1}\lVert d_t\rVert_{\alpha^{(t)}}^2 \le\mathcal{V}_t \le\langle\nabla F(\alpha^{(t+1)}),d_t\rangle+(1-\tau_t)^{-1}\lVert d_t\rVert_{\alpha^{(t)}}^2 .

Thus, G\mathcal{G} is a certified optimization gap, while Vt\mathcal{V}_t is a smoother EM diagnostic as we shall further show in Figure B3.

Comparison of the variance gap and infinity-norm gap for the IgG example

Figure B3. Comparison of the “variance gap” from Equation (21) (red) against the ∞\infty-norm gap ∥∇F(α(t))−1M∥∞\lVert\nabla F(\alpha^{(t)})^{-1}M \rVert_{\infty}, for the IgG example considered in Appendix B, with the same data as figure B2.

Practical considerations on stopping criteria

For early stopping, we compare the variance-gap criterion with the interior fixed-point diagnostic ∥∇F(α)−1M∥∞<η\lVert\nabla F(\alpha)-1_M\rVert_\infty< \eta, using η∈[10−3,10−2]\eta\in[10^{-3},10^{-2}]. Empirically, the square root of the variance gap behaves like a smoothed version of this diagnostic, so we use

⟨∇αF(α(t)),α(t+1)−α(t)⟩1/2<η\left\langle\nabla_\alpha F(\alpha^{(t)}),\alpha^{(t+1)}-\alpha^{(t)}\right\rangle^{1/2}<\eta

with η∈[10−3,10−2]\eta\in[10^{-3},10^{-2}].

The square root of the variance gap can be interpreted as a weighted L2L_2-norm of ∇F(α(t))−1\nabla F(\alpha^{(t)})-1 with respect to the weights α(t)\alpha^{(t)}, and is also related to the time-derivative of FF for a continuous-time flow on the simplex (Chok and Vasil, 2025) [10]. Connecting these interpretations to our results, with the goal of stronger numerical guidance for practitioners, is an important focus for future development.

Statistical properties

Stability under an additional noise component

Recall that the ambient image dimension is NN. As above, yiy_i is obtained from the signal plus noise model. Let ηi\eta_i, i=1,…,ni=1,\ldots,n, be noise vectors in RN\mathbb{R}^N and set y~i=yi+ηi\widetilde{y}_i=y_i+\eta_i. Let F~\widetilde{F} denote the objective computed from y~i\widetilde{y}_i. Let α^\widehat{\alpha} maximize FF, and let α~\widetilde{\alpha} be any maximizer of F~\widetilde{F}. Set

μN:=inf⁡α∈ΔM−1λmin⁡T(−∇2F(α)).\mu_N := \inf_{\alpha\in\Delta_{M-1}} \lambda_{\min}^{T}\left(-\nabla^2 F(\alpha)\right).

Theorem 2. Assume that the collection (ηi)i=1n(\eta_i)_{i=1}^{n} is independent of the uncorrupted data, that Eηi=0\mathbb{E}\eta_i=0, and that E(ηiηiT)=Σi\mathbb{E}(\eta_i\eta_i^{\mathsf{T}})=\Sigma_i. Write εN:=max⁡i∥Σi∥op1/2\varepsilon_N:=\max_i\|\Sigma_i\|_{\mathrm{op}}^{1/2}. For fixed nn, MM, suppose (8) holds almost surely. Then, as N→∞N\to\infty,

∥α~−α^∥=Op(RxεNμNσ2).(23)\|\widetilde{\alpha}-\widehat{\alpha}\|=O_p\left(\frac{R_x\varepsilon_N}{\mu_N\sigma^2}\right). \tag*{(23)}

Here, XN=Op(AN)X_N=O_p(A_N) for a positive random scale ANA_N means XN/AN=Op(1)X_N/A_N=O_p(1). The random factor μN−1\mu_N^{-1} records the conditioning of the uncorrupted problem. In particular, their distance converges to zero whenever RxεN/(μNσ2)→0R_x\varepsilon_N/(\mu_N\sigma^2)\to0 in probability. Therefore, for heteroscedastic noise that is not “too spiky” in the sense that εN\varepsilon_N is not too large, the estimates will be robust to noise.

Asymptotic distribution of the finite-grid

We use the standard constrained MM-estimation limit [17]. We now return to the fixed ambient dimension dd of Section 2, keep the finite grid fixed as n→∞n\to\infty, and parametrize the simplex by

Θ:={θ∈RM−1:θj≥0, 1Tθ≤1},α(θ):=(θ1,…,θM−1,1−1Tθ).\Theta:=\{\theta\in\mathbb{R}^{M-1}:\theta_j\geq0,\ \mathbf{1}^{\mathsf{T}}\theta\leq1\},\qquad\alpha(\theta):=(\theta_1,\ldots,\theta_{M-1},1-\mathbf{1}^{\mathsf{T}}\theta).

This parametrization is useful for the central limit theorem below as any element on the simplex has M−1M-1 degrees of freedom. Let fθ:=pσ∗PMα(θ)f_\theta:=p_\sigma* P_M^{\alpha(\theta)} and

sθ(y):=∇θlog⁡fθ(y)=(pσ(y∣xj)−pσ(y∣xM))j=1M−1fθ(y).s_\theta(y):=\nabla_\theta\log f_\theta(y)=\frac{\left(p_\sigma(y\mid x_j)-p_\sigma(y\mid x_M)\right)_{j=1}^{M-1}}{f_\theta(y)}.

Let α∗\alpha^* be a maximizer of E[log⁡{(pσ∗PMα)(Y)}]\mathbb{E}[\log\{(p_\sigma*P_M^\alpha)(Y)\}] over ΔM−1\Delta_{M-1}. Let θ∗\theta^* and θ^n\widehat{\theta}_n be the first M−1M-1 coordinates of α∗\alpha^* and α^n\widehat{\alpha}_n, respectively. Put

Iθ∗:=E[sθ∗(Y)sθ∗(Y)T],b:=E[sθ∗(Y)].I_{\theta^*}:=\mathbb{E}[s_{\theta^*}(Y)s_{\theta^*}(Y)^{\mathsf{T}}],\qquad b:=\mathbb{E}[s_{\theta^*}(Y)].

As the MLE is constrained, the constraint will be relevant for the limiting distribution. We thus introduce the critical cone

C:={u∈RM−1:uj≥0whenever θj∗=0,1Tu≤0whenever 1Tθ∗=1,bTu=0}.C:=\left\{u\in\mathbb{R}^{M-1}: \begin{array}{ll} u_j\geq0 & \text{whenever }\theta_j^*=0,\\ \mathbf{1}^{\mathsf{T}}u\leq0 & \text{whenever }\mathbf{1}^{\mathsf{T}}\theta^*=1,\\ b^{\mathsf{T}}u=0 \end{array} \right\}.

Theorem 3. Suppose that P0P_0 has density pσ∗ρp_{\sigma} * \rho, where ρ\rho is a boundedly supported probability measure, and that the candidate points are pairwise distinct. Then Iθ∗I_{\theta^*} is positive definite and

n(θ^n−θ∗)⇒arg⁡min⁡u∈C{12uTIθ∗u−ZTu},Z∼NM−1(0,Iθ∗−bbT).(24)\sqrt{n}(\hat{\theta}_n-\theta^*) \Rightarrow\underset{u\in\mathcal{C}}{\arg\min}\left\{\frac{1}{2}u^{\mathsf{T}}I_{\theta^*}u-Z^{\mathsf{T}}u\right\}, \qquad Z\sim N_{M-1}(0,I_{\theta^*}-bb^{\mathsf{T}}). \tag*{(24)}

where ⇒\Rightarrow denotes convergence in distribution. If θ∗\theta^* is in the interior of Θ\Theta, the limit is NM−1(0,Iθ∗−1)N_{M-1}(0,I_{\theta^*}^{-1}).

Remark 2 (Correct specification and boundary points). If P0P_0 has density fθ∗f_{\theta^*}, then b=0b=0 and C\mathcal{C} is the tangent cone. After multiplication by Iθ∗1/2I_{\theta^*}^{1/2}, the limit in (24) is the Euclidean projection of a standard Gaussian vector onto Iθ∗1/2CI_{\theta^*}^{1/2}\mathcal{C}. At a boundary point this projection is generally non-Gaussian.

Remark 3 (Misspecification at the boundary). At a boundary maximizer under misspecification, bb need not vanish. For a feasible displacement u/nu/\sqrt{n}, the full log-likelihood has leading deterministic term nbTu\sqrt{n}b^{\mathsf{T}}u, while its random fluctuation is Op(1)O_p(1). If bTu<0b^{\mathsf{T}}u<0, the deterministic loss rules out that direction asymptotically. Thus C\mathcal{C} retains only feasible directions satisfying bTu=0b^{\mathsf{T}}u=0.

Simulation results

In this section, we present numerical experiments illustrating the issues that were theoretically investigated above. In Section 7.1, we present the biomolecular system and cryo-EM setup that we used as an example. In Section 7.2, we exhibit how the weights can be biased when the structures do not cover the space well enough, thereby illustrating the practical consequence of Theorem 1. In Section 7.3, the question is similar, but instead of having part of the conformational space not covered, the density of the structures is too low in a certain region. In Section 7.4, we then numerically investigate what determines where the spike appears, echoing Proposition 4.

A running example: Hsp90

The molecule of interest for our experiments will be Hsp90, a classical example used by the community to simulate biomolecular changes in cryo-EM [40]. In a nutshell, the heat shock protein 90 is a molecule assisting other proteins to fold properly. Images are generated with the cryojax library [35]. For each candidate structure we read centered, hydrogen-free atomic coordinates from a PDB file and build the electron-scattering potential as a sum of per-atom Gaussians from the tabulated Peng scattering factors, retaining atomic BB-factors. A Gaussian-mixture integrator (with error-function pixel integration) then yields the clean projection on a grid of N=128×128N=128\times128 pixels at 2.02.0 Å/pixel and 300300 kV, with no PSF or translation and a single fixed pose (Euler angles (45∘,90∘,90∘)(45^\circ,90^\circ,90^\circ)) shared across all structures. Two images therefore differ only through their conformation and noise, which is the identity-forward-operator setting analyzed above. Let M\mathcal{M} be a cosine-tapered circular mask of radius 42 pixels (one third of the image width, one-pixel roll-off). Each clean projection ImI_m is standardized within the mask to unit signal variance,

I~m=Im−μM(Im)sM(Im),\widetilde{I}_m = \frac{I_m-\mu_{\mathcal{M}}(I_m)}{s_{\mathcal{M}}(I_m)},

and the observed image for a sampled conformation mm is y=SNR I~m+εy = \sqrt{\mathrm{SNR}}\,\widetilde{I}_m+\varepsilon, ε∼N(0,IN)\varepsilon\sim\mathcal{N}(0,I_N). The noise is thus i.i.d. standard Gaussian (σ2=1\sigma^2=1 per pixel) and the signal scales as SNR\sqrt{\mathrm{SNR}}, so SNR is the in-mask signal-to-noise power ratio and small SNR is the per-pixel high-noise regime of interest. Example images are shown in Figure 1b.

The ground truth distribution is generated by picking conformations at random among the frames of the trajectory and assigning each to the nearest candidate structure; in some cases, we chose a uniform distribution while in other cases a bimodal one, see the orange line on the figures below.

Reference structures not covering the space

Recall that Figure 1 showed how early stopping in the EM acted as a regularizer while the exact solution to the reweighting optimization problem is spiky. We now turn to addressing the question of the importance of the candidates points covering the space well enough, which is the setting of Theorem 1. We first consider the set of reference structures with the motion shown in Figure 1a to generate the data with varying signal-to-noise ratio and number of sample points. To reflect the possibility that the set of reference structures might not be complete enough, we remove the structures with opening angle between 15 and 20 degrees in the set of candidates.

On Figure 4, one clearly sees that the candidate structure closer to the structures that have no representative in the dataset gets too much weight, which matches intuition. Also, even though less pronounced, a larger sample size yields a distribution that is closer to the uniform, while decreasing the signal-to-noise ratio also deteriorates the recovery (Figure 4b).

Output of the optimization procedure showing the distribution and KL divergence between truth and reconstruction

Figure 4. Output of the optimization procedure when the reference structures do not cover a part of the conformational space, while the ground truth is uniform over a larger set of structures. Left: Comparison of the exact solutions of the problem for a sample size n=1000n = 1000 and SNR=2=2. Right: KL(αb∥α0)(\alpha_b\Vert\alpha_0), averaged over 10 replications of the same random setting. For larger sample sizes and signal-to-noise ratio, the KL diminishes. It does not converge to zero as the set of candidate structures is not covering part of the conformational space. For very small SNR, the spikiness translates into higher KL-divergence.

Recovery of low-population states

In practice, even if the reference structures cover the entire conformational space, there will be regions in which the density of the structures is larger and regions of lower density (i.e., low-populated states). This is what we explore in the following example. As the spacing between the bins in the middle suggest, there are less structures in the middle of the conformational trajectory than at the boundaries. The actual motion is an opening, similar to that of Figure 1a. The optimal weights given a dataset are optimized using an exact convex solver and the results are reported in Figure 5. In this example, as the reference structures do not cover the space well enough compared with the ground truth, they receive a much larger weight.

Output of the optimization procedure when the reference structures coarsely cover the central part of the conformational space

Figure 5. Output of the optimization procedure when the reference structures coarsely cover the central part of the conformational space, while the ground truth is uniform over a larger set of structures. Left: Comparison of the exact solutions of the problem for a sample size n=1000n = 1000 and SNR=2. Right: KL(α^∥α0\hat{\alpha}\Vert\alpha_0), averaged over 10 replications of the same random setting. For larger sample sizes and signal-to-noise ratio, the KL diminishes. It does not converge to zero as the set of candidate structures is not covering part of the conformational space. For very small SNR, the spikiness translates into higher KL-divergence.

Where do the spikes appear?

Section 4.4 showed how empirical frequencies and candidate geometry enter the objective. Figures 6 and 7 illustrate that frequencies alone do not predict the reconstructed weights. Indeed, the fact that a structure has been sampled (red) more than the ground truth does not imply that the reconstructed weight (green) is above the ground truth as well. The observed spike locations reflect the joint effect of candidate geometry, empirical frequencies and noise.

Conclusions

In this work, we analyzed cryo-EM ensemble reweighting over a fixed set of candidate structures as a computational surrogate for a nonparametric maximum-likelihood estimation problem. We developed theoretical results and complemented them with numerical experiments illustrating the behavior of the recovered weights.

Our first conclusion is that the discretization induced by fixing a set of candidate structures is more than a mere approximation and is not a harmless computational convenience: the interplay between the noise level and the coverage of the candidate set of structures fundamentally controls identifiability, conditioning, and interpretability, as also discussed in [33]. Candidate structures that are closer than the noise scale induce ill-conditioning, thereby making the weights sensitive to small perturbations. Conversely, when the candidates do not cover the space well enough, there is an unavoidable bias. These insights explain why spiky solutions are neither an

Comparison between the reconstructed weights, the actual frequencies in the data and the ground truth

Figure 6. Comparison between the reconstructed weights, the actual frequencies in the data and the ground truth. From left to right, the sample size is n=1000n = 1000 and the SNR is 0.1 and 0.01, respectively.

Comparison between the reconstructed weights, the actual frequencies in the data and the ground truth

Figure 7. Comparison between the reconstructed weights, the actual frequencies in the data and the ground truth. From left to right, the sample sizes are n=1000n = 1000 and n=10000n = 10000, and the SNR is 0.1.

anomaly nor necessarily conveying important information about the data; rather, they are an expected outcome of maximum-likelihood fitting on a discrete support.

We also showed that the choice of algorithm (and its parameters) has an important impact on the recovered solutions. In particular, EM slowdowns arise precisely in the near-nonidentifiable regime where structures are similar. Its tendency toward increasingly spiky fits motivates viewing the iteration count as an implicit regularization parameter. Importantly, this means that early stopping might be beneficial. The intricate nonlinear nature of the recursion prevents providing a clear theoretical rule, but we proposed principled stopping diagnostic tools. Future work may benefit from drawing analogs to regularization and early stopping in Richardson-Lucy iteration for image deconvolution (see for example [5], a close historical analog to our problem.

Overall, our work suggests shifting emphasis from solving the reweighting problem to a more nuanced approach in which one iteratively crafts the candidate set and carefully investigates the early-stopping potential for each dataset analyzed, extending simulation frameworks such as in [41]. Extending to the full cryo-EM model, i.e., including the pose and projection parts of the forward model, as well as reconstruction, is a step we aim to take in the future.

Acknowledgment

Research reported in this publication was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number R35GM157226. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The work was also supported by the Alfred P. Sloan Foundation (FG-2023-20853), and the Simons Foundation (1288155). The Flatiron Institute is a division of the Simons Foundation.

Proofs

Proof of Proposition 1. Direct differentiation gives the two formulas, and

−vT∇2F(α)v=1n∑i=1n(Ki,∙Tv)2si(α)2≥0.-v^{\mathsf{T}}\nabla^{2}F(\alpha)v = \frac{1}{n}\sum_{i=1}^{n}\frac{\left(K_{i,\bullet}^{\mathsf{T}}v\right)^{2}}{s_i(\alpha)^{2}} \ge0.

This proves the curvature claims. If two maximizers had different fitted vectors, strict concavity of z↦n−1∑ilog⁡ziz \mapsto n^{-1}\sum_i \log z_i would give a contradiction at their midpoint.

Proof of Proposition 2. Write K(h)K^{(h)} for the likelihood matrix defining FhF_h and let s(h)(α):=K(h)αs^{(h)}(\alpha) := K^{(h)}\alpha. For v=(ek−eℓ)/2v=(e_k-e_\ell)/\sqrt{2}, the Rayleigh quotient and (7) give

λmin⁡T(−∇2Fh(α))≤12n∑i=1n(Kik(h)−Kiℓ(h)si(h)(α))2.(25)\lambda_{\min}^{\mathsf{T}}\left(-\nabla^{2}F_h(\alpha)\right) \le\frac{1}{2n}\sum_{i=1}^{n}\left(\frac{K_{ik}^{(h)}-K_{i\ell}^{(h)}}{s_i^{(h)}(\alpha)}\right)^{2}. \tag*{(25)}

On a fixed neighborhood of zero, smoothness and positivity of the Gaussian kernel give constants C,c>0C,c>0 such that, uniformly in hh, ii, and α\alpha,

∣Kik(h)−Kiℓ(h)∣≤C∥h∥,si(h)(α)≥c.\left|K_{ik}^{(h)}-K_{i\ell}^{(h)}\right| \le C\lVert h\rVert,\qquad s_i^{(h)}(\alpha)\ge c.

The bound (A.1) is therefore O(∥h∥2)O(\lVert h\rVert^{2}) uniformly in α\alpha. Under the stated high-noise scaling, divide the numerator and denominator by Kiℓ(h)K_{i\ell}^{(h)}. The likelihood ratios of the fixed candidates are 1+Op(σ−1)1+O_p(\sigma^{-1}), while Kik(h)/Kiℓ(h)−1=Op(∥h∥/σ)K_{ik}^{(h)}/K_{i\ell}^{(h)}-1=O_p(\lVert h\rVert/\sigma) locally uniformly in hh. This gives the final claim.

Proof of Proposition 3. For u∈RMu\in\mathbb{R}^{M}, put zi:=Ki,∙Tu/Ki,∙Tαz_i:=K_{i,\bullet}^{\mathsf{T}}u/K_{i,\bullet}^{\mathsf{T}}\alpha. Direct differentiation gives

D2f(α)[u,u]=1n∑i=1nzi2,D3f(α)[u,u,u]=−2n∑i=1nzi3.D^{2}f(\alpha)[u,u]=\frac{1}{n}\sum_{i=1}^{n}z_i^{2},\qquad D^{3}f(\alpha)[u,u,u]=-\frac{2}{n}\sum_{i=1}^{n}z_i^{3}.

Therefore,

∣D3f(α)[u,u,u]∣≤2n(∑i=1nzi2)3/2=2n(D2f(α)[u,u])3/2.\left|D^{3}f(\alpha)[u,u,u]\right|\le\frac{2}{n}\left(\sum_{i=1}^{n}z_i^{2}\right)^{3/2}=2\sqrt{n}\left(D^{2}f(\alpha)[u,u]\right)^{3/2}.

Multiplication by nn changes the self-concordance constant from 2n2\sqrt{n} to 22, and the displayed Hessian vanishes in direction uu precisely when Ku=0Ku=0.

Proof of Theorem 1. For 1≤k≤m+11\le k\le m+1, define the symmetric moment-difference tensor

Tk(α):=∫x⊗k d(PMα−ρ)(x).T_k(\alpha):=\int x^{\otimes k}\,\mathrm{d}\left(P_M^\alpha-\rho\right)(x).

Matching all coordinate moments of total degree kk is equivalent to Tk(α)=0T_k(\alpha)=0. By maximality of mm, no α∈ΔM−1\alpha\in\Delta_{M-1} makes

T1(α),…,Tm+1(α)T_1(\alpha),\ldots,T_{m+1}(\alpha)

all zero. These tensors depend continuously on α\alpha, and the simplex is compact. Consequently,

δ:=min⁡α∈ΔM−1{∑k=1m+1∥Tk(α)∥F2}1/2>0.(26)\delta:= \min_{\alpha\in\Delta_{M-1}} \left\{ \sum_{k=1}^{m+1} \lVert T_k(\alpha) \rVert_F^2 \right\}^{1/2} > 0. \tag*{(26)}

Let

Δα(u):=F[PMα](u)−F[ρ](u),\Delta_\alpha(u) := \mathcal{F}[P_M^\alpha](u) - \mathcal{F}[\rho](u),

where, for a finite measure μ\mu,

F[μ](u):=∫ei⟨u,x⟩ dμ(x).\mathcal{F}[\mu](u) := \int e^{i\langle u,x\rangle} \, \mathrm{d}\mu(x).

The supports of ρ\rho and of every PMαP_M^\alpha lie in one fixed ball. Taylor’s formula for ei⟨u,x⟩e^{i\langle u,x\rangle} therefore gives, uniformly in α\alpha and for ∥u∥≤1\lVert u\rVert\le1,

Δα(u)=∑k=1m+1ikk!⟨Tk(α),u⊗k⟩+Eα(u),∣Eα(u)∣≤C1∥u∥m+2.(27)\Delta_\alpha(u) = \sum_{k=1}^{m+1} \frac{i^k}{k!} \langle T_k(\alpha),u^{\otimes k}\rangle+ E_\alpha(u), \qquad\lvert E_\alpha(u)\rvert\le C_1\lVert u\rVert^{m+2}. \tag*{(27)}

Let B1:={z∈Rd:∥z∥≤1}B_1 := \{z \in\mathbb{R}^d : \lVert z\rVert\le1\}. The map

(A1,…,Am+1)⟼∑k=1m+1ikk!⟨Ak,z⊗k⟩(A_1,\ldots,A_{m+1}) \longmapsto\sum_{k=1}^{m+1} \frac{i^k}{k!}\langle A_k,z^{\otimes k}\rangle

is injective on ⨁k=1m+1Sym⁡k(Rd)\bigoplus_{k=1}^{m+1} \operatorname{Sym}^k(\mathbb{R}^d). Hence the L2(B1)L^2(B_1)-norm of the resulting polynomial is equivalent to the Euclidean norm of its coefficients. It follows from (A.2) that, for every σ≥1\sigma\ge1,

∥∑k=1m+1ikσ−kk!⟨Tk(α),z⊗k⟩∥L2(B1)≥c1{∑k=1m+1σ−2k∥Tk(α)∥F2}1/2≥c1δσ−(m+1).(28)\left\lVert\sum_{k=1}^{m+1} \frac{i^k\sigma^{-k}}{k!}\langle T_k(\alpha),z^{\otimes k}\rangle\right\rVert_{L^2(B_1)} \ge c_1\left\{ \sum_{k=1}^{m+1} \sigma^{-2k}\lVert T_k(\alpha)\rVert_F^2 \right\}^{1/2} \ge c_1\delta\sigma^{-(m+1)}. \tag*{(28)}

Set u=z/σu=z/\sigma. By (A.3), the L2(B1)L^2(B_1)-norm of the remainder satisfies

∥Eα(z/σ)∥L2(B1)≤C2σ−(m+2)\left\lVert E_\alpha(z/\sigma)\right\rVert_{L^2(B_1)} \le C_2\sigma^{-(m+2)}

uniformly in α\alpha. Therefore,

∥Δα(z/σ)∥L2(B1)≥c1δσ−(m+1)−C2σ−(m+2).\left\lVert\Delta_\alpha(z/\sigma)\right\rVert_{L^2(B_1)} \ge c_1\delta\sigma^{-(m+1)} - C_2\sigma^{-(m+2)}.

Choosing σ0≥max⁡{1,2C2c1δ}\sigma_0 \ge\max\left\{1,\frac{2C_2}{c_1\delta}\right\}, it holds, for every σ≥σ0\sigma\ge\sigma_0, uniformly in α∈ΔM−1\alpha\in\Delta_{M-1}, that

∥Δα(z/σ)∥L2(B1)≥c1δ2σ−(m+1).\left\lVert\Delta_\alpha(z/\sigma)\right\rVert_{L^2(B_1)} \ge\frac{c_1\delta}{2}\sigma^{-(m+1)}.

After squaring, this gives

∫B1∣Δα(z/σ)∣2 dz≥c2σ−2(m+1).(29)\int_{B_1} \lvert\Delta_\alpha(z/\sigma)\rvert^2 \,\mathrm{d}z \ge c_2\sigma^{-2(m+1)}. \tag*{(29)}

Since

F[pσ](u)=e−σ2∥u∥2/2,\mathcal{F}[p_{\sigma}](u)=e^{-\sigma^{2}\lVert u\rVert^{2}/2},

Plancherel’s identity and the change of variables z=σuz=\sigma u yield

∥pσ∗PMα−pσ∗ρ∥22=cd∫Rde−σ2∥u∥2∣Δα(u)∣2 du=cdσ−d∫Rde−∥z∥2∣Δα(z/σ)∣2 dz≥cde−1σ−d∫B1∣Δα(z/σ)∣2 dz≥c3σ−d−2(m+1).(30)\begin{aligned} \lVert p_{\sigma} * P_{M}^{\alpha}-p_{\sigma} * \rho\rVert_{2}^{2} &=c_{d}\int_{\mathbb{R}^{d}}e^{-\sigma^{2}\lVert u\rVert^{2}}\lvert\Delta_{\alpha}(u)\rvert^{2}\,du\\ &=c_{d}\sigma^{-d}\int_{\mathbb{R}^{d}}e^{-\lVert z\rVert^{2}}\lvert\Delta_{\alpha}(z/\sigma)\rvert^{2}\,dz\\ &\geq c_{d}e^{-1}\sigma^{-d}\int_{B_{1}}\lvert\Delta_{\alpha}(z/\sigma)\rvert^{2}\,dz\\ &\geq c_{3}\sigma^{-d-2(m+1)}. \tag*{(30)} \end{aligned}

Both pσ∗ρp_{\sigma} * \rho and pσ∗PMαp_{\sigma} * P_{M}^{\alpha} are bounded above by Bσ:=(2πσ2)−d/2B_{\sigma}:=(2\pi\sigma^{2})^{-d/2}. The KL–Hellinger inequality gives

KL⁡(pσ∗ρ∥pσ∗PMα)≥14Bσ∥(pσ∗ρ)−(pσ∗PMα)∥22.\operatorname{KL}(p_{\sigma} * \rho\mathbin\Vert p_{\sigma} * P_{M}^{\alpha})\geq\frac{1}{4B_{\sigma}}\lVert(p_{\sigma} * \rho)-(p_{\sigma} * P_{M}^{\alpha})\rVert_{2}^{2}.

Combining this with (A.6) gives, uniformly in α\alpha, KL⁡(pσ∗ρ∥pσ∗PMα)≥cσ−2(m+1)\operatorname{KL}(p_{\sigma} * \rho\mathbin\Vert p_{\sigma} * P_{M}^{\alpha})\geq c\sigma^{-2(m+1)}. □

Proof of Proposition 4. Write

gα(y):=log⁡{(pσ∗PMα)(y)}.g_{\alpha}(y):=\log\{(p_{\sigma} * P_{M}^{\alpha})(y)\}.

If Y=xk+σZY=x_{k}+\sigma Z, write Ek\mathbb{E}_{k} for expectation under its law. Finiteness of the candidate grid gives, uniformly in α\alpha, β\beta and kk,

∣gα(Y)∣≤C(1+∥Z∥2),∣gα(Y)−gβ(Y)∣≤CeC∥Z∥∥α−β∥1.\lvert g_{\alpha}(Y)\rvert\leq C(1+\lVert Z\rVert^{2}),\qquad\lvert g_{\alpha}(Y)-g_{\beta}(Y)\rvert\leq Ce^{C\lVert Z\rVert}\lVert\alpha-\beta\rVert_{1}.

Indeed, the mixture density is a weighted average of the component densities, so it lies between their minimum and maximum; this gives the first bound. For the second, the mean-value inequality for the logarithm gives

∣gα(Y)−gβ(Y)∣≤max⁡mpσ(Y∣xm)min⁡mpσ(Y∣xm)∥α−β∥1.\lvert g_{\alpha}(Y)-g_{\beta}(Y)\rvert\leq\frac{\max_{m}p_{\sigma}(Y\mid x_{m})}{\min_{m}p_{\sigma}(Y\mid x_{m})}\lVert\alpha-\beta\rVert_{1}.

For Gaussian components with the same variance, the ratio on the right is bounded by CeC∥Z∥Ce^{C\lVert Z\rVert}. Both bounds are square integrable. The standard maximal bound for this fixed-dimensional Lipschitz class therefore gives, conditionally on the labels,

sup⁡α∈ΔM−1∣1Nk∑i:zi=kgα(Yi)−Ekgα(Y)∣=Op(Nk−1/2).\sup_{\alpha\in\Delta_{M-1}}\left\lvert\frac{1}{N_{k}}\sum_{i:z_{i}=k}g_{\alpha}(Y_{i})-\mathbb{E}_{k}g_{\alpha}(Y)\right\rvert=O_{p}(N_{k}^{-1/2}).

Moreover,

Ekgα(Y)=−d2{1+log⁡(2πσ2)}−KL⁡(pσ(⋅∣xk)∥pσ∗PMα).\mathbb{E}_{k}g_{\alpha}(Y)=-\frac{d}{2}\{1+\log(2\pi\sigma^{2})\}-\operatorname{KL}(p_{\sigma}(\cdot\mid x_{k})\mathbin\Vert p_{\sigma} * P_{M}^{\alpha}).

Summing over the fixed number of clusters and using Nk/n→πk>0N_{k}/n\to\pi_{k}>0 proves (13). □

Proof of Proposition 5. Up to an α\alpha-independent constant,

Q(α∣α(k))=∑jαj(k+1)log⁡αj,Q(\alpha\mid\alpha^{(k)}) = \sum_j \alpha_j^{(k+1)} \log\alpha_j,

whose unique simplex maximizer is α(k+1)\alpha^{(k+1)}. Expanding the KL divergence in D\mathcal{D} gives (18).

Proof of Proposition 6. Put L:=Tσ+1L := T_\sigma+ 1, Gσ(α):=Φn(α)−αG_\sigma(\alpha) := \Phi_n(\alpha) - \alpha, and

δσ:=max⁡i,j,ℓ∣log⁡KijKiℓ∣,εσ:=eδσ−1.\delta_\sigma:= \max_{i,j,\ell} \left| \log\frac{K_{ij}}{K_{i\ell}} \right|, \qquad\varepsilon_\sigma:= e^{\delta_\sigma} - 1.

The Gaussian likelihood gives

log⁡KijKiℓ=⟨Zi,xj−xℓ⟩σ−∥xj−μi∥2−∥xℓ−μi∥22σ2.\log\frac{K_{ij}}{K_{i\ell}} = \frac{\langle Z_i,x_j-x_\ell\rangle}{\sigma} - \frac{\lVert x_j-\mu_i\rVert^2-\lVert x_\ell-\mu_i\rVert^2}{2\sigma^2}.

Let

W:=max⁡i,j<ℓxj≠xℓ∣⟨Zi,xj−xℓ⟩∣∥xj−xℓ∥.W := \max_{\substack{i,j<\ell\\x_j\ne x_\ell}} \frac{\left|\langle Z_i,x_j-x_\ell\rangle\right|}{\lVert x_j-x_\ell\rVert}.

The preceding identity and polarization give

δσ≤RxσW+AxRxσ2≤aσmax⁡(W,1).\delta_\sigma\le\frac{R_x}{\sigma}W+\frac{A_xR_x}{\sigma^2}\le a_\sigma\max(W,1).

Moreover,

P(W>t)≤nM(M−1)e−t2/2.\mathbb{P}(W>t)\le nM(M-1)e^{-t^2/2}.

Since the law of WW does not depend on σ\sigma, this bound is uniform for the continuum limit σ→∞\sigma\to\infty, and therefore δσ=Op(aσ)\delta_\sigma=O_p(a_\sigma). In particular, P(δσ≤1)→1\mathbb{P}(\delta_\sigma\le1)\to1, and on this event εσ≤(e−1)δσ\varepsilon_\sigma\le(e-1)\delta_\sigma. Hence εσ=Op(aσ)\varepsilon_\sigma=O_p(a_\sigma). Since si(α)s_i(\alpha) is a convex combination of the entries in row ii,

e−δσ≤Kijsi(α)≤eδσ,∣Kijsi(α)−1∣≤εσ.e^{-\delta_\sigma}\le\frac{K_{ij}}{s_i(\alpha)}\le e^{\delta_\sigma}, \qquad\left|\frac{K_{ij}}{s_i(\alpha)}-1\right|\le\varepsilon_\sigma.

It follows that

[Gσ(α)]j=αjn∑i(Kijsi(α)−1),sup⁡α∈ΔM−1∥Gσ(α)∥2≤εσ.[G_\sigma(\alpha)]_j=\frac{\alpha_j}{n}\sum_i\left(\frac{K_{ij}}{s_i(\alpha)}-1\right), \qquad\sup_{\alpha\in\Delta_{M-1}}\lVert G_\sigma(\alpha)\rVert_2\le\varepsilon_\sigma.

For a tangent vector v∈Tv\in\mathcal{T}, direct differentiation and ∑mvm=0\sum_m v_m=0 give

[∇Gσ(α)v]j=1n∑i[(Kijsi(α)−1)vj−rj(yi;α)∑m(Kimsi(α)−1)vm].[\nabla G_\sigma(\alpha)v]_j=\frac{1}{n}\sum_i\left[\left(\frac{K_{ij}}{s_i(\alpha)}-1\right)v_j-r_j(y_i;\alpha)\sum_m\left(\frac{K_{im}}{s_i(\alpha)}-1\right)v_m\right].

Since each r(yi;α)r(y_i;\alpha) is a probability vector, the preceding row-wise bound implies

∥∇Gσ(α)v∥2≤(1+M)εσ∥v∥2.\lVert\nabla G_\sigma(\alpha)v\rVert_2\le(1+\sqrt{M})\varepsilon_\sigma\lVert v\rVert_2.

Integrating along the segment joining α\alpha and β\beta therefore gives

∥Gσ(α)−Gσ(β)∥2≤CMεσ∥α−β∥2,CM:=1+M.\lVert G_\sigma(\alpha)-G_\sigma(\beta)\rVert_2 \le C_M\varepsilon_\sigma\lVert\alpha-\beta\rVert_2,\qquad C_M:=1+\sqrt{M}.

The bound sup⁡α∈ΔM−1∥Gσ(α)∥2≤εσ\sup_{\alpha\in\Delta_{M-1}}\lVert G_\sigma(\alpha)\rVert_2\le\varepsilon_\sigma gives ∥α(t)−u∥2≤tεσ\lVert\alpha^{(t)}-u\rVert_2\le t\varepsilon_\sigma. Moreover, the EM recursion yields

α(L)−u−LGσ(u)=∑t=0L−1{Gσ(α(t))−Gσ(u)},\alpha^{(L)}-u-LG_\sigma(u)=\sum_{t=0}^{L-1}\left\{G_\sigma(\alpha^{(t)})-G_\sigma(u)\right\},

and

∥α(L)−u−LGσ(u)∥2≤CMεσ∑t=0L−1tεσ≤CM2L2εσ2.\lVert\alpha^{(L)}-u-LG_\sigma(u)\rVert_2\le C_M\varepsilon_\sigma\sum_{t=0}^{L-1}t\varepsilon_\sigma\le\frac{C_M}{2}L^2\varepsilon_\sigma^2.

Write α~:=α~σ,Tσ\widetilde{\alpha}:=\widetilde{\alpha}_{\sigma,T_\sigma}. Since KL⁡(u∥α)\operatorname{KL}(u\mathbin{\|}\alpha) diverges at the boundary and is strictly convex in the relative interior, the penalized objective has a unique interior maximizer. Its stationarity equations are

[∇F(α~)]j+1LMα~j=ν.[\nabla F(\widetilde{\alpha})]_j+\frac{1}{LM\widetilde{\alpha}_j}=\nu.

Multiplying by α~j\widetilde{\alpha}_j and summing gives ν=1+L−1\nu=1+L^{-1}. Multiplying the jjth stationarity equation by α~j\widetilde{\alpha}_j and using [Φn(α)]j=αj[∇F(α)]j[\Phi_n(\alpha)]_j=\alpha_j[\nabla F(\alpha)]_j gives

[Φn(α~)]j+1LM=(1+L−1)α~j,as well asL[Gσ(α~)]j=α~j−1M.[\Phi_n(\widetilde{\alpha})]_j+\frac{1}{LM}=(1+L^{-1})\widetilde{\alpha}_j,\qquad\text{as well as}\qquad L[G_\sigma(\widetilde{\alpha})]_j=\widetilde{\alpha}_j-\frac{1}{M}.

Thus α~−u=LGσ(α~)\widetilde{\alpha}-u=LG_\sigma(\widetilde{\alpha}), and consequently

∥α~−u∥2≤Lεσ,∥α~−u−LGσ(u)∥2≤CML2εσ2.\lVert\widetilde{\alpha}-u\rVert_2\le L\varepsilon_\sigma,\qquad\lVert\widetilde{\alpha}-u-LG_\sigma(u)\rVert_2\le C_ML^2\varepsilon_\sigma^2.

Together with the preceding bound on ∥α(L)−u−LGσ(u)∥2\lVert\alpha^{(L)}-u-LG_\sigma(u)\rVert_2, this shows that the numerator in (19) is Op(L2aσ2)O_p(L^2a_\sigma^2).

It remains to control the denominator. With xˉ:=M−1∑jxj\bar{x}:=M^{-1}\sum_jx_j and Zˉ:=n−1∑iZi\bar{Z}:=n^{-1}\sum_iZ_i, a first-order softmax expansion at the uniform weights gives

[Gσ(u)]j=⟨Zˉ,xj−xˉ⟩Mσ+Op(σ−2).[G_\sigma(u)]_j=\frac{\langle\bar{Z},x_j-\bar{x}\rangle}{M\sigma}+O_p(\sigma^{-2}).

Since the grid is fixed, aσ∼Rx/σa_\sigma\sim R_x/\sigma. The bound on α(L)−u−LGσ(u)\alpha^{(L)}-u-LG_\sigma(u) and Laσ→0La_\sigma\to0 therefore give

α(L)−uLaσ=Gσ(u)aσ+Op(Laσ)=1MRx(⟨Zˉ,xj−xˉ⟩)j=1M+op(1).\frac{\alpha^{(L)}-u}{La_\sigma}=\frac{G_\sigma(u)}{a_\sigma}+O_p(La_\sigma)=\frac{1}{MR_x}\left(\langle\bar{Z},x_j-\bar{x}\rangle\right)_{j=1}^{M}+o_p(1).

Because Rx>0R_x>0, the map z↦(⟨z,xj−xˉ⟩)j=1Mz\mapsto\left(\langle z,x_j-\bar{x}\rangle\right)_{j=1}^{M} has positive rank, so the Gaussian vector on the right is nonzero almost surely. Consequently, Laσ/∥α(L)−u∥2=Op(1)La_\sigma/\lVert\alpha^{(L)}-u\rVert_2=O_p(1). Combining this with the bound on the numerator gives

∥α(L)−α~∥2∥α(L)−u∥2=∥α(L)−α~∥2L2aσ2Laσ∥α(L)−u∥2Laσ=Op(Laσ)=op(1).\frac{\lVert\alpha^{(L)}-\widetilde{\alpha}\rVert_2}{\lVert\alpha^{(L)}-u\rVert_2} =\frac{\lVert\alpha^{(L)}-\widetilde{\alpha}\rVert_2}{L^2a_\sigma^2}\frac{La_\sigma}{\lVert\alpha^{(L)}-u\rVert_2}La_\sigma =O_p(La_\sigma)=o_p(1).

Proof of Proposition 7. Let f=−Ff=-F, ϕ(s)=f(β+sd)\phi(s)=f(\beta+sd), and q(s)=ϕ′′(s)q(s)=\sqrt{\phi''(s)}. If q(0)=0q(0)=0, then Kd=0Kd=0 and the claim is immediate. Otherwise, Proposition 3 gives ∣q′(s)∣≤n q(s)2|q'(s)|\le\sqrt{n}\,q(s)^2, whence

q(0)1+sτ≤q(s)≤q(0)1−sτ.\frac{q(0)}{1+s\tau}\le q(s)\le\frac{q(0)}{1-s\tau}.

Squaring and integrating over s∈[0,1]s\in[0,1] gives the two bounds in (22).

Proof of Theorem 2. Start by locally defining

δN:=1σ2max⁡i,m,ℓ∣⟨xm−xℓ,ηi⟩∣.\delta_N:=\frac{1}{\sigma^2}\max_{i,m,\ell}|\langle x_m-x_\ell,\eta_i\rangle|.

For some CηC_\eta independent of α\alpha, the perturbed objective is

F~(α)=Cη+1n∑ilog⁡(∑mαmKimexp⁡{⟨xm,ηi⟩σ2}).\widetilde{F}(\alpha)=C_\eta+\frac{1}{n}\sum_i\log\left(\sum_m\alpha_mK_{im}\exp\left\{\frac{\langle x_m,\eta_i\rangle}{\sigma^2}\right\}\right).

For each ii, the exponents in the sum above differ by at most δN\delta_N. Consequently, coordinatewise, the corresponding summands of the two gradients have a ratio between e−δNe^{-\delta_N} and eδNe^{\delta_N}; the same is true after averaging over ii. The KKT conditions at α~\widetilde{\alpha} then give

∥∇F(α~)−∇F~(α~)∥2≤M(eδN−1).\lVert\nabla F(\widetilde{\alpha})-\nabla\widetilde{F}(\widetilde{\alpha})\rVert_2\le\sqrt{M}\left(e^{\delta_N}-1\right).

Under (8), continuity and compactness give μN>0\mu_N>0. With d=α~−α^d=\widetilde{\alpha}-\widehat{\alpha}, strong concavity and the optimality conditions give

μN∥d∥2≤⟨∇F(α^)−∇F(α~),d⟩≤⟨∇F~(α~)−∇F(α~),d⟩.\mu_N\lVert d\rVert^2\le\langle\nabla F(\widehat{\alpha})-\nabla F(\widetilde{\alpha}),d\rangle\le\langle\nabla\widetilde{F}(\widetilde{\alpha})-\nabla F(\widetilde{\alpha}),d\rangle.

Therefore

∥d∥≤MμN(eδN−1).\lVert d\rVert\le\frac{\sqrt{M}}{\mu_N}\left(e^{\delta_N}-1\right).

At the uniform vector uu, restrict −∇2F(u)-\nabla^2F(u) to the (M−1)(M-1)-dimensional space T\mathcal{T}. The sum of its eigenvalues is M2n−1∑i∥r(yi;u)−u∥2M^2n^{-1}\sum_i\lVert r(y_i;u)-u\rVert^2. Since μN\mu_N is no larger than any of these eigenvalues and r(yi;u)r(y_i;u) is a probability vector,

(M−1)μN≤M21n∑i∥r(yi;u)−u∥2≤M(M−1).(M-1)\mu_N\le M^2\frac{1}{n}\sum_i\lVert r(y_i;u)-u\rVert^2\le M(M-1).

Here the last inequality uses ∥r(yi;u)−u∥2=∥r(yi;u)∥2−1/M≤1−1/M\lVert r(y_i;u)-u\rVert^2=\lVert r(y_i;u)\rVert^2-1/M\le1-1/M. Thus μN≤M\mu_N\le M. If δN≤1\delta_N\le1, the preceding bound and eδN−1≤2δNe^{\delta_N}-1\le2\delta_N give

∥d∥≤2MμNδN≤2MμNδN.\lVert d\rVert\le\frac{2\sqrt{M}}{\mu_N}\delta_N\le\frac{2M}{\mu_N}\delta_N.

If δN>1\delta_N>1, both maximizers lie in the simplex, whose diameter is 2\sqrt{2}. Using also μN≤M\mu_N\le M gives

∥d∥≤2≤2MμNδN.\lVert d\rVert\le\sqrt{2}\le\frac{2M}{\mu_N}\delta_N.

Hence the last bound holds in both cases. Finally,

E⟨xm−xℓ,ηi⟩2≤Rx2εN2.\mathbb{E}\langle x_m-x_\ell,\eta_i\rangle^2 \le R_x^2\varepsilon_N^2.

Chebyshev’s inequality and a finite union bound give δN=Op(RxεN/σ2)\delta_N=O_p(R_x\varepsilon_N/\sigma^2), which proves (23).

Proof of Theorem 3. Write mθ(y)=log⁡fθ(y)m_\theta(y)=\log f_\theta(y) and Pnh=n−1∑i=1nh(Yi)\mathbb{P}_n h=n^{-1}\sum_{i=1}^n h(Y_i). Write Y=X+σWY=X+\sigma W, where X∼ρX\sim\rho and W∼N(0,Id)W\sim\mathcal{N}(0,I_d) are independent. Bounded support of ρ\rho gives E∥Y∥2<∞\mathbb{E}\lVert Y\rVert^2<\infty, and

max⁡m∣log⁡pσ(Y∣xm)∣≤C(1+∥Y∥2).\max_m\left|\log p_\sigma(Y\mid x_m)\right|\le C(1+\lVert Y\rVert^2).

Since ∣mθ∣|m_\theta| is bounded by the left-hand side, compactness of Θ\Theta and continuity give

sup⁡θ∈Θ∣Pnmθ−Emθ∣⟶0in probability.\sup_{\theta\in\Theta}|\mathbb{P}_n m_\theta-\mathbb{E}m_\theta|\longrightarrow0\quad\text{in probability}.

Choose kk with αk∗>0\alpha_k^*>0. Since fθ∗≥αk∗pσ(⋅∣xk)f_{\theta^*}\ge\alpha_k^*p_\sigma(\cdot\mid x_k),

∥sθ∗(y)∥≤C∑mpσ(y∣xm)pσ(y∣xk).\lVert s_{\theta^*}(y)\rVert\le C\sum_m\frac{p_\sigma(y\mid x_m)}{p_\sigma(y\mid x_k)}.

Each ratio on the right is the exponential of an affine function of yy. The random vector Y=X+σWY=X+\sigma W has finite exponential moments in every linear direction, so Iθ∗I_{\theta^*} is finite. Moreover, for v∈RM−1v\in\mathbb{R}^{M-1},

vTIθ∗v=E[{∑j<Mvj(pσ(Y∣xj)−pσ(Y∣xM))}2fθ∗(Y)2].v^\mathsf{T}I_{\theta^*}v=\mathbb{E}\left[\frac{\left\{\sum_{j<M}v_j\left(p_\sigma(Y\mid x_j)-p_\sigma(Y\mid x_M)\right)\right\}^2}{f_{\theta^*}(Y)^2}\right].

The density pσ∗ρp_\sigma*\rho is positive everywhere, and distinct Gaussian translates are linearly independent, which can be seen by taking Fourier transforms. Hence this display can vanish only for v=0v=0, proving that Iθ∗I_{\theta^*} is positive definite.

Linearity of fθf_\theta gives the exact identity

mθ∗+h(y)−mθ∗(y)=log⁡{1+hTsθ∗(y)}.m_{\theta^*+h}(y)-m_{\theta^*}(y)=\log\{1+h^\mathsf{T}s_{\theta^*}(y)\}.

for every feasible hh. If θ∗+v\theta^*+v were another population maximizer, concavity would make the population criterion constant on the segment joining the two maximizers, whereas this identity and the finite second score moment give the strictly negative second directional derivative −vTIθ∗v-v^\mathsf{T}I_{\theta^*}v. Thus the population maximizer is unique and the uniform law above gives θ^n→θ∗\hat{\theta}_n\to\theta^* in probability.

Put Si=sθ∗(Yi)S_i=s_{\theta^*}(Y_i) and

Zn:=n(Pn−P0)sθ∗.Z_n:=\sqrt{n}(\mathbb{P}_n-\mathbb{P}_0)s_{\theta^*}.

Finite second moments give

max⁡i≤n∥Si∥n=op(1),1n∑i=1nSiSiT⟶Iθ∗,\frac{\max_{i\le n}\lVert S_i\rVert}{\sqrt{n}}=o_p(1),\qquad\frac{1}{n}\sum_{i=1}^n S_iS_i^\mathsf{T}\longrightarrow I_{\theta^*},

and the multivariate central limit theorem gives Zn⇝Z∼NM−1(0,Iθ∗−bb⊤)Z_n \rightsquigarrow Z \sim\mathcal{N}_{M-1}(0, I_{\theta^*}-bb^\top). Uniformly for uu in a fixed compact set with θ∗+u/n∈Θ\theta^* + u/\sqrt{n} \in\Theta, Taylor expansion in the exact identity now gives

n(Pnmθ∗+u/n−Pnmθ∗)=nb⊤u+Zn⊤u−12u⊤Iθ∗u+op(1).n\left(\mathbb{P}_n m_{\theta^*+u/\sqrt{n}}-\mathbb{P}_n m_{\theta^*}\right) = \sqrt{n}b^\top u + Z_n^\top u - \frac{1}{2}u^\top I_{\theta^*}u + o_p(1).

Population optimality gives b⊤u≤0b^\top u \le0 in every feasible local direction. If λ\lambda is the smallest eigenvalue of Iθ∗I_{\theta^*}, the last display is therefore at most R∥Zn∥−λR2/2+op(1)R\lVert Z_n\rVert-\lambda R^2/2+o_p(1) on the feasible sphere ∥u∥=R\lVert u\rVert=R. Consistency, concavity, and then R→∞R \to\infty give n(θ^n−θ∗)=Op(1)\sqrt{n}(\hat{\theta}_n-\theta^*)=O_p(1).

Because Θ\Theta is defined by linear inequalities, its root-nn local feasible directions eventually satisfy exactly the first two conditions in the definition of C\mathcal{C} on every compact set. Along a convergent feasible sequence, the local criterion tends to −∞-\infty if b⊤u<0b^\top u<0; if u∈Cu\in\mathcal{C}, its upper limit is at most

Z⊤u−12u⊤Iθ∗u.Z^\top u-\frac{1}{2}u^\top I_{\theta^*}u.

Conversely, every u∈Cu\in\mathcal{C} is eventually feasible and attains this limit. Tightness and the argmax argument for these concave criteria give (24); positive definiteness of Iθ∗I_{\theta^*} gives uniqueness of the limiting maximizer.

If θ∗\theta^* is interior, first-order optimality gives b=0b=0 and C=RM−1\mathcal{C}=\mathbb{R}^{M-1}. The limit is then Iθ∗−1ZI_{\theta^*}^{-1}Z, with covariance Iθ∗−1I_{\theta^*}^{-1}.

In the case where the model is correctly specified,

bj=∫{pσ(y∣xj)−pσ(y∣xM)} dy=0,b_j=\int\left\{p_{\sigma}(y\mid x_j)-p_{\sigma}(y\mid x_M)\right\}\,dy=0,

so C\mathcal{C} is the tangent cone and Z∼NM−1(0,Iθ∗)Z\sim\mathcal{N}_{M-1}(0,I_{\theta^*}). Moreover,

12u⊤Iθ∗u−Z⊤u=12∥Iθ∗1/2u−Iθ∗−1/2Z∥2−12Z⊤Iθ∗−1Z.\frac{1}{2}u^\top I_{\theta^*}u-Z^\top u =\frac{1}{2}\left\lVert I_{\theta^*}^{1/2}u-I_{\theta^*}^{-1/2}Z\right\rVert^2-\frac{1}{2}Z^\top I_{\theta^*}^{-1}Z.

Therefore, after multiplication by Iθ∗−1/2I_{\theta^*}^{-1/2}, the minimizer is the Euclidean projection of Iθ∗−1/2Z∼NM−1(0,IM−1)I_{\theta^*}^{-1/2}Z\sim\mathcal{N}_{M-1}(0,I_{M-1}) onto Iθ∗1/2CI_{\theta^*}^{1/2}\mathcal{C}.

Additional Stopping Criterion on IgG example

We now consider a different benchmark example to illustrate the stopping-criterion issue discussed in Section 5. The simulation is from the 100 structures of the IgG-1d example from the benchmark [21], but simulated with conformations taking a multimodal distribution as considered in [15] (Figure B1). Unlike the Hsp90 example above, the EM algorithm overfits very quickly, within 1000 iterations. Here, the threshold criteria at 10−410^{-4} is too strict, unlike the previous example in Figure 3. The iteration at which overfitting begins varies considerably between examples, and understanding this stopping time is an important direction for future work. Figures B2 and B3 report the corresponding stopping diagnostics.

Comparison of thresholds for the interior fixed-point diagnostic

Figure 3. Comparison of thresholds for the interior fixed-point diagnostic ∥∇F(α(t))−1M∥∞\|\nabla F(\alpha^{(t)}) - 1_M\|_{\infty} on the Hsp90 example (n=100,000n = 100{,}000, M=20M = 20). a.–c. Estimated weights at thresholds 10−210^{-2}, 10−410^{-4}, 10−610^{-6}. d.–f. Loss, diagnostic, and weight divergence KL⁡(α(t)∥αtrue)\operatorname{KL}(\alpha^{(t)}\|\alpha_{\mathrm{true}}) to the displayed true weights. The saved stopping iterations are 19, 603, and 369458.

Example structures of the IgG molecule and distribution along the dihedral angle

Figure B1. Example structures of the IgG molecule used in Appendix B, with diagram of the distribution of structures along the dihedral angle θ\theta, as considered in [15].

Comparison of various thresholds for the gap

Figure B2. Comparison of various thresholds for the gap ∥∇F(αt)−1∥∞\lVert\nabla F(\alpha_t) - 1 \rVert_{\infty} on the IgG example from Appendix B. a.-c. Estimated weights at thresholds 10−210^{-2}, 10−410^{-4}, 10−610^{-6} respectively. d.-f. Loss, gap, and divergence to true weights respectively, plotted against iteration number. The blue, orange, and green lines denote the iteration index where the 10−210^{-2}, 10−410^{-4}, 10−610^{-6} gap thresholds are reached.

References

  1. [1]Abramson, Josh et al. (2024). “Accurate structure prediction of biomolecular interactions with AlphaFold 3”. In: Nature 630.8016, pp. 493–500.DOI
  2. [2]Astore, Miro A et al. (2025). “The inaugural flatiron institute cryo-em conformational heterogeneity challenge”. In: bioRxiv.DOI
  3. [3]Balakrishnan, Sivaram an, Martin J Wainwright, and Bin Yu (2017). “Statistical guarantees for the EM algorithm: From population to sample-based analysis”. In: The Annals of Statistics 45.1, pp. 77–120.
  4. [4]Balanov, Amnon, Wasim Huleihel, and Tamir Bendory (2025). “Expectation-maximization for multi-reference alignment: Two pitfalls and one remedy”. In: arXiv preprint arXiv:2505.21435.
  5. [5]Bertero, Mario et al. (2009). “Image deblurring with Poisson data: from cells to galaxies”. In: Inverse Problems 25.12, p. 123006.DOI
  6. [6]Bottaro, Sandro, Tone Bengtsen, and Kresten Lindorff-Larsen (2020). “Integrating molecular simulation and experimental data: a Bayesian/maximum entropy reweighting approach”. In: Structural bioinformatics: methods and protocols. Springer, pp. 219–240.
  7. [7]Bottaro, Sandro and Kresten Lindorff-Larsen (2018). “Biophysical experiments and biomolecular simulations: A perfect match?” In: Science 361.6400, pp. 355–360.DOI
  8. [8]Boyd, Stephen and Lieven Vandenberghe (2004). Convex optimization. Cambridge university press.
  9. [9]Chen, Hong-Bin and Jonathan Niles-Weed (2021). “Asymptotics of smoothed Wasserstein distances”. In: Potential Analysis, pp. 1–25.
  10. [10]Chok, James and Geoffrey M Vasil (2025). “Optimization Over a Probability Simplex”. In: Journal of Machine Learning Research 26.73, pp. 1–35.
  11. [11]Chrétien, Stéphane and Alfred O Hero (2008). “On EM algorithms and their proximal generalizations”. In: ESAIM: Probability and Statistics 12, pp. 308–326.
  12. [12]Cover, T. (1984). “An algorithm for maximizing expected log investment return”. In: IEEE Transactions on Information Theory 30.2, pp. 369–373.DOI
  13. [13]Dingeldein, Lars et al. (2025). “Amortized template matching of molecular conformations from cryoelectron microscopy images using simulation-based inference”. In: Proceedings of the National Academy of Sciences 122.23, e2420158122.DOI
  14. [14]Doerr, Allison (2022). “A dynamic direction for cryo-EM”. In: Nature Methods 19.1, pp. 29–29.
  15. [15]Evans, Luke et al. (2026). “Counting particles in cryo-electron microscopy may result in incorrect population estimates”. In: Communications Biology 9.1, p. 421.DOI
  16. [16]Fan, Zhou et al. (2024). “Maximum likelihood for high-noise group orbit estimation and single-particle cryo-EM”. In: The Annals of Statistics 52.1, pp. 52–77.
  17. [17]Geyer, Charles J. (1994). “On the asymptotics of constrained M-estimation”. In: The Annals of Statistics 22.4, pp. 1993–2010.
  18. [18]Gilles, Marc Aurèle and Amit Singer (2025). “Cryo-EM heterogeneity analysis using regularized covariance estimation and kernel regression”. In: Proceedings of the National Academy of Sciences 122.9, e2419140122.
  19. [19]Giraldo-Barreto, Julian et al. (2021). “A Bayesian approach to extracting free-energy profiles from cryo-electron microscopy experiments”. In: Scientific reports 11.1, p. 13657.
  20. [20]Ignatiadis, Nikolaos and Bodhisattva Sen (2024). Empirical Bayes. Preliminary lecture notes.
  21. [21]Jeon, Minkyu et al. (2024). “CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM”. In: Advances in Neural Information Processing Systems 37, pp. 89468–89512.arxiv.org/abs/2408.05526
  22. [22]Jiang, Wenhua and Cun-Hui Zhang (2009). “General maximum likelihood empirical Bayes estimation of normal means”. In: The Annals of Statistics 37.4, pp. 1647–1684.DOI
  23. [23]Johnstone, Iain M. and Bernard W. Silverman (1991). “Discretization effects in statistical inverse problems”. In: Journal of Complexity 7.1, pp. 1–34.
  24. [24]Jung, Hendrik et al. (2023). “Machine-guided path sampling to discover mechanisms of molecular self-organization”. In: Nature Computational Science 3.4, pp. 334–345.DOI
  25. [25]Katsevich, Anya and Afonso S Bandeira (2023). “Likelihood maximization and moment matching in low SNR Gaussian mixture models”. In: Communications on Pure and Applied Mathematics 76.4, pp. 788–842.
  26. [26]Keys, Kevin L, Hua Zhou, and Kenneth Lange (2019). “Proximal distance algorithms: Theory and practice”. In: Journal of Machine Learning Research 20.66, pp. 1–38.
  27. [27]Kim, Youngseok et al. (2020). “A fast algorithm for maximum likelihood estimation of mixture proportions using sequential quadratic programming”. In: Journal of Computational and Graphical Statistics 29.2, pp. 261–273.
  28. [28]Koenker, Roger and Ivan Mizera (2014). “Convex optimization, shape constraints, compound decisions, and empirical Bayes rules”. In: Journal of the American Statistical Association 109.506, pp. 674–685.DOI
  29. [29]Laird, Nan (1978). “Nonparametric maximum likelihood estimation of a mixing distribution”. In: Journal of the American Statistical Association 73.364, pp. 805–811.
  30. [30]Lindsay, Bruce G (1995). Mixture models: theory, geometry, and applications. Vol. 5. NSF-CBMS Regional Conference Series in Probability and Statistics. Institute of Mathematical Statistics.
  31. [31]Lucy, Leon B (1974). “An Iterative Technique for the Rectification of Observed Distributions”. In: Astronomical Journal, Vol. 79, p. 745 (1974) 79, p. 745.DOI
  32. [32]Lyumkis, Dmitry et al. (2013). “Likelihood-Based Classification of Cryo-EM Images Using FREALIGN”. In: Journal of structural biology 183.3, pp. 377–388.DOI
  33. [33]Mattingly, Henry H, Luke Evans, and Pilar Cossio (2026). “Measurement-limited learning of conformational heterogeneity in cryo-electron microscopy”. In: arXiv preprint arXiv:2606.14449.arxiv.org/abs/2606.14449
  34. [34]Nakane, Takanori et al. (2020). “Single-particle cryo-EM at atomic resolution”. In: Nature 587.7832, pp. 152–156.
  35. [35]O’Brien, Michael J et al. (2026). “CryoJAX: a cryo-electron microscopy image-simulation library in JAX”. In: Acta Crystallographica Section D: Structural Biology 82.3, pp. 155–167.
  36. [36]Punjani, Ali et al. (2017). “cryoSPARC: algorithms for rapid unsupervised cryo-EM structure determination”. In: Nature Methods 14, pp. 290–296.DOI
  37. [37]Redner, Richard A and Homer F Walker (1984). “Mixture densities, maximum likelihood and the EM algorithm”. In: SIAM review 26.2, pp. 195–239.
  38. [38]Robbins, Herbert (1950). “A generalization of the method of maximum likelihood-estimating a mixing distribution”. In: Annals of mathematical statistics. Vol. 21. 2, pp. 314–315.
  39. [39]Scheres, Sjors H. W. (2012). “RELION: implementation of a Bayesian approach to cryo-EM structure determination”. In: Journal of Structural Biology 180.3, pp. 519–530.DOI
  40. [40]Seitz, Evan et al. (2019). “Simulation of cryo-EM ensembles from atomic models of molecules exhibiting continuous conformations”. In: BioRxiv, p. 864116.DOI
  41. [41]Silva-Sánchez, David et al. (2026). “Cryo-Electron Microscopy Structural Ensemble Optimization Using Individual Particles”. In: Journal of Chemical Theory and Computation 22.10, pp. 4922–4938.
  42. [42]Singer, Amit and Fred J Sigworth (2020). “Computational methods for single-particle electron cryomicroscopy”. In: Annual review of biomedical data science 3.1, pp. 163–190.DOI
  43. [43]Tang, Wai Shing et al. (2023). “Ensemble reweighting using cryo-EM particle images”. In: The Journal of Physical Chemistry B 127.24, pp. 5410–5421.
  44. [44]Vardi, Y. and D. Lee (1993). “From Image Deblurring to Optimal Investments: Maximum Likelihood Solutions for Positive Linear Inverse Problems”. In: Journal of the Royal Statistical Society: Series B (Methodological) 55.3, pp. 569–598.
  45. [45]Ward, Andrew B, Andrej Sali, and Ian A Wilson (2013). “Integrative structural biology”. In: Science 339.6122, pp. 913–915.
  46. [46]Zhang, Yangjing et al. (2024). “On efficient and scalable computation of the nonparametric maximum likelihood estimator in mixture models”. In: Journal of Machine Learning Research 25.8, pp. 1–46.

Paper details

vlnc:W_cat5afqt
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 3

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents