betaFeedback

WorksMathematics

Equilibrium measures for anisotropic exponential interaction energies with applications to transformers

Abstract

We study the minimization of anisotropic exponential interaction energies over probability measures on the sphere, motivated by mean-field models of self-attention in transformer architectures. For every symmetric positive definite interaction matrix, we relate the minimizer to a corresponding anisotropic Brownian motion and use the Feynman–Kac formula and elliptic regularity theory to show that it admits a smooth, strictly positive density on the sphere. We further show that it is the only stationary measure with full support for the dynamics considered. We use the stochastic viewpoint to derive an algorithm for sampling from the minimizer. In dimension two, we also propose a numerical scheme for approximating its density based on a Mathieu series expansion.

Introduction

The equilibrium behavior of interacting particle systems is a classical topic in statistical physics, including a variety of different phenomena such as clustering, screening, or crystallization (cf. [42, 31, 20, 35]). Analytical insights for large particle systems can be gained from a mean-field perspective, which opens the way to methods in calculus of variations (in the space of measures) or partial differential equations (cf. [25, 7, 8, 38, 11, 10, 37]), recently also extending the range of applications far beyond physics, e.g. to systems in biology or social behavior. A recent unexpected development in the field is the study of large-scale behavior in transformer architectures, widely used in modern machine learning, through interacting particle systems. This perspective helps explain some of the rich behavior observed in computations. The normalization mechanisms in these architectures naturally lead to problems on the sphere, whereas most of particle systems in other settings are studied in the full space. Moreover, even with a strong level of simplification relative to realistic transformer architectures, the anisotropy of the interaction remains important, distinguishing these problems from the predominant isotropic settings in the study of equilibrium measures.

In this paper, our main object of interest is the following energy functional on the space of Borel probability measures on the unit sphere Sd−1\mathbb{S}^{d-1}, d≥2d \ge2,

ED(μ)≔12∬Sd−1×Sd−1KD(x,y) μ(dx)μ(dy),μ∈P(Sd−1),(1)\mathcal{E}_{D}(\mu) \coloneqq\frac{1}{2} \iint_{\mathbb{S}^{d-1} \times\mathbb{S}^{d-1}} K_{D}(x,y)\,\mu(\mathrm{d}x)\mu(\mathrm{d}y), \qquad\mu\in\mathcal{P}(\mathbb{S}^{d-1}), \tag*{(1)}

where the interaction kernel is given by KD(x,y):=exTDyK_D(x,y) := e^{x^{\mathsf{T}}Dy} with a symmetric and positive definite matrix D∈Rd×dD \in\mathbb{R}^{d \times d}.

Variational problems of this form, with various interaction kernels KDK_D, arise in several areas, including aggregation and swarming models, where attractive and repulsive interactions determine the shape of equilibrium configurations, e.g. Carrillo et al. [12], models of collective alignment on the sphere, e.g. Frouvelle and Liu [16], and in particular mean-field self-attention models in transformer architectures, e.g. Sander et al. [32]; Geshkovski et al. [17] and subsequent works (due to the large and ever expanding literature on transformers we do not attempt a review; we restrict attention to works most directly relevant to our setting). For D=λID = \lambda I with λ>0\lambda> 0, it is known Borodachov et al. [5] that the unique minimizer is the uniform distribution on the sphere; this has also been noted by Geshkovski et al. [17] in the context of transformers. In the anisotropic case, the properties of the minimizer depend heavily on the eigenvalues of the matrix DD [9]. When DD is not positive definite, the minimizer may be supported on a proper subspace spanned by eigenvectors of DD, and is therefore singular with respect to the surface measure. For D≻0D \succ0, however, it was conjectured that the minimizer has full support. In this paper we prove this conjecture using techniques from stochastic analysis and elliptic PDE theory.

Main contributions and structure of the paper. Our main contributions are as follows.

  • We show that the minimizer of (1) is the exit distribution of a certain stochastic process (killed anisotropic Brownian motion; § 3.1) and, furthermore, that it has a smooth and strictly positive density with respect to the surface measure on the sphere (§ 3.2), thus resolving the full-support conjecture of Burger et al. [9].

  • We show that, for the dynamics considered, the minimizer is the only stationary measure having full support and that every other stationary measure is singular with respect to the surface measure (§ 4.1).

  • We analyze the stability of the minimizer (in the maximum mean discrepancy induced by the interaction kernel) with respect to perturbations in the matrix DD (§ 4.2).

  • We propose an algorithm for sampling from the minimizer using the stochastic process introduced in § 3.1 (§ 5.1).

  • For d=2d = 2, we propose a numerical scheme for approximating the minimizing density based on a Mathieu series expansion (§ 5.2).

In § 2 we collect the properties of the variational problem that will be used in the rest of the paper. Our notation is summarized below.

Notation. Throughout the paper, D∈Rd×dD \in\mathbb{R}^{d \times d} with d≥2d \ge2 is a symmetric positive definite matrix, i.e. DT=DD^{\mathsf{T}} = D and D≻0D \succ0. For y∈Rdy \in\mathbb{R}^d we denote by diag⁡(y)∈Rd×d\operatorname{diag}(y) \in\mathbb{R}^{d \times d} the diagonal matrix with diag⁡(y)ii=yi\operatorname{diag}(y)_{ii} = y_i for all i∈{1,…,d}i \in\{1,\ldots,d\}. We use ∥⋅∥\|\cdot\| for the Euclidean norm and Sd−1={x∈Rd:∥x∥=1}\mathbb{S}^{d-1} = \{x \in\mathbb{R}^d : \|x\| = 1\} and Bd:={y∈Rd:∥y∥<1}\mathbb{B}^d := \{y \in\mathbb{R}^d : \|y\| < 1\} for the unit sphere and the open unit ball, respectively. For matrices, we denote by II, det⁡\det, Tr⁡\operatorname{Tr}, and ∥⋅∥op\|\cdot\|_{\mathrm{op}} respectively the identity, determinant, trace, and operator norm induced by ∥⋅∥\|\cdot\|. We write a∧b:=min⁡{a,b}a \wedge b := \min\{a,b\}. For a twice continuously differentiable function uu, we denote by ∇2u\nabla^2 u its Hessian. Let ∇Sd−1\nabla_{\mathbb{S}^{d-1}} and div⁡Sd−1\operatorname{div}_{\mathbb{S}^{d-1}} denote the intrinsic gradient and divergence.

We denote by P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}) the space of Borel probability measures over Sd−1\mathbb{S}^{d-1}, by dSdS the surface measure, and by dσd−1d\sigma_{d-1} the normalized surface measure. We write δz\delta_z for the Dirac measure at zz, supp⁡μ\operatorname{supp}\mu for the support of μ\mu, and F#μF_{\#}\mu for its push-forward under FF, defined by (F#μ)(A)=μ(F−1(A))(F_{\#}\mu)(A) = \mu(F^{-1}(A)). We write μ⊥ν\mu\perp\nu when the measures μ\mu and ν\nu are mutually singular, that is, concentrated on disjoint Borel sets. The spaces of real-valued continuous and smooth functions on Sd−1\mathbb{S}^{d-1} are denoted by C(Sd−1)C(\mathbb{S}^{d-1}) and C∞(Sd−1)C^\infty(\mathbb{S}^{d-1}), respectively. We write Cc2(Rd)C_c^2(\mathbb{R}^d) for the space of twice continuously differentiable functions on Rd\mathbb{R}^d with compact support. Furthermore, we denote by L2(Sd−1)L^2(\mathbb{S}^{d-1}) the Hilbert space of Lebesgue functions on Sd−1\mathbb{S}^{d-1} with respect to the surface measure dSdS, equipped with the inner product

⟨f,g⟩L2:=∫Sd−1fg dS,f,g∈L2(Sd−1).\langle f,g\rangle_{L^2}:=\int_{\mathbb{S}^{d-1}}fg\,\mathrm{d}S,\qquad f,g\in L^2(\mathbb{S}^{d-1}).

For a sequence (μn)n≥1⊂P(Sd−1)(\mu_n)_{n\geq1}\subset\mathcal{P}(\mathbb{S}^{d-1}) and μ∈P(Sd−1)\mu\in\mathcal{P}(\mathbb{S}^{d-1}), weak convergence, written μn⇀μ\mu_n\rightharpoonup\mu, means that

∫Sd−1φ dμn⟶∫Sd−1φ dμfor every φ∈C(Sd−1).\int_{\mathbb{S}^{d-1}}\varphi\,\mathrm{d}\mu_n\longrightarrow\int_{\mathbb{S}^{d-1}}\varphi\,\mathrm{d}\mu\qquad\text{for every }\varphi\in C(\mathbb{S}^{d-1}).

For a finite signed Borel measure σ\sigma on Sd−1\mathbb{S}^{d-1}, we denote its total variation norm by

∥σ∥TV:=sup⁡f∈C(Sd−1)∥f∥∞≤1∣∫Sd−1f dσ∣.\|\sigma\|_{\mathrm{TV}}:=\sup_{\substack{f\in C(\mathbb{S}^{d-1})\\\|f\|_{\infty}\leq1}}\left|\int_{\mathbb{S}^{d-1}}f\,\mathrm{d}\sigma\right|.

The Wasserstein distance (of order 2) is defined by

W2(μ,ν):=(inf⁡γ∈Π(μ,ν)∫Sd−1×Sd−1dSd−1(x,y)2 γ(dx,dy))1/2,μ,ν∈P(Sd−1),W_2(\mu,\nu):=\left(\inf_{\gamma\in\Pi(\mu,\nu)}\int_{\mathbb{S}^{d-1}\times\mathbb{S}^{d-1}}d_{\mathbb{S}^{d-1}}(x,y)^2\,\gamma(\mathrm{d}x,\mathrm{d}y)\right)^{1/2},\qquad\mu,\nu\in\mathcal{P}(\mathbb{S}^{d-1}),

where Π(μ,ν)\Pi(\mu,\nu) is the set of Borel probability measures on Sd−1×Sd−1\mathbb{S}^{d-1}\times\mathbb{S}^{d-1} with marginals μ\mu and ν\nu, and dSd−1(x,y):=arccos⁡(x⊤y)d_{\mathbb{S}^{d-1}}(x,y):=\arccos(x^\top y) is the geodesic distance. Since Sd−1\mathbb{S}^{d-1} is compact, the Wasserstein distance metrizes the weak topology [45 Corollary 6.13].

Properties of the variational problem

In this section we collect some results about the variational problem associated with the energy ED\mathcal{E}_D that will be used throughout the paper. We consider

argmin⁡μ∈P(Sd−1){ED(μ)=12∬Sd−1×Sd−1KD(x,y) μ(dx)μ(dy)}withKD(x,y)=ex⊤Dy,(2)\underset{\mu\in\mathcal{P}(\mathbb{S}^{d-1})}{\operatorname{argmin}}\left\{\mathcal{E}_D(\mu)=\frac{1}{2}\iint_{\mathbb{S}^{d-1}\times\mathbb{S}^{d-1}}K_D(x,y)\,\mu(\mathrm{d}x)\mu(\mathrm{d}y)\right\} \qquad\text{with}\qquad K_D(x,y)=e^{x^\top D y}, \tag*{(2)}

where, as before, DD is symmetric and positive definite. We first establish existence and uniqueness of a global minimizer. We then introduce the Euler–Lagrange criterion, which will guide the construction of the minimizer in § 3. Finally, we relate the energy gap to a distance between probability measures induced by the interaction kernel.

Proposition 1 (Existence and uniqueness). The problem (2) admits a unique minimizer, which we denote by μD\mu_D.

Proof. The space P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}) is weakly compact, and continuity of KDK_D on Sd−1×Sd−1\mathbb{S}^{d-1}\times\mathbb{S}^{d-1} makes ED\mathcal{E}_D weakly continuous. Hence the minimum is attained. By Burger et al. [9], the energy ED\mathcal{E}_D is strictly convex on P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}), that is

ED((1−t)μ+tν)<(1−t)ED(μ)+tED(ν),μ,ν∈P(Sd−1), μ≠ν.(3)\mathcal{E}_D((1-t)\mu+t\nu)<(1-t)\mathcal{E}_D(\mu)+t\mathcal{E}_D(\nu),\qquad\mu,\nu\in\mathcal{P}(\mathbb{S}^{d-1}),\ \mu\neq\nu. \tag*{(3)}

Therefore the minimizer is unique. □

Euler–Lagrange optimality criterion. For a finite signed Borel measure ρ\rho on Sd−1\mathbb{S}^{d-1}, define its potential by

Uρ(x):=∫Sd−1KD(x,y) ρ(dy),x∈Sd−1,(4)U_\rho(x):=\int_{\mathbb{S}^{d-1}}K_D(x,y)\,\rho(\mathrm{d}y),\qquad x\in\mathbb{S}^{d-1}, \tag*{(4)}

where we simplify notation by suppressing the dependence on DD.

Proposition 2 (Euler–Lagrange criterion (Bilyk et al., [4]).) For μ∈P(Sd−1)\mu\in\mathcal{P}(\mathbb{S}^{d-1}),

μ=μD⟺{Uμ(x)≥2ED(μ)for every x∈Sd−1,Uμ(x)=2ED(μ)for every x∈supp⁡μ.(5)\mu= \mu_D \quad\Longleftrightarrow\quad \begin{cases} U_\mu(x) \ge2\mathcal{E}_D(\mu) & \text{for every } x \in\mathbb{S}^{d-1},\\ U_\mu(x) = 2\mathcal{E}_D(\mu) & \text{for every } x \in\operatorname{supp}\mu. \end{cases} \tag*{(5)}

In particular, by uniqueness of the minimizer, any probability measure whose potential is constant on Sd−1\mathbb{S}^{d-1} is equal to μD\mu_D.

The criterion says that the support of an equilibrium measure lies where its potential is smallest. A measure with constant potential on the whole sphere therefore provides a particularly convenient candidate: once it is known to be a probability measure, minimality follows. This observation will be the starting point of the proofs of main results in § 3.

Interpretation using reproducing kernel Hilbert spaces. We next show that the interaction kernel KDK_D is strictly positive definite whenever DD is symmetric and positive definite. Set

w(x):=ex⊤Dx/2,g(u,v):=e−∥u−v∥2/2.w(x) := e^{x^\top D x/2}, \qquad g(u,v) := e^{-\lVert u-v\rVert^2/2}.

The identity

KD(x,y)=w(x)w(y)g(D1/2x,D1/2y)(6)K_D(x,y) = w(x)w(y)g(D^{1/2}x,D^{1/2}y) \tag*{(6)}

shows that KDK_D is strictly positive definite: the Gaussian kernel gg is strictly positive definite (Sriperumbudur et al., [41]), the map x↦D1/2xx \mapsto D^{1/2}x is injective, and ww is strictly positive.

Thus there exists a unique reproducing kernel Hilbert spaces (RKHS) HD\mathcal{H}_D with reproducing kernel KDK_D (Aronszajn, [2]), whose canonical feature map is

x⟼KD(x,⋅)∈HD,x∈Sd−1.x \longmapsto K_D(x,\cdot) \in\mathcal{H}_D, \qquad x \in\mathbb{S}^{d-1}.

For a finite signed Borel measure ρ\rho, its potential

Uρ=∫Sd−1KD(y,⋅) ρ(dy)∈HD(7)U_\rho= \int_{\mathbb{S}^{d-1}} K_D(y,\cdot)\,\rho(dy) \in\mathcal{H}_D \tag*{(7)}

is its kernel mean embedding (Simon-Gabriel and Schölkopf, [39]). The Bochner integral (7) is well defined since ∥KD(y,⋅)∥HD2=KD(y,y)\lVert K_D(y,\cdot)\rVert_{\mathcal{H}_D}^2 = K_D(y,y) is bounded on Sd−1\mathbb{S}^{d-1}. The reproducing property gives, for finite signed Borel measures ρ,η\rho,\eta,

⟨Uρ,Uη⟩HD=∬Sd−1×Sd−1KD(x,y) ρ(dx)η(dy),(8)\langle U_\rho,U_\eta\rangle_{\mathcal{H}_D} = \iint_{\mathbb{S}^{d-1}\times\mathbb{S}^{d-1}} K_D(x,y)\,\rho(dx)\eta(dy), \tag*{(8)}

so that

ED(μ)=12∥Uμ∥HD2,μ∈P(Sd−1).(9)\mathcal{E}_D(\mu) = \frac{1}{2}\lVert U_\mu\rVert_{\mathcal{H}_D}^2, \qquad\mu\in\mathcal{P}(\mathbb{S}^{d-1}). \tag*{(9)}

For μ,ν∈P(Sd−1)\mu,\nu\in\mathcal{P}(\mathbb{S}^{d-1}), the associated maximum mean discrepancy (Gretton et al., [19]) satisfies

MMD⁡D(μ,ν):=sup⁡∥f∥HD≤1∣∫Sd−1f d(μ−ν)∣=∥Uμ−Uν∥HD.\operatorname{MMD}_D(\mu,\nu) := \sup_{\lVert f\rVert_{\mathcal{H}_D}\le1}\left|\int_{\mathbb{S}^{d-1}} f\,d(\mu-\nu)\right| = \lVert U_\mu-U_\nu\rVert_{\mathcal{H}_D}.

Proposition 3. The distance MMD⁡D\operatorname{MMD}_D metrizes the weak topology on P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}); that is, for (μn)n≥1⊂P(Sd−1)(\mu_n)_{n\ge1} \subset\mathcal{P}(\mathbb{S}^{d-1}) and μ∈P(Sd−1)\mu\in\mathcal{P}(\mathbb{S}^{d-1}),

μn⇀μ⟺MMD⁡D(μn,μ)⟶0.\mu_n \rightharpoonup\mu\quad\Longleftrightarrow\quad\operatorname{MMD}_D(\mu_n,\mu)\longrightarrow0.

Proof. We use Gaussian factorization of the kernel KDK_D given in (6). For a finite signed Borel measure ρ\rho on Sd−1\mathbb{S}^{d-1}, define the finite signed measure ρ~:=D#1/2(wρ)\tilde{\rho}:=D_{\#}^{1/2}(w\rho) on Rd\mathbb{R}^{d}. Substituting this into (8) yields

∥Uρ∥HD2=⟨Uρ,Uρ⟩HD=∬Rd×Rdg(u,v)ρ~(du)ρ~(dv).\lVert U_{\rho}\rVert_{\mathcal{H}_{D}}^{2}=\langle U_{\rho},U_{\rho}\rangle_{\mathcal{H}_{D}}=\iint_{\mathbb{R}^{d}\times\mathbb{R}^{d}}g(u,v)\tilde{\rho}(\mathrm{d}u)\tilde{\rho}(\mathrm{d}v).

The Gaussian kernel gg is integrally strictly positive definite (Sriperumbudur et al. [41]), so this integral is strictly positive whenever ρ~≠0\tilde{\rho}\ne0. Since D1/2D^{1/2} is invertible and ww is strictly positive on Sd−1\mathbb{S}^{d-1}, the map ρ↦ρ~\rho\mapsto\tilde{\rho} is injective. Thus ∥Uρ∥HD2>0\lVert U_{\rho}\rVert_{\mathcal{H}_{D}}^{2}>0 for every ρ≠0\rho\ne0.

In particular, the kernel mean embedding is injective on P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}), so KDK_{D} is characteristic. Since KDK_{D} is continuous and Sd−1\mathbb{S}^{d-1} is compact, Simon-Gabriel et al. [40] implies that MMD⁡D\operatorname{MMD}_{D} metrizes the weak topology on P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}).

We now use this norm to relate the energy gap to the distance from the minimizer.

Proposition 4 (Energy gap). For every ν∈P(Sd−1)\nu\in\mathcal{P}(\mathbb{S}^{d-1}),

ED(ν)−ED(μD)≥12MMD⁡D(ν,μD)2.(10)\mathcal{E}_{D}(\nu)-\mathcal{E}_{D}(\mu_{D})\ge\frac{1}{2}\operatorname{MMD}_{D}(\nu,\mu_{D})^{2}. \tag*{(10)}

Proof. Writing Uν=UμD+(Uν−UμD)U_{\nu}=U_{\mu_{D}}+(U_{\nu}-U_{\mu_{D}}) and using (9), we obtain

ED(ν)−ED(μD)=⟨UμD,Uν−UμD⟩HD+12∥Uν−UμD∥HD2.\mathcal{E}_{D}(\nu)-\mathcal{E}_{D}(\mu_{D})=\langle U_{\mu_{D}},U_{\nu}-U_{\mu_{D}}\rangle_{\mathcal{H}_{D}}+\frac{1}{2}\lVert U_{\nu}-U_{\mu_{D}}\rVert_{\mathcal{H}_{D}}^{2}.

By linearity of the kernel mean embedding and the reproducing property, the cross-term is

⟨UμD,Uν−UμD⟩HD=∫Sd−1⟨UμD,KD(y,⋅)⟩HD⏟UμD(y)(ν−μD)(dy).\langle U_{\mu_{D}},U_{\nu}-U_{\mu_{D}}\rangle_{\mathcal{H}_{D}}=\int_{\mathbb{S}^{d-1}}\underbrace{\langle U_{\mu_{D}},K_{D}(y,\cdot)\rangle_{\mathcal{H}_{D}}}_{U_{\mu_{D}}(y)}(\nu-\mu_{D})(\mathrm{d}y).

Therefore,

ED(ν)−ED(μD)=∫Sd−1UμD d(ν−μD)+12∥Uν−UμD∥HD2=∫Sd−1(UμD−2ED(μD))⏟≥0 dν+12MMD⁡D(ν,μD)2.(11)\begin{aligned} \mathcal{E}_{D}(\nu)-\mathcal{E}_{D}(\mu_{D}) &=\int_{\mathbb{S}^{d-1}}U_{\mu_{D}\,}\mathrm{d}(\nu-\mu_{D})+\frac{1}{2}\lVert U_{\nu}-U_{\mu_{D}}\rVert_{\mathcal{H}_{D}}^{2}\\ &=\int_{\mathbb{S}^{d-1}}\underbrace{(U_{\mu_{D}}-2\mathcal{E}_{D}(\mu_{D}))}_{\ge0}\,\mathrm{d}\nu+\frac{1}{2}\operatorname{MMD}_{D}(\nu,\mu_{D})^{2}. \tag*{(11)} \end{aligned}

where we used ∫Sd−1UμD dμD=2ED(μD)\int_{\mathbb{S}^{d-1}}U_{\mu_{D}}\,\mathrm{d}\mu_{D}=2\mathcal{E}_{D}(\mu_{D}) and ν(Sd−1)=1\nu(\mathbb{S}^{d-1})=1. The integral there is nonnegative by the Euler–Lagrange criterion (5), which proves the inequality. □\square

Remark 1. If supp⁡μD=Sd−1\operatorname{supp}\mu_{D}=\mathbb{S}^{d-1}, as will be proved further in Corollary 8, then (5) gives UμD=2ED(μD)U_{\mu_{D}}=2\mathcal{E}_{D}(\mu_{D}) on Sd−1\mathbb{S}^{d-1}. The integral in (10) therefore vanishes, yielding an exact equality

ED(ν)−ED(μD)=12MMD⁡D(ν,μD)2.\mathcal{E}_{D}(\nu)-\mathcal{E}_{D}(\mu_{D})=\frac{1}{2}\operatorname{MMD}_{D}(\nu,\mu_{D})^{2}.

In particular, for every ε≥0\varepsilon\ge0,

ED(ν)−ED(μD)≤ε⟹MMD⁡D(ν,μD)≤2ε.\mathcal{E}_{D}(\nu)-\mathcal{E}_{D}(\mu_{D})\le\varepsilon\quad\Longrightarrow\quad\operatorname{MMD}_{D}(\nu,\mu_{D})\le\sqrt{2\varepsilon}.

Together with Proposition 3 and weak continuity of ED\mathcal{E}_{D}, this gives the following characterization of convergence to the minimizer.

Corollary 5 (Convergence of approximate minimizers). For a sequence (νi)i≥1⊂P(Sd−1)(\nu_{i})_{i\ge1}\subset\mathcal{P}(\mathbb{S}^{d-1}), the following are equivalent:

ED(νi)⟶ED(μD)⟺MMD⁡D(νi,μD)⟶0⟺νi⇀μD.\mathcal{E}_{D}(\nu_{i})\longrightarrow\mathcal{E}_{D}(\mu_{D})\quad\Longleftrightarrow\quad\operatorname{MMD}_{D}(\nu_{i},\mu_{D})\longrightarrow0\quad\Longleftrightarrow\quad\nu_{i}\rightharpoonup\mu_{D}.

Proof. The lower bound in (10) gives the first implication, and Proposition 3 gives the second. Conversely, weak convergence to μD\mu_D implies convergence of the energies by weak continuity of ED\mathcal{E}_D. □\square

Consequently, a sequence of probability measures has energies converging to the minimum if and only if it converges weakly to μD\mu_D. These observations will allow us to translate energy estimates into approximation and stability statements.

Properties of the minimizer

We are now ready to state the main results. We show that μD\mu_D is the distribution of the exit point of a certain stochastic process and use this representation to establish the existence of a smooth, strictly positive density on Sd−1S^{d-1}, resolving conjecture of Burger et al., [9]. This representation also inspires the numerical sampling method discussed in § 5.1.

A probabilistic representation

We start with an informal sketch of the construction; a precise formulation follows below. Consider the ellipsoid DBdD\mathbb{B}^d, obtained by applying DD to the unit ball, and start an isotropic Brownian motion Bt=2WtB_t=\sqrt{2}W_t at the origin. Let τD\tau_D denote its exit time, that is, the first time the path reaches the boundary of the ellipsoid, and let BτDB_{\tau_D} be its exit location. Independently of the Brownian motion, assign to each path an exponentially distributed killing time, at which the path is deleted. Retain the exit locations of those paths which reach the boundary before being killed. Theorem 6 below states that these locations define a measure with a smooth, strictly positive density on the boundary of the ellipsoid. Mapping this measure back to the sphere by D−1D^{-1} and normalizing its mass gives precisely μD\mu_D.

While passing from the sphere to the boundary of an ellipsoid and back gives a clean geometric intuition, for the formal argument, we work directly in the unit ball with Sd−1S^{d-1} boundary. Instead of varying the domain, we use an anisotropic Brownian motion, with different rates of diffusion along the eigendirections of DD.

Tools and notation. Let (Ω,F,P)(\Omega,\mathcal{F},\mathbb{P}) be a probability space supporting a standard Brownian motion W=(Wt)t≥0W=(W_t)_{t\geq0} in Rd\mathbb{R}^d, started at the origin, and write E\mathbb{E} for expectation under P\mathbb{P}. Here a σ\sigma-algebra on Ω\Omega is a collection of subsets containing Ω\Omega and closed under complements and countable unions. A filtration is an increasing family of sub-σ\sigma-algebras of F\mathcal{F}. We denote by Ft:=σ(Ws:0≤s≤t)\mathcal{F}_t:=\sigma(W_s:0\leq s\leq t) the natural filtration of WW, where σ(Ws:0≤s≤t)\sigma(W_s:0\leq s\leq t) is the smallest σ\sigma-algebra making all WsW_s, 0≤s≤t0\leq s\leq t, measurable. A process (Xt)t≥0(X_t)_{t\geq0} is adapted to (Ft)t≥0(\mathcal{F}_t)_{t\geq0} if XtX_t is Ft\mathcal{F}_t-measurable for every t≥0t\geq0. An event is a measurable set A∈FA\in\mathcal{F}. For an introduction to the theoretical tools used in the proof, such as stochastic processes, martingales, Brownian motion, and stochastic differential equations (SDE), we refer the reader to Baldi [3]; Williams [47]; Le Gall [24].

Theorem 6 (Conditional exit measure). Consider a stochastic process XX on (Ω,F,P)(\Omega,\mathcal{F},\mathbb{P}) given by Xt:=2D−1WtX_t:=\sqrt{2}D^{-1}W_t. Define

τD:=inf⁡{t≥0:Xt∉Bd}.\tau_D:=\inf\{t\geq0:X_t\notin\mathbb{B}^d\}.

Let ζ∼Exp⁡(1)\zeta\sim\operatorname{Exp}(1) be independent of the Brownian path, so that P(ζ>t)=e−t\mathbb{P}(\zeta>t)=e^{-t} for t≥0t\geq0. Then,

μD(A)=P(XτD∈A∣τD<ζ),A⊆Sd−1 Borel.\mu_D(A)=\mathbb{P}(X_{\tau_D}\in A\mid\tau_D<\zeta),\qquad A\subseteq S^{d-1}\ \text{Borel}.

Proof. Let NP\mathcal{N}_{\mathbb{P}} denote the collection of all subsets of P\mathbb{P}-null sets in F\mathcal{F}, and set

Gt:=σ(Ft∪NP), t≥0,G∞:=σ(⋃t≥0Gt).\mathcal{G}_t:=\sigma(\mathcal{F}_t\cup\mathcal{N}_{\mathbb{P}}),\ t\geq0,\qquad\mathcal{G}_{\infty}:=\sigma\left(\bigcup_{t\geq0}\mathcal{G}_t\right).

Then (Gt)t≥0(\mathcal{G}_t)_{t\ge0} is the completed natural filtration of WW. Since DD is invertible, XX and WW generate the same natural filtration. Thus (Gt)t≥0(\mathcal{G}_t)_{t\ge0} is also the completed natural filtration of XX. Since D≻0D \succ0, the first coordinate of XX is a non-degenerate real Brownian motion, which leaves [−1,1][-1,1] almost surely (Le Gall, [24]), therefore P(τD<∞)=1\mathbb{P}(\tau_D < \infty)=1. X0=0∈BdX_0=0\in\mathbb{B}^d and XX has continuous paths, so

XτD∈Sd−1 on {τD<∞}.(12)X_{\tau_D}\in\mathbb{S}^{d-1}\text{ on }\{\tau_D<\infty\}. \tag*{(12)}

The process XX is adapted to (Gt)t≥0(\mathcal{G}_t)_{t\ge0}, therefore τD\tau_D is a (Gt)t≥0(\mathcal{G}_t)_{t\ge0}-stopping time.

Define event E:={τD<ζ}E:=\{\tau_D<\zeta\}. Conditioning on WW and using the independence of ζ\zeta, we get

MD:=P(E)=E[e−τD]>0,M_D:=\mathbb{P}(E)=\mathbb{E}[e^{-\tau_D}]>0,

since τD<∞\tau_D<\infty almost surely. By (12),

P(XτD∈Sd−1∣E)=1.\mathbb{P}(X_{\tau_D}\in\mathbb{S}^{d-1}\mid E)=1.

Let νD\nu_D be given by

νD(A):=P({XτD∈A}∩E),A⊆Sd−1 Borel.\nu_D(A):=\mathbb{P}(\{X_{\tau_D}\in A\}\cap E),\qquad A\subseteq\mathbb{S}^{d-1}\ \mathrm{Borel}.

Since XτDX_{\tau_D} is measurable, νD\nu_D is a Borel measure on Sd−1\mathbb{S}^{d-1}, with νD(Sd−1)=MD\nu_D(\mathbb{S}^{d-1})=M_D. Hence νD/MD\nu_D/M_D is a Borel probability measure on Sd−1\mathbb{S}^{d-1}, and

P(XτD∈A∣E)=P({XτD∈A}∩E)P(E)=νD(A)MD,A⊆Sd−1 Borel.\mathbb{P}(X_{\tau_D}\in A\mid E)=\frac{\mathbb{P}(\{X_{\tau_D}\in A\}\cap E)}{\mathbb{P}(E)}=\frac{\nu_D(A)}{M_D},\qquad A\subseteq\mathbb{S}^{d-1}\ \mathrm{Borel}.

By Proposition 1 and the linearity of ρ↦Uρ\rho\mapsto U_\rho, it is enough to show that UνDU_{\nu_D} is constant on Sd−1\mathbb{S}^{d-1}. Since τD\tau_D and XτDX_{\tau_D} are G∞\mathcal{G}_\infty-measurable, conditioning on G∞\mathcal{G}_\infty gives

νD(A)=E[1{XτD∈A}1{τD<ζ}]=E[1{XτD∈A}P(ζ>τD∣G∞)]=E[e−τD1{XτD∈A}],\nu_D(A)=\mathbb{E}\left[1_{\{X_{\tau_D}\in A\}}1_{\{\tau_D<\zeta\}}\right]=\mathbb{E}\left[1_{\{X_{\tau_D}\in A\}}\mathbb{P}(\zeta>\tau_D\mid\mathcal{G}_\infty)\right]=\mathbb{E}\left[e^{-\tau_D}1_{\{X_{\tau_D}\in A\}}\right],

where the last equality follows from the independence of ζ\zeta and G∞\mathcal{G}_\infty, together with P(ζ>t)=e−t\mathbb{P}(\zeta>t)=e^{-t}. Hence, by the standard machine (Williams, [47]), for every bounded Borel φ\varphi on Sd−1\mathbb{S}^{d-1},

∫Sd−1φ dνD=E[e−τDφ(XτD)].(13)\int_{\mathbb{S}^{d-1}}\varphi\,d\nu_D=\mathbb{E}[e^{-\tau_D}\varphi(X_{\tau_D})]. \tag*{(13)}

Fix x∈Sd−1x\in\mathbb{S}^{d-1}. Taking φ=KD(x,⋅)\varphi=K_D(x,\cdot) gives

UνD(x)=∫Sd−1KD(x,y) dνD(y)=E[e−τDKD(x,XτD)]=E[e−τDex⊤DXτD].(14)U_{\nu_D}(x)=\int_{\mathbb{S}^{d-1}}K_D(x,y)\,d\nu_D(y)=\mathbb{E}[e^{-\tau_D}K_D(x,X_{\tau_D})]=\mathbb{E}[e^{-\tau_D}e^{x^\top D X_{\tau_D}}]. \tag*{(14)}

Set βt:=x⊤Wt\beta_t:=x^\top W_t. Since ∥x∥=1\lVert x\rVert=1, β\beta is a real Brownian motion, and since βt−βs=x⊤(Wt−Ws)\beta_t-\beta_s=x^\top(W_t-W_s) is independent of Gs\mathcal{G}_s for s<ts<t, it is a (Gt)t≥0(\mathcal{G}_t)_{t\ge0}-Brownian motion. Hence

Zt:=exp⁡(2 βt−t)=e−tex⊤DXtZ_t:=\exp(\sqrt{2}\,\beta_t-t)=e^{-t}e^{x^\top D X_t}

is a (Gt)t≥0(\mathcal{G}_t)_{t\ge0}-martingale; see, e.g., Durrett [14], Le Gall [24]. Since X0=0∈BdX_0=0\in\mathbb{B}^d and XX has continuous paths, Xt∧τD∈BdX_{t\wedge\tau_D}\in\mathbb{B}^d for all t≥0t\ge0, so

0<Zt∧τD=e−(t∧τD)ex⊤DXt∧τD≤e∥D∥op,t≥0.0<Z_{t\wedge\tau_D}=e^{-(t\wedge\tau_D)}e^{x^\top D X_{t\wedge\tau_D}}\le e^{\lVert D\rVert_{\mathrm{op}}},\qquad t\ge0.

Thus the stopped martingale (Zt∧τD)t≥0(Z_{t\wedge\tau_D})_{t\ge0} is bounded, hence uniformly integrable. Applying the optional stopping theorem [24 Theorem 3.22] to this martingale gives

E[e−τDex⊤DXτD]=EZτD=EZ0=1.(15)\mathbb{E}\left[e^{-\tau_D}e^{x^\top D X_{\tau_D}}\right]=\mathbb{E}Z_{\tau_D}=\mathbb{E}Z_0=1. \tag*{(15)}

Putting (15) into (14) yields UνD(x)=1U_{\nu_D}(x)=1, independent of xx. Consequently, by linearity,

UνDMD(x)=1MD,x∈Sd−1.(16)\frac{U_{\nu_D}}{M_D}(x)=\frac{1}{M_D}, \qquad x\in S^{d-1}. \tag*{(16)}

The claim follows by Proposition 1.

Remark 2. The process XX is an anisotropic Brownian motion, with a covariance matrix

E[XtXt⊤]=2tD−2.\mathbb{E}[X_tX_t^\top]=2tD^{-2}.

In particular, if vv is a unit eigenvector of DD with eigenvalue λ\lambda, then v⊤Xtv^\top X_t has variance 2t/λ22t/\lambda^2. Thus diffusion is faster in directions corresponding to smaller eigenvalues of DD.

This identity gives a comparison between interaction matrices. Let E≻0E\succ0 be symmetric and suppose that D−2−E−2⪰0D^{-2}-E^{-2}\succeq0. Construct

XtD:=2D−1Wt,XtE:=2E−1Wt.X_t^D:=\sqrt{2}D^{-1}W_t,\qquad X_t^E:=\sqrt{2}E^{-1}W_t.

using the same Brownian path. Then, for every t≥0t\geq0,

∥XtD∥2−∥XtE∥2=2Wt⊤(D−2−E−2)Wt≥0.\lVert X_t^D\rVert^2-\lVert X_t^E\rVert^2=2W_t^\top(D^{-2}-E^{-2})W_t\geq0.

Whenever XEX^E reaches the sphere, XDX^D has already reached it. Their exit times therefore satisfy τD≤τE\tau_D\leq\tau_E almost surely, and hence

MD=E[e−τD]≥E[e−τE]=ME.M_D=\mathbb{E}[e^{-\tau_D}]\geq\mathbb{E}[e^{-\tau_E}]=M_E.

By (16) and Proposition 2,

min⁡μ∈P(Sd−1)ED(μ)≤min⁡μ∈P(Sd−1)EE(μ).\min_{\mu\in\mathcal{P}(S^{d-1})}\mathcal{E}_D(\mu)\leq\min_{\mu\in\mathcal{P}(S^{d-1})}\mathcal{E}_E(\mu).

In particular, the condition holds when DD and EE admit a common orthonormal eigenbasis and each eigenvalue of DD is at most the corresponding eigenvalue of EE. The comparison thus follows by coupling the paths, without computing either minimizing density.

Smoothness and positivity of the density

In this section we show that the measure μD\mu_D admits a density with respect to the surface measure, and obtain an expression via the Dirichlet Green function of a certain elliptic operator built from the generator of the diffusion process XX. The argument combines the Feynman–Kac formula with Green’s representation formula. Using elliptic regularity and Hopf’s boundary point lemma we then prove smoothness and strict positivity of the density, and hence full support of the minimizer. For a classical treatment of the elliptic theory used in the proof, we refer to Gilbarg and Trudinger [18].

Theorem 7. The measure μD\mu_D admits a density with respect to the normalized surface measure σd−1\sigma_{d-1}, given by

μD(dy)=wD(y)ZD dσd−1(y),ZD:=∫Sd−1wD dσd−1.\mu_D(\mathrm{d}y)=\frac{w_D(y)}{Z_D}\,\mathrm{d}\sigma_{d-1}(y),\qquad Z_D:=\int_{S^{d-1}}w_D\,\mathrm{d}\sigma_{d-1}.

The function wDw_D is smooth and strictly positive on Sd−1S^{d-1}.

Proof. The generator AD\mathcal{A}_D of the process XX is defined on Cc2(Rd)C_c^2(\mathbb{R}^d) by

ADg(z):=lim⁡h↓0Ez[g(Xh)]−g(z)h=lim⁡h↓0E[g(z+2D−1Wh)]−g(z)h.\mathcal{A}_D g(z) := \lim_{h\downarrow0} \frac{\mathbb{E}_z[g(X_h)]-g(z)}{h} = \lim_{h\downarrow0} \frac{\mathbb{E}\left[g\left(z+\sqrt{2}D^{-1}W_h\right)\right]-g(z)}{h}.

By Itô’s formula (Karatzas and Shreve, [21]),

ADg=12Tr⁡((2D−1)(2D−1)T∇2g)=Tr⁡(D−2∇2g)=∇⋅(D−2∇g),g∈Cc2(Rd),\mathcal{A}_D g = \frac{1}{2}\operatorname{Tr}\left((\sqrt{2}D^{-1})(\sqrt{2}D^{-1})^{\mathsf{T}}\nabla^2 g\right) = \operatorname{Tr}(D^{-2}\nabla^2 g) = \nabla\cdot(D^{-2}\nabla g), \qquad g\in C_c^2(\mathbb{R}^d),

where we used DT=DD^{\mathsf{T}}=D.

For f∈C∞(Sd−1)f\in C^\infty(\mathbb{S}^{d-1}), let u∈C2(Bd‾)u\in C^2(\overline{\mathbb{B}^d}) be the unique solution of

{(AD−I)u=0in Bd,u=fon Sd−1,(17)\begin{cases} (\mathcal{A}_D-I)u=0 & \text{in }\mathbb{B}^d,\\ u=f & \text{on }\mathbb{S}^{d-1}, \tag*{(17)} \end{cases}

whose existence follows from Gilbarg and Trudinger [18]. Since E[τD]<∞\mathbb{E}[\tau_D]<\infty by Karatzas and Shreve [21], applying the Feynman–Kac representation (Karatzas and Shreve, [21]) with operator AD\mathcal{A}_D, killing rate k=1k=1, and zero source term g=0g=0 gives

u(0)=E[e−τDf(XτD)].(18)u(0)=\mathbb{E}\left[e^{-\tau_D}f(X_{\tau_D})\right]. \tag*{(18)}

Set LD:=I−AD\mathcal{L}_D:=I-\mathcal{A}_D. The operator LD\mathcal{L}_D is uniformly elliptic, since

ξTD−2ξ=∥D−1ξ∥2≥∥D∥op−2∥ξ∥2,\xi^{\mathsf{T}}D^{-2}\xi=\lVert D^{-1}\xi\rVert^2\geq\lVert D\rVert_{\mathrm{op}}^{-2}\lVert\xi\rVert^2,

and it is formally self-adjoint because D−2D^{-2} is symmetric. On Sd−1\mathbb{S}^{d-1}, the outward unit normal is n(y)=yn(y)=y. Set nD(y):=D−2n(y)n_D(y):=D^{-2}n(y), and write

∂nDv(y):=nD(y)T∇v(y)=n(y)TD−2∇v(y),v∈C1(Bd‾∖{0}),\partial_{n_D}v(y):=n_D(y)^{\mathsf{T}}\nabla v(y)=n(y)^{\mathsf{T}}D^{-2}\nabla v(y), \qquad v\in C^1(\overline{\mathbb{B}^d}\setminus\{0\}),

for the conormal derivative. Let GDG_D denote the negative of the Dirichlet Green function of LD\mathcal{L}_D in Bd\mathbb{B}^d with pole at the origin, so that

{LDGD=−δ0in Bd,GD=0on Sd−1.\begin{cases} \mathcal{L}_D G_D=-\delta_0 & \text{in }\mathbb{B}^d,\\ G_D=0 & \text{on }\mathbb{S}^{d-1}. \end{cases}

Its existence and Green’s representation formula are shown in, e.g., Véron [44]. Since LDu=0\mathcal{L}_D u=0, this representation gives

u(0)=∫Sd−1f(y) ∂nDGD(y) dS(y).(19)u(0)=\int_{\mathbb{S}^{d-1}}f(y)\,\partial_{n_D}G_D(y)\,\mathrm{d}S(y). \tag*{(19)}

Combining (13) with the Feynman–Kac representation (18) and (19) yields, for every f∈C∞(Sd−1)f\in C^\infty(\mathbb{S}^{d-1}),

∫Sd−1f dνD=E[e−τDf(XτD)]=u(0)=∫Sd−1f(y) ∂nDGD(y) dS(y).\int_{\mathbb{S}^{d-1}}f\,\mathrm{d}\nu_D=\mathbb{E}\left[e^{-\tau_D}f(X_{\tau_D})\right]=u(0)=\int_{\mathbb{S}^{d-1}}f(y)\,\partial_{n_D}G_D(y)\,\mathrm{d}S(y).

Since C∞(Sd−1)C^\infty(\mathbb{S}^{d-1}) is dense in C(Sd−1)C(\mathbb{S}^{d-1}), this identifies νD=wD dS\nu_D=w_D\,\mathrm{d}S, where wD:=∂nDGDw_D:=\partial_{n_D}G_D. Elliptic regularity away from the pole gives GD∈C∞(Bd‾∖{0})G_D\in C^\infty(\overline{\mathbb{B}^d}\setminus\{0\}) (Gilbarg and Trudinger, [18]). In particular, ∂nDGD∈C∞(Sd−1)\partial_{n_D}G_D\in C^\infty(\mathbb{S}^{d-1}). Moreover, the maximum principle gives GD<0G_D<0 in Bd∖{0}\mathbb{B}^d\setminus\{0\}, while GD=0G_D=0 on Sd−1\mathbb{S}^{d-1}. Hopf’s boundary lemma, applied near each boundary point, therefore gives

∂nGD(y)>0,y∈Sd−1;\partial_nG_D(y)>0,\qquad y\in\mathbb{S}^{d-1};

see Gilbarg and Trudinger [18]. Since GD=0G_D=0 on Sd−1\mathbb{S}^{d-1}, i.e., constant, its tangential derivatives vanish there. Decomposing the ordinary gradient into its tangential and normal components therefore gives

∇GD(y)=∇Sd−1GD(y)+∂nGD(y)y=∂nGD(y)y,y∈Sd−1.\nabla G_D(y)=\nabla_{\mathbb{S}^{d-1}}G_D(y)+\partial_nG_D(y)y=\partial_nG_D(y)y,\qquad y\in\mathbb{S}^{d-1}.

Consequently,

wD(y)=y⊤D−2∇GD(y)=(y⊤D−2y)∂nGD(y)>0,y∈Sd−1.w_D(y)=y^\top D^{-2}\nabla G_D(y)=\left(y^\top D^{-2}y\right)\partial_nG_D(y)>0,\qquad y\in\mathbb{S}^{d-1}.

Write ad−1:=∫Sd−1dSa_{d-1}:=\int_{\mathbb{S}^{d-1}}dS, so that dS=ad−1 dσd−1dS=a_{d-1}\,d\sigma_{d-1}. Since MD=νD(Sd−1)=∫Sd−1wD dSM_D=\nu_D(\mathbb{S}^{d-1})=\int_{\mathbb{S}^{d-1}}w_D\,dS, Theorem 6 gives

μD(A)=1MD∫AwD(y) dS(y)=ad−1MD∫AwD(y) dσd−1(y),A⊆Sd−1 Borel.\mu_D(A)=\frac{1}{M_D}\int_Aw_D(y)\,dS(y)=\frac{a_{d-1}}{M_D}\int_Aw_D(y)\,d\sigma_{d-1}(y),\qquad A\subseteq\mathbb{S}^{d-1}\ \text{Borel}.

Thus μD\mu_D has the strictly positive smooth density ad−1wD/MDa_{d-1}w_D/M_D with respect to σd−1\sigma_{d-1}, and supp⁡μD=Sd−1\operatorname{supp}\mu_D=\mathbb{S}^{d-1}.

Remark 3. The function wD=∂nGDw_D=\partial_nG_D is the Poisson kernel at the origin for the operator LD=I−AD\mathcal{L}_D=I-\mathcal{A}_D in Bd\mathbb{B}^d, with respect to surface measure dSdS. Thus the density of μD\mu_D is obtained by normalizing this kernel. For a general discussion of Poisson kernels and their relation to Dirichlet problems, we refer to Véron [44].

Resolution of the conjecture. The following immediate consequence of the result above resolves the conjecture of Burger et al. [9].

Corollary 8. The density of μD\mu_D is bounded away from zero on Sd−1\mathbb{S}^{d-1}:

wD(y)ZD≥cD>0,y∈Sd−1.\frac{w_D(y)}{Z_D}\ge c_D>0,\qquad y\in\mathbb{S}^{d-1}.

In particular, supp⁡μD=Sd−1\operatorname{supp}\mu_D=\mathbb{S}^{d-1}.

Proof. The continuous, strictly positive function wD/ZDw_D/Z_D attains a positive minimum on the compact set Sd−1\mathbb{S}^{d-1}.

We now formulate the following conjecture concerning the minimizers of wDw_D.

Conjecture. The minimum of wDw_D on Sd−1\mathbb{S}^{d-1} is attained at every unit vector belonging to the eigenspace associated with the largest eigenvalue of DD.

The probabilistic analysis in § 3.1 provides intuition for this conjecture. Since the law of the diffusion is invariant under orthogonal transformations within each eigenspace of DD, the density wDw_D takes the same value at all unit vectors in that eigenspace. However, despite the apparent simplicity of the statement, we have not been able to establish it rigorously.

Energy landscape

A standard approach to the minimization problem (2) is to equip P(Sd−1)\mathcal{P}(\mathbb{S}^{d-1}) with a suitable metric and consider the corresponding gradient flow of ED\mathcal{E}_D. Different choices of metric lead to different descent dynamics. For the Wasserstein distance W2W_2, the evolution satisfies

∂tμt=div⁡Sd−1(μt∇Sd−1Uμt);(20)\partial_t\mu_t=\operatorname{div}_{\mathbb{S}^{d-1}}\left(\mu_t\nabla_{\mathbb{S}^{d-1}}U_{\mu_t}\right); \tag*{(20)}

see, for instance, Villani [45]; Santambrogio (2017). A closely related evolution is the normalized repulsive self-attention equation studied by Geshkovski et al. [17] and Burger et al. [9],

∂tμt=div⁡Sd−1(μtUμt∇Sd−1Uμt)=div⁡Sd−1(μt∇Sd−1log⁡Uμt).(21)\partial_t\mu_t=\operatorname{div}_{\mathbb{S}^{d-1}}\left(\frac{\mu_t}{U_{\mu_t}}\nabla_{\mathbb{S}^{d-1}}U_{\mu_t}\right)=\operatorname{div}_{\mathbb{S}^{d-1}}\left(\mu_t\nabla_{\mathbb{S}^{d-1}}\log U_{\mu_t}\right). \tag*{(21)}

This evolution is a gradient flow of the same energy in a modified transport geometry with nonlocal mobility Uμ−1U_{\mu}^{-1} (induced by a modified Wasserstein distance). Both equations are understood in the distributional sense.

The existence of a unique global minimizer does not by itself ensure convergence of either flow to it. We therefore study stationary measures of ED\mathcal{E}_D and show that every stationary measure except for μD\mu_D is singular with respect to the surface measure. We also remark that these measures can’t be local minimizers with respect to the above Wasserstein-type distances. We then study the stability of μD\mu_D under perturbations of the interaction matrix.

Stationary measures with respect to Wasserstein flows

We say that a probability measure μ∈P(Sd−1)\mu\in\mathcal{P}(\mathbb{S}^{d-1}) is stationary if

∇Sd−1Uμ(x)=0for μ-a.e. x.(22)\nabla_{\mathbb{S}^{d-1}}U_\mu(x)=0\qquad\text{for }\mu\text{-a.e. }x. \tag*{(22)}

Indeed, testing the respective stationary equations against UμU_\mu gives

0=∫Sd−1∥∇Sd−1Uμ(x)∥2μ(dx),0=\int_{\mathbb{S}^{d-1}}\left\|\nabla_{\mathbb{S}^{d-1}}U_\mu(x)\right\|^2\mu(\mathrm{d}x),

and,

0=∫Sd−1⟨∇Sd−1Uμ(x),∇Sd−1log⁡Uμ(x)⟩μ(dx)=∫Sd−1∥∇Sd−1Uμ(x)∥2Uμ(x)μ(dx).0=\int_{\mathbb{S}^{d-1}}\left\langle\nabla_{\mathbb{S}^{d-1}}U_\mu(x),\nabla_{\mathbb{S}^{d-1}}\log U_\mu(x)\right\rangle\mu(\mathrm{d}x)=\int_{\mathbb{S}^{d-1}}\frac{\left\|\nabla_{\mathbb{S}^{d-1}}U_\mu(x)\right\|^2}{U_\mu(x)}\mu(\mathrm{d}x).

Since UμU_\mu is strictly positive, either identity implies (22). Conversely, (22) implies that both ∇Sd−1Uμ\nabla_{\mathbb{S}^{d-1}}U_\mu and ∇Sd−1log⁡Uμ\nabla_{\mathbb{S}^{d-1}}\log U_\mu vanish μ\mu-almost everywhere, so μ\mu satisfies both stationary equations (20) and (21). Hence the constant curve μt=μ\mu_t=\mu solves either evolution. Smoothness of KDK_D on the compact sphere and the bound Uμ≥e−∥D∥opU_\mu\geq e^{-\lVert D\rVert_{\mathrm{op}}} make both velocity fields Lipschitz in space and in the measure argument for W2W_2. The corresponding evoluations are therefore unique, so a solution initialized at μ\mu remains there. This characterization also agrees with stationarity under transport variations; see Burger et al. [9].

The potential UμDU_{\mu_D} is constant on Sd−1\mathbb{S}^{d-1}, so μD\mu_D is stationary. Other stationary measures exist. For example, if v∈Sd−1v\in\mathbb{S}^{d-1} is an eigenvector of DD, then

∇Sd−1Uδv(v)=e⊤Dv(I−vv⊤)Dv=0,\nabla_{\mathbb{S}^{d-1}}U_{\delta_v}(v)=e^{\top}Dv(I-vv^{\top})Dv=0,

so δv\delta_v is stationary. We now show that every stationary measure distinct from μD\mu_D has support of zero surface measure. Thus μD\mu_D is the only stationary probability measure having a density with respect to dσd−1\mathrm{d}\sigma_{d-1}.

Proposition 9 (Singularity of non-minimizing stationary measures). Let μ∈P(Sd−1)\mu\in\mathcal{P}(\mathbb{S}^{d-1}) satisfy (22). If μ≠μD\mu\ne\mu_D, then supp⁡μ\operatorname{supp}\mu is contained in the zero set of a nonzero real-analytic function on Sd−1\mathbb{S}^{d-1}. In particular,

σd−1(supp⁡μ)=0,μ⊥dσd−1.\sigma_{d-1}(\operatorname{supp}\mu)=0,\qquad\mu\perp\mathrm{d}\sigma_{d-1}.

Consequently, μD\mu_D is the only stationary probability measure absolutely continuous with respect to dσd−1\mathrm{d}\sigma_{d-1}. For d=2d=2, every stationary measure distinct from μD\mu_D has finite support.

Proof. We first verify that UμU_\mu extends to a real-analytic function on Rd\mathbb{R}^d. For x0,h∈Rdx_0,h\in\mathbb{R}^d, expansion of the exponential gives

Uμ(x0+h)=∑m=0∞1m!∫Sd−1ex0TDy(hTDy)m μ(dy).U_\mu(x_0+h)=\sum_{m=0}^{\infty}\frac{1}{m!}\int_{\mathbb{S}^{d-1}}e^{x_0^{\mathsf T}D_y}(h^{\mathsf T}D_y)^m\,\mu(\mathrm{d}y).

The exchange of the sum and the integral is justified by

ex0TDy∣hTDy∣mm!≤e∥x0∥∥D∥op(∥h∥∥D∥op)mm!,y∈Sd−1.\frac{e^{x_0^{\mathsf T}D_y}|h^{\mathsf T}D_y|^m}{m!}\leq e^{\lVert x_0\rVert\lVert D\rVert_{\mathrm{op}}}\frac{(\lVert h\rVert\lVert D\rVert_{\mathrm{op}})^m}{m!},\qquad y\in\mathbb{S}^{d-1}.

Thus the power series converges absolutely and uniformly for hh in bounded sets, proving analyticity.

On the sphere,

∇Sd−1Uμ(x)=(I−xxT)∇Uμ(x),x∈Sd−1.\nabla_{\mathbb{S}^{d-1}}U_\mu(x)=(I-xx^{\mathsf T})\nabla U_\mu(x),\qquad x\in\mathbb{S}^{d-1}.

It follows that g:Sd−1→Rg:\mathbb{S}^{d-1}\to\mathbb{R} given by

g(x):=∥∇Sd−1Uμ(x)∥2,x∈Sd−1,g(x):=\lVert\nabla_{\mathbb{S}^{d-1}}U_\mu(x)\rVert^2,\qquad x\in\mathbb{S}^{d-1},

is real-analytic. By (22), g=0g=0 μ\mu-almost everywhere. Continuity of gg then gives

supp⁡μ⊂{x∈Sd−1:g(x)=0}=:Z(g).\operatorname{supp}\mu\subset\{x\in\mathbb{S}^{d-1}:g(x)=0\}=:\mathcal{Z}(g).

If g=0g=0, connectedness of Sd−1\mathbb{S}^{d-1} implies that UμU_\mu is constant on Sd−1\mathbb{S}^{d-1}. Proposition 2 then gives μ=μD\mu=\mu_D.

Suppose now that μ≠μD\mu\ne\mu_D. Then g≢0g\not\equiv0. Since Sd−1\mathbb{S}^{d-1} is a connected real-analytic manifold, Langrenez et al. [23] gives

σd−1(Z(g))=0.\sigma_{d-1}(\mathcal{Z}(g))=0.

Hence σd−1(supp⁡μ)=0\sigma_{d-1}(\operatorname{supp}\mu)=0, which proves singularity. Since μD\mu_D has a smooth, strictly positive density by Theorem 7, it is the only absolutely continuous stationary probability measure.

For d=2d=2, the zeros of a nonzero real-analytic function on the circle are isolated by the one-dimensional identity theorem; see also Álvarez-López et al. [1]. Compactness of S1\mathbb{S}^{1} therefore makes the zero set finite, and the same holds for supp⁡μ\operatorname{supp}\mu.

Remark 4. The fact that absolute continuity is a distinguishing property of the minimizer among all stationary points has an important consequence. If an evolutionary process (e.g., a gradient flow) converges weakly to an absolutely continuous stationary measure, then its limit must be μD\mu_D. In particular, a uniform bound on the relative entropy with respect to σd−1\sigma_{d-1} ensures absolute continuity of every weak limit, by lower semicontinuity of relative entropy. Note, however, that preservation of absolute continuity at each finite time alone does not suffice to exclude a singular limit.

Remark 5 (Absence of local minimizers). Although the associated Wasserstein-type gradient flows admit non-minimizing stationary measures, there are no local minimizers except for μD\mu_D. Indeed, take any metric dd that metrizes weak convergence of measures (e.g., one of the Wasserstein-type distances considered in the previous section) and define an open ball

Vr(μ)={ν∈P(Sd−1):d(μ,ν)<r},μ∈P(Sd−1), r>0.\mathcal{V}_r(\mu)=\{\nu\in\mathcal{P}(\mathbb{S}^{d-1}):d(\mu,\nu)<r\},\qquad\mu\in\mathcal{P}(\mathbb{S}^{d-1}),\ r>0.

Assume ν∈P(Sd−1)\nu\in\mathcal{P}(\mathbb{S}^{d-1}), ν≠μD\nu\ne\mu_D, is a local minimizer of ED\mathcal{E}_D with respect to dd, that is, there exists an r>0r>0 such that

ED(ν)≤ED(μ)for every μ∈Vr(ν).\mathcal{E}_D(\nu)\leq\mathcal{E}_D(\mu)\qquad\text{for every }\mu\in\mathcal{V}_r(\nu).

Consider the segment

νt=(1−t)ν+tμD,t∈[0,1].\nu_t=(1-t)\nu+t\mu_D,\qquad t\in[0,1].

The map t↦νtt \mapsto\nu_t is continuous with respect to weak convergence of measures, and ν0=ν\nu_0=\nu. Thus, an initial segment is contained in Vr(ν)\mathcal{V}_r(\nu): there exists t0∈(0,1]t_0 \in(0,1] such that νt∈Vr(ν)\nu_t \in\mathcal{V}_r(\nu) for every t∈[0,t0)t \in[0,t_0). By local minimality and strict convexity (3), we have

ED(ν)≤ED(νt)<(1−t)ED(ν)+tED(μD),0<t<t0.\mathcal{E}_D(\nu) \leq\mathcal{E}_D(\nu_t) < (1-t)\mathcal{E}_D(\nu) + t\mathcal{E}_D(\mu_D), \qquad0 < t < t_0.

Subtracting (1−t)ED(ν)(1-t)\mathcal{E}_D(\nu) and dividing by tt gives ED(ν)<ED(μD)\mathcal{E}_D(\nu) < \mathcal{E}_D(\mu_D), which contradicts the global minimality of μD\mu_D, hence ν=μD\nu=\mu_D.

Stability with respect to the interaction matrix

We now study the dependence of μD\mu_D on the interaction matrix DD. The lower bound in (10) allows us to control the distance between two minimizers by comparing their energies udner the two interaction kernels.

Proposition 10 (Stability with respect to the interaction matrix). Let E1,E2≻0E_1,E_2 \succ0 be symmetric, and suppose that

max⁡(∥E1∥op,∥E2∥op)≤R.\max\left(\lVert E_1\rVert_{\mathrm{op}},\lVert E_2\rVert_{\mathrm{op}}\right) \leq R.

Then

MMD⁡E1(μE1,μE2)2≤2eR∥E1−E2∥op.(23)\operatorname{MMD}_{E_1}(\mu_{E_1},\mu_{E_2})^2 \leq2e^R\lVert E_1-E_2\rVert_{\mathrm{op}}. \tag*{(23)}

In particular, if D≻0D \succ0 and (Di)i≥1(D_i)_{i\geq1} is a sequence of symmetric positive definite matrices such that ∥Di−D∥op→0\lVert D_i-D\rVert_{\mathrm{op}}\to0, then μDi⇀μD\mu_{D_i}\rightharpoonup\mu_D.

Proof. For every x,y∈Sd−1x,y \in\mathbb{S}^{d-1}, the mean value theorem gives

∣KE1(x,y)−KE2(x,y)∣≤eR∣xT(E1−E2)y∣≤eR∥E1−E2∥op.\lvert K_{E_1}(x,y)-K_{E_2}(x,y)\rvert\leq e^R\lvert x^{\mathsf{T}}(E_1-E_2)y\rvert\leq e^R\lVert E_1-E_2\rVert_{\mathrm{op}}.

Integrating against any ρ∈P(Sd−1)\rho\in\mathcal{P}(\mathbb{S}^{d-1}), we obtain

∣EE1(ρ)−EE2(ρ)∣≤12eR∥E1−E2∥op.\lvert\mathcal{E}_{E_1}(\rho)-\mathcal{E}_{E_2}(\rho)\rvert\leq\frac{1}{2}e^R\lVert E_1-E_2\rVert_{\mathrm{op}}.

By minimality of μE2\mu_{E_2} for EE2\mathcal{E}_{E_2},

EE1(μE2)−EE1(μE1)=(EE1(μE2)−EE2(μE2))+(EE2(μE2)−EE2(μE1))+(EE2(μE1)−EE1(μE1))≤eR∥E1−E2∥op.\begin{aligned} \mathcal{E}_{E_1}(\mu_{E_2})-\mathcal{E}_{E_1}(\mu_{E_1}) &= \bigl(\mathcal{E}_{E_1}(\mu_{E_2})-\mathcal{E}_{E_2}(\mu_{E_2})\bigr) + \bigl(\mathcal{E}_{E_2}(\mu_{E_2})-\mathcal{E}_{E_2}(\mu_{E_1})\bigr) \\ &\quad+ \bigl(\mathcal{E}_{E_2}(\mu_{E_1})-\mathcal{E}_{E_1}(\mu_{E_1})\bigr) \\ &\leq e^R\lVert E_1-E_2\rVert_{\mathrm{op}}. \end{aligned}

Applying the lower bound in (10) proves (23).

Now suppose that ∥Di−D∥op→0\lVert D_i-D\rVert_{\mathrm{op}}\to0. The sequence (Di)i≥1(D_i)_{i\geq1} is bounded in operator norm, so we may choose RR such that ∥D∥op≤R\lVert D\rVert_{\mathrm{op}}\leq R and ∥Di∥op≤R\lVert D_i\rVert_{\mathrm{op}}\leq R for every ii. Therefore,

MMD⁡D(μDi,μD)2≤2eR∥Di−D∥op⟶0.\operatorname{MMD}_D(\mu_{D_i},\mu_D)^2 \leq2e^R\lVert D_i-D\rVert_{\mathrm{op}} \longrightarrow0.

Proposition 3 gives the claimed weak convergence. □\square

Numerical methods

In this section we use the derived representations of the minimizer μD\mu_D to construct a sampling scheme in arbitrary dimension and, in dimension two, to approximate the unnormalized density wDw_D by an expansion in Mathieu functions. The code for reproducing the numerical experiments is available at github.com/TimRoith/EquilibriumTransformers.

Sampling from the minimizer

Theorem 6 gives a construction which allows to sample from the minimizer without evaluating the density. As explained in § 3.1, one may run Bt=2WtB_t=\sqrt{2}W_t in ΩD=DBd\Omega_D= D\mathbb{B}^d, starting at the origin, and kill it at an independent exponential time of rate one. Mapping a surviving exit back by D−1D^{-1} gives a sample with law μD\mu_D. We approximate these exits by a Walk on Spheres (WoS) [29, 15, 34].

Survival during one step. Suppose that the process starts at the center of a ball of radius rr contained in ΩD\Omega_D. Rotational invariance makes the exit direction uniform and independent of the exit time. For the generator Δ\Delta and killing rate one, the probability of reaching the boundary of this ball before killing is

pd(r):=rν2νΓ(ν+1)Iν(r),ν:=d2−1,pd(0):=1.(24)p_d(r) := \frac{r^\nu}{2^\nu\Gamma(\nu+1)I_\nu(r)}, \qquad\nu:= \frac{d}{2}-1, \qquad p_d(0):=1. \tag*{(24)}

Here IνI_\nu denotes the modified Bessel function of the first kind of order ν\nu. By Rasila and Sottinen [30], pd(r)p_d(r) is the total mass of the centered-ball panharmonic measure. The strong Markov property of the exponentially killed process allows these steps to be repeated.

Remark 6 (Numerical evaluation). For numerical evaluation, it is convenient to use the exponentially scaled modified Bessel function

Iνsc(r):=e−rIν(r).I_\nu^{\mathrm{sc}}(r):=e^{-r}I_\nu(r).

Multiplying the numerator and denominator in (24) by e−re^{-r} gives, for r>0r>0,

pd(r)=rνe−r2νΓ(ν+1)Iνsc(r).p_d(r)=\frac{r^\nu e^{-r}}{2^\nu\Gamma(\nu+1)I_\nu^{\mathrm{sc}}(r)}.

Evaluating IνscI_\nu^{\mathrm{sc}} directly avoids overflow due to the exponential growth of IνI_\nu for large arguments. The value pd(0)=1p_d(0)=1 is defined by continuity.

For z∈ΩDz\in\Omega_D, the radius

r(z):=λmin⁡(D)(1−∥D−1z∥)(25)r(z):=\lambda_{\min}(D)\left(1-\lVert D^{-1}z\rVert\right) \tag*{(25)}

gives an admissible ball. Indeed, for every b∈∂ΩDb\in\partial\Omega_D,

1−∥D−1z∥≤∥D−1(b−z)∥≤λmin⁡−1(D)∥b−z∥.1-\lVert D^{-1}z\rVert\leq\lVert D^{-1}(b-z)\rVert\leq\lambda_{\min}^{-1}(D)\lVert b-z\rVert.

Thus r(z)≤dist⁡(z,∂ΩD)r(z)\leq\operatorname{dist}(z,\partial\Omega_D). We use a dimensionless tolerance 0<ε<10<\varepsilon<1 on 1−∥D−1z∥1-\lVert D^{-1}z\rVert, and project to the sphere when this quantity is at most ε\varepsilon.

Algorithm 1 One walk-on-spheres trial for an approximate sample from μD\mu_D.

Require: Symmetric D≻0D\succ0, tolerance 0<ε<10<\varepsilon<1

  1. z←0z\leftarrow0

  2. while true do

  3. y←D−1z,δ←1−∥y∥\quad y\leftarrow D^{-1}z,\quad\delta\leftarrow1-\lVert y\rVert

  4. \quadif δ≤ε\delta\leq\varepsilon then

  5. \qquadreturn y/∥y∥y/\lVert y\rVert

  6. r←λmin⁡(D)δ\quad r\leftarrow\lambda_{\min}(D)\delta

  7. \quadDraw UU uniformly on [0,1][0,1]

  8. \quadif U>pd(r)U>p_d(r) then

  9. \qquadreturn Killed

  10. \quadDraw V∼σd−1V\sim\sigma_{d-1}, independently

  11. z←z+rV\quad z\leftarrow z+rV

Convergence as the tolerance vanishes. We next show that the law of an accepted output converges to μD\mu_D as the stopping tolerance tends to zero.

Proposition 11 (Convergence of the sampling law). Let D≻0D \succ0 be symmetric. For 0<ε<10 < \varepsilon< 1, denote by μD,ε∈P(Sd−1)\mu_{D,\varepsilon} \in\mathcal{P}(\mathbb{S}^{d-1}) the law of the output of Algorithm 1, conditional on the trial being successful. Then

μD,ε⇀μDas ε↓0.\mu_{D,\varepsilon} \rightharpoonup\mu_D \qquad\text{as } \varepsilon\downarrow0.

Remark 7. Convergence of the classical walk on spheres is recalled in Kyprianou et al. [22]. Related results for walks with killing and for the error due to the boundary stopping tolerance are given by Bossy et al. [6] and Mascagni and Hwang [26], respectively. These results do not state convergence of the conditional sampling law for the anisotropic domain (i.e., ellipsoids) and stopping rule used here. We therefore give a full proof.

Proof. Throughout the proof, convergence of random variables is understood to hold almost surely.

Let TjT_j denote the time along the Brownian path corresponding to the jj-th jump of the walk without killing. Set T0=0T_0 = 0 and, whenever Tj<τDT_j < \tau_D, define

Tj+1:=inf⁡{t≥Tj:∥D(Xt−XTj)∥≥r(XTj)}.(26)T_{j+1} := \inf\{t \ge T_j : \lVert D(X_t - X_{T_j}) \rVert\ge r(X_{T_j})\}. \tag*{(26)}

If Tj=τDT_j = \tau_D, set Tj+1=τDT_{j+1} = \tau_D. By the strong Markov property and the local exit law, the positions XTjX_{T_j} follow the jump rule of Algorithm 1, with killing omitted. The choice of rr ensures that Tj+1≤τDT_{j+1} \le\tau_D, so Tj↑T∞≤τD<∞T_j \uparrow T_\infty\le\tau_D < \infty almost surely. Moreover, continuity of XX gives

XTj+1−XTj⟶0.X_{T_{j+1}} - X_{T_j} \longrightarrow0.

By the definition of the successive exit times,

λmin⁡(D)(1−∥XTj∥)=∥D(XTj+1−XTj)∥⟶0.\lambda_{\min}(D)(1-\lVert X_{T_j}\rVert)=\lVert D(X_{T_{j+1}}-X_{T_j})\rVert\longrightarrow0.

Thus XT∞∈Sd−1X_{T_\infty} \in\mathbb{S}^{d-1}, and T∞=τDT_\infty= \tau_D.

Define NεN_\varepsilon the number of jumps needed to reach the ε\varepsilon-neighbourhood of the boundary, along a given trajectory of the walk without killing, i.e.,

Nε:=min⁡{j≥0:1−∥XTj∥≤ε},Tε:=TNε,Yε:=XTε∥XTε∥.N_\varepsilon:= \min\{j \ge0 : 1-\lVert X_{T_j}\rVert\le\varepsilon\}, \qquad T_\varepsilon:= T_{N_\varepsilon}, \qquad Y_\varepsilon:= \frac{X_{T_\varepsilon}}{\lVert X_{T_\varepsilon}\rVert}.

Then Nε<∞N_\varepsilon< \infty, and

∥Yε−XτD∥≤ε+∥XTε−XτD∥⟶0.\lVert Y_\varepsilon-X_{\tau_D}\rVert\le\varepsilon+\lVert X_{T_\varepsilon}-X_{\tau_D}\rVert\longrightarrow0.

So that

Tε↑τD,Yε⟶XτDas ε↓0.T_\varepsilon\uparrow\tau_D, \qquad Y_\varepsilon\longrightarrow X_{\tau_D} \qquad\text{as } \varepsilon\downarrow0.

At each step, the exit location is independent of the exit time, and pd(r)p_d(r) is the probability of surviving until that exit. The strong Markov property of the killed process therefore implies that the accepted output has the law of YεY_\varepsilon conditional on Tε<ζT_\varepsilon< \zeta. Its acceptance probability satisfies

P(Tε<ζ)=E[e−Tε]≥E[e−τD]>0.\mathbb{P}(T_\varepsilon< \zeta)=\mathbb{E}[e^{-T_\varepsilon}] \ge\mathbb{E}[e^{-\tau_D}] > 0.

Consequently, for every h∈C(Sd−1)h \in C(\mathbb{S}^{d-1}), dominated convergence and Theorem 6 yield

∫Sd−1h dμD,ε=E[e−Tεh(Yε)]E[e−Tε]⟶E[e−τDh(XτD)]E[e−τD]=∫Sd−1h dμD,\int_{\mathbb{S}^{d-1}} h\,\mathrm{d}\mu_{D,\varepsilon} = \frac{\mathbb{E}[e^{-T_\varepsilon}h(Y_\varepsilon)]}{\mathbb{E}[e^{-T_\varepsilon}]} \longrightarrow \frac{\mathbb{E}[e^{-\tau_D}h(X_{\tau_D})]}{\mathbb{E}[e^{-\tau_D}]} = \int_{\mathbb{S}^{d-1}} h\,\mathrm{d}\mu_D,

recovering the claim.

Note that the acceptance probability converges to MD=P(τD<ζ)M_D=\mathbb{P}(\tau_D<\zeta). Combining (16) with (5), we obtain the following relation between the normalization, the sampling cost, and the minimum:

E[number of exact trials per accepted sample]=1MD=2ED(μD).\mathbb{E}[\text{number of exact trials per accepted sample}] = \frac{1}{M_D} = 2\mathcal{E}_D(\mu_D).

Sampling error and the energy gap. Independent repetitions, retaining only successful trials, yield a sequence of independent random variables (Xi)i≥1(X_i)_{i\geq1} with common law μD,ε\mu_{D,\varepsilon}. By the strong law of large numbers for empirical measures [43], for each fixed ε\varepsilon,

μ^D,ε,N:=1N∑i=1NδXi⇀μD,εalmost surely as N→∞.(27)\widehat{\mu}_{D,\varepsilon,N} := \frac{1}{N}\sum_{i=1}^{N}\delta_{X_i} \rightharpoonup\mu_{D,\varepsilon} \qquad\text{almost surely as } N\to\infty. \tag*{(27)}

We now compare the empirical energy of the accepted outputs with the minimum energy.

Proposition 12 (Sampling error at fixed tolerance). Fix 0<ε<10<\varepsilon<1 and N≥1N\geq1. Then,

E[MMD⁡D(μ^D,ε,N,μD)2]≤MMD⁡D(μD,ε,μD)2+e∥D∥op−1N,(28)\mathbb{E}\left[\operatorname{MMD}_D(\widehat{\mu}_{D,\varepsilon,N},\mu_D)^2\right] \leq\operatorname{MMD}_D(\mu_{D,\varepsilon},\mu_D)^2+\frac{e^{\lVert D\rVert_{\mathrm{op}}}-1}{N}, \tag*{(28)}

where μ^D,ε,N\widehat{\mu}_{D,\varepsilon,N} is defined in (27).

Proof. Independence gives

E[ED(μ^D,ε,N)]=12N2∑i,j=1NE[KD(Xi,Xj)]=N(N−1)2N2∬Sd−1×Sd−1KD(x,y) μD,ε(dx)μD,ε(dy)+12N∫Sd−1KD(x,x) μD,ε(dx)=(1−1N)ED(μD,ε)+12N∫Sd−1KD(x,x) μD,ε(dx).\begin{aligned} \mathbb{E}[\mathcal{E}_D(\widehat{\mu}_{D,\varepsilon,N})] = \frac{1}{2N^2}\sum_{i,j=1}^{N}\mathbb{E}[K_D(X_i,X_j)] \\ &= \frac{N(N-1)}{2N^2}\iint_{\mathbb{S}^{d-1}\times\mathbb{S}^{d-1}} K_D(x,y)\,\mu_{D,\varepsilon}(dx)\mu_{D,\varepsilon}(dy) +\frac{1}{2N}\int_{\mathbb{S}^{d-1}}K_D(x,x)\,\mu_{D,\varepsilon}(dx) \\ &= \left(1-\frac{1}{N}\right)\mathcal{E}_D(\mu_{D,\varepsilon}) +\frac{1}{2N}\int_{\mathbb{S}^{d-1}}K_D(x,x)\,\mu_{D,\varepsilon}(dx). \end{aligned}

Consequently, Proposition 4 yields

E[MMD⁡D(μ^D,ε,N,μD)2]=2(E[ED(μ^D,ε,N)]−ED(μD))=MMD⁡D(μD,ε,μD)2+1N(∫Sd−1KD(x,x) μD,ε(dx)−2ED(μD,ε))≤MMD⁡D(μD,ε,μD)2+e∥D∥op−1N,\begin{aligned} \mathbb{E}\left[\operatorname{MMD}_D(\widehat{\mu}_{D,\varepsilon,N},\mu_D)^2\right] =2\left(\mathbb{E}[\mathcal{E}_D(\widehat{\mu}_{D,\varepsilon,N})]-\mathcal{E}_D(\mu_D)\right) \\ &=\operatorname{MMD}_D(\mu_{D,\varepsilon},\mu_D)^2 +\frac{1}{N}\left(\int_{\mathbb{S}^{d-1}}K_D(x,x)\,\mu_{D,\varepsilon}(dx)-2\mathcal{E}_D(\mu_{D,\varepsilon})\right) \\ &\leq\operatorname{MMD}_D(\mu_{D,\varepsilon},\mu_D)^2+\frac{e^{\lVert D\rVert_{\mathrm{op}}}-1}{N}, \end{aligned}

where the last step uses KD(x,x)=exTDx≤e∥D∥opK_D(x,x)=e^{x^{\mathsf{T}}Dx}\leq e^{\lVert D\rVert_{\mathrm{op}}} on Sd−1\mathbb{S}^{d-1} and 2ED(μD,ε)≥12\mathcal{E}_D(\mu_{D,\varepsilon})\geq1.

The first term in (28) accounts for the stopping tolerance, and the second for the finite number of accepted samples. Since μD,ε⇀μD\mu_{D,\varepsilon}\rightharpoonup\mu_D as ε↓0\varepsilon\downarrow0, the first term tends to zero. Taking N→∞N\to\infty and ε↓0\varepsilon\downarrow0 therefore gives convergence in mean squared MMD⁡D\operatorname{MMD}_D, and hence convergence of the expected energy to its minimum. Increasing NN at fixed ε\varepsilon does not remove the stopping-tolerance error. An explicit rate in ε\varepsilon would require a quantitative estimate of that term.

For a numerical comparison, we apply the projected gradient descent scheme from Geshkovski et al. [17], which optimizes the configuration of an ensemble x=(x1,…,xN)\mathbf{x}=(x_1,\ldots,x_N). Set ED(x):=ED(1N∑i=1Nδxi)\mathcal{E}_D(\mathbf{x}):=\mathcal{E}_D\left(\frac{1}{N}\sum_{i=1}^{N}\delta_{x_i}\right). Starting from an initial ensemble x(0)\mathbf{x}^{(0)}, we iterate

x(k+1)=Π(x(k)−γ∇xED(x(k))),(29)\mathbf{x}^{(k+1)}=\Pi\left(\mathbf{x}^{(k)}-\gamma\nabla_{\mathbf{x}}\mathcal{E}_D(\mathbf{x}^{(k)})\right), \tag*{(29)}

where Π(z)=z/∥z∥\Pi(z)=z/\lVert z\rVert for z≠0z\ne0 is applied to each particle and γ>0\gamma>0 is a fixed step size. We compare the resulting energy with that of the empirical measure formed from NN independent accepted outputs of Algorithm 1. Figure 1 reports the two energies and their relative difference for increasing values of NN. Over the range shown, the relative difference decreases approximately as N−1N^{-1}.

Figure 1. Comparison of Algorithm 1 with projected gradient descent (29), for a fixed randomly generated matrix DD in dimension d=20d=20. Left: particle energies at equal ensemble sizes NN. Right: the sampling energy minus the gradient descent energy, divided by the latter. The dashed line is proportional to N−1N^{-1}.

Density approximation using Mathieu functions

In this section we use the Dirichlet representation to expand the density wDw_D of the unnormalized measure νD\nu_D (13) in an orthogonal basis. In dimension two, separation of variables in elliptic coordinates gives an explicit expansion in Mathieu functions.

Throughout this section, we assume that DD is given in diagonal form. Since μD\mu_D is invariant under coordinate reflections by Burger et al. [9], its unnormalized density satisfies wD∈Lsym2(Sd−1)w_D\in L^2_{\mathrm{sym}}(\mathbb{S}^{d-1}), where

Lsym2(Sd−1):=⋂r∈{−1,1}d{f∈L2(Sd−1):f=f∘diag⁡(r)}.L^2_{\mathrm{sym}}(\mathbb{S}^{d-1}) := \bigcap_{r\in\{-1,1\}^d}\left\{f\in L^2(\mathbb{S}^{d-1}): f=f\circ\operatorname{diag}(r)\right\}.

is the space of L2L^2 functions on the sphere invariant under coordinate reflections. For every r∈{−1,1}dr\in\{-1,1\}^d, the matrix diag⁡(r)\operatorname{diag}(r) is orthogonal and therefore preserves surface measure on Sd−1\mathbb{S}^{d-1}. Consequently, f↦f∘diag⁡(r)f\mapsto f\circ\operatorname{diag}(r) defines a linear isometry of L2(Sd−1)L^2(\mathbb{S}^{d-1}). The fixed-point space of each such isometry is a closed linear subspace of L2(Sd−1)L^2(\mathbb{S}^{d-1}). Hence their intersection, Lsym2(Sd−1)L^2_{\mathrm{sym}}(\mathbb{S}^{d-1}), is also a closed linear subspace of L2(Sd−1)L^2(\mathbb{S}^{d-1}).

For an orthogonal basis {fn}n∈N0⊂C∞(Sd−1)\{f_n\}_{n\in\mathbb{N}_0}\subset C^\infty(\mathbb{S}^{d-1}) of Lsym2(Sd−1)L^2_{\mathrm{sym}}(\mathbb{S}^{d-1}) we write the orthogonal expansion of wDw_D and, for each integer K≥1K\ge1, its truncation as

wD=∑n∈N0⟨fn,wD⟩fn∥fn∥2,andwD,K:=∑n=0K⟨fn,wD⟩fn∥fn∥2.w_D=\sum_{n\in\mathbb{N}_0}\langle f_n,w_D\rangle\frac{f_n}{\lVert f_n\rVert^2},\qquad\text{and}\qquad w_{D,K}:=\sum_{n=0}^{K}\langle f_n,w_D\rangle\frac{f_n}{\lVert f_n\rVert^2}.

Since {fn}n∈N0\{f_n\}_{n\in\mathbb{N}_0} is an orthogonal basis of Lsym2(Sd−1)L^2_{\mathrm{sym}}(\mathbb{S}^{d-1}), we have

lim⁡K→∞∥wD,K−wD∥L2=0.\lim_{K\to\infty}\lVert w_{D,K}-w_D\rVert_{L^2}=0.

In particular, the associated measures wD,K dS⇀νDw_{D,K}\,\mathrm{d}S\rightharpoonup\nu_D as K→∞K\to\infty. For any $n\in\mathbb{N}_0, the Dirichlet problem (17) with boundary values fnf_n admits a unique solution un∈C2(Bd‾)u_n \in C^2(\overline{B^d}). It directly follows from (19) that ⟨fn,wD⟩L2=un(0)\langle f_n,w_D\rangle_{L^2}=u_n(0). This yields

wD=∑n∈N0un(0)fn∥fn∥2.(30)w_D=\sum_{n\in\mathbb{N}_0}u_n(0)\frac{f_n}{\lVert f_n\rVert^2}. \tag*{(30)}

To evaluate the coefficients, we need solutions with a given boundary data. In dimension d=2d=2, pairs of boundary values and solutions are available through Mathieu functions, see Meixner and Schäfke [28]; Whittaker and Watson [46]. These functions arise by separation of variables in the modified Helmholtz equation,

u~−Δ~u~=0.(31)\widetilde{u}-\widetilde{\Delta}\widetilde{u}=0. \tag*{(31)}

More precisely, let us assume that the interaction matrix is given in diagonal form D=diag⁡(λ1,λ2)D=\operatorname{diag}(\lambda_1,\lambda_2) with λ1>λ2>0\lambda_1>\lambda_2>0. For c=λ12−λ22c=\sqrt{\lambda_1^2-\lambda_2^2}, we consider the elliptical coordinate transform

Φ:R≥0×(−π,π]→R2,(ξ,η)↦c(cosh⁡(ξ)cos⁡(η),sinh⁡(ξ)sin⁡(η))T.\Phi:\mathbb{R}_{\geq0}\times(-\pi,\pi]\to\mathbb{R}^2,\qquad(\xi,\eta)\mapsto c\left(\cosh(\xi)\cos(\eta),\sinh(\xi)\sin(\eta)\right)^{\mathsf{T}}.

We note that for ξ0=arctanh⁡(λ2/λ1)\xi_0=\operatorname{arctanh}(\lambda_2/\lambda_1) and η∈(−π,π]\eta\in(-\pi,\pi] it holds that Φ(ξ0,η)=D(cos⁡(η),sin⁡(η))T∈DS1\Phi(\xi_0,\eta)=D(\cos(\eta),\sin(\eta))^{\mathsf{T}}\in D\mathbb{S}^1. We consider solutions of (31) of the form

u~n(Φ(ξ,η))=Cen(ξ) cen(η),\widetilde{u}_n(\Phi(\xi,\eta))=\mathrm{Ce}_n(\xi)\,\mathrm{ce}_n(\eta),

where cen\mathrm{ce}_n and Cen\mathrm{Ce}_n denote the angular and modified Mathieu functions, respectively. They satisfy Mathieu’s equation and its modified form (Wolf, [48], Sections 28.2, 28.20):

cen′′(η)+(an−2qcos⁡(2η))⋅cen(η)=0,Cen′′(ξ)−(an−2qcosh⁡(2ξ))⋅Cen(ξ)=0.\mathrm{ce}_n^{\prime\prime}(\eta)+(a_n-2q\cos(2\eta))\cdot\mathrm{ce}_n(\eta)=0,\qquad\mathrm{Ce}_n^{\prime\prime}(\xi)-(a_n-2q\cosh(2\xi))\cdot\mathrm{Ce}_n(\xi)=0.

Here, q=−c2/4q=-c^2/4 and an∈Ra_n\in\mathbb{R} depends on qq. Accordingly, the functions cen,Cen\mathrm{ce}_n,\mathrm{Ce}_n depend on qq, too, however we suppress the dependence on qq for brevity. We refer to (McLachlan, [27], Chapter 2) for a detailed introduction. The functions {fn}n∈N0\{f_n\}_{n\in\mathbb{N}_0} defined by

fn(x):=Ce2n(ξ0) ce2n(η),x=(cos⁡(η),sin⁡(η))T, η∈(−π,π],f_n(x):=\mathrm{Ce}_{2n}(\xi_0)\,\mathrm{ce}_{2n}(\eta),\qquad x=(\cos(\eta),\sin(\eta))^{\mathsf{T}},\ \eta\in(-\pi,\pi],

form an orthogonal basis of Lsym2(S1)L^2_{\mathrm{sym}}(\mathbb{S}^1), which can be seen as follows. As shown in (McLachlan, [27], Chapter 2), the set {cen,sen+1}n∈N0\{\mathrm{ce}_n,\mathrm{se}_{n+1}\}_{n\in\mathbb{N}_0} is an orthogonal basis of L2((−π,π))L^2((-\pi,\pi)), where sen\mathrm{se}_n are further solutions of Mathieu’s equation, substituting ana_n with different values bnb_n. In fact, {ce2n}n∈N0\{\mathrm{ce}_{2n}\}_{n\in\mathbb{N}_0} consists precisely of those basis elements satisfying the symmetry conditions ce2n(η)=ce2n(−η)\mathrm{ce}_{2n}(\eta)=\mathrm{ce}_{2n}(-\eta) and ce2n(π/2+η)=ce2n(π/2−η)\mathrm{ce}_{2n}(\pi/2+\eta)=\mathrm{ce}_{2n}(\pi/2-\eta), which correspond to the symmetries defining Lsym2(S1)L^2_{\mathrm{sym}}(\mathbb{S}^1). In contrast, sen+1\mathrm{se}_{n+1} is odd and ce2n+1(π/2+η)=−ce2n+1(π/2−η)\mathrm{ce}_{2n+1}(\pi/2+\eta)=-\mathrm{ce}_{2n+1}(\pi/2-\eta) for every n∈N0n\in\mathbb{N}_0. Hence the corresponding functions on S1\mathbb{S}^1 are orthogonal to Lsym2(S1)L^2_{\mathrm{sym}}(\mathbb{S}^1).

We next compute un(0)u_n(0) and ∥fn∥2\lVert f_n\rVert^2, which determine the coefficients in (30). To this end, let f~2n:=u~2n∣DS1\widetilde{f}_{2n}:=\widetilde{u}_{2n}|_{D\mathbb{S}^1} denote the restriction of u~2n\widetilde{u}_{2n} to the ellipse DS1D\mathbb{S}^1. It directly follows that u~2n\widetilde{u}_{2n} solves the Dirichlet problem

{u~−Δ~u~=0in DB2,u~=f~2non DS1.\begin{cases} \widetilde{u}-\widetilde{\Delta}\widetilde{u}=0 & \text{in }D B^2,\\ \widetilde{u}=\widetilde{f}_{2n} & \text{on }D\mathbb{S}^1. \end{cases}

Equivalently, since fn=f~2n∘Df_n=\widetilde{f}_{2n}\circ D, the function un:=u~2n∘Du_n:=\widetilde{u}_{2n}\circ D solves initial Dirichlet problem on R2\mathbb{R}^2 (17) with boundary function fnf_n. With the conventional definition of Mathieu functions (McLachlan, [27], Section 2.21), it holds that ∥cen∥2=π\lVert\mathrm{ce}_n\rVert^2=\pi. Therefore,

un(0)=u~2n(0)=Ce2n(0) ce2n(π/2),and∥fn∥2=Ce2n(ξ0)2∥ce2n∥2=Ce2n(ξ0)2π.u_n(0)=\widetilde{u}_{2n}(0)=\mathrm{Ce}_{2n}(0)\,\mathrm{ce}_{2n}(\pi/2),\qquad\text{and}\qquad\lVert f_n\rVert^2=\mathrm{Ce}_{2n}(\xi_0)^2\lVert\mathrm{ce}_{2n}\rVert^2=\mathrm{Ce}_{2n}(\xi_0)^2\pi.

Figure 2: Mathieu approximation wD,Kw_{D,K} for matrices D=diag⁡(1,λ2)D=\operatorname{diag}(1,\lambda_2) with varying λ2>0\lambda_2>0.

Note that Ce⁡2n(ξ0)≠0\operatorname{Ce}_{2n}(\xi_0)\ne0: otherwise, the function un≠0u_n\ne0 would solve the Dirichlet problem with boundary data fn=0f_n=0, contradicting uniqueness.

Numerical evaluation involves two approximations. We first truncate the series in (30) at order K∈NK\in\mathbb{N}, obtaining

wD,K(x)=∑n=0KCe⁡2n(0)ce⁡2n(π/2)Ce⁡2n(ξ0)πce⁡2n(η),for x=(cos⁡(η),sin⁡(η))T.w_{D,K}(x)=\sum_{n=0}^{K}\frac{\operatorname{Ce}_{2n}(0)\operatorname{ce}_{2n}(\pi/2)}{\operatorname{Ce}_{2n}(\xi_0)\pi}\operatorname{ce}_{2n}(\eta),\qquad\text{for }x=(\cos(\eta),\sin(\eta))^{\mathsf{T}}.

Since wD∈C∞(S1)w_D\in C^\infty(\mathbb{S}^{1}), it follows from [36 Theorem 4.1] that, for every s≥1s\ge1

max⁡x∈S1∣wD,K(x)−wD(x)∣≤C(s,q)K1/2−s∥wD∥Hs\max_{x\in\mathbb{S}^{1}}|w_{D,K}(x)-w_D(x)|\le C(s,q)K^{1/2-s}\lVert w_D\rVert_{H^s}

where C(s,q)C(s,q) is independent of KK and ∥⋅∥Hs\lVert\cdot\rVert_{H^s} denotes the norm of the Sobolev space Hs(S1)H^s(\mathbb{S}^{1}). In particular, this yields uniform convergence of the approximation as K→∞K\to\infty.

The evaluation of Ce⁡2n,ce⁡2n\operatorname{Ce}_{2n},\operatorname{ce}_{2n} is also done approximately by a truncated Fourier series as in [13 eq. 16–20], where the Fourier coefficients are calculated by the term-recurrences specified therein. In Figure 2 we visualize the approximations obtained this way for different interaction matrices DD. We evaluate the energy of an empirical approximation supported on equally spaced points of S1\mathbb{S}^{1}. For N∈NN\in\mathbb{N}, set

Figure 2. (a) Visualization of the approximation wD,K,Nw_{D,K,N}. (b) Energies of Mathieu approximation compared to the ones obtained with (29) for N=1000N=1000.

sℓ=(cos⁡(ηℓ),sin⁡(ηℓ)),ηℓ=2πℓN,ℓ=1,…,N,s_\ell=(\cos(\eta_\ell),\sin(\eta_\ell)),\qquad\eta_\ell=\frac{2\pi\ell}{N},\qquad\ell=1,\ldots,N,

and define

wD,K,N:=∑ℓ=1N(wD,K(sℓ)∑j=1NwD,K(sj))δsℓ.\mathbf{w}_{D,K,N}:=\sum_{\ell=1}^{N}\left(\frac{w_{D,K}(s_\ell)}{\sum_{j=1}^{N}w_{D,K}(s_j)}\right)\delta_{s_\ell}.

We then report ED(wD,K,N)\mathcal{E}_D(\mathbf{w}_{D,K,N}). By Corollary 8, wDw_D is bounded away from zero. Uniform convergence therefore implies the existence of KD∈NK_D\in\mathbb{N} such that wD,K>0w_{D,K}>0 on S1\mathbb{S}^{1} for every K≥KDK\ge K_D. In the implementation, we start with K=60K=60 and double KK until all evaluated weights are positive. This condition ensures that the normalized discrete measure is a probability measure; it does not provide an estimate of the truncation error.

Acknowledgments

The authors thank the organizers of the 2nd Workshop on Machine Learning in Infinite Dimensions, held at ETH Zürich on 8–11 September 2025, where this work originated. Y.K. is grateful to the University of Genova for supporting his research stay in Autumn 2026, when part of this work was undertaken.

O.K. acknowledges financial support of the European Commission (Horizon Europe grant ELIAS 101120237). M.B., S.K. and T.R. acknowledge support from DESY (Hamburg, Germany), a member of the Helmholtz Association HGF. This work was carried in part while T.R. was a member of DESY and was supported in part through the Maxwell computational resources operated at Deutsches Elektronen-Synchrotron DESY, Hamburg, Germany. M.B. and T.R. acknowledge funding by the German Federal Ministry of Research, Technology and Space (BMFTR) under grant agreement No. 16IS24072B and 16IS24072D (COMFORT). M.B. and S.K. acknowledge support from the German Research Foundation, project BU 2327/19-2. T.R. acknowledges the support of COST Action CA24136 - InterCoML, the Munich Center for Machine Learning and the ERC Advanced Grant NEITALG, grant agreement No. 101198055. Funded by the European Union. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them.

Tool and computational resource disclosure. Generative AI was used for literature review, assistance with code for the numerical experiments, as well as language and grammar corrections.

References

  1. [1]Antonio Álvarez-López, Borjan Geshkovski, and Domènec Ruiz-Balet. Perceptrons and localization of attention’s mean-field landscape. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 2180–2206. PMLR, 2026. URL https://proceedings.mlr.press/v306/alvarez-lopez26a.html.
  2. [2]Nachman Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3):337–404, 1950. doi: 10.1090/S0002-9947-1950-0051437-7.DOI
  3. [3]Paolo Baldi. Stochastic Calculus: An Introduction Through Theory and Exercises. Universitext. Springer, Cham, 2017. ISBN 978-3-319-62225-5. doi: 10.1007/978-3-319-62226-2.DOI
  4. [4]Dmitriy Bilyk, Ryan W. Matzke, and Oleksandr Vlasiuk. Positive definiteness and the Stolarsky invariance principle. Journal of Mathematical Analysis and Applications, 513(2):126220, 2022. doi: 10.1016/j.jmaa.2022.126220. URL https://doi.org/10.1016/j.jmaa.2022.126220.
  5. [5]Sergiy V. Borodachov, Douglas P. Hardin, and Edward B. Saff. Discrete Energy on Rectifiable Sets. Springer Monographs in Mathematics. Springer, New York, 2019. doi: 10.1007/978-0-387-84808-2. URL https://doi.org/10.1007/978-0-387-84808-2.
  6. [6]Mireille Bossy, Nicolas Champagnat, Sylvain Maire, and Denis Talay. Probabilistic interpretation and random walk on spheres algorithms for the Poisson–Boltzmann equation in molecular dynamics. ESAIM: Mathematical Modelling and Numerical Analysis, 44(5):997–1048, 2010. doi: 10.1051/m2an/2010050.DOI
  7. [7]Martin Burger and Marco Di Francesco. Large time behavior of nonlocal aggregation models with nonlinear diffusion. Networks and Heterogeneous Media, 3(4):749–785, 2008.DOI
  8. [8]Martin Burger, Razvan Fetecau, and Yanghong Huang. Stationary states and asymptotic behavior of aggregation models with nonlinear local repulsion. SIAM Journal on Applied Dynamical Systems, 13(1):397–424, 2014.DOI
  9. [9]Martin Burger, Samira Kabri, Yury Korolev, Tim Roith, and Lukas Weigand. Analysis of mean-field models arising from self-attention dynamics in transformer architectures with layer normalization. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 383(2298):20240233, 2025. doi: 10.1098/rsta.2024.0233. URL https://doi.org/10.1098/rsta.2024.0233.
  10. [10]José A. Cañizo and Alejandro Ramos-Lora. Discrete minimizers of the interaction energy in collective behavior: a brief numerical and analytic review. In José Antonio Carrillo and Eitan Tadmor, editors, Active Particles, Volume 4: Theory, Models, Applications, Modeling and Simulation in Science, Engineering and Technology, pages 55–78. Birkhäuser, Cham, 2024. doi: 10.1007/978-3-031-73423-6_2.DOI
  11. [11]José A. Carrillo, Alessio Figalli, and Francesco S. Patacchini. Geometry of minimizers for the interaction energy with mildly repulsive potentials. Annales de l’Institut Henri Poincaré C, Analyse non linéaire, 34(5):1299–1308, 2017. doi: 10.1016/j.anihpc.2016.10.004.
  12. [12]José Antonio Carrillo, Michel Chipot, and Yanghong Huang. On global minimizers of repulsive-attractive power-law interaction energies. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 372(2028):20130399, 2014. doi: 10.1098/rsta.2013.0399. URL https://doi.org/10.1098/rsta.2013.0399.
  13. [13]Lorea Chaos-Cador and E. Ley-Koo. Mathieu functions revisited: matrix evaluation and generating functions. Revista Mexicana de Física, 48(1):67–75, 2002.
  14. [14]Rick Durrett. Probability: Theory and Examples, volume 49 of Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, fifth edition, 2019. ISBN 978-1-108-47368-2.
  15. [15]Boris Stepanovich Elepov and Guennady Alekseevich Mikhailov. Solution of the Dirichlet problem for the equation Δu − cu = −q by a model of “walks on spheres”. USSR Computational Mathematics and Mathematical Physics, 9(3):194–204, 1969.DOI
  16. [16]Amic Frouvelle and Jian-Guo Liu. Dynamics in a kinetic model of oriented particles with phase transition. SIAM Journal on Mathematical Analysis, 44(2):791–826, 2012. doi: 10.1137/110823912. URL https://doi.org/10.1137/110823912.
  17. [17]Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. A mathematical perspective on transformers. Bulletin of the American Mathematical Society, 62(3):427–479, 2025. doi: 10.1090/bull/1863. URL https://doi.org/10.1090/bull/1863.
  18. [18]David Gilbarg and Neil S. Trudinger. Elliptic Partial Differential Equations of Second Order. Classics in Mathematics. Springer, Berlin, Heidelberg, second edition, 2001. doi: 10.1007/978-3-642-61798-0. URL https://doi.org/10.1007/978-3-642-61798-0.
  19. [19]Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012. URL https://jmlr.org/papers/v13/gretton12a.html.DOI
  20. [20]Douglas P Hardin and Edward B Saff. Minimal riesz energy point configurations for rectifiable d-dimensional manifolds. Advances in Mathematics, 193(1):174–204, 2005.
  21. [21]Ioannis Karatzas and Steven E. Shreve. Brownian Motion and Stochastic Calculus, volume 113 of Graduate Texts in Mathematics. Springer, New York, NY, second edition, 1991. ISBN 978-0-387-97655-6. doi: 10.1007/978-1-4612-0949-2.
  22. [22]Andreas E. Kyprianou, Ana Osojnik, and Tony Shardlow. Unbiased ‘walk-on-spheres’ Monte Carlo methods for the fractional Laplacian. IMA Journal of Numerical Analysis, 38(3):1550–1578, 2018. doi: 10.1093/imanum/drx042.DOI
  23. [23]Christopher Langrenez, Wilfred Salmon, Stephan De Bièvre, Jonathan J. Thio, Christopher K. Long, and David R. M. Arvidsson-Shukur. Almost sure minimality of the set of experiments with classical Kirkwood–Dirac representations. Physical Review A, 113(6):062215, June 2026. doi: 10.1103/v7z4-qsz8.DOI
  24. [24]Jean-François Le Gall. Brownian Motion, Martingales, and Stochastic Calculus, volume 274 of Graduate Texts in Mathematics. Springer, Cham, 2016. ISBN 978-3-319-31088-6. doi: 10.1007/978-3-319-31089-3.
  25. [25]J Maćkowiak. On the mean field description of systems of point particles in equilibrium. Physica A: Statistical Mechanics and its Applications, 110(1-2):302–320, 1982.DOI
  26. [26]Michael Mascagni and Chi-Ok Hwang. ϵ-shell error analysis for “Walk On Spheres” algorithms. Mathematics and Computers in Simulation, 63(2):93–104, 2003. doi: 10.1016/S0378-4754(03)00038-7.DOI
  27. [27]Norman W. McLachlan. Theory and Application of Mathieu Functions. Oxford University Press, Oxford, 1951.
  28. [28]Josef Meixner and Friedrich Wilhelm Schäfke. Mathieusche Funktionen und Sphäroidfunktionen: Mit Anwendungen auf physikalische und technische Probleme, volume 71 of Die Grundlehren der mathematischen Wissenschaften. Springer, Berlin, Heidelberg, 1954. doi: 10.1007/978-3-662-00941-3.DOI
  29. [29]Mervin E. Muller. Some continuous Monte Carlo methods for the Dirichlet problem. The Annals of Mathematical Statistics, 27(3):569–589, 1956. doi: 10.1214/aoms/1177728169.DOI
  30. [30]Antti Rasila and Tommi Sottinen. Yukawa potential, panharmonic measure and Brownian motion. Axioms, 7(2):28, 2018. doi: 10.3390/axioms7020028. URL https://doi.org/10.3390/axioms7020028.
  31. [31]Edward B Saff and Amo BJ Kuijlaars. Distributing many points on a sphere. Mathematical Intelligencer, 19(1):5, 1997.DOI
  32. [32]Michael E. Sander, Pierre Ablin, Mathieu Blondel, and Gabriel Peyré. Sinkformers: Transformers with doubly stochastic attention. In Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, volume 151 of Proceedings of Machine Learning Research, pages 3515–3530. PMLR, 2022. URL https://proceedings.mlr.press/v151/sander22a.html.arxiv.org/abs/2110.11773
  33. [34]Rohan Sawhney and Keenan Crane. Monte Carlo geometry processing: A grid-free approach to PDE-based methods on volumetric domains. ACM Transactions on Graphics, 39(4), 2020. doi: 10.1145/3386569.3392374.DOI
  34. [35]Sylvia Serfaty. Ginzburg–landau vortices, coulomb gases, and abrikosov lattices. Comptes Rendus. Physique, 15(6):539–546, 2014.DOI
  35. [36]Jie Shen and Li-Lian Wang. On spectral approximations in elliptical geometries using Mathieu functions. Mathematics of Computation, 78(266):815–844, 2009.DOI
  36. [37]Ruiwen Shu. Wasserstein-infinity stability and mean field limit of discrete interaction energy minimizers. arXiv preprint arXiv:2407.18395, 2024. URL https://arxiv.org/abs/2407.18395.
  37. [38]Robert Simione, Dejan Slepčev, and Ihsan Topaloglu. Existence of ground states of nonlocal-interaction energies. Journal of Statistical Physics, 159(4):972–986, 2015.
  38. [39]Carl-Johann Simon-Gabriel and Bernhard Schölkopf. Kernel distribution embeddings: Universal kernels, characteristic kernels and kernel metrics on distributions. Journal of Machine Learning Research, 19(44):1–29, 2018. URL https://jmlr.org/papers/v19/16-291.html.
  39. [40]Carl-Johann Simon-Gabriel, Alessandro Barp, Bernhard Schölkopf, and Lester Mackey. Metrizing weak convergence with maximum mean discrepancies. Journal of Machine Learning Research, 24 (184):1–20, 2023. URL https://jmlr.org/papers/v24/21-0599.html.
  40. [41]Bharath K. Sriperumbudur, Kenji Fukumizu, and Gert R. G. Lanckriet. Universality, characteristic kernels and RKHS embedding of measures. Journal of Machine Learning Research, 12(70):2389–2410, 2011. URL https://jmlr.org/papers/v12/sriperumbudur11a.html.arxiv.org/abs/1003.0887
  41. [42]PAJ Tindemans and HW Capel. An exact calculation of the free energy in systems with separable interactions. Physica, 72(3):433–464, 1974.DOI
  42. [43]V. S. Varadarajan. On the convergence of sample probability distributions. Sankhyā: The Indian Journal of Statistics, 19(1/2):23–26, 1958. URL https://www.jstor.org/stable/25048365.
  43. [44]Laurent Véron. Elliptic equations involving measures. In Michel Chipot and Pavol Quittner, editors, Handbook of Differential Equations: Stationary Partial Differential Equations, volume 1, pages 593–712. Elsevier/North-Holland, Amsterdam, 2004. ISBN 978-0-444-51126-3. doi: 10.1016/S1874-5733(04)80010-X.DOI
  44. [45]Cédric Villani. Optimal Transport: Old and New, volume 338 of Grundlehren der mathematischen Wissenschaften. Springer, 2009. doi: 10.1007/978-3-540-71050-9.
  45. [46]Edmund Taylor Whittaker and George Neville Watson. A Course of Modern Analysis. Cambridge University Press, Cambridge, fourth edition, 1927.
  46. [47]David Williams. Probability with Martingales. Cambridge Mathematical Textbooks, Cambridge University Press, Cambridge, 1991. ISBN 978-0-521-40605-5. doi: 10.1017/CBO9780511813658.DOI
  47. [48]Gerhard Wolf. Mathieu functions and Hill’s equation. In F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain, editors, NIST Digital Library of Mathematical Functions, chapter 28. National Institute of Standards and Technology, 2026. URL https://dlmf.nist.gov/28. Release 1.2.8 of 2026-09-15.

Paper details

vlnc:W_rgeen4b3
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Generative AI

View statement
Looking for related papers…
Contents