Introduction
In modern statistical practice, variational inference ([3 , 14 , 28 ] ) has become a primary methodology for scalable approximate inference, most notably for approximating intractable posterior distributions in Bayesian models. Rather than sampling directly from a complex target distribution p p p on a sample space X \mathcal{X} X , variational inference casts inference as an optimization problem: one seeks a distribution q ∈ Q q \in\mathcal{Q} q ∈ Q within a tractable variational family that minimizes a statistical divergence—most commonly the Kullback–Leibler (KL) divergence KL ( q ∥ p ) \operatorname{KL}(q\|p) KL ( q ∥ p ) . In practice, the variational family Q \mathcal{Q} Q is nearly always misspecified with respect to p p p : factorized mean-field families sacrifice dependence structure for tractability, while structured transformation families and normalizing flows ([24 , 26 ] ) trade computational cost for richer geometry. Because p p p rarely belongs to Q \mathcal{Q} Q , a foundational statistical question arises: even under model misspecification, when does the variational optimum exactly recover meaningful structural parameters of the target distribution?
Symmetry offers a natural geometric principle to address this question. In classical mathematical statistics, group invariance is foundational: equivariance principles govern the theory of optimal invariant estimators and hypothesis tests ([8 , 18 , 30 ] ), the Hunt–Stein theorem establishes conditions under which minimax procedures can be chosen to be group-invariant ([4 ] ), and the interplay between sufficiency and invariance is central to statistical decision theory ([12 ] ). In contemporary machine learning, group invariance and equivariance have similarly guided the design of structured neural architectures ([6 , 7 , 29 ] ). By contrast, the role of distributional symmetry in shaping the optimization landscape of variational inference—specifically, how shared invariances between the variational base distribution and the standardized target distribution constrain stationary points of the variational objective—has only recently begun to receive systematic study.
A striking step in this direction was taken by Margossian and Saul (2025), who demonstrated that variational inference can achieve exact parameter recovery in misspecified location families (generated by translations x ↦ x + μ x \mapsto x+\mu x ↦ x + μ ) and location-scale families (generated by affine transformations x ↦ μ + A x x \mapsto\mu+Ax x ↦ μ + A x ) whenever the base distribution q 0 q_0 q 0 and the standardized target share a common symmetry. Specifically, they established that the gradient of the KL divergence vanishes at the true location parameter under central reflection symmetry (x ↦ − x x \mapsto-x x ↦ − x ), and at the true correlation matrix under elliptical symmetry, while uncovering a persistent scale bias when estimating the full covariance matrix. These findings led the authors to pose the following general question:
“Finally, an intriguing conjecture, also deserving of study, is that for each symmetry of p p p , there is a corresponding statistic that VI under mild conditions can exactly recover.”
In subsequent work, Margossian et al. [21 ] extended these location-scale recovery guarantees from the standard variational objective KL ( q ∥ p ) \operatorname{KL}(q\Vert p) KL ( q ∥ p ) to general f f f -divergences without requiring log-concavity. In the present paper, we address this conjecture in full generality by developing a Lie-theoretic and representation-theoretic framework for transformation-based variational inference. While our first-order symmetry constraint (Theorem 6.2 ) relies only on diffeomorphism invariance of the divergence and therefore applies broadly to any invariant loss functional, working explicitly with the KL divergence KL ( q g ∥ p ) \operatorname{KL}(q_g\Vert p) KL ( q g ∥ p ) allows us to derive exact Stein-type moment-matching conditions that govern the remaining non-stationary directions. Our analysis reveals precisely how the algebraic compatibility between the symmetry group of the distribution and the Lie group of variational transports determines which parameter components are recovered without bias and which remain susceptible to model misspecification.
Concretely, we study the interaction between two group actions on the sample space X \mathcal{X} X : a compact symmetry group H H H acting by isometries, and a transport Lie group G G G acting by diffeomorphisms to generate the variational family. This formulation connects classical statistical transformation models (Eaton, [8 ] ) and Lie group representation theory ([9 , 11 , 15 ] ) with information geometry ([2 , 23 ] ) and Riemannian optimization on manifolds ([1 , 5 ] ).
To formalize the estimation problem, we consider variational families obtained by pushing forward a fixed reference distribution q 0 ∈ P ( X ) q_0 \in\mathcal{P}(\mathcal{X}) q 0 ∈ P ( X ) under the action of G G G : if Z ∼ q 0 Z \sim q_0 Z ∼ q 0 , then X = g ⋅ Z ∼ q g X=g\cdot Z \sim q_g X = g ⋅ Z ∼ q g , yielding the variational optimization problem
min g ∈ G KL ( q g ∥ p ) . \min_{g\in G} \operatorname{KL}(q_g\Vert p). g ∈ G min KL ( q g ∥ p ) .
On the data side, we assume that the target distribution p p p is generated by transporting an unknown standardized distribution p ~ \tilde{p} p ~ by a true parameter g ∗ ∈ G g^* \in G g ∗ ∈ G that we wish to estimate; equivalently, p ~ \tilde{p} p ~ is the pullback of p p p under g ∗ g^* g ∗ , meaning that if X ∼ p X\sim p X ∼ p , then the standardized variable ξ = ( g ∗ ) − 1 ⋅ X \xi=(g^*)^{-1}\cdot X ξ = ( g ∗ ) − 1 ⋅ X follows p ~ \tilde{p} p ~ . Importantly, we do not assume that p ~ \tilde{p} p ~ coincides with q 0 q_0 q 0 , so the variational family Q G = { q g : g ∈ G } \mathcal{Q}_G=\{q_g:g\in G\} Q G = { q g : g ∈ G } may be misspecified (p ∉ Q G p\notin\mathcal{Q}_G p ∈ / Q G ). Instead, we assume a shared structural symmetry : both the variational base q 0 q_0 q 0 and the standardized target p ~ \tilde{p} p ~ are invariant under the action of the symmetry group H H H . Geometrically, this means that q 0 q_0 q 0 and p ~ \tilde{p} p ~ share the same notion of centering or alignment around the origin, and the statistical question is whether this shared invariance suffices to make g ∗ g^* g ∗ a stationary point (and local minimizer) of the variational loss g ↦ KL ( q g ∥ p ) g\mapsto\operatorname{KL}(q_g\Vert p) g ↦ KL ( q g ∥ p ) .
Main contributions. We characterize the first-order geometry of transport variational inference at g ∗ g^* g ∗ through the conjugation action of H H H on G G G and its linearization, the adjoint representation of H H H on the Lie algebra g = T e G \mathfrak{g}=T_eG g = T e G :
A Lie-theoretic framework for symmetric transport VI (§ 3 –§ 4 ). We formulate a general statistical framework pairing a compact symmetry group H H H with an H H H -normalized transport Lie group G G G . Analyzing the total transformation group K = G H K = GH K = G H and the intersection subgroup N = G ∩ H N = G \cap H N = G ∩ H allows us to separate parameter identifiability on the coset manifold M = G / N \mathcal{M} = G/N M = G / N from first-order estimation bias, unifying location, scale, directional, and permutation models under a single geometric theory.
Gradient constraint and orthogonal Lie algebra decomposition (Theorem 6.2 and Proposition 6.3). We prove that the Riemannian gradient ∇ G L ( g ∗ ) \nabla_G \mathcal{L}(g^*) ∇ G L ( g ∗ ) is constrained to lie in the trivial representation g H = { u ∈ g : Ad h u = u ∀ h ∈ H } \mathfrak{g}^{H} = \{u \in\mathfrak{g} : \operatorname{Ad}_h u = u \ \forall h \in H\} g H = { u ∈ g : Ad h u = u ∀ h ∈ H } of the adjoint action of H H H on g \mathfrak{g} g . Combining this invariance with the identifiability structure yields an orthogonal decomposition of the transport Lie algebra,
g = n ⊕ g b i a s H ⊕ g ⊥ , \mathfrak{g} = \mathfrak{n} \oplus\mathfrak{g}^{H}_{\mathrm{bias}} \oplus\mathfrak{g}^{\perp}, g = n ⊕ g bias H ⊕ g ⊥ ,
into an identifiability kernel n \mathfrak{n} n (along which the loss is locally constant), a bias subspace g b i a s H = g H ∩ n ⊥ \mathfrak{g}^{H}_{\mathrm{bias}} = \mathfrak{g}^{H} \cap\mathfrak{n}^{\perp} g bias H = g H ∩ n ⊥ (consisting of symmetry-preserving perturbations where model mismatch can induce a nonzero gradient), and a symmetry-breaking subspace g ⊥ = ( g H ) ⊥ ∩ n ⊥ \mathfrak{g}^{\perp} = (\mathfrak{g}^{H})^{\perp} \cap\mathfrak{n}^{\perp} g ⊥ = ( g H ) ⊥ ∩ n ⊥ (along which the gradient is forced to vanish by symmetry alone). Whenever g b i a s H = { 0 } \mathfrak{g}^{H}_{\mathrm{bias}} = \{0\} g bias H = { 0 } , g ∗ g^* g ∗ is guaranteed to be a first-order stationary point regardless of model misspecification between q 0 q_0 q 0 and p ~ \tilde{p} p ~ .
Stein-type moment-matching condition and debiasing recipe (Theorem 6.5 ). When the bias subspace g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H is nontrivial, we derive an exact necessary and sufficient condition for g ∗ g^* g ∗ to be a critical point. Using integration by parts along the infinitesimal generators V u V_u V u of the group action, we show that the gradient along u ∈ g b i a s H u \in\mathfrak{g}^{H}_{\mathrm{bias}} u ∈ g bias H vanishes if and only if the directional scores of the log-potentials of q 0 q_0 q 0 and p ~ \tilde{p} p ~ match in expectation under q 0 q_0 q 0 . This characterization provides a constructive calibration procedure to eliminate estimation bias by rescaling or shifting the base distribution q 0 q_0 q 0 along exp ( g b i a s H ) \exp(\mathfrak{g}^{H}_{\mathrm{bias}}) exp ( g bias H ) while preserving its H H H -invariance.
Organization. The remainder of the paper is organized as follows. § 2 situates our work within the literature on variational inference, group invariance in statistics, and Riemannian optimization. § 3 introduces transport variational families, H H H -conjugate symmetry, and the structural normalization condition, together with six guiding examples that we trace throughout the paper. § 4 analyzes the total transformation group K = G H K = GH K = G H , the conjugation homomorphism, and the three resulting identifiability regimes on the quotient space M = G / N \mathcal{M} = G/N M = G / N . § 5 develops the required differential calculus on G G G and introduces the adjoint representation Ad H \operatorname{Ad}_H Ad H and its trivial subspace g H \mathfrak{g}^{H} g H . § 6 presents our main theoretical results: the gradient constraint theorem, the orthogonal decomposition of g \mathfrak{g} g , and the moment-matching theorem. § 7 discusses gauge-fixing vs. base calibration, second-order curvature conditions, and extensions. Technical proofs and detailed example derivations are deferred to the Appendices.
Related work
Variational inference and transport-based families. The variational formulation of approximate inference, rooted in statistical physics and convex duality, was brought to prominence in statistics and machine learning by Jordan et al. [14 ] , Wainwright and Jordan [28 ] , and Blei et al. [3 ] . The development of black-box variational inference and reparameterization gradients ([25 ] ; [26 ] ) subsequently enabled optimization over general continuous families where expectations under q g q_g q g are evaluated by pushing forward samples Z ∼ q 0 Z \sim q_0 Z ∼ q 0 through a differentiable transformation x = g ( Z ) x = g(Z) x = g ( Z ) . In the normalizing flow literature ([16 ] ; [24 ] ; [26 ] ), the transformation g g g is typically parameterized by a deep composition of general diffeomorphisms. Our framework studies the structured setting in which the transport maps form a finite-dimensional Lie group G G G acting on X \mathcal{X} X : this encompasses classical parametric families (such as location, scale, and directional families) while endowing the variational parameter space with an explicit Lie algebra that makes global and local symmetry analysis tractable.
Group invariance in mathematical statistics and machine learning. Group actions have long served as a unifying language in theoretical statistics [8 , 18 , 30 ] . In classical statistical decision theory, when a family of distributions is closed under a group action, one seeks estimators that are equivariant —such as the Pitman estimator for location or invariant Bayes estimators under right Haar measure—and hypothesis tests that are invariant with respect to nuisance group actions. In those classical works, the statistical model is typically well-specified and the group acts transitively on the parameter space to reduce risk. Our perspective differs in two important respects: first, we study the deterministic optimization landscape of a divergence functional under model misspecification (p ≠ Q G p \ne Q_G p = Q G ); second, the symmetry group H H H of the standardized distribution and the transport group G G G indexing the variational family need not coincide, and their interaction via conjugation governs which parameters are unbiasedly estimated. In a complementary direction, geometric deep learning and equivariant neural networks [6 , 7 , 29 ] build group symmetries directly into function approximators and normalizing flows [17 ] ; rather than constraining the variational family to produce strictly H H H -invariant distributions (which would preclude estimating symmetry-breaking parameters g ∗ g^{*} g ∗ ), our work uses H H H -invariant base distributions pushed forward by G G G to estimate the unknown transformation g ∗ g^{*} g ∗ .
Information geometry and optimization on Lie groups. Differential-geometric approaches to statistical estimation have been extensively studied in information geometry [2 , 23 ] , where the Fisher–Rao metric defines a natural Riemannian structure on parametric families, and in optimization on smooth manifolds and Lie groups [1 , 5 ] . When a variational family is indexed by a Lie group G G G , left-trivialization identifies the tangent space T g G T_gG T g G at any parameter g g g with the Lie algebra g = T e G \mathfrak{g}=T_eG g = T e G . By equipping g \mathfrak{g} g with an Ad H \operatorname{Ad}_H Ad H -invariant inner product (guaranteed by the compactness of H H H ), we connect Riemannian first-order stationarity directly to the decomposition of g g g into irreducible representations of H H H .
Symmetry in variational inference. The most direct predecessors of the present work are Margossian and Saul [19 ] and Margossian et al. [21 ] , who proved that location-scale variational inference exactly recovers the target mean under even symmetry and the target correlation matrix under elliptical symmetry across general f f f -divergences, and Zellinger and Vergari [31 ] , who extended even-symmetry location recovery to forward KL and α \alpha α -divergences without log-concavity assumptions. Complementing these positive results, Margossian et al. [20 ] established an impossibility theorem for factorized (mean-field) variational approximations of non-factorized targets, proving that a mean-field family can accurately recover at most one of three canonical dispersion summaries (marginal variances, marginal precisions, or generalized variance). This impossibility underscores the necessity of non-factorized group-transport families such as full GL ( d ) \operatorname{GL}(d) GL ( d ) covariance models. Concurrently with our work, Marks et al. [22 ] developed a global theory of symmetry-induced statistic recovery in variational inference, characterizing when global minimizers inherit target symmetries and applying their framework to von Mises–Fisher distributions on the sphere, while Ghafourpour et al. [10 ] investigated radial transport families. Our differential-geometric approach is directly complementary: by linearizing the conjugation action of H H H on G G G via the adjoint representation, we provide an explicit subspace decomposition of the transport Lie algebra g \mathfrak{g} g that simultaneously explains both why symmetry-breaking gradient directions vanish and why symmetry-preserving directions (g b i a s H \mathfrak{g}_{\mathrm{bias}}^{H} g bias H ) exhibit estimation bias unless calibrated by moment matching.
Problem description
In parametric variational inference [3 , 25 , 28 ] , one approximates an intractable target distribution p ∈ P ( X ) p \in\mathcal{P}(\mathcal{X}) p ∈ P ( X ) on a sample space X \mathcal{X} X by selecting a candidate distribution q θ q_\theta q θ from a parameterized family Q Θ = { q θ ∈ P ( X ) : θ ∈ Θ } \mathcal{Q}_\Theta= \{q_\theta\in\mathcal{P}(\mathcal{X}) : \theta\in\Theta\} Q Θ = { q θ ∈ P ( X ) : θ ∈ Θ } that minimizes a divergence functional over θ ∈ Θ \theta\in\Theta θ ∈ Θ . In this paper, we study structured variational families generated by the smooth action of a transport Lie group G G G on X \mathcal{X} X , in the presence of a symmetry group H H H acting isometrically on X \mathcal{X} X . For any g ∈ G g \in G g ∈ G (respectively, h ∈ H h \in H h ∈ H ) and any point x ∈ X x \in\mathcal{X} x ∈ X , we denote the group action on the sample space by g ⋅ x ∈ X g \cdot x \in\mathcal{X} g ⋅ x ∈ X (respectively, h ⋅ x ∈ X h \cdot x \in\mathcal{X} h ⋅ x ∈ X ).
Viewing both G G G and H H H through their actions on the common domain X \mathcal{X} X serves two fundamental purposes. First, any measurable action on X \mathcal{X} X canonically induces a pushforward action on probability measures over X \mathcal{X} X : transforming a random variable Z ∼ q 0 Z \sim q_0 Z ∼ q 0 via X = g ⋅ Z X = g \cdot Z X = g ⋅ Z transports its law to q g = g ♯ q 0 q_g = g_{\sharp}q_0 q g = g ♯ q 0 . Second, because both G G G and H H H act as transformations of X \mathcal{X} X , they can be composed within the ambient group Aut ( X ) \operatorname{Aut}(\mathcal{X}) Aut ( X ) of diffeomorphisms of X \mathcal{X} X , even when G G G and H H H are defined a priori as distinct algebraic groups. As we show throughout the paper, the non-commutativity of these two actions on X \mathcal{X} X —captured algebraically by the conjugation of transports in G G G by symmetries in H H H —is the central mechanism governing parameter identifiability and unbiased estimation in variational inference.
Group transport variational inference
We begin by formalizing variational families generated by pushing forward a fixed reference measure q 0 q_0 q 0 under a Lie group action. From a statistical standpoint, this construction generalizes classical location-scale transformation models [8 ] to arbitrary smooth manifolds and Lie groups; from a machine learning standpoint, it corresponds to a normalizing flow [24 , 26 ] whose transformation maps are structured as a Lie group G G G . Throughout the paper, we restrict attention to distributions admitting strictly positive, smooth (or piecewise smooth, locally Lipschitz) densities with respect to the canonical reference measure on X \mathcal{X} X (Lebesgue measure when X = R d \mathcal{X} = \mathbb{R}^{d} X = R d , or the Riemannian volume measure when X \mathcal{X} X is a Riemannian manifold), and we identify each probability measure with its density.
Definition 3.1 (Transport variational inference). Let X \mathcal{X} X be a smooth data domain (such as Euclidean space R d \mathbb{R}^{d} R d or a smooth Riemannian manifold) and let G G G be a Lie group acting smoothly and effectively on X \mathcal{X} X by diffeomorphisms, written ( g , x ) ↦ g ⋅ x ∈ X (g,x) \mapsto g \cdot x \in\mathcal{X} ( g , x ) ↦ g ⋅ x ∈ X (where acting effectively simply means that distinct group elements g ≠ g ′ g \ne g' g = g ′ define distinct transformations on X \mathcal{X} X ). We call G G G the parameter transport group . Given a base distribution q 0 ∈ P ( X ) q_0 \in\mathcal{P}(\mathcal{X}) q 0 ∈ P ( X ) and a target data distribution p ∈ P ( X ) p \in\mathcal{P}(\mathcal{X}) p ∈ P ( X ) , both with positive densities on X \mathcal{X} X , we define:
q g ( x ) = q 0 ( g − 1 ⋅ x ) ∣ det J g − 1 ( x ) ∣ , q_g(x) = q_0(g^{-1} \cdot x) \left|\det J_{g^{-1}}(x)\right|, q g ( x ) = q 0 ( g − 1 ⋅ x ) det J g − 1 ( x ) ,
where J g − 1 ( x ) J_{g^{-1}}(x) J g − 1 ( x ) denotes the Jacobian of the inverse map x ↦ g − 1 ⋅ x x \mapsto g^{-1} \cdot x x ↦ g − 1 ⋅ x with respect to the reference volume measure on X \mathcal{X} X (in particular, ∣ det J g − 1 ( x ) ∣ = 1 \left|\det J_{g^{-1}}(x)\right| = 1 det J g − 1 ( x ) = 1 whenever G G G acts by isometries).
The transport variational family Q G = { q g ∈ P ( X ) ∣ g ∈ G } \mathcal{Q}_G = \{q_g \in\mathcal{P}(\mathcal{X}) \mid g \in G\} Q G = { q g ∈ P ( X ) ∣ g ∈ G } .
The variational inference objective L : G → R \mathcal{L}: G \to\mathbb{R} L : G → R , defined via the Kullback–Leibler divergence:
L ( g ) = KL ( q g ∥ p ) = E x ∼ q g [ log q g ( x ) − log p ( x ) ] . \mathcal{L}(g) = \operatorname{KL}(q_g \| p) = \mathbb{E}_{x \sim q_g}\left[\log q_g(x) - \log p(x)\right]. L ( g ) = KL ( q g ∥ p ) = E x ∼ q g [ log q g ( x ) − log p ( x ) ] .
Illustrative examples of transport families. To ground Definition Definition 3.1 , consider three classical parametric families obtained as group pushforwards of a standard base distribution q 0 q_0 q 0 :
Gaussian location family: On X = R d \mathcal{X} = \mathbb{R}^{d} X = R d , let G = ( R d , + ) G = (\mathbb{R}^{d}, +) G = ( R d , + ) act by translation g ⋅ x = x + μ g g \cdot x = x + \mu_{g} g ⋅ x = x + μ g . Taking the standard Gaussian base q 0 = N ( 0 , I d ) q_{0} = \mathcal{N}(0, I_{d}) q 0 = N ( 0 , I d ) yields the homoscedastic location family
Q G m e a n = { N ( μ g , I d ) ∣ μ g ∈ R d } . Q_{G}^{\mathrm{mean}} = \{\mathcal{N}(\mu_{g}, I_{d}) \mid\mu_{g} \in\mathbb{R}^{d}\}. Q G mean = { N ( μ g , I d ) ∣ μ g ∈ R d } .
Gaussian scale/covariance family: On X = R d \mathcal{X} = \mathbb{R}^{d} X = R d , let the general linear group G = G L ( d ) G = \mathrm{GL}(d) G = GL ( d ) act linearly by g ⋅ x = A g x g \cdot x = A_{g}x g ⋅ x = A g x . If Z ∼ q 0 = N ( 0 , I d ) Z \sim q_{0} = \mathcal{N}(0, I_{d}) Z ∼ q 0 = N ( 0 , I d ) , then A g Z ∼ N ( 0 , A g A g T ) A_{g}Z \sim\mathcal{N}(0, A_{g}A_{g}^{\mathsf{T}}) A g Z ∼ N ( 0 , A g A g T ) , generating the centered Gaussian covariance family
Q G c o v = { N ( 0 , A g A g T ) ∣ A g ∈ G L ( d ) } . Q_{G}^{\mathrm{cov}} = \{\mathcal{N}(0, A_{g}A_{g}^{\mathsf{T}}) \mid A_{g} \in\mathrm{GL}(d)\}. Q G cov = { N ( 0 , A g A g T ) ∣ A g ∈ GL ( d )} .
Directional von Mises–Fisher family on the sphere: On the unit sphere X = S 2 ⊂ R 3 \mathcal{X} = S^{2} \subset\mathbb{R}^{3} X = S 2 ⊂ R 3 , let G = S O ( 3 ) G = \mathrm{SO}(3) G = SO ( 3 ) act by rotations g ⋅ x = R g x g \cdot x = R_{g}x g ⋅ x = R g x . Taking a von Mises–Fisher base distribution q 0 ( x ) = c κ exp ( κ e 3 T x ) q_{0}(x) = c_{\kappa}\exp(\kappa e_{3}^{\mathsf{T}}x) q 0 ( x ) = c κ exp ( κ e 3 T x ) (κ > 0 \kappa> 0 κ > 0 ) concentrated around the polar axis e 3 = ( 0 , 0 , 1 ) T e_{3} = (0, 0, 1)^{\mathsf{T}} e 3 = ( 0 , 0 , 1 ) T (the North pole) yields the rotational family
Q G a x = { c κ exp ( κ v g T x ) ∣ v g = R g e 3 , R g ∈ S O ( 3 ) } , Q_{G}^{\mathrm{ax}} = \{c_{\kappa}\exp(\kappa v_{g}^{\mathsf{T}}x) \mid v_{g} = R_{g}e_{3},\ R_{g} \in\mathrm{SO}(3)\}, Q G ax = { c κ exp ( κ v g T x ) ∣ v g = R g e 3 , R g ∈ SO ( 3 )} ,
indexed by the transported modal direction v g = R g e 3 ∈ S 2 v_{g} = R_{g}e_{3} \in S^{2} v g = R g e 3 ∈ S 2 .
When the target distribution p p p is itself generated by transporting a centered distribution p ~ \widetilde{p} p via an unknown true transformation g ∗ ∈ G g^{*} \in G g ∗ ∈ G , a fundamental statistical question is whether minimizing L ( g ) \mathcal{L}(g) L ( g ) recovers g ∗ g^{*} g ∗ —or, at a first-order differential level, whether g ∗ g^{*} g ∗ is a stationary point of L \mathcal{L} L . Because G G G is a smooth manifold rather than a flat vector space in general, first-order stationarity is characterized through the Riemannian gradient ∇ G L ( g ∗ ) \nabla_{G}\mathcal{L}(g^{*}) ∇ G L ( g ∗ ) , which is an element of the Lie algebra g = T e G \mathfrak{g} = T_{e}G g = T e G tangent to G G G at the identity (defined formally in § 5 ). By left-trivialization, ∇ G L ( g ) \nabla_{G}\mathcal{L}(g) ∇ G L ( g ) governs the first-order variation of L \mathcal{L} L along one-parameter subgroup perturbations t ↦ g exp ( t u ) t \mapsto g\exp(tu) t ↦ g exp ( t u ) for u ∈ g u \in\mathfrak{g} u ∈ g :
L ( g ⋅ exp ( t u ) ) = L ( g ) + t ⟨ ∇ G L ( g ) , u ⟩ g + o ( t ) . \mathcal{L}(g \cdot\exp(tu)) = \mathcal{L}(g) + t\langle\nabla_{G}\mathcal{L}(g), u\rangle_{\mathfrak{g}} + o(t). L ( g ⋅ exp ( t u )) = L ( g ) + t ⟨ ∇ G L ( g ) , u ⟩ g + o ( t ) .
Thus, g ∗ g^{*} g ∗ is a first-order critical point of the variational loss if and only if ∇ G L ( g ∗ ) = 0 \nabla_{G}\mathcal{L}(g^{*}) = 0 ∇ G L ( g ∗ ) = 0 in g \mathfrak{g} g .
Remarks on pushforwards, overparametrization, and notation.
Pushforward versus pullback: In measure-theoretic terminology, the transported distribution q g = g # q 0 q_{g} = g_{\#}q_{0} q g = g # q 0 is the pushforward of q 0 q_{0} q 0 by the action of g g g . Conversely, standardizing a distribution p p p by applying the inverse transformation ( g ∗ ) − 1 (g^{*})^{-1} ( g ∗ ) − 1 defines its pullback p ~ = ( g ∗ ) # − 1 p \widetilde{p} = (g^{*})^{-1}_{\#}p p = ( g ∗ ) # − 1 p (Definition 3.2 ).
Overparametrization and coset identifiability: In several natural families—such as the Gaussian covariance family ( G = G L ( d ) ) (G = \mathrm{GL}(d)) ( G = GL ( d )) and the spherical axis family ( G = S O ( 3 ) ) (G = \mathrm{SO}(3)) ( G = SO ( 3 )) —the map g ↦ q g g \mapsto q_{g} g ↦ q g is not injective: distinct group elements g ≠ g ′ g \ne g' g = g ′ can produce the exact same distribution q g = q g ′ q_{g} = q_{g'} q g = q g ′ (for instance, whenever A g A g T = A g ′ A g ′ T A_{g}A_{g}^{\mathsf{T}} = A_{g'}A_{g'}^{\mathsf{T}} A g A g T = A g ′ A g ′ T , which occurs when A g ′ = A g U A_{g'} = A_{g}U A g ′ = A g U for U ∈ O ( d ) U \in O(d) U ∈ O ( d ) ). In group-theoretic terminology, the set of transformations n ∈ G n \in G n ∈ G that leave the base distribution unchanged (q n = q 0 q_{n} = q_{0} q n = q 0 ) forms a subgroup called the stabilizer of q 0 q_{0} q 0 , and the equivalence class of all parameters { g n : n in the stabilizer } \{g n : n \text{ in the stabilizer}\} { g n : n in the stabilizer } that generate the same distribution q g q_{g} q g is called a left coset. In such settings, g ∗ g^{*} g ∗ is identifiable only up to this stabilizer subgroup, while the induced distribution q g ∗ q_{g^{*}} q g ∗ and the coset parameter on the quotient space (§ 4 ) remain well-defined.
Notation: Where no ambiguity arises, we identify a group element g ∈ G g \in G g ∈ G directly with its concrete transformation parameter, writing μ ∈ R d \mu\in\mathbb{R}^{d} μ ∈ R d for translations, A ∈ G L ( d ) A \in\mathrm{GL}(d) A ∈ GL ( d ) for invertible matrices, or R ∈ S O ( d ) R \in\mathrm{SO}(d) R ∈ SO ( d ) for rotations.
Distribution symmetries
To understand when variational inference can recover g ∗ g^{*} g ∗ despite functional misspecification between q 0 q_{0} q 0 and the target distribution, we formalize the notion of distributional invariance under a group H H H of
Transport variational inference with symmetry
isometries of X \mathcal{X} X , and define conjugate symmetry around a transport parameter g ∗ ∈ G g^{*} \in G g ∗ ∈ G . Statistically, conjugate symmetry expresses the familiar modeling assumption that a random variable X ∼ p X \sim p X ∼ p becomes symmetric around the origin after being standardized by ( g ∗ ) − 1 (g^{*})^{-1} ( g ∗ ) − 1 , i.e., the standardized variable ξ = ( g ∗ ) − 1 ⋅ X ∼ p ~ \xi= (g^{*})^{-1} \cdot X \sim\widetilde{p} ξ = ( g ∗ ) − 1 ⋅ X ∼ p has an H H H -invariant distribution.
Definition 3.2 (H H H -symmetry and conjugate symmetry). Let H H H be a non-trivial compact group acting effectively on X \mathcal{X} X by isometries (written linearly as h ⋅ x = M h x h \cdot x = M_hx h ⋅ x = M h x with M h ∈ O ( d ) M_h \in O(d) M h ∈ O ( d ) when X ⊆ R d \mathcal{X} \subseteq\mathbb{R}^{d} X ⊆ R d ).
A distribution q ∈ P ( X ) q \in\mathcal{P}(\mathcal{X}) q ∈ P ( X ) is H H H -symmetric (or H H H -invariant) if q ( h ⋅ z ) = q ( z ) q(h \cdot z) = q(z) q ( h ⋅ z ) = q ( z ) for all h ∈ H h \in H h ∈ H and z ∈ X z \in\mathcal{X} z ∈ X .
Given a target distribution p ∈ P ( X ) p \in\mathcal{P}(\mathcal{X}) p ∈ P ( X ) and a transport element g ∗ ∈ G g^{*} \in G g ∗ ∈ G , the pullback distribution p ~ = ( g ∗ ) # − 1 p \widetilde{p} = (g^{*})^{-1}_{\#}p p = ( g ∗ ) # − 1 p is the law of the standardized variable ξ = ( g ∗ ) − 1 ⋅ X \xi= (g^{*})^{-1} \cdot X ξ = ( g ∗ ) − 1 ⋅ X when X ∼ p X \sim p X ∼ p , with density
p ~ ( ξ ) = p ( g ∗ ⋅ ξ ) ∣ det J g ∗ ( ξ ) ∣ . \widetilde{p}(\xi) = p(g^{*} \cdot\xi) \left\lvert\det J_{g^{*}}(\xi) \right\rvert. p ( ξ ) = p ( g ∗ ⋅ ξ ) ∣ det J g ∗ ( ξ ) ∣ . A distribution p ∈ P ( X ) p \in\mathcal{P}(\mathcal{X}) p ∈ P ( X ) has H H H -conjugate symmetry around g ∗ ∈ G g^{*} \in G g ∗ ∈ G if its pullback p ~ \widetilde{p} p under g ∗ g^{*} g ∗ is H H H -symmetric, meaning that for all h ∈ H h \in H h ∈ H and ξ ∈ X \xi\in\mathcal{X} ξ ∈ X ,
p ~ ( h ⋅ ξ ) = p ~ ( ξ ) . \widetilde{p}(h \cdot\xi) = \widetilde{p}(\xi). p ( h ⋅ ξ ) = p ( ξ ) .
By construction, the pullback operation inverts the transport pushforward: if X ∼ p X \sim p X ∼ p , then ( g ∗ ) − 1 ⋅ X ∼ p ~ (g^{*})^{-1} \cdot X \sim\widetilde{p} ( g ∗ ) − 1 ⋅ X ∼ p , and conversely if ξ ∼ p ~ \xi\sim\widetilde{p} ξ ∼ p , then g ∗ ⋅ ξ ∼ p g^{*} \cdot\xi\sim p g ∗ ⋅ ξ ∼ p . Thus, p p p has H H H -conjugate symmetry around g ∗ g^{*} g ∗ precisely when p p p is obtained by transporting an H H H -symmetric reference distribution p ~ \widetilde{p} p by g ∗ g^{*} g ∗ .
Examples of H H H -symmetry and conjugate symmetry.
Central reflection symmetry around a location parameter: Let X = R d \mathcal{X} = \mathbb{R}^{d} X = R d , let H = Z / 2 Z H = \mathbb{Z}/2\mathbb{Z} H = Z /2 Z act by central reflection { I d , − I d } \{I_d, -I_d\} { I d , − I d } , and let G = ( R d , + ) G = (\mathbb{R}^{d}, +) G = ( R d , + ) act by translation x ↦ x + μ x \mapsto x + \mu x ↦ x + μ . A distribution q ∈ P ( R d ) q \in\mathcal{P}(\mathbb{R}^{d}) q ∈ P ( R d ) is H H H -symmetric if and only if it is even, q ( − z ) = q ( z ) q(-z) = q(z) q ( − z ) = q ( z ) . For a target distribution p ∈ P ( R d ) p \in\mathcal{P}(\mathbb{R}^{d}) p ∈ P ( R d ) and a location parameter μ ∗ ∈ R d \mu^{*} \in\mathbb{R}^{d} μ ∗ ∈ R d , the pullback is the centered density p ~ ( ξ ) = p ( μ ∗ + ξ ) \widetilde{p}(\xi) = p(\mu^{*} + \xi) p ( ξ ) = p ( μ ∗ + ξ ) , corresponding to the centered random variable ξ = X − μ ∗ \xi= X - \mu^{*} ξ = X − μ ∗ . Therefore, p p p has H H H -conjugate symmetry around μ ∗ \mu^{*} μ ∗ if and only if p ~ ( ξ ) = p ~ ( − ξ ) \widetilde{p}(\xi) = \widetilde{p}(-\xi) p ( ξ ) = p ( − ξ ) , or equivalently p ( μ ∗ + ξ ) = p ( μ ∗ − ξ ) p(\mu^{*} + \xi) = p(\mu^{*} - \xi) p ( μ ∗ + ξ ) = p ( μ ∗ − ξ ) for all ξ ∈ R d \xi\in\mathbb{R}^{d} ξ ∈ R d . This recovers the classical setting of location estimation for centrally symmetric distributions studied in Margossian and Saul (2025).
Canonical isometry groups in statistics: Beyond central reflection Z / 2 Z \mathbb{Z}/2\mathbb{Z} Z /2 Z , several compact isometry groups H H H arise naturally in statistical modeling: the full orthogonal group O ( d ) O(d) O ( d ) on R d \mathbb{R}^{d} R d (characterizing spherical and radial distributions, or elliptical distributions after affine standardization); the symmetric group S n \mathbb{S}_n S n acting on R n \mathbb{R}^{n} R n by coordinate permutation (characterizing exchangeable random vectors); the dihedral groups D k D_k D k on R 2 \mathbb{R}^{2} R 2 (characterizing discrete rotational and reflectional symmetries, such as ℓ m \ell_m ℓ m -norm contours); and the planar rotation group S O ( 2 ) ⊂ S O ( 3 ) SO(2) \subset SO(3) S O ( 2 ) ⊂ S O ( 3 ) fixing a polar axis e 3 e_3 e 3 on the sphere S 2 \mathbb{S}^{2} S 2 (characterizing axially symmetric directional distributions).
Remark (Geometric meaning of conjugate symmetry). Written directly in terms of the target density p p p without reference to the pullback p ~ \widetilde{p} p , H H H -conjugate symmetry around g ∗ g^{*} g ∗ states that for every h ∈ H h \in H h ∈ H and x ∈ X x \in\mathcal{X} x ∈ X ,
p ( ( g ∗ ∘ h ∘ ( g ∗ ) − 1 ) ⋅ x ) ∣ det J g ∗ ∘ h ∘ ( g ∗ ) − 1 ( x ) ∣ = p ( x ) . p\left((g^{*} \circ h \circ(g^{*})^{-1}) \cdot x\right) \left\lvert\det J_{g^{*}\circ h\circ(g^{*})^{-1}}(x) \right\rvert= p(x). p ( ( g ∗ ∘ h ∘ ( g ∗ ) − 1 ) ⋅ x ) det J g ∗ ∘ h ∘ ( g ∗ ) − 1 ( x ) = p ( x ) .
That is, while the standardized distribution p ~ \widetilde{p} p is invariant under the base symmetry group H H H centered at the origin, transporting p ~ \widetilde{p} p by g ∗ g^{*} g ∗ shifts its symmetries to the conjugated subgroup g ∗ H ( g ∗ ) − 1 g^{*}H(g^{*})^{-1} g ∗ H ( g ∗ ) − 1 . Consequently, specifying a symmetric variational inference problem involves an inherently coupled modeling choice between the symmetry group H H H of the standardized data and the transport group G G G used to relocate the distribution. We formalize this coupling next.
Transport variational inference with symmetry
We now combine the transport family of Definition 3.1 with the symmetry structure of Definition 3.2 . Suppose that the variational base distribution q 0 q_0 q 0 is H H H -symmetric, and that the target distribution p p p has H H H -conjugate symmetry around an unknown true parameter g ∗ ∈ G g^* \in G g ∗ ∈ G . Even when the functional form of q 0 q_0 q 0 differs substantially from that of the pullback p ~ \widetilde{p} p (for instance, when q 0 q_0 q 0 is Gaussian while p ~ \widetilde{p} p is heavy-tailed or multimodal), both distributions share the same invariance group H H H . Our central question is whether this shared symmetry suffices for variational inference to identify g ∗ g^* g ∗ , and specifically whether ∇ G L ( g ∗ ) = 0 \nabla_G \mathcal{L}(g^*) = 0 ∇ G L ( g ∗ ) = 0 .
As Margossian and Saul (2025) analyzed two special instances of this problem (corresponding to Examples 1 and 4 below), showing that the variational gradient vanishes at the true location parameter under reflection symmetry, but exhibits a nonzero radial scaling bias when estimating a covariance matrix under elliptical symmetry. Our goal is to establish a general theory covering arbitrary pairs ( G , H ) (G,H) ( G , H ) that explains both phenomena and characterizes ∇ G L ( g ∗ ) \nabla_G \mathcal{L}(g^*) ∇ G L ( g ∗ ) completely.
Definition 3.3 (Transport variational inference with symmetry). In the setting of Definition 3.1 , let H H H be a non-trivial compact group acting effectively on X \mathcal{X} X by isometries (the symmetry group ). Assume that:
(i) the variational base distribution q 0 ∈ P ( X ) q_0 \in\mathcal{P}(\mathcal{X}) q 0 ∈ P ( X ) is H H H -symmetric (with stabilizer Stab G ( q 0 ) = G ∩ H \operatorname{Stab}_G(q_0) = G \cap H Stab G ( q 0 ) = G ∩ H in G G G );
(ii) the target distribution p ∈ P ( X ) p \in\mathcal{P}(\mathcal{X}) p ∈ P ( X ) has H H H -conjugate symmetry around a true parameter g ∗ ∈ G g^* \in G g ∗ ∈ G ;
and
(iii) the transport group G G G is normalized by H H H within Aut ( X ) \operatorname{Aut}(\mathcal{X}) Aut ( X ) , meaning that Φ h ( g ) : = h ∘ g ∘ h − 1 ∈ G \Phi_h(g) := h \circ g \circ h^{-1} \in G Φ h ( g ) := h ∘ g ∘ h − 1 ∈ G for all g ∈ G g \in G g ∈ G and $h \in H—in plain terms, expressing any parameter transformation g ∈ G g \in G g ∈ G in a coordinate system rotated or reflected by h ∈ H h \in H h ∈ H yields another valid parameter transformation inside G G G .
Symmetric transport variational inference then consists in solving the optimization problem
min g ∈ G L ( g ) = min g ∈ G KL ( q g ∥ p ) . \min_{g \in G} \mathcal{L}(g) = \min_{g \in G} \operatorname{KL}(q_g \mathbin{\|} p). g ∈ G min L ( g ) = g ∈ G min KL ( q g ∥ p ) .
Remark (Why G G G must be normalized by H H H ). Condition (iii) in Definition 3.3 requires that conjugating any transport g ∈ G g \in G g ∈ G by any symmetry h ∈ H h \in H h ∈ H produces a transformation Φ h ( g ) = h ∘ g ∘ h − 1 \Phi_h(g) = h \circ g \circ h^{-1} Φ h ( g ) = h ∘ g ∘ h − 1 that still belongs to the variational transport group G G G . This structural consistency condition has a clear geometric and statistical motivation.
Consider perturbing the variational parameter away from g ∗ g^* g ∗ by a relative transport g ∈ G g \in G g ∈ G , evaluating L ( g ∗ g ) = KL ( q g ∥ p ~ ) \mathcal{L}(g^*g) = \operatorname{KL}(q_g \mathbin{\|} \widetilde{p}) L ( g ∗ g ) = KL ( q g ∥ p ) . Because both q 0 q_0 q 0 and p ~ \widetilde{p} p are H H H -invariant, rotating the perturbation by any symmetry h ∈ H h \in H h ∈ H produces a symmetrically equivalent perturbation h ∘ g ∘ h − 1 h \circ g \circ h^{-1} h ∘ g ∘ h − 1 that achieves the exact same KL divergence against p ~ \widetilde{p} p . However, for this symmetry of the objective to constrain the gradient within our parameter space G G G , the rotated perturbation h ∘ g ∘ h − 1 h \circ g \circ h^{-1} h ∘ g ∘ h − 1 must actually lie in G G G . If h ∘ g ∘ h − 1 ∉ G h \circ g \circ h^{-1} \notin G h ∘ g ∘ h − 1 ∈ / G , the variational family truncates the symmetry orbit of the perturbation. For example, if p ~ \widetilde{p} p is rotationally invariant in the plane (H = SO ( 2 ) H = \operatorname{SO}(2) H = SO ( 2 ) ) but the variational family only allows horizontal translations along the x 1 x_1 x 1 -axis, rotating a horizontal translation by 90 ∘ 90^\circ 9 0 ∘ produces a vertical translation along the x 2 x_2 x 2 -axis, which lies outside G G G . Requiring G G G to be normalized by H H H ensures that the variational parameter space is closed under the symmetry transformations we seek to exploit, and guarantees that the quotient parameter space M = G / ( G ∩ H ) \mathcal{M} = G/(G \cap H) M = G / ( G ∩ H ) (§ 4 ) and the adjoint representation on the Lie algebra g \mathfrak{g} g (§ 5 ) are well-defined.
Guiding examples. Throughout the paper, we trace six concrete examples that span location estimation, directional statistics, covariance estimation, non-Gaussian orientation estimation, and exchangeable models. These examples allow every algebraic object in our theory to be interpreted directly in familiar statistical terms:
Reflection (Univariate symmetric location model): On the real line X = R \mathcal{X} = \mathbb{R} X = R , let H = Z / 2 Z = { 1 , − 1 } H = \mathbb{Z}/2\mathbb{Z} = \{1,-1\} H = Z /2 Z = { 1 , − 1 } act by sign reflection x ↦ ± x x \mapsto\pm x x ↦ ± x , and let G = ( R , + ) G = (\mathbb{R}, +) G = ( R , + ) act by translation x ↦ x + μ x \mapsto x + \mu x ↦ x + μ . Choosing the standard normal base q 0 = N ( 0 , 1 ) q_0 = \mathcal{N}(0,1) q 0 = N ( 0 , 1 ) generates the unit-variance Gaussian location family Q R = { N ( μ , 1 ) ∣ μ ∈ R } Q_{\mathbb{R}} = \{\mathcal{N}(\mu,1) \mid\mu\in\mathbb{R}\} Q R = { N ( μ , 1 ) ∣ μ ∈ R } . Any target distribution p p p that is symmetric around a true location μ ∗ ∈ R \mu^* \in\mathbb{R} μ ∗ ∈ R —such as a symmetric bimodal Gaussian mixture p = 1 2 N ( μ ∗ + δ , 1 ) + 1 2 N ( μ ∗ − δ , 1 ) p = \frac{1}{2}\mathcal{N}(\mu^*+\delta,1) + \frac{1}{2}\mathcal{N}(\mu^*-\delta,1) p = 2 1 N ( μ ∗ + δ , 1 ) + 2 1 N ( μ ∗ − δ , 1 ) , a Student-t t t distribution centered at μ ∗ \mu^* μ ∗ , or a Laplace distribution—has H H H -conjugate symmetry around μ ∗ \mu^* μ ∗ even though p ∉ Q R p \notin Q_{\mathbb{R}} p ∈ / Q R .
Euclidean (Multivariate radially symmetric location model): On X = R d \mathcal{X} = \mathbb{R}^d X = R d (d ≥ 2 d \ge2 d ≥ 2 ), let H = O ( d ) H = O(d) H = O ( d ) be the orthogonal group of rotations and reflections around the origin, and let G = ( R d , + ) G = (\mathbb{R}^d,+) G = ( R d , + ) act by translation x ↦ x + μ x \mapsto x+\mu x ↦ x + μ . With q 0 = N ( 0 , I d ) q_0 = \mathcal{N}(0,I_d) q 0 = N ( 0 , I d ) , the variational family is Q R d = { N ( μ , I d ) ∣ μ ∈ R d } Q_{\mathbb{R}^d} = \{\mathcal{N}(\mu,I_d) \mid\mu\in\mathbb{R}^d\} Q R d = { N ( μ , I d ) ∣ μ ∈ R d } , while the target p ( x ) = p ~ ( x − μ ∗ ) p(x) = \tilde{p}(x-\mu^*) p ( x ) = p ~ ( x − μ ∗ ) can be any radially symmetric distribution p ~ ( ξ ) = f ( ∥ ξ ∥ 2 ) \tilde{p}(\xi) = f(\|\xi\|_2) p ~ ( ξ ) = f ( ∥ ξ ∥ 2 ) shifted to center μ ∗ ∈ R d \mu^* \in\mathbb{R}^d μ ∗ ∈ R d .
Sphere (Directional estimation for axially symmetric distributions): On the unit sphere X = S 2 ⊂ R 3 \mathcal{X} = S^2 \subset\mathbb{R}^3 X = S 2 ⊂ R 3 , let H = S O ( 2 ) H = \mathrm{SO}(2) H = SO ( 2 ) be the subgroup of S O ( 3 ) \mathrm{SO}(3) SO ( 3 ) consisting of rotations around the polar axis e 3 = ( 0 , 0 , 1 ) T e_3 = (0,0,1)^\mathsf{T} e 3 = ( 0 , 0 , 1 ) T (the North pole), and let G = S O ( 3 ) G = \mathrm{SO}(3) G = SO ( 3 ) act by 3D rotations x ↦ R x x \mapsto Rx x ↦ R x . Using a von Mises–Fisher base distribution q 0 ( x ) = c κ exp ( κ e 3 T x ) q_0(x) = c_\kappa\exp(\kappa e_3^\mathsf{T}x) q 0 ( x ) = c κ exp ( κ e 3 T x ) (κ > 0 \kappa> 0 κ > 0 ) produces the variational family Q S O ( 3 ) = { q R ( x ) = c κ exp ( κ ( R e 3 ) T x ) ∣ R ∈ S O ( 3 ) } Q_{\mathrm{SO}(3)} = \{q_R(x) = c_\kappa\exp(\kappa(Re_3)^\mathsf{T}x) \mid R \in\mathrm{SO}(3)\} Q SO ( 3 ) = { q R ( x ) = c κ exp ( κ ( R e 3 ) T x ) ∣ R ∈ SO ( 3 )} , which is effectively indexed by the modal direction v = R e 3 ∈ S 2 v = Re_3 \in S^2 v = R e 3 ∈ S 2 . The target p ( x ) = p ~ ( ( R ∗ ) T x ) p(x) = \tilde{p}((R^*)^\mathsf{T}x) p ( x ) = p ~ (( R ∗ ) T x ) is obtained by rotating any rotationally symmetric spherical distribution p ~ \tilde{p} p ~ around e 3 e_3 e 3 so that its axis of symmetry aligns with v ∗ = R ∗ e 3 v^* = R^*e_3 v ∗ = R ∗ e 3 .
Covariance (Elliptical scatter and covariance estimation): On X = R d \mathcal{X} = \mathbb{R}^d X = R d (d ≥ 2 d \ge2 d ≥ 2 ), let H = O ( d ) H = O(d) H = O ( d ) act by orthogonal transformations and let G = G L ( d ) G = \mathrm{GL}(d) G = GL ( d ) act linearly by x ↦ A x x \mapsto Ax x ↦ A x . Taking q 0 = N ( 0 , I d ) q_0 = \mathcal{N}(0,I_d) q 0 = N ( 0 , I d ) yields the zero-mean Gaussian family Q G L ( d ) c o v = { N ( 0 , A A T ) ∣ A ∈ G L ( d ) } Q_{\mathrm{GL}(d)}^{\mathrm{cov}} = \{\mathcal{N}(0,AA^\mathsf{T}) \mid A \in\mathrm{GL}(d)\} Q GL ( d ) cov = { N ( 0 , A A T ) ∣ A ∈ GL ( d )} , which parameterizes all symmetric positive-definite covariance matrices Σ = A A T ∈ S P D ( d ) \Sigma= AA^\mathsf{T} \in\mathrm{SPD}(d) Σ = A A T ∈ SPD ( d ) . If the target p ( x ) = ∣ det A ∗ ∣ − 1 p ~ ( ( A ∗ ) − 1 x ) p(x) = |\det A^*|^{-1}\tilde{p}((A^*)^{-1}x) p ( x ) = ∣ det A ∗ ∣ − 1 p ~ (( A ∗ ) − 1 x ) is generated by applying an invertible linear transformation A ∗ A^* A ∗ to any spherically symmetric distribution p ~ \tilde{p} p ~ (such as a multivariate t t t or Kotz-type elliptical distribution), then p p p is an elliptical distribution with scatter structure Σ ∗ = A ∗ ( A ∗ ) T \Sigma^* = A^*(A^*)^\mathsf{T} Σ ∗ = A ∗ ( A ∗ ) T and possesses O ( d ) O(d) O ( d ) -conjugate symmetry around A ∗ A^* A ∗ .
Square (Orientation estimation under discrete dihedral symmetry): On the plane X = R 2 \mathcal{X} = \mathbb{R}^2 X = R 2 , let H = D 4 H = D_4 H = D 4 be the dihedral symmetry group of the square (generated by 90 ∘ 90^\circ 9 0 ∘ rotations and coordinate-axis reflections), and let G = S O ( 2 ) G = \mathrm{SO}(2) G = SO ( 2 ) act by continuous planar rotations R θ R_\theta R θ . A natural D 4 D_4 D 4 -invariant base distribution is the generalized Gaussian q 0 ( x ) ∝ exp ( − γ m ∥ x ∥ m ) q_0(x) \propto\exp(-\gamma_m\|x\|^m) q 0 ( x ) ∝ exp ( − γ m ∥ x ∥ m ) with m > 2 m > 2 m > 2 (for example, m = 4 m = 4 m = 4 ), whose level sets are ℓ m \ell_m ℓ m -balls with fourfold corners. Rotating q 0 q_0 q 0 by R θ ∈ S O ( 2 ) R_\theta\in\mathrm{SO}(2) R θ ∈ SO ( 2 ) generates the orientation family Q S O ( 2 ) s q = { q 0 ( R θ T x ) ∣ R θ ∈ S O ( 2 ) } Q_{\mathrm{SO}(2)}^{\mathrm{sq}} = \{q_0(R_\theta^\mathsf{T}x) \mid R_\theta\in\mathrm{SO}(2)\} Q SO ( 2 ) sq = { q 0 ( R θ T x ) ∣ R θ ∈ SO ( 2 )} modeling distributions whose square-symmetric contours are tilted by an angle θ \theta θ , against a target p ( x ) = p ~ ( R θ ∗ T x ) p(x) = \tilde{p}(R_{\theta^*}^\mathsf{T}x) p ( x ) = p ~ ( R θ ∗ T x ) tilted by a true angle θ ∗ \theta^* θ ∗ .
Permutation (Location estimation for exchangeable coordinates): On X = R n \mathcal{X} = \mathbb{R}^n X = R n , let H = S n H = S_n H = S n be the symmetric group of coordinate permutations ( x 1 , … , x n ) ↦ ( x σ ( 1 ) , … , x σ ( n ) ) (x_1,\ldots,x_n) \mapsto(x_{\sigma(1)},\ldots,x_{\sigma(n)}) ( x 1 , … , x n ) ↦ ( x σ ( 1 ) , … , x σ ( n ) ) , which characterizes exchangeable random variables, and let G = ( R n , + ) G = (\mathbb{R}^n,+) G = ( R n , + ) act by coordinate-wise translation x ↦ x + μ x \mapsto x+\mu x ↦ x + μ . For instance, taking an exchangeable product Laplace base q 0 ( x ) = c κ exp ( − κ ∥ x ∥ 1 ) q_0(x) = c_\kappa\exp(-\kappa\|x\|_1) q 0 ( x ) = c κ exp ( − κ ∥ x ∥ 1 ) yields the location family Q R n p e r m = { c κ exp ( − κ ∥ x − μ ∥ 1 ) ∣ μ ∈ R n } Q_{\mathbb{R}^n}^{\mathrm{perm}} = \{c_\kappa\exp(-\kappa\|x-\mu\|_1) \mid\mu\in\mathbb{R}^n\} Q R n perm = { c κ exp ( − κ ∥ x − μ ∥ 1 ) ∣ μ ∈ R n } , while the target p ( x ) = p ~ ( x − μ ∗ ) p(x) = \tilde{p}(x-\mu^*) p ( x ) = p ~ ( x − μ ∗ ) is obtained by shifting any exchangeable distribution p ~ \tilde{p} p ~ (potentially with dependent coordinates, such as an equicorrelated normal or exchangeable mixture) by a location vector μ ∗ ∈ R n \mu^* \in\mathbb{R}^n μ ∗ ∈ R n .
Table 1 summarizes the sample spaces, transport groups, and symmetry groups for these six guiding examples.
Table 1. Summary of sample spaces, transport groups, and symmetry groups for the six guiding examples.
Group interactions and identifiability
Before analyzing first-order stationarity of the variational loss L ( g ) = KL ( q g ∥ p ) \mathcal{L}(g) = \operatorname{KL}(q_{g} \mathbin\Vert p) L ( g ) = KL ( q g ∥ p ) , we must address a foundational statistical question: which parameters are identifiable from the variational family Q G = { q g : g ∈ G } Q_{G} = \{q_{g} : g \in G\} Q G = { q g : g ∈ G } ? Because the base distribution q 0 q_{0} q 0 is invariant under the symmetry group H H H , any transport n ∈ G n \in G n ∈ G that also belongs to H H H leaves q 0 q_{0} q 0 unchanged (q n = q 0 q_{n} = q_{0} q n = q 0 ), and more generally right-multiplying any g ∈ G g \in G g ∈ G by n ∈ G ∩ H n \in G \cap H n ∈ G ∩ H produces the identical distribution q g n = q g q_{gn} = q_{g} q g n = q g . Furthermore, even when h ∈ H h \in H h ∈ H does not belong to G G G , composing transports in G G G with symmetries in H H H on the sample space X \mathcal{X} X generates a larger ambient group of transformations. In this section, we formalize this ambient group K = G H K = GH K = G H , analyze the conjugation action Φ h ( g ) = h ∘ g ∘ h − 1 \Phi_{h}(g) = h \circ g \circ h^{-1} Φ h ( g ) = h ∘ g ∘ h − 1 across our six guiding examples, and classify the resulting statistical identifiability regimes on the quotient manifold M = G / ( G ∩ H ) \mathcal{M} = G/(G \cap H) M = G / ( G ∩ H ) .
Total transformation group and semidirect product structure
When G G G and H H H both act on the sample space X \mathcal{X} X , their compositions generate a combined group of transformations inside Aut ( X ) \operatorname{Aut}(\mathcal{X}) Aut ( X ) . The algebraic structure of this total group is governed by the normalization condition Φ h ( g ) = h ∘ g ∘ h − 1 ∈ G \Phi_{h}(g) = h \circ g \circ h^{-1} \in G Φ h ( g ) = h ∘ g ∘ h − 1 ∈ G introduced in Definition 3.3 .
Definition 4.1 (Total transformation group). Let K = G H ⊆ Aut ( X ) K = GH \subseteq\operatorname{Aut}(\mathcal{X}) K = G H ⊆ Aut ( X ) denote the total group of transformations on X \mathcal{X} X generated by composing actions of G G G and H H H . Every element k ∈ K k \in K k ∈ K can be written as k = g ∘ h k = g \circ h k = g ∘ h for some g ∈ G g \in G g ∈ G and h ∈ H h \in H h ∈ H , acting on x ∈ X x \in\mathcal{X} x ∈ X by applying the symmetry h h h first and the transport g g g second:
k ⋅ x = g ⋅ ( h ⋅ x ) . k \cdot x = g \cdot(h \cdot x). k ⋅ x = g ⋅ ( h ⋅ x ) . Because G G G is normalized by H H H , the composition of any two elements k 2 = g 2 ∘ h 2 k_{2} = g_{2} \circ h_{2} k 2 = g 2 ∘ h 2 and k 1 = g 1 ∘ h 1 k_{1} = g_{1} \circ h_{1} k 1 = g 1 ∘ h 1 in K K K remains in G H GH G H and obeys the semidirect multiplication law
( g 2 ∘ h 2 ) ∘ ( g 1 ∘ h 1 ) = ( g 2 ∘ Φ h 2 ( g 1 ) ) ∘ ( h 2 ∘ h 1 ) , (g_{2} \circ h_{2}) \circ(g_{1} \circ h_{1}) = (g_{2} \circ\Phi_{h_{2}}(g_{1})) \circ(h_{2} \circ h_{1}), ( g 2 ∘ h 2 ) ∘ ( g 1 ∘ h 1 ) = ( g 2 ∘ Φ h 2 ( g 1 )) ∘ ( h 2 ∘ h 1 ) , where Φ h ( g ) = h ∘ g ∘ h − 1 ∈ G \Phi_{h}(g) = h \circ g \circ h^{-1} \in G Φ h ( g ) = h ∘ g ∘ h − 1 ∈ G denotes the conjugation of g g g by h h h . Depending on the intersection subgroup N = G ∩ H N = G \cap H N = G ∩ H , the total group K K K takes one of three algebraic forms:
When N = G ∩ H = { e } N = G \cap H = \{e\} N = G ∩ H = { e } (disjoint actions), K K K is isomorphic to the internal semidirect product G ⋊ Φ H G \rtimes_{\Phi} H G ⋊ Φ H : every transformation k ∈ K k \in K k ∈ K decomposes uniquely into a symmetry h ∈ H h \in H h ∈ H followed by a transport g ∈ G g \in G g ∈ G , and composing two such transformations twists the first transport parameter g 1 g_{1} g 1 by h 2 h_{2} h 2 via Φ h 2 ( g 1 ) \Phi_{h_{2}}(g_{1}) Φ h 2 ( g 1 ) (just as rotating an affine-shifted variable rotates its translation vector). For location families where G = ( R d , + ) G = (\mathbb{R}^{d}, +) G = ( R d , + ) acts by translation and H H H acts linearly (Examples 1, 2, and 6), K = R d ⋊ H K = \mathbb{R}^{d} \rtimes H K = R d ⋊ H is the group of affine isometries generated by H H H and translations (in particular, the full Euclidean group E ( d ) = R d ⋊ O ( d ) E(d) = \mathbb{R}^{d} \rtimes O(d) E ( d ) = R d ⋊ O ( d ) in Example 2).
(ii) When H ⊆ G H \subseteq G H ⊆ G , so that N = H N = H N = H (internal symmetries), every symmetry is already a transport transformation, and the total group collapses to K = G K = G K = G (as in Example 3 where K = S O ( 3 ) K = \mathrm{SO}(3) K = SO ( 3 ) and Example 4 where K = G L ( d ) K = \mathrm{GL}(d) K = GL ( d ) ).
(iii) In the general overlapping case where { e } ⊊ N ⊊ H \{e\} \subsetneq N \subsetneq H { e } ⊊ N ⊊ H , K K K is isomorphic to the quotient ( G ⋊ Φ H ) / N (G \rtimes_{\Phi} H)/N ( G ⋊ Φ H ) / N , where N N N is embedded anti-diagonally via n ↦ ( n , n − 1 ) n \mapsto(n,n^{-1}) n ↦ ( n , n − 1 ) (as in Example 5, where G = S O ( 2 ) G = \mathrm{SO}(2) G = SO ( 2 ) and H = D 4 H = D_4 H = D 4 intersect in N = C 4 N = C_4 N = C 4 and together generate the full planar orthogonal group K = O ( 2 ) K = \mathrm{O}(2) K = O ( 2 ) ).
Remark (Closure under conjugation and well-posedness of K K K ). Definition 4.1 underscores why the normalization requirement Φ h ( g ) ∈ G \Phi_h(g) \in G Φ h ( g ) ∈ G from § 3 is algebraically indispensable. Rewriting the composition h 2 ∘ g 1 h_2 \circ g_1 h 2 ∘ g 1 as ( h 2 ∘ g 1 ∘ h 2 − 1 ) ∘ h 2 = Φ h 2 ( g 1 ) ∘ h 2 (h_2 \circ g_1 \circ h_2^{-1}) \circ h_2 = \Phi_{h_2}(g_1) \circ h_2 ( h 2 ∘ g 1 ∘ h 2 − 1 ) ∘ h 2 = Φ h 2 ( g 1 ) ∘ h 2 allows any sequence of alternating transports and symmetries to be re-ordered into a single symmetry followed by a single transport in G G G . If G G G were not normalized by H H H , composing transports with symmetries would generate transformations outside G H GH G H , and differentiating the conjugation map at the identity (§ 5 ) would produce tangent vectors pointing outside the Lie algebra g \mathfrak{g} g of the variational family.
The conjugation action in detail
The conjugation map Φ : H → Aut ( G ) \Phi: H \to\operatorname{Aut}(G) Φ : H → Aut ( G ) , h ↦ Φ h h \mapsto\Phi_h h ↦ Φ h , where Φ h ( g ) = h ∘ g ∘ h − 1 \Phi_h(g) = h \circ g \circ h^{-1} Φ h ( g ) = h ∘ g ∘ h − 1 , is the fundamental algebraic bridge connecting the symmetry group H H H to the transport group G G G . Before passing to the tangent space in § 5 , we establish the basic structural properties of Φ \Phi Φ , examine its geometric meaning on X \mathcal{X} X , and compute its action, kernel, and fixed-point subgroup across all six guiding examples.
Proposition 4.2 (Conjugation as a group homomorphism). The map Φ : H → Aut ( G ) \Phi: H \to\operatorname{Aut}(G) Φ : H → Aut ( G ) , h ↦ Φ h h \mapsto\Phi_h h ↦ Φ h , is a group homomorphism : Φ h 1 h 2 = Φ h 1 ∘ Φ h 2 \Phi_{h_1h_2} = \Phi_{h_1} \circ\Phi_{h_2} Φ h 1 h 2 = Φ h 1 ∘ Φ h 2 for all h 1 , h 2 ∈ H h_1,h_2 \in H h 1 , h 2 ∈ H . Moreover, for each h ∈ H h \in H h ∈ H , the map Φ h : G → G \Phi_h : G \to G Φ h : G → G is a Lie group automorphism, satisfying Φ h ( g 1 g 2 ) = Φ h ( g 1 ) Φ h ( g 2 ) \Phi_h(g_1g_2) = \Phi_h(g_1)\Phi_h(g_2) Φ h ( g 1 g 2 ) = Φ h ( g 1 ) Φ h ( g 2 ) and Φ h ( e G ) = e G \Phi_h(e_G) = e_G Φ h ( e G ) = e G .
Proof. For any h 1 , h 2 ∈ H h_1,h_2 \in H h 1 , h 2 ∈ H and g ∈ G g \in G g ∈ G , associativity of composition in Aut ( X ) \operatorname{Aut}(\mathcal{X}) Aut ( X ) yields
Φ h 1 h 2 ( g ) = ( h 1 h 2 ) ∘ g ∘ ( h 1 h 2 ) − 1 = h 1 ∘ ( h 2 ∘ g ∘ h 2 − 1 ) ∘ h 1 − 1 = Φ h 1 ( Φ h 2 ( g ) ) . \Phi_{h_1h_2}(g) = (h_1h_2) \circ g \circ(h_1h_2)^{-1} = h_1 \circ(h_2 \circ g \circ h_2^{-1}) \circ h_1^{-1} = \Phi_{h_1}\bigl(\Phi_{h_2}(g)\bigr). Φ h 1 h 2 ( g ) = ( h 1 h 2 ) ∘ g ∘ ( h 1 h 2 ) − 1 = h 1 ∘ ( h 2 ∘ g ∘ h 2 − 1 ) ∘ h 1 − 1 = Φ h 1 ( Φ h 2 ( g ) ) .
Similarly, for any g 1 , g 2 ∈ G g_1,g_2 \in G g 1 , g 2 ∈ G , inserting h − 1 ∘ h = id X h^{-1} \circ h = \operatorname{id}_{\mathcal{X}} h − 1 ∘ h = id X gives
Φ h ( g 1 g 2 ) = h ∘ g 1 ∘ ( h − 1 ∘ h ) ∘ g 2 ∘ h − 1 = Φ h ( g 1 ) Φ h ( g 2 ) , \Phi_h(g_1g_2) = h \circ g_1 \circ(h^{-1} \circ h) \circ g_2 \circ h^{-1} = \Phi_h(g_1)\Phi_h(g_2), Φ h ( g 1 g 2 ) = h ∘ g 1 ∘ ( h − 1 ∘ h ) ∘ g 2 ∘ h − 1 = Φ h ( g 1 ) Φ h ( g 2 ) ,
and Φ h ( e G ) = h ∘ e G ∘ h − 1 = e G \Phi_h(e_G) = h \circ e_G \circ h^{-1} = e_G Φ h ( e G ) = h ∘ e G ∘ h − 1 = e G . Smoothness of Φ h \Phi_h Φ h follows from the smooth group structure of K K K . □ \square □
Two subgroups associated with the conjugation action Φ \Phi Φ play a recurring role in our analysis:
ker ( Φ ) = { h ∈ H : Φ h ( g ) = g ∀ g ∈ G } = C H ( G ) , \operatorname{ker}(\Phi) = \{h \in H : \Phi_h(g) = g\ \forall g \in G\} = C_H(G), ker ( Φ ) = { h ∈ H : Φ h ( g ) = g ∀ g ∈ G } = C H ( G ) ,
which consists of all symmetries in H H H that commute with every transport in G G G (meaning that transforming the coordinate system by h h h leaves every parameter g ∈ G g \in G g ∈ G unchanged). When ker ( Φ ) = { e H } \operatorname{ker}(\Phi) = \{e_H\} ker ( Φ ) = { e H } , the conjugation action of H H H on G G G is faithful: every non-trivial symmetry induces a non-trivial reparametrization of the transport group.
G H = { g ∈ G : Φ h ( g ) = g ∀ h ∈ H } , G^H = \{g \in G : \Phi_h(g) = g\ \forall h \in H\}, G H = { g ∈ G : Φ h ( g ) = g ∀ h ∈ H } ,
which consists of all transports in G G G that commute with every symmetry in H H H . Because each Φ h \Phi_h Φ h is a group automorphism fixing e G e_G e G , G H G^H G H is a closed Lie subgroup of G G G . Statistically, G H G^H G H consists of those parameter transformations (such as global isotropic scaling) that do not break the H H H -symmetry of the model: if q 0 q_0 q 0 is H H H -symmetric and g ∈ G H g \in G^H g ∈ G H , then q g q_g q g remains H H H -symmetric.
Geometric interpretation. Conjugation describes how a change of reference frame by a symmetry h ∈ H h \in H h ∈ H re-expresses a transport transformation g ∈ G g \in G g ∈ G . Suppose we wish to apply a transport g g g in a coordinate system that has been transformed by h h h . Moving from the original coordinates to the h h h -transformed coordinates applies h − 1 h^{-1} h − 1 ; performing the transport in those coordinates applies g g g ; and returning to the original frame applies h h h . The resulting transformation on x ∈ X x \in\mathcal{X} x ∈ X is ( h ∘ g ∘ h − 1 ) ⋅ x = Φ h ( g ) ⋅ x (h \circ g \circ h^{-1}) \cdot x = \Phi_h(g) \cdot x ( h ∘ g ∘ h − 1 ) ⋅ x = Φ h ( g ) ⋅ x . Equivalently, for every x ∈ X x \in\mathcal{X} x ∈ X , g ∈ G g \in G g ∈ G , and h ∈ H h \in H h ∈ H , the following diagram commutes:
x → g g ⋅ x h ↓ ↓ h h ⋅ x → Φ h ( g ) h ⋅ ( g ⋅ x ) = Φ h ( g ) ⋅ ( h ⋅ x ) \begin{CD}
x @>g>> g \cdot x \\
@VhVV @VVhV \\
h \cdot x @>>\Phi_h(g)> h \cdot(g \cdot x) = \Phi_h(g) \cdot(h \cdot x)
\end{CD} x h ↓ ⏐ h ⋅ x g Φ h ( g ) g ⋅ x ↓ ⏐ h h ⋅ ( g ⋅ x ) = Φ h ( g ) ⋅ ( h ⋅ x )
That is, transporting a point by g g g and then applying the symmetry h h h is identical to first applying the symmetry h h h and then transporting by the conjugated parameter Φ h ( g ) ∈ G \Phi_h(g) \in G Φ h ( g ) ∈ G .
Detailed analysis of the guiding examples. We now compute the total group K K K , the conjugation map Φ h \Phi_h Φ h , the kernel ker ( Φ ) \ker(\Phi) ker ( Φ ) , and the fixed-point subgroup G H G^H G H explicitly for each of our six guiding examples:
Reflection (G = ( R , + ) G = (\mathbb{R}, +) G = ( R , + ) , H = Z / 2 Z = { e , σ } H = \mathbb{Z}/2\mathbb{Z} = \{e, \sigma\} H = Z /2 Z = { e , σ } with σ ⋅ x = − x \sigma\cdot x = -x σ ⋅ x = − x ). The total transformation group is the one-dimensional semidirect product K = R ⋊ ( Z / 2 Z ) K = \mathbb{R} \rtimes(\mathbb{Z}/2\mathbb{Z}) K = R ⋊ ( Z /2 Z ) of translations and sign reflections. Denoting translation by T μ ( x ) = x + μ T_\mu(x) = x + \mu T μ ( x ) = x + μ , conjugating T μ T_\mu T μ by the reflection σ \sigma σ yields
( σ ∘ T μ ∘ σ − 1 ) ( x ) = σ ( T μ ( − x ) ) = − ( − x + μ ) = x − μ = T − μ ( x ) . (\sigma\circ T_\mu\circ\sigma^{-1})(x) = \sigma(T_\mu(-x)) = -(-x + \mu) = x - \mu= T_{-\mu}(x). ( σ ∘ T μ ∘ σ − 1 ) ( x ) = σ ( T μ ( − x )) = − ( − x + μ ) = x − μ = T − μ ( x ) .
Thus Φ σ ( μ ) = − μ \Phi_\sigma(\mu) = -\mu Φ σ ( μ ) = − μ : reflecting the coordinate axis reverses the direction of the translation shift. The kernel ker ( Φ ) = { e } \ker(\Phi) = \{e\} ker ( Φ ) = { e } is trivial, and the fixed-point subgroup is G H = { μ ∈ R : − μ = μ } = { 0 } G^H = \{\mu\in\mathbb{R} : -\mu= \mu\} = \{0\} G H = { μ ∈ R : − μ = μ } = { 0 } , indicating that no nonzero translation preserves reflection symmetry around the origin.
Euclidean (G = ( R d , + ) G = (\mathbb{R}^d, +) G = ( R d , + ) , H = O ( d ) H = O(d) H = O ( d ) with h ⋅ x = U x h \cdot x = Ux h ⋅ x = U x for U ∈ O ( d ) U \in O(d) U ∈ O ( d ) ). The total group is the full Euclidean group K = E ( d ) = R d ⋊ O ( d ) K = E(d) = \mathbb{R}^d \rtimes O(d) K = E ( d ) = R d ⋊ O ( d ) . Conjugating a translation T μ ( x ) = x + μ T_\mu(x) = x + \mu T μ ( x ) = x + μ by an orthogonal transformation U ∈ O ( d ) U \in O(d) U ∈ O ( d ) gives
( U ∘ T μ ∘ U − 1 ) ( x ) = U ( U − 1 x + μ ) = x + U μ = T U μ ( x ) , (U \circ T_\mu\circ U^{-1})(x) = U(U^{-1}x + \mu) = x + U\mu= T_{U\mu}(x), ( U ∘ T μ ∘ U − 1 ) ( x ) = U ( U − 1 x + μ ) = x + U μ = T U μ ( x ) ,
so Φ U ( μ ) = U μ \Phi_U(\mu) = U\mu Φ U ( μ ) = U μ : an orthogonal change of basis rotates or reflects the location parameter μ \mu μ by the same matrix U U U . The conjugation orbits in G = R d G = \mathbb{R}^d G = R d are concentric spheres { μ ∈ R d : ∥ μ ∥ 2 = r } \{\mu\in\mathbb{R}^d : \|\mu\|_2 = r\} { μ ∈ R d : ∥ μ ∥ 2 = r } . Because only the identity matrix fixes every vector in R d \mathbb{R}^d R d , the action is faithful (ker ( Φ ) = { I d } \ker(\Phi) = \{I_d\} ker ( Φ ) = { I d } ), and the fixed-point subgroup is trivial, G H = { 0 } G^H = \{0\} G H = { 0 } .
Sphere (G = S O ( 3 ) G = \mathrm{SO}(3) G = SO ( 3 ) , H = S O ( 2 ) H = \mathrm{SO}(2) H = SO ( 2 ) embedded as planar rotations around e 3 e_3 e 3 ). Because H ⊂ G H \subset G H ⊂ G , the total group collapses to K = S O ( 3 ) K = \mathrm{SO}(3) K = SO ( 3 ) . For an axial symmetry rotation h = R α e 3 ∈ S O ( 2 ) h = R^{e_3}_\alpha\in\mathrm{SO}(2) h = R α e 3 ∈ SO ( 2 ) by angle α \alpha α around e 3 e_3 e 3 and a general 3D rotation g = R β ν ∈ S O ( 3 ) g = R^\nu_\beta\in\mathrm{SO}(3) g = R β ν ∈ SO ( 3 ) by angle β \beta β around a unit axis ν ∈ S 2 \nu\in S^2 ν ∈ S 2 , matrix conjugation yields
Φ h ( g ) = R α e 3 R β ν ( R α e 3 ) − 1 = R β R α e 3 ν . \Phi_h(g) = R^{e_3}_\alpha R^\nu_\beta(R^{e_3}_\alpha)^{-1} = R^{R^{e_3}_\alpha\nu}_\beta. Φ h ( g ) = R α e 3 R β ν ( R α e 3 ) − 1 = R β R α e 3 ν .
Geometrically, conjugating a rotation by R α e 3 R^{e_3}_\alpha R α e 3 preserves the rotation angle β \beta β while rotating its axis ν \nu ν around the North–South pole e 3 e_3 e 3 by angle α \alpha α . A rotation R β ν R^\nu_\beta R β ν commutes with all rotations around e 3 e_3 e 3 if and only if its axis ν \nu ν is parallel to e 3 e_3 e 3 ; hence the fixed-point subgroup is G H = { R γ e 3 : γ ∈ [ 0 , 2 π ) } = S O ( 2 ) = H G^H = \{R^{e_3}_\gamma: \gamma\in[0, 2\pi)\} = \mathrm{SO}(2) = H G H = { R γ e 3 : γ ∈ [ 0 , 2 π )} = SO ( 2 ) = H , which coincides exactly with the internal symmetry group N = G ∩ H = H N = G \cap H = H N = G ∩ H = H .
Covariance (G = G L ( d ) G = \mathrm{GL}(d) G = GL ( d ) , H = O ( d ) H = O(d) H = O ( d ) with g ⋅ x = A x g \cdot x = Ax g ⋅ x = A x ). Here again H ⊂ G H \subset G H ⊂ G , so the total group is K = G L ( d ) K = \mathrm{GL}(d) K = GL ( d ) . For an orthogonal matrix U ∈ O ( d ) U \in O(d) U ∈ O ( d ) and an invertible matrix A ∈ G L ( d ) A \in\mathrm{GL}(d) A ∈ GL ( d ) , conjugation acts by orthogonal similarity:
Φ U ( A ) = U A U − 1 = U A U T . (1) \Phi_U(A) = UAU^{-1} = UAU^{\mathsf{T}}. \tag*{(1)} Φ U ( A ) = U A U − 1 = U A U T . (1)
Statistically, conjugating A A A by U U U corresponds to rotating the coordinate axes of the covariance matrix Σ = A A T ↦ U Σ U T \Sigma= AA^{\mathsf{T}} \mapsto U\Sigma U^{\mathsf{T}} Σ = A A T ↦ U Σ U T , which rotates the principal axes of the ellipsoid while preserving its eigenvalues (and hence its trace and determinant). Any matrix A ∈ GL ( d ) A \in\operatorname{GL}(d) A ∈ GL ( d ) satisfying U A U T = A UAU^{\mathsf{T}} = A U A U T = A for all U ∈ O ( d ) U \in O(d) U ∈ O ( d ) must remain invariant under all orthogonal rotations and coordinate reflections, which forces A A A to have equal scaling in every direction and no off-diagonal couplings (formally, by Schur’s lemma applied to the irreducible action of O ( d ) O(d) O ( d ) on R d \mathbb{R}^{d} R d ). Thus the fixed-point subgroup is the one-dimensional subgroup of isotropic dilations, G H = { λ I d : λ ∈ R ∖ { 0 } } G^{H} = \{\lambda I_{d} : \lambda\in\mathbb{R} \setminus\{0\}\} G H = { λ I d : λ ∈ R ∖ { 0 }} , whereas the kernel of Φ \Phi Φ is ker ( Φ ) = { ± I d } \ker(\Phi) = \{\pm I_{d}\} ker ( Φ ) = { ± I d } .
Square (G = SO ( 2 ) G = \operatorname{SO}(2) G = SO ( 2 ) , H = D 4 H = D_{4} H = D 4 ). The planar rotation group G = SO ( 2 ) G = \operatorname{SO}(2) G = SO ( 2 ) and the dihedral group H = D 4 H = D_{4} H = D 4 intersect in the fourfold rotational subgroup N = C 4 = { R k π / 2 : k = 0 , 1 , 2 , 3 } N = C_{4} = \{R_{k\pi/2} : k = 0,1,2,3\} N = C 4 = { R k π /2 : k = 0 , 1 , 2 , 3 } and together generate the full orthogonal group K = O ( 2 ) K = O(2) K = O ( 2 ) . The eight elements of D 4 D_{4} D 4 consist of four rotations r k = R k π / 2 r_{k} = R_{k\pi/2} r k = R k π /2 and four reflections s k s_{k} s k . Because SO ( 2 ) \operatorname{SO}(2) SO ( 2 ) is abelian, all rotations r k ∈ C 4 r_{k} \in C_{4} r k ∈ C 4 commute with every R θ ∈ SO ( 2 ) R_{\theta} \in\operatorname{SO}(2) R θ ∈ SO ( 2 ) , giving Φ r k ( R θ ) = R θ \Phi_{r_{k}}(R_{\theta}) = R_{\theta} Φ r k ( R θ ) = R θ . By contrast, any reflection s k ∈ D 4 s_{k} \in D_{4} s k ∈ D 4 reverses planar orientation and therefore inverts the rotation angle:
Φ s k ( R θ ) = s k R θ s k − 1 = R − θ . \Phi_{s_{k}}(R_{\theta}) = s_{k}R_{\theta}s_{k}^{-1} = R_{-\theta}. Φ s k ( R θ ) = s k R θ s k − 1 = R − θ .
Consequently, the conjugation homomorphism Φ : D 4 → Aut ( SO ( 2 ) ) \Phi: D_{4} \to\operatorname{Aut}(\operatorname{SO}(2)) Φ : D 4 → Aut ( SO ( 2 )) factors through D 4 / C 4 ≅ Z / 2 Z D_{4}/C_{4} \cong\mathbb{Z}/2\mathbb{Z} D 4 / C 4 ≅ Z /2 Z with kernel ker ( Φ ) = C 4 \ker(\Phi) = C_{4} ker ( Φ ) = C 4 , and its fixed-point set in SO ( 2 ) \operatorname{SO}(2) SO ( 2 ) is the discrete two-element subgroup G H = { R 0 , R π } ≅ Z / 2 Z G^{H} = \{R_{0}, R_{\pi}\} \cong\mathbb{Z}/2\mathbb{Z} G H = { R 0 , R π } ≅ Z /2 Z .
Permutation (G = ( R n , + ) G = (\mathbb{R}^{n}, +) G = ( R n , + ) , H = S n H = \mathbb{S}_{n} H = S n with σ ⋅ x = P σ x \sigma\cdot x = P_{\sigma}x σ ⋅ x = P σ x ). The total group is the semidirect product K = R n ⋊ S n K = \mathbb{R}^{n} \rtimes\mathbb{S}_{n} K = R n ⋊ S n of translations and coordinate permutations. Conjugating a translation T μ ( x ) = x + μ T_{\mu}(x) = x + \mu T μ ( x ) = x + μ by a permutation matrix P σ P_{\sigma} P σ gives
( P σ ∘ T μ ∘ P σ − 1 ) ( x ) = P σ ( P σ − 1 x + μ ) = x + P σ μ = T P σ μ ( x ) , (P_{\sigma} \circ T_{\mu} \circ P_{\sigma}^{-1})(x) = P_{\sigma}(P_{\sigma}^{-1}x + \mu) = x + P_{\sigma}\mu= T_{P_{\sigma}\mu}(x), ( P σ ∘ T μ ∘ P σ − 1 ) ( x ) = P σ ( P σ − 1 x + μ ) = x + P σ μ = T P σ μ ( x ) ,
so Φ σ ( μ ) = P σ μ \Phi_{\sigma}(\mu) = P_{\sigma}\mu Φ σ ( μ ) = P σ μ : coordinate permutations act on the location parameter by permuting its entries. The conjugation orbit of μ ∈ R n \mu\in\mathbb{R}^{n} μ ∈ R n consists of all vectors sharing the same empirical multiset of coordinates. The action is faithful (ker ( Φ ) = { e } \ker(\Phi) = \{e\} ker ( Φ ) = { e } ), and a location vector μ \mu μ is fixed by all permutations if and only if all of its coordinates are equal:
G H = { μ ∈ R n : P σ μ = μ ∀ σ ∈ S n } = span ( 1 ) , G^{H} = \{\mu\in\mathbb{R}^{n} : P_{\sigma}\mu= \mu\ \forall\sigma\in\mathbb{S}_{n}\} = \operatorname{span}(1), G H = { μ ∈ R n : P σ μ = μ ∀ σ ∈ S n } = span ( 1 ) ,
where 1 = ( 1 , … , 1 ) T ∈ R n 1 = (1,\ldots,1)^{\mathsf{T}} \in\mathbb{R}^{n} 1 = ( 1 , … , 1 ) T ∈ R n denotes the all-ones vector. Statistically, shifting an exchangeable distribution by c 1 c1 c 1 shifts the common location of all coordinates simultaneously while preserving exchangeability.
Conjugation and distributional invariance of the variational objective. The following proposition explains why the conjugation map Φ h \Phi_{h} Φ h directly governs the symmetries of the variational loss landscape L ( g ) = KL ( q g ∥ p ) \mathcal{L}(g) = \operatorname{KL}(q_{g}\Vert p) L ( g ) = KL ( q g ∥ p ) around g ∗ g^{*} g ∗ .
Proposition 4.3 (Conjugation and transport of symmetric distributions). Assume that the base distribution q 0 ∈ P ( X ) q_{0} \in\mathcal{P}(\mathcal{X}) q 0 ∈ P ( X ) is H H H -symmetric, and let g ∈ G g \in G g ∈ G . For any h ∈ H h \in H h ∈ H , let q h g : = h # q g q_{hg} := h_{\#}q_{g} q h g := h # q g denote the pushforward of q g q_{g} q g under the symmetry h h h , i.e., the distribution of h ⋅ ( g ⋅ Z ) h \cdot(g \cdot Z) h ⋅ ( g ⋅ Z ) when Z ∼ q 0 Z \sim q_{0} Z ∼ q 0 (note that the notation q h g q_{hg} q h g refers to the pushforward measure ( h ∘ g ) # q 0 (h \circ g)_{\#}q_{0} ( h ∘ g ) # q 0 , even when h ∉ G h \notin G h ∈ / G ). Then:
(i) Pushforward equivariance: q Φ h ( g ) = q h g q_{\Phi_{h}(g)} = q_{hg} q Φ h ( g ) = q h g , meaning that q Φ h ( g ) ( x ) = q g ( h − 1 ⋅ x ) q_{\Phi_{h}(g)}(x) = q_{g}(h^{-1} \cdot x) q Φ h ( g ) ( x ) = q g ( h − 1 ⋅ x ) for all x ∈ X x \in\mathcal{X} x ∈ X .
(ii) Loss under conjugated transport: L ( Φ h ( g ) ) = KL ( q h g ∥ p ) \mathcal{L}(\Phi_{h}(g)) = \operatorname{KL}(q_{hg}\Vert p) L ( Φ h ( g )) = KL ( q h g ∥ p ) for all h ∈ H h \in H h ∈ H and g ∈ G g \in G g ∈ G .
(iii) Conjugation invariance of the relative loss around g ∗ g^{*} g ∗ . If, in addition, the target p p p has H H H -conjugate symmetry around g ∗ ∈ G g^{*} \in G g ∗ ∈ G , then for every relative transport g ∈ G g \in G g ∈ G and every symmetry h ∈ H h \in H h ∈ H ,
L ( g ∗ Φ h ( g ) ) = L ( g ∗ g ) . \mathcal{L}(g^{*}\Phi_{h}(g)) = \mathcal{L}(g^{*}g). L ( g ∗ Φ h ( g )) = L ( g ∗ g ) .
Proof. To prove (i), observe that q Φ h ( g ) q_{\Phi_h(g)} q Φ h ( g ) is the law of Φ h ( g ) ⋅ Z = ( h ∘ g ∘ h − 1 ) ⋅ Z \Phi_h(g) \cdot Z = (h \circ g \circ h^{-1}) \cdot Z Φ h ( g ) ⋅ Z = ( h ∘ g ∘ h − 1 ) ⋅ Z when Z ∼ q 0 Z \sim q_0 Z ∼ q 0 . Because q 0 q_0 q 0 is H H H -symmetric, the random variable Z ′ = h − 1 ⋅ Z Z' = h^{-1} \cdot Z Z ′ = h − 1 ⋅ Z has the same distribution q 0 q_0 q 0 as Z Z Z . Therefore,
Φ h ( g ) ⋅ Z = h ⋅ ( g ⋅ ( h − 1 ⋅ Z ) ) = d h ⋅ ( g ⋅ Z ) ∼ q h g , \Phi_h(g) \cdot Z = h \cdot(g \cdot(h^{-1} \cdot Z)) \overset{d}{=} h \cdot(g \cdot Z) \sim q_{hg}, Φ h ( g ) ⋅ Z = h ⋅ ( g ⋅ ( h − 1 ⋅ Z )) = d h ⋅ ( g ⋅ Z ) ∼ q h g ,
which establishes q Φ h ( g ) = q h g q_{\Phi_h(g)} = q_{hg} q Φ h ( g ) = q h g , and part (ii) follows immediately from the definition of L \mathcal{L} L . For part (iii), recall that p = ( g ∗ ) # p ~ p = (g^*)_{\#}\tilde{p} p = ( g ∗ ) # p ~ where p ~ \tilde{p} p ~ is H H H -symmetric (p ~ = h # p ~ \tilde{p} = h_{\#}\tilde{p} p ~ = h # p ~ ). By the invariance of the KL divergence under the diffeomorphism ( g ∗ ) − 1 (g^*)^{-1} ( g ∗ ) − 1 , we have L ( g ∗ g ′ ) = KL ( q g ∗ g ′ ∥ p ) = KL ( q g ′ ∥ p ~ ) \mathcal{L}(g^*g') = \operatorname{KL}(q_{g^*g'} \mathbin\Vert p) = \operatorname{KL}(q_{g'} \mathbin\Vert\tilde{p}) L ( g ∗ g ′ ) = KL ( q g ∗ g ′ ∥ p ) = KL ( q g ′ ∥ p ~ ) for any g ′ ∈ G g' \in G g ′ ∈ G . Applying this identity with g ′ = Φ h ( g ) g' = \Phi_h(g) g ′ = Φ h ( g ) and invoking part (i) together with the H H H -invariance of p ~ \tilde{p} p ~ yields
L ( g ∗ Φ h ( g ) ) = KL ( q Φ h ( g ) ∥ p ~ ) = KL ( h # q g ∥ p ~ ) = KL ( h # q g ∥ h # p ~ ) = KL ( q g ∥ p ~ ) = L ( g ∗ g ) , \mathcal{L}(g^*\Phi_h(g)) = \operatorname{KL}(q_{\Phi_h(g)} \mathbin\Vert\tilde{p}) = \operatorname{KL}(h_{\#}q_g \mathbin\Vert\tilde{p}) = \operatorname{KL}(h_{\#}q_g \mathbin\Vert h_{\#}\tilde{p}) = \operatorname{KL}(q_g \mathbin\Vert\tilde{p}) = \mathcal{L}(g^*g), L ( g ∗ Φ h ( g )) = KL ( q Φ h ( g ) ∥ p ~ ) = KL ( h # q g ∥ p ~ ) = KL ( h # q g ∥ h # p ~ ) = KL ( q g ∥ p ~ ) = L ( g ∗ g ) ,
where the penultimate equality again uses the invariance of KL divergence under the diffeomorphism h h h . □
Proposition 4.3 provides the key geometric insight into how symmetry shapes the variational objective. Part (i) shows that because q 0 q_0 q 0 is H H H -symmetric, applying a symmetry h h h to the transported distribution q g q_g q g is equivalent to replacing the parameter g g g by its conjugate Φ h ( g ) ∈ G \Phi_h(g) \in G Φ h ( g ) ∈ G —at the distribution level, q g q_g q g and q Φ h ( g ) q_{\Phi_h(g)} q Φ h ( g ) have the exact same shape, viewed from an h h h -rotated angle. Part (iii) then shows that when p p p is H H H -conjugate symmetric around g ∗ g^* g ∗ , stepping away from g ∗ g^* g ∗ by a relative transport g g g or by its conjugated version Φ h ( g ) \Phi_h(g) Φ h ( g ) yields the exact same KL divergence, so the local optimization landscape around g ∗ g^* g ∗ cannot distinguish between conjugate relative transports and L \mathcal{L} L is constant along every conjugation orbit { g ∗ Φ h ( g ) : h ∈ H } \{g^*\Phi_h(g) : h \in H\} { g ∗ Φ h ( g ) : h ∈ H } . Notice that this conjugation invariance L ( g ∗ Φ h ( g ) ) = L ( g ∗ g ) \mathcal{L}(g^*\Phi_h(g)) = \mathcal{L}(g^*g) L ( g ∗ Φ h ( g )) = L ( g ∗ g ) is far richer than the right-invariance L ( g n ) = L ( g ) \mathcal{L}(gn) = \mathcal{L}(g) L ( g n ) = L ( g ) for n ∈ N = G ∩ H n \in N = G \cap H n ∈ N = G ∩ H : right-invariance under N N N merely reflects parameter redundancy within G G G (since q g n = q g q_{gn} = q_g q g n = q g ), whereas conjugation invariance captures how symmetries in H H H act on G G G from the outside to relate distinct distributions q g ∗ g q_{g^*g} q g ∗ g and q g ∗ Φ h ( g ) q_{g^*\Phi_h(g)} q g ∗ Φ h ( g ) that are symmetric counterparts of one another around g ∗ g^* g ∗ .
Quotient space and statistical identifiability regimes
We now formalize the parameter redundancy induced by the overlap between the transport group G G G and the symmetry group H H H . Because q 0 q_0 q 0 is H H H -invariant, any transformation in the total group K = G H K = GH K = G H that differs on the right by an element of H H H produces the exact same pushforward distribution on X \mathcal{X} X . Restricting to the transport group G G G , two parameters g , g ′ ∈ G g, g' \in G g , g ′ ∈ G satisfy g ′ = g n g' = gn g ′ = g n for some n ∈ N : = G ∩ H n \in N := G \cap H n ∈ N := G ∩ H if and only if they belong to the same left coset g N gN g N in the quotient space
M : = G / N ≃ K / H , N = G ∩ H . \mathcal{M} := G/N \simeq K/H, \qquad N = G \cap H. M := G / N ≃ K / H , N = G ∩ H .
In statistical terms, the subgroup Stab G ( q 0 ) : = { g ∈ G : g # q 0 = q 0 } \operatorname{Stab}_G(q_0) := \{g \in G : g_{\#}q_0 = q_0\} Stab G ( q 0 ) := { g ∈ G : g # q 0 = q 0 } (the stabilizer of q 0 q_0 q 0 in $G) is the set of redundant parameter transformations that leave the base distribution q 0 q_0 q 0 unchanged, and forming the quotient space M = G / N \mathcal{M} = G/N M = G / N simply means grouping together all parameters { g n : n ∈ N } \{gn : n \in N\} { g n : n ∈ N } that produce the exact same distribution q g q_g q g into a single equivalence class [ g ] = g N [g] = gN [ g ] = g N (just as a covariance matrix Σ = A A T \Sigma= AA^{\mathsf{T}} Σ = A A T groups together all matrix square roots { A U : U ∈ O ( d ) } \{AU : U \in O(d)\} { A U : U ∈ O ( d )} ). Assuming that the stabilizer of q 0 q_0 q 0 within G G G is precisely N = G ∩ H N = G \cap H N = G ∩ H (i.e., Stab G ( q 0 ) = G ∩ H \operatorname{Stab}_G(q_0) = G \cap H Stab G ( q 0 ) = G ∩ H ), the map [ g ] = g N ↦ q g [g] = gN \mapsto q_g [ g ] = g N ↦ q g is injective on M \mathcal{M} M , so the smooth coset manifold M = G / N \mathcal{M} = G/N M = G / N represents the space of statistically identifiable parameters of the variational family Q G \mathcal{Q}_G Q G . Depending on the structure of the intersection N = G ∩ H N = G \cap H N = G ∩ H , our examples fall into three distinct identifiability regimes:
Separable symmetry (N = G ∩ H = { e } N = G \cap H = \{e\} N = G ∩ H = { e } ). When the transport group G G G and the symmetry group H H H intersect only at the identity transformation, the quotient manifold coincides with the transport group itself, M ≃ G \mathcal{M} \simeq G M ≃ G . In this regime, no non-trivial transport leaves q 0 q_0 q 0 invariant, so the parameter g ∈ G g \in G g ∈ G is globally identifiable from q g q_g q g :
Examples 1, 2, and 6 (Reflection, Euclidean, and Permutation location families): In all three translation models, G = ( R d , + ) G = (\mathbb{R}^{d}, +) G = ( R d , + ) (or R n \mathbb{R}^{n} R n ) contains no non-trivial linear isometries around the origin, so N = G ∩ H = { 0 } N = G \cap H = \{0\} N = G ∩ H = { 0 } and M ≅ R d \mathcal{M} \cong\mathbb{R}^{d} M ≅ R d (or R n \mathbb{R}^{n} R n ). Every location parameter μ \mu μ uniquely indexes a distinct distribution q μ ∈ Q G q_{\mu} \in\mathcal{Q}_{G} q μ ∈ Q G . Crucially, however, global parameter identifiability on M \mathcal{M} M does not by itself guarantee that μ ∗ \mu^{*} μ ∗ minimizes L ( μ ) \mathcal{L}(\mu) L ( μ ) under model misspecification: as we show in § 6 , Examples 1 and 2 are always stationary at μ ∗ \mu^{*} μ ∗ , whereas Example 6 can exhibit an additive location bias along the diagonal span ( 1 ) \operatorname{span}(\mathbf{1}) span ( 1 ) .
Internal symmetry
2. Internal symmetry (H ⊂ G H \subset G H ⊂ G , so N = H N = H N = H ). When the symmetry group H H H is a subgroup of the transport group G G G , the total group collapses to K = G K = G K = G and the identifiable parameter space is the homogeneous quotient manifold M = G / H \mathcal{M} = G/H M = G / H of left cosets [ g ] = g H [g] = gH [ g ] = g H . In this regime, the group element g ∈ G g \in G g ∈ G is overparametrized , and only the coset [ g ] ∈ G / H [g] \in G/H [ g ] ∈ G / H is statistically identifiable:
Example 3 (Sphere): With G = SO ( 3 ) G = \operatorname{SO}(3) G = SO ( 3 ) and H = N = SO ( 2 ) H = N = \operatorname{SO}(2) H = N = SO ( 2 ) (rotations around e 3 e_{3} e 3 ), the quotient manifold is M = SO ( 3 ) / SO ( 2 ) ≅ S 2 \mathcal{M} = \operatorname{SO}(3)/\operatorname{SO}(2) \cong S^{2} M = SO ( 3 ) / SO ( 2 ) ≅ S 2 . Right-multiplying a rotation matrix R ∈ SO ( 3 ) R \in\operatorname{SO}(3) R ∈ SO ( 3 ) by any axial rotation R α e 3 ∈ SO ( 2 ) R_{\alpha}^{e_{3}} \in\operatorname{SO}(2) R α e 3 ∈ SO ( 2 ) leaves the image of the North pole v = R e 3 ∈ S 2 v = R e_{3} \in S^{2} v = R e 3 ∈ S 2 unchanged, so the identifiable statistic is precisely the modal axis v ∗ = R ∗ e 3 ∈ S 2 v^{*} = R^{*}e_{3} \in S^{2} v ∗ = R ∗ e 3 ∈ S 2 .
Example 4 (Covariance): With G = GL ( d ) G = \operatorname{GL}(d) G = GL ( d ) and H = N = O ( d ) H = N = \operatorname{O}(d) H = N = O ( d ) , the quotient manifold is M = GL ( d ) / O ( d ) ≅ SPD ( d ) \mathcal{M} = \operatorname{GL}(d)/\operatorname{O}(d) \cong\operatorname{SPD}(d) M = GL ( d ) / O ( d ) ≅ SPD ( d ) , the cone of d × d d \times d d × d symmetric positive-definite matrices. Two invertible matrices A , A ′ ∈ GL ( d ) A, A' \in\operatorname{GL}(d) A , A ′ ∈ GL ( d ) belong to the same left coset A O ( d ) A\operatorname{O}(d) A O ( d ) if and only if A ′ = A U A' = AU A ′ = A U for some U ∈ O ( d ) U \in\operatorname{O}(d) U ∈ O ( d ) , which is equivalent to A ′ ( A ′ ) T = A A T = Σ A'(A')^{\mathsf{T}} = AA^{\mathsf{T}} = \Sigma A ′ ( A ′ ) T = A A T = Σ . Thus, the factor A ∗ A^{*} A ∗ is identifiable up to right-multiplication by O ( d ) \operatorname{O}(d) O ( d ) (with continuous gradient optimization operating within a single connected component such as GL + ( d ) \operatorname{GL}^{+}(d) GL + ( d ) , where the connected stabilizer fiber is GL + ( d ) ∩ O ( d ) = SO ( d ) \operatorname{GL}^{+}(d) \cap\operatorname{O}(d) = \operatorname{SO}(d) GL + ( d ) ∩ O ( d ) = SO ( d ) ), and the identifiable parameter on M \mathcal{M} M is the covariance matrix Σ ∗ = A ∗ ( A ∗ ) T \Sigma^{*} = A^{*}(A^{*})^{\mathsf{T}} Σ ∗ = A ∗ ( A ∗ ) T .
Partial overlap
3. Partial overlap ({ e } ⊊ N ⊊ H \{e\} \subsetneq N \subsetneq H { e } ⊊ N ⊊ H ). When G G G and H H H share a proper, non-trivial subgroup N = G ∩ H N = G \cap H N = G ∩ H , the identifiable parameter space is the quotient M = G / N \mathcal{M} = G/N M = G / N , while the remaining symmetries in H ∖ N H \setminus N H ∖ N act non-trivially on M \mathcal{M} M from the outside via conjugation:
Example 5 (Square): With G = SO ( 2 ) G = \operatorname{SO}(2) G = SO ( 2 ) and H = D 4 H = D_{4} H = D 4 , the intersection is the discrete fourfold rotation group N = C 4 = { R k π / 2 : k = 0 , 1 , 2 , 3 } N = C_{4} = \{R_{k\pi/2}: k = 0, 1, 2, 3\} N = C 4 = { R k π /2 : k = 0 , 1 , 2 , 3 } . Because rotating a square-symmetric distribution by 90 ∘ 90^\circ 9 0 ∘ produces the exact same distribution, the orientation angle θ ∈ [ 0 , 2 π ) \theta\in[0, 2\pi) θ ∈ [ 0 , 2 π ) is identifiable only modulo π / 2 \pi/2 π /2 , on the quotient manifold M = SO ( 2 ) / C 4 ≅ S 1 \mathcal{M} = \operatorname{SO}(2)/C_{4} \cong S^{1} M = SO ( 2 ) / C 4 ≅ S 1 . Meanwhile, the reflections in D 4 ∖ C 4 D_{4} \setminus C_{4} D 4 ∖ C 4 do not belong to G G G , yet they act on M \mathcal{M} M via conjugation ( θ ↦ − θ ) (\theta\mapsto-\theta) ( θ ↦ − θ ) , which will prove decisive in forcing the gradient at θ ∗ \theta^{*} θ ∗ to vanish.
Remark (Identifiability versus unbiased stationarity). Because q g n = q g q_{gn} = q_{g} q g n = q g for all n ∈ N n \in N n ∈ N , the variational loss L ( g n ) = L ( g ) \mathcal{L}(gn) = \mathcal{L}(g) L ( g n ) = L ( g ) for every g ∈ G g \in G g ∈ G and n ∈ N n \in N n ∈ N . When N N N is a positive-dimensional Lie subgroup (as in Examples 3 and 4), L \mathcal{L} L is identically constant along the coset fibers g N gN g N , so the directional derivative of L \mathcal{L} L at g ∗ g^{*} g ∗ automatically vanishes along all tangent directions to N N N . When N N N is finite (as in Example 5), L \mathcal{L} L is periodic across the ∣ N ∣ |N| ∣ N ∣ equivalent representatives { g ∗ n : n ∈ N } \{g^{*}n : n \in N\} { g ∗ n : n ∈ N } . However, this fiber invariance only accounts for directions inside N N N ; whether the gradient vanishes along the identifiable directions transverse to N N N depends on the adjoint action of the full symmetry group H H H on the Lie algebra of G G G , which we develop in the next two sections.
Differentiation on the transport Lie group
To characterize when the true transport parameter g ∗ ∈ G g^{*} \in G g ∗ ∈ G is a first-order stationary point of the variational loss L ( g ) = KL ( q g ∥ p ) \mathcal{L}(g) = \operatorname{KL}(q_{g}\|p) L ( g ) = KL ( q g ∥ p ) , we require differential calculus on the Lie group G G G . In classical parametric statistics, one typically differentiates a loss function by adding a small coordinate vector θ + t u \theta+ t u θ + t u in R p \mathbb{R}^{p} R p . For transformation parameters such as rotation matrices R ∈ SO ( d ) R \in\operatorname{SO}(d) R ∈ SO ( d ) or scale matrices A ∈ GL ( d ) A \in\operatorname{GL}(d) A ∈ GL ( d ) , however, an additive perturbation A + t U A + tU A + t U either leaves the manifold of valid parameters (breaking orthogonality) or depends on the current scale A A A . Because G G G is both a group and a smooth manifold, local perturbations around any parameter g ∈ G g \in G g ∈ G can instead be parameterized canonically through the tangent space at the identity—the Lie algebra g = T e G G \mathfrak{g} = T_{e_G}G g = T e G G —via left multiplication t ↦ g exp ( t u ) t \mapsto g\exp(tu) t ↦ g exp ( t u ) (where exp ( t u ) ≈ I + t u \exp(tu) \approx I + tu exp ( t u ) ≈ I + t u for matrix groups). Writing local perturbations multiplicatively as g ⋅ exp ( t u ) g \cdot\exp(tu) g ⋅ exp ( t u ) rather than additively (known geometrically as left-trivialization ) measures relative (or equivariant) parameter changes in a single fixed vector space g \mathfrak{g} g at the identity. From a statistical perspective, this multiplicative parameterization is natural for transformation models: perturbing g ∗ exp ( t u ) g^* \exp(tu) g ∗ exp ( t u ) corresponds, after pulling back by ( g ∗ ) − 1 (g^*)^{-1} ( g ∗ ) − 1 , to perturbing the standardized base distribution q 0 q_0 q 0 by exp ( t u ) \exp(tu) exp ( t u ) near the identity e G e_G e G . We briefly review the necessary tools from Lie theory and Riemannian optimization; see [11 , 5 ] for comprehensive treatments.
Lie group calculus and left-trivialized Riemannian gradients
Definition 5.1 (Lie algebra, exponential map, and Riemannian gradient). Let G G G be a Lie group with identity element e G e_G e G . We recall the following differential-geometric objects:
The Lie algebra g : = T e G G \mathfrak{g} := T_{e_G}G g := T e G G is the tangent vector space to G G G at the identity, which we equip with a positive-definite inner product ⟨ ⋅ , ⋅ ⟩ g \langle\cdot,\cdot\rangle_{\mathfrak{g}} ⟨ ⋅ , ⋅ ⟩ g (defining a left-invariant Riemannian metric on G G G ).
The exponential map exp : g → G \exp: \mathfrak{g} \to G exp : g → G maps each tangent vector u ∈ g u \in\mathfrak{g} u ∈ g to the time-1 point γ u ( 1 ) \gamma_u(1) γ u ( 1 ) of the unique one-parameter subgroup γ u ( t ) = exp ( t u ) \gamma_u(t) = \exp(tu) γ u ( t ) = exp ( t u ) satisfying γ u ( 0 ) = e G \gamma_u(0) = e_G γ u ( 0 ) = e G and γ u ′ ( 0 ) = u \gamma_u'(0) = u γ u ′ ( 0 ) = u . The map exp \exp exp is a smooth local diffeomorphism from a neighborhood of 0 ∈ g 0 \in\mathfrak{g} 0 ∈ g onto a neighborhood of e G ∈ G e_G \in G e G ∈ G , so a neighborhood of any g ∈ G g \in G g ∈ G is parameterized via u ↦ g ⋅ exp ( u ) u \mapsto g \cdot\exp(u) u ↦ g ⋅ exp ( u ) .
The (left-trivialized) Riemannian gradient ∇ G L ( g ) ∈ g \nabla_G \mathcal{L}(g) \in\mathfrak{g} ∇ G L ( g ) ∈ g of a smooth objective L : G → R \mathcal{L} : G \to\mathbb{R} L : G → R at g ∈ G g \in G g ∈ G is the unique element of g \mathfrak{g} g representing the first-order Taylor expansion of u ↦ L ( g ⋅ exp ( u ) ) u \mapsto\mathcal{L}(g \cdot\exp(u)) u ↦ L ( g ⋅ exp ( u )) at u = 0 u = 0 u = 0 :
L ( g ⋅ exp ( u ) ) = L ( g ) + ⟨ ∇ G L ( g ) , u ⟩ g + O ( ∥ u ∥ 2 ) ∀ u ∈ g . (2) \mathcal{L}(g \cdot\exp(u)) = \mathcal{L}(g) + \langle\nabla_G \mathcal{L}(g),u\rangle_{\mathfrak{g}} + O(\lVert u\rVert^2) \quad\forall u \in\mathfrak{g}. \tag*{(2)} L ( g ⋅ exp ( u )) = L ( g ) + ⟨ ∇ G L ( g ) , u ⟩ g + O (∥ u ∥ 2 ) ∀ u ∈ g . (2)
Concrete form of the exponential map and Riemannian gradient. In statistical applications, G G G is either an additive vector space or a matrix Lie group, where both the exponential map and the left-trivialized gradient take explicit forms:
Additive translation group G = ( R d , + ) G = (\mathbb{R}^d,+) G = ( R d , + ) : The group operation is vector addition, the identity is e G = 0 e_G = 0 e G = 0 , the Lie algebra is g = R d \mathfrak{g} = \mathbb{R}^d g = R d equipped with the standard Euclidean inner product, and the exponential map is the identity exp ( u ) = u \exp(u) = u exp ( u ) = u . Perturbing μ ∈ G \mu\in G μ ∈ G via μ ⋅ exp ( t u ) \mu\cdot\exp(tu) μ ⋅ exp ( t u ) is simply μ + t u \mu+ tu μ + t u , and the Riemannian gradient coincides with the ordinary Euclidean gradient:
∇ G L ( μ ) = ∇ E u c L ( μ ) ∈ R d . \nabla_G \mathcal{L}(\mu) = \nabla_{\mathrm{Euc}}\mathcal{L}(\mu) \in\mathbb{R}^d. ∇ G L ( μ ) = ∇ Euc L ( μ ) ∈ R d .
Matrix Lie groups G ⊆ GL ( d ) G \subseteq\operatorname{GL}(d) G ⊆ GL ( d ) : When G G G is a closed subgroup of GL ( d ) \operatorname{GL}(d) GL ( d ) , exp ( U ) = ∑ k = 0 ∞ 1 k ! U k \exp(U) = \sum_{k=0}^{\infty} \frac{1}{k!}U^k exp ( U ) = ∑ k = 0 ∞ k ! 1 U k is the standard matrix exponential, and g ⋅ exp ( t U ) = g e t U g \cdot\exp(tU) = ge^{tU} g ⋅ exp ( t U ) = g e t U is matrix multiplication. If L \mathcal{L} L is extended smoothly to an open neighborhood in R d × d \mathbb{R}^{d \times d} R d × d with Euclidean matrix of partial derivatives ∇ E u c L ( g ) ∈ R d × d \nabla_{\mathrm{Euc}}\mathcal{L}(g) \in\mathbb{R}^{d \times d} ∇ Euc L ( g ) ∈ R d × d , then for any U ∈ g U \in\mathfrak{g} U ∈ g , the chain rule gives
d d t ∣ t = 0 L ( g e t U ) = ⟨ ∇ E u c L ( g ) , g U ⟩ F r o b = ⟨ g T ∇ E u c L ( g ) , U ⟩ F r o b . \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\mathcal{L}(ge^{tU}) = \langle\nabla_{\mathrm{Euc}}\mathcal{L}(g),gU\rangle_{\mathrm{Frob}} = \langle g^{\mathsf{T}}\nabla_{\mathrm{Euc}}\mathcal{L}(g),U\rangle_{\mathrm{Frob}}. d t d t = 0 L ( g e t U ) = ⟨ ∇ Euc L ( g ) , g U ⟩ Frob = ⟨ g T ∇ Euc L ( g ) , U ⟩ Frob .
Consequently, under the Frobenius inner product ⟨ U , V ⟩ g = tr ( U T V ) \langle U,V\rangle_{\mathfrak{g}} = \operatorname{tr}(U^{\mathsf{T}}V) ⟨ U , V ⟩ g = tr ( U T V ) , the Riemannian gradient in g \mathfrak{g} g is obtained by left-multiplying the Euclidean gradient by g T g^{\mathsf{T}} g T (transporting the tangent vector from T g G T_gG T g G back to T I G = g T_IG = \mathfrak{g} T I G = g ) and projecting orthogonally onto the subspace g ⊆ R d × d \mathfrak{g} \subseteq\mathbb{R}^{d \times d} g ⊆ R d × d :
∇ G L ( g ) = proj g ( g T ∇ E u c L ( g ) ) . \nabla_G \mathcal{L}(g) = \operatorname{proj}_{\mathfrak{g}}\left(g^{\mathsf{T}}\nabla_{\mathrm{Euc}}\mathcal{L}(g)\right). ∇ G L ( g ) = proj g ( g T ∇ Euc L ( g ) ) .
For the general linear group G = GL ( d ) G = \operatorname{GL}(d) G = GL ( d ) , the Lie algebra is the full matrix space g = g l ( d ) = R d × d \mathfrak{g} = \mathfrak{gl}(d) = \mathbb{R}^{d \times d} g = gl ( d ) = R d × d , so proj g \operatorname{proj}_{\mathfrak{g}} proj g is the identity and ∇ G L ( A ) = A T ∇ E u c L ( A ) \nabla_G\mathcal{L}(A) = A^{\mathsf{T}}\nabla_{\mathrm{Euc}}\mathcal{L}(A) ∇ G L ( A ) = A T ∇ Euc L ( A ) . For the special orthogonal group G = SO ( d ) G = \operatorname{SO}(d) G = SO ( d ) , the Lie algebra g = s o ( d ) = { U ∈ R d × d : U T = − U } \mathfrak{g} = \mathfrak{so}(d) = \{U \in\mathbb{R}^{d \times d}: U^{\mathsf{T}} = -U\} g = so ( d ) = { U ∈ R d × d : U T = − U } consists of skew-symmetric matrices, and orthogonal projection onto s o ( d ) \mathfrak{so}(d) so ( d ) is the skew-symmetrization proj s o ( d ) ( M ) = 1 2 ( M − M T ) \operatorname{proj}_{\mathfrak{so}(d)}(M) = \frac{1}{2}(M - M^{\mathsf{T}}) proj so ( d ) ( M ) = 2 1 ( M − M T ) .
Adjoint representation and the trivial subspace $\mathfrak{g}^{H}`
In Section 4.2, we saw that the symmetry group H H H acts on the transport group G G G by conjugation automorphisms Φ h ( g ) = h ∘ g ∘ h − 1 \Phi_h(g) = h \circ g \circ h^{-1} Φ h ( g ) = h ∘ g ∘ h − 1 . While Φ h : G → G \Phi_h : G \to G Φ h : G → G is in general a nonlinear map on the manifold G G G , it fixes the identity element (Φ h ( e G ) = e G \Phi_h(e_G) = e_G Φ h ( e G ) = e G for every h ∈ H h \in H h ∈ H ). Differentiating Φ h \Phi_h Φ h at e G e_G e G therefore induces a linear representation of H H H on the tangent space g = T e G G \mathfrak{g} = T_{e_G}G g = T e G G : the adjoint representation.
Definition 5.2 (Adjoint representation and trivial subspace). The adjoint representation of H H H on g \mathfrak{g} g is the group homomorphism Ad : H → GL ( g ) \operatorname{Ad}: H \to\operatorname{GL}(\mathfrak{g}) Ad : H → GL ( g ) , h ↦ Ad h h \mapsto\operatorname{Ad}_h h ↦ Ad h , defined as the differential of the conjugation map Φ h \Phi_h Φ h at the identity e G e_G e G . For any h ∈ H h \in H h ∈ H and u ∈ g u \in\mathfrak{g} u ∈ g :
Ad h ⋅ u : = ( d Φ h ) e G ( u ) = d d t ∣ t = 0 Φ h ( exp ( t u ) ) ∈ g . \operatorname{Ad}_h \cdot u := (d\Phi_h)_{e_G}(u) = \left.\frac{d}{dt}\right|_{t=0}\Phi_h(\exp(tu)) \in\mathfrak{g}. Ad h ⋅ u := ( d Φ h ) e G ( u ) = d t d t = 0 Φ h ( exp ( t u )) ∈ g .
The trivial representation (or H H H -fixed subspace) g H ⊆ g \mathfrak{g}^{H} \subseteq\mathfrak{g} g H ⊆ g is the linear subspace of tangent directions left invariant by the adjoint action of every element of H H H (called “trivial” in representation theory because every h ∈ H h \in H h ∈ H acts on g H \mathfrak{g}^{H} g H as the identity operator, leaving its vectors unchanged):
g H = { v ∈ g : Ad h ⋅ v = v ∀ h ∈ H } . \mathfrak{g}^{H} = \{v \in\mathfrak{g} : \operatorname{Ad}_h \cdot v = v \ \forall h \in H\}. g H = { v ∈ g : Ad h ⋅ v = v ∀ h ∈ H } .
Compactness of H H H and Ad H \operatorname{Ad}_H Ad H -invariant inner products. Because H H H is assumed to be a compact topological group, it admits a unique normalized Haar probability measure. By averaging any initial inner product ⟨ ⋅ , ⋅ ⟩ 0 \langle\cdot,\cdot\rangle_0 ⟨ ⋅ , ⋅ ⟩ 0 on g \mathfrak{g} g over H H H via ⟨ u , v ⟩ g : = ∫ H ⟨ Ad h u , Ad h v ⟩ 0 d H a a r ( h ) \langle u,v\rangle_{\mathfrak{g}} := \int_H \langle\operatorname{Ad}_h u,\operatorname{Ad}_h v\rangle_0 \, \mathrm{dHaar}(h) ⟨ u , v ⟩ g := ∫ H ⟨ Ad h u , Ad h v ⟩ 0 dHaar ( h ) , we are guaranteed an Ad H \operatorname{Ad}_H Ad H -invariant inner product on g \mathfrak{g} g satisfying
⟨ Ad h u , Ad h v ⟩ g = ⟨ u , v ⟩ g ∀ h ∈ H , ∀ u , v ∈ g . \langle\operatorname{Ad}_h u,\operatorname{Ad}_h v\rangle_{\mathfrak{g}} = \langle u,v\rangle_{\mathfrak{g}} \quad\forall h \in H,\ \forall u,v \in\mathfrak{g}. ⟨ Ad h u , Ad h v ⟩ g = ⟨ u , v ⟩ g ∀ h ∈ H , ∀ u , v ∈ g .
With respect to this inner product, each linear operator Ad h : g → g \operatorname{Ad}_h : \mathfrak{g} \to\mathfrak{g} Ad h : g → g is orthogonal (Ad h T = Ad h − 1 = Ad h − 1 \operatorname{Ad}_h^{\mathsf{T}} = \operatorname{Ad}_{h}^{-1} = \operatorname{Ad}_{h^{-1}} Ad h T = Ad h − 1 = Ad h − 1 ). Note that for all of our guiding examples, the standard Euclidean inner product on R d \mathbb{R}^d R d and the Frobenius inner product on g l ( d ) \mathfrak{gl}(d) gl ( d ) or s o ( d ) \mathfrak{so}(d) so ( d ) are already Ad H \operatorname{Ad}_H Ad H -invariant without requiring explicit averaging.
Statistical and geometric interpretation of Ad h \operatorname{Ad}_h Ad h and g H \mathfrak{g}^{H} g H . Just as Φ h ( g ) \Phi_h(g) Φ h ( g ) represents the transport g g g viewed in an h h h -transformed coordinate frame, Ad h ⋅ u \operatorname{Ad}_h \cdot u Ad h ⋅ u represents the infinitesimal parameter perturbation u ∈ g u \in\mathfrak{g} u ∈ g rotated by the symmetry h h h . In our two primary classes of transport groups, Ad h \operatorname{Ad}_h Ad h takes a particularly transparent form:
Translation groups (G = ( R d , + ) G = (\mathbb{R}^d,+) G = ( R d , + ) with H H H acting linearly by h ⋅ x = M h x h \cdot x = M_hx h ⋅ x = M h x ): Because Φ h ( μ ) = M h μ \Phi_h(\mu) = M_h\mu Φ h ( μ ) = M h μ is already linear in μ \mu μ , its derivative at 0 0 0 is simply matrix-vector multiplication:
Ad h ⋅ u = M h u ( u ∈ R d ) . (3) \operatorname{Ad}_h \cdot u = M_hu \qquad(u \in\mathbb{R}^d). \tag*{(3)} Ad h ⋅ u = M h u ( u ∈ R d ) . (3)
Matrix groups (G ⊆ GL ( d ) G \subseteq\operatorname{GL}(d) G ⊆ GL ( d ) with H H H acting linearly by M h ∈ O ( d ) M_h \in\operatorname{O}(d) M h ∈ O ( d ) ): Because Φ h ( g ) = M h g M h − 1 \Phi_h(g) = M_h g M_h^{-1} Φ h ( g ) = M h g M h − 1 is matrix conjugation, differentiating t ↦ M h e t U M h − 1 t \mapsto M_h e^{tU} M_h^{-1} t ↦ M h e t U M h − 1 at t = 0 t = 0 t = 0 yields matrix similarity on the Lie algebra:
Ad h ⋅ U = M h U M h − 1 = M h U M h T ( U ∈ g ⊆ R d × d ) . \operatorname{Ad}_h \cdot U = M_h U M_h^{-1} = M_h U M_h^{\mathsf{T}} \qquad(U \in\mathfrak{g} \subseteq\mathbb{R}^{d \times d}). Ad h ⋅ U = M h U M h − 1 = M h U M h T ( U ∈ g ⊆ R d × d ) .
The trivial subspace g H = { u ∈ g : Ad h ⋅ u = u ∀ h ∈ H } \mathfrak{g}^{H} = \{u \in\mathfrak{g} : \operatorname{Ad}_h \cdot u = u \ \forall h \in H\} g H = { u ∈ g : Ad h ⋅ u = u ∀ h ∈ H } is precisely the Lie algebra of the fixed-point subgroup G H = { g ∈ G : Φ h ( g ) = g ∀ h ∈ H } G^{H} = \{g \in G : \Phi_h(g) = g \ \forall h \in H\} G H = { g ∈ G : Φ h ( g ) = g ∀ h ∈ H } introduced in § 4.2 . Statistically, a tangent vector u u u belongs to g H \mathfrak{g}^{H} g H if and only if the one-parameter flow exp ( t u ) \exp(tu) exp ( t u ) commutes with every symmetry h ∈ H h \in H h ∈ H , which means that pushing an H H H -symmetric distribution q 0 q_0 q 0 forward along exp ( t u ) \exp(tu) exp ( t u ) preserves its H H H -symmetry . Conversely, any direction u u u orthogonal to g H \mathfrak{g}^{H} g H breaks the H H H -symmetry of q 0 q_0 q 0 .
Guiding examples: Adjoint action and trivial subspace g H \mathfrak{g}^{H} g H . We now determine g \mathfrak{g} g , the linear operator Ad h \operatorname{Ad}_h Ad h , and the trivial subspace g H \mathfrak{g}^{H} g H for each of the six guiding examples:
Reflection (G = ( R , + ) G = (\mathbb{R}, +) G = ( R , + ) , H = Z / 2 Z = { 1 , − 1 } H = \mathbb{Z}/2\mathbb{Z} = \{1, -1\} H = Z /2 Z = { 1 , − 1 } ): The Lie algebra is g = R \mathfrak{g} = \mathbb{R} g = R , and the non-trivial reflection σ = − 1 \sigma= -1 σ = − 1 acts on tangent shifts u ∈ R u \in\mathbb{R} u ∈ R by sign inversion, Ad σ u = − u \operatorname{Ad}_{\sigma}u = -u Ad σ u = − u . The only shift direction satisfying − u = u -u = u − u = u is u = 0 u = 0 u = 0 , so the trivial subspace vanishes: g H = { 0 } \mathfrak{g}^{H} = \{0\} g H = { 0 } . Every nonzero translation breaks reflection symmetry around the origin.
Euclidean (G = ( R d , + ) G = (\mathbb{R}^{d}, +) G = ( R d , + ) , H = O ( d ) H = \operatorname{O}(d) H = O ( d ) ): The Lie algebra is g = R d \mathfrak{g} = \mathbb{R}^{d} g = R d , and each orthogonal matrix U ∈ O ( d ) U \in\operatorname{O}(d) U ∈ O ( d ) acts on tangent vectors u ∈ R d u \in\mathbb{R}^{d} u ∈ R d by Ad U u = U u \operatorname{Ad}_{U}u = Uu Ad U u = U u . Because no nonzero vector in R d \mathbb{R}^{d} R d is invariant under all orthogonal transformations, the trivial subspace is again trivial: g H = { 0 } \mathfrak{g}^{H} = \{0\} g H = { 0 } .
Sphere (G = SO ( 3 ) G = \operatorname{SO}(3) G = SO ( 3 ) , H = SO ( 2 ) H = \operatorname{SO}(2) H = SO ( 2 ) fixing e 3 e_3 e 3 ): The Lie algebra g = s o ( 3 ) \mathfrak{g} = \mathfrak{so}(3) g = so ( 3 ) of 3 × 3 3 \times3 3 × 3 skew-symmetric matrices can be identified with R 3 \mathbb{R}^{3} R 3 via the hat isomorphism ω ∈ R 3 ↔ [ ω ] × ∈ s o ( 3 ) \omega\in\mathbb{R}^{3} \leftrightarrow[\omega]_{\times} \in\mathfrak{so}(3) ω ∈ R 3 ↔ [ ω ] × ∈ so ( 3 ) , where [ ω ] × x = ω × x [\omega]_{\times}x = \omega\times x [ ω ] × x = ω × x . Under this identification, matrix conjugation Ad h ( [ ω ] × ) = R α e 3 [ ω ] × ( R α e 3 ) T = [ R α e 3 ω ] × \operatorname{Ad}_{h}([\omega]_{\times}) = R_{\alpha}^{e_3}[\omega]_{\times}(R_{\alpha}^{e_3})^{\mathsf{T}} = [R_{\alpha}^{e_3}\omega]_{\times} Ad h ([ ω ] × ) = R α e 3 [ ω ] × ( R α e 3 ) T = [ R α e 3 ω ] × acts on the instantaneous rotation axis ω ∈ R 3 \omega\in\mathbb{R}^{3} ω ∈ R 3 simply by rotating ω \omega ω around e 3 e_3 e 3 by angle α \alpha α . The vectors invariant under all rotations around e 3 e_3 e 3 are those parallel to e 3 e_3 e 3 itself, yielding a one-dimensional trivial subspace:
g H = span ( [ e 3 ] × ) ≅ span ( e 3 ) . \mathfrak{g}^{H} = \operatorname{span}([\!e_3\!]_{\times}) \cong\operatorname{span}(e_3). g H = span ([ e 3 ] × ) ≅ span ( e 3 ) .
Note that this subspace coincides with the Lie algebra n = s o ( 2 ) \mathfrak{n} = \mathfrak{so}(2) n = so ( 2 ) of the stabilizer N = SO ( 2 ) N = \operatorname{SO}(2) N = SO ( 2 ) of q 0 q_0 q 0 .
Covariance (G = GL ( d ) G = \operatorname{GL}(d) G = GL ( d ) , H = O ( d ) H = \operatorname{O}(d) H = O ( d ) ): The Lie algebra is the space of all d × d d \times d d × d matrices g = g l ( d ) = R d × d \mathfrak{g} = \mathfrak{gl}(d) = \mathbb{R}^{d \times d} g = gl ( d ) = R d × d , on which U ∈ O ( d ) U \in\operatorname{O}(d) U ∈ O ( d ) acts by orthogonal similarity Ad U ( M ) = U M U T \operatorname{Ad}_{U}(M) = UMU^{\mathsf{T}} Ad U ( M ) = U M U T . A matrix M ∈ g l ( d ) M \in\mathfrak{gl}(d) M ∈ gl ( d ) belongs to g H \mathfrak{g}^{H} g H if and only if U M U T = M UMU^{\mathsf{T}} = M U M U T = M (i.e., U M = M U UM = MU U M = M U ) for all U ∈ O ( d ) U \in\operatorname{O}(d) U ∈ O ( d ) . By Schur’s lemma, the only matrices commuting with all orthogonal transformations are scalar multiples of the identity matrix:
g H = span ( I d ) = { λ I d : λ ∈ R } . \mathfrak{g}^{H} = \operatorname{span}(I_d) = \{\lambda I_d : \lambda\in\mathbb{R}\}. g H = span ( I d ) = { λ I d : λ ∈ R } .
Statistically, span ( I d ) \operatorname{span}(I_d) span ( I d ) generates the one-parameter subgroup exp ( t I d ) = e t I d \exp(tI_d) = e^t I_d exp ( t I d ) = e t I d of isotropic scalings, which dilate or contract the overall variance of an O ( d ) \operatorname{O}(d) O ( d ) -symmetric distribution without breaking its spherical symmetry.
Square (G = SO ( 2 ) G = \operatorname{SO}(2) G = SO ( 2 ) , H = D 4 H = D_4 H = D 4 ): The Lie algebra of planar rotations is one-dimensional, g = s o ( 2 ) ≅ R \mathfrak{g} = \mathfrak{so}(2) \cong\mathbb{R} g = so ( 2 ) ≅ R , parameterized by the angular velocity θ ˙ ∈ R \dot{\theta} \in\mathbb{R} θ ˙ ∈ R . Differentiating Φ r k ( R θ ) = R θ \Phi_{r_k}(R_\theta) = R_\theta Φ r k ( R θ ) = R θ and Φ s k ( R θ ) = R − θ \Phi_{s_k}(R_\theta) = R_{-\theta} Φ s k ( R θ ) = R − θ at θ = 0 \theta= 0 θ = 0 shows that rotations r k ∈ C 4 r_k \in C_4 r k ∈ C 4 act trivially (Ad r k θ ˙ = θ ˙ \operatorname{Ad}_{r_k}\dot{\theta} = \dot{\theta} Ad r k θ ˙ = θ ˙ ), whereas reflections s k ∈ D 4 ∖ C 4 s_k \in D_4 \setminus C_4 s k ∈ D 4 ∖ C 4 reverse the angular velocity (Ad s k θ ˙ = − θ ˙ \operatorname{Ad}_{s_k}\dot{\theta} = -\dot{\theta} Ad s k θ ˙ = − θ ˙ ). Because an invariant direction must be fixed by all elements of D 4 D_4 D 4 including reflections, we obtain g H = { 0 } \mathfrak{g}^{H} = \{0\} g H = { 0 } .
Permutation (G = ( R n , + ) G = (\mathbb{R}^{n}, +) G = ( R n , + ) , H = S n H = S_n H = S n ): The Lie algebra is g = R n \mathfrak{g} = \mathbb{R}^{n} g = R n , and each permutation σ ∈ S n \sigma\in S_n σ ∈ S n acts on tangent shift vectors u ∈ R n u \in\mathbb{R}^{n} u ∈ R n by permuting their coordinates, Ad σ u = P σ u \operatorname{Ad}_{\sigma}u = P_{\sigma}u Ad σ u = P σ u . Under the standard representation of S n S_n S n , R n \mathbb{R}^{n} R n decomposes orthogonally into the one-dimensional trivial representation
of constant vectors and the ( n − 1 ) (n-1) ( n − 1 ) -dimensional standard irreducible representation of zero-sum vectors:
g H = { u ∈ R n : P σ u = u ∀ σ ∈ S n } = span ( 1 ) , ( g H ) ⊥ = { u ∈ R n : ∑ i = 1 n u i = 0 } . \mathfrak{g}^{H}=\{u\in\mathbb{R}^{n}:P_{\sigma}u=u\ \forall\sigma\in\mathbb{S}_{n}\}=\operatorname{span}(1),\qquad(\mathfrak{g}^{H})^{\perp}=\left\{u\in\mathbb{R}^{n}:\sum_{i=1}^{n}u_i=0\right\}. g H = { u ∈ R n : P σ u = u ∀ σ ∈ S n } = span ( 1 ) , ( g H ) ⊥ = { u ∈ R n : i = 1 ∑ n u i = 0 } .
Statistically, perturbations along span ( 1 ) \operatorname{span}(1) span ( 1 ) shift all coordinates by a common scalar additive constant, preserving exchangeability, whereas perturbations in ( g H ) ⊥ (\mathfrak{g}^{H})^{\perp} ( g H ) ⊥ alter the relative offsets between coordinates and break exchangeability.
As we prove in § 6 , the Riemannian gradient ∇ G L ( g ∗ ) \nabla_{G}\mathcal{L}(g^{*}) ∇ G L ( g ∗ ) is forced to lie inside g H \mathfrak{g}^{H} g H . Whenever g H = { 0 } \mathfrak{g}^{H}=\{0\} g H = { 0 } (Examples 1, 2, and 5), first-order stationarity ∇ G L ( g ∗ ) = 0 \nabla_{G}\mathcal{L}(g^{*})=0 ∇ G L ( g ∗ ) = 0 follows immediately from symmetry alone; when g H ≠ { 0 } \mathfrak{g}^{H}\ne\{0\} g H = { 0 } (Examples 3, 4, and 6), the interplay between g H \mathfrak{g}^{H} g H and the identifiability kernel n \mathfrak{n} n determines whether any nonzero gradient component can survive.
Main results: symmetry constraints, Lie algebra decomposition, and moment matching
We now establish our main theoretical results characterizing the first-order optimization landscape of the variational objective L ( g ) = KL ( q g ∥ p ) \mathcal{L}(g)=\operatorname{KL}(q_{g}\mathbin{\|}p) L ( g ) = KL ( q g ∥ p ) at the true transport parameter g ∗ g^{*} g ∗ . Geometrically, our results rest on a fundamental principle of invariant analysis: the gradient of a group-invariant functional at a symmetric point must itself be invariant under the group action. To picture this concretely, consider a rotationally invariant (radial) function on R d \mathbb{R}^{d} R d : computing its gradient at a rotated point on the same circular level set yields an equally rotated gradient vector, and at the center of symmetry itself (the origin), the gradient vector must remain unchanged under every rotation—which forces it to be the zero vector. When translated to the Lie group setting via the pullback loss around g ∗ g^{*} g ∗ , this principle implies that the Riemannian gradient ∇ G L ( g ∗ ) \nabla_{G}\mathcal{L}(g^{*}) ∇ G L ( g ∗ ) is fixed by the adjoint action Ad h \operatorname{Ad}_{h} Ad h for every h ∈ H h\in H h ∈ H , and therefore must reside in the trivial representation g H \mathfrak{g}^{H} g H .
Warm-up: Gradient constraint for translation families
To build concrete statistical intuition before treating general Lie groups, we first state and prove the gradient constraint for additive location families G = ( R d , + ) G=(\mathbb{R}^{d},+) G = ( R d , + ) , where the Lie algebra is R d \mathbb{R}^{d} R d and the Riemannian gradient reduces to the standard Euclidean gradient. This proposition generalizes the univariate reflection argument of Margossian and Saul (2025) (where d = 1 d=1 d = 1 and H = { ± 1 } H=\{\pm1\} H = { ± 1 } ) to arbitrary linear isometry groups H ⊆ O ( d ) H\subseteq O(d) H ⊆ O ( d ) in d d d dimensions, and illustrates how the expected score of the standardized target interacts with H H H -invariance.
Proposition 6.1 (Gradient constraint — translation case). In the setting of Definition 3.3 , suppose that G = ( R d , + ) G=(\mathbb{R}^{d},+) G = ( R d , + ) acts on X = R d \mathcal{X}=\mathbb{R}^{d} X = R d by translation g ⋅ x = x + μ g g\cdot x=x+\mu_{g} g ⋅ x = x + μ g , and that H H H acts linearly by orthogonal transformations h ⋅ x = M h x h\cdot x=M_{h}x h ⋅ x = M h x . Then the Euclidean gradient of L ( μ ) = KL ( q μ ∥ p ) \mathcal{L}(\mu)=\operatorname{KL}(q_{\mu}\mathbin{\|}p) L ( μ ) = KL ( q μ ∥ p ) at the true location parameter μ ∗ \mu^{*} μ ∗ satisfies
∇ L ( μ ∗ ) = M h ∇ L ( μ ∗ ) , ∀ h ∈ H . \nabla\mathcal{L}(\mu^{*})=M_{h}\nabla\mathcal{L}(\mu^{*}),\qquad\forall h\in H. ∇ L ( μ ∗ ) = M h ∇ L ( μ ∗ ) , ∀ h ∈ H . Equivalently, ∇ L ( μ ∗ ) \nabla\mathcal{L}(\mu^{*}) ∇ L ( μ ∗ ) lies in the H H H -fixed subspace ( R d ) H = { v ∈ R d : M h v = v ∀ h ∈ H } (\mathbb{R}^{d})^{H}=\{v\in\mathbb{R}^{d}:M_{h}v=v\ \forall h\in H\} ( R d ) H = { v ∈ R d : M h v = v ∀ h ∈ H } .
Proof. Applying the translation change of variables x = μ + ξ x=\mu+\xi x = μ + ξ (with unit Jacobian determinant), the variational objective at any location μ ∈ R d \mu\in\mathbb{R}^{d} μ ∈ R d can be written as an expectation under the fixed base distribution ξ ∼ q 0 \xi\sim q_{0} ξ ∼ q 0 :
L ( μ ) = ∫ R d q 0 ( ξ ) [ log q 0 ( ξ ) − log p ( μ + ξ ) ] d ξ . (4) \mathcal{L}(\mu)=\int_{\mathbb{R}^{d}}q_{0}(\xi)\left[\log q_{0}(\xi)-\log p(\mu+\xi)\right]\mathrm{d}\xi. \tag*{(4)} L ( μ ) = ∫ R d q 0 ( ξ ) [ log q 0 ( ξ ) − log p ( μ + ξ ) ] d ξ . (4)
Differentiating with respect to μ \mu μ under the integral sign and evaluating at μ = μ ∗ \mu=\mu^* μ = μ ∗ , where p ( μ ∗ + ξ ) = p ~ ( ξ ) p(\mu^*+\xi)=\tilde{p}(\xi) p ( μ ∗ + ξ ) = p ~ ( ξ ) by definition of the standardized pullback p ~ \tilde{p} p ~ , yields the expected score of p ~ \tilde{p} p ~ under q 0 q_0 q 0 :
∇ G L ( μ ∗ ) = ∇ L ( μ ∗ ) = − ∫ R d q 0 ( ξ ) ∇ μ [ log p ( μ ∗ + ξ ) ] d ξ = − ∫ R d q 0 ( ξ ) ∇ log p ~ ( ξ ) d ξ . \nabla_G\mathcal{L}(\mu^*)=\nabla\mathcal{L}(\mu^*)=-\int_{\mathbb{R}^d}q_0(\xi)\nabla_\mu\left[\log p(\mu^*+\xi)\right]\,\mathrm{d}\xi=-\int_{\mathbb{R}^d}q_0(\xi)\nabla\log\tilde{p}(\xi)\,\mathrm{d}\xi. ∇ G L ( μ ∗ ) = ∇ L ( μ ∗ ) = − ∫ R d q 0 ( ξ ) ∇ μ [ log p ( μ ∗ + ξ ) ] d ξ = − ∫ R d q 0 ( ξ ) ∇ log p ~ ( ξ ) d ξ .
Because p ~ \tilde{p} p ~ is H H H -symmetric (p ~ ( M h ξ ′ ) = p ~ ( ξ ′ ) \tilde{p}(M_h\xi')=\tilde{p}(\xi') p ~ ( M h ξ ′ ) = p ~ ( ξ ′ ) ), the chain rule implies that its score function is H H H -equivariant: M h T ∇ log p ~ ( M h ξ ′ ) = ∇ log p ~ ( ξ ′ ) M_h^{\mathsf{T}}\nabla\log\tilde{p}(M_h\xi')=\nabla\log\tilde{p}(\xi') M h T ∇ log p ~ ( M h ξ ′ ) = ∇ log p ~ ( ξ ′ ) , or equivalently ∇ log p ~ ( M h ξ ′ ) = M h ∇ log p ~ ( ξ ′ ) \nabla\log\tilde{p}(M_h\xi')=M_h\nabla\log\tilde{p}(\xi') ∇ log p ~ ( M h ξ ′ ) = M h ∇ log p ~ ( ξ ′ ) since M h ∈ O ( d ) M_h\in O(d) M h ∈ O ( d ) is orthogonal. Performing the orthogonal change of variables ξ = M h ξ ′ \xi=M_h\xi' ξ = M h ξ ′ (with ∣ det M h ∣ = 1 |\det M_h|=1 ∣ det M h ∣ = 1 ) and using the H H H -invariance q 0 ( M h ξ ′ ) = q 0 ( ξ ′ ) q_0(M_h\xi')=q_0(\xi') q 0 ( M h ξ ′ ) = q 0 ( ξ ′ ) , we obtain for every h ∈ H h\in H h ∈ H :
∇ L ( μ ∗ ) = − ∫ R d q 0 ( M h ξ ′ ) ∇ log p ~ ( M h ξ ′ ) d ξ ′ = − M h ∫ R d q 0 ( ξ ′ ) ∇ log p ~ ( ξ ′ ) d ξ ′ = M h ∇ L ( μ ∗ ) . \nabla\mathcal{L}(\mu^*)=-\int_{\mathbb{R}^d}q_0(M_h\xi')\nabla\log\tilde{p}(M_h\xi')\,\mathrm{d}\xi'=-M_h\int_{\mathbb{R}^d}q_0(\xi')\nabla\log\tilde{p}(\xi')\,\mathrm{d}\xi'=M_h\nabla\mathcal{L}(\mu^*) . ∇ L ( μ ∗ ) = − ∫ R d q 0 ( M h ξ ′ ) ∇ log p ~ ( M h ξ ′ ) d ξ ′ = − M h ∫ R d q 0 ( ξ ′ ) ∇ log p ~ ( ξ ′ ) d ξ ′ = M h ∇ L ( μ ∗ ) .
Thus v ∗ = ∇ L ( μ ∗ ) v^*=\nabla\mathcal{L}(\mu^*) v ∗ = ∇ L ( μ ∗ ) satisfies M h v ∗ = v ∗ M_hv^*=v^* M h v ∗ = v ∗ for all h ∈ H h\in H h ∈ H , i.e., ∇ L ( μ ∗ ) ∈ ( R d ) H \nabla\mathcal{L}(\mu^*)\in(\mathbb{R}^d)^H ∇ L ( μ ∗ ) ∈ ( R d ) H . In Examples 1 (Reflection) and 2 (Euclidean), ( R d ) H = { 0 } (\mathbb{R}^d)^H=\{0\} ( R d ) H = { 0 } , which immediately forces ∇ L ( μ ∗ ) = 0 \nabla\mathcal{L}(\mu^*)=0 ∇ L ( μ ∗ ) = 0 ; in Example 6 (Permutation), it constrains ∇ L ( μ ∗ ) ∈ span ( 1 ) \nabla\mathcal{L}(\mu^*)\in\operatorname{span}(\mathbf{1}) ∇ L ( μ ∗ ) ∈ span ( 1 ) . □
Gradient constraint: the general Lie group case
We now establish our first main theorem for an arbitrary transport Lie group G G G normalized by a compact symmetry group H H H . In the translation setting of Proposition 6.1 , the matrix M h M_h M h acted directly on the parameter gradient because the conjugation of a translation μ \mu μ by M h M_h M h is simply Φ h ( μ ) = M h μ \Phi_h(\mu)=M_h\mu Φ h ( μ ) = M h μ . For a general Lie group G G G , the natural counterpart of M h M_h M h on the tangent space g = T e G G \mathfrak{g}=T_{e_G}G g = T e G G is the adjoint operator Ad h = ( d Φ h ) e G \operatorname{Ad}_h=(\mathrm{d}\Phi_h)_{e_G} Ad h = ( d Φ h ) e G . Replacing Euclidean invariance with the conjugation invariance of the relative loss established in Proposition 4.3 (iii) yields the general gradient constraint.
Theorem 6.2 (Gradient constraint from symmetry). Under the assumptions of Definition 3.3 , let g \mathfrak{g} g be equipped with an Ad H \operatorname{Ad}_H Ad H -invariant inner product ⟨ ⋅ , ⋅ ⟩ g \langle\cdot,\cdot\rangle_{\mathfrak{g}} ⟨ ⋅ , ⋅ ⟩ g . Then the Riemannian gradient of the variational objective L ( g ) = KL ( q g ∥ p ) \mathcal{L}(g)=\operatorname{KL}(q_g\|p) L ( g ) = KL ( q g ∥ p ) at the true transport parameter g ∗ ∈ G g^*\in G g ∗ ∈ G satisfies
∇ G L ( g ∗ ) = Ad h ⋅ ∇ G L ( g ∗ ) , ∀ h ∈ H . \nabla_G\mathcal{L}(g^*)=\operatorname{Ad}_h\cdot\nabla_G\mathcal{L}(g^*),\qquad\forall h\in H . ∇ G L ( g ∗ ) = Ad h ⋅ ∇ G L ( g ∗ ) , ∀ h ∈ H . Equivalently, ∇ G L ( g ∗ ) \nabla_G\mathcal{L}(g^*) ∇ G L ( g ∗ ) is constrained to lie in the trivial representation of H H H in g \mathfrak{g} g :
∇ G L ( g ∗ ) ∈ g H . \nabla_G\mathcal{L}(g^*)\in\mathfrak{g}^H . ∇ G L ( g ∗ ) ∈ g H .
Proof. For any tangent vector u ∈ g u\in\mathfrak{g} u ∈ g , consider the perturbed parameter g ∗ exp ( u ) ∈ G g^*\exp(u)\in G g ∗ exp ( u ) ∈ G . Because the Kullback–Leibler divergence is invariant under simultaneous application of the diffeomorphism ( g ∗ ) − 1 (g^*)^{-1} ( g ∗ ) − 1 to both measures, we can pull the variational objective back to the standardized reference space around the identity e G e_G e G :
L ( g ∗ exp ( u ) ) = KL ( q g ∗ exp ( u ) ∥ p ) = KL ( q exp ( u ) ∥ p ~ ) = : L 0 ( exp ( u ) ) . \mathcal{L}(g^*\exp(u))=\operatorname{KL}\left(q_{g^*\exp(u)}\middle\|p\right)=\operatorname{KL}\left(q_{\exp(u)}\middle\|\tilde{p}\right)=:\mathcal{L}_0(\exp(u)) . L ( g ∗ exp ( u )) = KL ( q g ∗ e x p ( u ) p ) = KL ( q e x p ( u ) p ~ ) =: L 0 ( exp ( u )) .
By Definition 5.1 , the left-trivialized Riemannian gradient of L \mathcal{L} L at g ∗ g^* g ∗ coincides with the Riemannian gradient of the standardized loss L 0 \mathcal{L}_0 L 0 at e G e_G e G , and hence with the Euclidean gradient at 0 ∈ g 0\in\mathfrak{g} 0 ∈ g of the pullback objective J : g → R J:\mathfrak{g}\to\mathbb{R} J : g → R defined by J ( u ) : = L 0 ( exp ( u ) ) = L ( g ∗ exp ( u ) ) J(u):=\mathcal{L}_0(\exp(u))=\mathcal{L}(g^*\exp(u)) J ( u ) := L 0 ( exp ( u )) = L ( g ∗ exp ( u )) .
We first show that the function J : g → R J:\mathfrak{g}\to\mathbb{R} J : g → R is strictly invariant under the adjoint action of H H H , i.e., J ( Ad h ⋅ u ) = J ( u ) J(\operatorname{Ad}_h\cdot u)=J(u) J ( Ad h ⋅ u ) = J ( u ) for all h ∈ H h\in H h ∈ H and u ∈ g u\in\mathfrak{g} u ∈ g :
J ( Ad h ⋅ u ) = L 0 ( exp ( Ad h ⋅ u ) ) = L 0 ( Φ h ( exp ( u ) ) ) ( † ) = KL ( q Φ h ( exp ( u ) ) ∥ p ~ ) = KL ( q h exp ( u ) ∥ p ~ ) ( ∗ ) = KL ( q h exp ( u ) ∥ h # p ~ ) ( ∗ ∗ ) = KL ( q exp ( u ) ∥ p ~ ) = J ( u ) . \begin{aligned}
J(\operatorname{Ad}_h\cdot u)&=\mathcal{L}_0\left(\exp(\operatorname{Ad}_h\cdot u)\right)=\mathcal{L}_0\left(\Phi_h(\exp(u))\right) \qquad(\dagger)\\
&=\operatorname{KL}\left(q_{\Phi_h(\exp(u))}\middle\|\tilde{p}\right)=\operatorname{KL}\left(q_{h\exp(u)}\middle\|\tilde{p}\right) \qquad(\ast)\\
&=\operatorname{KL}\left(q_{h\exp(u)}\middle\|h_{\#}\tilde{p}\right) \qquad(\ast\ast)\\
&=\operatorname{KL}\left(q_{\exp(u)}\middle\|\tilde{p}\right)=J(u).
\end{aligned} J ( Ad h ⋅ u ) = L 0 ( exp ( Ad h ⋅ u ) ) = L 0 ( Φ h ( exp ( u )) ) ( † ) = KL ( q Φ h ( e x p ( u )) p ~ ) = KL ( q h e x p ( u ) p ~ ) ( ∗ ) = KL ( q h e x p ( u ) h # p ~ ) ( ∗ ∗ ) = KL ( q e x p ( u ) p ~ ) = J ( u ) .
Each step is justified by the algebraic and measure-theoretic structure established earlier:
Step (†) (Naturality of the exponential map): exp ( Ad h ⋅ u ) = Φ h ( exp ( u ) ) \exp(\operatorname{Ad}_h \cdot u) = \Phi_h(\exp(u)) exp ( Ad h ⋅ u ) = Φ h ( exp ( u )) . Since Φ h : G → G \Phi_h : G \to G Φ h : G → G is a Lie group automorphism (Proposition 4.2 ), the curve t ↦ Φ h ( exp ( t u ) ) t \mapsto\Phi_h(\exp(tu)) t ↦ Φ h ( exp ( t u )) is a one-parameter subgroup of G G G whose initial velocity at t = 0 t = 0 t = 0 is ( d Φ h ) e G ( u ) = Ad h ⋅ u (d\Phi_h)_{e_G}(u) = \operatorname{Ad}_h \cdot u ( d Φ h ) e G ( u ) = Ad h ⋅ u by Definition 5.2 ; by uniqueness of one-parameter subgroups, Φ h ( exp ( t u ) ) = exp ( t Ad h ⋅ u ) \Phi_h(\exp(tu)) = \exp(t\operatorname{Ad}_h \cdot u) Φ h ( exp ( t u )) = exp ( t Ad h ⋅ u ) .
Step ( ) (H H H -invariance of the pullback p ~ \tilde p p ~ ):** Because p p p has H H H -conjugate symmetry around g ∗ g^* g ∗ , its pullback p ~ \tilde p p ~ satisfies h ♯ p ~ = p ~ h_{\sharp}\tilde p = \tilde p h ♯ p ~ = p ~ , and the final equality follows by applying the diffeomorphism h − 1 h^{-1} h − 1 to both arguments of the KL divergence.
Differentiating the invariance identity J ( Ad h ⋅ u ) = J ( u ) J(\operatorname{Ad}_h \cdot u) = J(u) J ( Ad h ⋅ u ) = J ( u ) at u = 0 u = 0 u = 0 along an arbitrary direction u ∈ g u \in\mathfrak{g} u ∈ g via the first-order expansion (2) gives
⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = ⟨ ∇ J ( 0 ) , Ad h ⋅ u ⟩ g = ⟨ Ad h T ∇ J ( 0 ) , u ⟩ g ∀ u ∈ g . \langle\nabla_G \mathcal{L}(g^*),u\rangle_{\mathfrak{g}} = \langle\nabla J(0),u\rangle_{\mathfrak{g}} = \langle\nabla J(0),\operatorname{Ad}_h \cdot u\rangle_{\mathfrak{g}} = \langle\operatorname{Ad}_h^{\mathsf{T}}\nabla J(0),u\rangle_{\mathfrak{g}} \qquad\forall u \in\mathfrak{g}. ⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = ⟨ ∇ J ( 0 ) , Ad h ⋅ u ⟩ g = ⟨ Ad h T ∇ J ( 0 ) , u ⟩ g ∀ u ∈ g .
Because the inner product ⟨ ⋅ , ⋅ ⟩ g \langle\cdot,\cdot\rangle_{\mathfrak{g}} ⟨ ⋅ , ⋅ ⟩ g is Ad H \operatorname{Ad}_H Ad H -invariant, Ad h \operatorname{Ad}_h Ad h is an orthogonal operator on g \mathfrak{g} g , so Ad h T = Ad h − 1 = Ad h − 1 \operatorname{Ad}_h^{\mathsf{T}} = \operatorname{Ad}_h^{-1} = \operatorname{Ad}_{h^{-1}} Ad h T = Ad h − 1 = Ad h − 1 (as is also immediately verified in coordinates: for translation groups Ad h T = M h T = M h − 1 \operatorname{Ad}_h^{\mathsf{T}} = M_h^{\mathsf{T}} = M_h^{-1} Ad h T = M h T = M h − 1 , and for matrix groups under the Frobenius inner product ⟨ Ad h U , V ⟩ = tr ( ( M h U M h T ) T V ) = tr ( U T M h − 1 V M h ) = ⟨ U , Ad h − 1 V ⟩ \langle\operatorname{Ad}_h U,V\rangle= \operatorname{tr}((M_hUM_h^{\mathsf{T}})^{\mathsf{T}}V) = \operatorname{tr}(U^{\mathsf{T}}M_h^{-1}VM_h) = \langle U,\operatorname{Ad}_{h^{-1}}V\rangle ⟨ Ad h U , V ⟩ = tr (( M h U M h T ) T V ) = tr ( U T M h − 1 V M h ) = ⟨ U , Ad h − 1 V ⟩ ). Since u ∈ g u \in\mathfrak{g} u ∈ g is arbitrary, we conclude that ∇ G L ( g ∗ ) = Ad h − 1 ⋅ ∇ G L ( g ∗ ) \nabla_G\mathcal{L}(g^*) = \operatorname{Ad}_{h^{-1}} \cdot\nabla_G\mathcal{L}(g^*) ∇ G L ( g ∗ ) = Ad h − 1 ⋅ ∇ G L ( g ∗ ) for all h ∈ H h \in H h ∈ H . Replacing h h h by h − 1 h^{-1} h − 1 yields ∇ G L ( g ∗ ) ∈ g H \nabla_G\mathcal{L}(g^*) \in\mathfrak{g}^H ∇ G L ( g ∗ ) ∈ g H .
Gradient localization and orthogonal Lie algebra decomposition
Theorem 6.2 establishes that the Riemannian gradient ∇ G L ( g ∗ ) \nabla_G\mathcal{L}(g^*) ∇ G L ( g ∗ ) is confined to the trivial representation g H \mathfrak{g}^H g H . At the same time, our identifiability analysis in § 4 showed that whenever the intersection subgroup N = G ∩ H N = G \cap H N = G ∩ H has positive dimension, the variational objective L ( g ∗ exp ( t u ) ) = L ( g ∗ ) \mathcal{L}(g^*\exp(tu)) = \mathcal{L}(g^*) L ( g ∗ exp ( t u )) = L ( g ∗ ) is identically constant along the Lie algebra n = T e N \mathfrak{n} = T_eN n = T e N of N N N . Combining these two independent structural constraints restricts the gradient to the orthogonal complement of n \mathfrak{n} n inside g H \mathfrak{g}^H g H , and induces a canonical three-way orthogonal decomposition of the entire transport Lie algebra g \mathfrak{g} g . From a statistical estimation standpoint, this decomposes all local parameter perturbations around g ∗ g^* g ∗ into three intuitive categories: (1) unidentifiable directions n \mathfrak{n} n that do not change the distribution q g ∗ q_{g^*} q g ∗ at all; (2) symmetry-preserving directions g b i a s H \mathfrak{g}_{\mathrm{bias}}^H g bias H (such as global scale) where mismatch between q 0 q_0 q 0 and p ~ \tilde p p ~ can induce estimation bias; and (3) symmetry-breaking directions g ⊥ \mathfrak{g}^{\perp} g ⊥ (such as location, orientation, or correlation shape) where symmetry forces the gradient at g ∗ g^* g ∗ to vanish.
Proposition 6.3 (Gradient localization and Lie algebra decomposition). Under the setting of Definition 3.3 with an Ad H \operatorname{Ad}_H Ad H -invariant inner product on g \mathfrak{g} g , the transport Lie algebra g \mathfrak{g} g admits the orthogonal direct sum decomposition
g = n ⏟ identifiability kernel (flat) ⊕ g b i a s H ⏟ bias directions ⊕ g ⊥ ⏟ symmetry-breaking (forced to 0) . \mathfrak{g} = \underbrace{\mathfrak{n}}_{\text{identifiability kernel (flat)}} \oplus\underbrace{\mathfrak{g}_{\mathrm{bias}}^H}_{\text{bias directions}} \oplus\underbrace{\mathfrak{g}^{\perp}}_{\text{symmetry-breaking (forced to 0)}}. g = identifiability kernel (flat) n ⊕ bias directions g bias H ⊕ symmetry-breaking (forced to 0) g ⊥ . where the three mutually orthogonal subspaces are defined as follows:
Identifiability kernel n : = { u ∈ g ∣ exp ( t u ) ∈ N = G ∩ H ∀ t ∈ R } \mathfrak{n} := \{u \in\mathfrak{g} \mid\exp(tu) \in N = G \cap H \ \forall t \in\mathbb{R}\} n := { u ∈ g ∣ exp ( t u ) ∈ N = G ∩ H ∀ t ∈ R } : the Lie algebra of the stabilizer intersection N = G ∩ H N = G \cap H N = G ∩ H . Along any direction u ∈ n u \in\mathfrak{n} u ∈ n , the transported distribution q g ∗ exp ( t u ) = q g ∗ q_{g^*\exp(tu)} = q_{g^*} q g ∗ e x p ( t u ) = q g ∗ is unchanged, so L ( g ∗ exp ( t u ) ) = L ( g ∗ ) \mathcal{L}(g^*\exp(tu)) = \mathcal{L}(g^*) L ( g ∗ exp ( t u )) = L ( g ∗ ) is locally constant and proj n ( ∇ G L ( g ∗ ) ) = 0 \operatorname{proj}_{\mathfrak{n}}(\nabla_G\mathcal{L}(g^*)) = 0 proj n ( ∇ G L ( g ∗ )) = 0 .
Bias subspace g b i a s H : = g H ∩ n ⊥ \mathfrak{g}_{\mathrm{bias}}^H := \mathfrak{g}^H \cap\mathfrak{n}^{\perp} g bias H := g H ∩ n ⊥ : the subspace of H H H -invariant tangent directions orthogonal to the identifiability kernel n \mathfrak{n} n . Perturbations along g b i a s H \mathfrak{g}_{\mathrm{bias}}^H g bias H deform the distribution q g ∗ q_{g^*} q g ∗ while preserving its H H H -conjugate symmetry, and represent the only directions along which ∇ G L ( g ∗ ) \nabla_G\mathcal{L}(g^*) ∇ G L ( g ∗ ) can be nonzero.
Symmetry-breaking subspace g ⊥ \mathfrak{g}^{\perp} g ⊥ : g ⊥ : = ( g H ) ⊥ ∩ n ⊥ \mathfrak{g}^{\perp} := (\mathfrak{g}^{H})^{\perp} \cap\mathfrak{n}^{\perp} g ⊥ := ( g H ) ⊥ ∩ n ⊥ : the subspace of identifiable tangent directions that transform non-trivially under Ad H \operatorname{Ad}_{H} Ad H . By Theorem 6.2 , the gradient vanishes identically on this subspace:
proj g ⊥ ( ∇ G L ( g ∗ ) ) = 0. \operatorname{proj}_{\mathfrak{g}^{\perp}}\left(\nabla_{G}\mathcal{L}(g^{*})\right)=0. proj g ⊥ ( ∇ G L ( g ∗ ) ) = 0.
Consequently, the Riemannian gradient at the true transport parameter g ∗ g^{*} g ∗ satisfies
∇ G L ( g ∗ ) ∈ g b i a s H . \nabla_{G}\mathcal{L}(g^{*})\in\mathfrak{g}^{H}_{\mathrm{bias}}. ∇ G L ( g ∗ ) ∈ g bias H .
Proof. By Theorem 6.2 , ∇ G L ( g ∗ ) ∈ g H \nabla_{G}\mathcal{L}(g^{*})\in\mathfrak{g}^{H} ∇ G L ( g ∗ ) ∈ g H . Furthermore, for any u ∈ n u\in\mathfrak{n} u ∈ n and any t ∈ R t\in\mathbb{R} t ∈ R , we have exp ( t u ) ∈ N = G ∩ H ⊆ H \exp(tu)\in N=G\cap H\subseteq H exp ( t u ) ∈ N = G ∩ H ⊆ H . Because q 0 q_{0} q 0 is H H H -invariant, q exp ( t u ) = ( exp ( t u ) ) ♯ q 0 = q 0 q_{\exp(tu)}=(\exp(tu))_{\sharp}q_{0}=q_{0} q e x p ( t u ) = ( exp ( t u ) ) ♯ q 0 = q 0 , which implies q g ∗ exp ( t u ) = q g ∗ q_{g^{*}\exp(tu)}=q_{g^{*}} q g ∗ e x p ( t u ) = q g ∗ and hence L ( g ∗ exp ( t u ) ) = L ( g ∗ ) \mathcal{L}(g^{*}\exp(tu))=\mathcal{L}(g^{*}) L ( g ∗ exp ( t u )) = L ( g ∗ ) for all t ∈ R t\in\mathbb{R} t ∈ R . Differentiating at t = 0 t=0 t = 0 gives ⟨ ∇ G L ( g ∗ ) , u ⟩ g = 0 \langle\nabla_{G}\mathcal{L}(g^{*}),u\rangle_{\mathfrak{g}}=0 ⟨ ∇ G L ( g ∗ ) , u ⟩ g = 0 for all u ∈ n u\in\mathfrak{n} u ∈ n , so ∇ G L ( g ∗ ) ∈ n ⊥ \nabla_{G}\mathcal{L}(g^{*})\in\mathfrak{n}^{\perp} ∇ G L ( g ∗ ) ∈ n ⊥ . Combining both inclusions yields ∇ G L ( g ∗ ) ∈ g H ∩ n ⊥ = g b i a s H \nabla_{G}\mathcal{L}(g^{*})\in\mathfrak{g}^{H}\cap\mathfrak{n}^{\perp}=\mathfrak{g}^{H}_{\mathrm{bias}} ∇ G L ( g ∗ ) ∈ g H ∩ n ⊥ = g bias H .
To prove that g = n ⊕ g b i a s H ⊕ g ⊥ \mathfrak{g}=\mathfrak{n}\oplus\mathfrak{g}^{H}_{\mathrm{bias}}\oplus\mathfrak{g}^{\perp} g = n ⊕ g bias H ⊕ g ⊥ is an orthogonal direct sum spanning g \mathfrak{g} g , note first that g b i a s H ⊆ n ⊥ \mathfrak{g}^{H}_{\mathrm{bias}}\subseteq\mathfrak{n}^{\perp} g bias H ⊆ n ⊥ and g ⊥ ⊆ n ⊥ \mathfrak{g}^{\perp}\subseteq\mathfrak{n}^{\perp} g ⊥ ⊆ n ⊥ are both orthogonal to n \mathfrak{n} n by definition, and g b i a s H ⊆ g H \mathfrak{g}^{H}_{\mathrm{bias}}\subseteq\mathfrak{g}^{H} g bias H ⊆ g H is orthogonal to g ⊥ ⊆ ( g H ) ⊥ \mathfrak{g}^{\perp}\subseteq(\mathfrak{g}^{H})^{\perp} g ⊥ ⊆ ( g H ) ⊥ . Next, observe that N = G ∩ H N=G\cap H N = G ∩ H is normalized by H H H : for any n ∈ N n\in N n ∈ N and h ∈ H h\in H h ∈ H , Φ h ( n ) = h n h − 1 \Phi_{h}(n)=hnh^{-1} Φ h ( n ) = hn h − 1 lies in G G G (since G G G is normalized by H H H ) and in H H H (since H H H is a group), so Φ h ( N ) = N \Phi_{h}(N)=N Φ h ( N ) = N . Differentiating at e G e_{G} e G implies that n \mathfrak{n} n is an Ad H \operatorname{Ad}_{H} Ad H -invariant subspace of g \mathfrak{g} g (Ad h ( n ) = n \operatorname{Ad}_{h}(\mathfrak{n})=\mathfrak{n} Ad h ( n ) = n for all h ∈ H h\in H h ∈ H ; see Appendix § 10 for details). Because n \mathfrak{n} n is Ad H \operatorname{Ad}_{H} Ad H -invariant and the inner product is Ad H \operatorname{Ad}_{H} Ad H -invariant, the orthogonal projection onto g H \mathfrak{g}^{H} g H maps n \mathfrak{n} n into itself, so n \mathfrak{n} n decomposes orthogonally as
n = ( n ∩ g H ) ⊕ ( n ∩ ( g H ) ⊥ ) . \mathfrak{n}=(\mathfrak{n}\cap\mathfrak{g}^{H})\oplus(\mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp}). n = ( n ∩ g H ) ⊕ ( n ∩ ( g H ) ⊥ ) .
Taking orthogonal complements inside g H \mathfrak{g}^{H} g H and ( g H ) ⊥ (\mathfrak{g}^{H})^{\perp} ( g H ) ⊥ respectively gives g H = ( n ∩ g H ) ⊕ g b i a s H \mathfrak{g}^{H}=(\mathfrak{n}\cap\mathfrak{g}^{H})\oplus\mathfrak{g}^{H}_{\mathrm{bias}} g H = ( n ∩ g H ) ⊕ g bias H and ( g H ) ⊥ = ( n ∩ ( g H ) ⊥ ) ⊕ g ⊥ (\mathfrak{g}^{H})^{\perp}=(\mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp})\oplus\mathfrak{g}^{\perp} ( g H ) ⊥ = ( n ∩ ( g H ) ⊥ ) ⊕ g ⊥ . Summing these decompositions over g = g H ⊕ ( g H ) ⊥ \mathfrak{g}=\mathfrak{g}^{H}\oplus(\mathfrak{g}^{H})^{\perp} g = g H ⊕ ( g H ) ⊥ yields g = n ⊕ g b i a s H ⊕ g ⊥ \mathfrak{g}=\mathfrak{n}\oplus\mathfrak{g}^{H}_{\mathrm{bias}}\oplus\mathfrak{g}^{\perp} g = n ⊕ g bias H ⊕ g ⊥ . □ \square □
Guiding examples: Three-way decomposition and statistical interpretation. We now evaluate the orthogonal decomposition g = n ⊕ g b i a s H ⊕ g ⊥ \mathfrak{g}=\mathfrak{n}\oplus\mathfrak{g}^{H}_{\mathrm{bias}}\oplus\mathfrak{g}^{\perp} g = n ⊕ g bias H ⊕ g ⊥ for each of our six guiding examples, illustrating how Proposition 6.3 separates overparametrization, estimation bias, and unbiased recovery:
Reflection (G = ( R , + ) G=(\mathbb{R},+) G = ( R , + ) , H = Z / 2 Z H=\mathbb{Z}/2\mathbb{Z} H = Z /2 Z ): The Lie algebra is g = R \mathfrak{g}=\mathbb{R} g = R . Because N = { 0 } N=\{0\} N = { 0 } is trivial and Ad σ u = − u \operatorname{Ad}_{\sigma}u=-u Ad σ u = − u leaves only 0 0 0 fixed, both the identifiability kernel and the bias subspace vanish:
n = { 0 } , g b i a s H = { 0 } , g ⊥ = R . \mathfrak{n}=\{0\},\qquad\mathfrak{g}^{H}_{\mathrm{bias}}=\{0\},\qquad\mathfrak{g}^{\perp}=\mathbb{R}. n = { 0 } , g bias H = { 0 } , g ⊥ = R .
Every nonzero translation breaks reflection symmetry, so the entire parameter space lies in g ⊥ \mathfrak{g}^{\perp} g ⊥ and ∇ G L ( μ ∗ ) = 0 \nabla_{G}\mathcal{L}(\mu^{*})=0 ∇ G L ( μ ∗ ) = 0 is guaranteed for any symmetric target p p p .
Euclidean (G = ( R d , + ) G=(\mathbb{R}^{d},+) G = ( R d , + ) , H = O ( d ) H=O(d) H = O ( d ) ): The Lie algebra is g = R d \mathfrak{g}=\mathbb{R}^{d} g = R d , with N = { 0 } N=\{0\} N = { 0 } and g H = { 0 } \mathfrak{g}^{H}=\{0\} g H = { 0 } . Thus
n = { 0 } , g b i a s H = { 0 } , g ⊥ = R d . \mathfrak{n}=\{0\},\qquad\mathfrak{g}^{H}_{\mathrm{bias}}=\{0\},\qquad\mathfrak{g}^{\perp}=\mathbb{R}^{d}. n = { 0 } , g bias H = { 0 } , g ⊥ = R d .
All d d d translation coordinates are symmetry-breaking directions, forcing ∇ G L ( μ ∗ ) = 0 \nabla_{G}\mathcal{L}(\mu^{*})=0 ∇ G L ( μ ∗ ) = 0 .
Sphere (G = S O ( 3 ) G=SO(3) G = S O ( 3 ) , H = S O ( 2 ) H=SO(2) H = S O ( 2 ) fixing e 3 e_{3} e 3 ): Under the isomorphism g = s o ( 3 ) ≅ R 3 \mathfrak{g}=\mathfrak{so}(3)\cong\mathbb{R}^{3} g = so ( 3 ) ≅ R 3 , the stabilizer N = S O ( 2 ) N=SO(2) N = S O ( 2 ) has Lie algebra n = span ( e 3 ) \mathfrak{n}=\operatorname{span}(e_{3}) n = span ( e 3 ) , consisting of infinitesimal rotations around the symmetry axis e 3 e_{3} e 3 . As computed in Section 5, the trivial representation is also g H = span ( e 3 ) \mathfrak{g}^{H}=\operatorname{span}(e_{3}) g H = span ( e 3 ) . Because g H \mathfrak{g}^{H} g H is entirely contained within the identifiability kernel n \mathfrak{n} n , the bias subspace vanishes:
n = span ( e 3 ) , g b i a s H = g H ∩ n ⊥ = { 0 } , g ⊥ = span ( e 1 , e 2 ) . \mathfrak{n}=\operatorname{span}(e_{3}),\qquad\mathfrak{g}^{H}_{\mathrm{bias}}=\mathfrak{g}^{H}\cap\mathfrak{n}^{\perp}=\{0\},\qquad\mathfrak{g}^{\perp}=\operatorname{span}(e_{1},e_{2}). n = span ( e 3 ) , g bias H = g H ∩ n ⊥ = { 0 } , g ⊥ = span ( e 1 , e 2 ) .
Geometrically, g ⊥ = span ( e 1 , e 2 ) \mathfrak{g}^{\perp}=\operatorname{span}(e_{1},e_{2}) g ⊥ = span ( e 1 , e 2 ) consists of infinitesimal tilts of the modal axis e 3 e_{3} e 3 toward the equatorial plane (spanning the tangent space T e 3 S 2 T_{e_{3}}S^{2} T e 3 S 2 of the quotient manifold M = S O ( 3 ) / S O ( 2 ) ≅ S 2 \mathcal{M}=SO(3)/SO(2)\cong S^{2} M = S O ( 3 ) / S O ( 2 ) ≅ S 2 ). Symmetry forces the gradient along all equatorial tilts to vanish, while the axial spin direction n \mathfrak{n} n is flat by identifiability, guaranteeing ∇ G L ( R ∗ ) = 0 \nabla_{G}\mathcal{L}(R^{*})=0 ∇ G L ( R ∗ ) = 0 .
Covariance (G = GL ( d ) G = \operatorname{GL}(d) G = GL ( d ) , H = O ( d ) H = \operatorname{O}(d) H = O ( d ) ): The Lie algebra g = g l ( d ) = R d × d \mathfrak{g} = \mathfrak{gl}(d) = \mathbb{R}^{d \times d} g = gl ( d ) = R d × d equipped with the Frobenius inner product decomposes into skew-symmetric matrices and symmetric matrices. The stabilizer N = O ( d ) N = \operatorname{O}(d) N = O ( d ) has Lie algebra n = s o ( d ) = { U ∈ R d × d : U T = − U } \mathfrak{n} = \mathfrak{so}(d) = \{U \in\mathbb{R}^{d \times d}: U^{\mathsf{T}} = -U\} n = so ( d ) = { U ∈ R d × d : U T = − U } , while the trivial subspace is the one-dimensional space of isotropic scalings g H = span ( I d ) \mathfrak{g}^{H} = \operatorname{span}(I_d) g H = span ( I d ) . Since I d I_d I d is symmetric and every matrix in s o ( d ) \mathfrak{so}(d) so ( d ) is skew-symmetric (and traceless), g H \mathfrak{g}^{H} g H is completely orthogonal to n \mathfrak{n} n (tr ( I d T U ) = tr ( U ) = 0 \operatorname{tr}(I_d^{\mathsf{T}}U) = \operatorname{tr}(U) = 0 tr ( I d T U ) = tr ( U ) = 0 for all U ∈ s o ( d ) U \in\mathfrak{so}(d) U ∈ so ( d ) ). We therefore obtain the classical Cartan-type matrix decomposition:
n = s o ( d ) , g b i a s H = span ( I d ) , g ⊥ = Sym 0 ( d ) : = { S ∈ R d × d : S = S T , tr ( S ) = 0 } . \mathfrak{n} = \mathfrak{so}(d), \qquad\mathfrak{g}^{H}_{\mathrm{bias}} = \operatorname{span}(I_d), \qquad\mathfrak{g}^{\perp} = \operatorname{Sym}_0(d) := \{S \in\mathbb{R}^{d \times d}: S = S^{\mathsf{T}}, \operatorname{tr}(S) = 0\}. n = so ( d ) , g bias H = span ( I d ) , g ⊥ = Sym 0 ( d ) := { S ∈ R d × d : S = S T , tr ( S ) = 0 } .
This decomposition has a striking statistical interpretation. The identifiability kernel n = s o ( d ) \mathfrak{n} = \mathfrak{so}(d) n = so ( d ) generates orthogonal right-rotations of A ∗ A^{*} A ∗ that leave the covariance matrix Σ ∗ = A ∗ ( A ∗ ) T \Sigma^{*} = A^{*}(A^{*})^{\mathsf{T}} Σ ∗ = A ∗ ( A ∗ ) T invariant. The symmetry-breaking subspace g ⊥ = Sym 0 ( d ) \mathfrak{g}^{\perp} = \operatorname{Sym}_0(d) g ⊥ = Sym 0 ( d ) of traceless symmetric matrices parameterizes anisotropic shape and correlation deformations (changing the ratios of eigenvalues and orientation of principal axes while preserving volume to first order); along all 1 2 d ( d + 1 ) − 1 \frac{1}{2}d(d+1) - 1 2 1 d ( d + 1 ) − 1 such directions, the gradient at A ∗ A^{*} A ∗ is forced to vanish by elliptical symmetry! However, along the single remaining scalar direction g b i a s H = span ( I d ) \mathfrak{g}^{H}_{\mathrm{bias}} = \operatorname{span}(I_d) g bias H = span ( I d ) , the gradient satisfies ∇ G L ( A ∗ ) = λ I d \nabla_G \mathcal{L}(A^{*}) = \lambda I_d ∇ G L ( A ∗ ) = λ I d for some scalar λ ∈ R \lambda\in\mathbb{R} λ ∈ R , corresponding to a multiplicative global scale bias between the dispersion of q 0 q_0 q 0 and that of p ~ \widetilde{p} p .
Square (G = SO ( 2 ) G = \operatorname{SO}(2) G = SO ( 2 ) , H = D 4 H = D_4 H = D 4 ): The Lie algebra is g = s o ( 2 ) ≅ R \mathfrak{g} = \mathfrak{so}(2) \cong\mathbb{R} g = so ( 2 ) ≅ R . Because the overlap subgroup N = C 4 N = C_4 N = C 4 is discrete, its Lie algebra is zero-dimensional (n = { 0 } \mathfrak{n} = \{0\} n = { 0 } ). At the same time, the reflections in D 4 D_4 D 4 force g H = { 0 } \mathfrak{g}^{H} = \{0\} g H = { 0 } . Consequently,
n = { 0 } , g b i a s H = { 0 } , g ⊥ = R . \mathfrak{n} = \{0\}, \qquad\mathfrak{g}^{H}_{\mathrm{bias}} = \{0\}, \qquad\mathfrak{g}^{\perp} = \mathbb{R}. n = { 0 } , g bias H = { 0 } , g ⊥ = R .
Every infinitesimal planar rotation breaks the reflection symmetry of the square, so ∇ G L ( R θ ∗ ) = 0 \nabla_G \mathcal{L}(R_{\theta^{*}}) = 0 ∇ G L ( R θ ∗ ) = 0 is guaranteed by D 4 D_4 D 4 -invariance.
Permutation (G = ( R n , + ) G = (\mathbb{R}^{n}, +) G = ( R n , + ) , H = S n H = \mathbb{S}_n H = S n ): The Lie algebra is g = R n \mathfrak{g} = \mathbb{R}^{n} g = R n with trivial intersection N = { 0 } N = \{0\} N = { 0 } , so n = { 0 } \mathfrak{n} = \{0\} n = { 0 } . The trivial representation of S n \mathbb{S}_n S n is the one-dimensional diagonal subspace g H = span ( 1 ) \mathfrak{g}^{H} = \operatorname{span}(\mathbf{1}) g H = span ( 1 ) , which therefore coincides entirely with the bias subspace:
n = { 0 } , g b i a s H = span ( 1 ) , g ⊥ = { u ∈ R n : ∑ i = 1 n u i = 0 } . \mathfrak{n} = \{0\}, \qquad\mathfrak{g}^{H}_{\mathrm{bias}} = \operatorname{span}(\mathbf{1}), \qquad\mathfrak{g}^{\perp} = \left\{u \in\mathbb{R}^{n}: \sum_{i=1}^{n} u_i = 0\right\}. n = { 0 } , g bias H = span ( 1 ) , g ⊥ = { u ∈ R n : i = 1 ∑ n u i = 0 } .
Statistically, the ( n − 1 ) (n - 1) ( n − 1 ) -dimensional symmetry-breaking subspace g ⊥ \mathfrak{g}^{\perp} g ⊥ of zero-sum vectors governs all relative contrasts μ i − μ j \mu_i - \mu_j μ i − μ j between coordinates: exchangeability forces the gradient at μ ∗ \mu^{*} μ ∗ to vanish along every contrast vector u ∈ g ⊥ u \in\mathfrak{g}^{\perp} u ∈ g ⊥ , ensuring that relative coordinate differences are estimated without bias. By contrast, along the common-location direction g b i a s H = span ( 1 ) \mathfrak{g}^{H}_{\mathrm{bias}} = \operatorname{span}(\mathbf{1}) g bias H = span ( 1 ) , the gradient takes the form ∇ G L ( μ ∗ ) = λ 1 \nabla_G \mathcal{L}(\mu^{*}) = \lambda\mathbf{1} ∇ G L ( μ ∗ ) = λ 1 for some λ ∈ R \lambda\in\mathbb{R} λ ∈ R , representing a uniform additive location bias across all coordinates if q 0 q_0 q 0 and p ~ \widetilde{p} p are not centered at the same scalar mean/median level.
Proposition 6.3 unifies the phenomena observed in Margossian and Saul (2025) and extends them across general Lie groups. Whenever g b i a s H = { 0 } \mathfrak{g}^{H}_{\mathrm{bias}} = \{0\} g bias H = { 0 } —either because g H = { 0 } \mathfrak{g}^{H} = \{0\} g H = { 0 } (Reflection, Euclidean, Square) or because g H = n \mathfrak{g}^{H} = \mathfrak{n} g H = n is absorbed into the stabilizer of q 0 q_0 q 0 (Sphere)—symmetry alone guarantees ∇ G L ( g ∗ ) = 0 \nabla_G \mathcal{L}(g^{*}) = 0 ∇ G L ( g ∗ ) = 0 . Conversely, when g b i a s H ≠ { 0 } \mathfrak{g}^{H}_{\mathrm{bias}} \ne\{0\} g bias H = { 0 } (Covariance and Permutation), symmetry eliminates all gradient components in g ⊥ \mathfrak{g}^{\perp} g ⊥ while isolating any potential misspecification bias to the low-dimensional subspace g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H . Table 2 summarizes the algebraic invariants and decomposition across all six guiding examples.
Table 2. Algebraic invariants and Lie algebra decomposition g = n ⊕ g b i a s H ⊕ g ⊥ \mathfrak{g}=\mathfrak{n}\oplus\mathfrak{g}^{H}_{\mathrm{bias}}\oplus\mathfrak{g}^{\perp} g = n ⊕ g bias H ⊕ g ⊥ for the six guiding examples. The final column indicates whether ∇ G L ( g ∗ ) = 0 \nabla_{G}\mathcal{L}(g^{*})=0 ∇ G L ( g ∗ ) = 0 is guaranteed by symmetry alone (g b i a s H = { 0 } \mathfrak{g}^{H}_{\mathrm{bias}}=\{0\} g bias H = { 0 } ).
Bias from model mismatch and the Stein moment-matching condition
When the bias subspace g b i a s H = g H ∩ n ⊥ \mathfrak{g}^{H}_{\mathrm{bias}} = \mathfrak{g}^{H} \cap\mathfrak{n}^{\perp} g bias H = g H ∩ n ⊥ is non-trivial, Proposition 6.3 shows that ∇ G L ( g ∗ ) \nabla_G \mathcal{L}(g^{*}) ∇ G L ( g ∗ ) lies in g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H but need not vanish. Statistically, a nonzero gradient component along g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H arises because perturbations exp ( t u ) \exp(tu) exp ( t u ) with u ∈ g b i a s H u\in\mathfrak{g}^{H}_{\mathrm{bias}} u ∈ g bias H transform the base distribution q 0 q_{0} q 0 into a distinct H H H -symmetric distribution q exp ( t u ) q_{\exp(tu)} q e x p ( t u ) . If the standardized target p ~ \widetilde{p} p and the base q 0 q_{0} q 0 differ in their scale or position along this symmetry-preserving subgroup exp ( g b i a s H ) \exp(\mathfrak{g}^{H}_{\mathrm{bias}}) exp ( g bias H ) , the KL divergence pulls the variational parameter away from g ∗ g^{*} g ∗ along g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H to compensate for the mismatch between q 0 q_{0} q 0 and p ~ \widetilde{p} p .
Illustrative examples of estimation bias under model mismatch. Before deriving the general analytical criterion for when the bias vanishes, we illustrate how mismatch along g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H manifests in three concrete settings:
Multiplicative scale bias in covariance estimation (Example 4): Suppose the base distribution is q 0 = N ( 0 , I d ) q_{0}=\mathcal{N}(0,I_{d}) q 0 = N ( 0 , I d ) , while the standardized pullback of the target is p ~ = N ( 0 , σ 2 I d ) \widetilde{p}=\mathcal{N}(0,\sigma^{2}I_{d}) p = N ( 0 , σ 2 I d ) with σ ≠ 1 \sigma\ne1 σ = 1 (so that p = N ( 0 , σ 2 A ∗ ( A ∗ ) T ) p=\mathcal{N}(0,\sigma^{2}A^{*}(A^{*})^{\mathsf{T}}) p = N ( 0 , σ 2 A ∗ ( A ∗ ) T ) ). Both q 0 q_{0} q 0 and p ~ \widetilde{p} p are O ( d ) O(d) O ( d ) -symmetric, yet their radial scales differ by σ \sigma σ . Consequently, L ( A ) \mathcal{L}(A) L ( A ) is minimized at A ^ = σ A ∗ \widehat{A}=\sigma A^{*} A = σ A ∗ rather than A ∗ A^{*} A ∗ , and the gradient at A ∗ A^{*} A ∗ has a nonzero component ∇ G L ( A ∗ ) = ( σ − 2 − 1 ) I d ∈ span ( I d ) = g b i a s H \nabla_{G}\mathcal{L}(A^{*})=(\sigma^{-2}-1)I_{d}\in\operatorname{span}(I_{d})=\mathfrak{g}^{H}_{\mathrm{bias}} ∇ G L ( A ∗ ) = ( σ − 2 − 1 ) I d ∈ span ( I d ) = g bias H (so that the negative gradient − ∇ G L ( A ∗ ) = ( 1 − σ − 2 ) I d -\nabla_{G}\mathcal{L}(A^{*})=(1-\sigma^{-2})I_{d} − ∇ G L ( A ∗ ) = ( 1 − σ − 2 ) I d drives A A A toward σ A ∗ \sigma A^{*} σ A ∗ ).
Common-location additive bias under exchangeability (Example 6): Suppose the base distribution is q 0 ( x ) ∝ exp ( − κ ∥ x ∥ 1 ) q_{0}(x)\propto\exp(-\kappa\lVert x\rVert_{1}) q 0 ( x ) ∝ exp ( − κ ∥ x ∥ 1 ) (centered at the origin), while the standardized pullback is p ~ ( x ) ∝ exp ( − κ ∥ x − β 1 ∥ 1 ) \widetilde{p}(x)\propto\exp(-\kappa\lVert x-\beta\mathbf{1}\rVert_{1}) p ( x ) ∝ exp ( − κ ∥ x − β 1 ∥ 1 ) with β ≠ 0 \beta\ne0 β = 0 . Because shifting all coordinates by β 1 \beta\mathbf{1} β 1 commutes with all coordinate permutations in S n \mathbb{S}_{n} S n , p ~ \widetilde{p} p remains fully S n \mathbb{S}_{n} S n -symmetric! However, q 0 q_{0} q 0 and p ~ \widetilde{p} p have different common coordinate medians (0 0 0 versus β \beta β ), so L ( μ ) \mathcal{L}(\mu) L ( μ ) is minimized at μ ^ = μ ∗ + β 1 \widehat{\mu}=\mu^{*}+\beta\mathbf{1} μ = μ ∗ + β 1 rather than μ ∗ \mu^{*} μ ∗ , producing a nonzero gradient at μ ∗ \mu^{*} μ ∗ along span ( 1 ) = g b i a s H \operatorname{span}(\mathbf{1})=\mathfrak{g}^{H}_{\mathrm{bias}} span ( 1 ) = g bias H .
Angular drift under chiral C 4 C_{4} C 4 symmetry (Modified Example 5): Suppose we weaken the symmetry of the standardized target p ~ \widetilde{p} p in Example 5 from the full dihedral group D 4 D_{4} D 4 to the pure rotational subgroup H ′ = C 4 H'=C_{4} H ′ = C 4 (eliminating reflections) by setting p ~ ( x ) = q 0 ( R α T x ) \widetilde{p}(x)=q_{0}(R_{\alpha}^{\mathsf{T}}x) p ( x ) = q 0 ( R α T x ) for an angle α ∉ ( π / 4 ) Z \alpha\notin(\pi/4)\mathbb{Z} α ∈ / ( π /4 ) Z . Because C 4 ⊂ S O ( 2 ) C_{4}\subset SO(2) C 4 ⊂ S O ( 2 ) is abelian, conjugation by r k ∈ C 4 r_{k}\in C_{4} r k ∈ C 4 acts trivially on S O ( 2 ) SO(2) S O ( 2 ) , so Ad r k = id \operatorname{Ad}_{r_{k}}=\operatorname{id} Ad r k = id and the trivial subspace expands from { 0 } \{0\} { 0 } to the entire Lie algebra: g b i a s C 4 = g = s o ( 2 ) ≅ R \mathfrak{g}^{C_{4}}_{\mathrm{bias}}=\mathfrak{g}=\mathfrak{so}(2)\cong\mathbb{R} g bias C 4 = g = so ( 2 ) ≅ R . With reflections removed, symmetry imposes no constraint on the rotational gradient, and the variational minimizer shifts to θ ∗ + α ( m o d π / 2 ) \theta^{*}+\alpha\pmod{\pi/2} θ ∗ + α ( mod π /2 ) .
Infinitesimal generators and the Stein moment-matching condition. To characterize precisely when ∇ G L ( g ∗ ) = 0 \nabla_{G}\mathcal{L}(g^{*})=0 ∇ G L ( g ∗ ) = 0 in the presence of a non-trivial bias subspace g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H , we express the directional derivative of the pullback loss J ( u ) = KL ( q exp ( u ) ∥ p ~ ) J(u)=\operatorname{KL}(q_{\exp(u)}\Vert\widetilde{p}) J ( u ) = KL ( q e x p ( u ) ∥ p ) at u = 0 u=0 u = 0 in terms of the log-density potentials of q 0 q_{0} q 0 and p ~ \widetilde{p} p . Write
q 0 ( z ) ∝ exp ( − ψ ( z ) ) , p ~ ( z ) ∝ exp ( − φ ( z ) ) , q_{0}(z)\propto\exp(-\psi(z)),\qquad\widetilde{p}(z)\propto\exp(-\varphi(z)), q 0 ( z ) ∝ exp ( − ψ ( z )) , p ( z ) ∝ exp ( − φ ( z )) ,
where ψ , φ : X → R \psi,\varphi:\mathcal{X}\to\mathbb{R} ψ , φ : X → R are smooth H H H -invariant potential functions. Describing how an infinitesimal transport u ∈ g u\in\mathfrak{g} u ∈ g deforms points z ∈ X z\in\mathcal{X} z ∈ X requires the fundamental vector field (or velocity field) induced by the group action:
Definition 6.4 (Infinitesimal generator / velocity field). For each tangent direction u ∈ g u \in\mathfrak{g} u ∈ g , the velocity field (or infinitesimal generator ) V u : X → T X V_u:\mathcal{X}\to T\mathcal{X} V u : X → T X is the vector field on X \mathcal{X} X assigning to each point x ∈ X x\in\mathcal{X} x ∈ X its instantaneous velocity under the flow t ↦ exp ( t u ) ⋅ x t\mapsto\exp(tu)\cdot x t ↦ exp ( t u ) ⋅ x :
V u ( x ) : = d d t ∣ t = 0 ( exp ( t u ) ⋅ x ) . V_u(x) := \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\bigl(\exp(tu)\cdot x\bigr). V u ( x ) := d t d t = 0 ( exp ( t u ) ⋅ x ) .
For our primary transport groups, the velocity field V u ( x ) V_u(x) V u ( x ) takes an immediate closed form:
Additive translations (G = ( R d , + ) G=(\mathbb{R}^{d},+) G = ( R d , + ) , exp ( t u ) ⋅ x = x + t u \exp(tu)\cdot x=x+tu exp ( t u ) ⋅ x = x + t u ): V u ( x ) = u V_u(x)=u V u ( x ) = u (a constant vector field on R d \mathbb{R}^{d} R d ).
Linear matrix transformations (G = G L ( d ) G=\mathrm{GL}(d) G = GL ( d ) , exp ( t U ) ⋅ x = e t U x \exp(tU)\cdot x=e^{tU}x exp ( t U ) ⋅ x = e t U x ): V U ( x ) = U x V_U(x)=Ux V U ( x ) = U x (a linear vector field on R d \mathbb{R}^{d} R d ).
Rotations on the sphere (G = S O ( 3 ) G=\mathrm{SO}(3) G = SO ( 3 ) , g ≅ R 3 \mathfrak{g}\cong\mathbb{R}^{3} g ≅ R 3 ): V ω ( x ) = ω × x V_\omega(x)=\omega\times x V ω ( x ) = ω × x (the tangential rotational velocity field on S 2 S^{2} S 2 ).
When a random variable Z ∼ q 0 Z\sim q_0 Z ∼ q 0 is pushed forward along the flow x ( t ) = exp ( t u ) ⋅ Z x(t)=\exp(tu)\cdot Z x ( t ) = exp ( t u ) ⋅ Z , the KL divergence J ( t u ) = KL ( q exp ( t u ) ∥ p ~ ) = E Z ∼ q 0 [ log q 0 ( Z ) − log p ~ ( exp ( t u ) ⋅ Z ) − log ∣ det J exp ( t u ) ( Z ) ∣ ] J(tu)=\operatorname{KL}(q_{\exp(tu)}\mathbin\Vert\widetilde{p})=\mathbb{E}_{Z\sim q_0}\left[\log q_0(Z)-\log\widetilde{p}(\exp(tu)\cdot Z)-\log\left|\det J_{\exp(tu)}(Z)\right|\right] J ( t u ) = KL ( q e x p ( t u ) ∥ p ) = E Z ∼ q 0 [ log q 0 ( Z ) − log p ( exp ( t u ) ⋅ Z ) − log det J e x p ( t u ) ( Z ) ] varies at first order through two distinct mechanisms:
(i) Cross-entropy / target potential variation: Differentiating − log p ~ ( exp ( t u ) ⋅ Z ) = φ ( exp ( t u ) ⋅ Z ) + c o n s t -\log\widetilde{p}(\exp(tu)\cdot Z)=\varphi(\exp(tu)\cdot Z)+\mathrm{const} − log p ( exp ( t u ) ⋅ Z ) = φ ( exp ( t u ) ⋅ Z ) + const at t = 0 t=0 t = 0 yields the directional derivative of the target potential along the velocity field, E q 0 [ ⟨ ∇ φ ( Z ) , V u ( Z ) ⟩ ] \mathbb{E}_{q_0}[\langle\nabla\varphi(Z),V_u(Z)\rangle] E q 0 [⟨ ∇ φ ( Z ) , V u ( Z )⟩] .
(ii) Entropic / Jacobian volume variation: By Jacobi’s formula, differentiating the log-Jacobian determinant − log ∣ det J exp ( t u ) ( Z ) ∣ -\log\left|\det J_{\exp(tu)}(Z)\right| − log det J e x p ( t u ) ( Z ) at t = 0 t=0 t = 0 yields the negative divergence of the velocity field, − E q 0 [ div ( V u ) ( Z ) ] -\mathbb{E}_{q_0}[\operatorname{div}(V_u)(Z)] − E q 0 [ div ( V u ) ( Z )] . Applying integration by parts (Stein’s identity [27 ] ; cf. Hyvärinen [13 ] ) against the density q 0 ( z ) ∝ exp ( − ψ ( z ) ) q_0(z)\propto\exp(-\psi(z)) q 0 ( z ) ∝ exp ( − ψ ( z )) re-expresses this volume expansion rate directly in terms of the base potential score:
E Z ∼ q 0 [ div ( V u ) ( Z ) ] = − E Z ∼ q 0 [ ⟨ ∇ log q 0 ( Z ) , V u ( Z ) ⟩ ] = E Z ∼ q 0 [ ⟨ ∇ ψ ( Z ) , V u ( Z ) ⟩ ] . \mathbb{E}_{Z\sim q_0}[\operatorname{div}(V_u)(Z)]
=-\mathbb{E}_{Z\sim q_0}[\langle\nabla\log q_0(Z),V_u(Z)\rangle]
=\mathbb{E}_{Z\sim q_0}[\langle\nabla\psi(Z),V_u(Z)\rangle]. E Z ∼ q 0 [ div ( V u ) ( Z )] = − E Z ∼ q 0 [⟨ ∇ log q 0 ( Z ) , V u ( Z )⟩] = E Z ∼ q 0 [⟨ ∇ ψ ( Z ) , V u ( Z )⟩] .
Combining these two terms yields our second main theorem, proven in detail in Appendix § 9 .
Theorem 6.5 (Stein moment-matching condition for stationarity). *Under the setting of Definition 3.3 , assume that q 0 ∝ exp ( − ψ ) q_0\propto\exp(-\psi) q 0 ∝ exp ( − ψ ) and p ~ ∝ exp ( − φ ) \widetilde{p}\propto\exp(-\varphi) p ∝ exp ( − φ ) have differentiable (or locally Lipschitz and weakly differentiable a.e.) log-potentials ψ \psi ψ and φ \varphi φ satisfying the integrability condition (locally uniformly for small ∣ t ∣ |t| ∣ t ∣ ) E Z ∼ q 0 [ ∥ ∇ ψ ( Z ) ∥ ∥ V u ( Z ) ∥ + ∥ ∇ φ ( Z ) ∥ ∥ V u ( Z ) ∥ + ∣ div ( V u ) ( Z ) ∣ ] < ∞ \mathbb{E}_{Z\sim q_0}\left[\|\nabla\psi(Z)\|\|V_u(Z)\|+\|\nabla\varphi(Z)\|\|V_u(Z)\|+|\operatorname{div}(V_u)(Z)|\right]<\infty E Z ∼ q 0 [ ∥∇ ψ ( Z ) ∥∥ V u ( Z ) ∥ + ∥∇ φ ( Z ) ∥∥ V u ( Z ) ∥ + ∣ div ( V u ) ( Z ) ∣ ] < ∞ and the boundary condition lim R → ∞ ∫ ∂ B R q 0 ( z ) ∥ V u ( z ) ∥ d S ( z ) = 0 \lim_{R\to\infty}\int_{\partial B_R}q_0(z)\|V_u(z)\|\,\mathrm{d}S(z)=0 lim R → ∞ ∫ ∂ B R q 0 ( z ) ∥ V u ( z ) ∥ d S ( z ) = 0 (which holds automatically when X \mathcal{X} X is compact, and on R d \mathbb{R}^{d} R d whenever q 0 q_0 q 0 has sub-Gaussian or faster tail decay and ∥ V u ( z ) ∥ \|V_u(z)\| ∥ V u ( z ) ∥ grows at most polynomially). Then for any u ∈ g u\in\mathfrak{g} u ∈ g , the directional derivative of the pullback loss J ( u ) = KL ( q exp ( u ) ∥ p ~ ) J(u)=\operatorname{KL}(q_{\exp(u)}\mathbin\Vert\widetilde{p}) J ( u ) = KL ( q e x p ( u ) ∥ p ) at u = 0 u=0 u = 0 is given by
⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = E Z ∼ q 0 [ ⟨ ∇ φ ( Z ) , V u ( Z ) ⟩ ] − E Z ∼ q 0 [ ⟨ ∇ ψ ( Z ) , V u ( Z ) ⟩ ] . \langle\nabla_G\mathcal{L}(g^*),u\rangle_{\mathfrak{g}}
=\langle\nabla J(0),u\rangle_{\mathfrak{g}}
=\mathbb{E}_{Z\sim q_0}[\langle\nabla\varphi(Z),V_u(Z)\rangle]
-\mathbb{E}_{Z\sim q_0}[\langle\nabla\psi(Z),V_u(Z)\rangle]. ⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = E Z ∼ q 0 [⟨ ∇ φ ( Z ) , V u ( Z )⟩] − E Z ∼ q 0 [⟨ ∇ ψ ( Z ) , V u ( Z )⟩] . In particular, since ∇ G L ( g ∗ ) ∈ g H \nabla_G\mathcal{L}(g^*)\in\mathfrak{g}^{H} ∇ G L ( g ∗ ) ∈ g H by Proposition 6.3, the true transport parameter g ∗ g^* g ∗ is a first-order critical point of L \mathcal{L} L (∇ G L ( g ∗ ) = 0 \nabla_G\mathcal{L}(g^*)=0 ∇ G L ( g ∗ ) = 0 ) if and only if the following moment-matching condition holds for every basis direction u ∈ g b i a s u\in\mathfrak{g}_{\mathrm{bias}} u ∈ g bias :
E Z ∼ q 0 [ ⟨ ∇ ψ ( z ) , V u ( z ) ⟩ ] = E Z ∼ q 0 [ ⟨ ∇ φ ( z ) , V u ( z ) ⟩ ] . (5) \mathbb{E}_{Z\sim q_0}[\langle\nabla\psi(z),V_u(z)\rangle]
=\mathbb{E}_{Z\sim q_0}[\langle\nabla\varphi(z),V_u(z)\rangle]. \tag*{(5)} E Z ∼ q 0 [⟨ ∇ ψ ( z ) , V u ( z )⟩] = E Z ∼ q 0 [⟨ ∇ φ ( z ) , V u ( z )⟩] . (5)
Statistically, the quantity ⟨ ∇ ψ ( z ) , V u ( z ) ⟩ = − ⟨ ∇ log q 0 ( z ) , V u ( z ) ⟩ \langle\nabla\psi(z),V_u(z)\rangle=-\langle\nabla\log q_0(z),V_u(z)\rangle ⟨ ∇ ψ ( z ) , V u ( z )⟩ = − ⟨ ∇ log q 0 ( z ) , V u ( z )⟩ is the directional score function of the base model along the transformation generator V u V_u V u (for instance, the radial virial score z T ∇ ψ ( z ) z^{\mathsf{T}}\nabla\psi(z) z T ∇ ψ ( z ) measuring overall dispersion when V u ( z ) = z V_u(z)=z V u ( z ) = z is an isotropic scaling, or the summed score 1 T ∇ ψ ( z ) \mathbf{1}^{\mathsf{T}}\nabla\psi(z) 1 T ∇ ψ ( z ) measuring common location when V u ( z ) = 1 V_u(z)=1 V u ( z ) = 1 is a diagonal shift). Equation (5) therefore has a transparent statistical meaning: the true parameter g ∗ g^* g ∗ is a stationary point of the variational objective if and only if the standardized target p ~ \tilde{p} p ~ and the base distribution q 0 q_0 q 0 have matching expected directional scores under q 0 q_0 q 0 along every symmetry-preserving bias direction u ∈ g b i a s H u\in\mathfrak{g}^{H}_{\mathrm{bias}} u ∈ g bias H .
Remark (Constructive debiasing by base distribution calibration). When the moment-matching identity (5) is violated, ∇ G L ( g ∗ ) \nabla_{\mathcal{G}}\mathcal{L}(g^*) ∇ G L ( g ∗ ) has a nonzero component in g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H , reflecting a calibration mismatch between the variational base q 0 q_0 q 0 and the standardized target p ~ \tilde{p} p ~ along the symmetry-preserving subgroup exp ( g b i a s H ) \exp(\mathfrak{g}^{H}_{\mathrm{bias}}) exp ( g bias H ) . Crucially, Theorem 6.5 provides a direct constructive procedure to eliminate this bias without altering the symmetry structure of the variational family: one recalibrates the base distribution by pushing q 0 q_0 q 0 forward along an element g Ω = exp ( u Ω ) g_{\Omega}=\exp(u_{\Omega}) g Ω = exp ( u Ω ) with u Ω ∈ g b i a s H u_{\Omega}\in\mathfrak{g}^{H}_{\mathrm{bias}} u Ω ∈ g bias H , defining the calibrated base
q Ω : = ( g Ω ) ♯ q 0 . q_{\Omega}:=(g_{\Omega})_{\sharp}q_0. q Ω := ( g Ω ) ♯ q 0 .
Because every u Ω ∈ g b i a s H ⊆ g H u_{\Omega}\in\mathfrak{g}^{H}_{\mathrm{bias}}\subseteq\mathfrak{g}^{H} u Ω ∈ g bias H ⊆ g H generates transformations in the centralizer G H G^{H} G H commuting with H H H , the recalibrated base q Ω q_{\Omega} q Ω remains strictly H H H -symmetric. Specifically, replacing q 0 ∝ exp ( − ψ ) q_0\propto\exp(-\psi) q 0 ∝ exp ( − ψ ) by q Ω ∝ exp ( − ψ Ω ) q_{\Omega}\propto\exp(-\psi_{\Omega}) q Ω ∝ exp ( − ψ Ω ) in (5) and pulling back to z ′ ∼ q 0 z'\sim q_0 z ′ ∼ q 0 via z = g Ω ⋅ z ′ z=g_{\Omega}\cdot z' z = g Ω ⋅ z ′ (using Stein’s identity on the left-hand side) yields the calibration equation
E z ′ ∼ q 0 [ div ( V u ) ( g Ω ⋅ z ′ ) ] = E z ′ ∼ q 0 [ ⟨ ∇ φ ( g Ω ⋅ z ′ ) , V u ( g Ω ⋅ z ′ ) ⟩ ] , ∀ u ∈ g b i a s H , \mathbb{E}_{z'\sim q_0}\left[\operatorname{div}(V_u)(g_{\Omega}\cdot z')\right]
=
\mathbb{E}_{z'\sim q_0}\left[\left\langle\nabla\varphi(g_{\Omega}\cdot z'),V_u(g_{\Omega}\cdot z')\right\rangle\right],
\qquad\forall u\in\mathfrak{g}^{H}_{\mathrm{bias}}, E z ′ ∼ q 0 [ div ( V u ) ( g Ω ⋅ z ′ ) ] = E z ′ ∼ q 0 [ ⟨ ∇ φ ( g Ω ⋅ z ′ ) , V u ( g Ω ⋅ z ′ ) ⟩ ] , ∀ u ∈ g bias H ,
which admits a solution u Ω ∈ g b i a s H u_{\Omega}\in\mathfrak{g}^{H}_{\mathrm{bias}} u Ω ∈ g bias H whenever the left-hand side lies in the range of the right-hand side along exp ( g b i a s H ) \exp(\mathfrak{g}^{H}_{\mathrm{bias}}) exp ( g bias H ) . Because dim ( g b i a s H ) \dim(\mathfrak{g}^{H}_{\mathrm{bias}}) dim ( g bias H ) is typically very small (equal to 1 in both the Covariance and Permutation examples), solving this condition reduces to a simple scalar algebraic equation for u Ω u_{\Omega} u Ω . When the functional form of the standardized potential φ \varphi φ is known (for example, when approximating a known heavy-tailed elliptical family by a Gaussian family), u Ω u_{\Omega} u Ω can be computed offline prior to optimization; when φ \varphi φ is unknown, one can alternatively eliminate g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H by restricting the transport group G G G to a subgroup G 0 ⊂ G G_0\subset G G 0 ⊂ G (structural gauge fixing), as discussed in § 7 .
Guiding examples: Explicit moment-matching equations and base calibration. We conclude this section by working out the moment-matching condition (5) explicitly in our three examples with non-vanishing directional derivatives (full derivations are provided in Appendix § 11 ):
Covariance (Example 4): Here g b i a s H = span ( I d ) \mathfrak{g}^{H}_{\mathrm{bias}}=\operatorname{span}(I_d) g bias H = span ( I d ) is one-dimensional, and the single basis generator u = I d u=I_d u = I d induces the radial scaling velocity field V I d ( z ) = I d z = z V_{I_d}(z)=I_dz=z V I d ( z ) = I d z = z . For a recalibrated isotropic Gaussian base q Ω = N ( 0 , λ 2 I d ) q_{\Omega}=\mathcal{N}(0,\lambda^2I_d) q Ω = N ( 0 , λ 2 I d ) with potential ψ λ ( z ) = ∥ z ∥ 2 2 / ( 2 λ 2 ) \psi_{\lambda}(z)=\lVert z\rVert_2^2/(2\lambda^2) ψ λ ( z ) = ∥ z ∥ 2 2 / ( 2 λ 2 ) , the left-hand side of (5) is the virial expectation
E z ∼ q Ω [ ⟨ ∇ ψ λ ( z ) , z ⟩ ] = 1 λ 2 E z ∼ q Ω [ ∥ z ∥ 2 2 ] = d . \mathbb{E}_{z\sim q_{\Omega}}\left[\left\langle\nabla\psi_{\lambda}(z),z\right\rangle\right]
=
\frac{1}{\lambda^2}\mathbb{E}_{z\sim q_{\Omega}}\left[\lVert z\rVert_2^2\right]
=
d. E z ∼ q Ω [ ⟨ ∇ ψ λ ( z ) , z ⟩ ] = λ 2 1 E z ∼ q Ω [ ∥ z ∥ 2 2 ] = d .
When the standardized target is p ~ = N ( 0 , σ 2 I d ) \tilde{p}=\mathcal{N}(0,\sigma^2I_d) p ~ = N ( 0 , σ 2 I d ) with φ ( z ) = ∥ z ∥ 2 2 / ( 2 σ 2 ) \varphi(z)=\lVert z\rVert_2^2/(2\sigma^2) φ ( z ) = ∥ z ∥ 2 2 / ( 2 σ 2 ) , the right-hand side is σ − 2 E q Ω [ ∥ z ∥ 2 2 ] = d λ 2 / σ 2 \sigma^{-2}\mathbb{E}_{q_{\Omega}}[\lVert z\rVert_2^2]=d\lambda^2/\sigma^2 σ − 2 E q Ω [∥ z ∥ 2 2 ] = d λ 2 / σ 2 , so (5) holds if and only if λ = σ \lambda=\sigma λ = σ . More generally, for any non-Gaussian spherically symmetric target p ~ ( z ) ∝ exp ( − φ ( z ) ) \tilde{p}(z)\propto\exp(-\varphi(z)) p ~ ( z ) ∝ exp ( − φ ( z )) with radial potential φ ( z ) = ρ ( ∥ z ∥ 2 ) \varphi(z)=\rho(\lVert z\rVert_2) φ ( z ) = ρ (∥ z ∥ 2 ) , we have ⟨ ∇ φ ( z ) , z ⟩ = ρ ′ ( ∥ z ∥ 2 ) ∥ z ∥ 2 \langle\nabla\varphi(z),z\rangle=\rho'(\lVert z\rVert_2)\lVert z\rVert_2 ⟨ ∇ φ ( z ) , z ⟩ = ρ ′ (∥ z ∥ 2 ) ∥ z ∥ 2 , and the exact scale calibration condition for λ > 0 \lambda>0 λ > 0 reads
1 λ 2 E z ∼ N ( 0 , λ 2 I d ) [ ∥ z ∥ 2 2 ] = d = E z ∼ N ( 0 , λ 2 I d ) [ ρ ′ ( ∥ z ∥ 2 ) ∥ z ∥ 2 ] . \frac{1}{\lambda^2}\mathbb{E}_{z\sim\mathcal{N}(0,\lambda^2I_d)}\left[\lVert z\rVert_2^2\right]
=
d
=
\mathbb{E}_{z\sim\mathcal{N}(0,\lambda^2I_d)}\left[\rho'(\lVert z\rVert_2)\lVert z\rVert_2\right]. λ 2 1 E z ∼ N ( 0 , λ 2 I d ) [ ∥ z ∥ 2 2 ] = d = E z ∼ N ( 0 , λ 2 I d ) [ ρ ′ (∥ z ∥ 2 ) ∥ z ∥ 2 ] .
Choosing λ \lambda λ to satisfy this scalar equation eliminates the radial scale bias at A ∗ A^* A ∗ , making A ^ = A ∗ \hat{A}=A^* A ^ = A ∗ stationary and recovering the target scatter parameter Σ ∗ = A ∗ ( A ∗ ) T \Sigma^*=A^*(A^*)^{\mathsf{T}} Σ ∗ = A ∗ ( A ∗ ) T (while the fitted Gaussian density itself has covariance λ 2 Σ ∗ \lambda^2\Sigma^* λ 2 Σ ∗ ).
Permutation (Example 6): Here g b i a s H = span ( 1 ) \mathfrak{g}^{H}_{\mathrm{bias}} = \operatorname{span}(1) g bias H = span ( 1 ) is one-dimensional, and the generator u = 1 u = 1 u = 1 induces the constant velocity field V 1 ( z ) = 1 V_{1}(z) = 1 V 1 ( z ) = 1 . The moment-matching condition (5) therefore requires the expected sum of partial derivatives of the potentials to match under q 0 q_{0} q 0 :
E z ∼ q 0 [ ⟨ ∇ ψ ( z ) , 1 ⟩ ] = E z ∼ q 0 [ ⟨ ∇ φ ( z ) , 1 ⟩ ] . \mathbb{E}_{z\sim q_{0}}\left[\langle\nabla\psi(z),1\rangle\right] = \mathbb{E}_{z\sim q_{0}}\left[\langle\nabla\varphi(z),1\rangle\right]. E z ∼ q 0 [ ⟨ ∇ ψ ( z ) , 1 ⟩ ] = E z ∼ q 0 [ ⟨ ∇ φ ( z ) , 1 ⟩ ] .
For the product Laplace base q 0 ( z ) ∝ exp ( − κ ∥ z ∥ 1 ) q_{0}(z) \propto\exp(-\kappa\lVert z\rVert_{1}) q 0 ( z ) ∝ exp ( − κ ∥ z ∥ 1 ) , we have ⟨ ∇ ψ ( z ) , 1 ⟩ = κ ∑ i = 1 n sign ( z i ) \langle\nabla\psi(z),1\rangle= \kappa\sum_{i=1}^{n}\operatorname{sign}(z_{i}) ⟨ ∇ ψ ( z ) , 1 ⟩ = κ ∑ i = 1 n sign ( z i ) , whose expectation under q 0 q_{0} q 0 is 0. If p ~ ( z ) ∝ exp ( − κ ∥ z − β 1 ∥ 1 ) \widetilde{p}(z) \propto\exp(-\kappa\lVert z-\beta1\rVert_{1}) p ( z ) ∝ exp ( − κ ∥ z − β 1 ∥ 1 ) , shifting the base distribution along g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H by u Ω = λ 1 u_{\Omega} = \lambda1 u Ω = λ 1 to q Ω ( z ) ∝ exp ( − κ ∥ z − λ 1 ∥ 1 ) q_{\Omega}(z) \propto\exp(-\kappa\lVert z-\lambda1\rVert_{1}) q Ω ( z ) ∝ exp ( − κ ∥ z − λ 1 ∥ 1 ) achieves E q Ω [ ⟨ ∇ φ ( z ) , 1 ⟩ ] = 0 \mathbb{E}_{q_{\Omega}}[\langle\nabla\varphi(z),1\rangle] = 0 E q Ω [⟨ ∇ φ ( z ) , 1 ⟩] = 0 precisely when λ = β \lambda= \beta λ = β .
Square vs. Chiral C 4 C_{4} C 4 rotation (Example 5): For the quartic potential ψ ( z ) = κ ( x 4 + y 4 ) \psi(z) = \kappa(x^{4}+y^{4}) ψ ( z ) = κ ( x 4 + y 4 ) on R 2 \mathbb{R}^{2} R 2 (written in polar coordinates z = ( r cos ϕ , r sin ϕ ) z=(r\cos\phi,r\sin\phi) z = ( r cos ϕ , r sin ϕ ) as ψ ( r , ϕ ) = κ r 4 ( 3 4 + 1 4 cos 4 ϕ ) \psi(r,\phi)=\kappa r^{4}\left(\frac{3}{4}+\frac{1}{4}\cos4\phi\right) ψ ( r , ϕ ) = κ r 4 ( 4 3 + 4 1 cos 4 ϕ ) ) and the tilted target pullback φ ( z ) = ψ ( R − α z ) \varphi(z)=\psi(R_{-\alpha}z) φ ( z ) = ψ ( R − α z ) , the rotational generator θ ˙ ∈ s o ( 2 ) ≅ R \dot{\theta}\in\mathfrak{so}(2)\cong\mathbb{R} θ ˙ ∈ so ( 2 ) ≅ R induces the tangential velocity field V θ ˙ ( x , y ) = θ ˙ ( − y , x ) T = θ ˙ ∂ ∂ ϕ V_{\dot{\theta}}(x,y)=\dot{\theta}(-y,x)^{\mathsf{T}}=\dot{\theta}\frac{\partial}{\partial\phi} V θ ˙ ( x , y ) = θ ˙ ( − y , x ) T = θ ˙ ∂ ϕ ∂ . The directional derivatives of ψ \psi ψ and φ \varphi φ along V θ ˙ V_{\dot{\theta}} V θ ˙ are
⟨ ∇ ψ ( z ) , V θ ˙ ( z ) ⟩ = − κ θ ˙ r 4 sin ( 4 ϕ ) , ⟨ ∇ φ ( z ) , V θ ˙ ( z ) ⟩ = − κ θ ˙ r 4 sin ( 4 ( ϕ − α ) ) . \langle\nabla\psi(z),V_{\dot{\theta}}(z)\rangle=-\kappa\dot{\theta}r^{4}\sin(4\phi),\qquad\langle\nabla\varphi(z),V_{\dot{\theta}}(z)\rangle=-\kappa\dot{\theta}r^{4}\sin(4(\phi-\alpha)). ⟨ ∇ ψ ( z ) , V θ ˙ ( z )⟩ = − κ θ ˙ r 4 sin ( 4 ϕ ) , ⟨ ∇ φ ( z ) , V θ ˙ ( z )⟩ = − κ θ ˙ r 4 sin ( 4 ( ϕ − α )) .
Because q 0 q_{0} q 0 is invariant under the reflection ϕ ↦ − ϕ \phi\mapsto-\phi ϕ ↦ − ϕ , the odd trigonometric moment vanishes (E q 0 [ r 4 sin ( 4 ϕ ) ] = 0 \mathbb{E}_{q_{0}}[r^{4}\sin(4\phi)]=0 E q 0 [ r 4 sin ( 4 ϕ )] = 0 ), and expanding sin ( 4 ( ϕ − α ) ) = sin ( 4 ϕ ) cos ( 4 α ) − cos ( 4 ϕ ) sin ( 4 α ) \sin(4(\phi-\alpha))=\sin(4\phi)\cos(4\alpha)-\cos(4\phi)\sin(4\alpha) sin ( 4 ( ϕ − α )) = sin ( 4 ϕ ) cos ( 4 α ) − cos ( 4 ϕ ) sin ( 4 α ) yields
⟨ ∇ J ( 0 ) , θ ˙ ⟩ g = − κ M 4 sin ( 4 α ) θ ˙ , where M 4 : = − E z ∼ q 0 [ r 4 cos ( 4 ϕ ) ] > 0. \langle\nabla J(0),\dot{\theta}\rangle_{\mathfrak{g}}=-\kappa M_{4}\sin(4\alpha)\dot{\theta},\qquad\text{where }M_{4}:=-\mathbb{E}_{z\sim q_{0}}\left[r^{4}\cos(4\phi)\right]>0. ⟨ ∇ J ( 0 ) , θ ˙ ⟩ g = − κ M 4 sin ( 4 α ) θ ˙ , where M 4 := − E z ∼ q 0 [ r 4 cos ( 4 ϕ ) ] > 0.
Consequently, ∇ J ( 0 ) = 0 \nabla J(0)=0 ∇ J ( 0 ) = 0 if and only if sin ( 4 α ) = 0 \sin(4\alpha)=0 sin ( 4 α ) = 0 , i.e., α ≡ 0 ( m o d π / 4 ) \alpha\equiv0\pmod{\pi/4} α ≡ 0 ( mod π /4 ) , which is precisely the condition that p ~ \widetilde{p} p possesses the reflection symmetries of D 4 D_{4} D 4 aligned with the axes or diagonals of q 0 q_{0} q 0 .
Discussion and future directions
We have developed a general Lie-theoretic framework characterizing how distributional symmetries of the target and base measures interact with transport-based variational families. By studying the conjugation action of the symmetry group H H H on the transport group G G G and its linearization—the adjoint representation Ad H \operatorname{Ad}_{H} Ad H on the transport Lie algebra g \mathfrak{g} g —Theorem 6.2 and Proposition 6.3 establish that the Riemannian gradient ∇ G L ( g ∗ ) \nabla_{G}L(g^{*}) ∇ G L ( g ∗ ) decomposes across three mutually orthogonal subspaces: the identifiability kernel n \mathfrak{n} n (where the objective is locally constant), the symmetry-breaking subspace g ⊥ \mathfrak{g}^{\perp} g ⊥ (where the gradient is forced to vanish by symmetry), and the symmetry-preserving bias subspace g b i a s H = g H ∩ n ⊥ \mathfrak{g}^{H}_{\mathrm{bias}}=\mathfrak{g}^{H}\cap\mathfrak{n}^{\perp} g bias H = g H ∩ n ⊥ (where model misspecification can induce a nonzero gradient). Theorem 6.5 complements this geometric decomposition by providing an exact Stein-type moment-matching condition characterizing when the gradient along g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H vanishes.
Addressing the conjecture of Margossian and Saul (2025). Our results provide a precise algebraic answer to the conjecture of Margossian and Saul (2025) that each symmetry of a target distribution p p p corresponds to a statistic that variational inference can recover under mild conditions. Within the class of Lie group transformation models, given a compact symmetry group H H H of the standardized target p ~ \widetilde{p} p , any H H H -normalized transport group G G G defines a smooth manifold of identifiable parameters M = G / ( G ∩ H ) \mathcal{M}=G/(G\cap H) M = G / ( G ∩ H ) whose tangent space at the true coset [ g ∗ ] [g^{*}] [ g ∗ ] is canonically identified with n ⊥ = g b i a s H ⊕ g ⊥ \mathfrak{n}^{\perp}=\mathfrak{g}^{H}_{\mathrm{bias}}\oplus\mathfrak{g}^{\perp} n ⊥ = g bias H ⊕ g ⊥ . The components of [ g ∗ ] [g^{*}] [ g ∗ ] along the symmetry-breaking directions g ⊥ \mathfrak{g}^{\perp} g ⊥ are automatically first-order stationary points of the variational objective KL ( q g ∥ p ) \operatorname{KL}(q_{g}\Vert p) KL ( q g ∥ p ) for any H H H -symmetric base distribution q 0 q_{0} q 0 , without requiring parametric specification (p ∈ Q G p\in Q_{G} p ∈ Q G ). Whenever g b i a s H = { 0 } \mathfrak{g}^{H}_{\mathrm{bias}}=\{0\} g bias H = { 0 } , the entire coset [ g ∗ ] ∈ G / N [g^{*}]\in G/N [ g ∗ ] ∈ G / N is recovered at first order by symmetry alone.
Structural gauge fixing versus base calibration. When the bias subspace g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H is nontrivial, our framework highlights two distinct remedies for eliminating estimation bias:
Base calibration via moment matching (Theorem 6.5 ): One retains the full transport group G G G and calibrates the base distribution q 0 ↦ q Ω = ( exp ( u Ω ) ) # q 0 q_0 \mapsto q_{\Omega} = (\exp(u_{\Omega}))_{\#}q_0 q 0 ↦ q Ω = ( exp ( u Ω ) ) # q 0 along u Ω ∈ g b i a s H u_{\Omega} \in\mathfrak{g}^{H}_{\mathrm{bias}} u Ω ∈ g bias H so that the directional score moments (5) match, recovering the full parameter g ∗ g^{*} g ∗ (modulo N N N ).
Structural gauge fixing (subgroup restriction) : When calibrating q 0 q_0 q 0 is impractical because the radial profile of p ~ \tilde{p} p ~ is unknown, one can instead restrict the variational family to a smaller, H H H -normalized Lie subgroup G 0 ⊂ G G_0 \subset G G 0 ⊂ G whose Lie algebra g 0 = T e G 0 \mathfrak{g}_0 = T_eG_0 g 0 = T e G 0 is a Lie subalgebra ([ g 0 , g 0 ] ⊆ g 0 [\mathfrak{g}_0,\mathfrak{g}_0] \subseteq\mathfrak{g}_0 [ g 0 , g 0 ] ⊆ g 0 ) transverse to g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H , i.e., g 0 ∩ g b i a s H = { 0 } \mathfrak{g}_0 \cap\mathfrak{g}^{H}_{\mathrm{bias}} = \{0\} g 0 ∩ g bias H = { 0 } . In the restricted family Q G 0 Q_{G_0} Q G 0 , the bias subspace vanishes identically, so the projection g 0 ∗ g_0^{*} g 0 ∗ of g ∗ g^{*} g ∗ onto G 0 / ( G 0 ∩ H ) G_0/(G_0 \cap H) G 0 / ( G 0 ∩ H ) is guaranteed to be a stationary point for any H H H -symmetric base q 0 q_0 q 0 (whenever g ∗ g^{*} g ∗ factors as g ∗ = g 0 ∗ g b i a s ∗ n g^{*} = g_0^{*}g_{\mathrm{bias}}^{*}n g ∗ = g 0 ∗ g bias ∗ n with g 0 ∗ ∈ G 0 g_0^{*} \in G_0 g 0 ∗ ∈ G 0 , g b i a s ∗ ∈ G H g_{\mathrm{bias}}^{*} \in G^{H} g bias ∗ ∈ G H , and n ∈ N n \in N n ∈ N , so that pulling p p p back by g 0 ∗ g_0^{*} g 0 ∗ preserves H H H -invariance), at the cost of no longer estimating the parameter component along g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H .
Covariance estimation (Example 4, G = G L ( d ) G = \mathrm{GL}(d) G = GL ( d ) , H = O ( d ) H = \mathrm{O}(d) H = O ( d ) ) provides an instructive comparison between these strategies. Here the bias subspace is the one-dimensional trace direction g b i a s H = span ( I d ) \mathfrak{g}^{H}_{\mathrm{bias}} = \operatorname{span}(I_d) g bias H = span ( I d ) . From a Lie-theoretic standpoint, the canonical H H H -normalized subgroup of G L ( d ) \mathrm{GL}(d) GL ( d ) that excises span ( I d ) \operatorname{span}(I_d) span ( I d ) is the special linear group G 0 = S L ( d ) = { A ∈ G L ( d ) : det A = 1 } G_0 = \mathrm{SL}(d) = \{A \in\mathrm{GL}(d) : \det A = 1\} G 0 = SL ( d ) = { A ∈ GL ( d ) : det A = 1 } , whose Lie algebra s l ( d ) = { M ∈ g l ( d ) : tr ( M ) = 0 } \mathfrak{sl}(d) = \{M \in\mathfrak{gl}(d) : \operatorname{tr}(M) = 0\} sl ( d ) = { M ∈ gl ( d ) : tr ( M ) = 0 } is precisely n ⊕ g ⊥ = s o ( d ) ⊕ Sym 0 ( d ) \mathfrak{n} \oplus\mathfrak{g}^{\perp} = \mathfrak{so}(d) \oplus\operatorname{Sym}_0(d) n ⊕ g ⊥ = so ( d ) ⊕ Sym 0 ( d ) . Restricting transport to S L ( d ) \mathrm{SL}(d) SL ( d ) yields the quotient manifold M 0 = S L ( d ) / S O ( d ) \mathcal{M}_0 = \mathrm{SL}(d)/\mathrm{SO}(d) M 0 = SL ( d ) / SO ( d ) of unit-determinant shape matrices (det Σ = 1 \det\Sigma= 1 det Σ = 1 ), recovering the exact covariance shape while leaving only the single scalar generalized variance det Σ ∗ \det\Sigma^{*} det Σ ∗ unestimated. By comparison, Margossian and Saul (2025) optimize over the full unconstrained group G = G L ( d ) G = \mathrm{GL}(d) G = GL ( d ) (obtaining a stationary point Σ ^ = c Σ ∗ \hat{\Sigma} = c\Sigma^{*} Σ ^ = c Σ ∗ proportional to Σ ∗ \Sigma^{*} Σ ∗ ) and eliminate the scale factor c > 0 c > 0 c > 0 post-hoc by normalizing Σ ^ \hat{\Sigma} Σ ^ into the correlation matrix (Σ i i = 1 \Sigma_{ii} = 1 Σ ii = 1 for i = 1 , … , d i = 1, \ldots, d i = 1 , … , d ). Our decomposition clarifies both post-hoc normalization and prior subgroup restriction: from the unconstrained optimum Σ ^ = c Σ ∗ \hat{\Sigma} = c\Sigma^{*} Σ ^ = c Σ ∗ , one can in fact extract the full unit-determinant shape matrix ( det Σ ^ ) − 1 / d Σ ^ ∈ S L ( d ) / S O ( d ) (\det\hat{\Sigma})^{-1/d}\hat{\Sigma} \in\mathrm{SL}(d)/\mathrm{SO}(d) ( det Σ ^ ) − 1/ d Σ ^ ∈ SL ( d ) / SO ( d ) (preserving the d − 1 d - 1 d − 1 ratios of marginal variances that are discarded by the correlation matrix), while if one wishes to restrict the optimization domain a priori to a subgroup G 0 ⊂ G L ( d ) G_0 \subset\mathrm{GL}(d) G 0 ⊂ GL ( d ) , restricting to S L ( d ) \mathrm{SL}(d) SL ( d ) preserves full O ( d ) \mathrm{O}(d) O ( d ) conjugation invariance whereas enforcing unit diagonal entries breaks it. Similarly, in the Permutation model (Example 6, G = ( R n , + ) G = (\mathbb{R}^{n}, +) G = ( R n , + ) , H = S n H = S_n H = S n ), restricting G G G to the ( n − 1 ) (n - 1) ( n − 1 ) -dimensional zero-sum hyperplane G 0 = { μ ∈ R n : ∑ i = 1 n μ i = 0 } ⊂ g ⊥ G_0 = \{\mu\in\mathbb{R}^{n} : \sum_{i=1}^{n}\mu_i = 0\} \subset\mathfrak{g}^{\perp} G 0 = { μ ∈ R n : ∑ i = 1 n μ i = 0 } ⊂ g ⊥ is normalized by S n S_n S n and guarantees unbiased recovery of all centered coordinate offsets μ ∗ − μ ˉ ∗ 1 \mu^{*} - \bar{\mu}^{*}\mathbf{1} μ ∗ − μ ˉ ∗ 1 .
From first-order stationarity to second-order and global optimality. Theorem 6.2 is a first-order result: it guarantees that ∇ G L ( g ∗ ) = 0 \nabla_G\mathcal{L}(g^{*}) = 0 ∇ G L ( g ∗ ) = 0 whenever g b i a s H = { 0 } \mathfrak{g}^{H}_{\mathrm{bias}} = \{0\} g bias H = { 0 } (or when moment matching holds), which means g ∗ g^{*} g ∗ is a critical point of L \mathcal{L} L . Whether g ∗ g^{*} g ∗ is a strict local minimizer (modulo the flat coset fiber N N N ) or a global minimizer depends on higher-order curvature properties of the target and base distributions. Specifically, g ∗ g^{*} g ∗ defines a strict local minimum on the quotient manifold M = G / N \mathcal{M} = G/N M = G / N if the Riemannian Hessian Hess G L ( g ∗ ) = ∇ 2 J ( 0 ) \operatorname{Hess}_{G}\mathcal{L}(g^{*}) = \nabla^{2}J(0) Hess G L ( g ∗ ) = ∇ 2 J ( 0 ) is strictly positive definite on the horizontal subspace n ⊥ = g b i a s H ⊕ g ⊥ \mathfrak{n}^{\perp} = \mathfrak{g}^{H}_{\mathrm{bias}} \oplus\mathfrak{g}^{\perp} n ⊥ = g bias H ⊕ g ⊥ . When the target p p p is strongly multimodal—for instance, in Example 1 when p = 1 2 N ( μ ∗ + δ , 1 ) + 1 2 N ( μ ∗ − δ , 1 ) p = \frac{1}{2}\mathcal{N}(\mu^{*} + \delta, 1) + \frac{1}{2}\mathcal{N}(\mu^{*} - \delta, 1) p = 2 1 N ( μ ∗ + δ , 1 ) + 2 1 N ( μ ∗ − δ , 1 ) has two widely separated modes (δ ≫ 1 \delta\gg1 δ ≫ 1 ) while q 0 = N ( 0 , 1 ) q_0 = \mathcal{N}(0,1) q 0 = N ( 0 , 1 ) is unimodal with unit variance—the symmetry center μ ∗ \mu^{*} μ ∗ remains a critical point by Theorem 6.2 , but bifurcates from a global minimum into a local maximum (or a saddle point in higher dimensions) straddled by two symmetry-broken global minima near μ ∗ ± δ \mu^{*} \pm\delta μ ∗ ± δ . For translation families (G = R d G = \mathbb{R}^{d} G = R d ), log-concavity of p p p ensures convexity of L ( μ ) \mathcal{L}(\mu) L ( μ ) and hence global uniqueness of the stationary point μ ∗ \mu^{*} μ ∗ ; extending such global uniqueness guarantees to non-abelian Lie groups via geodesic convexity on Riemannian quotient manifolds M = G / N \mathcal{M} = G/N M = G / N (Boumal, [5 ] ) is an important direction for future research.
Further extensions. Several additional directions merit investigation. First, in empirical applications, target distributions often satisfy H H H -conjugate symmetry only approximately; combining our representation-theoretic decomposition with perturbation bounds on ∇ J ( 0 ) \nabla J(0) ∇ J ( 0 ) would quantify how an ϵ \epsilon ϵ -asymmetry in p ~ \widetilde{p} p induces an O ( ϵ ) O(\epsilon) O ( ϵ ) drift of the variational stationary point along g ⊥ \mathfrak{g}^{\perp} g ⊥ . Second, while Theorem 6.2 already holds for any divergence functional that is invariant under diffeomorphisms of X \mathcal{X} X (including all f f f -divergences and Rényi α \alpha α -divergences, since steps (†), (*), and (**) in its proof rely solely on pushforward invariance), extending the Stein moment-matching formula of Theorem 6.5 to general f f f -divergences and Wasserstein distances will yield analogous base-calibration recipes beyond the KL divergence. Finally, our assumption that H H H acts on X \mathcal{X} X by linear isometries can be relaxed to arbitrary isometric actions of compact Lie groups on Riemannian manifolds ( X , vol X ) (\mathcal{X},\operatorname{vol}_{\mathcal{X}}) ( X , vol X ) : replacing the Euclidean gradient and divergence in Theorem 6.5 by the Riemannian gradient ∇ X \nabla_{\mathcal{X}} ∇ X and Laplace–Beltrami divergence div X \operatorname{div}_{\mathcal{X}} div X leaves all our main theorems intact.
References
Proof of Theorem 6.5
We provide the complete proof of the Stein moment-matching condition (Theorem 6.5 ). Recall that q 0 ( z ) = Z 0 − 1 exp ( − ψ ( z ) ) q_0(z) = Z_0^{-1}\exp(-\psi(z)) q 0 ( z ) = Z 0 − 1 exp ( − ψ ( z )) and p ~ ( z ) = Z ~ p − 1 exp ( − φ ( z ) ) \tilde{p}(z) = \tilde{Z}_p^{-1}\exp(-\varphi(z)) p ~ ( z ) = Z ~ p − 1 exp ( − φ ( z )) have differentiable potentials ψ , φ : X → R \psi,\varphi: \mathcal{X} \to\mathbb{R} ψ , φ : X → R satisfying the integrability and boundary conditions stated in Theorem 6.5 .
Proof of Theorem 6.5. Recall from the proof of Theorem 6.2 that the pullback objective around g ∗ g^* g ∗ is given by J ( u ) = L ( g ∗ exp ( u ) ) = KL ( q exp ( u ) ∥ p ~ ) J(u) = \mathcal{L}(g^* \exp(u)) = \operatorname{KL}(q_{\exp(u)} \mathbin{\|} \tilde{p}) J ( u ) = L ( g ∗ exp ( u )) = KL ( q e x p ( u ) ∥ p ~ ) for u ∈ g u \in\mathfrak{g} u ∈ g . We compute the directional derivative
⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = d d t ∣ t = 0 J ( t u ) \langle\nabla_G \mathcal{L}(g^*),u\rangle_{\mathfrak{g}} = \langle\nabla J(0),u\rangle_{\mathfrak{g}} = \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}J(tu) ⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = d t d t = 0 J ( t u )
for an arbitrary tangent vector u ∈ g u \in\mathfrak{g} u ∈ g .
Step 1: Pullback to the fixed base measure q 0 q_0 q 0 . Under the diffeomorphism x = exp ( t u ) ⋅ z x = \exp(tu)\cdot z x = exp ( t u ) ⋅ z with z ∼ q 0 z \sim q_0 z ∼ q 0 , the pushforward density q exp ( t u ) q_{\exp(tu)} q e x p ( t u ) satisfies q exp ( t u ) ( exp ( t u ) ⋅ z ) ∣ det J exp ( t u ) ( z ) ∣ = q 0 ( z ) q_{\exp(tu)}(\exp(tu)\cdot z)\lvert\det J_{\exp(tu)}(z)\rvert= q_0(z) q e x p ( t u ) ( exp ( t u ) ⋅ z ) ∣ det J e x p ( t u ) ( z )∣ = q 0 ( z ) . Substituting the change of variables x = exp ( t u ) ⋅ z x = \exp(tu)\cdot z x = exp ( t u ) ⋅ z (with d x = ∣ det J exp ( t u ) ( z ) ∣ d z \mathrm{d}x = \lvert\det J_{\exp(tu)}(z)\rvert\mathrm{d}z d x = ∣ det J e x p ( t u ) ( z )∣ d z ) into the definition of the KL divergence gives
J ( t u ) = ∫ X q 0 ( z ) [ log q 0 ( z ) − log p ~ ( exp ( t u ) ⋅ z ) − log ∣ det J exp ( t u ) ( z ) ∣ ] d z . J(tu) = \int_{\mathcal{X}} q_0(z)\left[\log q_0(z)-\log\tilde{p}(\exp(tu)\cdot z)-\log\left\lvert\det J_{\exp(tu)}(z)\right\rvert\right]\mathrm{d}z. J ( t u ) = ∫ X q 0 ( z ) [ log q 0 ( z ) − log p ~ ( exp ( t u ) ⋅ z ) − log det J e x p ( t u ) ( z ) ] d z .
Note that the first integrand term log q 0 ( z ) \log q_0(z) log q 0 ( z ) is independent of t t t and therefore does not contribute to the derivative with respect to t t t .
Step 2: Differentiation at t = 0 t=0 t = 0 . Differentiating under the integral sign at t = 0 t=0 t = 0 (justified by dominated convergence under a standard integrable local dominating bound on t ∈ ( − ϵ , ϵ ) t \in(-\epsilon,\epsilon) t ∈ ( − ϵ , ϵ ) together with the integrability assumption of Theorem 6.5 ), we obtain
d d t ∣ t = 0 J ( t u ) = − ∫ X q 0 ( z ) [ d d t ∣ t = 0 log p ~ ( exp ( t u ) ⋅ z ) + d d t ∣ t = 0 log ∣ det J exp ( t u ) ( z ) ∣ ] d z . \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}J(tu) = -\int_{\mathcal{X}}q_0(z)\left[\left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\log\tilde{p}(\exp(tu)\cdot z)+\left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\log\left\lvert\det J_{\exp(tu)}(z)\right\rvert\right]\mathrm{d}z. d t d t = 0 J ( t u ) = − ∫ X q 0 ( z ) [ d t d t = 0 log p ~ ( exp ( t u ) ⋅ z ) + d t d t = 0 log det J e x p ( t u ) ( z ) ] d z .
For the first term, since log p ~ ( x ) = − φ ( x ) − log Z ~ p \log\tilde{p}(x)=-\varphi(x)-\log\tilde{Z}_p log p ~ ( x ) = − φ ( x ) − log Z ~ p and d d t ∣ t = 0 ( exp ( t u ) ⋅ z ) = V u ( z ) \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}(\exp(tu)\cdot z)=V_u(z) d t d t = 0 ( exp ( t u ) ⋅ z ) = V u ( z ) by Definition 6.4, the chain rule yields
d d t ∣ t = 0 log p ~ ( exp ( t u ) ⋅ z ) = ⟨ ∇ log p ~ ( z ) , V u ( z ) ⟩ = − ⟨ ∇ φ ( z ) , V u ( z ) ⟩ . \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\log\tilde{p}(\exp(tu)\cdot z)=\langle\nabla\log\tilde{p}(z),V_u(z)\rangle=-\langle\nabla\varphi(z),V_u(z)\rangle. d t d t = 0 log p ~ ( exp ( t u ) ⋅ z ) = ⟨ ∇ log p ~ ( z ) , V u ( z )⟩ = − ⟨ ∇ φ ( z ) , V u ( z )⟩ .
For the second term, the flow map ϕ t ( z ) = exp ( t u ) ⋅ z \phi_t(z)=\exp(tu)\cdot z ϕ t ( z ) = exp ( t u ) ⋅ z satisfies ϕ 0 ( z ) = z \phi_0(z)=z ϕ 0 ( z ) = z (with J ϕ 0 ( z ) = I d J_{\phi_0}(z)=I_d J ϕ 0 ( z ) = I d ) and d d t ∣ t = 0 J ϕ t ( z ) = ∇ V u ( z ) \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}J_{\phi_t}(z)=\nabla V_u(z) d t d t = 0 J ϕ t ( z ) = ∇ V u ( z ) . Applying Jacobi’s formula for the derivative of a log-determinant gives the divergence of the velocity field V u V_u V u :
d d t ∣ t = 0 log ∣ det J exp ( t u ) ( z ) ∣ = tr ( ∇ V u ( z ) ) = div ( V u ) ( z ) = ∑ i = 1 d ∂ ( V u ) i ∂ z i ( z ) . \left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\log\left\lvert\det J_{\exp(tu)}(z)\right\rvert=\operatorname{tr}(\nabla V_u(z))=\operatorname{div}(V_u)(z)=\sum_{i=1}^{d}\frac{\partial(V_u)_i}{\partial z_i}(z). d t d t = 0 log det J e x p ( t u ) ( z ) = tr ( ∇ V u ( z )) = div ( V u ) ( z ) = i = 1 ∑ d ∂ z i ∂ ( V u ) i ( z ) .
Step 3: Integration by parts (Stein’s identity). Using the product rule div ( q 0 V u ) = q 0 div ( V u ) + ⟨ ∇ q 0 , V u ⟩ \operatorname{div}(q_0V_u)=q_0\operatorname{div}(V_u)+\langle\nabla q_0,V_u\rangle div ( q 0 V u ) = q 0 div ( V u ) + ⟨ ∇ q 0 , V u ⟩ and the divergence theorem on X \mathcal{X} X , the boundary integral vanishes by our boundary condition, and since ∇ q 0 ( z ) = q 0 ( z ) ∇ log q 0 ( z ) = − q 0 ( z ) ∇ ψ ( z ) \nabla q_0(z)=q_0(z)\nabla\log q_0(z)=-q_0(z)\nabla\psi(z) ∇ q 0 ( z ) = q 0 ( z ) ∇ log q 0 ( z ) = − q 0 ( z ) ∇ ψ ( z ) , we obtain
∫ X q 0 ( z ) div ( V u ) ( z ) d z = − ∫ X ⟨ ∇ q 0 ( z ) , V u ( z ) ⟩ d z = − ∫ X q 0 ( z ) ⟨ ∇ log q 0 ( z ) , V u ( z ) ⟩ d z = ∫ X q 0 ( z ) ⟨ ∇ ψ ( z ) , V u ( z ) ⟩ d z . \begin{aligned}
\int_{\mathcal{X}}q_0(z)\operatorname{div}(V_u)(z)\,\mathrm{d}z=-\int_{\mathcal{X}}\langle\nabla q_0(z),V_u(z)\rangle\,\mathrm{d}z \\
&=-\int_{\mathcal{X}}q_0(z)\langle\nabla\log q_0(z),V_u(z)\rangle\,\mathrm{d}z=\int_{\mathcal{X}}q_0(z)\langle\nabla\psi(z),V_u(z)\rangle\,\mathrm{d}z.
\end{aligned} ∫ X q 0 ( z ) div ( V u ) ( z ) d z = − ∫ X ⟨ ∇ q 0 ( z ) , V u ( z )⟩ d z = − ∫ X q 0 ( z ) ⟨ ∇ log q 0 ( z ) , V u ( z )⟩ d z = ∫ X q 0 ( z ) ⟨ ∇ ψ ( z ) , V u ( z )⟩ d z .
Step 4: Combining both contributions. Substituting the results of Steps 2 and 3 into the derivative of J ( t u ) J(tu) J ( t u ) yields
⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = E z ∼ q 0 [ ⟨ ∇ φ ( z ) , V u ( z ) ⟩ ] − E z ∼ q 0 [ ⟨ ∇ ψ ( z ) , V u ( z ) ⟩ ] . \langle\nabla_G \mathcal{L}(g^\ast),u\rangle_{\mathfrak{g}}=\langle\nabla J(0),u\rangle_{\mathfrak{g}}=\mathbb{E}_{z\sim q_0}\left[\left\langle\nabla\varphi(z),V_u(z)\right\rangle\right]-\mathbb{E}_{z\sim q_0}\left[\left\langle\nabla\psi(z),V_u(z)\right\rangle\right]. ⟨ ∇ G L ( g ∗ ) , u ⟩ g = ⟨ ∇ J ( 0 ) , u ⟩ g = E z ∼ q 0 [ ⟨ ∇ φ ( z ) , V u ( z ) ⟩ ] − E z ∼ q 0 [ ⟨ ∇ ψ ( z ) , V u ( z ) ⟩ ] .
Finally, by Proposition 6.3, ∇ G L ( g ∗ ) \nabla_G\mathcal{L}(g^\ast) ∇ G L ( g ∗ ) already lies in g b i a s H \mathfrak{g}^{H}_{\mathrm{bias}} g bias H (its components along n \mathfrak{n} n and g ⊥ \mathfrak{g}^{\perp} g ⊥ vanish identically). Thus ∇ G L ( g ∗ ) = 0 \nabla_G\mathcal{L}(g^\ast)=0 ∇ G L ( g ∗ ) = 0 if and only if ⟨ ∇ G L ( g ∗ ) , u ⟩ g = 0 \langle\nabla_G\mathcal{L}(g^\ast),u\rangle_{\mathfrak{g}}=0 ⟨ ∇ G L ( g ∗ ) , u ⟩ g = 0 for all u ∈ g b i a s H u\in\mathfrak{g}^{H}_{\mathrm{bias}} u ∈ g bias H , which is equivalent to (5) . □
AdH-invariance of n and subspace decomposition details
The orthogonal decomposition g = n ⊕ g b i a s H ⊕ g ⊥ \mathfrak{g}=\mathfrak{n}\oplus\mathfrak{g}^{H}_{\mathrm{bias}}\oplus\mathfrak{g}^{\perp} g = n ⊕ g bias H ⊕ g ⊥ in Proposition 6.3 uses the fact that the identifiability kernel n = T e N \mathfrak{n}=T_eN n = T e N is an Ad H \operatorname{Ad}_H Ad H -invariant subspace of g \mathfrak{g} g . We record the detailed verification here and display the explicit decomposition n = ( n ∩ g H ) ⊕ ( n ∩ ( g H ) ⊥ ) \mathfrak{n}=(\mathfrak{n}\cap\mathfrak{g}^{H})\oplus(\mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp}) n = ( n ∩ g H ) ⊕ ( n ∩ ( g H ) ⊥ ) across all six guiding examples.
Proof of AdH-invariance of n
Recall that N = G ∩ H N=G\cap H N = G ∩ H is the intersection of the transport group G G G and the symmetry group H H H inside the total group K = G H K=GH K = G H . For any n ∈ N n\in N n ∈ N and h ∈ H h\in H h ∈ H , we have Φ h ( n ) = h ∘ n ∘ h − 1 ∈ G \Phi_h(n)=h\circ n\circ h^{-1}\in G Φ h ( n ) = h ∘ n ∘ h − 1 ∈ G because G G G is normalized by H H H (Definition 3.3), and simultaneously h ∘ n ∘ h − 1 ∈ H h\circ n\circ h^{-1}\in H h ∘ n ∘ h − 1 ∈ H because n ∈ H n\in H n ∈ H and H H H is a group. Hence Φ h ( n ) ∈ G ∩ H = N \Phi_h(n)\in G\cap H=N Φ h ( n ) ∈ G ∩ H = N for all n ∈ N n\in N n ∈ N and h ∈ H h\in H h ∈ H .
Now let u ∈ n u\in\mathfrak{n} u ∈ n . By definition of n \mathfrak{n} n , the one-parameter subgroup t ↦ exp ( t u ) t\mapsto\exp(tu) t ↦ exp ( t u ) lies entirely in N N N for all t ∈ R t\in\mathbb{R} t ∈ R . Applying Φ h \Phi_h Φ h , the smooth curve t ↦ Φ h ( exp ( t u ) ) t\mapsto\Phi_h(\exp(tu)) t ↦ Φ h ( exp ( t u )) also lies entirely in N N N for all t ∈ R t\in\mathbb{R} t ∈ R , and passes through e G e_G e G at t = 0 t=0 t = 0 . Its velocity vector at t = 0 t=0 t = 0 therefore belongs to the tangent space T e N = n T_eN=\mathfrak{n} T e N = n :
Ad h ⋅ u = d d t ∣ t = 0 Φ h ( exp ( t u ) ) ∈ n . \operatorname{Ad}_h\cdot u=\left.\frac{\mathrm{d}}{\mathrm{d}t}\right|_{t=0}\Phi_h(\exp(tu))\in\mathfrak{n}. Ad h ⋅ u = d t d t = 0 Φ h ( exp ( t u )) ∈ n .
Thus Ad h ( n ) = n \operatorname{Ad}_h(\mathfrak{n})=\mathfrak{n} Ad h ( n ) = n for every h ∈ H h\in H h ∈ H . Because the inner product ⟨ ⋅ , ⋅ ⟩ g \langle\cdot,\cdot\rangle_{\mathfrak{g}} ⟨ ⋅ , ⋅ ⟩ g is Ad H \operatorname{Ad}_H Ad H -invariant, the orthogonal averaging projection P g H = ∫ H Ad h d H a a r ( h ) P_{\mathfrak{g}^{H}}=\int_H\operatorname{Ad}_h\,\mathrm{dHaar}(h) P g H = ∫ H Ad h dHaar ( h ) onto the trivial representation g H \mathfrak{g}^{H} g H maps n \mathfrak{n} n into n ∩ g H \mathfrak{n}\cap\mathfrak{g}^{H} n ∩ g H , and the complementary projection I − P g H I-P_{\mathfrak{g}^{H}} I − P g H maps n \mathfrak{n} n into n ∩ ( g H ) ⊥ \mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp} n ∩ ( g H ) ⊥ . Consequently, n \mathfrak{n} n splits orthogonally as
n = ( n ∩ g H ) ⊕ ( n ∩ ( g H ) ⊥ ) , \mathfrak{n}=(\mathfrak{n}\cap\mathfrak{g}^{H})\oplus(\mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp}), n = ( n ∩ g H ) ⊕ ( n ∩ ( g H ) ⊥ ) ,
which ensures that n \mathfrak{n} n , g b i a s H = g H ∩ n ⊥ \mathfrak{g}^{H}_{\mathrm{bias}}=\mathfrak{g}^{H}\cap\mathfrak{n}^{\perp} g bias H = g H ∩ n ⊥ , and g ⊥ = ( g H ) ⊥ ∩ n ⊥ \mathfrak{g}^{\perp}=(\mathfrak{g}^{H})^{\perp}\cap\mathfrak{n}^{\perp} g ⊥ = ( g H ) ⊥ ∩ n ⊥ are mutually orthogonal and sum to g \mathfrak{g} g .
Explicit subspace verification for the six guiding examples
Separable regime (Examples 1, 2, and 6): Here N = { e } N=\{e\} N = { e } , so n = { 0 } \mathfrak{n}=\{0\} n = { 0 } and both n ∩ g H = { 0 } \mathfrak{n}\cap\mathfrak{g}^{H}=\{0\} n ∩ g H = { 0 } and n ∩ ( g H ) ⊥ = { 0 } \mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp}=\{0\} n ∩ ( g H ) ⊥ = { 0 } are trivial. Consequently, g b i a s H = g H \mathfrak{g}^{H}_{\mathrm{bias}}=\mathfrak{g}^{H} g bias H = g H and g ⊥ = ( g H ) ⊥ \mathfrak{g}^{\perp}=(\mathfrak{g}^{H})^{\perp} g ⊥ = ( g H ) ⊥ .
Sphere (Example 3): Here N = H = S O ( 2 ) N=H=\mathrm{SO}(2) N = H = SO ( 2 ) is abelian, so the adjoint action of S O ( 2 ) \mathrm{SO}(2) SO ( 2 ) on its own Lie algebra n = span ( e 3 ) \mathfrak{n}=\operatorname{span}(e_3) n = span ( e 3 ) is trivial, giving n = g H = span ( e 3 ) \mathfrak{n}=\mathfrak{g}^{H}=\operatorname{span}(e_3) n = g H = span ( e 3 ) . Thus n ∩ g H = n \mathfrak{n}\cap\mathfrak{g}^{H}=\mathfrak{n} n ∩ g H = n and n ∩ ( g H ) ⊥ = { 0 } \mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp}=\{0\} n ∩ ( g H ) ⊥ = { 0 } : the entire identifiability kernel lies inside the trivial subspace g H \mathfrak{g}^{H} g H , leaving g b i a s H = g H ∩ n ⊥ = { 0 } \mathfrak{g}^{H}_{\mathrm{bias}}=\mathfrak{g}^{H}\cap\mathfrak{n}^{\perp}=\{0\} g bias H = g H ∩ n ⊥ = { 0 } .
Covariance (Example 4): Here N = H = O ( d ) N=H=\mathrm{O}(d) N = H = O ( d ) has Lie algebra n = s o ( d ) \mathfrak{n}=\mathfrak{so}(d) n = so ( d ) (skew-symmetric matrices), whereas g H = span ( I d ) \mathfrak{g}^{H}=\operatorname{span}(I_d) g H = span ( I d ) (isotropic scalar matrices). Because the only matrix that is both symmetric (a multiple of I d I_d I d ) and skew-symmetric is the zero matrix, we have n ∩ g H = { 0 } \mathfrak{n}\cap\mathfrak{g}^{H}=\{0\} n ∩ g H = { 0 } and n ∩ ( g H ) ⊥ = s o ( d ) \mathfrak{n}\cap(\mathfrak{g}^{H})^{\perp}=\mathfrak{so}(d) n ∩ ( g H ) ⊥ = so ( d ) : for d ≥ 2 d\ge2 d ≥ 2 , the entire identifiability kernel s o ( d ) \mathfrak{so}(d) so ( d ) lies in the non-trivial representation ( g H ) ⊥ (\mathfrak{g}^{H})^{\perp} ( g H ) ⊥ . Consequently, g b i a s H = g H = span ( I d ) \mathfrak{g}^{H}_{\mathrm{bias}}=\mathfrak{g}^{H}=\operatorname{span}(I_d) g bias H = g H = span ( I d ) , and g ⊥ = ( g H ) ⊥ ∩ s o ( d ) ⊥ = Sym 0 ( d ) \mathfrak{g}^{\perp}=(\mathfrak{g}^{H})^{\perp}\cap\mathfrak{so}(d)^{\perp}=\operatorname{Sym}_0(d) g ⊥ = ( g H ) ⊥ ∩ so ( d ) ⊥ = Sym 0 ( d ) is the space of traceless symmetric matrices.
Square (Example 5): Here N = C 4 N=C_4 N = C 4 is a finite group of order 4, so its Lie algebra is zero-dimensional (n = { 0 } \mathfrak{n}=\{0\} n = { 0 } ), and the decomposition of n \mathfrak{n} n is again trivial.
Detailed derivations for the guiding examples
Covariance example: Explicit scale bias and base calibration
Consider Example 4 (G = G L ( d ) G = \mathrm{GL}(d) G = GL ( d ) , H = O ( d ) H = \mathrm{O}(d) H = O ( d ) ) with standard Gaussian base q 0 = N ( 0 , I d ) q_0 = \mathcal{N}(0,I_d) q 0 = N ( 0 , I d ) and standardized target pullback p ~ = N ( 0 , σ 2 I d ) \tilde{p} = \mathcal{N}(0,\sigma^2 I_d) p ~ = N ( 0 , σ 2 I d ) for some σ > 0 \sigma> 0 σ > 0 . The Lie algebra is g = g l ( d ) \mathfrak{g} = \mathfrak{gl}(d) g = gl ( d ) , and by Proposition 6.3 the bias subspace is g b i a s = span ( I d ) \mathfrak{g}_{\mathrm{bias}} = \operatorname{span}(I_d) g bias = span ( I d ) .
For the basis generator u = I d u = I_d u = I d , the velocity field is V I d ( z ) = I d z = z V_{I_d}(z) = I_dz = z V I d ( z ) = I d z = z . The log-density potentials (up to additive constants) are ψ ( z ) = 1 2 ∥ z ∥ 2 2 \psi(z) = \frac{1}{2}\lVert z\rVert_2^2 ψ ( z ) = 2 1 ∥ z ∥ 2 2 and φ ( z ) = 1 2 σ 2 ∥ z ∥ 2 2 \varphi(z) = \frac{1}{2\sigma^2}\lVert z\rVert_2^2 φ ( z ) = 2 σ 2 1 ∥ z ∥ 2 2 , with gradients ∇ ψ ( z ) = z \nabla\psi(z) = z ∇ ψ ( z ) = z and ∇ φ ( z ) = σ − 2 z \nabla\varphi(z) = \sigma^{-2}z ∇ φ ( z ) = σ − 2 z . Substituting these into Theorem 6.5 gives the directional derivative of the pullback loss along I d I_d I d :
⟨ ∇ G L ( A ∗ ) , I d ⟩ F r o b = E z ∼ q 0 [ ⟨ ∇ φ ( z ) , z ⟩ ] − E z ∼ q 0 [ ⟨ ∇ ψ ( z ) , z ⟩ ] = 1 σ 2 E z ∼ q 0 [ ∥ z ∥ 2 2 ] − E z ∼ q 0 [ ∥ z ∥ 2 2 ] = d ( 1 σ 2 − 1 ) . \begin{aligned}
\left\langle\nabla_G\mathcal{L}(A^*), I_d \right\rangle_{\mathrm{Frob}} = \mathbb{E}_{z\sim q_0}\left[\left\langle\nabla\varphi(z),z\right\rangle\right] - \mathbb{E}_{z\sim q_0}\left[\left\langle\nabla\psi(z),z\right\rangle\right] \\
&= \frac{1}{\sigma^2}\mathbb{E}_{z\sim q_0}\left[\lVert z\rVert_2^2\right] - \mathbb{E}_{z\sim q_0}\left[\lVert z\rVert_2^2\right] = d\left(\frac{1}{\sigma^2}-1\right).
\end{aligned} ⟨ ∇ G L ( A ∗ ) , I d ⟩ Frob = E z ∼ q 0 [ ⟨ ∇ φ ( z ) , z ⟩ ] − E z ∼ q 0 [ ⟨ ∇ ψ ( z ) , z ⟩ ] = σ 2 1 E z ∼ q 0 [ ∥ z ∥ 2 2 ] − E z ∼ q 0 [ ∥ z ∥ 2 2 ] = d ( σ 2 1 − 1 ) .
Because ∥ I d ∥ F r o b 2 = d \lVert I_d\rVert_{\mathrm{Frob}}^2 = d ∥ I d ∥ Frob 2 = d and ∇ G L ( A ∗ ) ∈ span ( I d ) \nabla_G\mathcal{L}(A^*) \in\operatorname{span}(I_d) ∇ G L ( A ∗ ) ∈ span ( I d ) , we obtain the exact Riemannian gradient at A ∗ A^* A ∗ :
∇ G L ( A ∗ ) = ( 1 σ 2 − 1 ) I d . \nabla_G\mathcal{L}(A^*) = \left(\frac{1}{\sigma^2}-1\right)I_d. ∇ G L ( A ∗ ) = ( σ 2 1 − 1 ) I d .
This vanishes if and only if σ 2 = 1 \sigma^2 = 1 σ 2 = 1 . When σ 2 ≠ 1 \sigma^2 \ne1 σ 2 = 1 , recalibrating the base distribution along exp ( g b i a s H ) = { λ I d : λ > 0 } \exp(\mathfrak{g}_{\mathrm{bias}}^H) = \{\lambda I_d:\lambda> 0\} exp ( g bias H ) = { λ I d : λ > 0 } by choosing g Ω = σ I d g_\Omega= \sigma I_d g Ω = σ I d produces the calibrated base q Ω = ( g Ω ) ♯ q 0 = N ( 0 , σ 2 I d ) q_\Omega= (g_\Omega)_{\sharp}q_0 = \mathcal{N}(0,\sigma^2 I_d) q Ω = ( g Ω ) ♯ q 0 = N ( 0 , σ 2 I d ) with potential ψ Ω ( z ) = 1 2 σ 2 ∥ z ∥ 2 2 \psi_\Omega(z) = \frac{1}{2\sigma^2}\lVert z\rVert_2^2 ψ Ω ( z ) = 2 σ 2 1 ∥ z ∥ 2 2 . The moment-matching condition (5) is then satisfied identically, and ∇ G L ( A ∗ ) = 0 \nabla_G\mathcal{L}(A^*) = 0 ∇ G L ( A ∗ ) = 0 .
Square example: Polar coordinate derivation of the angular gradient
Consider Example 5 (G = S O ( 2 ) G = \mathrm{SO}(2) G = SO ( 2 ) , H = D 4 H = D_4 H = D 4 ) with quartic base distribution q 0 ( z ) ∝ exp ( − κ ( x 4 + y 4 ) ) q_0(z) \propto\exp\left(-\kappa(x^4+y^4)\right) q 0 ( z ) ∝ exp ( − κ ( x 4 + y 4 ) ) on R 2 \mathbb{R}^2 R 2 , where z = ( x , y ) T z=(x,y)^\mathsf{T} z = ( x , y ) T and κ > 0 \kappa> 0 κ > 0 . Passing to polar coordinates x = r cos ϕ x=r\cos\phi x = r cos ϕ , y = r sin ϕ y=r\sin\phi y = r sin ϕ and using the identity cos 4 ϕ + sin 4 ϕ = ( cos 2 ϕ + sin 2 ϕ ) 2 − 1 2 sin 2 ( 2 ϕ ) = 3 4 + 1 4 cos ( 4 ϕ ) \cos^4\phi+\sin^4\phi=(\cos^2\phi+\sin^2\phi)^2-\frac{1}{2}\sin^2(2\phi)=\frac{3}{4}+\frac{1}{4}\cos(4\phi) cos 4 ϕ + sin 4 ϕ = ( cos 2 ϕ + sin 2 ϕ ) 2 − 2 1 sin 2 ( 2 ϕ ) = 4 3 + 4 1 cos ( 4 ϕ ) , the base potential takes the form
ψ ( r , ϕ ) = κ ( r 4 cos 4 ϕ + r 4 sin 4 ϕ ) = κ r 4 ( 3 4 + 1 4 cos ( 4 ϕ ) ) . \psi(r,\phi)=\kappa\left(r^4\cos^4\phi+r^4\sin^4\phi\right)=\kappa r^4\left(\frac{3}{4}+\frac{1}{4}\cos(4\phi)\right). ψ ( r , ϕ ) = κ ( r 4 cos 4 ϕ + r 4 sin 4 ϕ ) = κ r 4 ( 4 3 + 4 1 cos ( 4 ϕ ) ) .
For an angular velocity u = θ ˙ ∈ s o ( 2 ) ≃ R u=\dot{\theta}\in\mathfrak{so}(2)\simeq\mathbb{R} u = θ ˙ ∈ so ( 2 ) ≃ R , the flow exp ( t θ ˙ ) ⋅ z = R t θ ˙ z \exp(t\dot{\theta})\cdot z=R_{t\dot{\theta}}z exp ( t θ ˙ ) ⋅ z = R t θ ˙ z has velocity field V θ ˙ ( x , y ) = θ ˙ ( − y , x ) T V_{\dot{\theta}}(x,y)=\dot{\theta}(-y,x)^\mathsf{T} V θ ˙ ( x , y ) = θ ˙ ( − y , x ) T . Consequently, the directional derivative of ψ \psi ψ along V θ ˙ V_{\dot{\theta}} V θ ˙ is
T 0 ( z ) : = ⟨ ∇ ψ ( z ) , V θ ˙ ( z ) ⟩ = 4 κ θ ˙ ( − x 3 y + x y 3 ) = θ ˙ ∂ ψ ∂ ϕ ( r , ϕ ) = − κ θ ˙ r 4 sin ( 4 ϕ ) . T_0(z):=\left\langle\nabla\psi(z),V_{\dot{\theta}}(z)\right\rangle=4\kappa\dot{\theta}\left(-x^3y+xy^3\right)=\dot{\theta}\frac{\partial\psi}{\partial\phi}(r,\phi)=-\kappa\dot{\theta}r^4\sin(4\phi). T 0 ( z ) := ⟨ ∇ ψ ( z ) , V θ ˙ ( z ) ⟩ = 4 κ θ ˙ ( − x 3 y + x y 3 ) = θ ˙ ∂ ϕ ∂ ψ ( r , ϕ ) = − κ θ ˙ r 4 sin ( 4 ϕ ) .
Now suppose the standardized target pullback is rotated by an angle α \alpha α , so that φ ( z ) = ψ ( R − α z ) = κ r 4 ( 3 4 + 1 4 cos ( 4 ( ϕ − α ) ) ) \varphi(z)=\psi(R_{-\alpha}z)=\kappa r^4\left(\frac{3}{4}+\frac{1}{4}\cos(4(\phi-\alpha))\right) φ ( z ) = ψ ( R − α z ) = κ r 4 ( 4 3 + 4 1 cos ( 4 ( ϕ − α )) ) . The directional derivative of φ \varphi φ along V θ ˙ V_{\dot{\theta}} V θ ˙ is similarly
T α ( z ) : = ⟨ ∇ φ ( z ) , V θ ˙ ( z ) ⟩ = − κ θ ˙ r 4 sin ( 4 ( ϕ − α ) ) . T_\alpha(z):=\left\langle\nabla\varphi(z),V_{\dot{\theta}}(z)\right\rangle=-\kappa\dot{\theta}r^4\sin(4(\phi-\alpha)). T α ( z ) := ⟨ ∇ φ ( z ) , V θ ˙ ( z ) ⟩ = − κ θ ˙ r 4 sin ( 4 ( ϕ − α )) .
By Theorem 6.5 , the directional derivative of the pullback loss J J J at 0 0 0 is
⟨ ∇ J ( 0 ) , θ ˙ ⟩ g = E z ∼ q 0 [ T α ( z ) − T 0 ( z ) ] = − κ θ ˙ E z ∼ q 0 [ r 4 ( sin ( 4 ( ϕ − α ) ) − sin ( 4 ϕ ) ) ] . \left\langle\nabla J(0),\dot{\theta}\right\rangle_{\mathfrak{g}}=\mathbb{E}_{z\sim q_0}\left[T_\alpha(z)-T_0(z)\right]=-\kappa\dot{\theta}\mathbb{E}_{z\sim q_0}\left[r^4\left(\sin(4(\phi-\alpha))-\sin(4\phi)\right)\right]. ⟨ ∇ J ( 0 ) , θ ˙ ⟩ g = E z ∼ q 0 [ T α ( z ) − T 0 ( z ) ] = − κ θ ˙ E z ∼ q 0 [ r 4 ( sin ( 4 ( ϕ − α )) − sin ( 4 ϕ ) ) ] .
Because the density q 0 ( r , ϕ ) ∝ exp ( − κ r 4 ( 3 4 + 1 4 cos ( 4 ϕ ) ) ) q_0(r,\phi)\propto\exp\left(-\kappa r^4\left(\frac{3}{4}+\frac{1}{4}\cos(4\phi)\right)\right) q 0 ( r , ϕ ) ∝ exp ( − κ r 4 ( 4 3 + 4 1 cos ( 4 ϕ ) ) ) is an even function of ϕ \phi ϕ while sin ( 4 ϕ ) \sin(4\phi) sin ( 4 ϕ ) is odd, we have E q 0 [ r 4 sin ( 4 ϕ ) ] = 0 \mathbb{E}_{q_0}[r^4\sin(4\phi)]=0 E q 0 [ r 4 sin ( 4 ϕ )] = 0 . Expanding sin ( 4 ( ϕ − α ) ) = sin ( 4 ϕ ) cos ( 4 α ) − cos ( 4 ϕ ) sin ( 4 α ) \sin(4(\phi-\alpha))=\sin(4\phi)\cos(4\alpha)-\cos(4\phi)\sin(4\alpha) sin ( 4 ( ϕ − α )) = sin ( 4 ϕ ) cos ( 4 α ) − cos ( 4 ϕ ) sin ( 4 α ) therefore simplifies the gradient to
⟨ ∇ J ( 0 ) , θ ˙ ⟩ g = κ θ ˙ sin ( 4 α ) E z ∼ q 0 [ r 4 cos ( 4 ϕ ) ] = − κ M 4 sin ( 4 α ) θ ˙ . \left\langle\nabla J(0),\dot{\theta}\right\rangle_{\mathfrak{g}}=\kappa\dot{\theta}\sin(4\alpha)\mathbb{E}_{z\sim q_0}\left[r^4\cos(4\phi)\right]=-\kappa M_4\sin(4\alpha)\dot{\theta}. ⟨ ∇ J ( 0 ) , θ ˙ ⟩ g = κ θ ˙ sin ( 4 α ) E z ∼ q 0 [ r 4 cos ( 4 ϕ ) ] = − κ M 4 sin ( 4 α ) θ ˙ .
where M 4 : = − E z ∼ q 0 [ r 4 cos ( 4 ϕ ) ] M_{4} := -\mathbb{E}_{z \sim q_{0}}[r^{4}\cos(4\phi)] M 4 := − E z ∼ q 0 [ r 4 cos ( 4 ϕ )] . Note that M 4 M_{4} M 4 is strictly positive for every κ > 0 \kappa> 0 κ > 0 : for each fixed radius r > 0 r > 0 r > 0 , the conditional angular density p ( ϕ ∣ r ) ∝ exp ( − 1 4 κ r 4 cos ( 4 ϕ ) ) p(\phi\mid r) \propto\exp\left(-\frac{1}{4}\kappa r^{4}\cos(4\phi)\right) p ( ϕ ∣ r ) ∝ exp ( − 4 1 κ r 4 cos ( 4 ϕ ) ) places strictly greater mass on angles where cos ( 4 ϕ ) < 0 \cos(4\phi) < 0 cos ( 4 ϕ ) < 0 (near the diagonals ϕ = ± π / 4 , ± 3 π / 4 \phi= \pm\pi/4, \pm3\pi/4 ϕ = ± π /4 , ± 3 π /4 ) than on angles where cos ( 4 ϕ ) > 0 \cos(4\phi) > 0 cos ( 4 ϕ ) > 0 (near the coordinate axes), ensuring E q 0 [ r 4 cos ( 4 ϕ ) ] < 0 \mathbb{E}_{q_{0}}[r^{4}\cos(4\phi)] < 0 E q 0 [ r 4 cos ( 4 ϕ )] < 0 . Thus ∇ J ( 0 ) = 0 \nabla J(0) = 0 ∇ J ( 0 ) = 0 if and only if sin ( 4 α ) = 0 \sin(4\alpha) = 0 sin ( 4 α ) = 0 , i.e., α ∈ ( π / 4 ) Z \alpha\in(\pi/4)\mathbb{Z} α ∈ ( π /4 ) Z .