betaFeedback

WorksMathematics

Bregman geometry in regular exponential families: dual representations and relative smoothness

Abstract

Regular exponential-family distributions are ubiquitous in statistics due to their appealing and convenient properties. Notably, they can be expressed in terms of Bregman divergences, revealing fundamental connections between statistics, convex analysis, and information geometry. However, this geometric structure has not been systematically exploited to characterize these distributions with respect to their expectation parametrization. Building on a projective characterization, we establish a general construction of equivalent dual Bregman representations, together with normalization and uniqueness results. These representations express log-densities as ordinary Bregman divergences in dual coordinate systems, providing geometric characterizations of the parameter space. So-called ββ- and αα-scaled dual representations emerge as particular instances of this framework. We further derive dual Bregman representations of conjugate priors and establish that they preserve conjugacy. Applications to classical distributions illustrate the geometric structures underlying seemingly different statistical models. Moreover, we show that these representations provide natural reference geometries for establishing relative smoothness of the primal Bregman representation of the distribution, with explicit bounds that are sharp in the scalar and separable settings. An illustrative application to the Poisson distribution sheds new light on several works in the literature, providing complementary geometric justifications for reference geometries while deriving existing algorithms.

Introduction

Exponential families constitute one of the most important classes of statistical models. Their broad applicability stems from a combination of analytical and statistical properties, including tractable likelihoods, finite-dimensional sufficient statistics, and the existence of natural parametrizations in terms of both natural and expectation parameters [14] (Barndorff-Nielsen 2014). They encompass many classical distributions, including Gaussian, Poisson, Bernoulli, Binomial, Gamma, and Beta distributions, and play a central role in statistical inference, information geometry, and machine learning.

An important connection between regular exponential families and convex analysis is provided by Bregman divergences. It is well established that, under standard regularity and minimality assumptions, the log-density of a regular exponential-family distribution can be expressed in terms of a Bregman divergence generated by the convex conjugate of the log-partition function [4]. This representation provides a geometric viewpoint of exponential families and has proved useful well beyond statistical interpretation. In particular, Bregman divergences constitute fundamental tools in optimization and statistical learning, where they underlie generalized projection methods, mirror descent, Bregman proximal algorithms, and a variety of estimation and learning procedures [9, 6]. The Bregman representation of an exponential-family distribution also endows its data space with a natural geometry, with the sufficient statistics playing the role of geometric coordinates.

This representation, however, reveals an important asymmetry between the two arguments of the Bregman divergence. In the canonical representation, the first argument corresponds to the sufficient statistics, whereas the second one is associated with the expectation parameter. Convexity with respect to the first argument is guaranteed by the convexity of the Bregman generator. In contrast, convexity with respect to the second argument is not generally guaranteed. Thus, although the Bregman divergence naturally describes the geometry of the data space, its reverse form does not necessarily define an ordinary Bregman geometry on the expectation-parameter space. This difficulty is particularly pronounced when conducting asymptotically exact data augmentation [27] and deriving splitting-based Monte Carlo algorithms [26] with non-Euclidean and, in particular, Bregman kernels. With respect to these two tasks, respectively, it has recently been overcome by augmenting the statistical model with the same divergence evaluated with swapped arguments [15] or, equivalently, by replacing the Bregman divergences by symmetrized counterparts defined as Jensen-Bregman divergences [13]. More generally and beyond these purely modeling and algorithmic workarounds, this divergence asymmetry raises a more fundamental question: can a reverse Bregman divergence itself be represented as an ordinary Bregman divergence after a suitable transformation of the coordinates? Then, can the Bregman geometry underlying a regular exponential-family distribution be transferred to its parameter space through an appropriate dual coordinate system? From an information geometry perspective, this question can be formulated as the search for a geometric structure underlying conformal divergences as studied by Amari and Nagaoka [2] and Nock et al. [21].

The objective of this paper is to answer this question. A sound, albeit incomplete, response is naturally provided by leveraging the role of Legendre duality in the geometry of exponential families [23]. However, the term duality, used hereafter and underlying the main rationale of this work, refers to a property intended to go beyond the sole Legendre duality. Thus, starting from the primal Bregman representation of regular exponential-family distributions, we build on a projective characterization providing necessary and sufficient conditions for the existence of such dual Bregman representations. We further establish a general construction of the corresponding dual coordinates and Bregman generators, together with normalization and uniqueness results. Among admissible representations, we exhibit two particular models, distinguished by the form of an associated scaling function and, consequently, by the resulting Bregman generator and parameter coordinates. The first one, referred to as the β\beta-scaled dual Bregman representation, recovers the self-evident representation induced by Legendre duality in exponential families. In this case, the resulting geometry is determined by the gradient of the original Bregman generator together with a scaled version of its Legendre conjugate. The second class, referred to as the α\alpha-scaled dual Bregman representation, relies on a different transformed coordinate system and an associated Bregman generator. Both constructions arise as particular instances of the general projective framework. These two constructions there fore provide distinct but equivalent geometric descriptions of the same exponential-family distribution. In particular, different parametrizations of the distribution can induce different Bregman geometries on its parameter space while remaining equivalent at the level of the underlying probability model.

An important application of the proposed framework concerns conjugate priors for exponential families. By applying the two dual Bregman representations previously identified to the conjugate distributions associated with regular exponential families, we obtain alternative geometric representations of these priors. These representations differ from the primal Bregman formulation of conjugate priors established by Agarwal and Daumé III (2010) in both their coordinate systems and associated Bregman generators, while retaining the defining conjugacy property under Bayesian updating. In this sense, the proposed framework does not modify the underlying probabilistic structure of conjugate priors, but provides alternative geometric descriptions of the same distributions. The resulting framework also recovers, as particular cases, dual Bregman representations associated with several classical statistical models and their conjugate priors. These examples illustrate how apparently different models can share common geometric structures and provide concrete interpretations of the proposed constructions.

Finally, the proposed framework also leads to consequences beyond the sole construction of these alternative Bregman representations. It is well known that relative smoothness generalizes the classical Lipschitz-gradient condition by measuring the variation of a function relative to a reference convex function, and has become an important notion in modern Bregman-based optimization [7, 18]. We show that the geometries induced by the β\beta- and α\alpha-scaled models identified in this work provide natural reference geometries for establishing relative smoothness of the negative log-density associated with the original, primal Bregman representation. These results are extended to separable multidimensional models, thereby covering a broad class of statistical models with component-wise structure. For the geometries induced by both models, we derive explicit relative-smoothness bounds, with sharp constants in the scalar and separable settings. These results establish a direct connection between the geometry of exponential-family distributions and the convex-analytic notion of relative smoothness, which is central to modern Bregman-based optimization.

Addressing Poisson inverse problems leads to useful illustrations in which the proposed geometric framework provides a principled interpretation of choices that have appeared in previous optimization and statistical-learning approaches.

The main contributions of this work can be summarized as follows:

  • Building on the projective characterization stated in Theorem 2, which provides necessary and sufficient conditions for a reverse Bregman divergence to admit an ordinary Bregman representation in transformed coordinates, Theorem 3 clarifies the normalization freedom and the associated uniqueness properties. Assuming the existence of admissible dual Bregman representations, Proposition 4 then derives reconstruction formulas of such representations;

  • Theorem 4 then establishes the existence and uniqueness of dual Bregman representations for regular minimal exponential families, with explicit construction formulas given in Theorem 5. Theorem 6 specifies how these dual representations translate to the log-density of regular exponential family. Theorem 7 identifies two classes of admissible dual representations, corresponding to the so-called β\beta- and α\alpha-scaled constructions, as particular instances of the general projective framework;

  • Starting from Proposition 5, which establishes the primal representation of posterior distributions resulting from conjugate priors, Theorems 8 and 9 show that the β\beta- and α\alpha-scaled dual Bregman representations can be instantiated to derive as alternatives of these priors, while preserving conjugacy properties;

  • Table 1 reports the main geometrical characteristics of these two particular representations for several classical exponential-family distributions, the corresponding derivations being detailed in Appendix § 10;

DistributionX\mathcal{X}β\beta-scaled dual model τ(x)\tau(x)h(t)h(t)T\mathcal{T}α\alpha-scaled dual model τ(x)\tau(x)h(t)h(t)T\mathcal{T}
Gaussian N(μ,σ2)\mathcal{N}(\mu,\sigma^2)R\mathbb{R}xσ2\frac{x}{\sigma^2}σ2t22\frac{\sigma^2t^2}{2}R\mathbb{R}N/A
Poisson P(λ)\mathcal{P}(\lambda)R+∗\mathbb{R}^{*}_{+}log⁡x\log xete^tR\mathbb{R}xx−log⁡t-\log tR+∗\mathbb{R}^{*}_{+}
Exponential E(b)\mathcal{E}(b)R+∗\mathbb{R}^{*}_{+}−1x-\frac{1}{x}−1−log⁡(−t)-1-\log(-t)R−\mathbb{R}_{-}log⁡x−1\log x-1e−t−1e^{-t-1}R\mathbb{R}
Gamma G(k,b)\mathcal{G}(k,b)R+∗\mathbb{R}^{*}_{+}−kx-\frac{k}{x}−klog⁡(−t)-k\log(-t)R−\mathbb{R}_{-}klog⁡x−kk\log x-kke−(t+k)/kke^{-(t+k)/k}R\mathbb{R}
Bernoulli Ber⁡(p)\operatorname{Ber}(p)(0,1)(0,1)log⁡x1−x\log\frac{x}{1-x}log⁡(1+et)\log(1+e^t)R\mathbb{R}−log⁡(1−x)-\log(1-x)−log⁡(et−1)-\log(e^t-1)R+∗\mathbb{R}^{*}_{+}
Binomial B(k,p)\mathcal{B}(k,p)(0,k)(0,k)log⁡xk−x\log\frac{x}{k-x}klog⁡(1+et)k\log(1+e^t)R\mathbb{R}−klog⁡(k−x)-k\log(k-x)−log⁡(et/k−1)-\log(e^{t/k}-1)(−klog⁡k,+∞)(-k\log k,+\infty)
Negative Binomial NB(r,p)\mathcal{NB}(r,p)R+∗\mathbb{R}^{*}_{+}log⁡xr+x\log\frac{x}{r+x}−rlog⁡(1−et)-r\log(1-e^t)R−\mathbb{R}_{-}rlog⁡(r+x)r\log(r+x)tr−log⁡(et/r−r)\frac{t}{r}-\log(e^{t/r}-r)(rlog⁡r,+∞)(r\log r,+\infty)
Inverse-Gaussian IG(μ,λ)\mathcal{IG}(\mu,\lambda)R+∗\mathbb{R}^{*}_{+}−λ2x2-\frac{\lambda}{2x^2}−−2λt-\sqrt{-2\lambda t}R−\mathbb{R}_{-}−λx-\frac{\lambda}{x}t22λ\frac{t^2}{2\lambda}R−∗\mathbb{R}^{*}_{-}
Gaussian N(μ,σ2)\mathcal{N}(\mu,\sigma^2){x1∈R,x2>x12}\{x_1\in\mathbb{R},x_2>x_1^2\}1x2−x12(x12,−12)\frac{1}{x_2-x_1^2}\left(\frac{x_1}{2},-\frac{1}{2}\right)see (117)R×R−∗\mathbb{R}\times\mathbb{R}^{*}_{-}N/A
Gamma G(k,b)\mathcal{G}(k,b){x2∈R+∗,x1<log⁡x2}\{x_2\in\mathbb{R}^{*}_{+},x_1<\log x_2\}no closed-formsee (118){t1>−1,t2<0}\{t_1>-1,t_2<0\}N/A
Beta Be(a,b)\mathcal{B}e(a,b)ran⁡∇f∗\operatorname{ran}\nabla f^{*}no closed-formsee (120){t1>−1,t2>−1}\{t_1>-1,t_2>-1\}N/A

Table 1. Dual Bregman representations of classical exponential family models. Fixed parameters appear in gray. Whenever well-identified, the divergences resulting from the α\alpha-scaled or β\beta-scaled generators hh are specified in Appendix C.

  • Propositions 6 and 7 show that both representations provide suitable reference geometries to ensure relative smoothness of the negative log-density associated with exponential-family distributions, with explicit bounds that are sharp in the scalar and separable settings;

  • We apply the proposed framework to Poisson inverse problems to recover several results from the literature. In particular, we show that Bregman gradient iterations under the geometries induced by the α\alpha-scaled and β\beta-scaled model recover the NoLips ([7]) and the expectation-maximization ([24]) algorithms, respectively.

The remainder of the paper is organized as follows. § 2 recalls the necessary background on Legendre duality, Bregman geometry, relative smoothness and regular exponential families. § 3 states the projective characterization of dual Bregman representations and establishes their general construction, normalization and uniqueness properties. In particular, we instantiate these representations to exhibit dual representations of prior distributions required in a Bayesian framework and show that they naturally recover expected conjugacy properties. We also derive these representations for several classical exponential-family distributions. § 4 shows that these representations also provide suitable reference geometries to ensure relative smoothness properties. As an illustrative example, § 5 considers the case of Poisson inverse problems. Finally, § 6 concludes the paper and discusses possible extensions.

Background and preliminaries

Legendre duality

Notions of Legendre duality are recalled below following the terminology adopted by Rockafellar [23].

Definition 1 (Legendre function). Let f:X⊆Rn→R∪{+∞}f : \mathcal{X} \subseteq\mathbb{R}^{n} \to\mathbb{R} \cup\{+\infty\} be a proper, lower semicontinuous, convex function. The function ff is said to be essentially smooth if it is differentiable throughout int⁡(dom⁡f)\operatorname{int}(\operatorname{dom} f) and ∥∇f(xk)∥→+∞\lVert\nabla f(x_k) \rVert\to+\infty whenever xk∈int⁡(dom⁡f)x_k \in\operatorname{int}(\operatorname{dom} f) converges to a boundary point of dom⁡f\operatorname{dom} f. Moreover, if ff is strictly convex on int⁡(dom⁡f)\operatorname{int}(\operatorname{dom} f), it is called a Legendre function.

Definition 2 (Fenchel conjugate). Let ff be a Legendre function with X=int⁡(dom⁡f)\mathcal{X} = \operatorname{int}(\operatorname{dom} f). The Fenchel conjugate of ff is defined for θ∈dom⁡f∗\theta\in\operatorname{dom} f^{*} by

f∗(θ)=sup⁡x∈dom⁡f{⟨θ,x⟩−f(x)}.f^{*}(\theta) = \sup_{\mathbf{x} \in\operatorname{dom} f} \left\lbrace\langle\theta,\mathbf{x}\rangle- f(\mathbf{x}) \right\rbrace.

The associated dual coordinate is

θ=∇f(x).\theta= \nabla f(\mathbf{x}).

A Legendre function and its conjugate satisfy the following duality property.

Proposition 1 (Legendre duality). The function f:X⊆Rn→R∪{+∞}f:\mathcal{X}\subseteq\mathbb{R}^{n}\to\mathbb{R}\cup\{+\infty\} is a Legendre function if and only if its Fenchel conjugate f∗f^{*} is also a Legendre function. Moreover, its gradient ∇f:int⁡(dom⁡f)→int⁡(dom⁡f∗)\nabla f:\operatorname{int}(\operatorname{dom} f)\to\operatorname{int}(\operatorname{dom} f^{*}) is a bijection and satisfies the identity

∇f∗=(∇f)−1\nabla f^{*}=(\nabla f)^{-1}

or, equivalently, ∀x∈X, θ=∇f(x)⇔x=∇f∗(θ)\forall x\in\mathcal{X},\ \theta=\nabla f(x)\Leftrightarrow x=\nabla f^{*}(\theta). Moreover, one has

f(x)+f∗(∇f(x))=⟨x,∇f(x)⟩.f(x)+f^{*}(\nabla f(x))=\langle x,\nabla f(x)\rangle.

Thus, for a Legendre function, the gradient establishes a one-to-one correspondence between the respective interiors of the effective domains of ff and f∗f^{*}. These properties are central in the construction of divergences underlying the Bregman geometry.

Bregman geometry

Definition 3 (Bregman divergences). ([9]) Let f:X⊆Rn→R∪{+∞}f:\mathcal{X}\subseteq\mathbb{R}^{n}\to\mathbb{R}\cup\{+\infty\} be a proper convex function that is differentiable on int⁡(dom⁡f)\operatorname{int}(\operatorname{dom} f). For y∈dom⁡fy\in\operatorname{dom} f and x∈int⁡(dom⁡f)x\in\operatorname{int}(\operatorname{dom} f), the Bregman divergence induced by ff is defined as

df(y,x)=f(y)−f(x)−⟨∇f(x),y−x⟩.d_f(y,x)=f(y)-f(x)-\langle\nabla f(x),y-x\rangle.

When ff is strictly convex, df(y,x)≥0d_f(y,x)\geq0, with

df(y,x)=0 ⟺ y=x.d_f(y,x)=0\ \Longleftrightarrow\ y=x.

It is well-known that the Bregman divergences satisfy the duality identity recalled in the following proposition.

Proposition 2 (Duality of Bregman divergences). ([9]) Let ff be a Legendre function. For every x,y∈int⁡(dom⁡f)x,y\in\operatorname{int}(\operatorname{dom} f), one has

df(y,x)=df∗(∇f(x),∇f(y)).(1)d_f(y,x)=d_{f^{*}}(\nabla f(x),\nabla f(y)). \tag*{(1)}

Relative smoothness

Relative smoothness generalizes the definition of Lipschitz continuity of the gradient by replacing the Euclidean quadratic distance with a Bregman divergence ([7]).

References

  1. [1]Arvind Agarwal and Hal Daumé III. A geometric view of conjugate priors. Machine learning, 81(1):99–113, 2010.arxiv.org/abs/1005.0047
  2. [2]Shun-ichi Amari and Hiroshi Nagaoka. Methods of information geometry, volume 191. American Mathematical Soc., 2000.
  3. [3]Alfred Auslender and Marc Teboulle. Interior gradient and proximal methods for convex and conic optimization. SIAM Journal on Optimization, 16(3):697–725, 2006.DOI
  4. [4]Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, and Joydeep Ghosh. Clustering with Bregman divergences. Journal of Machine Learning Research, 6(58):1705–1749, 2005.DOI
  5. [6]Heinz H Bauschke, Jonathan M Borwein, et al. Legendre functions and the method of random Bregman projections. J. Convex Anal, 4(1):27–67, 1997.DOI
  6. [7]Heinz H. Bauschke, Jérôme Bolte, and Marc Teboulle. A descent lemma beyond Lipschitz gradient continuity: First-order methods revisited and applications. Math. Oper. Res., 42:330–348, 2017.DOI
  7. [8]Amir Beck and Marc Teboulle. Mirror descent and nonlinear projected subgradient methods for convex optimization. Operations Research Letters, 31(3):167–175, 2003.DOI
  8. [9]Lev M Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR computational mathematics and mathematical physics, 7(3):200–217, 1967.DOI
  9. [10]Jonathan Chirinos-Rodríguez, Christian Daniele, Cédric Févotte, and Emmanuel Soubies. On the linear convergence of Bregman proximal gradient methods with applications to Kullback-Leibler regression. arXiv preprint, 2026. URL https://arxiv.org/abs/2607.05539.
  10. [11]Antonin Clerc, Ségolène Martin, Nicolas Papadakis, and Gabriele Steidl. Reinterpreting EMML as mirror descent for constrained maximum likelihood estimation. In Proc. European Signal Process. Conf. (EUSIPCO), 2026.arxiv.org/abs/2602.13063
  11. [12]Christian Daniele, Silvia Villa, Samuel Vaiter, and Luca Calatroni. Deep equilibrium models for Poisson imaging inverse problems via mirror descent. SIAM Journal on Imaging Sciences, 19(2):1077–1109, 2026.arxiv.org/abs/2507.11461
  12. [13]Nicolas Dobigeon, Mame Diarra Fall, and Eric Barat. Jensen-Bregman split Gibbs sampling. arXiv preprint, 2026. URL http://arxiv.org/abs/2610.15853.
  13. [14]Bradley Efron. Exponential families in theory and practice. Cambridge University Press Cambridge, 2023.DOI
  14. [15]Elhadji Cisse Faye, Mame Diarra Fall, Nicolas Dobigeon, and Eric Barat. Bregman geometry-aware split Gibbs sampling for Bayesian Poisson inverse problems. arXiv preprint, 2026. URL https://arxiv.org/abs/2511.12257.
  15. [16]Samuel Hurault, Ulugbek Kamilov, Arthur Leclaire, and Nicolas Papadakis. Convergent Bregman plug-and-play image restoration for Poisson inverse problems. In Advances in Neural Information Processing Systems, volume 36, 2023.arxiv.org/abs/2306.03466
  16. [17]Frederik Kunstner, Raunak Kumar, and Mark Schmidt. Homeomorphic-invariance of EM: Non-asymptotic convergence in KL divergence for exponential families via mirror descent. In International Conference on Artificial Intelligence and Statistics, pages 3295–3303. PMLR, 2021.arxiv.org/abs/2011.01170
  17. [18]Haihao Lu, Robert M Freund, and Yurii Nesterov. Relatively smooth convex optimization by first-order methods, and applications. SIAM Journal on Optimization, 28(1):333–354, 2018.arxiv.org/abs/1610.05708
  18. [19]Leon B Lucy. An iterative technique for the rectification of observed distributions. Astronomical Journal, Vol. 79, p. 745 (1974), 79:745, 1974.DOI
  19. [20]Thibaut Modrzyk, Élie Bretin, and Voichita Maxim. Right Bregman proximal gradient with application to Poisson inverse problems. arXiv preprint, 2026. URL https://arxiv.org/abs/2610.06579.
  20. [21]Richard Nock, Frank Nielsen, and Shun-ichi Amari. On conformal divergences and their population minimizers. IEEE Transactions on Information Theory, 62(1):527–538, 2015.arxiv.org/abs/1311.5125
  21. [22]William Hadley Richardson. Bayesian-based iterative method of image restoration. Journal of the optical society of America, 62(1):55–59, 1972.DOI
  22. [23]R. Tyrrell Rockafellar. Convex Analysis, volume 11 of Princeton Mathematical Series. Princeton University Press, Princeton, N.J., 1997.
  23. [24]Lawrence A Shepp and Yehuda Vardi. Maximum likelihood reconstruction for emission tomography. IEEE Transactions on Medical Imaging, 1(2):113–122, 1982.
  24. [25]Ryan J Tibshirani, Samy Wu Fung, Howard Heaton, and Stanley Osher. Laplace meets moreau: Smooth approximation to infimal convolutions using Laplace’s method. Journal of Machine Learning Research, 26(72):1–36, 2025.arxiv.org/abs/2406.02003
  25. [26]Maxime Vono, Nicolas Dobigeon, and Pierre Chainais. Split-and-augmented Gibbs sampler – application to large-scale inference problems. IEEE Transactions on Signal Processing, 67(6):1648–1661, 2019.DOI
  26. [27]Maxime Vono, Nicolas Dobigeon, and Pierre Chainais. Asymptotically exact data augmentation: Models, properties, and algorithms. Journal of Computational and Graphical Statistics, 30(2):335–348, 2020.

Paper details

vlnc:W_8mn7gkwt
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents