betaFeedback

WorksMathematics

Lipschitz-Constrained Discriminators in Generative Modeling: Asymptotic Behavior and Support Mismatch

Abstract

We study the asymptotic behavior of optimal discriminators associated to L−L-Lipschitz regularized KL-divergences of two probability distributions QQ and PP that interpolate between the rescaled Wasserstein−1-1 metric by some parameter L>0L>0 and the KL divergence. The parameter LL plays the role of a Lipschitz constraint and is viewed as a regularization parameter: small values of LL enforce a more geometry-aware discriminator, while as LL increases the geometric regularization is progressively relaxed. It is a known result [3] that such divergences converge to the KL divergence as L→∞L\to\infty. In this paper, we characterize the asymptotic behavior of the optimal discriminators as L→∞L\to\infty in different regimes: Firstly, under Q≪PQ\ll P and the Radon-Nikodym derivative dQdP\frac{dQ}{dP} is bounded from above and bounded away from zero. Secondly, when QQ and PP are discrete distributions satisfying Q≪PQ\ll P, and finally when QQ and PP are non absolutely continuous discrete distributions. This result motivates training strategies which involve discriminators, such as adversarial generative models. In practice, the data distribution is often represented by a discrete empirical measure, while, especially at the early stages of the training, the model distribution may have little or no overlap with the empirical data distribution.

Paper details

vlnc:W_5cjyrq2g
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 7

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…