betaFeedback

WorksComputer Science

Error accumulation in reverse diffusion with an imperfect score: beyond path-space analyses

Abstract

Diffusion modeling is a popular method of sampling from a complex, high-dimensional distribution pp using learned approximations of its (noised) score functions. Generating a single sample requires simulating the reverse diffusion process, which requires many queries to the learned scores. Thus, a priori, the learning errors can easily compound. Indeed, in existing theoretical analyses, the sampling error (as measured in total variation distance to pp) accumulates polynomially with the horizon of the stochastic process. This is a consequence of bounding the error of the output law by the error of the law of the entire reverse diffusion path. In this paper, we demonstrate that by circumventing the "path-space" analysis, the error accumulation of the diffusion sampler can be mitigated when pp satisfies a Poincaré inequality. Moreover, we achieve this result as a special case of a broader framework that applies to any distribution pp, but relaxes the error metric from total variation distance to a weaker metric capturing all sample statistics that do not "witness" the multimodality of pp. We then zoom out from analyzing the diffusion sampler to the information-theoretic question: given a learned score with some error bound, how accurately can we identify pp? We show that, for any given bound on the Poincaré constant of pp, the diffusion sampler already nearly achieves optimal total variation error.

Paper details

vlnc:W_9v9sp6d7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
arXiv nonexclusive license
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…