betaFeedback

WorksComputer Science

sDMD: Sinkhorn Distribution Matching Distillation

Abstract

Distribution Matching Distillation (DMD) distills multi-step diffusion models into few-step generators by minimizing reverse KL divergence. Despite its strong performance, DMD guides samples by local density alone, which introduces two limitations: teacher scores can be unreliable in regions of low teacher density, and the mode-seeking behavior of reverse KL can collapse the student onto a subset of the teacher modes. We first connect DMD to Wasserstein gradient flow, showing that its gradient follows from a fixed-point formulation with integral KL (IKL) as the energy functional. This perspective motivates augmenting IKL with an optimal transport term that addresses these limitations by guiding outlier samples toward the target and promoting mode coverage. We realize this term using Sinkhorn divergence estimated from finite batches. In practice, however, errors in the estimated Sinkhorn velocity can perturb in-domain samples for which IKL guidance is already accurate. To this end, we introduce sDMD, which adaptively activates the Sinkhorn correction based on discriminator outputs, focusing on outlier samples where IKL may be unreliable. With 2–4 sampling steps on FLUX.1-dev, Qwen-Image-20B, and Wan2.1-14B, sDMD outperforms distillation baselines on GenEval, DPG, WISE, and VBench while improving sample diversity over DMD2.

Introduction

Diffusion models ([67, 69, 33]) generate high-quality images and videos but require many sampling steps. Diffusion distillation reduces this cost by transferring knowledge from a pretrained teacher to a student that generates samples in one or a few steps. Existing approaches have advanced few-step generation by learning mappings along teacher trajectories ([23, 70]), using adversarial supervision ([65, 45]), or directly matching student and teacher distributions ([81, 91]). Among these, a representative approach is DMD ([84]), which optimizes a reverse KL objective using the difference between teacher and student scores. It has been widely adopted due to its high sample fidelity and scalability in large-scale image and video generation ([14, 89, 20]).

To better understand and improve the training dynamics of DMD, we view diffusion distillation from the perspective of Wasserstein gradient flow (WGF) ([1]). WGF evolves a probability distribution by steepest descent of an energy functional in Wasserstein space, with the corresponding velocity field specifying the update direction for each sample. While recent works ([29, 31]) use WGF to train one-step generators from scratch, we establish its connection to diffusion distillation by recovering the DMD gradient through fixed-point regression under the integral KL (IKL) energy, defined as reverse KL averaged over diffusion noise levels (§ 4.1). This identifies the energy underlying DMD and the velocity field that guides generated

samples, allowing us to analyze the behavior and limitations of DMD and explore complementary energy functionals to improve distillation.

The IKL velocity of DMD reveals two practical limitations (Figure 2). First, learned teacher scores can be unreliable for outlier samples, and the resulting updates may fail to guide them toward target modes. Although adding diffusion noise can improve distributional overlap, outliers may remain poorly supported at low noise levels. Higher noise levels can improve their support but blur the local structure of target modes. Second, the mode-seeking behavior of reverse KL can concentrate the student on a subset of target modes, potentially reducing the diversity of generated samples.

IKL and Sinkhorn velocity fields

Figure 2. IKL and Sinkhorn Velocity Fields. We visualize velocity fields from a 2D Gaussian mixture experiment. Black circles mark student particles and contours show target density. Solid (dashed) arrows use exact (learned) teacher scores for IKL and large (small) batches for Sinkhorn. (a) For in-domain student samples, the IKL velocity carries the particles onto the target. (b) Outlier samples under poor initialization, the IKL velocity can be inaccurate in regions of low teacher density. (c) The mode-seeking behavior of IKL can concentrate particles on a subset of the target modes. (d) The Sinkhorn velocity guides particles toward the target modes through the transport plan, remaining robust even for outlier samples (blue box). In practice, finite-sample errors in the Sinkhorn velocity can perturb samples for which IKL already provides reliable guidance (red boxes). We address this using our gated scheduler in Sec. 5.2. (a) IKL: aligned samples. (b) IKL: outlier samples. (c) IKL: mode seeking. (d) Sinkhorn transport.

To address these limitations, we augment IKL with an optimal transport (OT) term. OT couples student and target samples globally, providing robust guidance for outlier samples and promoting mode coverage without relying on learned teacher scores. To exploit their complementary strengths, we propose combining global OT guidance with local IKL score updates that help capture fine details. Our analysis shows that sufficiently strong exact OT regularization ensures convergence of the combined flow to the target under regularity assumptions (§ 4.2). For efficient computation, we use Sinkhorn divergence, a debiased form of entropic OT. While Sinkhorn does not automatically inherit this guarantee, we establish sufficient conditions on its approximation error and regularization strength for the combined flow to converge to the target (Sec. 5).

In practice, the transport maps used to compute Sinkhorn updates are estimated from finite samples, with larger batches generally reducing estimation error at greater computational cost [22]. Notably, our experiments show that even with relatively few samples, Sinkhorn can provide more informative guidance than IKL for outlier samples where learned teacher scores are unreliable. However, for well-supported samples, estimation errors in Sinkhorn can instead interfere with the accurate guidance already provided by IKL. To this end, we introduce sDMD, which adaptively activates Sinkhorn guidance for samples with relatively weak teacher support based on discriminator outputs. We reuse the discriminator trained for adversarial regularization in DMD2 [83], without training a separate model.

We evaluate sDMD using 2–4 sampling steps on FLUX.1-dev [42] and Qwen-Image-20B [78] for text-to-image generation (Figure 1) and Wan2.1 [74] (1.3B and 14B) for text-to-video generation. sDMD outperforms the distillation baselines in most metrics on GenEval [25], DPG [35], and WISE [57], achieves the highest overall VBench scores on both Wan2.1 models, and improves all reported diversity metrics over the strong DMD2 [83] baseline.

Teaser. 4-step generated images using sDMD with FLUX.1 and Qwen-Image.

Figure 1. Teaser. 4-step generated images using sDMD with FLUX.1 and Qwen-Image.

Related Work

Diffusion Distillation. To reduce the sampling cost of multi-step diffusion models, trajectory-based distillation methods enable one or few-step generation by learning shortcuts along denoising trajectories [70, 5, 18]. Continuous-time formulations use JVPs [48, 23], whose efficient computation at scale requires specialized implementations [89]. Recent works aim to avoid JVP computation through specialized architectures that decouple sampling substeps from backbone evaluations [7, 66]. Adversarial approaches use discriminator-based supervision for distillation [64, 65] with potential training instability [45].

Distribution-matching distillation methods align generated and target distributions through ff-divergence minimization [81, 80] or score matching [91, 51]. Among these, DMD [84, 50] minimizes the reverse KL divergence using the difference between teacher and student scores. With their high sample fidelity and scalability, DMD-based methods have been widely adopted for high-resolution image synthesis and video generation [85, 36, 89, 83]. However, DMD remains sensitive to outliers and the mode-seeking behavior of reverse KL [4, 52]. Our work augments DMD with a complementary velocity field to address these limitations and improve sample quality and diversity.

Wasserstein Gradient Flow for Generative Modeling. WGF [39, 1] evolves a probability distribution by steepest descent of an energy functional under the 2-Wasserstein metric. When a divergence to the target distribution is chosen as the energy, flow reduces the discrepancy between the two distributions, providing a framework for generative modeling. For example, MMD flows [2] depend on kernel gradients and can provide weak guidance when generated and target samples are far apart relative to the kernel bandwidth. This requires careful kernel design, such as noise-adaptive kernels [19] or kernels built from multiple feature encoders and scales [16, 12].

Sinkhorn divergence provides transport-based guidance [21, 17], and recent works [29, 31] use its gradient flow to train one-step generators. However, estimating transport from finite batches can introduce error into the resulting velocity field [60, 22]. Flows based on reverse KL between clean distributions use local score information to guide samples toward the target distribution [44, 47, 30]. However, these approaches require known target densities or score estimates obtained through methods such as kernel density estimation, which can limit scalability in high-dimensional generation [30, 29]. In contrast, DMD offers a scalable diffusion distillation approach by applying reverse KL to noise-perturbed distributions with noise annealing. In this work, we connect DMD to WGF and propose sDMD, which incorporates an optimal transport term to improve guidance for outlier samples and promote mode coverage.

Preliminaries

Problem Setup and Notation. We work on Rd\mathbb{R}^{d} and denote by P2(Rd)\mathcal{P}_{2}(\mathbb{R}^{d}) the set of probability distributions. Let p∈P2(Rd)p \in\mathcal{P}_{2}(\mathbb{R}^{d}) denote the target distribution and let pref=N(0,Id)p_{\mathrm{ref}} = \mathcal{N}(0, I_{d}) be the reference distribution. We assume access to a pretrained multi-step teacher for pp that gives scores at different noise levels. Our goal is to distill this teacher into a student generator fθ:Rd→Rdf_{\theta} : \mathbb{R}^{d} \to\mathbb{R}^{d} that generates a sample in one or a few steps, such that its output distribution q=(fθ)#prefq = (f_{\theta})_{\#}p_{\mathrm{ref}} approximates pp.

Distribution Matching Distillation (DMD). DMD [84] distills a multi-step diffusion teacher into a few-step generator by matching the student distribution qq to the teacher distribution pp. The original formulation starts from reverse KL between clean distributions, whose gradient involves the difference between the student and target scores. To approximate the gradient, it introduces diffusion annealing, replacing the clean-distribution scores with scores of noisy distributions. For a few-step generated sample x=fθ(z)x = f_{\theta}(z) with z∼prefz \sim p_{\mathrm{ref}}, the forward diffusion process gives xt=αtx+σtϵx_{t} = \alpha_{t}x + \sigma_{t}\epsilon, where ϵ∼N(0,Id)\epsilon\sim\mathcal{N}(0, I_{d}) and (αt,σt)(\alpha_{t}, \sigma_{t}) defines the noise schedule. We denote the corresponding perturbed marginals by qtq_{t} and ptp_{t}, then the distribution-matching gradient takes the form:

∇DMD(θ)=−Et,z,ϵ[αt(∇log⁡pt(xt)−∇log⁡qt(xt))∂fθ(z)∂θ].(1)\nabla_{\mathrm{DMD}}(\theta) = -\mathbb{E}_{t,z,\epsilon}\left[\alpha_{t}\left(\nabla\log p_{t}(x_{t}) - \nabla\log q_{t}(x_{t})\right)\frac{\partial f_{\theta}(z)}{\partial\theta}\right]. \tag*{(1)}

In practice, the pretrained teacher provides target scores, while an auxiliary diffusion model learns the student score online from generated samples using a score-matching objective [73].

Wasserstein Gradient Flow (WGF). Let F:P2(Rd)→R\mathcal{F} : \mathcal{P}_{2}(\mathbb{R}^{d}) \to\mathbb{R} be an energy functional on distributions that satisfies arg⁡min⁡qF(q)=p\arg\min_{q}\mathcal{F}(q) = p. WGF [1, 39] describes the evolution of the probability density {qτ}τ≥0\{q_{\tau}\}_{\tau\ge0}, where τ\tau denotes gradient-flow time. The evolution is characterized by the following partial differential equation [72]:

∂τqτ+∇⋅(qτVτ)=0,Vτ(x)=−∇δFδq(qτ)(x),(2)\partial_{\tau}q_{\tau} + \nabla\cdot(q_{\tau}V_{\tau}) = 0, \qquad V_{\tau}(x) = -\nabla\frac{\delta\mathcal{F}}{\delta q}(q_{\tau})(x), \tag*{(2)}

where the velocity field VτV_{\tau} is the negative gradient of the first variation of F\mathcal{F}. Discretizing this flow in time, each iteration displaces every particle xτ∼qτx_{\tau} \sim q_{\tau} by an Euler step, i.e., xτ+1=xτ+Vτ(xτ)x_{\tau+1} = x_{\tau} + V_{\tau}(x_{\tau}). At the equilibrium qτ=pq_{\tau} = p, the velocity vanishes and each step leaves every particle in place. Writing the particles as generator output x=fθ(z)x = f_{\theta}(z), the fixed-point relation gives the following regression objective on generator parameters [12]:

L(θ)=Ez∼pref[∥fθ(z)−sg⁡(fθ(z)+Vτ(fθ(z)))∥2],(3)\mathcal{L}(\theta) = \mathbb{E}_{z \sim p_{\mathrm{ref}}}\left[\left\|f_{\theta}(z) - \operatorname{sg}\left(f_{\theta}(z) + V_{\tau}(f_{\theta}(z))\right)\right\|^{2}\right], \tag*{(3)}

where sg⁡(⋅)\operatorname{sg}(\cdot) denotes the stop-gradient, and the loss decreases as Vτ→0V_{\tau} \to0. Note that different energy functionals induce different velocity fields and shape the training dynamics. Therefore, we connect DMD to WGF to explore the design space of energy functionals for distillation and introduce a complementary transport term to address the limitations of DMD.

Entropic Optimal Transport and Sinkhorn Divergence. Entropic optimal transport (EOT) compares qq and pp via a regularized transport cost which can be computed using the Sinkhorn algorithm [10, 41]. For the quadratic cost c(x,y)=12∥x−y∥2c(x,y) = \frac{1}{2}\lVert x-y\rVert^2 and regularization strength β>0\beta> 0, EOT is defined as:

OT⁡β(q,p)=min⁡π∈Π(q,p)∫c(x,y) dπ(x,y)+βDKL(π∥q⊗p)(4)\operatorname{OT}_{\beta}(q,p)=\min_{\pi\in\Pi(q,p)}\int c(x,y)\,\mathrm{d}\pi(x,y)+\beta D_{\mathrm{KL}}(\pi\mathbin\Vert q\otimes p) \tag*{(4)}

where Π(q,p)\Pi(q,p) denotes the couplings with marginals qq and pp. The debiased Sinkhorn divergence removes entropic bias by subtracting the self-transport costs [21]:

Sβ(q,p)=OT⁡β(q,p)−12OT⁡β(q,q)−12OT⁡β(p,p).(5)S_{\beta}(q,p)=\operatorname{OT}_{\beta}(q,p)-\frac{1}{2}\operatorname{OT}_{\beta}(q,q)-\frac{1}{2}\operatorname{OT}_{\beta}(p,p). \tag*{(5)}

SβS_{\beta} is nonnegative and vanishes if and only if q=pq=p [17]. Let πq,p\pi_{q,p} denote the optimal entropic transport plan in Eq. (4), and define its barycentric projection as Tq,p(x)=∫y dπq,p(y∣x)T_{q,p}(x)=\int y\,\mathrm{d}\pi_{q,p}(y\mid x), the weighted average of target points under the transport plan; Tq,q(x)T_{q,q}(x) denotes the corresponding projection under the self-transport plan. Then, the WGF velocity can be written as the difference between the two barycentric projections [21]:

VSβ(x)=−∇δSβ(q,p)δq(x)=Tq,p(x)−Tq,q(x).(6)V_{S_{\beta}}(x)=-\nabla\frac{\delta S_{\beta}(q,p)}{\delta q}(x)=T_{q,p}(x)-T_{q,q}(x). \tag*{(6)}

Analyzing DMD through Wasserstein Gradient Flow

Connection of DMD to WGF

DMD Yin et al., [84] has shown strong empirical performance in few-step generation, but the original derivation of the gradient introduces diffusion noise without a formal connection to WGF. We show that the DMD gradient in Eq. (1) follows from the WGF fixed-point relation in Eq. (3). Specifically, the DMD update can be viewed through the WGF framework by choosing the integral KL (IKL) divergence [50], i.e., reverse KL averaged over diffusion noise levels, as the energy functional:

FIKL(q)=Et[DKL(qt∥pt)],VIKL(x)≡(i)Et,ϵ[αt(∇log⁡pt(xt)−∇log⁡qt(xt))],(7)\mathcal{F}_{\mathrm{IKL}}(q)=\mathbb{E}_{t}\left[D_{\mathrm{KL}}(q_t\mathbin\Vert p_t)\right],\qquad V_{\mathrm{IKL}}(x)\mathrel{\mathop{\equiv}^{\mathrm{(i)}}}\mathbb{E}_{t,\epsilon}\left[\alpha_t\left(\nabla\log p_t(x_t)-\nabla\log q_t(x_t)\right)\right], \tag*{(7)}

where (i) follows by substituting FIKL\mathcal{F}_{\mathrm{IKL}} into the WGF velocity in Eq. (2). Plugging this velocity into the fixed-point relation in Eq. (3) and taking the gradient gives:

∇θLIKL(θ)≔∇θEz[∥fθ(z)−sg⁡(fθ(z)+VIKL(fθ(z)))∥2]=−2Ez[VIKL(fθ(z))∂fθ(z)∂θ],(8)\nabla_{\theta}\mathcal{L}_{\mathrm{IKL}}(\theta)\coloneqq\nabla_{\theta}\mathbb{E}_{z}\left[\left\|f_{\theta}(z)-\operatorname{sg}\left(f_{\theta}(z)+V_{\mathrm{IKL}}(f_{\theta}(z))\right)\right\|^2\right]=-2\mathbb{E}_{z}\left[V_{\mathrm{IKL}}(f_{\theta}(z))\frac{\partial f_{\theta}(z)}{\partial\theta}\right], \tag*{(8)}

which recovers the DMD gradient in Eq. (1) up to a constant after substituting VIKLV_{\mathrm{IKL}} from Eq. (7). We present the detailed derivation in Appendix § 11.§ 1. This connection allows us to analyze the behavior and limitations of DMD through its velocity field and explore synergistic energies to improve distillation.

Connection to Prior and Concurrent Work. Diff-Instruct [50] derives the generator score-difference update directly from IKL, but does not explicitly connect it to WGF. ProlificDreamer [76] derives a WGF update for text-to-3D particles, but does not consider distillation into few-step image or video generators. Concurrent analyses [27, 71] study WGF fixed-point regression for drifting models [12] using kernel-smoothed KL rather than IKL, and also do not explicitly connect DMD to WGF. In this work, we connect DMD to WGF under diffusion-noise-averaged KL from fixed-point regression and use this interpretation to introduce complementary Sinkhorn guidance for diffusion distillation.

Limitations of IKL

DMD [84] and its follow-up methods [15, 20] have demonstrated empirical success in capturing fine details and scaling to high-resolution image generation [14, 83] and video generation [28, 89, 36]. However, the gradient of IKL in DMD guides particles by local density alone, which brings two practical limitations. First, the clean reverse KL gradient flow is ill-defined when pp and qq are mutually singular [26]. Although forward diffusion gives ptp_t and qtq_t full support, in practice the learned teacher score may remain unreliable for generated samples with low teacher density, particularly at low noise levels. Second, the mode-seeking behavior of reverse KL can cause the student to cover only a subset of the teacher modes.

References

  1. [1]Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient Flows in Metric Spaces and in the Space of Probability Measures. Birkhäuser, 2nd edition, 2008.
  2. [2]Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. In NeurIPS, 2019.arxiv.org/abs/1906.04370
  3. [3]Mahmoud Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, Vasil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil Krishnakumar, Yong Li, Xiaodong Ma, Sarath Chandar, Franziska Meier, Yann LeCun, Michael Rabbat, and Nicolas Ballas. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025.
  4. [4]Lichen Bai, Zikai Zhou, Shitong Shao, Wenliang Zhong, Shuo Yang, Shuo Chen, Bojun Chen, and Zeke Xie. Optimizing few-step generation with adaptive matching distillation. arXiv preprint arXiv:2602.07345, 2026.arxiv.org/abs/2602.07345
  5. [5]Nicholas M. Boffi, Michael S. Albergo, and Eric Vanden-Eijnden. How to build a consistency model: Learning flow maps via self-distillation. arXiv preprint arXiv:2505.18825, 2025.arxiv.org/abs/2505.18825
  6. [6]Clément Bonet, Nicolas Courty, François Septier, and Lucas Drumetz. Efficient gradient flows in sliced-wasserstein space. arXiv preprint arXiv:2110.10972, 2021.arxiv.org/abs/2110.10972
  7. [7]Hansheng Chen, Kai Zhang, Hao Tan, Leonidas Guibas, Gordon Wetzstein, and Sai Bi. pi-flow: Policy-based few-step generation via imitation distillation. In ICLR, 2026.arxiv.org/abs/2510.14974
  8. [8]Zhenglin Cheng, Peng Sun, Jianguo Li, and Tao Lin. Twinflow: Realizing one-step generation on large models with self-adversarial flows. In ICLR, 2026.arxiv.org/abs/2512.05150
  9. [9]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scalable wasserstein gradient flow for generative modeling through unbalanced optimal transport. arXiv preprint arXiv:2402.05443, 2024.arxiv.org/abs/2402.05443
  10. [10]Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In NeurIPS, 2013.
  11. [11]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
  12. [12]Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. Generative modeling via drifting. arXiv preprint arXiv:2602.04770, 2026.arxiv.org/abs/2602.04770
  13. [13]Jiaojiao Fan, Qinsheng Zhang, Amirhossein Taghvaei, and Yongxin Chen. Variational wasserstein gradient flow. arXiv preprint arXiv:2112.02424, 2021.arxiv.org/abs/2112.02424
  14. [14]Xiangyu Fan, Zesong Qiu, Fanzhou Wang, Peng Gao, and Lei Yang. Qwen-image-lightning, 2025. URL https://github.com/ModelTC/LightX2V-Qwen-Image-Lightning.
  15. [15]Xiangyu Fan, Zesong Qiu, Zhuguanyu Wu, Fanzhou Wang, Zhiqian Lin, Tianxiang Ren, Dahua Lin, Ruihao Gong, and Lei Yang. Phased dmd: Few-step distribution matching distillation via score matching within subintervals. In CVPR, 2026.arxiv.org/abs/2510.27684
  16. [16]Lan Feng, Wuyang Li, Eloi Zablocki, Matthieu Cord, and Alexandre Alahi. Representation distribution matching for one-step visual generation. arXiv preprint arXiv:2607.02375, 2026.arxiv.org/abs/2607.02375
  17. [17]Jean Feydy, Thibault Séjourné, François-Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. In AISTATS, 2019.arxiv.org/abs/1810.08278
  18. [18]Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In ICML, 2025.arxiv.org/abs/2410.12557
  19. [19]Alexandre Galashov, Valentin De Bortoli, and Arthur Gretton. Deep MMD gradient flow without adversarial training. In ICLR, 2025.arxiv.org/abs/2405.06780
  20. [20]Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation. arXiv preprint arXiv:2506.00523, 2025.arxiv.org/abs/2506.00523
  21. [21]Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with sinkhorn divergences. In AISTATS, 2018.arxiv.org/abs/1706.00292
  22. [22]Aude Genevay, Lénaïc Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complexity of sinkhorn divergences. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89, pp. 1574–1583. PMLR, 2019. URL https://proceedings.mlr.press/v89/genevay19a.html.arxiv.org/abs/1810.02733
  23. [23]Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In NeurIPS, 2025.arxiv.org/abs/2505.13447
  24. [24]Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models. In CVPR, 2026.arxiv.org/abs/2512.02012
  25. [25]Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. In NeurIPS, 2023.arxiv.org/abs/2310.11513
  26. [26]Pierre Glaser, Michael Arbel, and Arthur Gretton. KALE flow: A relaxed KL gradient flow for probabilities with disjoint support. In NeurIPS, 2021.arxiv.org/abs/2106.08929
  27. [27]Arthur Gretton, Li Kevin Wenliang, Alexandre Galashov, James Thornton, Valentin De Bortoli, and Arnaud Doucet. On the wasserstein gradient flow interpretation of drifting models. arXiv preprint arXiv:2605.05118, 2026.arxiv.org/abs/2605.05118
  28. [28]Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, and Mike Zheng Shou. Anyflow: Any-step video diffusion model with on-policy flow map distillation. arXiv preprint arXiv:2605.13724, 2026.arxiv.org/abs/2605.13724
  29. [29]Jiaqi Han, Puheng Li, Qiushan Guo, Renyuan Xu, Stefano Ermon, and Emmanuel J. Candès. One-step generative modeling via wasserstein gradient flows. arXiv preprint arXiv:2605.11755, 2026.arxiv.org/abs/2605.11755
  30. [30]Jiajun He, Wenlin Chen, Mingtian Zhang, David Barber, and José Miguel Hernández-Lobato. Training neural samplers with reverse diffusive kl divergence. In AISTATS, 2025.arxiv.org/abs/2410.12456
  31. [31]Ping He, Om Khangaonkar, Hamed Pirsiavash, Yikun Bai, and Soheil Kolouri. Sinkhorn-drifting generative models. arXiv preprint arXiv:2603.12366, 2026.arxiv.org/abs/2603.12366
  32. [32]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local Nash equilibrium. NeurIPS, 2017.
  33. [33]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.arxiv.org/abs/2006.11239
  34. [34]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.arxiv.org/abs/2106.09685
  35. [35]Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135, 2024.arxiv.org/abs/2403.05135
  36. [36]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In NeurIPS, 2025.arxiv.org/abs/2506.08009
  37. [37]Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, 2024.arxiv.org/abs/2311.17982
  38. [38]Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In CVPR, 2024.arxiv.org/abs/2401.09603
  39. [39]Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker–planck equation. SIAM Journal on Mathematical Analysis, 1998.
  40. [40]Sunwoo Kim, Minky u Kim, and Dongmin Park. Test-time alignment of diffusion models without reward over-optimization. In ICLR, 2025.arxiv.org/abs/2501.05803
  41. [41]Philip A. Knight. The sinkhorn–knopp algorithm: convergence and applications. SIAM Journal on Matrix Analysis and Applications, 2008.DOI
  42. [42]Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2024.
  43. [43]Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2.5: Three insights towards enhancing aesthetic quality in text-to-image generation, 2024.arxiv.org/abs/2402.17245
  44. [44]Lingxiao Li, Samuel Hurault, and Justin Solomon. Self-consistent velocity matching of probability flows. In NeurIPS, 2023.arxiv.org/abs/2301.13737
  45. [45]Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang, Xuefeng Xiao, and Lu Jiang. Diffusion adversarial post-training for one-step video generation. In ICML, 2025.arxiv.org/abs/2501.08316
  46. [46]Dongyang Liu, Gao Peng, David Liu, DU DU, Zhen Li, Qilong Wu, Xin Jin, Sihan Cao, Shifeng Zhang, Steven Hoi, et al. Decoupled dmd: Cfg augmentation as the spear, distribution matching as the shield. In ICLR, 2026.arxiv.org/abs/2511.22677
  47. [47]Qiang Liu. Stein variational gradient descent as gradient flow. In NeurIPS, 2017.arxiv.org/abs/1704.07520
  48. [48]Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. In ICLR, 2025.arxiv.org/abs/2410.11081
  49. [49]Yiyang Lu, Susie Lu, Qiao Sun, Hanhong Zhao, Zhicheng Jiang, Xianbang Wang, Tianhong Li, Zhengyang Geng, and Jiaming He. One-step latent-free image generation with pixel mean flows. arXiv preprint arXiv:2601.22158, 2026.arxiv.org/abs/2601.22158
  50. [50]Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In NeurIPS, 2023.arxiv.org/abs/2305.18455
  51. [51]Weijian Luo, Zemin Huang, Zhengyang Geng, J Zico Kolter, and Guo-jun Qi. One-step diffusion distillation through score implicit matching. In NeurIPS, 2024.arxiv.org/abs/2410.16794
  52. [52]Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang. Learning few-step diffusion models by trajectory distribution matching. In ICCV, 2025.arxiv.org/abs/2503.06674
  53. [53]Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In ECCV, 2024.arxiv.org/abs/2401.08740
  54. [54]Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, and Evgeny Burnaev. Large-scale wasserstein gradient flows. In NeurIPS, 2021.arxiv.org/abs/2106.00736
  55. [55]Weili Nie, Julius Berner, Chao Liu, and Arash Vahdat. Nvidia fastgen: Fast generation from diffusion models, 2026a. URL https://github.com/NVlabs/FastGen.
  56. [56]Weili Nie, Julius Berner, Nanye Ma, Chao Liu, Saining Xie, and Arash Vahdat. Transition matching distillation for fast video generation. arXiv preprint arXiv:2601.09881, 2026b.arxiv.org/abs/2601.09881
  57. [57]Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Fanqing Meng, Kunpeng Ning, et al. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07265, 2025.arxiv.org/abs/2503.07265
  58. [58]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vỏ, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francis Massa, Alaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.arxiv.org/abs/2304.07193
  59. [59]Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2024.arxiv.org/abs/2410.13720
  60. [60]Aram-Alexandre Pooladian and Jonathan Niles-Weed. Entropic estimation of optimal transport maps. arXiv preprint arXiv:2109.12004, 2021.arxiv.org/abs/2109.12004
  61. [61]Aram-Alexandre Pooladian, Marco Cuturi, and Jonathan Niles-Weed. Debiaser beware: Pitfalls of centering regularized transport maps. In ICML, volume 162, pp. 17830–17847. PMLR, 2022. URL https://proceedings.mlr.press/v162/pooladian22a.html.arxiv.org/abs/2202.08919
  62. [62]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.arxiv.org/abs/2103.00020
  63. [63]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
  64. [64]Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042, 2023.arxiv.org/abs/2311.17042
  65. [65]Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403.12015, 2024.arxiv.org/abs/2403.12015
  66. [66]Neta Shaul, Chao Liu, Arash Vahdat, and Julius Berner. Parallel decoding distillation for fast image and video generation. arXiv preprint arXiv:2607.26004, 2026.arxiv.org/abs/2607.26004
  67. [67]Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
  68. [68]Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In ICML, 2024.arxiv.org/abs/2310.14189
  69. [69]Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
  70. [70]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023.arxiv.org/abs/2303.01469
  71. [71]Erkan Turan, Nicolas Dufour, and Maks Ovsianikov. Generative drifting is secretly score matching: a spectral and variational perspective. arXiv preprint arXiv:2603.09936, 2026. URL https://arxiv.org/abs/2603.09936.
  72. [72]Cédric Villani et al. Optimal transport: old and new. Springer, 2009.
  73. [73]Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 2011.DOI
  74. [74]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jingren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, Keyu Yan, Lianhua Huang, Mengyang Feng, Ningyi Zhang, Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, Tingyu Yang, Tong Shen, Wei Lin, Wei Wang, Wei Wang, Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yitong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025.arxiv.org/abs/2503.20314
  75. [75]Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yinan He, Yi Wang, Yali Wang, and Yu Qiao. VideoMAE V2: Scaling video masked autoencoders with dual masking. In CVPR, 2023a.arxiv.org/abs/2303.16727
  76. [76]Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. NeurIPS, 2023b.arxiv.org/abs/2305.16213
  77. [77]Zidong Wang, Yiyuan Zhang, Xiaoyu Yue, Xiangyu Yue, Yangguang Li, Wanli Ouyang, and Lei Bai. Transition models: Rethinking the generative learning objective. In CVPR, 2026.arxiv.org/abs/2509.04394
  78. [78]Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Youxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-image technical report, 2025.arxiv.org/abs/2508.02324
  79. [79]Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023.arxiv.org/abs/2306.09341
  80. [80]Sirui Xie, Zhisheng Xiao, Diederik P Kingma, Tingbo Hou, Ying N Wu, Kevin Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em distillation for one-step diffusion models. In NeurIPS, volume 37, pp. 45073–45104, 2024.arxiv.org/abs/2405.16852
  81. [81]Yilun Xu, Weili Nie, and Arash Vahdat. One-step diffusion models with f-divergence distribution matching. arXiv preprint arXiv:2502.15681, 2025.arxiv.org/abs/2502.15681
  82. [82]Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbiang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time interactive long video generation. arXiv preprint arXiv:2509.22622, 2025.arxiv.org/abs/2509.22622
  83. [83]Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Frédo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis. In NeurIPS, 2024a.arxiv.org/abs/2405.14867
  84. [84]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024b.arxiv.org/abs/2311.18828
  85. [85]Tianwei Yin, Qiang Zhang, Richard Zhang, William T. Freeman, Frédo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.arxiv.org/abs/2412.07772
  86. [86]Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940, 2024.arxiv.org/abs/2410.06940
  87. [87]Huijie Zhang, Aliaksandr Siarohin, Willi Menapace, Michael Vasilkovsky, Sergey Tulyakov, Qing Qu, and Ivan Skorokhodov. Alphaflow: Understanding and improving meanflow models. In ICLR, 2026.arxiv.org/abs/2510.20771
  88. [88]Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie. Diffusion transformers with representation autoencoders. In ICLR, 2026a.arxiv.org/abs/2510.11690
  89. [89]Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency. In ICLR, 2026b.arxiv.org/abs/2510.08431
  90. [90]Linqi Zhou, Stefano Ermon, and Jiaming Song. Inductive moment matching. arXiv preprint arXiv:2503.07565, 2025.arxiv.org/abs/2503.07565
  91. [91]Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In ICML, 2024.arxiv.org/abs/2404.04057

Paper details

vlnc:W_gdwaspmt
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents