Distribution Matching Distillation (DMD) distills multi-step diffusion models into few-step generators by minimizing reverse KL divergence. Despite its strong performance, DMD guides samples by local density alone, which introduces two limitations: teacher scores can be unreliable in regions of low teacher density, and the mode-seeking behavior of reverse KL can collapse the student onto a subset of the teacher modes. We first connect DMD to Wasserstein gradient flow, showing that its gradient follows from a fixed-point formulation with integral KL (IKL) as the energy functional. This perspective motivates augmenting IKL with an optimal transport term that addresses these limitations by guiding outlier samples toward the target and promoting mode coverage. We realize this term using Sinkhorn divergence estimated from finite batches. In practice, however, errors in the estimated Sinkhorn velocity can perturb in-domain samples for which IKL guidance is already accurate. To this end, we introduce sDMD, which adaptively activates the Sinkhorn correction based on discriminator outputs, focusing on outlier samples where IKL may be unreliable. With 2–4 sampling steps on FLUX.1-dev, Qwen-Image-20B, and Wan2.1-14B, sDMD outperforms distillation baselines on GenEval, DPG, WISE, and VBench while improving sample diversity over DMD2.
Introduction
Diffusion models ([67, 69, 33]) generate high-quality images and videos but require many sampling steps. Diffusion distillation reduces this cost by transferring knowledge from a pretrained teacher to a student that generates samples in one or a few steps. Existing approaches have advanced few-step generation by learning mappings along teacher trajectories ([23, 70]), using adversarial supervision ([65, 45]), or directly matching student and teacher distributions ([81, 91]). Among these, a representative approach is DMD ([84]), which optimizes a reverse KL objective using the difference between teacher and student scores. It has been widely adopted due to its high sample fidelity and scalability in large-scale image and video generation ([14, 89, 20]).
To better understand and improve the training dynamics of DMD, we view diffusion distillation from the perspective of Wasserstein gradient flow (WGF) ([1]). WGF evolves a probability distribution by steepest descent of an energy functional in Wasserstein space, with the corresponding velocity field specifying the update direction for each sample. While recent works ([29, 31]) use WGF to train one-step generators from scratch, we establish its connection to diffusion distillation by recovering the DMD gradient through fixed-point regression under the integral KL (IKL) energy, defined as reverse KL averaged over diffusion noise levels (§ 4.1). This identifies the energy underlying DMD and the velocity field that guides generated
samples, allowing us to analyze the behavior and limitations of DMD and explore complementary energy functionals to improve distillation.
The IKL velocity of DMD reveals two practical limitations (Figure 2). First, learned teacher scores can be unreliable for outlier samples, and the resulting updates may fail to guide them toward target modes. Although adding diffusion noise can improve distributional overlap, outliers may remain poorly supported at low noise levels. Higher noise levels can improve their support but blur the local structure of target modes. Second, the mode-seeking behavior of reverse KL can concentrate the student on a subset of target modes, potentially reducing the diversity of generated samples.
Figure 2. IKL and Sinkhorn Velocity Fields. We visualize velocity fields from a 2D Gaussian mixture experiment. Black circles mark student particles and contours show target density. Solid (dashed) arrows use exact (learned) teacher scores for IKL and large (small) batches for Sinkhorn. (a) For in-domain student samples, the IKL velocity carries the particles onto the target. (b) Outlier samples under poor initialization, the IKL velocity can be inaccurate in regions of low teacher density. (c) The mode-seeking behavior of IKL can concentrate particles on a subset of the target modes. (d) The Sinkhorn velocity guides particles toward the target modes through the transport plan, remaining robust even for outlier samples (blue box). In practice, finite-sample errors in the Sinkhorn velocity can perturb samples for which IKL already provides reliable guidance (red boxes). We address this using our gated scheduler in Sec. 5.2. (a) IKL: aligned samples. (b) IKL: outlier samples. (c) IKL: mode seeking. (d) Sinkhorn transport.
To address these limitations, we augment IKL with an optimal transport (OT) term. OT couples student and target samples globally, providing robust guidance for outlier samples and promoting mode coverage without relying on learned teacher scores. To exploit their complementary strengths, we propose combining global OT guidance with local IKL score updates that help capture fine details. Our analysis shows that sufficiently strong exact OT regularization ensures convergence of the combined flow to the target under regularity assumptions (§ 4.2). For efficient computation, we use Sinkhorn divergence, a debiased form of entropic OT. While Sinkhorn does not automatically inherit this guarantee, we establish sufficient conditions on its approximation error and regularization strength for the combined flow to converge to the target (Sec. 5).
In practice, the transport maps used to compute Sinkhorn updates are estimated from finite samples, with larger batches generally reducing estimation error at greater computational cost [22]. Notably, our experiments show that even with relatively few samples, Sinkhorn can provide more informative guidance than IKL for outlier samples where learned teacher scores are unreliable. However, for well-supported samples, estimation errors in Sinkhorn can instead interfere with the accurate guidance already provided by IKL. To this end, we introduce sDMD, which adaptively activates Sinkhorn guidance for samples with relatively weak teacher support based on discriminator outputs. We reuse the discriminator trained for adversarial regularization in DMD2 [83], without training a separate model.
We evaluate sDMD using 2–4 sampling steps on FLUX.1-dev [42] and Qwen-Image-20B [78] for text-to-image generation (Figure 1) and Wan2.1 [74] (1.3B and 14B) for text-to-video generation. sDMD outperforms the distillation baselines in most metrics on GenEval [25], DPG [35], and WISE [57], achieves the highest overall VBench scores on both Wan2.1 models, and improves all reported diversity metrics over the strong DMD2 [83] baseline.
Figure 1. Teaser. 4-step generated images using sDMD with FLUX.1 and Qwen-Image.
Related Work
Diffusion Distillation. To reduce the sampling cost of multi-step diffusion models, trajectory-based distillation methods enable one or few-step generation by learning shortcuts along denoising trajectories [70, 5, 18]. Continuous-time formulations use JVPs [48, 23], whose efficient computation at scale requires specialized implementations [89]. Recent works aim to avoid JVP computation through specialized architectures that decouple sampling substeps from backbone evaluations [7, 66]. Adversarial approaches use discriminator-based supervision for distillation [64, 65] with potential training instability [45].
Distribution-matching distillation methods align generated and target distributions through f-divergence minimization [81, 80] or score matching [91, 51]. Among these, DMD [84, 50] minimizes the reverse KL divergence using the difference between teacher and student scores. With their high sample fidelity and scalability, DMD-based methods have been widely adopted for high-resolution image synthesis and video generation [85, 36, 89, 83]. However, DMD remains sensitive to outliers and the mode-seeking behavior of reverse KL [4, 52]. Our work augments DMD with a complementary velocity field to address these limitations and improve sample quality and diversity.
Wasserstein Gradient Flow for Generative Modeling. WGF [39, 1] evolves a probability distribution by steepest descent of an energy functional under the 2-Wasserstein metric. When a divergence to the target distribution is chosen as the energy, flow reduces the discrepancy between the two distributions, providing a framework for generative modeling. For example, MMD flows [2] depend on kernel gradients and can provide weak guidance when generated and target samples are far apart relative to the kernel bandwidth. This requires careful kernel design, such as noise-adaptive kernels [19] or kernels built from multiple feature encoders and scales [16, 12].
Sinkhorn divergence provides transport-based guidance [21, 17], and recent works [29, 31] use its gradient flow to train one-step generators. However, estimating transport from finite batches can introduce error into the resulting velocity field [60, 22]. Flows based on reverse KL between clean distributions use local score information to guide samples toward the target distribution [44, 47, 30]. However, these approaches require known target densities or score estimates obtained through methods such as kernel density estimation, which can limit scalability in high-dimensional generation [30, 29]. In contrast, DMD offers a scalable diffusion distillation approach by applying reverse KL to noise-perturbed distributions with noise annealing. In this work, we connect DMD to WGF and propose sDMD, which incorporates an optimal transport term to improve guidance for outlier samples and promote mode coverage.
Preliminaries
Problem Setup and Notation. We work on Rd and denote by P2(Rd) the set of probability distributions. Let p∈P2(Rd) denote the target distribution and let pref=N(0,Id) be the reference distribution. We assume access to a pretrained multi-step teacher for p that gives scores at different noise levels. Our goal is to distill this teacher into a student generator fθ:Rd→Rd that generates a sample in one or a few steps, such that its output distribution q=(fθ)#pref approximates p.
Distribution Matching Distillation (DMD). DMD [84] distills a multi-step diffusion teacher into a few-step generator by matching the student distribution q to the teacher distribution p. The original formulation starts from reverse KL between clean distributions, whose gradient involves the difference between the student and target scores. To approximate the gradient, it introduces diffusion annealing, replacing the clean-distribution scores with scores of noisy distributions. For a few-step generated sample x=fθ(z) with z∼pref, the forward diffusion process gives xt=αtx+σtϵ, where ϵ∼N(0,Id) and (αt,σt) defines the noise schedule. We denote the corresponding perturbed marginals by qt and pt, then the distribution-matching gradient takes the form:
In practice, the pretrained teacher provides target scores, while an auxiliary diffusion model learns the student score online from generated samples using a score-matching objective [73].
Wasserstein Gradient Flow (WGF). Let F:P2(Rd)→R be an energy functional on distributions that satisfies argminqF(q)=p. WGF [1, 39] describes the evolution of the probability density {qτ}τ≥0, where τ denotes gradient-flow time. The evolution is characterized by the following partial differential equation [72]:
∂τqτ+∇⋅(qτVτ)=0,Vτ(x)=−∇δqδF(qτ)(x),(2)
where the velocity field Vτ is the negative gradient of the first variation of F. Discretizing this flow in time, each iteration displaces every particle xτ∼qτ by an Euler step, i.e., xτ+1=xτ+Vτ(xτ). At the equilibrium qτ=p, the velocity vanishes and each step leaves every particle in place. Writing the particles as generator output x=fθ(z), the fixed-point relation gives the following regression objective on generator parameters [12]:
where sg(⋅) denotes the stop-gradient, and the loss decreases as Vτ→0. Note that different energy functionals induce different velocity fields and shape the training dynamics. Therefore, we connect DMD to WGF to explore the design space of energy functionals for distillation and introduce a complementary transport term to address the limitations of DMD.
Entropic Optimal Transport and Sinkhorn Divergence. Entropic optimal transport (EOT) compares q and p via a regularized transport cost which can be computed using the Sinkhorn algorithm [10, 41]. For the quadratic cost c(x,y)=21∥x−y∥2 and regularization strength β>0, EOT is defined as:
where Π(q,p) denotes the couplings with marginals q and p. The debiased Sinkhorn divergence removes entropic bias by subtracting the self-transport costs [21]:
Sβ(q,p)=OTβ(q,p)−21OTβ(q,q)−21OTβ(p,p).(5)
Sβ is nonnegative and vanishes if and only if q=p[17]. Let πq,p denote the optimal entropic transport plan in Eq. (4), and define its barycentric projection as Tq,p(x)=∫ydπq,p(y∣x), the weighted average of target points under the transport plan; Tq,q(x) denotes the corresponding projection under the self-transport plan. Then, the WGF velocity can be written as the difference between the two barycentric projections [21]:
VSβ(x)=−∇δqδSβ(q,p)(x)=Tq,p(x)−Tq,q(x).(6)
Analyzing DMD through Wasserstein Gradient Flow
Connection of DMD to WGF
DMD Yin et al., [84] has shown strong empirical performance in few-step generation, but the original derivation of the gradient introduces diffusion noise without a formal connection to WGF. We show that the DMD gradient in Eq. (1) follows from the WGF fixed-point relation in Eq. (3). Specifically, the DMD update can be viewed through the WGF framework by choosing the integral KL (IKL) divergence [50], i.e., reverse KL averaged over diffusion noise levels, as the energy functional:
where (i) follows by substituting FIKL into the WGF velocity in Eq. (2). Plugging this velocity into the fixed-point relation in Eq. (3) and taking the gradient gives:
which recovers the DMD gradient in Eq. (1) up to a constant after substituting VIKL from Eq. (7). We present the detailed derivation in Appendix § 11.§ 1. This connection allows us to analyze the behavior and limitations of DMD through its velocity field and explore synergistic energies to improve distillation.
Connection to Prior and Concurrent Work. Diff-Instruct [50] derives the generator score-difference update directly from IKL, but does not explicitly connect it to WGF. ProlificDreamer [76] derives a WGF update for text-to-3D particles, but does not consider distillation into few-step image or video generators. Concurrent analyses [27, 71] study WGF fixed-point regression for drifting models [12] using kernel-smoothed KL rather than IKL, and also do not explicitly connect DMD to WGF. In this work, we connect DMD to WGF under diffusion-noise-averaged KL from fixed-point regression and use this interpretation to introduce complementary Sinkhorn guidance for diffusion distillation.
Limitations of IKL
DMD [84] and its follow-up methods [15, 20] have demonstrated empirical success in capturing fine details and scaling to high-resolution image generation [14, 83] and video generation [28, 89, 36]. However, the gradient of IKL in DMD guides particles by local density alone, which brings two practical limitations. First, the clean reverse KL gradient flow is ill-defined when p and q are mutually singular [26]. Although forward diffusion gives pt and qt full support, in practice the learned teacher score may remain unreliable for generated samples with low teacher density, particularly at low noise levels. Second, the mode-seeking behavior of reverse KL can cause the student to cover only a subset of the teacher modes.
References
[1]Luigi Ambrosio, Nicola Gigli, and Giuseppe Savaré. Gradient Flows in Metric Spaces and in the Space of Probability Measures. Birkhäuser, 2nd edition, 2008.
[2]Michael Arbel, Anna Korba, Adil Salim, and Arthur Gretton. Maximum mean discrepancy gradient flow. In NeurIPS, 2019.arxiv.org/abs/1906.04370
[3]Mahmoud Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Mojtaba Komeili, Matthew Muckley, Ammar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, Sergio Arnaud, Abha Gejji, Ada Martin, Francois Robert Hogan, Daniel Dugas, Piotr Bojanowski, Vasil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil Krishnakumar, Yong Li, Xiaodong Ma, Sarath Chandar, Franziska Meier, Yann LeCun, Michael Rabbat, and Nicolas Ballas. V-JEPA 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025.
[5]Nicholas M. Boffi, Michael S. Albergo, and Eric Vanden-Eijnden. How to build a consistency model: Learning flow maps via self-distillation. arXiv preprint arXiv:2505.18825, 2025.arxiv.org/abs/2505.18825
[6]Clément Bonet, Nicolas Courty, François Septier, and Lucas Drumetz. Efficient gradient flows in sliced-wasserstein space. arXiv preprint arXiv:2110.10972, 2021.arxiv.org/abs/2110.10972
[7]Hansheng Chen, Kai Zhang, Hao Tan, Leonidas Guibas, Gordon Wetzstein, and Sai Bi. pi-flow: Policy-based few-step generation via imitation distillation. In ICLR, 2026.arxiv.org/abs/2510.14974
[8]Zhenglin Cheng, Peng Sun, Jianguo Li, and Tao Lin. Twinflow: Realizing one-step generation on large models with self-adversarial flows. In ICLR, 2026.arxiv.org/abs/2512.05150
[9]Jaemoo Choi, Jaewoong Choi, and Myungjoo Kang. Scalable wasserstein gradient flow for generative modeling through unbalanced optimal transport. arXiv preprint arXiv:2402.05443, 2024.arxiv.org/abs/2402.05443
[10]Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In NeurIPS, 2013.
[11]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In CVPR, 2009.
[12]Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. Generative modeling via drifting. arXiv preprint arXiv:2602.04770, 2026.arxiv.org/abs/2602.04770
[14]Xiangyu Fan, Zesong Qiu, Fanzhou Wang, Peng Gao, and Lei Yang. Qwen-image-lightning, 2025. URL https://github.com/ModelTC/LightX2V-Qwen-Image-Lightning.
[15]Xiangyu Fan, Zesong Qiu, Zhuguanyu Wu, Fanzhou Wang, Zhiqian Lin, Tianxiang Ren, Dahua Lin, Ruihao Gong, and Lei Yang. Phased dmd: Few-step distribution matching distillation via score matching within subintervals. In CVPR, 2026.arxiv.org/abs/2510.27684
[16]Lan Feng, Wuyang Li, Eloi Zablocki, Matthieu Cord, and Alexandre Alahi. Representation distribution matching for one-step visual generation. arXiv preprint arXiv:2607.02375, 2026.arxiv.org/abs/2607.02375
[17]Jean Feydy, Thibault Séjourné, François-Xavier Vialard, Shun-ichi Amari, Alain Trouvé, and Gabriel Peyré. Interpolating between optimal transport and mmd using sinkhorn divergences. In AISTATS, 2019.arxiv.org/abs/1810.08278
[18]Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In ICML, 2025.arxiv.org/abs/2410.12557
[19]Alexandre Galashov, Valentin De Bortoli, and Arthur Gretton. Deep MMD gradient flow without adversarial training. In ICLR, 2025.arxiv.org/abs/2405.06780
[20]Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation. arXiv preprint arXiv:2506.00523, 2025.arxiv.org/abs/2506.00523
[21]Aude Genevay, Gabriel Peyré, and Marco Cuturi. Learning generative models with sinkhorn divergences. In AISTATS, 2018.arxiv.org/abs/1706.00292
[22]Aude Genevay, Lénaïc Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complexity of sinkhorn divergences. In Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics, volume 89, pp. 1574–1583. PMLR, 2019. URL https://proceedings.mlr.press/v89/genevay19a.html.arxiv.org/abs/1810.02733
[23]Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In NeurIPS, 2025.arxiv.org/abs/2505.13447
[24]Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models. In CVPR, 2026.arxiv.org/abs/2512.02012
[25]Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. In NeurIPS, 2023.arxiv.org/abs/2310.11513
[26]Pierre Glaser, Michael Arbel, and Arthur Gretton. KALE flow: A relaxed KL gradient flow for probabilities with disjoint support. In NeurIPS, 2021.arxiv.org/abs/2106.08929
[27]Arthur Gretton, Li Kevin Wenliang, Alexandre Galashov, James Thornton, Valentin De Bortoli, and Arnaud Doucet. On the wasserstein gradient flow interpretation of drifting models. arXiv preprint arXiv:2605.05118, 2026.arxiv.org/abs/2605.05118
[28]Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, and Mike Zheng Shou. Anyflow: Any-step video diffusion model with on-policy flow map distillation. arXiv preprint arXiv:2605.13724, 2026.arxiv.org/abs/2605.13724
[29]Jiaqi Han, Puheng Li, Qiushan Guo, Renyuan Xu, Stefano Ermon, and Emmanuel J. Candès. One-step generative modeling via wasserstein gradient flows. arXiv preprint arXiv:2605.11755, 2026.arxiv.org/abs/2605.11755
[30]Jiajun He, Wenlin Chen, Mingtian Zhang, David Barber, and José Miguel Hernández-Lobato. Training neural samplers with reverse diffusive kl divergence. In AISTATS, 2025.arxiv.org/abs/2410.12456
[31]Ping He, Om Khangaonkar, Hamed Pirsiavash, Yikun Bai, and Soheil Kolouri. Sinkhorn-drifting generative models. arXiv preprint arXiv:2603.12366, 2026.arxiv.org/abs/2603.12366
[32]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local Nash equilibrium. NeurIPS, 2017.
[33]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020.arxiv.org/abs/2006.11239
[34]Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.arxiv.org/abs/2106.09685
[35]Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with llm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135, 2024.arxiv.org/abs/2403.05135
[36]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In NeurIPS, 2025.arxiv.org/abs/2506.08009
[37]Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, 2024.arxiv.org/abs/2311.17982
[38]Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In CVPR, 2024.arxiv.org/abs/2401.09603
[39]Richard Jordan, David Kinderlehrer, and Felix Otto. The variational formulation of the fokker–planck equation. SIAM Journal on Mathematical Analysis, 1998.
[40]Sunwoo Kim, Minky u Kim, and Dongmin Park. Test-time alignment of diffusion models without reward over-optimization. In ICLR, 2025.arxiv.org/abs/2501.05803
[41]Philip A. Knight. The sinkhorn–knopp algorithm: convergence and applications. SIAM Journal on Matrix Analysis and Applications, 2008.DOI
[43]Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2.5: Three insights towards enhancing aesthetic quality in text-to-image generation, 2024.arxiv.org/abs/2402.17245
[44]Lingxiao Li, Samuel Hurault, and Justin Solomon. Self-consistent velocity matching of probability flows. In NeurIPS, 2023.arxiv.org/abs/2301.13737
[45]Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang, Xuefeng Xiao, and Lu Jiang. Diffusion adversarial post-training for one-step video generation. In ICML, 2025.arxiv.org/abs/2501.08316
[46]Dongyang Liu, Gao Peng, David Liu, DU DU, Zhen Li, Qilong Wu, Xin Jin, Sihan Cao, Shifeng Zhang, Steven Hoi, et al. Decoupled dmd: Cfg augmentation as the spear, distribution matching as the shield. In ICLR, 2026.arxiv.org/abs/2511.22677
[47]Qiang Liu. Stein variational gradient descent as gradient flow. In NeurIPS, 2017.arxiv.org/abs/1704.07520
[48]Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. In ICLR, 2025.arxiv.org/abs/2410.11081
[50]Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In NeurIPS, 2023.arxiv.org/abs/2305.18455
[51]Weijian Luo, Zemin Huang, Zhengyang Geng, J Zico Kolter, and Guo-jun Qi. One-step diffusion distillation through score implicit matching. In NeurIPS, 2024.arxiv.org/abs/2410.16794
[52]Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang. Learning few-step diffusion models by trajectory distribution matching. In ICCV, 2025.arxiv.org/abs/2503.06674
[53]Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In ECCV, 2024.arxiv.org/abs/2401.08740
[54]Petr Mokrov, Alexander Korotin, Lingxiao Li, Aude Genevay, Justin M Solomon, and Evgeny Burnaev. Large-scale wasserstein gradient flows. In NeurIPS, 2021.arxiv.org/abs/2106.00736
[55]Weili Nie, Julius Berner, Chao Liu, and Arash Vahdat. Nvidia fastgen: Fast generation from diffusion models, 2026a. URL https://github.com/NVlabs/FastGen.
[56]Weili Nie, Julius Berner, Nanye Ma, Chao Liu, Saining Xie, and Arash Vahdat. Transition matching distillation for fast video generation. arXiv preprint arXiv:2601.09881, 2026b.arxiv.org/abs/2601.09881
[57]Yuwei Niu, Munan Ning, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Fanqing Meng, Kunpeng Ning, et al. Wise: A world knowledge-informed semantic evaluation for text-to-image generation. arXiv preprint arXiv:2503.07265, 2025.arxiv.org/abs/2503.07265
[58]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vỏ, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francis Massa, Alaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.arxiv.org/abs/2304.07193
[59]Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2024.arxiv.org/abs/2410.13720
[60]Aram-Alexandre Pooladian and Jonathan Niles-Weed. Entropic estimation of optimal transport maps. arXiv preprint arXiv:2109.12004, 2021.arxiv.org/abs/2109.12004
[61]Aram-Alexandre Pooladian, Marco Cuturi, and Jonathan Niles-Weed. Debiaser beware: Pitfalls of centering regularized transport maps. In ICML, volume 162, pp. 17830–17847. PMLR, 2022. URL https://proceedings.mlr.press/v162/pooladian22a.html.arxiv.org/abs/2202.08919
[62]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.arxiv.org/abs/2103.00020
[63]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
[64]Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042, 2023.arxiv.org/abs/2311.17042
[65]Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. arXiv preprint arXiv:2403.12015, 2024.arxiv.org/abs/2403.12015
[66]Neta Shaul, Chao Liu, Arash Vahdat, and Julius Berner. Parallel decoding distillation for fast image and video generation. arXiv preprint arXiv:2607.26004, 2026.arxiv.org/abs/2607.26004
[67]Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015.
[68]Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In ICML, 2024.arxiv.org/abs/2310.14189
[69]Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021.
[70]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023.arxiv.org/abs/2303.01469
[71]Erkan Turan, Nicolas Dufour, and Maks Ovsianikov. Generative drifting is secretly score matching: a spectral and variational perspective. arXiv preprint arXiv:2603.09936, 2026. URL https://arxiv.org/abs/2603.09936.
[72]Cédric Villani et al. Optimal transport: old and new. Springer, 2009.
[73]Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 2011.DOI
[75]Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yinan He, Yi Wang, Yali Wang, and Yu Qiao. VideoMAE V2: Scaling video masked autoencoders with dual masking. In CVPR, 2023a.arxiv.org/abs/2303.16727
[76]Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. NeurIPS, 2023b.arxiv.org/abs/2305.16213
[77]Zidong Wang, Yiyuan Zhang, Xiaoyu Yue, Xiangyu Yue, Yangguang Li, Wanli Ouyang, and Lei Bai. Transition models: Rethinking the generative learning objective. In CVPR, 2026.arxiv.org/abs/2509.04394
[78]Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Youxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-image technical report, 2025.arxiv.org/abs/2508.02324
[79]Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023.arxiv.org/abs/2306.09341
[80]Sirui Xie, Zhisheng Xiao, Diederik P Kingma, Tingbo Hou, Ying N Wu, Kevin Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em distillation for one-step diffusion models. In NeurIPS, volume 37, pp. 45073–45104, 2024.arxiv.org/abs/2405.16852
[81]Yilun Xu, Weili Nie, and Arash Vahdat. One-step diffusion models with f-divergence distribution matching. arXiv preprint arXiv:2502.15681, 2025.arxiv.org/abs/2502.15681
[82]Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbiang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time interactive long video generation. arXiv preprint arXiv:2509.22622, 2025.arxiv.org/abs/2509.22622
[83]Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Frédo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis. In NeurIPS, 2024a.arxiv.org/abs/2405.14867
[84]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In CVPR, 2024b.arxiv.org/abs/2311.18828
[85]Tianwei Yin, Qiang Zhang, Richard Zhang, William T. Freeman, Frédo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. In CVPR, 2025.arxiv.org/abs/2412.07772
[86]Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940, 2024.arxiv.org/abs/2410.06940
[87]Huijie Zhang, Aliaksandr Siarohin, Willi Menapace, Michael Vasilkovsky, Sergey Tulyakov, Qing Qu, and Ivan Skorokhodov. Alphaflow: Understanding and improving meanflow models. In ICLR, 2026.arxiv.org/abs/2510.20771
[88]Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie. Diffusion transformers with representation autoencoders. In ICLR, 2026a.arxiv.org/abs/2510.11690
[89]Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency. In ICLR, 2026b.arxiv.org/abs/2510.08431
[90]Linqi Zhou, Stefano Ermon, and Jiaming Song. Inductive moment matching. arXiv preprint arXiv:2503.07565, 2025.arxiv.org/abs/2503.07565
[91]Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In ICML, 2024.arxiv.org/abs/2404.04057