betaFeedback

WorksComputer Science

FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning

Abstract

Reinforcement learning (RL) has become a standard post-training method for diffusion models, but most RL objectives roll out the current policy at every iteration, and with a many-step sampler these rollouts typically dominate the cost of training. Few-step models cut this cost, yet they model an average velocity over an interval, whereas existing RL objectives are built for an instantaneous velocity with a many-step sampler. We adapt RL objectives to few-step models systematically in three phases, namely a transformation, a decomposition and an adaptation. The transformation to xx-space expresses each objective we consider through a common policy prediction, at the clean end by default and at an explicitly marked level for transition or flow-map objectives, so that objectives stated over densities, velocities or rewards become comparable. The decomposition writes the gradient of each of these objectives as the sum of a self-regression toward a reference and a reward augmentation, so that a method reduces, at the gradient level, to a choice of anchors and weights. The adaptation computes each anchor from a few-step model at the end of its transport or along its tangent, with the instantaneous velocity recovered from the network itself or from an auxiliary network. This yields few-step versions of FlowGRPO, AWM, DiffusionNFT, DiffusionOPSD and ReFL in one modular codebase. In our experiments, reward-gradient-based methods are the most effective when the reward is differentiable and its gradient is affordable; otherwise DiffusionNFT-style methods generally reach high rewards in relatively few iterations despite a sometimes higher per-iteration cost. Extra inference steps show no systematic aggregate benefit in our experiments, although selected outputs visibly improve with two to four times as many sampling steps as in training.

Affiliation: 1^{1}Tencent Hunyuan, 2^{2} Westlake University, 3^{3} University of Chinese Academy of Sciences

*Equal contribution; †Project Leader; ‡Corresponding author: [email protected]

Project Page: fewsteprl.page

Code: fewsteprl-sd35 (images), fewsteprl-wan21 (videos)

Checkpoints: Hugging Face (coming soon)

Introduction

Diffusion models and their flow-matching counterparts generate data by integrating a learned transport from noise to samples, and have become a dominant approach to image and video synthesis [31, 15] (Song et al. 2021b; Lipman et al. 2023; Liu et al. 2023; Rombach et al. 2022). Because the learned field is accurate only locally, sampling typically takes tens of network evaluations (Song et al., 2021a).

Few-step models remove most of this cost by learning the transport over long intervals directly. Consistency models and MeanFlow, which models the average velocity over an interval, and on-policy distilled flow maps such as DMD and AnyFlow generate in one to four evaluations [23, 24, 26] (Song et al. 2023; Yin et al. 2024b).

Reinforcement learning (RL) has become a standard post-training method for diffusion models. Fine-tuning against reward models of human preference, text rendering or composition measurably improves the outputs [3, 19, 11] (Xu et al. 2023; Liu et al. 2025a; Xue et al. 2026; Zheng et al. 2026a). This gain, however, typically requires samples from the current policy at every iteration and must be protected against reward hacking, in which the policy exploits the reward model instead of improving.

With a many-step sampler, these rollouts typically account for most of the time and memory of training, and more so for long video sequences. A few-step model shrinks this cost, but it models an average velocity over an interval, whereas the RL objectives above are built for an instantaneous velocity or for the transitions of a many-step sampler. Applied directly, they leave open which network evaluation should be differentiated, how the reference policy should be evaluated, and which parts of the interval are trained. Prior work typically resolves these choices for one objective at a time (Huang et al., 2026b,a; Li et al., 2026b); what is, to our knowledge, missing is a systematic way to adapt RL objectives to few-step models. Figure 1 illustrates the effect of leaving them unresolved. Substituting the average velocity of a distilled few-step model for the instantaneous velocity required by FlowGRPO, AWM and DiffusionNFT, i.e., applying the original objective to the distilled model without modification, gives a lower best GenEval2 than the adapted objective derived in this paper in each of the three cases.

Figure 1. Held-out GenEval2 against the RL iteration in the mixed-reward setting of § 6.1, from the shared pre-RL few-step checkpoint. Dashed: the original objective applied to the distilled model with the average velocity substituted for the instantaneous velocity (FlowGRPO, AWM, DiffusionNFT). Solid: the corresponding few-step adaptation of this paper (FewStepGRPO, MeanFlowAWM, MeanFlowNFT; § 5). Each annotated gap is the adaptation minus the baseline, at the best checkpoint of the baseline and at iteration 400; the two coincide when the baseline is best at iteration 400.

This paper builds one in three phases (Figure 2), namely a transformation, a decomposition and an adaptation. The transformation maps each objective to xx-space. Under a linear noise schedule, a velocity defines an affine map between noise levels, so each objective we consider can be expressed through a common policy prediction, at the clean end by default and at an explicitly marked level for transition or flow-map objectives. In xx-space, objectives stated over densities, velocities or rewards become directly comparable. The decomposition splits the loss gradient. The gradient of every objective we consider is the sum of a self-regression term, which keeps the policy close to a reference, and a reward-augmentation term, which contains the reward, so that, at the gradient level, a method reduces to a choice of anchors and weights. The adaptation extends the anchors to few-step models. We show how each anchor is computed from a few-step model, either at the end of its transport or along the tangent of the trajectory, with the instantaneous velocity recovered from the network itself, as in MeanFlow, or from an auxiliary network, as in DMD.

Figure 2. Overview of the framework. (a) RL increases the relative mass of the high-reward mode under the rollout distribution poldp_{\mathrm{old}} of the old policy. The default objective is evaluated at the clean end, on the policy prediction (renoise-and-denoise sample) x~θ=X[x~t,vθ,0]=x~t−tvθ\tilde{x}_{\theta}=X[\tilde{x}_{t},v_{\theta},0]=\tilde{x}_{t}-tv_{\theta}, whose reward augmentation attracts it toward high-reward rollout samples and repels it from low-reward ones; in the figure, x^+\hat{x}^{+} and x^−\hat{x}^{-} denote a high-reward and a low-reward rollout sample x^(i)\hat{x}^{(i)}, respectively. (b) The RL-loss gradient can be decoupled into a self-regression term, visualised as a regression force toward the regression anchor x~reg\tilde{x}_{\mathrm{reg}}, and a reward-augmentation term, visualised as a reward force along the displacement x~rwd−x~base\tilde{x}_{\mathrm{rwd}}-\tilde{x}_{\mathrm{base}} from the baseline anchor x~base\tilde{x}_{\mathrm{base}}, which lies between the two modes, to the rewarded anchor x~rwd\tilde{x}_{\mathrm{rwd}}. A high-reward sample produces attraction along that displacement (blue); a low-reward sample produces repulsion against it (orange), so both updates favour higher reward (§ 4.2). (c) For a few-step model, the prediction and the anchors {x~rwd,x~base,x~reg}\{\tilde{x}_{\mathrm{rwd}},\tilde{x}_{\mathrm{base}},\tilde{x}_{\mathrm{reg}}\} are computed either at the end of the transport curve, x~t−tu\tilde{x}_{t}-tu (u-form), or along the tangent at x~t\tilde{x}_{t}, x~t−tv\tilde{x}_{t}-tv (v-form). The instantaneous velocity is recovered from the network itself for MeanFlow, v=u+(t−r)sg⁡(∂tu)v=u+(t-r)\operatorname{sg}(\partial_tu) with the detached correction evaluated on the old policy in the implemented methods (Equation (56)), and from an auxiliary network for DMD, v=vϕv=v_{\phi}, fitted so that vϕ(x~t,t)≈E[ϵ−x^∣x~t]v_{\phi}(\tilde{x}_{t},t)\approx\mathbb{E}[\epsilon-\hat{x}\mid\tilde{x}_{t}] (§ 4.3).

All methods share one codebase, which is represented by a single training step with several method-specific stages, namely rollout, reward and advantage, anchors and weights, and loss (plus auxiliary fitting and the refresh of the old policy), where each method overrides only the hooks in which it differs (§ 5). The methods are therefore compared under shared rollout, reward and evaluation code, and a new variant is specified mainly by its anchors and weights. Experimentally, we compare seven implementations on image generation with an AnyFlow-distilled SD3.5-Medium ([26] (Stability AI 2024)), under a non-differentiable OCR reward and a differentiable mixed reward, and on video generation with an AnyFlow-distilled Wan2.1-T2V-1.3B ([26] (Team Wan et al. 2025)); we further test whether the fine-tuned few-step models improve when sampled with more steps.

The contributions of this paper are as follows.

  • A general view. After the transformation to xx-space, the gradients of the RL objectives considered here share one decoupled form, the sum of a self-regression term and a reward-augmentation term, so that their gradients differ only in their anchors and weights.

  • A systematic adaptation to few-step models. Each of these anchors can be computed from an average-velocity model at the end of the transport or along its tangent, which yields few-step versions of FlowGRPO, AWM, DiffusionNFT, DiffusionOPSD and ReFL within one modular codebase.

  • Empirical guidance with experimental analysis. Where the reward is differentiable and its gradient affordable, reward-gradient-based methods are the most effective in our experiments, and ReFL alone is a strong baseline. Otherwise, group-advantage methods apply. Among them, DiffusionNFT-style methods generally reach high rewards in relatively few iterations, even when one iteration is expensive, and DMD-based variants are faster per iteration on image but slower on video, depending on the relative cost of forward and backward passes, and often coincide with larger held-out degradation. Additional inference steps bring little aggregate improvement in generation quality, while visual gains appear in selected outputs with more sampling steps than training.

Background

Diffusion and flow-based generators. Diffusion models learn to reverse a fixed noising process [31]; score-based models cast this reversal as a reverse-time SDE whose probability-flow ODE gives a deterministic sampler, of which DDIM is a discrete instance (Song et al., 2021b,a). Flow matching and rectified flow instead regress the velocity of a prescribed interpolation between noise and data, the latter straightening the paths so that fewer integration steps suffice (Lipman et al., 2023; Liu et al., 2023). These models were scaled by moving to a latent space (Rombach et al., 2022), by transformer backbones (Peebles and Xie, 2023; Ma et al., 2024) and by larger, more efficient architectures [15] (Podell et al. 2024; Xie et al. 2025a), with classifier-free guidance as a standard way to trade diversity for prompt fidelity [30]. Related generators include current text-to-image systems [2, 4, 5] (OpenAI 2022; Saharia et al. 2022; Team Seedream et al. 2025; Z-Image Team et al. 2025; Qin et al. 2025), multimodal models that both generate and understand images in one network [10, 9] (Ma et al. 2025a; Xie et al. 2025b; Wang et al. 2024b; OpenAI et al. 2024), trained on web-scale image–text data (Schuhmann et al., 2022; Zou, 2024), and video generators such as Wan and LongCat-Video (Team Wan et al., 2025; Meituan LongCat Team et al., 2025). These models typically learn the instantaneous velocity of the sampling flow, or an affine transformation of it such as the score or the predicted noise. Since this velocity is instantaneous, the sampler integrates it with many small steps, typically 32 or 64, which makes inference costly.

Few-step generation. Sampling cost has been cut along several lines. Progressive distillation halves the number of steps repeatedly by training a student to match two teacher steps with one (Salimans and Ho, 2022). Consistency models map every point of a trajectory to its endpoint, and were extended to latent space, arbitrary trajectory jumps, multiple sub-intervals and segmented trajectories (Song et al., 2023; Luo et al., 2023a; Kim et al., 2024; Wang et al., 2024a; Ren et al., 2024); consistency has also been used as a training regulariser against sampling drift and to learn from noisy data [12, 13]. Shortcut models condition on the step size and learn to take large steps directly [20]. Adversarial distillation adds a discriminator so that one to four steps produce sharp samples [6] (Wang et al. 2023a; Sauer et al. 2025; Sauer et al. 2024; Lin et al. 2024; Lu and Song 2025). Distribution matching distillation (DMD) trains the generator so that its output distribution matches the teacher’s, using the difference between a teacher score and the score of an auxiliary network fitted to the generator’s samples (Yin et al., 2024b,a), an idea shared with Diff-Instruct and variational score distillation (Luo et al., 2023b; Wang et al., 2023b); later work scales it to flow models, splits it into sub-intervals, decouples guidance from matching, moves it to continuous time, regularises it with consistency, and adapts it to video (Ge et al., 2026a; Fan et al., 2026b; Liu et al., 2026a,b; Zheng et al., 2026b; Ge et al., 2026b; Nie et al., 2026; Luo et al., 2025). Flow maps learn the transport between any two noise levels. MeanFlow parameterises the average velocity over an interval and trains it through its identity with the instantaneous velocity [23, 24] (Lin et al. 2026), and AnyFlow distils such a map on policy for any step count [26]. Unlike the many-step generators above, these flow-map models learn an average velocity over an interval, which can be read as the direction of a secant of the sampling trajectory, differing substantially from the instantaneous velocity where the trajectory is curved.

Reinforcement learning for diffusion models. RL fine-tuning of generators builds on policy optimisation (Levine, 2018; Schulman et al., 2017), preference optimisation (Rafailov et al., 2023) and group-relative advantages, which normalise rewards within a group of samples for the same prompt [27] (Shao et al. 2024). For diffusion models, early work fine-tunes on human feedback (Lee et al., 2023), treats denoising as a Markov decision process optimised by policy gradients [3, 19], or optimises pairwise preferences directly (Wallace et al., 2024). FlowGRPO brings group-relative policy optimisation to flow models by turning the ODE sampler into an SDE with tractable transition likelihoods (Liu et al., 2025a); follow-ups extend it to video, mix ODE and SDE steps, weight timesteps, and preserve the noise coefficients of the sampler [28] (Xue et al. 2025; Li et al. 2026a; Wang and Yu 2025). A second family avoids transition likelihoods by acting on the forward process: AWM weights the pretraining loss by the advantage (Xue et al., 2026), reward-weighted and policy-gradient flow matching reweight or rescale the velocity regression [16] (McAllister et al. 2026), adjoint matching is scaled to RL post-training [1], and DiffusionNFT contrasts positive and negative implicit policies built from the model and the frozen policy that generated its rollouts (Zheng et al., 2026a), following NFT in language models [8]. A third family backpropagates differentiable rewards through the sampler, as in DRaFT and ReFL [11] (Xu et al. 2023; Wang et al. 2026); the recently proposed DiffusionOPSD (Zhou et al., 2026) belongs to this family but constructs detached points with normalised reward-gradient steps and fits the policy to them, thereby forming a surrogate gradient. These methods are built for an instantaneous velocity, and most of them need on-policy rollouts at every iteration, which are expensive with the many-step samplers discussed above; how to apply RL to few-step models is therefore an important question.

RL for few-step generators. Closest to this paper are methods that fine-tune few-step models with rewards. Several combine DMD with RL, by adding a reward term to distillation, guiding distillation with reward gradients, or recasting distribution matching itself as a reward [7, 14, 17] (Jiang et al. 2026); others tune distilled models with non-differentiable rewards, Kullback–Leibler (KL) regularised alignment or pairwise sample optimisation (Luo et al., 2026; Luo, 2025; Miao et al., 2025). For flow maps, Flow-Map GRPO composes stochastic steps of a flow-map generator (Li et al., 2026b), MeanFlowNFT applies DiffusionNFT to an average-velocity model through its self-induced instantaneous field [34], and RTDMD optimises a reward-tilted distribution matching objective [33]. These works mostly adapt one objective to one kind of few-step model; we instead give a common construction that extends FlowGRPO, AWM, DiffusionNFT, DiffusionOPSD and ReFL to average-velocity models, and compare them under one implementation.

Preliminaries

Few-step Diffusion Models with On-Policy Distillation

We write t∈[0,1]t \in[0,1] for the noise level, with data at t=0t=0 and Gaussian noise at t=1t=1. A plain xtx_t denotes a point at level tt on a sampling trajectory, and x~t\tilde{x}_t denotes a renoised state at level tt obtained by adding fresh noise to a clean sample Equation (2); every map below applies to either. The setting we focus on throughout is distillation, where a pretrained instantaneous-velocity field vreal(x~t,t)v_{\mathrm{real}}(\tilde{x}_t,t) is available and frozen, which we call the teacher, and the student uθ(x~t,t,r)u_\theta(\tilde{x}_t,t,r) must reproduce its transport in one or a few network calls. The teacher is an ordinary flow-matching model, trained by regressing onto the conditional velocity vcond=ϵ−xv_{\mathrm{cond}}=\epsilon-x of a clean sample xx and Gaussian noise ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I), and it provides the student’s regression target in Equation (8) (Lipman et al., 2023; Liu et al., 2023). The student is the average-velocity generator [23, 24, 26],

uθ(xt,t,r)=1t−r∫rtvθ(xv,v) dv,(1)u_\theta(x_t,t,r) = \frac{1}{t-r}\int_r^t v_\theta(x_v,v)\,\mathrm{d}v, \tag*{(1)}

where vθv_\theta is the instantaneous velocity induced by the student, xνx_\nu is the state at level ν\nu on the probability-flow trajectory of vθv_\theta through xtx_t, and uθ(xt,t,t):=vθ(xt,t)u_\theta(x_t,t,t) := v_\theta(x_t,t) closes the definition at r=tr=t. Throughout, we identify this induced field with the flow-matching marginal velocity of the student’s sample distribution pθp_\theta, vθ(xt,t)≡E[ϵ−x^∣xt]v_\theta(x_t,t) \equiv\mathbb{E}[\epsilon-\hat{x}\mid x_t] with x^∼pθ\hat{x}\sim p_\theta (Appendix A); the two coincide when the flow map is exact.

Unlike off-policy methods, which query the model at x~t=AddNoise⁡(x0,t)\tilde{x}_t=\operatorname{AddNoise}(x_0,t) with x0∼pdatax_0\sim p_{\mathrm{data}}, on-policy training draws the noised state from the student’s own distribution:

x~t=AddNoise⁡(x^θ,t)=(1−t)x^θ+tϵ,x^θ∼pθ(x),ϵ∼N(0,I).(2)\tilde{x}_t=\operatorname{AddNoise}(\hat{x}_\theta,t)=(1-t)\hat{x}_\theta+t\epsilon,\qquad\hat{x}_\theta\sim p_\theta(x),\qquad\epsilon\sim\mathcal{N}(0,I). \tag*{(2)}

The subscript θ\theta in Equation (2) is kept only where the sampler is differentiated. The samples x^θ\hat{x}_\theta are produced by a rollout operator Rollout⁡\operatorname{Rollout}, an ODE or SDE solver that maps Gaussian noise at level 1 to a clean sample at level 0 for a text prompt cc in MstepM_{\mathrm{step}} steps, on a grid 1=t0>t1>⋯>tMstep=01=t_0>t_1>\cdots>t_{M_{\mathrm{step}}}=0:

x^θ=Rollout⁡(ϵ,uθ,c),x^θ,t0=ϵ∼N(0,I).(3)\hat{x}_\theta=\operatorname{Rollout}(\epsilon,u_\theta,c),\qquad\hat{x}_{\theta,t_0}=\epsilon\sim\mathcal{N}(0,I). \tag*{(3)}

For an instantaneous field, one Euler step of the solver between tjt_j and tj+1t_{j+1} is

x^θ,tj+1=x^θ,tj−(tj−tj+1)vθ(x^θ,tj,tj),(4)\hat{x}_{\theta,t_{j+1}}=\hat{x}_{\theta,t_j}-(t_j-t_{j+1})v_\theta(\hat{x}_{\theta,t_j},t_j), \tag*{(4)}

which is only first-order accurate, so MstepM_{\mathrm{step}} must be large. A displacement is the time integral of the velocity, and by Equation (1) (t−r)uθ(t-r)u_\theta equals this integral over [r,t][r,t]. With uθu_\theta as the generator, the Euler step is replaced by an exact jump,

x^θ,r=x^θ,t−(t−r)uθ(x^θ,t,t,r),(5)\hat{x}_{\theta,r}=\hat{x}_{\theta,t}-(t-r)u_\theta(\hat{x}_{\theta,t},t,r), \tag*{(5)}

so a few-step sampler is a chain of MstepM_{\mathrm{step}} such jumps, each a single subtraction rather than an integration. Taking the whole interval at once, Mstep=1M_{\mathrm{step}}=1, t=1t=1 and r=0r=0, recovers the one-call generator.

MeanFlow. According to the definition of uθu_\theta in Equation (1) (Geng et al. [23]), multiplying by (t−r)(t-r) and differentiating in tt eliminates the integral and leaves the MeanFlow identity

References

  1. [1]Andreas Bergmeister, Stefanie Jegelka, Nikolas Nüsken, Carles Domingo-Enrich, and Jakiw Pidstrigach. Reinforce adjoint matching: Scaling RL post-training of diffusion and flow-matching models, 2026. URL https://arxiv.org/abs/2605.10759.
  2. [2]James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra, Prafulla Dhariwal, Casey Chu, Yunxin Jiao, and Aditya Ramesh. Improving image generation with better captions. Technical report, OpenAI, 2023. URL https://cdn.openai.com/papers/dall-e-3.pdf.
  3. [3]Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, volume 2024, pages 4965–4987, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/hash/14f75513f0f1ca01de1e826b52e6b840-Abstract-Conference.html.arxiv.org/abs/2305.13301
  4. [4]Black Forest Labs. FLUX.1 model family. Software repository, 2024. URL https://github.com/black-forest-labs/flux.
  5. [5]Black Forest Labs. FLUX.2: Frontier visual intelligence, 2025. URL https://bfl.ai/blog/flux-2.
  6. [6]Clément Chadebec, Onur Tasar, Eyal Benaroche, and Benjamin Aubin. Flash diffusion: Accelerating any conditional diffusion model for few steps image generation. Proceedings of the AAAI Conference on Artificial Intelligence, 39(15):15686–15695, 2025. doi: 10.1609/aaai.v39i15.33722. URL https://ojs.aaai.org/index.php/AAAI/article/view/33722.DOI
  7. [7]Guanjie Chen, Shirui Huang, Yifu Sun, Kai Liu, Jianchen Zhu, Xiaoye Qu, Yu Cheng, and Peng Chen. Flash-DMD: Towards high-fidelity few-step image generation with efficient distillation and joint reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6010–6020, June 2026a. URL https://openaccess.thecvf.com/content/CVPR2026/html/Chen_Flash-DMD_Towards_High-Fidelity_Few-Step_Image_Generation_with_Efficient_Distillation_and_CVPR_2026_paper.html.arxiv.org/abs/2511.20549
  8. [8]Huayu Chen, Kaiwen Zheng, Qinsheng Zhang, Ganqu Cui, Yin Cui, Haotian Ye, Tsung-Yi Lin, Ming-Yu Liu, Jun Zhu, and Haoxiang Wang. NFT: Bridging supervised learning and reinforcement learning in math reasoning. In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust, editors, International Conference on Learning Representations, volume 2026, pages 124025–124042, 2026b. URL https://proceedings.iclr.cc/paper_files/paper/2026/hash/c9867d5a22653ce98b02595061e40f12-Abstract-Conference.html.arxiv.org/abs/2505.18116
  9. [9]Junying Chen, Zhenyang Cai, Pengcheng Chen, Shunian Chen, Ke Ji, Xidong Wang, Yunjin Yang, and Benyou Wang. ShareGPT-4o-Image: Aligning multimodal models with GPT-4o-level image generation, 2025a. URL https://arxiv.org/abs/2506.18095.
  10. [10]Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, 2025b. URL https://arxiv.org/abs/2501.17811.
  11. [11]Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly fine-tuning diffusion models on differentiable rewards. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, volume 2024, pages 4793–4822, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/hash/13e8be77982beb73d7ed0bb122f9f3c-Abstract-Conference.html.arxiv.org/abs/2309.17400
  12. [12]Giannis Daras, Yuval Dagan, Alex Dimakis, and Constantinos Daskalakis. Consistent diffusion models: Mitigating sampling drift by learning to be consistent. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 42038–42063. Curran Associates, Inc., 2023. doi: 10.52202/075280-1821. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/831406cfe7e4a0aed5ac5c8a8389d1f5-Abstract-Conference.html.DOI
  13. [13]Giannis Daras, Alex Dimakis, and Constantinos Costis Daskalakis. Consistent diffusion meets tweedie: Training exact ambient diffusion models with noisy data. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 10091–10108. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/daras24a.html.arxiv.org/abs/2404.10177
  14. [14]Linwei Dong, Ruoyu Guo, Ge Bai, Zehuan Yuan, Yawei Luo, and Changqing Zou. Guiding distribution matching distillation with gradient-based reinforcement learning, 2026. URL https://arxiv.org/abs/2604.19009.
  15. [15]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 12606–12633. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/esser24a.html.arxiv.org/abs/2403.03206
  16. [16]Jiajun Fan, Shuaike Shen, Chaoran Cheng, Yuxin Chen, Chumeng Liang, and Ge Liu. Online reward-weighted fine-tuning of flow matching with wasserstein regularization. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 60053–60113, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/hash/9690d4746230cfea3d067fca695ba648-Abstract-Conference.html.arxiv.org/abs/2502.06061
  17. [17]Linqian Fan, Peiqin Sun, Tiancheng Wen, Shun Lu, and Chengru Song. Rdm: Re-conceptualizing distribution matching as a reward for diffusion distillation, 2026a. URL https://arxiv.org/abs/2603.28460.
  18. [19]Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. DPOK: Reinforcement learning for fine-tuning text-to-image diffusion models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 79858–79885. Curran Associates, Inc., 2023. doi: 10.52202/075280-3497. URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/fc65fab891d83433bd3c8d966ed de311-Abstract-Conference.html.
  19. [20]Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 34668–34684, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/hash/559a0998fab1d19b80e7e43a5852401c-Abstract-Conference.html.arxiv.org/abs/2410.12557
  20. [23]Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. In Advances in Neural Information Processing Systems, volume 38, pages 75460–75482, 2025. doi: 10.52202/085713-2534. URL https://papers.neurips.cc/paper_files/paper/2025/hash/6d13e085b79d454da5910e4ca82a3d9d-Abstract-Conference.html.DOI
  21. [24]Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, and Kaiming He. Improved mean flows: On the challenges of fastforward generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 30467–30476, 2026. URL https://openaccess.thecvf.com/content/CVPR2026/html/Geng_Improved_Mean_Flows_On_the_Challenges_of_Fastforward_Generative_Models_CVPR_2026_paper.html.arxiv.org/abs/2512.02012
  22. [25]Dhruba Ghosh, Hannah Hajishirzi, and Ludwig Schmidt. GenEval: An object-focused framework for evaluating text-to-image alignment. In Advances in Neural Information Processing Systems, volume 36, pages 52132–52152, 2023. doi: 10.52202/075280-2270. URL https://papers.neurips.cc/paper_files/paper/2023/hash/a3bf71c7c63f0c3bcb7ff67c67b1e7b1-Abstract-Datasets_and_Benchmarks.html.DOI
  23. [26]Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, and Mike Zheng Shou. AnyFlow: Any-step video diffusion model with on-policy flow map distillation, 2026. URL https://arxiv.org/abs/2605.13724.
  24. [27]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chengxiang Zhao, Chengqi Deng, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Honghui Ding, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jingchang Chen, Jingyang Yuan, Jinhao Tu, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaichao You, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruiyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zhihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081):633–638, 2025. doi: 10.1038/s41586-025-09422-z. URL https://www.nature.com/articles/s41586-025-09422-z.DOI
  25. [28]Xiaoxuan He, Siming Fu, Yuke Zhao, Wanli Li, Jian Yang, Dacheng Yin, Fengyun Rao, and Bo Zhang. TEMPFLOW-GRPO: WHEN TIMING MATTERS FOR GRPO IN FLOW MODELS. In International Conference on Learning Representations, pages 132869–132895, 2026. URL https://proceedings.iclr.cc/paper_files/paper/2026/hash/d75f561eaaf2cb754bc8d7e36d8af362-Abstract-Conference.html.arxiv.org/abs/2508.04324
  26. [29]Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514–7528, 2021. doi: 10.18653/v1/2021.emnlp-main.595. URL https://aclanthology.org/2021.emnlp-main.595.DOI
  27. [30]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021. URL https://neurips.cc/virtual/2021/48591.arxiv.org/abs/2207.12598
  28. [31]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020. URL https://proceedings.neurips.cc/paper/2020/hash/4c5bcfec8584af0d967f1ab10179ca4b-Abstract.html.arxiv.org/abs/2006.11239
  29. [32]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://iclr.cc/virtual/2022/poster/6319.arxiv.org/abs/2106.09685
  30. [33]Yushi Huang, Xiangxin Zhou, Ruoyu Wang, Chi Zhang, Jun Zhang, and Tianyu Pang. Reinforcing few-step generators via reward-tilted distribution matching, 2026a. URL https://arxiv.org/abs/2605.26108.
  31. [34]Yushi Huang, Xiangxin Zhou, Jun Zhang, Liefeng Bo, and Tianyu Pang. MeanFlowNFT: Bringing forward-process RL to average-velocity generators, 2026b. URL https://arxiv.org/abs/2607.15273. arXiv:2607.15273v2.

Paper details

vlnc:W_8j33x5zd
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents