WorksComputer Science
VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation
Abstract
Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student's changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student's sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference.
Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student’s changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student’s sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference. Code is available at https://github.com/Luo-Yihong/VA-TDM.
Introduction
Few-step distillation seeks to make diffusion and flow models efficient without sacrificing generation quality [3, 6]. A central challenge is to provide effective supervision when a small number of student transitions must replace a long teacher trajectory. Trajectory distillation learns from individual teacher trajectories [37], whereas distribution matching allows the student to learn its own mapping by aligning student and teacher distributions [20, 44]. Trajectory Distribution Matching (TDM) [22] extends this principle to intermediate transitions. Each step then learns a partial denoising task, distributing supervision across the student trajectory and providing a natural foundation for few-step distillation.
For intermediate supervision to be effective, however, the training signal must faithfully reflect the distribution produced at each student step. In score-based methods, generator updates depend on the difference between teacher and student scores, requiring an auxiliary model to track the evolving student distribution. Intermediate transitions complicate this estimation: the student’s clean prediction and retained residual can be correlated, so perturbing its intermediate output generally differs from adding fresh Gaussian noise to the clean prediction. Our analysis of TDM shows why this distinction matters. Direct score estimation for the perturbed intermediate output can involve high-variance targets, while clean-target denoising can estimate a different distribution from the one used for generator supervision.
These observations motivate supervision that explicitly accounts for the student’s sampling process while allowing alternatives to its noisy-marginal score. Velocity targets offer this flexibility because they can be defined through paths with coupled endpoints [17], including dependencies inherited from a student transition. We build on this flexibility to introduce Velocity Adversarial Distillation (VAD), a framework for constructing student velocity fields and using their discrepancies from the teacher to guide distillation. Two prototypical velocity constructions are derived within VAD. A-form is a rectified velocity that captures the expected clean prediction conditioned on a perturbed sample; B-form is a score-equivalent Gaussian velocity for the student’s perturbed distribution. The two forms thus express conditional-moment and distributional supervision within the same sampling process. A vector-valued critic estimates the discrepancy between the chosen student field and the teacher field, and the generator learns from this discrepancy.
The resulting framework provides a unified perspective along two axes: which field to match and how to use its discrepancy to update the generator. With different generator updates, A-form connects to conditional moment matching, while B-form connects reverse-KL score distillation and Fisher-based field matching. These connections also clarify the relationship between an intended objective and its practical update. In particular, VAD explains the heuristic update of MMD [30] and derives an alternative surrogate that recovers the exact gradient of MMD’s desired objective.
To use both types of supervision along a few-step trajectory, we develop VA-TDM, retaining TDM’s supervision of intermediate transitions. VA-TDM combines A-form and B-form through a shared auxiliary backbone and blends complementary generator updates. Because samples from different student steps can follow different distributions even at the same noise level, we also condition field estimation on the transition being supervised. This provides different and accurate supervision signals among different steps.
Overall, our unified VAD framework connects existing distillation updates and derives new objectives, which VA-TDM combines with complementary velocity fields and transition-aware estimation for four-step generation, achieving state-of-the-art performance across backbones and tasks. On SD3.5-Medium, VA-TDM outperforms the compared few-step baselines across prompt following, image quality, and preference metrics, improving GenEval from TDM’s 0.61 to 0.67 while increasing sample diversity. On FLUX.2-Klein-Base 4B, it surpasses the released four-step model, raising GenEval from 0.78 to 0.80 and OCR similarity from 0.64 to 0.69, with diversity measured by LPIPS improving from 0.47 to 0.56. Most notably, VA-TDM scales to MiniMax-H3 (MiniMax, [25]), a state-of-the-art open-source audio-video model with 33B parameters. In our user study (Figure 2), VA-TDM wins 61.33% of comparisons against Fast-H3 at the same 4-NFE budget and achieves a 45.53% preference rate against the 100-NFE teacher, compared with 24.67% for Fast-H3. These results demonstrate that VA-TDM substantially narrows the perceptual quality gap to the teacher while requiring only 4-NFE.
Figure 2. Human preference on video synthesis.
Preliminaries
Diffusion and Flow Models. Diffusion models ([6, 34]) use the Gaussian forward process , where , independently, and denotes the perturbed marginal. Time runs from clean data to the noise prior; conditioning on is omitted. For , denoising score matching (DSM) learns the marginal score by regressing onto the conditional target:
Rectified flow (Liu et al., [17]) uses the linear schedule , and velocity . For Gaussian forward processes, it is equivalent to diffusion models.
Trajectory Distribution Matching (TDM). TDM (Luo et al., [22]) matches intermediate student and teacher distributions along a -step student trajectory . For a transition , perturbing with , gives student marginal . Learning is performed by reverse-KL divergence (Luo et al., [20]) along the student trajectory:
Teacher scores are supplied by the pretrained model; student scores require online estimation.
Method
Problem Setup. We consider distilling a pretrained conditional diffusion or flow model into an efficient few-step generator. Let and denote the teacher’s marginal distribution and velocity field at time , respectively, where corresponds to clean samples and to the noise prior. Given a condition and a desired number of sampling steps , our goal is to learn a student that transforms noise into a sample through transitions while matching the teacher’s marginal distributions along the generation trajectory.
Overview. We introduce Velocity Adversarial Distillation (VAD) as a unified framework for distilling diffusion models through aligning student and teacher velocity fields by an adversarial game. § 3.1 motivates this formulation by examining the challenges of student score estimation and introduces two constructions of student velocity. § 3.2 develops the adversarial velocity game, characterizes its matching equilibrium, and establishes connections to existing distillation methods. § 3.3 presents practical generator and critic updates and applies the framework along the student trajectory, with transition-aware field estimation.
From Score Distillation to Velocity Distillation
We focus on learning a one-step marginal transition from to a less noisy , where , whose output distribution should match the teacher marginal . Matching an intermediate distribution requires only partial denoising at each step, easing the learning task compared with requiring a clean output at every step, and is more efficient and effective (Luo et al., [22]).
Under the rectified-flow schedule, the student predicts a clean anchor and its residual given :
Given and , we can parameterize a DDIM-style intermediate transition (Song et al., [35]):
where and . Note that the clean anchor is a parameterization of the transition; the desired matching objective is imposed on , not separately on . By default, we let , which forms a deterministic transition, while its outputs remain random across inputs.
Distributional Supervision at a Perturbed Time. To supervise the intermediate distribution, existing works in the line of score distillation (Luo et al., [20, 22]) further perturb to , where . This smooths the distributions and makes the teacher’s knowledge at additional noise levels available for supervision. For , we use the teacher-compatible kernel
Thus , where . The full sampling chain factors into and . Unless otherwise specified, expectations are taken over sampled , this student transition and subsequent perturbation, with conditioning fixed. Let denote the distribution of the student’s intermediate output . After applying the perturbation kernel, the resulting perturbed marginal distribution is
This is the student distribution whose score is required for reverse-KL supervision at time . Because retains the residual inherited from the student transition, this distribution generally differs from one obtained by adding independent Gaussian noise directly to .
If the optimization is formed by reverse-KL following previous works (Luo et al., [22]), we would require the score difference . The teacher supplies the latter score, whereas the former must be learned online for the current student’s actual perturbation distribution.
The Difficulty of Learning the Student Score. DSM can estimate this score by regressing onto . Although population-correct, under the reference Gaussian forward process this target has larger conditional variance than standard DSM, which regresses (Xue et al., [42]), thereby making the online score harder to learn.
Remark 1 (Mismatch in TDM). To avoid this difficulty, TDM learns its online score through clean-target denoising with importance sampling (Luo et al., [22]). Although importance sampling allows training samples to be drawn from , the objective still estimates the score of the distribution obtained by independently perturbing to . This generally differs from the score of required by the generator update, introducing a distribution mismatch (See detailed derivation in Appendix § 8.§ 2).
Velocity as a Choice of Path and Coupling. This distinction motivates a shift from score distillation to velocity distillation, which can define student fields directly from the actual sampling chain. In particular, Velocity targets can be induced by differentiable paths with correlated or non-Gaussian endpoint couplings, therefore giving us greater flexibility. In the student transition , and share the same input and can be dependent, alongside optional fresh noise; the forward perturbation adds independent Gaussian noise. This gives two choices: use the clean-to-perturbed coupling induced by the composite process , or follow the path alone. Throughout the remaining sections, abbreviates , the perturbed marginal.
A-form: Rectified Velocity. The composite process couples the clean prediction to . Expanding Eq. (4) yields
The effective noise contains the residual inherited from the student and standard Gaussian noise. This can deliver a corresponding rectified velocity coupled with the endpoint:
The A-form rectified velocity uses the clean prediction as its anchor through the composite coupling. It produces reduced training variance when , which benefits learning of the online field. This marginal velocity is an affine reparameterization of a moment .
B-form: Gaussian Velocity. The perturbation defines a path starting at . Differentiating this path with respect to , and using , gives the conditional velocity
Dots denote derivatives with fixed. Its conditional average is the marginal velocity of the perturbation path. This marginal velocity is an affine reparameterization of the score of the actual perturbed distribution (see proof in Appendix § 7.§ 4).
The Induced Student Field. We use the common notation for these two conditional targets, with . The field to be compared with the teacher is
Both formulations can be computed via student samples, but their marginal fields require averaging over the implicit student distribution.
Velocity Adversarial Distillation
Diff-Instruct has discussed that reverse-KL-based score distillation is related to GANs ([5, 20]). We further find that, after shifting from score distillation to velocity distillation, explicitly defining an adversarial game that aims to align the teacher velocity with the student velocity forms a unified framework and view, which deepens the understanding of existing methods and might inspire the design of new distillation losses.
Matching Implicit Velocity. We propose to align the student and teacher velocity fields through an adversarial game defined by their inner products with a vector-valued critic , i.e., . In particular, the critic maximizes its alignment with the teacher velocity and minimizes its alignment with the student velocity, subject to a quadratic penalty. The student opposes the critic by maximizing its own velocity’s alignment with the critic, with the critic held fixed.
For brevity, we write for the sampled conditional velocity, and let denote expectation over and the full sampling chain . Since depends only on , replacing the sampled velocity by its conditional-mean velocity leaves the expected inner product unchanged:
This identity motivates two generator losses with the same expected value but different gradient conventions. The pathwise update differentiates through the full sampling process, changing both the outer sampling distribution and the student velocity at those locations. The conditional-mean update stops the gradient of the sampling distribution and adjusts only the student velocity field.
In particular, the adversarial game in the velocity space can be formulated as:
or
Here sg denotes stop-gradient. Only the outer distribution is stopped in ; retains its parameter dependence. The following lemma reveals the formulation of the optimal critic:
Lemma 2 (Optimal critic). For a fixed student , under the regularity and critic assumptions detailed in Appendix § 7, the maximizing critic is unique up to -null sets and satisfies
Matching Equilibrium. The optimal critic in Theorem 2 is the velocity residual. Its zero identifies the following equilibrium of the two-player game.
Proposition 3 (Matching equilibrium). Under the regularity, critic, and realizability assumptions in Appendix § 7, the game in Eq. (12) admits a Nash equilibrium satisfying
It attains zero velocity discrepancy, the minimum of the maximized critic payoff.
Directly computing the conditional-mean update is intractable because it requires differentiating the implicit field with respect to . The following lemma gives a general flow-product update through a tractable surrogate whose gradient agrees with the conditional-mean update.
Lemma 4 (General Flow-Product Update). Fix a reference parameter and let . Hold the parameters of and fixed, while retaining their input derivatives. Define
Under the regularity conditions detailed in Appendix § 7.§ 3,
Theorem 5. (Matching meanings and algorithmic unification). Use the compatible Gaussian kernel in Eq. (5) and an exact teacher. For the gradient identities, fix the input law and times , , take , and assume exact critics and reference fields with the differentiability conditions of Appendix § 7.§ 3.
(i) Matching. A-form velocity matching is equivalent to the conditional-moment matching
B-form velocity matching is equivalent to distribution matching , and hence .
(ii) Equivalence at . At , and coincide, with identical matching conditions and population gradients under the same generator update.
(iii) Unifying Algorithms. The two forms and two gradient choices yield the correspondences below, up to positive time weights.
| Form | Pathwise w/ detached critic | Flow-product w/ non-detach critic |
| A | MMD practical update (Salimans et al., [30]) | Exact Moment Matching |
| B | Reverse-KL (Luo et al., [20, 22]; Yin et al. [44]) | Fisher Divergence (Luo et al., [21]; Zhou et al. [50]) |
The proof and time coefficients are in Appendix § 7.§ 4.
:::
MMD: Objective and Approximate Update. MMD (Salimans et al., [30]) aims to minimize a squared conditional-moment discrepancy to align the teacher moment and student moment. Because differentiating the student’s implicit moment is intractable, it adopts a heuristic approximate gradient to update the student generator. Our unified framework provides both an exact gradient and a mechanistic explanation of this heuristic: A-form flow-product updates recover the intended objective’s exact gradient under MMD’s fixed-outer-measure convention, while A-form pathwise updates with a detached critic recover the surrogate update as a generator update in the adversarial velocity game, up to positive time weighting on the same sampling chain. Appendix § 7.§ 4 explicitly gives the target objective, the approximation leading to the surrogate, and both gradient identities.
VA-TDM: Distillation Along the Student Trajectory
Generator Learning. We observe unstable training when gradients are propagated through the critic output (Figure 3). We therefore stop the entire critic output in the practical update. Let denote this stop-gradient operation and define
Figure 3. Comparison between detached critic and non-detached critic. We found that the non-detached critic leads to unstable training and collapse.
Note that and correspond to different adversarial flexibility for the generator: the latter allows the generator to cheat the critic by altering both the outer sampling distribution and the velocity. Since the balance of the adversarial game is crucial for performance, we propose optimizing the hybrid loss to provide an explicit knob for controlling the balance.
Critic Learning. With the help of Theorem 2, we parameterize the critic as and fit the student field with
To retain the pre-trained knowledge, we add the regularization term , with , under the same sampling distribution. The optimal minimizer then becomes
We therefore should correctly parameterize the critic as to perform best.
Dual-Form Velocity Game with a Shared Backbone. The A-form produces low-variance online field learning, but it only captures the conditional-mean moment of the distribution; the B-form induces high-variance online field learning, yet it encodes the full distributional information. Since no single velocity construction enjoys both benefits, we propose to optimize the adversarial games induced by the two velocity constructions simultaneously, the generator loss becomes:
where each term follows Eq. (20) with its corresponding velocity target. To avoid the cost of maintaining two online fields, we propose a shared-backbone strategy: the two field networks share the backbone with separate output heads. Table 2 shows that this sharing strategy not only reduces memory and training overhead, but also improves performance.
| Configuration | Verifiable | ImgQ | Preference | Div. | ||||
| GenEval | OCR | Aest. | ImgRwd | PickScore | HPSv2 | HPSv3 | LPIPS | |
| (A-form only) | 0.64 | 0.55 | 5.53 | 1.00 | 0.8671 | 0.2949 | 11.20 | <u>0.58</u> |
| (B-form only) | 0.61 | 0.57 | 5.57 | 0.99 | 0.8665 | <u>0.2987</u> | <u>11.47</u> | 0.59 |
| w/o transition-aware field | 0.58 | 0.53 | 5.56 | 0.99 | 0.8655 | <u>0.2964</u> | 11.13 | 0.56 |
| w/o shared backbone | 0.62 | 0.56 | 5.55 | 0.99 | 0.8669 | 0.2971 | 11.33 | 0.57 |
| 0.63 | <u>0.58</u> | 5.56 | <u>1.01</u> | 0.8674 | 0.2983 | <u>11.47</u> | <u>0.58</u> | |
| + path/flow-product mixing () | 0.65 | <u>0.57</u> | 5.56 | 0.96 | <u>0.8687</u> | 0.2950 | <u>11.39</u> | 0.59 |
| + field regularization () | <u>0.66</u> | <u>0.58</u> | <u>5.58</u> | 0.96 | 0.8682 | 0.2952 | 11.44 | 0.57 |
| + regularization-corrected critic | 0.67 | 0.59 | 5.65 | 1.02 | 0.8702 | 0.3010 | 11.89 | <u>0.58</u> |
Table 2. Ablation study. Best / second-best results are bold / <u>underlined</u>.
Distillation Along the Student Trajectory. Matching only the final output places the entire distributional correction at the end of generation. We have discussed the algorithm for learning one-step marginal transition from to . Following TDM [22], we instead apply VAD to intermediate student transitions along the student trajectory. Let be a sampling grid. Starting from , the student repeatedly applies the student transition in Eq. (4) with and . Each transition learns a distributional mapping from its current noisy state to the next state; no pointwise teacher ODE solution is needed.
On-Trajectory ODE Regression Warm-up. We initialize VA-TDM with ODE regression along the student’s own sampling trajectory. For a sampled transition , we roll out the student from noise to obtain and integrate the teacher ODE over the same interval using multiple solver steps:
We train the student transition in Eq. (4), whose output is denoted by , with
where is the distance metric and is the student marginal at and sg denotes stop-gradient. Following the connection between ODE regression and distribution matching in TDM [22], perfect pointwise matching also implies trajectory distribution alignment. Specifically, assuming exact teacher integration and that every transition is sampled with positive probability,
where denotes the teacher ODE marginal. This follows by induction: each perfectly matched transition pushes the shared current marginal to the same next marginal. Thus, ODE regression imposes a stronger pointwise constraint whose exact satisfaction also fulfills trajectory distribution matching. Therefore, this initialization directly targets the intermediate transitions and marginals optimized by VA-TDM. We subsequently switch to VAD, relaxing pointwise regression to distributional alignment.
Transition-Aware Field Estimation. In both our framework and existing score distillation work, it is crucial for the auxiliary score to accurately track the student distribution in order to achieve strong performance. However, we identify an aspect that has been consistently overlooked in prior work [16, 45]: in the few-step generation setting, student transitions with different step indices generally induce distinct noisy distributions, yet existing methods typically model them with a shared auxiliary score, which theoretically precludes the possibility of precisely tracking the student distribution. TDM tries to avoid this by assigning student transitions with different step indices to disjoint perturbation intervals, which is partially effective but sacrifices part of supervision signals. To resolve this issue, we propose that the field network should additionally receive the identity of the transition—for instance, in our setting, the target timestep of the student transition can be injected as extra input, i.e., learn as online velocity. Results in Table 2 show that this modification is essential for providing accurate supervision signals, and substantially improves diversity and fine-grained rendering of images, such as text rendering.
Experiments
Models and Baselines. We distill SD3.5-Medium [3], FLUX.2-Klein-Base 4B [1] and MiniMax-H3 [25] into four-step generators. For SD3.5-Medium, the baselines include DMD2 [45], D-DMD [16], DPDMD [39], and TDM [22]. For FLUX.2 4B, we compare with the officially released four-step model.
Evaluation Protocol. We evaluate images along four complementary dimensions. Prompt following is assessed using GenEval [4] for compositional generation and OCR for visual text rendering. Image quality is measured by Aesthetic Score [32], while preference is assessed with ImageReward [40], PickScore [12], HPSv2, and HPSv3; these quality and preference metrics are evaluated on DrawBench [28]. Finally, diversity is measured by average pairwise LPIPS [47] between independently generated images by the same prompt. For audio-video generation, we conduct anonymized pairwise human comparisons of audio-visual quality and prompt alignment (Appendix § 9.§ 2).
Main Results
Quantitative Results. Table 1 compares VA-TDM with few-step baselines and their teachers on text-to-image generation. At four NFE, VA-TDM improves compositional prompt following and text rendering on both backbones. On SD3.5-Medium, it leads the compared few-step models on all verifiable, quality, and preference metrics, raising GenEval from the strongest baseline TDM’s 0.61 to 0.67 while increasing TDM’s diversity. On FLUX2, gains in prompt following and most quality and preference metrics are accompanied by an LPIPS increase from 0.47 to 0.56. Together, the results demonstrate the state-of-the-art performance of our proposed VA-TDM in few-step generation.
| Model | NFE | GenEval | OCR | Aest. | ImgRwd | PickScore | HPSv2 | HPSv3 | LPIPS |
| SD3.5-Medium (teacher, CFG=4.5) | 0.65 | 0.58 | 5.37 | 0.86 | 0.8628 | 0.2805 | 9.78 | 0.61 | |
| SD3.5-Medium (teacher, CFG=7) | 0.67 | 0.62 | 5.37 | 0.96 | 0.8640 | 0.2854 | 9.87 | 0.60 | |
| DMD2 | 4 | 0.60 | 0.53 | 5.55 | 1.00 | 0.8648 | <u>0.2977</u> | 11.18 | 0.53 |
| D-DMD | 4 | 0.55 | 0.56 | 5.51 | 0.99 | 0.8567 | <u>0.2897</u> | 10.93 | 0.57 |
| DPDMD | 4 | <u>0.62</u> | 0.33 | 5.28 | 0.80 | 0.8530 | 0.2707 | 8.13 | 0.59 |
| TDM | 4 | 0.61 | <u>0.57</u> | <u>5.58</u> | <u>1.01</u> | <u>0.8679</u> | 0.2969 | <u>11.54</u> | 0.57 |
| VA-TDM (Ours) | 4 | 0.67 | 0.59 | 5.65 | 1.02 | 0.8702 | 0.3010 | 11.89 | <u>0.58</u> |
| FLUX.2-Klein-Base 4B (teacher, CFG=4.0) | 0.77 | 0.60 | 5.31 | 0.85 | 0.8605 | 0.2768 | 10.28 | 0.56 | |
| FLUX.2-Klein-Base 4B (teacher, CFG=7) | 0.80 | 0.64 | 5.38 | 0.93 | 0.8620 | 0.2840 | 10.58 | 0.54 | |
| FLUX.2-Klein 4B | 4 | <u>0.78</u> | <u>0.64</u> | <u>5.28</u> | <u>1.04</u> | 0.8746 | <u>0.2886</u> | <u>11.27</u> | <u>0.47</u> |
| VA-TDM (Ours) | 4 | 0.80 | 0.69 | 5.43 | 1.07 | <u>0.8694</u> | 0.2930 | 11.58 | 0.56 |
Table 1. Table 1 | Systematic evaluation. Best / second-best few-step results per backbone are bold / underlined.
Table 1 | Systematic evaluation. Best / second-best few-step results per backbone are bold / <u>underlined</u>.
Qualitative Results. Across visual comparisons in image and video generation (Figure 1 and Figure 4), VA-TDM consistently delivers better visual quality compared with few-step baselines, significantly approaching the many-step base model. Moreover, we find that previous methods either suffer from oversaturation or excessively dark outputs, whereas VA-TDM effectively avoids these undesirable artifacts.
Figure 1. Comparison of VA-TDM-H3 and competing baselines. Our VA-TDM delivers better visual quality and suffers less oversaturation than the few-step competitor. More samples in Appendix § 10.
Figure 4. Qualitative comparison among VA-TDM and competing baselines. Printed labels: Base Model, 50 NFE; DMD2, 4 NFE; DP-DMD, 4 NFE; D-DMD, 4 NFE; TDM, 4 NFE; VA-TDM (Ours), 4 NFE. Prompts: A majestic lion with a thick mane in warm evening light, close-up wildlife portrait. A small wooden toy airplane on top of a stack of colorful books, soft studio light. Fresh strawberries and blueberries falling into a bowl of yogurt, high-speed food photograph.
Ablation Study
Ablation Study
We examine the components introduced in § 3.3 using four-step SD3.5-Medium (Table 2). Following Eq. (22), mixes the A-form and B-form games; the separate coefficient in Eq. (20) controls the path/flow-product mixture within each form, and weights the online field’s regularization toward the teacher velocity. Starting from , the final three rows report cumulative configurations with generator-update mixing, field regularization, and critic correction.
Dual-Form Velocity Mixing. The A-form rectified velocity and B-form Gaussian velocity in § 3.1 yield complementary results. A-form alone improves GenEval over B-form alone, but has lower OCR and HPS scores. Combining the two games produces higher OCR and ImageReward than either endpoint while retaining the A-form diversity. This supports combining their supervision.
Shared Backbone. Sharing the online-field backbone between the two forms not only reduces training cost but also improves performance. We believe that this is because the high-variance B-form is difficult to learn on its own, whereas the shared backbone allows the network to learn from both online fields jointly, which potentially leads to better internal representations and thus provides better supervision signals.
Path/Flow-Product Mixing. The mixed update notably improves GenEval and diversity, while the other metrics remain roughly unchanged. This indicates that the mixed update allows us to find a better-performing sweet spot.
Online Field Regularization and Regularization-Corrected Critic. We augment the field regression in Eq. (21) with the teacher-velocity regularizer, using . Even without critic correction, this also slightly improves GenEval, OCR, supporting the benefit of retaining teacher information during online field learning. By applying the regularization-corrected critic and adjusted loss coefficients, the resulting configuration notably improves the performance and achieves the strongest verifiable, quality, and preference scores among variants.
Transition-Aware Field Estimation. This enables accurate tracking of the student field and notably improves instruction following (GenEval), text rendering (OCR), and diversity (LPIPS).
Related Works
Training-Free Diffusion Acceleration. Training-free methods for accelerating diffusion model sampling have progressed substantially ([14, 18, 23, 33, 41, 48]). However, diffusion distillation remains essential for high-quality few-step generation.
Diffusion Distillation. Distilled models generate images from noise in one or a few steps. Two main approaches transfer knowledge from pre-trained DMs: 1) Trajectory distillation [19, 24, 29, 36, 37, 43] matches teacher ODE trajectories at the instance level, facing challenging instance-wise alignment and numerical errors in teacher PF-ODE solutions; 2) Distribution matching via score distillation [16, 20, 39, 44, 49 b] aligns student and teacher distributions using objectives such as reverse KL, Fisher divergence, and moment matching. FGM aligns velocity fields but considers only Gaussian forward processes [7], where its objective reduces to Fisher divergence [50]. TDM [22] has shown strong few-step performance by learning partial denoising at each step. Our VA-TDM unifies score distillation, Fisher divergence, and moment matching while yielding new distillation objectives. Combined with transition-aware online field learning, it achieves state-of-the-art few-step performance.
Generative adversarial networks. Early GAN research achieved strong results on domain-specific datasets [10, 11, 26, 46], while it has been effectively extended these successes to ImageNet [2, 27, 31]. However, training instability and limited scalability remain key challenges. Large-scale text-to-image GANs still rely on convolutional architectures with complex designs [9, 51], and scaling transformer-based GANs remains difficult [8, 13]. Adversarial Flow Models (AFM) [15] train many-step flow-matching models through an adversarial game in velocity space, requiring discriminator Jacobian-vector products (JVPs). Our work also formulates an adversarial game in velocity space, but targets few-step distillation and avoids expensive JVP computation.
Conclusions
We introduced Velocity Adversarial Distillation (VAD), a unified framework that guides distillation through velocity fields constructed from the student’s sampling process. By separating the choice of field from the generator update, VAD connects score distillation, Fisher-based matching, and moment matching, clarifies MMD’s heuristic updates, and derives new training objectives. Building on this framework, VA-TDM combines dual-form supervision with transition-aware online velocity learning to supervise partial denoising along few-step student trajectories, without adding inference cost. Experiments across text-to-image and text-to-video generation demonstrate VA-TDM consistently achieves state-of-the-art four-step performance. VA-TDM also scales effectively to 33B MiniMax-H3, outperforming Fast-H3 in human evaluation and substantially narrowing the perceptual quality gap to the many-step teacher. These results highlight the value of our work, establishing VAD as a principled foundation for designing effective few-step distillation methods.
Proofs of Main Results
Throughout, time variables and conditioning information are fixed. For the population game in Eq. (12), assume that the teacher field and sampled target have finite second moments, and that critic optimization ranges over all square-integrable fields under the student’s marginal law. Theorem 3 additionally assumes realizability: there exists a student with almost everywhere under . The critic maximizes , while the student minimizes with the critic fixed.
Proof of Lemma 2
Proof. Conditioning on gives the marginal payoff
Writing and completing the square yields
The upper bound is attained exactly when almost everywhere under , proving the optimal critic and value. □
Parameterizing makes critic optimization equivalent to the field regression in Eq. (21), by the conditional variance decomposition.
Proof of Proposition 3
Proof. Choose a realizable matching student and the globally zero critic . For any admissible critic,
so is a best response. With this critic, for every student, so is also a best response. Thus is a Nash equilibrium. By Theorem 2, the maximized critic payoff is nonnegative and vanishes at , establishing the claimed optimality. □
Proof of Lemma 4
Our proof uses the conditional-projection principle underlying denoising score matching [38] and the score-projection identity used in SiD [50] and SIM [21]. FGM [7] develops the corresponding flow-product identity. Here the argument applies to conditional target fields under the student’s composite sampling law, without requiring a Gaussian forward kernel.
Assume differentiable marginal densities and conditional fields near , integrability, and domination sufficient to interchange differentiation and integration. Reparameterized derivatives are assumed to exist and commute with expectation. Input laws and time variables are held fixed during differentiation; auxiliary parameters are frozen while their input derivatives are retained. Proof. Let be the joint sampling law, write and , and define . For any admissible test field , conditional expectation gives
Fix , set , and hold and fixed as functions. The projection identity gives
At , the centered difference vanishes pointwise, so its product with the sampling-density derivative is zero. Hence
where differentiates at a fixed spatial argument. Since , this is precisely
whose right-hand side holds the outer measure fixed at . The argument allows both the target and the joint law to depend on , and applies to both A-form and B-form. □
The practical mixed update in Eq. (20) uses learned auxiliary fields and detaches the critic output while retaining input gradients through the centering field.
Proof of Theorem 5
Fix , , and . Let denote the law of the local reparameterized chain
with fixed input law and independent standard Gaussian . Use the regularity conditions of Appendix § 7.§ 3, an exact teacher, and finite reverse KL with the boundary conditions needed for its pathwise gradient. Write for total sample derivatives with base noises fixed, and for field derivatives at fixed spatial arguments. All gradient identities are evaluated at a reference student , using the exact reference field and critic . For a teacher-regularized regressor, the unregularized reference field is .
Proof. (i) Matching. For A-form, conditioning the target on gives
with the moments defined in Eq. (18). Thus zero residual is equivalent to conditional-moment matching. For B-form, Gaussian differentiation gives . Using the coefficients of Eq. (5), with and , yields
The kernel maps to , since and . Gaussian smoothing gives positive densities on , so equal scores imply equal normalized densities. Since and the Gaussian characteristic function never vanishes, deconvolution makes equivalent to .
(ii) The clean endpoint. At , and , giving
The sampled targets coincide, so their conditional means, optimal critics, and gradients under identical differentiation conventions coincide.
(iii) Pathwise updates with the critic output detached. Set and . From Eq. (34),
and the sampled targets satisfy
The reverse-KL pathwise identity is
Substituting the optimal critics into Eq. (19) gives
The A-form expectation is the MMD surrogate update made explicit below; the B-form gives reverse KL. Both coefficients are positive time weights.
(iv) Flow-product updates without output detachment. Define the discrepancy with its outer measure held fixed:
Theorem 4, applied to the optimal reference critic, gives
The residual identities in part (i) identify the objectives as
These are the conditional-moment and scaled Fisher discrepancies, respectively, with the fixed-outer-measure gradient convention.
MMD objective and surrogate. To make the MMD correspondence explicit [30], write . Its moment objective and local fixed-measure version are
The exact local gradient is
MMD approximates the implicit conditional-mean derivative by the sampled clean-prediction derivative, giving
This is the gradient of the detached surrogate
Parts (iii)–(iv) therefore give both correspondences directly:
Thus flow-product recovers the exact local moment gradient, while the detached pathwise update recovers MMD’s surrogate direction up to time weighting. □
Supporting Derivations
Reference DSM Variance
Consider the reference Gaussian forward process with , , and , where the standard Gaussian noises are independent of each other and . The two DSM targets
have the same conditional mean . The Gaussian bridge and the law of total covariance give the comparison of Xue et al. [42] in our notation:
The excess is positive for and vanishes at . This reference-process comparison motivates the noisy-target DSM discussion in § 3.1.
Clean-Target and Path-Marginal Score Mismatch
Fix , , with and the input law. Let be the joint law of the intermediate output and clean anchor. The two perturbed marginals are
TDM’s ideal importance-weighted clean-target objective [22] uses the ratio of these conditional kernels. Since both kernels have positive densities, for any integrable loss ,
For with , an unrestricted regressor with finite second moments has optimum . Gaussian differentiation identifies its score and the required path score as
Thus importance weighting recovers the clean-denoising objective, whose population score can differ from the path score needed by the generator. For example, in one dimension with and , the transition gives : the path marginal is and the clean marginal is , with different variances for . At , and the two marginals coincide.
Experimental Details
This section provides the training details for the experiments in § 4.
Training Configurations
VA-TDM trains a generator and an online auxiliary field while keeping the teacher fixed. The generator combines the two velocity forms through Eq. (22), and the auxiliary field follows the regression and regularization objectives in § 3.3. By default, we apply LoRA fine-tuning with rank 64. We update the online score and generator once each iteration. We use the AdamW optimizer and set .
User Study
The study in Figure 2 compares MiniMax-H3 with Fast-H3, VA-TDM with MiniMax-H3, and VA-TDM with Fast-H3. We carried out a user study in which participants were shown two anonymized audio-videos, each produced by a different model, and were asked to pick the one with superior audio-visual quality and closer alignment with the prompt. For audio-video generation, we randomly chose 10 prompts, and every generated audio-video was manually checked to confirm it contained no inappropriate or harmful content.
Additional Visual Samples
Figs. 5 to 9 provide additional text-to-video comparisons complementing Figure 1. Each figure presents six selected frames per method, with the summarized prompt shown above the panels. From top to bottom, the rows correspond to MiniMax-H3 (100 NFE), Fast-H3 (4 NFE), and VA-TDM-H3 (4 NFE). The examples cover human–animal interactions, clothing changes, fantasy combat, stylized game scenes, and multi-scene cinematic sequences.
Figure 5. Summarized Prompt: A figure wearing a rabbit mask and pastel clothes walks and sprints beside a white rhino in spiked blue armor, charging the camera before stopping together as desert dust settles. Figure 5 | Additional visual comparison of MiniMax-H3 (100 NFE), Fast-H3 (4 NFE), and VA-TDM-H3 (4 NFE).
Figure 6. Summarized Prompt: In one continuous backward tracking shot, a walking model’s clothes morph through suits, silk dresses, denim, and coats amid passing buses, preserving her identity before she stops. Figure 6 | Additional visual comparison of MiniMax-H3 (100 NFE), Fast-H3 (4 NFE), and VA-TDM-H3 (4 NFE).
Figure 7. Additional visual comparison of MiniMax-H3 (100 NFE), Fast-H3 (4 NFE), and VA-TDM-H3 (4 NFE).
Figure 8. Summarized Prompt: Six chibi characters battle on a floating arena using cards, a golden beast, and shields, displaying “-120”, “-128”, “-150” damage as “TURN 1” becomes “TURN 2”, with “END TURN” visible. Additional visual comparison of MiniMax-H3 (100 NFE), Fast-H3 (4 NFE), and VA-TDM-H3 (4 NFE).
Figure 9. Summarized Prompt: Robots assemble a sports car around a man before neon drifting, wet smartphone camera closeups, and a showroom portrait distorted by a glass lens, ending with “Created with MiniMax H3”. Additional visual comparison of MiniMax-H3 (100 NFE), Fast-H3 (4 NFE), and VA-TDM-H3 (4 NFE).
References
- [1]Black Forest Labs. FLUX.2 [klein]. https://bfl.ai/models/flux-2-klein, 2026.
- [2]Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2019.arxiv.org/abs/1809.11096
- [3]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yash Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024. URL https://arxiv.org/abs/2403.03206.
- [4]Dhruba Ghosh, Hanneah Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132–52152, 2023.arxiv.org/abs/2310.11513
- [5]Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
- [6]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840–6851, 2020.arxiv.org/abs/2006.11239
- [7]Zemin Huang, Zhengyang Geng, Weijian Luo, and Guo jun Qi. Flow generator matching, 2024. URL https://arxiv.org/abs/2410.19310.
- [8]Yifan Jiang, Shiyu Chang, and Zhangyang Wang. Transgan: Two pure transformers can make one strong gan, and that can scale up. Advances in Neural Information Processing Systems, 34: 14745–14758, 2021.
- [9]Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10124–10134, 2023.arxiv.org/abs/2303.05511
- [10]Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110–8119, 2020.arxiv.org/abs/1912.04958
- [11]Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. Advances in Neural Information Processing Systems, 34:852–863, 2021.arxiv.org/abs/2106.12423
- [12]Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems, 36:36652–36663, 2023.arxiv.org/abs/2305.01569
- [13]Kwonjoon Lee, Huiwen Chang, Lu Jiang, Han Zhang, Zhuowen Tu, and Ce Liu. Vitgan: Training gans with vision transformers. arXiv preprint arXiv:2107.04589, 2021.arxiv.org/abs/2107.04589
- [14]Tiancheng Li, Weijian Luo, Zhiyang Chen, Liyuan Ma, and Guo-Jun Qi. Self-guidance: Boosting flow and diffusion generation on their own. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.arxiv.org/abs/2412.05827
- [15]Shanchuan Lin, Ceyuan Yang, Zhijie Lin, Hao Chen, and Haoqi Fan. Adversarial flow models, 2026. URL https://arxiv.org/abs/2511.22475.
- [16]Dongyang Liu, Peng Gao, David Liu, Ruoyi Du, Zhen Li, Qilong Wu, Xin Jin, Sihan Cao, Shifeng Zhang, Steven Hoi, and Hongsheng Li. Decoupled DMD: CFG augmentation as the spear, distribution matching as the shield. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=jBztv0iCKE.arxiv.org/abs/2511.22677
- [17]Xingchao Liu, Chengyue Gong, et al. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2022.arxiv.org/abs/2209.03003
- [18]Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775–5787, 2022.arxiv.org/abs/2206.00927
- [19]Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388, 2021.arxiv.org/abs/2101.02388
- [20]Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. Advances in Neural Information Processing Systems, 36, 2023.arxiv.org/abs/2305.18455
- [21]Weijian Luo, Zemin Huang, Zhengyang Geng, J Zico Kolter, and Guo-Jun Qi. One-step diffusion distillation through score implicit matching. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=ogk236hsJM.arxiv.org/abs/2410.16794
- [22]Yihong Luo, Tianyang Hu, Jiacheng Sun, Yujun Cai, and Jing Tang. Learning few-step diffusion models by trajectory distribution matching. arXiv preprint arXiv:2503.06674, 2025.arxiv.org/abs/2503.06674
- [23]Jiajun Ma, Shuchen Xue, Tianyang Hu, Wenjia Wang, Zhaoqiang Liu, Zhenguo Li, Zhi-Ming Ma, and Kenji Kawaguchi. The surprising effectiveness of skip-tuning in diffusion sampling. arXiv preprint arXiv:2402.15170, 2024.arxiv.org/abs/2402.15170
- [24]Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14297–14306, June 2023.arxiv.org/abs/2210.03142
- [25]MiniMax. MiniMax-H3. Hugging Face model card, 2026. URL https://huggingface.co/MiniMaxAI/MiniMax-H3. Accessed: 2026-09-26.
- [26]Scott Reed, Zeynep Akata, Xinchen Yan, Lajianlugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In International conference on machine learning, pages 1060–1069. PMLR, 2016.arxiv.org/abs/1605.05396
- [27]Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015.
- [28]Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479–36494, 2022.arxiv.org/abs/2205.11487
- [29]Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022.arxiv.org/abs/2202.00512
- [30]Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching, 2024. URL https://arxiv.org/abs/2406.04103.
- [31]Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pages 1–10, 2022.arxiv.org/abs/2202.00273
- [32]Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278–25294, 2022.
- [33]Chenyang Si, Ziqi Huang, Yuming Jiang, and Ziwei Liu. Freeu: Free lunch in diffusion u-net. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4733–4743, 2024.arxiv.org/abs/2309.11497
- [34]Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR, 2015.
- [35]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=St1giarCHLP.arxiv.org/abs/2010.02502
- [36]Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=WNzy9bRDvG.arxiv.org/abs/2310.14189
- [37]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.arxiv.org/abs/2303.01469
- [38]Pascal Vincent. A Connection Between Score Matching and Denoising Autoencoders. Neural Computation, 23(7):1661–1674, 2011.DOI
- [39]Tianhe Wu, Ruibin Li, Lei Zhang, and Kede Ma. Diversity-preserved distribution matching distillation for fast visual synthesis. arXiv preprint arXiv:2602.03139, 2026.
- [40]Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences for text-to-image generation. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 15903–15935, 2023.arxiv.org/abs/2304.05977
- [41]Shuchen Xue, Zhaoqiang Liu, Fei Chen, Shifeng Zhang, Tianyang Hu, Enze Xie, and Zhenguo Li. Accelerating diffusion sampling with optimized time steps. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8292–8301, 2024.arxiv.org/abs/2402.17376
- [42]Shuchen Xue, Chongjian Ge, Shilong Zhang, Yichen Li, and Zhi-Ming Ma. Advantage weighted matching: Aligning rl with pretraining in diffusion models, 2025. URL https://arxiv.org/abs/2509.25050.
- [43]Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng. Perflow: Piecewise rectified flow as universal plug-and-play accelerator. arXiv preprint arXiv:2405.07510, 2024.arxiv.org/abs/2405.07510
- [44]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. arXiv preprint arXiv:2311.18828, 2023.arxiv.org/abs/2311.18828
- [45]Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T. Freeman. Improved distribution matching distillation for fast image synthesis, 2024. URL https://arxiv.org/abs/2405.14867.
- [46]Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision, pages 5907–5915, 2017.arxiv.org/abs/1612.03242
- [47]Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018.arxiv.org/abs/1801.03924
- [48]Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36:49842–49869, 2023.arxiv.org/abs/2302.04867
- [49]Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, and Hai Huang. Adversarial score identity distillation: Rapidly surpassing the teacher in one step, 2024a. URL https://arxiv.org/abs/2410.14919.
- [50]Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In International Conference on Machine Learning, 2024b.arxiv.org/abs/2404.04057
- [51]Jiapeng Zhu, Ceyuan Yang, Kecheng Zheng, Yinghao Xu, Zifan Shi, Yifei Zhang, Qifeng Chen, and Yujun Shen. Exploring sparse moe in gans for text-conditioned image synthesis. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18411–18423, 2025.arxiv.org/abs/2309.03904








