betaFeedback

WorksComputer Science

Speed-Variation Distillation for Generative Modeling

Abstract

While diffusion models excel at image generation, their denoising trajectories pass through corrupted intermediate states far from the image manifold. We ask whether generation can instead proceed through recognizable images throughout the entire trajectory. We introduce a generative framework built around a variation-reducing degradation process that contracts diverse images toward a shared prototype while preserving recognizable intermediate states. Because such manifold traversal lacks a simple analytical form, we construct the trajectories using generative-prior-based image morphing. We then distill the reverse process into a diversification operator that transforms the prototype into increasingly varied samples. To resolve the one-to-many ambiguity of the reverse process, we associate each trajectory with its initial noise latent and apply perturbations to improve robustness against accumulated errors. At inference, every step yields a recognizable image, allowing early termination and enabling a controllable speed–variation trade-off.

Comparison of the standard diffusion process and the proposed degradation process

Figure 1. (a) Standard diffusion generates images by denoising Gaussian noise toward the data manifold, producing corrupted intermediate states for most of the trajectory. (b) Our method instead starts from a shared prototype already on the data manifold and reverses our variation-reducing degradation process, moving the point within the data distribution to gradually diversify it, enabling a controllable speed–variation trade-off.

Introduction

Diffusion [15, 35] and flow-matching [25] models have emerged as the dominant generative paradigm due to their stable training dynamics and high sample quality. They generate data by progressively transforming a simple prior distribution into the target data distribution through a sequence of denoising steps. However, because the intermediate states along this trajectory are corrupted by noise, they cannot be used as natural images during generation. This raises a question: Can we design a generative process whose entire trajectory consists of valid, clean images from the target distribution?

Several studies, including Cold Diffusion [2], Soft Diffusion [8], Scale-Space Diffusion [28], and physics-inspired diffusion models [12, 31], have explored degradation processes beyond Gaussian noise to improve generation quality, efficiency, and interpretability. Yet, these methods still fail to produce recognizable intermediate images as they progressively corrupt the input and move it away from the data manifold. As a result, the generated trajectories remain visually uninterpretable.

Our goal is to design a degradation process whose intermediate states remain on the data manifold. Unlike prior approaches that corrupt images with noise [15, 25, 35], blur [16, 31], or downsampling [1, 28], we gradually contract image variation toward a single prototype image while preserving visual realism throughout. By learning to reverse this trajectory, our model yields a generative process that can be terminated early at any step to produce a fully valid image—introducing a novel trade-off that exchanges sample diversity for faster generation.

We present a first realization of this principle through a distillation framework that extracts variation-reducing trajectories from pretrained generative models. Specifically, we construct a dataset of trajectories via generative-prior-based image morphing, progressively transforming teacher-generated images along the data manifold toward a single, shared prototype. However, learning to reverse these trajectories is an ill-posed one-to-many mapping from the single prototype to diverse targets, causing standard regression to collapse into blurry average predictions. Drawing inspiration from consistency distillation [27, 36, 42] and rectified flow distillation [26], we resolve this multi-modality by conditioning the model on the init noise latent of the teacher-generated target image. This anchors each trajectory and converts the multi-modal generative task into a deterministic, one-to-one mapping. Finally, to mitigate error accumulation during iterative generation, we inject small perturbations into the model input during both training and inference.

Our main contributions are twofold: First, we introduce a novel variation-reducing degradation that contracts image variation toward a shared prototype while preserving visual realism along the data manifold. Second, we propose a distillation framework to learn its reverse process, establishing a novel trade-off between speed and sample diversity.

Related Work

Diffusion Models Beyond Gaussian Noise

Diffusion models [15, 35] and flow models [25] have emerged as the leading paradigms for generative modeling, achieving state-of-the-art image synthesis quality and outperforming GAN-based methods [9]. Their success stems from progressively transforming samples drawn from a simple prior, typically a Gaussian distribution, into natural images. Recent work investigates whether the addition of noise is necessary to bridge the gap between a simple reference distribution and the target data distribution. Cold Diffusion [2] and Soft Diffusion [8] show that a wide range of degradations can be paired with a learned restoration process. Subsequent work has explored blur-based trajectories in particular. Inverse Heat Dissipation models [31] use blurring to encourage coarse-to-fine generation, Blurring Diffusion [16] connects blur to non-isotropic Gaussian diffusion, and Warm Diffusion [17] studies how to balance blur and noise while preserving proximity to the data manifold. Beyond Blur [12] proposes an advection-diffusion corruption process that improves both image diversity and generation quality. More recently, Scale Space Diffusion [28] leverages scale-space theory to progressively downsample images during the forward process and restore their original resolution during the reverse process, improving generation efficiency. However, these methods are primarily designed to improve sample quality and still rely on corruption processes that progressively transform images into visually unrecognizable intermediate states. In this work, we introduce a degradation process that progressively reduces image variation toward a single prototype while preserving recognizable intermediate images.

Image Morphing

Our variation-reducing degradation process uses image morphing to transform each image toward a shared prototype while preserving realistic intermediate states. Classical morphing methods rely on correspondence-based warping and blending, often requiring manual annotations and producing ghosting artifacts [3, 4, 7, 24, 34]. Deep generative models with continuous semantic latent spaces, such as variational autoencoders [21], StyleGAN [19, 20], and Diffusion Autoencoders [29], enable image morphing through latent interpolation that produces plausible intermediate states, but are typically limited to their training domains. Recent methods instead leverage large-scale text-to-image diffusion priors. Wang and Golland [39] interpolate inverted noises and textual-inversion embeddings with pose guidance. IMPUS [41] additionally introduces pair-specific LoRA adaptation and perceptually uniform sampling. DiffMorpher [43] fits and interpolates endpoint-specific LoRAs without textual inversion. In this work, we adopt DiffMorpher for its efficiency in our many-to-one setting, where the prototype LoRA is fitted once and reused across all trajectories. Note that our degradation framework is agnostic to the specific morphing technique.

Background

Diffusion models progressively add noise to an image during the forward process and learn to reverse this process by denoising the noisy image back to a clean image. Latent Diffusion Models [32] operate in the latent space of a Variational Autoencoder (VAE) [21] to enable efficient high-resolution image generation. This is achieved by first encoding the input image xx into a latent representation z~=z0\tilde{z} = z^{0} using the VAE encoder E\mathcal{E}. The forward diffusion process then perturbs z0z^{0} to a latent variable at diffusion timestep τ\tau, which can be expressed as:

zτ=ατ z0+1−ατ ϵ,ϵ∼N(0,I),(1)z^{\tau} = \sqrt{\alpha_{\tau}}\,z^{0} + \sqrt{1-\alpha_{\tau}}\,\epsilon,\qquad\epsilon\sim\mathcal{N}(0,I), \tag*{(1)}

where {ατ}τ=0T\{\alpha_{\tau}\}_{\tau=0}^{T} denotes the signal schedule over TT diffusion timesteps. During the backward process, the diffusion models predict noise ϵ^τ\hat{\epsilon}^{\tau} added in the input zτz^{\tau}, however, this predicted noise can be turn into predicted clean z0z^{0} denoted as z^0\hat{z}^{0} using the following equation:

z^=z^0=zτ−1−ατ ϵ^τατ.(2)\hat{z} = \hat{z}^{0} = \frac{z^{\tau} - \sqrt{1-\alpha_{\tau}}\,\hat{\epsilon}^{\tau}}{\sqrt{\alpha_{\tau}}}. \tag*{(2)}

Method

We propose a distillation framework that learns a generative process with a controllable trade-off between sampling speed and output variation. Our framework consists of two main components. First, we define a variation-reducing degradation process that maps diverse images toward a shared prototype through realistic intermediate states, materialized as a curated trajectory dataset for efficient training. Second, we distill its reverse process into a diversification operator, employing noise conditioning and input perturbations to improve robustness during iterative generation. An overview of our method is shown in Figure 2.

Figure 2. Overview. (a) Generate images with a pretrained LDM, select a prototype, and construct variation-reducing trajectories. (b) We train the diversification operator using noise and variation perturbations to improve robustness during iterative generation, conditioned on the associated noise latent η\eta. (c) The trained model generates clean, natural trajectories with a controllable speed-variation trade-off.

Variation-Reducing Degradation Process

We define a variation-reducing degradation process that progressively contracts diverse images toward a shared prototype through realistic intermediate states. Since this transformation traverse the image manifold rather than following an explicit corruption process, such as Gaussian noising, we realize it through image morphing with pretrained generative models to keep the trajectories close to the data manifold. Specifically, we first generate an image set and select a representative image as the shared prototype. We then connect each image to this prototype through a realistic morphing trajectory, producing trajectory data for training the reverse diversification process.

Generating images and selecting the prototype. We use a Latent Diffusion Model (LDM) [32] to generate a set of images X={xi}\mathcal{X} = \{x_i\} together with the initial noise latent ηi\eta_i associated with each image. To select a representative prototype while preserving realism, we avoid pixel-space averaging, which produces blurry artifacts. Instead, we adopt a simple approach: we encode all images using a pretrained CLIP image encoder [30] and select the prototype x⋆x^\star as the image with the highest average cosine similarity to all other images in X\mathcal{X}.

Constructing image trajectories. To obtain smooth trajectories between the prototype and each target image, we use DiffMorpher [43], an image morphing method, to generate image morphing sequences. Specifically, for each target image x∈Xx \in\mathcal{X}, DiffMorpher generates a sequence of T+1T + 1 images that gradually transforms the target image into the prototype image x⋆x^\star:

{x0,x1,x2,…,xT}\{x_0, x_1, x_2, \ldots, x_T\}

where x0=xx_0 = x is the target image and xT=x⋆x_T = x^\star is the prototype image. We repeat this process for every image in X\mathcal{X}, yielding a set of trajectories connecting the shared prototype to all images in the dataset. Since generating these morphing trajectories requires costly inference through a generative prior, we precompute and cache them as a trajectory dataset for efficient model training.

Diversification Operator

Our goal is to distill the reverse of our variation-reducing trajectories into a diversification operator. Given an image xtx_t at reduced variation, this operator predicts the higher-variation state xt−1x_{t-1}. Starting from the shared prototype xTx_T, iterative application of this operator acts as a generative process that yields increasingly diverse samples over progressive inference steps.

Model input. We train our diversification operator by fine-tuning a Latent Diffusion Model (LDM) on the curated trajectory dataset. The model is initialized from the same pretrained LDM used to generate the initial image set and its associated initial noise latents. Training is performed in the LDM latent space Z\mathcal{Z}, where each trajectory image xtx_t is encoded by the VAE as zt∈Zz_t \in\mathcal{Z}.

Because the reverse mapping is one-to-many, ztz_t alone leaves the diversification direction ambiguous, causing the model to average over plausible targets and produce blurry outputs. We therefore modify the model input by concatenating the associated initial noise latent η\eta with the current image latent ztz_t along the channel dimension CC at trajectory step tt, resulting in the input x=(zt,η)\mathbf{x}=(z_t,\eta), where x∈R2C×H×W\mathbf{x}\in\mathbb{R}^{2C \times H \times W}.

However, directly training on this input can lead to error accumulation, as small prediction errors may propagate and compound over successive inference steps. We improve robustness using two complementary strategies: variation perturbation and noise perturbation [6, 23]. During training, we perform variation perturbation by randomly replacing the input ztz_t with a lower-variation state zt+sz_{t+s} from its local neighborhood, where 0≤s<smax⁡0 \le s < s_{\max} and smax⁡s_{\max} denotes the maximum perturbation range. The perturbed input state thus becomes z˙t=zt+s\dot{z}_t=z_{t+s}. Next, we apply noise perturbation to z˙t\dot{z}_t via the forward diffusion process at a fixed global timestep τ\tau (Eq. (1)), yielding the noisy latent z˙tτ\dot{z}_t^\tau. The complete input to the model is therefore:

x=(z˙tτ,η).\mathbf{x}=(\dot{z}_t^\tau,\eta).

In addition to the latent input, we condition the denoising network on the current variation step tt. To preserve the timestep-conditioning mechanism of the pretrained LDM, we offset the variation timestep by the fixed diffusion perturbation timestep τ\tau: t~=t+τ\tilde{t}=t+\tau, and t~\tilde{t} is the timestep provided to the denoising network. The model therefore predicts

ϵ^t=ϵθ(x,t~).(3)\hat{\epsilon}_t=\epsilon_\theta(\mathbf{x},\tilde{t}). \tag*{(3)}

This conditioning allows the model to distinguish different stages of the diversification trajectory while remaining close to the timestep conditioning used by the pretrained LDM.

Training objective. During training, as outlined in Algorithm 1, we first sample a trajectory step tt, with zt−1z_{t-1} as the prediction target, and construct the input x=(z˙tτ,η)\mathbf{x}=(\dot{z}_t^\tau,\eta). Since our model is initialized from a pretrained LDM that Algorithm 1 Training

Require: Trajectory dataset D\mathcal{D}, model ϵθ\epsilon_\theta, noise perturbation τ\tau, variation perturbation smax⁡s_{\max}

  1. while not converged do

  2. Sample (zt,zt−1,η,t)∼D(z_t, z_{t-1}, \eta, t) \sim\mathcal{D}

  3. Sample s∼{0,…,smax⁡−1}s \sim\{0, \ldots, s_{\max} - 1\}

  4. Set z˙t←zt+s\dot{z}_t \leftarrow z_{t+s}

  5. Perturb z˙t\dot{z}_t to obtain z~t\tilde{z}_t using (1)

  6. Set t~←t+τ\tilde{t} \leftarrow t + \tau

  7. ϵ^t←ϵθ(concat⁡(z~t,η),t~)\hat{\epsilon}_t \leftarrow\epsilon_\theta(\operatorname{concat}(\tilde{z}_t, \eta), \tilde{t})

  8. Recover z^t−1\hat{z}_{t-1} using (2)

  9. Compute L\mathcal{L} using (4)

  10. Update θ\theta

  11. end while

Algorithm 2 Inference

Require: Model ϵθ\epsilon_\theta, noise perturbation τ\tau

  1. Sample η∼N(0,I)\eta\sim\mathcal{N}(0, I)

  2. Prototype latent zTz_T from ϵθ\epsilon_\theta

  3. for t=T,T−1,…,1t = T, T - 1, \ldots, 1 do

  4. Perturb ztz_t to obtain z~t\tilde{z}_t (1)

  5. Set t~←t+τ\tilde{t} \leftarrow t + \tau

  6. ϵ^t←ϵθ(concat⁡(z~t,η),t~)\hat{\epsilon}_t \leftarrow\epsilon_\theta(\operatorname{concat}(\tilde{z}_t, \eta), \tilde{t})

  7. Recover zt−1z_{t-1} using (2)

  8. end for

  9. Decode {zt}t=0T\{z_t\}_{t=0}^{T} using the VAE

  10. return {xt}t=0T\{x_t\}_{t=0}^{T}

is trained to predict the added noise, we retain ϵ\epsilon-parameterization during fine-tuning. Specifically, we first predict the noise residual ϵ^t\hat{\epsilon}_t and then convert it to the associated clean latent z^t−1\hat{z}_{t-1} using (2). Therefore, our training objective is:

L=Et,zt,zt−1,η[∥z^t−1−zt−1∥1].(4)\mathcal{L} = \mathbb{E}_{t,z_t,z_{t-1},\eta}\left[\left\|\hat{z}_{t-1}-z_{t-1}\right\|_1\right]. \tag*{(4)}

Inference. The model starts from the prototype latent zTz_T and iteratively applies the learned diversification operator. Similar to training, we perturb the latent using the same τ\tau value employed during training. The conditioning latent η\eta controls the direction of diversification. Since η\eta is the initial noise used to generate the target image during dataset construction, we can easily sample a new conditioning latent at inference time from the same Gaussian prior,

η∼N(0,I).\eta\sim\mathcal{N}(0,I).

The number of inference steps controls how far the output moves away from the prototype: fewer steps produce samples closer to the prototype, while more steps produce increasingly diverse outputs. Finally, the resulting latent is decoded back to image space using the LDM VAE decoder. The complete inference procedure is summarized in Algorithm 2.

Experiments

Experiment Setup

Implementation details. We use unconditional Latent Diffusion Models [32] from the official repository as our base models. We set the number of variation steps to T=50T = 50 and the number of variation perturbation to smax⁡=5s_{\max} = 5 for all datasets. Within a 1,000-step diffusion schedule, we use τ=300\tau= 300 for FFHQ, CelebA, and LSUN-Bedrooms, and τ=200\tau= 200 for LSUN-Churches. We fine-tune the models on the generated trajectory datasets using a batch size of 32 and a learning rate of 1×10−51 \times10^{-5} for 20,000 training steps, except for LSUN-Churches, which is trained for 50,000 steps.

Metrics. For quantitative evaluation, we use four metrics: Fréchet Inception Distance (FID) [14] to measure image fidelity, Inception Score (IS) [33] to assess sample diversity, and Precision (P) and Recall (R) [22] to evaluate the coverage of the generated distribution. To evaluate intermediate states, we use ImageReward [40] to measure perceptual image quality and the Vendi Score [11] to measure sample diversity.

Datasets. We conduct most experiments on FFHQ, and additionally evaluate our method on CelebA, LSUN-Churches, and LSUN-Bedrooms. For each dataset, we generate 10,000 images using an unconditional Latent Diffusion Model with DDIM sampling and retain the corresponding initial noisy latent η\eta as the conditioning input to our model. All experiments are performed at a resolution of 256×256256 \times256. Following § 4, we select a prototype image for each dataset and use DiffMorpher [43] to construct a 50-step (T=50T = 50) trajectory that progressively transforms each generated image into the prototype.

Main Results

We compare the trajectories produced by our diversification operator with those of a standard diffusion model (LDM). We visualize trajectories on FFHQ, CelebA, LSUN-Churches, and LSUN-Bedrooms, and evaluate 1,000 FFHQ samples per method using ImageReward and Vendi Score. For the diffusion baselines, we show the intermediate states xtx_t and predicted clean image x^0\hat{x}_0 at each denoising step, using a total of 50 deterministic DDIM steps.

Figure 3 presents the comparison. Our method produces sharp, recognizable images from the earliest inference steps while progressively developing variations in appearance and high-level semantic attributes. For example, in the third FFHQ trajectory, the apparent age of the subject decreases gradually from an adult to a child. In contrast, the diffusion baseline determines this attribute early and primarily refines visual details afterward. More generally, the baseline remains blurry during early denoising stages and becomes visually coherent only toward the end of generation.

Qualitative comparison of generated sequence strips across FFHQ, CelebA, Church, and Bedroom datasets

Figure 3. Qualitative comparison of generated sequence strips across FFHQ, CelebA, Church, and Bedroom datasets. For each dataset grid, the top section shows xtx_t, the middle section shows x^0\hat{x}_0, and the bottom section shows Ours.

Figure 4 illustrates the speed–variation trade-off by plotting image quality against sample diversity across inference steps. Our trajectory closely follows the variation-reducing trajectory in the curated training dataset, denoted by “Dataset,” while maintaining high image quality as diversity increases. In contrast, both the intermediate states xtx_t and predicted clean images x^0\hat{x}_0 of the diffusion baseline begin at lower quality (∙\bullet, ∙\bullet) and improve only during later denoising stages. These results show that our model produces high-quality samples early (∙\bullet) while progressively increasing diversity, enabling a controllable speed–variation trade-off.

Image quality versus diversity across inference steps

Figure 4. Image quality versus diversity across inference steps. Image quality is measured using ImageReward, while diversity is measured using the Vendi Score. Arrows indicate the direction of generation, while circles (•, •, •, •) and squares mark (■, ■, ■, ■) the initial and final steps, respectively. We compare our trajectory with the diffusion baseline’s intermediate states xtx_t and predicted clean images x^0\hat{x}_0. The dashed curve represents the variation-reducing trajectories in the curated training dataset. Our method maintains high image quality as diversity progressively increases.

Importance of noise association η\eta. We show the importance of noise association η\eta in our method by removing η\eta from the input to the model, leaving only the perturbed input image, to construct an unconditional version of our method. Figure 5 shows that without η\eta, the generated samples exhibit lower variation throughout the trajectory and change much less than those with η\eta. This shows that noise association η\eta acts as a guiding direction toward specific semantics, leading the model to generate correct attributes, such as changing the prototype from female to male, while without η\eta, the model struggles to do so.

Figure 5. Comparison of our method with and without noise association η\eta. Noise association η\eta enables greater variation throughout the trajectory, whereas omitting it results in trajectories with limited variation and minimal changes.

Importance of variation perturbation smax⁡s_{\max}. We analyze the effect of the maximum variation perturbation range smax⁡s_{\max} in Table 2. The results show that using no variation perturbation (smax⁡=1s_{\max} = 1) leads to a high FID score, indicating poorer generation quality. As shown in Figure 6, without variation perturbation, the intermediate trajectory generation changes more slowly compared to the ones with support smax⁡=5s_{\max}=5 and ground truth, leading to poorer generation. We find that smax⁡=5s_{\max}=5 performs best in our experiments, achieving the lowest FID and highest IS, while larger values of smax⁡s_{\max} progressively degrade FID performance.

Figure 6. Comparison of our method with (smax⁡=5s_{\max}=5) and without (smax⁡=1s_{\max}=1) variation perturbation smax⁡s_{\max}. Using variation perturbation produces trajectories that more faithfully match the curated ground-truth trajectories.

smax⁡s_{\max}FID ↓\downarrowIS ↑\uparrowP ↑\uparrowR ↑\uparrow
146.1202.7290.8660.098
525.4073.9120.7580.166
1027.8683.5840.7930.144
2030.5013.4880.8010.115
5032.4053.4660.7800.093

Table 2. Quantitative results under different variation perturbation smax⁡s_{\max}.

Importance of noise perturbation τ\tau. We show the impact of different noise perturbation levels τ\tau quantitatively in Table 1 and qualitatively in Figure 7. Without noise perturbation (τ=0\tau=0), the model struggles to learn the trajectory generation, leading to corrupted outputs. Conversely, excessively high noise levels, such as τ=500\tau=500 and τ=800\tau=800, introduce severe artifacts, distortion, and blurriness. We find that τ=300\tau=300 performs best, achieving the lowest FID score.

τ\tauFID ↓\downarrowIS ↑\uparrowP ↑\uparrowR ↑\uparrow
0199.0183.1780.0680.000
10047.6643.1050.7710.116
30025.4073.9120.7580.166
50030.5303.8250.8040.114
80040.5523.4270.8160.060

Table 1. Quantitative results under different numbers of noise perturbation (τ\tau).

Grid of generated face images across time steps and noise perturbations

Figure 7. Effect of noise perturbation τ\tau on image quality. Both insufficient (τ=0\tau= 0) and excessive (τ=500\tau= 500 and τ=800\tau= 800) noise perturbation degrade image quality, resulting in corrupted, distorted, or blurry images. In contrast, a moderate noise perturbation (τ=300\tau= 300) provides the best balance and produces the highest-quality results with lowest FID as shown in Table 1.

Limitations and Discussion

Data quality. The performance of our learned model depends on the quality of the morphing trajectory dataset. In this work, we employ DiffMorpher [43] to generate these trajectories because it produces smooth transitions and generalizes across diverse concepts. However, it does not guarantee that every intermediate state lies on the image manifold. This limitation is particularly evident for datasets with substantial structural and appearance variation, such as LSUN-Churches, where some trajectories contain distorted church structures or unrealistic intermediate images as shown in Figure 8. Moreover, the trajectory quality is also limited by the quality of the pretrained LDM-generated endpoint images, whose artifacts can propagate through the trajectories. However, the morphing algorithm is used only as a preprocessing step, making our framework compatible with future improvements or alternative morphing methods.

Examples of low-quality trajectories from the LSUN-Church dataset

Figure 8. Examples of low-quality trajectories from the LSUN-Church dataset.

Additionally, our prototype selection strategy relies on CLIP to identify a representative image. Future work could explore alternative approaches for obtaining prototypes that better reside on the data manifold, such as average image [10, 38], dataset distillation [5, 13, 37], and selecting representatives from the learned semantic latent space of generative models [18, 19, 20, 29].

Generated image quality. While our model generates smooth and coherent trajectories, our final generated images achieve a higher FID (25.407) than the baseline (5.032), which is initialized from an LDM pretrained on FFHQ and fine-tuned on our trajectory dataset. As shown in Figure 9, our model tends to produce images that contain fewer high-frequency details than those generated by the baseline, contributing to the degradation in FID. Despite this, the generated samples remain visually realistic and qualitatively similar to those of the baseline.

Image quality comparison with the baseline

Figure 9. Image quality comparison with the baseline. Despite achieving a higher FID than the baseline (Lower FID is better), our model generates samples that remain recognizable and visually realistic.

Computation. Due to limited computational resources, we mainly conduct our experiments on the FFHQ dataset. The hyperparameters for the other datasets have not been extensively tuned, and more careful optimization may further improve their performance.

Conclusion

We present Speed-Variation Distillation, a novel generative modeling idea. By leveraging variation-reduction trajectories obtained through image morphing together with our distillation framework, our method learns to transform a single shared prototype into high-quality and diverse images through a sequence of realistic intermediate states. This capability introduces a novel speed-diversity trade-off, enabling faster image generation at the cost of sample diversity.

References

  1. [1]Abu-Hussein, S., Giryes, R.: Udpm: Upsampling diffusion probabilistic models. Advances in Neural Information Processing Systems 37, 27616–27646 (2024)
  2. [2]Bansal, A., Borgnia, E., Chu, H.M., Li, J.S., Kazemi, H., Huang, F., Goldblum, M., Geiping, J., Goldstein, T.: Cold diffusion: Inverting arbitrary image transforms without noise (2023)arxiv.org/abs/2208.09392
  3. [3]Beier, T., Neely, S.: Feature-based image metamorphosis. In: Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 529–536 (2023)
  4. [4]Bhatt, B.G.: Comparative study of triangulation based and feature based image morphing. Signal & Image Processing 2(4), 235 (2011)
  5. [5]Chan Santiago, J.A., Tirupattur, P., Nayak, G.K., Liu, G., Shah, M.: MGD^3: Mode-guided dataset distillation using diffusion models. In: Proceedings of the 42nd International Conference on Machine Learning (ICML) (2025)
  6. [6]Chen, B., Martí Monsó, D., Du, Y., Simchowitz, M., Tedrake, R., Sitzmann, V.: Diffusion for next-token prediction meets full-sequence diffusion. Advances in Neural Information Processing Systems 37, 24081–24125 (2024)arxiv.org/abs/2407.01392
  7. [7]Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: Combining inconsistent images using patch-based synthesis. ACM Transactions on graphics (TOG) 31(4), 1–10 (2012)DOI
  8. [8]Daras, G., Delbracio, M., Talebi, H., Dimakis, A.G., Milanfar, P.: Soft diffusion: Score matching for general corruptions. arXiv preprint arXiv:2209.05442 (2022)arxiv.org/abs/2209.05442
  9. [9]Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34, 8780–8794 (2021)
  10. [10]Feng, C., Chen, Z., Holynski, A., Efros, A.A., Owens, A.: Gps as a control signal for image generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2766–2778 (June 2025)arxiv.org/abs/2501.12390
  11. [11]Friedman, D., Dieng, A.B.: The vendi score: A diversity evaluation metric for machine learning. arXiv preprint arXiv:2210.02410 (2022)
  12. [12]Gruszczynski, G., Meixner, J., Wlodarczyk, M., Musialski, P.: Beyond blur: A fluid perspective on generative diffusion models. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 17818–17827 (2025)DOI
  13. [13]Gu, J., Vahidiani, S., Kungurtsev, V., Wang, H., Jiang, W., You, Y., Chen, Y.: Efficient dataset distillation via minimax diffusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)arxiv.org/abs/2311.15529
  14. [14]Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30 (2017)
  15. [15]Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems 33, 6840–6851 (2020)arxiv.org/abs/2006.11239
  16. [16]Hoogeboom, E., Salimans, T.: Blurring diffusion models. arXiv preprint arXiv:2209.05557 (2022)arxiv.org/abs/2209.05557
  17. [17]Hsueh, H.C., Peng, W.H., Huang, C.C.: Warm diffusion: Recipe for blur-noise mixture diffusion models. In: International Conference on Learning Representations. vol. 2025, pp. 13020–13037 (2025)arxiv.org/abs/2511.16904
  18. [18]Hudson, D.A., Zoran, D., Malinowski, M., Lampinen, A.K., Jaegle, A., McClelland, J.L., Matthey, L., Hill, F., Lerchner, A.: Soda: Bottleneck diffusion models for representation learning. arXiv preprint arXiv:2311.17901 (2023)arxiv.org/abs/2311.17901
  19. [19]Karras, T., Laine, S., Aila, T.: A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 4401–4410 (2019)DOI
  20. [20]Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8110–8119 (2020)arxiv.org/abs/1912.04958
  21. [21]Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)arxiv.org/abs/1312.6114
  22. [22]Kynkäänniemi, T., Karras, T., Laine, S., Lehtinen, J., Aila, T.: Improved precision and recall metric for assessing generative models. Advances in neural information processing systems 32 (2019)arxiv.org/abs/1904.06991
  23. [23]Laskey, M., Lee, J., Fox, R., Dragan, A., Goldberg, K.: Dart: Noise injection for robust imitation learning. In: Conference on robot learning. pp. 143–156. PMLR (2017)arxiv.org/abs/1703.09327
  24. [24]Liao, J., Lima, R.S., Nehab, D., Hoppe, H., Sander, P.V., Yu, J.: Automating image morphing using structural similarity on a halfway domain. ACM Transactions on Graphics (TOG) 33(5), 1–12 (2014)DOI
  25. [25]Lipman, Y., Chen, R.T., Ben-Hamu, H., Nickel, M., Le, M.: Flow matching for generative modeling. In: The eleventh international conference on learning representations (2022)arxiv.org/abs/2210.02747
  26. [26]Liu, X., Zhang, X., Ma, J., Peng, J., et al.: Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In: The Twelfth International Conference on Learning Representations (2023)arxiv.org/abs/2309.06380
  27. [27]Luo, S., Tan, Y., Huang, L., Li, J., Zhao, H.: Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378 (2023)arxiv.org/abs/2310.04378
  28. [28]Mukhopadhyay, S., Udhayanan, P., Shrivastava, A.: Scale space diffusion (2026), https://arxiv.org/abs/2603.08709
  29. [29]Preechakul, K., Chatthee, N., Wizadwongsa, S., Suwajanakorn, S.: Diffusion autoencoders: Toward a meaningful and decodable representation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10619–10629 (2022)arxiv.org/abs/2111.15640
  30. [30]Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)arxiv.org/abs/2103.00020
  31. [31]Rissanen, S., Heinonen, M., Solin, A.: Generative modelling with inverse heat dissipation. arXiv preprint arXiv:2206.13397 (2022)arxiv.org/abs/2206.13397
  32. [32]Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 10684–10695 (2022)arxiv.org/abs/2112.10752
  33. [33]Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. Advances in neural information processing systems 29 (2016)arxiv.org/abs/1606.03498
  34. [34]Shechtman, E., Rav-Acha, A., Irani, M., Seitz, S.: Regenerative morphing. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 615–622. IEEE (2010)DOI
  35. [35]Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020)arxiv.org/abs/2010.02502
  36. [36]Song, Y., Dhariwal, P., Chen, M., Sutskever, I.: Consistency models (2023), https://arxiv.org/abs/2303.01469
  37. [37]Su, D., Hou, J., Gao, W., Tian, Y., Tang, B.: D^4m: Dataset distillation via disentangled diffusion model. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 5809–5818 (June 2024)arxiv.org/abs/2407.15138
  38. [38]Dhatwdramongkit, P., Seripanitkarn, S., Suwajanakorn, S.: Diffusion mental averages. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 35713–35725 (June 2026)
  39. [39]Wang, C., Golland, P.: Interpolating between images with diffusion models (2023)arxiv.org/abs/2307.12560
  40. [40]Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., Dong, Y.: Image-reward: Learning augmented human preferences for text-to-image generation. Advances in Neural Information Processing Systems 36, 15903–15935 (2023)
  41. [41]Yang, Z., Yu, Z., Xu, Z., Singh, J., Zhang, J., Campbell, D., Tu, P., Hartley, R.: Impus: Image morphing with perceptually-uniform sampling using diffusion models. In: International Conference on Learning Representations. vol. 2024, pp. 2059–2086 (2024)arxiv.org/abs/2311.06792
  42. [42]Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W.T., Park, T.: One-step diffusion with distribution matching distillation. In: CVPR (2024)arxiv.org/abs/2311.18828
  43. [43]Zhang, K., Zhou, Y., Xu, X., Dai, B., Pan, X.: Diffmorpher: Unleashing the capability of diffusion models for image morphing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7912–7921 (2024)arxiv.org/abs/2312.07409

Paper details

vlnc:W_9n9hthxn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 6

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents