betaFeedback

WorksComputer Science

Disentangled Embedding Optimization for Diffusion-Based Dataset Distillation

Abstract

Dataset distillation aims to compress large-scale datasets into compact synthetic subsets that enable efficient model training with comparable performance to the original data. Recent advances show that diffusion models are highly effective for this task due to their strong generative capability. Existing diffusion-based approaches typically perform class-wise clustering and use the obtained cluster prototypes as diffusion inputs to generate distilled samples. However, the uniformly distributed original samples in a category inevitably lead to entanglement near cluster boundaries, pulling the cluster prototypes closer and reducing the inter-cluster separation. Consequently, the diffusion model receives overly similar latent inputs, resulting in synthetic data with limited diversity. To address this issue, we propose Disentangled Embedding Optimization (DEO), a simple yet effective framework that replaces static clustering with trainable embeddings. These embeddings are optimized to match the Gaussian statistics (i.e., mean and covariance) of the original data distribution, while a diversity regularizer is designed to encourage separation among these embeddings to improve the intra-class variability. Extensive experiments demonstrate that our proposed DEO can generate more diversified and representative distilled data than existing cluster-based diffusion methods, achieving state-of-the-art performance and superior generalization across multiple datasets.

CCS Concepts

• Computing methodologies → Artificial intelligence.

Keywords

Dataset Distillation, Representative Learning, Semantic Diversity

Introduction

The success of deep learning in vision, language, and multimodal tasks largely stems from the availability of large-scale datasets [13, 17, 22, 25, 36]. However, the growing size of such datasets leads to substantial costs in storage, training time, and computation. To mitigate these issues, dataset distillation has emerged as a promising paradigm that synthesizes compact datasets capable of retaining the training efficacy of the original datasets [19, 34, 40]. Beyond efficiency, distilled datasets also benefit privacy-sensitive [8, 48] and resource-constrained scenarios [12, 37].

Early dataset distillation (DD) methods typically employ a bi-level optimization framework, where the outer loop updates model parameters and the inner loop refines synthetic data by aligning training trajectories [1, 9, 23] or gradients [18, 43, 44]. Despite their effectiveness, the interdependent optimization of models and distilled data incurs prohibitive computational costs, often restricting their application to low-resolution datasets like CIFAR-10/100. To mitigate this, subsequent research has decoupled the process by directly matching feature statistics [33, 39, 45, 46]. However, these methods still rely on pixel-space optimization, which makes scaling to high-resolution datasets inherently difficult [4, 24, 47].

To overcome the limitations of pixel-space optimization, several pioneering studies have incorporated generative priors into dataset distillation. For example, GLAD [2] improves distillation by projecting optimization losses onto the intermediate latent layers of a pre-trained GAN, thereby exploiting the structural and semantic information captured at different depths to regularize synthesized samples toward the natural image manifold. However, GAN-based frameworks still require backpropagation through complex generator architectures, resulting in substantial memory overhead. To alleviate this issue, more recent research has shifted toward diffusion-based distillation. Compared with GANs, diffusion models operate in a highly expressive and semantically linearized latent space induced by a pre-trained VAE. Within this paradigm, the VAE encoder first projects original images into a compact latent manifold, allowing distillation to be performed in a lower-dimensional yet semantically structured space. Specifically, D4M [31] facilitates distillation by using kk-means cluster prototypes as the latent foundation for diffusion. Building upon this paradigm, VLCP [49] integrates textual guidance by extracting LLaVA-generated descriptions for each cluster, leveraging these rich semantic prompts to further steer the synthesis of distilled samples.

However, a critical limitation is overlooked: in the latent space of a pretrained VAE encoder, features within each class tend to be uniformly distributed. Under such conditions, samples lying near the boundary of two clusters are inevitably and often arbitrarily assigned to one cluster or the other, which will reduce the distance between the corresponding cluster prototypes. As illustrated in Figure 1(a), we visualize the latent embeddings of a class, where samples belonging to two clusters are highlighted in orange and green, respectively. It is obvious that the entangled samples tend to pull their corresponding cluster prototypes closer in the feature space, thereby reducing inter-cluster separation. As shown in Figure 1(b), when these less separable cluster prototypes are used as latent inputs to the diffusion model, the generated distilled images appear visually similar and lack diversity. Consequently, the synthesized data fail to capture the full spectrum of intra-class semantic variations, limiting the performance of downstream tasks.

Figure 1. Illustration of the entanglement issue in distilled data generation. (a) t-SNE visualization shows that entangled sub-cluster prototypes in the original data reduce separability. (b) These less-separable latent inputs for the diffusion model result in indistinguishable synthesized images.

To address this issue, we propose Disentangled Embedding Optimization (DEO), a simple yet effective framework for diffusion-based dataset distillation. Instead of performing clustering analysis in the latent space, our method directly optimizes the latent embeddings, which serve as inputs of the diffusion model to generate distilled data. Specifically, we treat the latent embeddings as learnable parameters that are optimized to approximate the original data distribution. For each class, we estimate a Gaussian distribution in the latent space by computing the empirical mean and covariance matrix with the original data. Then, the mean and covariance of the trainable latent embeddings are aligned with those of the corresponding class distributions, ensuring semantic representativeness. To further enhance intra-class diversity of the distilled data, we introduce an explicit dispersion regularization that promotes mutual separation among the trainable latent embeddings. Finally, the optimized embeddings, together with their corresponding class textual prompts, are fed into the diffusion model to generate distilled data. Extensive experiments on CIFAR-10/100, ImageNet-1K, and several ImageNet subsets (ImageWoof, ImageNette, and ImageIDC) demonstrate that our DEO can generate diversified distilled data, which will consistently improve the performance of downstream tasks, all without introducing any external knowledge.

Our main contributions are summarized as follows:

  • We propose a simple yet effective framework, termed Disentangled Embedding Optimization (DEO) for diffusion-based dataset distillation, which makes the input latent space trainable rather than relying on conventional clustering.

  • We design a dispersion regularizer that explicitly encourages diversity among synthetic embeddings, effectively mitigating prototype collapse and promoting richer intra-class representations.

  • Extensive experiments on CIFAR-10/100, ImageNet-1K, and multiple ImageNet subsets show that DEO consistently outperforms prior diffusion-based baselines with strong cross-architecture generalization and lower computational cost.

Related Work

Matching-Based Methods

Early approaches to dataset distillation typically adopt a bi-level optimization framework, where the outer loop updates model parameters and the inner loop optimizes the distilled data to match either gradient signals or training trajectories. Specifically, gradient matching aligns the stepwise gradients computed on the original and distilled data [15, 18, 43, 44], whereas trajectory matching extends this idea by aligning the entire optimization paths [1, 5, 9, 23]. Despite demonstrating strong performance, these methods are computationally expensive due to the repeated alternation between model updates and data optimization.

To improve efficiency, recent methods leverage distribution matching, which aligns feature distributions without updating the model [45]. Early variants typically utilize randomly initialized networks to extract features. For example, CAFE [33] performs multi-layer prototype alignment; DataDAM [28] selects informative regions via attention mechanisms; and M3D [42] conducts feature alignment in a reproducing kernel Hilbert space. Building on this line of work, IDM [46] and DANCE [41] enhance representational capacity by employing multiple pretrained extractors, while IID [7] and DSDM [20] introduce feature moment matching to promote semantic diversity. However, these methods require both the dataset and the model to reside in memory, which limits scalability to large-scale settings.

To address the scalability challenge, decoupled distillation strategies have emerged [30, 32, 38]. A representative example is SRe2L [39], which first trains a reference model to absorb the knowledge from the full dataset, and then optimizes the distilled data by matching per-batch BatchNorm statistics to the model’s global statistics. Additionally, soft labels generated by the reference model provide supervisory signals for distillation. While this strategy substantially reduces computational cost, it may compromise diversity in synthetic samples, limiting generalization to unseen architectures. To mitigate this issue, DWA [10] introduces training perturbations and variance-aware weighting to encourage diverse sample generation. Complementarily, DEDA [21] incorporates a decorrelation regularization term that reduces redundancy among distilled features, thereby further enhancing semantic diversity. Although the decoupled distillation methods are more computationally efficient, they still follow the pixel-level optimization paradigm, where memory usage and computational cost grow greatly with image resolution, ultimately limiting their scalability to high-resolution datasets.

Generative-Based Methods

Recent research has shifted from pixel-space optimization to leveraging generative priors. As a pioneering work, GLAD [2] introduces a deep generative prior from GANs to regularize the distillation process, significantly enhancing cross-architecture generalization. However, GAN-based methods often struggle with high-resolution scaling or necessitate complex, memory-intensive optimization. Building on the success of generative modeling [26], diffusion models have emerged as robust backbones for dataset distillation due to their superior capacity for high-fidelity image synthesis. For instance, Gu et al. [11] enhance the representativeness of synthetic data by fine-tuning diffusion models to align predicted embeddings with those of original images. Although effective, this multi-stage fine-tuning pipeline is computationally expensive, especially for large-scale datasets. To mitigate the overhead, subsequent studies exploit a frozen VAE and perform distillation directly in its latent space. D⁴M [31] clusters latent codes using kk-means and conditions the generation process on cluster prototypes combined with label text prompts. MGD [3] further discovers intra-class patterns through clustering to guide the denoising trajectory of the diffusion model. Inspired by this idea, VLCP [49] enhances textual conditioning by leveraging LLaVA to generate per-image descriptions and selecting the most frequent sentence within each cluster to provide fine-grained textual descriptions for each cluster prototype. In these approaches, clustering is a crucial step for generating distilled data. However, clustering-based distillation often falls into a homogeneity trap. Because cluster prototypes frequently overlap in dense data regions, the resulting synthetic samples become repetitive, sacrificing dataset diversity for mere representative proximity.

Method

Preliminaries

Dataset distillation aims to construct a compact synthetic dataset S={(si,yi)}i=1∣S∣\mathcal{S} = \{(s_i, y_i)\}_{i=1}^{|\mathcal{S}|}, on which models trained can achieve comparable generalization performance to those trained on the full dataset T={(xi,yi)}i=1∣T∣\mathcal{T} = \{(x_i, y_i)\}_{i=1}^{|\mathcal{T}|}, where yi∈Y={0,1,…,C−1}y_i \in\mathcal{Y} = \{0, 1, \ldots, C-1\}. Typically, S\mathcal{S} contains only IPC ×C\times C samples (IPC: images per class), satisfying ∣S∣≪∣T∣|\mathcal{S}| \ll|\mathcal{T}|. The goal is to train θS\theta_{\mathcal{S}} on S\mathcal{S} such that it achieves comparable generalization to θT\theta_{\mathcal{T}} trained on T\mathcal{T}.

Latent Diffusion Model for Dataset Distillation. Recently, latent diffusion models (LDM) have emerged as a promising paradigm for dataset distillation, due to their strong cross-architecture generalization and efficient sample generation [31, 49]. A typical LDM pipeline consists of: (1) a VAE image encoder E\mathcal{E} for mapping input images into the latent space, (2) a conditioning text encoder that encodes class labels or descriptions, (3) a denoising U-Net ϵθ\epsilon_\theta trained to reverse noise corruption, and (4) a VAE decoder D\mathcal{D} that reconstructs images from the denoised latents.

Latent Diffusion Objective. The latent diffusion model is trained to predict additive noise injected into latent embeddings:

LLDM=Ex,y,ϵ,t[∥ϵ−ϵθ(zt,t,τϕ(y))∥22],(1)\mathcal{L}_{\mathrm{LDM}} = \mathbb{E}_{x, y, \epsilon, t}\left[\left\lVert\epsilon- \epsilon_\theta\left(z_t, t, \tau_\phi(y)\right)\right\rVert_2^2\right], \tag*{(1)}

where ztz_t is a noised latent obtained from an original image xx through a VAE encoder and a forward diffusion process, ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0, I) is the ground-truth noise, and t∼U{1,T}t \sim\mathcal{U}\{1, T\} is the diffusion timestep. The term τ(y)\tau(y) denotes the embedding of the class label yy produced by a text encoder τϕ\tau_\phi, which provides semantic conditioning for the denoising network ϵθ\epsilon_\theta. During inference, a scheduler progressively denoises the latent from zTz_T to z0z_0, which is then decoded into an image using the VAE decoder. In the context of diffusion-based dataset distillation, the collection of such generated images constitutes the distilled dataset S\mathcal{S}.

Disentangled Embedding Optimization

Recent diffusion-based dataset distillation methods typically perform clustering on the latent embeddings extracted by a VAE image encoder for each class and use the cluster prototypes as the inputs to the latent diffusion model to generate distilled images. However, since the samples of a class are often uniformly distributed in the latent space, the resulting cluster prototypes tend to lie close to one another. This will lead to the synthetic samples with limited intra-class diversity.

To mitigate this issue, instead of following the clustering-based pipeline, we propose to explicitly optimize the embeddings of distilled data Zc={zi}i=1IPC\mathcal{Z}_c = \{z_i\}_{i=1}^{\mathrm{IPC}}. This design enables the generated samples to preserve the diversity of the original data distribution. As shown in Figure 2(a), we initialize the learnable latent embeddings Zc\mathcal{Z}_c independently and align them with the Gaussian distribution of the original data in the latent space of the pre-trained VAE encoder E\mathcal{E} by minimizing a mean alignment loss Lμ\mathcal{L}_{\mu} and a covariance alignment loss Lcov\mathcal{L}_{\mathrm{cov}}. In addition, to promote sample-level diversity within each class, we introduce a diversity loss Ldiv\mathcal{L}_{\mathrm{div}} that encourages the optimized embeddings to be mutually dispersed in the latent space.

Figure 2. Overview of our Disentangled Embedding Optimization (DEO) for diffusion-based distillation. (a) Learnable latent embeddings Zc\mathcal{Z}_c are optimized to match the mean and covariance of the original data, with a diversity loss to encourage separation. (b) The optimized Zc\mathcal{Z}_c and text prompts are fed into the diffusion model to generate distilled samples S\mathcal{S}.

Specifically, for each class cc, let {ei=E(xi)}i=1Nc\{e_i = \mathcal{E}(x_i)\}_{i=1}^{N_c} denote the latent embeddings extracted from the original samples in class cc, where NcN_c is the number of original samples. We define the mean alignment loss as follows:

Lμ=∥μZ,c−μT,c∥22\mathcal{L}_{\mu} = \left\lVert\mu_{\mathcal{Z},c} - \mu_{\mathcal{T},c} \right\rVert_2^2
s.t.μZ,c=1IPC∑i=1IPCzi,μT,c=1Nc∑i=1Ncei,(2)\text{s.t.}\quad\mu_{\mathcal{Z},c} = \frac{1}{\mathrm{IPC}}\sum_{i=1}^{\mathrm{IPC}} z_i,\quad\mu_{\mathcal{T},c} = \frac{1}{N_c}\sum_{i=1}^{N_c} e_i, \tag*{(2)}

where μZ,c\mu_{\mathcal{Z},c} and μT,c\mu_{\mathcal{T},c} are the mean latent embeddings of the synthetic and original samples in class cc, respectively. Similarly, we define the covariance alignment loss as follows:

LCov=∥CovZ,c−CovT,c∥22\mathcal{L}_{\mathrm{Cov}} = \left\lVert\mathrm{Cov}_{\mathcal{Z},c} - \mathrm{Cov}_{\mathcal{T},c} \right\rVert_2^2
s.t.CovZ,c=1IPC∑i=1IPC(zi−μZ,c)(zi−μZ,c)T,\text{s.t.}\quad\mathrm{Cov}_{\mathcal{Z},c} = \frac{1}{\mathrm{IPC}}\sum_{i=1}^{\mathrm{IPC}}(z_i-\mu_{\mathcal{Z},c})(z_i-\mu_{\mathcal{Z},c})^{\mathsf{T}},
CovT,c=1Nc∑i=1Nc(ei−μT,c)(ei−μT,c)T.(3)\mathrm{Cov}_{\mathcal{T},c} = \frac{1}{N_c}\sum_{i=1}^{N_c}(e_i-\mu_{\mathcal{T},c})(e_i-\mu_{\mathcal{T},c})^{\mathsf{T}}. \tag*{(3)}

To further encourage sample-level diversity within each class, we introduce a separation regularization term that penalizes latent embeddings with high similarity:

Ldiv=1IPC∑i=1IPC∑j=1IPClog⁡(1+exp⁡(z~iTz~j)),(4)\mathcal{L}_{\mathrm{div}} = \frac{1}{\mathrm{IPC}}\sum_{i=1}^{\mathrm{IPC}}\sum_{j=1}^{\mathrm{IPC}}\log\left(1+\exp\left(\tilde{z}_i^{\mathsf{T}}\tilde{z}_j\right)\right), \tag*{(4)}

where z~i\tilde{z}_i denotes the ℓ2\ell_2-normalized version of ziz_i. This term encourages the learnable embeddings to be well separated in the latent space, thereby promoting intra-class diversity in the distilled data.

Overall Objective and Generation Procedure

The training objective combines distribution alignment with diversity regularization:

L=Lμ+λ1LCov+λ2Ldiv,(5)\mathcal{L} = \mathcal{L}_{\mu} + \lambda_1\mathcal{L}_{\mathrm{Cov}} + \lambda_2\mathcal{L}_{\mathrm{div}}, \tag*{(5)}
Algorithm 1: DEO
Input: T\mathcal{T}: Original dataset; CC: Number of classes; IPC: Images per class; TT: Number of optimization steps; η\eta: Learning rate.
1 (a): Disentangled Embedding Optimization
2 for each class c=1c = 1 to CC do
3 Initialize learnable latent embeddings Zc={zi}i=1IPC\mathcal{Z}_c = \{z_i\}_{i=1}^{\mathrm{IPC}}
4 Compute the class-wise feature mean μT,c\mu_{\mathcal{T},c} and covariance CovT,c\mathrm{Cov}_{\mathcal{T},c} from T\mathcal{T}.
5 for iteration t=1t = 1 to TT do
6 Compute the mean μZ,c\mu_{\mathcal{Z},c} and covariance CovZ,c\mathrm{Cov}_{\mathcal{Z},c} of the latent embeddings Zc\mathcal{Z}_c.
7 Compute the mean loss Lμ\mathcal{L}_{\mu} via Eq. (2).
8 Compute the covariance loss LCov\mathcal{L}_{\mathrm{Cov}} via Eq. (3).
9 Compute the diversity regularization loss Ldiv\mathcal{L}_{\mathrm{div}} via Eq. (4).
10 Compute the overall objective L\mathcal{L} via Eq. (5).
11 Update latent embeddings: Zc←Zc−η∇ZcL\mathcal{Z}_c \leftarrow\mathcal{Z}_c - \eta\nabla_{\mathcal{Z}_c}\mathcal{L}.
12 (b): Diffusion-Based Data Generation
13 for each class c=1c = 1 to CC do
14 Combine the optimized latent embeddings Zc\mathcal{Z}_c with the class-specific textual prompt PtextcP_{\mathrm{text}}^c (i.e., “An image of [class]”).
15 Generate synthetic images {si}\{s_i\} according to Eq. (6).
16 Append generated images to the distilled dataset: S←S∪{si}\mathcal{S} \leftarrow\mathcal{S}\cup\{s_i\}.
Output: Distilled synthetic dataset S\mathcal{S}.

Table 1.

where λ1\lambda_1 and λ2\lambda_2 are tunable hyperparameters. After the latent embeddings are optimized, we feed them into the diffusion model following prior work [31, 49]. As illustrated in Figure 2(b), the per-class distilled latent set Zc={zi}i=1IPC\mathcal{Z}_c = \{z_i\}_{i=1}^{\mathrm{IPC}} is paired with a class-specific textual prompt PtextcP_{\mathrm{text}}^c (e.g., “An image of [class”]). The prompt embedding τϕ(Ptextc)\tau_{\phi}(P_{\mathrm{text}}^{c}) conditions the U-Net, and each distilled image sis_i is synthesized as:

si=D(Ut(Concat⁡(z~i,tc,τϕ(Ptextc)))),(6)s_i = \mathcal{D}\left(\mathcal{U}_t\left(\operatorname{Concat}(\tilde{z}_{i,t}^{c},\tau_{\phi}(P_{\mathrm{text}}^{c}))\right)\right), \tag*{(6)}

where z~i,tc\tilde{z}_{i,t}^{c} is the noise-perturbed version of ziz_i at diffusion timestep tt, Ut(⋅)\mathcal{U}_t(\cdot) denotes the timestep-dependent U-Net backbone, D(⋅)\mathcal{D}(\cdot) is the image decoder, and Concat⁡(⋅,⋅)\operatorname{Concat}(\cdot,\cdot) denotes channel-wise concatenation. This formulation enables the optimized latent codes and textual semantics to jointly guide the denoising process to generate the distilled dataset SS. The complete training and generation pipeline is summarized in Algorithm 1.

Discussion

The advantage of DEO over clustering-based latent distillation methods can be understood from the perspective of prototype bias caused by boundary ambiguity. In the latent space of a pretrained VAE, samples within the same class are often uniformly distributed without clear sub-cluster boundaries. As a result, samples near the overlapping region may be arbitrarily assigned to different clusters, which will pull the estimated cluster prototypes near each other.

Consider two ideal semantic modes with centers μ1\mu_1 and μ2\mu_2. Due to boundary ambiguity, the estimated prototypes can be written as

μ^1=(1−α)μ1+αμ2,μ^2=(1−β)μ2+βμ1,(7)\hat{\mu}_1 = (1-\alpha)\mu_1+\alpha\mu_2,\qquad\hat{\mu}_2 = (1-\beta)\mu_2+\beta\mu_1, \tag*{(7)}

where α,β∈[0,0.5)\alpha,\beta\in[0,0.5) quantify cross-mode contamination. Their distance then becomes

∥μ^1−μ^2∥2=(1−α−β)∥μ1−μ2∥2<∥μ1−μ2∥2.(8)\lVert\hat{\mu}_1-\hat{\mu}_2\rVert_2=(1-\alpha-\beta)\lVert\mu_1-\mu_2\rVert_2<\lVert\mu_1-\mu_2\rVert_2. \tag*{(8)}

This simple analysis shows that boundary ambiguity naturally shrinks the separation between cluster prototypes, causing the diffusion model to receive similar latent inputs. In contrast, DEO avoids hard cluster assignment and directly optimizes learnable embeddings under distribution matching and dispersion regularization, thereby preserving richer intra-class semantic variability.

Experiment

Experimental Setup

Datasets. We evaluate our proposed DEO method on a diverse suite of image benchmarks. For low-resolution settings, we utilize CIFAR-10 and CIFAR-100 [16], two widely used datasets containing 60,000 images at 32 × 32 resolution, with 10 and 100 classes respectively. For high-resolution evaluation, we employ ImageNet-1K [6], a large-scale dataset comprising 1.28 million images across 1,000 categories at 224 × 224 resolution, which offers a comprehensive test of generalization under diverse visual concepts. Additionally, we assess performance on several widely adopted subsets of ImageNet-1K, specifically ImageWoof, ImageNette [14], and ImageIDC [49]. ImageWoof focuses on fine-grained recognition among ten visually similar dog breeds, while ImageNette and ImageIDC serve as more general benchmarks with ten coarse-grained categories each.

Implementation Details. We follow the experimental protocols established in prior work [31, 49]. Unless otherwise specified, we fine-tune Stable Diffusion v1.5 on each dataset using paired image-label supervision. The trainable latent embeddings are optimized using the Adam optimizer with a learning rate of 1×10−31 \times10^{-3} for 10,000 steps. To ensure both distribution alignment and sample diversity, we incorporate a covariance matching loss weighted by λ1=0.5\lambda_1=0.5, alongside a diversity-promoting separation regularizer applied to the denoised latents with a coefficient of λ2=0.004\lambda_2=0.004. All experiments are conducted on NVIDIA RTX A6000 or NVIDIA A100 (80GB) GPUs.

Evaluation Protocol. For a fair comparison, we follow the evaluation settings used in VLCP [49]. Two evaluation strategies are employed: (1) Direct Training: a classifier is trained from scratch using the generated images and their labels, with performance evaluated on the standard test split. This setting is used for ImageWoof, ImageNette, and ImageIDC. (2) Soft-Label Supervision: soft labels are obtained by applying the teacher model pre-trained by the original data to the distilled images, which are then used to train a new classifier. This setting is adopted for ImageNet-1K and CIFAR-10/100. All reported Top-1 accuracies are averaged over three random seeds.

Compared Methods. We compare DEO against a comprehensive set of state-of-the-art baselines: (1) Core-set selection methods, including K-Center [29] and Herding [35]; (2) Matching-based distillation methods, such as DM [45], IDC-1 [15], SRe2L [39], and RDED [32]; and (3) Diffusion-based distillation methods, including DiT [27], Minimax [11], D⁴M [31], and VLCP [49].

Main Results

ImageWoof, ImageNette, and ImageIDC. Table 1 reports the results on three benchmarks under a unified ResNetAP-10 evaluation protocol. DEO consistently achieves the best performance across all datasets and all IPC settings. On ImageWoof, DEO reaches 40.9%, 46.7%, and 59.9% at IPC=10, 20, and 50, respectively, outperforming the strongest baseline VLCP by 1.4%, 2.2%, and 2.6%. Similarly, on ImageNette, DEO improves over VLCP by 1.4%, 1.2%, and 1.0%, while on ImageIDC, the gains are 1.6%, 1.3%, and 0.7%. These results show that DEO produces more effective distilled samples across datasets with different visual characteristics and compression ratios.

ImageNet-1K. Table 2 further demonstrates the scalability of DEO on the challenging ImageNet-1K benchmark. DEO achieves 49.2% at IPC=10 and 61.4% at IPC=50, surpassing the strongest baseline VLCP by 2.5% and 0.9%, respectively. The consistent advantage on ImageNet-1K indicates that our embedding optimization strategy remains effective even when the dataset scale and semantic diversity increase substantially.

IPCSRe2LRDEDDiTMinimaxVLCPDEO (Ours)
1021.3±0.642.0±0.139.6±0.444.3±0.546.7±0.449.2±0.8
5046.8±0.256.5±0.152.9±0.658.6±0.360.5±0.261.4±0.3

Table 2. Performance comparison on ImageNet-1K with IPC=10/50.

CIFAR-10/100. We further evaluate DEO on low-resolution benchmarks in Table 3. On CIFAR-10, DEO achieves 41.8% and 64.1% at IPC=10 and 50, improving over VLCP by 2.8% and 0.9%, respectively. On CIFAR-100, DEO reaches 52.4% and 66.8%, exceeding VLCP by 1.8% and 0.7%. Compared with earlier baselines, the gains are even more pronounced; for example, on CIFAR-100 with IPC=10, DEO outperforms SRe2L and RDED by 25.4% and 9.8%, respectively. Overall, these results demonstrate that DEO consistently delivers superior distilled data quality on both large-scale high-resolution benchmarks and standard low-resolution datasets.

DatasetIPCSRe2LRDEDVLCPDEO (Ours)
CIFAR-101029.3±0.537.1±0.339.0±0.741.8±0.6
CIFAR-105045.0±0.762.1±0.163.2±0.364.1±0.7
CIFAR-1001027.0±0.442.6±0.250.6±0.752.4±0.3
CIFAR-1005050.2±0.462.6±0.166.1±0.366.8±0.5

Table 3. Performance comparison on CIFAR-10 and CIFAR-100 with IPC=10/50.

Cross-architecture Evaluation

Table 4 reports the results on ImageWoof using two target architectures, ConvNeXt and ResNet-18. DEO consistently achieves the best performance across all IPC settings on both backbones, demonstrating strong cross-architecture generalization of the distilled data.

BackboneIPCRandomK-CenterHerdingDiTDMIDC-1MinimaxD⁴MVLCPDEO (Ours)
ConvNet1024.3±1.119.4±0.926.7±0.534.2±1.126.9±1.233.3±1.133.3±1.729.4±0.934.8±2.436.2±0.8
ConvNet2029.1±0.721.5±0.829.5±0.336.1±0.829.9±1.035.5±0.837.3±0.134.0±2.337.9±1.939.4±0.3
ConvNet5041.3±0.636.5±1.040.3±0.746.5±0.844.4±1.043.9±1.250.9±0.847.4±0.954.5±0.655.6±0.3
ResNet-181027.7±0.921.1±0.430.2±1.234.7±0.433.4±0.737.3±0.235.7±1.632.3±1.239.9±2.641.5±1.1
ResNet-182029.7±0.523.6±0.332.2±0.640.5±0.529.8±1.738.6±0.241.8±1.938.4±1.144.5±2.045.0±0.8
ResNet-185047.9±1.839.6±1.048.3±1.250.1±0.546.2±0.648.3±0.853.7±0.653.7±2.258.9±1.560.1±0.9

Table 4. Performance comparison of dataset distillation methods on ImageWoof under varying IPC values. Evaluations are conducted using ConvNet and ResNet-18. The best results are shown in bold.

MethodImageWoof IPC=10ImageWoof IPC=20ImageWoof IPC=50ImageNette IPC=10ImageNette IPC=20ImageNette IPC=50ImageIDC IPC=10ImageIDC IPC=20ImageIDC IPC=50
Random29.4±0.832.7±0.447.2±1.354.2±1.663.5±0.576.1±1.148.1±0.852.5±0.968.1±0.7
K-Center22.1±0.125.1±0.740.6±0.4––––––
Herding32.0±0.334.9±0.149.1±0.7––––––
DiT34.7±0.541.1±0.849.3±0.259.1±0.764.8±1.273.3±0.954.1±0.458.9±0.264.3±0.6
DM30.3±1.235.2±0.647.1±1.160.8±0.666.5±1.176.2±0.452.8±0.558.5±0.469.1±0.8
IDC-139.1±0.543.4±0.348.3±1.0––––––
Minimax36.2±3.243.3±2.753.9±0.757.7±1.264.7±0.873.9±0.351.9±1.459.1±3.769.4±1.4
D⁴M33.2±2.140.1±1.651.7±3.260.9±1.766.3±1.377.7±1.150.3±1.055.8±0.269.1±2.4
VLCP39.5±1.544.5±2.257.3±0.564.8±3.671.4±0.581.2±0.857.0±1.463.3±1.271.9±0.4
DEO (Ours)40.9±0.546.7±0.459.9±0.666.2±1.272.6±0.482.2±0.458.6±0.764.6±0.672.6±0.3

Table 1 (PDF p. 6). Performance comparison on ImageWoof, ImageNette, and ImageIDC with ResNetAP-10 under different IPC settings. The best and second-best results are highlighted in bold and underline, respectively; “–” denotes unavailable results.

Specifically, on ConvNet, DEO improves over the strongest baseline VLCP by 1.4%, 1.5%, and 1.1% at IPC=10, 20, and 50, respectively. On ResNet-18, the corresponding gains are 1.6%, 0.5%, and 1.2%. Moreover, compared with the most relevant diffusion-based baseline D⁴M, DEO achieves substantially larger gains, ranging from 5.4% to 9.2% across different settings. These results indicate that DEO learns more transferable and discriminative latent representations, leading to superior performance across different model families rather than on a single evaluation architecture only.

Analysis

Ablation Study. To evaluate the effectiveness of each component in our DEO, we conduct ablation studies on ImageWoof under three IPC settings: IPC = 10, 20, and 50. We analyze the impact of three core losses: mean alignment (Lμ\mathcal{L}_{\mu}), covariance alignment (LCov\mathcal{L}_{Cov}), and diversity regularization (Ldiv\mathcal{L}_{div}). As summarized in Table 5, we can draw four key conclusions. First, using only Lμ\mathcal{L}_{\mu} already outperforms D⁴M by 1.1% and 0.5%, demonstrating that aligning the class-wise means of latent embeddings effectively preserves the core semantics of the original data. Second, adding LCov\mathcal{L}_{Cov} further captures intra-class variations around the class centers and brings the performance up to par with VLCP. This indicates that matching both the mean and covariance statistics is sufficient to reconstruct rich semantic information without relying on large pretrained models or long-horizon reasoning. Third, incorporating Ldiv\mathcal{L}_{div} alongside Lμ\mathcal{L}_{\mu} yields an additional gain of 1.9%–2.5%, suggesting that enforcing separation among embeddings promotes greater semantic diversity and expressiveness. Finally, the combination of all three losses consistently achieves the best performance across all IPC settings, validating the effectiveness of jointly optimizing for semantic alignment and intra-class diversity in the latent embedding space.

MethodLμL_{\mu}LCovL_{\mathrm{Cov}}LdivL_{\mathrm{div}}IPC=10IPC=20IPC=50
D⁴M---33.240.151.7
Ours✓--34.339.652.2
Ours✓✓-39.844.357.8
Ours✓-✓36.242.154.1
Ours✓✓✓40.946.759.9

Table 5. Ablation study on ImageWoof.

Effect of High IPC. We further analyze the effect of high IPC settings in Figure 3 by comparing D⁴M and DEO on ImageWoof. As IPC increases, both methods achieve better accuracy, showing

Figure 3. Performance of D⁴M and DEO under different high IPC settings on ImageWoof.

that a larger distilled budget generally improves downstream performance. DEO consistently surpasses D⁴M across all IPC settings, indicating its stronger ability to preserve informative class semantics. At the same time, the marginal gain of DEO gradually decreases as IPC grows, suggesting a clear diminishing-return effect. This implies that once sufficient semantic coverage is achieved, further increasing IPC brings only limited additional improvement.

Effect of λ1\lambda_1. In our DEO framework, the hyperparameter λ1\lambda_1 controls the weight of the covariance alignment loss LCov\mathcal{L}_{Cov}, which is designed to match the second-order statistics of the learnable embeddings to those of the original data. Covariance reflects both intra-class dispersion and inter-dimensional correlations, thereby capturing semantic variations around the class center. As shown in the left panel of Figure 4, on ImageWoof with IPC ∈{10,20,50}\in\{10, 20, 50\}, increasing λ1\lambda_1 from 0 to 0.5 consistently improves classification performance, demonstrating the effectiveness of incorporating higher-order statistical alignment. However, when λ1\lambda_1 exceeds 0.5, the performance drops noticeably. We attribute this decline to the trade-off between mean alignment and covariance alignment: an overly large λ1\lambda_1 may overemphasize noisy covariance estimates, especially in low-IPC settings, and drive the embeddings to fit outlier samples. Unless otherwise specified, we set λ1=0.5\lambda_1 = 0.5 in all experiments.

Line charts showing the effect of $\lambda_1$ and $\lambda_2$

Figure 4. Effect of λ1\lambda_1 and λ2\lambda_2 on ImageWoof with IPC ∈{10,20,50}\in\{10, 20, 50\}. Red, blue, and green lines represent IPC = 10, 20, and 50, respectively, while dashed lines denote the corresponding results from D⁴M.

Effect of λ2\lambda_2. The hyperparameter λ2\lambda_2 controls the strength of the diversity regularization, which encourages the learned embeddings to be more dispersed in the latent space. Such dispersion helps the diffusion model generate synthetic samples that are both more diverse and more representative. As shown in the right panel of Figure 4, we conduct a sensitivity analysis on ImageWoof with IPC ∈{10,20,50}\in\{10, 20, 50\}. The results indicate that performance reaches its peak at λ2=0.004\lambda_2 = 0.004. When λ2\lambda_2 becomes excessively large, however, the embeddings become over-dispersed, weakening intra-class compactness and leading to a gradual performance drop.

Diversity Analysis. The average cosine distance between samples and their class prototypes serves as a principled metric for intra-class diversity, which is fundamentally linked to the generalization capability of downstream models. To validate this, we report the average cosine distance for the first ten classes of CIFAR-100

and ImageNet-1K in Figure 5. As illustrated, our DEO consistently outperforms the D⁴M baseline across both datasets. Specifically, on CIFAR-100, DEO achieves an average cosine distance of 0.143, marking a 9.2% relative improvement over D⁴M (0.131). A similar trend is observed on ImageNet-1K, where DEO elevates the metric from 0.121 to 0.132 (a 9.1% gain). These quantitative results demonstrate that DEO more effectively captures intrinsic semantic

Figure 5. Comparison of intra-class diversity. Average cosine distance between distilled samples and class prototypes on CIFAR-100 and ImageNet-1K.

variability, thereby enriching the diversity of the distilled dataset and ultimately boosting downstream performance.

Computational Efficiency. We benchmark DEO against existing methods on a single NVIDIA A100 (80GB) GPU. As shown in Table 6, DEO substantially reduces the computational overhead of the distillation pipeline. On CIFAR-100, it decreases the per-class processing time from 9.5 to 7.4 minutes. The advantage is more pronounced on the large-scale ImageNet-1K dataset, where VLCP requires 25.2 minutes per class, mainly due to the heavy cost of LLaVA inference (21.7 minutes) and latent-space clustering (3.5 minutes). By removing these expensive stages and relying instead on efficient embedding-statistics computation with lightweight optimization, DEO reduces the total time to 13.4 minutes per class, yielding a 47% reduction in computational cost while also improving Top-1 accuracy. These results demonstrate that DEO achieves better performance with significantly lower resource demands.

Stage (min)CIFAR-100 VLCPCIFAR-100 DEO (Ours)ImageNet-1K VLCPImageNet-1K DEO (Ours)
Clustering1.3–3.5–
LLaVA Inf.8.2–21.7–
Emb. Stats–3.6–9.6
Optim.–3.8–3.8
Total Time9.57.425.213.4

Table 6. Per-class computational cost (minutes) at IPC=10 on CIFAR-100 and ImageNet-1K.

Visualization

Latent Space Visualization. To qualitatively demonstrate the effectiveness of our DEO in capturing the intrinsic semantic structure of original data, we visualize the distilled embeddings of the same class on ImageWoof using t-SNE under IPC = 10 and 50, comparing D⁴M with DEO as shown in Figure 6. In these plots, blue dots represent original samples, while red stars indicate distilled ones. For D⁴M, the clustering-based approach results in distilled points that are relatively close to each other, with two points nearly overlapping under IPC=10, and even more clustered under IPC=50, suggesting limited diversity. In contrast, DEO produces embeddings that better align with the original data distribution and maintain a higher degree of dispersion in the latent space. This suggests that optimizing learnable embeddings enables the diffusion model to capture and reproduce meaningful semantic structures from the dataset, rather than memorizing superficial patterns.

t-SNE visualization of latent distributions for original and distilled images

Figure 6. t-SNE visualization of latent distributions for original and distilled images. Results are shown for ImageWoof under IPC 10 and 50. Blue dots indicate original data; red stars denote distilled data from D⁴M and DEO.

Distilled Images. We further compare the visual quality and semantic fidelity of the distilled data generated by D⁴M and our DEO on CIFAR-100 and ImageWoof in Figure 7. Two major observations can be drawn: (1) On the low-resolution CIFAR-100, our DEO produces distilled samples that more faithfully retain the semantic characteristics of the original data. For example, in the bear and baby classes, DEO generates samples with diverse poses and expressions, whereas those from D⁴M tend to look very similar. (2) On the high-resolution ImageWoof, our DEO not only maintains clear class discriminability but also captures richer semantic diversity. For instance, in the Shih-Tzu class, D⁴M frequently generates seated samples with similar hair color and style, whereas DEO produces samples in both seated and lying poses with diverse hair colors (e.g., brown and black), better capturing the true intra-class variability. These results show that DEO more effectively captures semantic structure, yielding distilled samples with higher realism and diversity.

Figure 7. Visualization of distilled data. We compare original samples with distilled data generated by D⁴M and DEO on (a) CIFAR-100 (baby, bear) and (b) ImageWoof (Shih-Tzu, Beagle).

Conclusion

In this paper, we introduce Disentangled Embedding Optimization (DEO), a novel clustering-free framework for diffusion-based dataset distillation. Instead of relying on latent clustering, DEO directly optimizes the learnable embeddings, which are utilized to generate distilled data, by matching the Gaussian statistics (class-level means and covariances) of the original data distribution. To further enhance the diversity among embeddings, a separation regularization term is incorporated to encourage inter-embedding decorrelation. Extensive experiments across multiple benchmarks demonstrate that the proposed DEO consistently outperforms prior methods. It generates more diversified and informative distilled data, demonstrates strong generalization across different model architectures, and maintains lower computational overhead.

Acknowledgments

This work was supported by Strategic Priority Research Program of the Chinese Academy of Sciences under grant No.XDB0690302.

References

References

  1. [1]George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A. Efros, and Jun-Yan Zhu. 2022. Dataset Distillation by Matching Training Trajectories. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10718–10727.
  2. [2]George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei A. Efros, and Jun-Yan Zhu. 2023. Generalizing dataset distillation via deep generative prior. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3739–3748.arxiv.org/abs/2305.01649
  3. [3]Jeffrey A. Chan, Santiago, Praveen Tirumatur, Gaurav Kumar Nayak, Gaowen Liu, and Mubarak Shah. 2025. MGD³: Mode-Guided Dataset Distillation using Diffusion Models. In Proceedings of the 42nd International Conference on Machine Learning (ICML).arxiv.org/abs/2505.18963
  4. [4]Mingyang Chen, Jiawei Du, Bo Huang, Yi Wang, Xiaobo Zhang, and Wei Wang. 2025. Influence-Guided Diffusion for Dataset Distillation. In Proceedings of the International Conference on Learning Representations (ICLR).
  5. [5]Justin Cui, Ruchuan Wang, Si Si, and Cho-Jui Hsieh. 2023. Scaling up dataset distillation to imagenet-1k with constant memory. In International Conference on Machine Learning. PMLR, 6565–6590.arxiv.org/abs/2211.10586
  6. [6]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. IEEE, 248–255.
  7. [7]Wenxiao Deng, Wenbin Li, Tianyu Ding, Lei Wang, Hongguang Zhang, Kuihua Huang, Jing Huo, and Yang Gao. 2024. Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 17057–17066.
  8. [8]Tian Dong, Bo Zhao, and Lingjuan Lyu. 2022. Privacy for Free: How does Dataset Condensation Help Privacy? In Proceedings of the International Conference on Machine Learning (ICML). 5378–5396.arxiv.org/abs/2206.00240
  9. [9]Jiawei Du, Yidi Jiang, Vincent YF Tan, Joey Tianyi Zhou, and Haizhou Li. 2023. Minimizing the accumulated trajectory error to improve dataset distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3749–3758.arxiv.org/abs/2211.11004
  10. [10]Jiawei Du, Xin Zhang, Juncheng Hu, Wenxin Huang, and Joey Tianyi Zhou. 2024. Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment. In Adv. Neural Inf. Process. Syst. (NeurIPS).arxiv.org/abs/2409.17612
  11. [11]Jianyong Gu, Saeed Vahidian, Vyacheslav Kungurtsev, Haonan Wang, Wei Jiang, Yang You, and Yiran Chen. 2024. Efficient dataset distillation via minimax diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 15793–15803.arxiv.org/abs/2311.15529
  12. [12]Jianyong Gu, Kai Wang, Wei Jiang, and Yang You. 2024. Summarizing Stream Data for Memory-Restricted Online Continual Learning. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI).arxiv.org/abs/2305.16645
  13. [14]Jeremy Howard. 2019. Imagenette: A smaller subset of 10 easily classified classes from ImageNet (and a little more French). https://github.com/fastai/imagenette. GitHub repository.
  14. [15]Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joon-Hyun Jeong, Jung-Woo Ha, and Hyun Oh Song. 2022. Dataset condensation via efficient synthetic-data parameterization. In International Conference on Machine Learning. PMLR, 11102–11118.arxiv.org/abs/2205.14959
  15. [16]Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009).
  16. [18]Saehyung Lee, Sanghyuk Chun, Sangwon Jung, Sangdoo Yun, and Sungroh Yoon. 2022. Dataset condensation with contrastive signals. In International Conference on Machine Learning. PMLR, 12352–12364.
  17. [19]Shiye Lei and Dacheng Tao. 2023. A Comprehensive Survey to Dataset Distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 1 (2023), 17–32.DOI
  18. [20]Hongcheng Li, Yucan Zhou, Xiaoyan Gu, Bo Li, and Weiping Wang. 2024. Diversified Semantic Distribution Matching for Dataset Distillation. In Proceedings of the ACM International Conference on Multimedia (MM).
  19. [21]Hongcheng Li, Yucan Zhou, Xiaoyan Gu, Bo Li, and Weiping Wang. 2025. Diversity-Enhanced Distribution Alignment for Dataset Distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).DOI
  20. [23]Dai Liu, Jindong Gu, Hu Cao, Carsten Trinitis, and Martin Schulz. 2024. Dataset Distillation by Automatic Training Trajectories. In Proceedings of the European Conference on Computer Vision (ECCV).arxiv.org/abs/2407.14245
  21. [24]Ping Liu and Jiawei Du. 2025. The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions. arXiv preprint arXiv:2502.05673 (2025).arxiv.org/abs/2502.05673
  22. [26]Yufan Liu, Wanqian Zhang, Huashan Chen, Lin Wang, Xiaojun Jia, Zheng Lin, and Weiping Wang. 2025. AutoPrompt: Automated Red-Teaming of Text-to-Image Models via LLM-Driven Adversarial Prompts. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 17557–17566.
  23. [27]William Peebles and Saining Xie. 2023. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision. 4195–4205.arxiv.org/abs/2212.09748
  24. [28]Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Z Liu, Yuri A Lawryshyn, and Konstantinos N Plataniotis. 2023. Datadream: Efficient dataset distillation with attention matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 17097–17107.arxiv.org/abs/2310.00093
  25. [29]Ozan Sener and Silvio Savarese. 2018. Active Learning for Convolutional Neural Networks: A Core-Set Approach. In International Conference on Learning Representations.
  26. [30]Shitong Shao, Zeyuan Yin, Muxin Zhou, Xindong Zhang, and Zhiqiang Shen. 2024. Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 16709–16718.arxiv.org/abs/2311.17950
  27. [31]Duo Su, Junjie Hou, Weizhi Gao, Yingjie Tian, and Bowen Tang. 2024. D^4M: Dataset Distillation via Disentangled Diffusion Model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5809–5818.arxiv.org/abs/2407.15138
  28. [32]Peng Sun, Bei Shi, Daiwei Yu, and Tao Lin. 2024. On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2312.03526
  29. [33]Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. 2022. CAFE: Learning to Condense Dataset by Aligning Features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 12196–12205.arxiv.org/abs/2203.01531
  30. [34]Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. 2018. Dataset distillation. arXiv preprint arXiv:1811.10959 (2018).arxiv.org/abs/1811.10959
  31. [35]Max Welling. 2009. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference on Machine Learning (Montreal, Quebec, Canada) (ICML ’09). Association for Computing Machinery, New York, NY, USA, 1121–1128. doi:10.1145/1553374.1553517DOI
  32. [37]Enneng Yang, Li Shen, Zhenyi Wang, Tongliang Liu, and Guibing Guo. 2023. An Efficient Dataset Condensation Plugin and Its Application to Continual Learning. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS).DOI
  33. [38]Zeyuan Yin and Zhiqiang Shen. 2024. Dataset Distillation via Curriculum Data Synthesis in Large Data Era. Transactions on Machine Learning Research (2024).arxiv.org/abs/2311.18838
  34. [39]Zeyuan Yin, Eric Xing, and Zhiqiang Shen. 2023. Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective. In Thirty-seventh Conference on Neural Information Processing Systems.arxiv.org/abs/2306.13092
  35. [40]Ruonan Yu, Songhua Liu, and Xinchao Wang. 2023. A Comprehensive Survey to Dataset Distillation. IEEE Transactions on Pattern Analysis and Machine Intelligence 46, 1 (2023), 150–170.
  36. [41]Hansong Zhang, Shikun Li, Fanzhao Lin, Weiping Wang, Zhenxing Qian, and Shiming Ge. 2024. DANCE: Dual-View Distribution Alignment for Dataset Condensation. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI).arxiv.org/abs/2406.01063
  37. [42]Hansong Zhang, Shikun Li, Pengju Wang, Dan Zeng, and Shiming Ge. 2024. M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy. In The 38th Annual AAAI Conference on Artificial Intelligence (AAAI).arxiv.org/abs/2312.15927
  38. [43]Bo Zhao and Hakan Bilen. 2021. Dataset condensation with Differentiable Siamese Augmentation. In Proceedings of the International Conference on Machine Learning (ICML). 12674–12685.arxiv.org/abs/2102.08259
  39. [44]Bo Zhao and Hakan Bilen. 2021. Dataset Condensation with Gradient Matching. In Proceedings of the International Conference on Learning Representations (ICLR).
  40. [45]Bo Zhao and Hakan Bilen. 2023. Dataset Condensation with Distribution Matching. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). 6514–6523.arxiv.org/abs/2110.04181
  41. [46]Ganlong Zhao, Guanbin Li, Lipeng Qin, and Yizhou Yu. 2023. Improved distribution matching for dataset condensation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7856–7865.arxiv.org/abs/2307.09742
  42. [47]Lin Zhao, Yushu Wu, Xinru Jiang, Jianyong Gu, Yanzhi Wang, Xiaolin Xu, Pu Zhao, and Xue Lin. 2025. Taming Diffusion for Dataset Distillation with High Representativeness. In Proceedings of the International Conference on Machine Learning (ICML).arxiv.org/abs/2505.18399
  43. [48]Runkai Zheng, Vishnu Asutosh Dasu, Yining Olivier Wang, Haohan Wang, and Fernando De la Torre. 2025. Improving Noise Efficiency in Privacy-preserving Dataset Distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2508.01749
  44. [49]Yawen Zou, Guang Li, Duo Su, Zi Wang, Jun Yu, and Chao Zhang. 2025. Dataset Distillation via Vision-Language Category Prototype. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).arxiv.org/abs/2506.23580

Paper details

vlnc:W_9sbsj4bj
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 7

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents