WorksComputer Science
Guided Diffusion for Trustworthy Synthetic Text Generation
Abstract
The rapid evolution of Large Language Models is fueled by massive datasets, yet this reliance poses severe privacy risks, as models can inadvertently memorize and even directly output sensitive training data. Synthetic text generation offers a promising solution to this challenge; however, existing methods struggle to reconcile rigorous privacy guarantees with high data utility. In this paper, we propose a new paradigm for private synthetic text generation: Guided Diffusion. Leveraging the unique steerability of Diffusion Language Models, we introduce novel sampling and re-masking procedures that actively guide the iterative denoising steps. These mechanisms steer the generative process to reconstruct fluent text that adheres to a target semantic distribution. We demonstrate that our framework is naturally compatible with Differential Privacy (DP): once the semantic embeddings of the private corpus are protected, the DP guarantee carries over to the generated text by post-processing. Empirically, we show that our approach generates high-quality synthetic text with precise semantic alignment. Furthermore, we validate its effectiveness in DP synthetic text generation: across two datasets and two diffusion backbones, it outperforms Private Evolution and DP-SGD-based fine-tuning in distributional fidelity and downstream utility, while generating significantly faster than Private Evolution. We further extend the framework to Data-Specific Indistinguishability (DSI), which adds a formal certificate against data poisoning and memorization at a small utility cost.
Introduction
The rapid evolution of Large Language Models (LLMs) has fundamentally reshaped Natural Language Processing, a paradigm shift driven primarily by the massive scale of training data. However, this reliance on web-scale data accumulation comes with a steep risk of exposing sensitive information. Real-world datasets, ranging from private medical records and financial reports to personal communication logs, are invaluable for training robust models, but they often contain sensitive personal information that must not be exposed. Recent studies have demonstrated that generative models are prone to memorization ([6]), where they can inadvertently regenerate training samples verbatim. This vulnerability poses severe ethical concerns and violates strict privacy regulations such as GDPR ([40]) and CCPA ([5]).
To address this, synthetic text generation has emerged as a promising solution. Ideally, a synthetic dataset mirrors the statistical properties and semantic richness of the real distribution while ensuring that no individual data point is leaked. However, existing approaches typically struggle to balance these objectives, falling into two broad categories. The first relies on heuristic sanitization, such as entity masking or scrubbing ([24]). While these methods preserve linguistic fluency, they lack formal guarantees and have been repeatedly shown to fail against sophisticated reconstruction attacks ([6, 3, 58]). The second category employs rigorous formalisms like Differential Privacy (DP), often applied via DP-SGD ([1]). While provably privacy-preserving, these methods frequently result in a catastrophic loss of utility when applied to text, yielding outputs that are repetitive or incoherent. One fundamental limitation behind such heavy utility compromise is that provable privacy guarantees such as DP rely on noise to ensure statistical closeness. For example, DP requires statistical indistinguishability between the leakage produced by any two adjacent datasets for individual privacy. To support this, existing noise mechanisms require calibration to the absolute divergence or distance. However, text representations in vocabulary space have extremely high variance. For instance, the sentences “The movie is excellent” and “This is a superb film” are semantically identical yet possess disjoint vocabulary representations (orthogonal one-hot vectors). Standard distance metrics in this space fail to capture semantic proximity, and the required noise scale to mask these differences is often prohibitively large, destroying the linguistic structure of the output.
In this paper, we propose a novel framework for synthetic text generation that seamlessly integrates with off-the-shelf DP clustering algorithms to provide rigorous privacy guarantees. Our core insight is that, while raw text resides in a high-dimensional, discrete space, its semantic embedding can be captured on a low-dimensional, continuous manifold. We observe that DP mechanisms can achieve significantly higher utility if applied within this compact embedding space instead of the sparse vocabulary space. The challenge then shifts to a generation problem:
How to convert privatized, low-dimensional semantic embeddings into coherent, discrete text?
While latent-conditioned generation is well-established in computer vision (e.g., image editing ([32] (Ye et al. 2023)), transferring this success to text is challenging due to the discrete nature of language. Auto-regressive (AR) LLMs, which generate tokens sequentially, are notoriously difficult to steer globally using a fixed embedding vector. To bridge this gap, we leverage the unique properties of Diffusion Language Models (DLMs; [19, 33, 28, 49, 37, 44]). Unlike AR models, DLMs predict blocks of tokens (sentences) through an iterative denoising process. This paradigm allows us to effectively steer the generation by minimizing the distance between the generated sequence and the target semantic embedding at each diffusion step.
Specifically, we introduce a Guided Diffusion framework that generates coherent text conditioned on semantic embeddings. To enable precise control, we propose two novel components for masked DLMs: semantic-aware sampling and semantic-aware re-masking. They allow the DLM to align its discrete outputs with the continuous embedding guidance signal. By integrating DP mechanisms directly into the embedding stage, we present the first framework capable of utilizing DLMs for synthetic data generation with rigorous privacy protection. Beyond DP, the same framework accommodates Data-Specific Indistinguishability (DSI) ([41]), which targets risks such as data poisoning and unintended memorization (§ 5). In summary, we make the following contributions:
Guided Diffusion Framework: We propose a novel framework that, for the first time, exploits the steering capabilities of DLMs to generate coherent text conditioned on semantic embeddings.
DP and DSI Integration: Our Guided Diffusion framework naturally accommodates off-the-shelf DP clustering mechanisms in the embedding space, and the DP guarantee carries over to the synthetic text by post-processing. The same framework also supports DSI aggregation, which provides formal guarantees against corrupted data sources and memorization (§ 5).
Empirical Effectiveness: We validate our framework through extensive experiments, demonstrating superior synthetic data quality and strong downstream utility in DP synthetic text generation: across two datasets and two diffusion backbones, it outperforms Private Evolution and DP-SGD-based baselines in distributional fidelity and downstream utility, while generating substantially faster than Private Evolution. We will release our code publicly.
Preliminaries
Diffusion Language Models. Diffusion Language Models (DLMs) are non-autoregressive generative models that synthesize text through an iterative denoising process. Unlike AR LLMs that generate tokens sequentially from left to right, DLMs generate a sequence of tokens in parallel by progressively refining a noisy input state.
We focus on the denoising process, illustrated in Figure 1. Let denote the token sequence at step , where goes from down to 0. The process begins with , a fully masked sequence consisting entirely of the special token [MASK]. The core component is a mask predictor model , which takes a partially masked sequence as input and outputs a probability matrix for positions over the vocabulary . The transition from step to involves two critical operations:

Figure 1. High-level denoising pipeline for DLMs. At each step , the model predicts token probabilities for the masked input . A candidate sequence is sampled, and low-confidence tokens are re-masked to produce the refined state .
Sampling. Given the current state (e.g., “This paper [MASK] a [MASK] approach...”), the model predicts the probability distribution for every token position. From this probability matrix , we independently sample a candidate token for each masked position to form a fully populated candidate sequence , as an intermediate state (see Figure 1).
Re-Masking. Since early predictions may be inaccurate, we do not accept all sampled tokens immediately. We obtain a confidence score (probability) for each token in . As shown in Figure 1, tokens with high confidence (depicted in green) are retained, while tokens with low confidence (depicted in red) are reverted to the [MASK] token. This produces the input state for the next iteration. This cycle repeats until and all masks are removed, resulting in the final coherent sentence.
Differential Privacy. Differential Privacy (DP; [10]) bounds the influence of any single record on a randomized algorithm’s output. Throughout, two datasets are neighboring if one is obtained from the other by adding or removing a single record (add/remove adjacency). We present the formal definition for -DP.
Definition 2.1 (-DP). A randomized algorithm satisfies -DP if for all neighboring datasets and all measurable sets ,
Smaller and correspond to stronger privacy guarantees: an adversary distinguishing two neighboring datasets must accept a higher Type-I (or Type-II) error rate at any fixed Type-II (or Type-I) budget. Our framework relies on three standard properties.
The Gaussian mechanism ([2]) on a vector-valued query with outputs in and sensitivity achieves -DP with
where is the standard normal cumulative distribution function.
Post-processing invariance: any function applied to the output of an -DP algorithm that does not access the private input incurs no additional privacy loss. In other words, once the noisy embeddings are released under DP, any text generated from them alone, without touching the private data again, inherits the same DP guarantee.
Parallel composition: if a dataset is partitioned into disjoint subsets by a public rule and an -DP algorithm is run on each subset, the joint release is -DP under add/remove adjacency, since a single record affects only one subset. We use this to run DP clustering separately within each label class. Under replace-one adjacency, where a record may change its label and thereby move between two subsets, the same joint release is -DP.
Guided Diffusion Language Models
In this section, we present our framework of Guided Diffusion. The primary objective is to steer the generative process such that the resulting text achieves high similarity to a target semantic embedding , while preserving linguistic fluency. To achieve this, we introduce two novel mechanisms – Semantic-Aware Sampling and Semantic-Aware Re-Masking – and integrate them directly into the iterative denoising pipeline of standard DLMs.
Let denote the diffusion mask predictor. At each diffusion step , the model receives a partially masked sequence and outputs a probability matrix , where each row represents the probability distribution over the vocabulary for the corresponding token position. To output a coherent token block of length , DLMs typically employ sampling and re-masking steps and iteratively remove the noise, which is the [MASK] token, in the input. Now we introduce our Semantic-Aware Sampling and Semantic-Aware Re-Masking designs.
Semantic-Aware Sampling
Standard DLMs employ independent token sampling, where a candidate sequence is formed by sampling from the probability distribution at each token position independently. While efficient, this approach is agnostic to the global semantic trajectory required for our specific guidance task.
To enforce semantic alignment, we propose a two-stage reranking procedure that selects among sampled candidates by balancing exploration, fluency, and semantic distance. We present the algorithm in Algorithm 1. Specifically, at diffusion step , given the probability matrix , we first generate a pool of candidate sequences via independent sampling: .
We enforce that all candidate sequences differ by at least one token. We then evaluate these candidates using a hybrid scoring function to filter for quality and diversity:
where denotes the log-probability of the sequence under the model , and measures the Hamming distance between and the highest-likelihood sequence from . and are scalar weights for the two scoring terms; the values used throughout our experiments are reported in Appendix D, and Appendix E shows that generation quality is insensitive to the ratio over the tested range. We sort the candidates by and retain the top sequences, denoted as .
| Algorithm 1 | Semantic-Aware Sampling |
| Require: | Prob. matrix , target embedding , constants , , , , embedding model |
| 1: | Generate pool via |
| 2: | Compute , |
| 3: | Filter to top candidates based on |
| 4: | Compute for filtered candidates |
| 5: | Select , |
| 6: | Return |
Table 1.
Next, we select the candidate that best aligns with the target semantics. For each retained candidate , we compute its semantic embedding , where is the pre-trained semantic encoder. We evaluate the alignment with the target embedding using the Euclidean distance . The optimal candidate is selected as the minimizer of this distance:
This selected sequence serves as the base estimation for the current step, effectively steering the generation manifold toward the target semantics before the re-masking phase.
Semantic-Aware Re-Masking
After obtaining the candidate sequence , the denoising process requires a re-masking step to transition to the next noisy state . Standard strategies typically mask tokens based on low model confidence (low prediction probability) or simple random selection. However, these heuristics may inadvertently discard tokens that are critical for semantic alignment or retain tokens that steer the text away from the target semantic embedding .
To address this issue, we propose a contribution-based re-masking algorithm. Our insight is to strictly retain tokens that maximally contribute to minimizing the distance to the target semantic embedding, while masking those that are less relevant.
Algorithm 2 Semantic-Aware Re-Masking
Require: Sequence , target embedding , embedding model , denoise schedule
Compute base embedding
for position to do
Ablate token to get
Compute importance
Identify indices corresponding to the smallest scores , where
Update by setting for all
Identify indices corresponding to the lowest prediction probabilities, where and
Update by setting for all
Return
We present the algorithm in Algorithm 2. Let be the tokens in : and let denote its embedding. To quantify the contribution of the -th token , we perform a granular ablation: we replace with the special token to create a perturbed sequence and compute its embedding . We define the importance score as the change in distance to the target induced by ablating :
A large positive means masking moves the sequence away from the target, indicating that is critical to alignment. A near-zero means is irrelevant to alignment, and a negative means masking actually improves alignment, so the token is beneficial to mask. In accordance with the diffusion noise schedule, we retain the tokens with the largest and re-mask the remaining tokens. Specifically, at step , when the diffusion noise schedule calls for masking tokens, we first re-mask the tokens with the smallest and then re-mask an additional low-probability tokens. This simultaneously ensures semantic alignment and effective denoising.
By iterating these semantic-aware sampling and re-masking steps, we ensure that the generative process converges to a coherent sentence where the corresponding semantic embedding is close to the target reference embedding vector .
Necessity of Both Components. Semantic-aware sampling and re-masking play complementary roles: sampling steers candidate selection toward the target manifold, while re-masking preserves semantically critical tokens. We verify that disabling either component substantially degrades both semantic alignment (§ 3.3) and downstream DP utility (§ 4.2), confirming both stages are essential.
Semantic Alignment and Quality Preservation of Guided Diffusion
To demonstrate that Guided Diffusion can effectively generate high-quality sentences while closely aligning with a target semantic embedding, we conduct a comprehensive evaluation on standard benchmarks.
Experimental Setup. We use two datasets covering topical and sentiment-based steering: Yahoo Answers Topics (Zhang et al., [56]) (10 topic categories) and IMDB (Maas et al., [26]) (rich-sentiment movie reviews); from each we randomly draw 1,000 reference sentences exceeding 256 tokens as semantic targets. We employ LLaDA-8B-Instruct (Nie et al., [28]) as the backbone DLM, with target embeddings produced by sentence-t5-base (Ni et al., [27]) for Yahoo and stsb-roberta-base-v2 (Reimers & Gurevych, [31]) for IMDB. We report (i) semantic alignment as cosine similarity between generated and target embeddings, and (ii) generation quality as perplexity (PPL) computed with Qwen3-8B (Yang et al., [43]). We further include an ablation isolating the contribution of semantic-aware sampling and re-masking. Evaluation Results. The results are presented in Figure 2 and Table 1. Across both datasets, Guided Diffusion achieves strong semantic alignment with the target embeddings without sacrificing much on fluency, and both semantic-aware sampling and re-masking are necessary to reach this quality.
Figure 2. Evaluation of semantic alignment and generation quality. Left: Cosine similarity between generated and target embeddings. Guided Diffusion significantly improves alignment compared to unguided generation. Right: Perplexity (PPL) under Qwen3-8B of the original references (Orig.) and of guided and unguided generations. Guided text remains fluent: on IMDB its PPL matches unguided generation, and on Yahoo it is higher than unguided generation but still well below the references.
Semantic Alignment. Figure 2 (Left) shows that our Guided Diffusion achieves high fidelity to the target embeddings. The median cosine similarity between synthetic and original embeddings is 0.91 for Yahoo and 0.60 for IMDB. In contrast, unguided generation results in significantly lower semantic alignment (e.g., a median of 0.28 for IMDB), confirming that our Guided Diffusion can effectively preserve the high fidelity to the target embedding.
Generation Quality. In Figure 2 (Right), we demonstrate that Guided Diffusion preserves the linguistic fluency. The synthetic samples remain fluent, with median PPL scores of 6.51 (Yahoo) and 4.83 (IMDB). On IMDB this matches unguided generation (4.89); on Yahoo guided text has a higher PPL than unguided text, since steering toward a specific target constrains the sampler, but both remain well below the PPL of the human references. Our sampling and re-masking interventions therefore preserve fluency.
Ablation Study. To isolate the contribution of each guidance mechanism, we ablate semantic-aware sampling and semantic-aware re-masking on Yahoo and IMDB. When disabling one component, we replace it with its standard unguided counterpart (independent token sampling or low-confidence re-masking) while leaving the other component active, and report the mean cosine similarity over 1,000 samples in Table 1.
| Method | Yahoo | IMDB |
| Guided Diffusion (Full) | 0.90±0.04 | 0.59±0.11 |
| w/o semantic-aware re-masking | 0.79±0.02 | 0.35±0.10 |
| w/o semantic-aware sampling | 0.80±0.02 | 0.34±0.10 |
Table 1 (PDF p. 6). Ablation of semantic-aware sampling and re-masking. We report the cosine similarity between generated and target embeddings on Yahoo and IMDB as mean±std; bold marks the best result. Disabling either substantially degrades alignment, confirming both stages are essential.
The full framework achieves the best alignment on both datasets, and disabling either causes a sharp drop. The effect is most pronounced on IMDB, where the partial variants (0.35 and 0.34) collapse toward the fully unguided baseline (see Figure 2). The two components yield comparable scores in isolation (e.g., 0.79 vs. 0.80 on Yahoo), indicating that they capture complementary signals: semantic-aware sampling steers the candidate selection toward the target manifold, while semantic-aware re-masking preserves semantically critical tokens across diffusion steps. Combining both restores the full alignment quality, confirming that the two stages are essential and reinforcing.
Differentially Private Synthetic Data
In this section, we present our framework for generating differentially private (DP) synthetic text. The DP synthetic data generation pipeline is illustrated in Figure 3.
Figure 3. Trustworthy synthetic data generation with Guided Diffusion. For DP, the input is a private dataset, and the trustworthy aggregation is -DP aggregation. For DSI (§ 5), the input is a dataset collected from multiple, potentially malicious sources, and the trustworthy aggregation is DSI aggregation.
The fundamental challenge in applying DP to natural language is that text is discrete and symbolic; unlike numerical data, one cannot simply compute the “average” of multiple sentences and add DP noise to obfuscate individual contributions. Consequently, prior works have been forced to rely on complex and unstable methods like DP-SGD (Yu et al., 2022) or use an LLM to synthesize a huge corpus of synthetic data and select according to private samples iteratively [42].
The utility of the latter approach, Private Evolution (PE; Xie et al. [42]), is limited mainly by its generation pipeline rather than by the DP mechanism. If DP noise were the main limitation, removing it would raise utility substantially. Instead, on PubMed, PE improves only from 27.90% at to 28.60% without DP noise (; Table 4). PE’s own results show the same pattern: moving from to changes most metrics only modestly, leaves some unchanged or lower, and does not close the gap to training on the original data (Table 2 in Xie et al. [42]). A weaker privacy guarantee therefore does not remove PE’s proposal-and-selection bottleneck. This pipeline is also costly: PE generates a large pool of candidate texts in every iteration and then selects among them, so with the same backbone a single PE iteration already takes 1.53× as long as Guided Diffusion (§ 4.2). These limitations motivate a generation paradigm that is both more efficient and achieves higher utility for DP synthetic text.
| Backbone | Method | Utility (%) | FD | JSD | Dist-2 | MAUVE |
| ICLR26 (utility: macro-F1) | ||||||
| LLaDA-8B-Instruct | PE | |||||
| DP-SGD+LoRA | ||||||
| Guided Diffusion | ||||||
| Dream-7B | PE | |||||
| DP-SGD+LoRA | ||||||
| Guided Diffusion | ||||||
| PubMed (utility: accuracy) | ||||||
| LLaDA-8B-Instruct | PE | |||||
| DP-SGD+LoRA | ||||||
| Guided Diffusion | ||||||
| Dream-7B | PE | |||||
| DP-SGD+LoRA | ||||||
| Guided Diffusion |
Table 2. DP synthetic text generation at with 2,000 synthetic samples, reported as meanstd over four seeds. Utility is macro-F1 on ICLR26 and causal next-token accuracy on PubMed. Bold marks the best result per dataset and backbone.
| Method | Accuracy (%) | |
| Without fine-tuning | 0.07 | |
| Fine-tuning on real data (non-DP) | 34.60 | |
| PE† | 1 | 27.90 |
| PE† | 28.60 | |
| DP-SGD-FT (GPT-2 Large)† | 1 | 31.10 |
| DP-SGD-FT (GPT-2 Large)† | 33.10 | |
| DP-SGD+LoRA (Llama-3-8B) | 1 | 24.88 |
| DP-SGD+LoRA (Llama-3-8B) | 2 | 24.70 |
| DP-SGD+LoRA (Llama-3-8B) | 4 | 24.37 |
| Guided Diffusion | 1 | 32.68 |
| Guided Diffusion | 2 | 32.65 |
| Guided Diffusion | 4 | 32.48 |
Table 4. Effect of the privacy budget on PubMed accuracy, with . Guided Diffusion uses LLaDA-8B-Instruct. †Results from PE’s official implementation, where each baseline uses its default generator.
Our framework circumvents these limitations by shifting the privacy mechanism to the semantic embedding space. While embeddings allow for straightforward DP aggregation (e.g., computing noisy centroids), converting these abstract vectors back into coherent text has historically been an ill-posed inverse problem. Our key insight is that our Guided Diffusion algorithm (Algorithms 1 and 2) effectively solves this “inversion gap,” acting as a robust decoder that reconstructs high-fidelity text conditioned on the DP embeddings. This enables a streamlined “embed-privatize-generate” workflow that was previously infeasible.
Framework for DP Synthetic Text Generation with Guided Diffusion
As depicted in Figure 3, our framework contains three distinct stages. We assume that the base DLM and the text embedding model contain no information about the private samples, i.e., they are not trained on the private samples, and that all pipeline hyperparameters (the number of clusters, the per-class sample budget, the generation hyperparameters, and the choice of encoder) are fixed before the private data are accessed. To ensure that the first assumption holds in our evaluation, we construct ICLR26 from reviews published after these models were released (§ 4.2); for PubMed, whose abstracts predate the DLMs, the formal guarantee is unaffected, but the empirical utility may partly reflect pretraining exposure. We now introduce the three stages of DP synthetic text generation with Guided Diffusion:
Semantic Embedding: We project the sensitive private dataset into a semantic space using a pre-trained text embedding model . This yields a set of continuous embedding vectors . These vectors are normalized with norm equal to 1, simplifying the sensitivity bound of the aggregated embedding vectors.
DP Aggregation: We apply a DP aggregation mechanism to these vectors. To avoid collapsing the diverse semantic information of the dataset into a single mean, we employ the practical DP clustering algorithm (Chang & Kamath, [7]). This algorithm partitions the high-dimensional embeddings into semantically coherent groups and computes a set of representative noisy centroids , satisfying -DP. For labeled corpora (e.g., ICLR26 in § 4.2, where each review carries a score), DP clustering is run class-conditionally: separately within each label class on its embedding subset, yielding per-class noisy centroids . Each synthetic sample inherits the class label of its source per-class centroid, and the per-class sample budget is fixed in advance as a public hyperparameter. Since the per-class subsets are disjoint, the joint release of all centroids is -DP under add/remove adjacency by parallel composition (§ 2). These centroids effectively capture the distribution of the private text while ensuring privacy protection for individual samples. The only data-dependent objects that leave the private environment are the noisy centroids returned by the DP clustering algorithm (and, if the algorithm returns a data-dependent number of centroids, that number, which is part of its DP output). The requested number of clusters is a public constant, the number of synthetic samples generated per centroid is fixed uniformly in advance, and raw cluster sizes and cluster assignments are neither released nor used by the decoder.
Guided Diffusion Generation: Finally, we utilize the DP embedding as the guidance signal () for our Guided Diffusion algorithm with DLM . The model iteratively denoises a random sequence, steered by our semantic-aware sampling and re-masking strategies (§ 3), to generate an arbitrarily large set of synthetic texts that are semantically aligned with . The original private samples are rigorously protected with -DP guarantees.
Privacy Guarantee. Under standard conditions of public pretrained and , class-conditional DP aggregation via the DP clustering algorithm (Chang & Kamath, [7]), hyperparameters fixed before accessing , and a fixed generation procedure that does not access , the released synthetic dataset is -DP with respect to under add/remove adjacency by post-processing invariance. The formal statement (Proposition I.1) and proof are in Appendix § 19.
Evaluation: DP-Compliant Synthetic Text Generation
To demonstrate the efficacy of our DP synthetic text paradigm with Guided Diffusion, we generate synthetic text under -DP guarantees and evaluate its downstream utility and its distributional fidelity to the private data.
Experimental Setup. To isolate the generation algorithm from the capability of the underlying generator, we run every method on the same DLM backbone, either of two widely used DLMs, LLaDA-8B-Instruct (Nie et al., [28]) or Dream-7B (Ye et al., [46]), and compare against PE (Xie et al., [42]) and DP-SGD+LoRA (Li et al., [18]). That is, PE and DP-SGD+LoRA also use the same DLM to generate synthetic text. For each backbone, all of the methods use the same private data, privacy parameters, synthetic-data size, prompts, and downstream evaluation.
We evaluate on two private datasets. PubMed (Yu et al., 2023) contains 75,316 medical abstracts; following the official PE protocol, we fine-tune BERT-small (Turc et al., [39]) on the synthetic DP samples and report causal next-token prediction accuracy on real held-out abstracts in PubMed’s test split. To rule out the possibility that the training data of the evaluated models, including the DLMs and the embedding models, contains the private samples, we construct a new dataset, ICLR26, of paper reviews from ICLR 2026, which were released after all evaluated models. We follow the setup and train/test split of the ICLR 2023 review dataset used by PE ([42]). Specifically, we train a roberta-base ([25]) classifier on the synthetic reviews to predict the five-class review score and report macro-F1 on real held-out reviews in the test split. Consistent with PE, we use the embedding model sentence-t5-base for PubMed and stsb-roberta-base-v2 for ICLR26. We run PE for four proposal-and-selection iterations, which already takes more than computational cost compared with Guided Diffusion.
Every method generates 2,000 synthetic samples at , and results are averaged over four seeds. Beyond downstream utility, we also measure distributional fidelity to the real data with four metrics: the Fréchet distance (FD) between MPNet ([35]) embeddings of synthetic and real texts, the Jensen–Shannon divergence (JSD) between their unigram distributions, Distinct-2 ([17]) for lexical diversity, and MAUVE ([29]), which compares the two distributions in a language-model embedding space and penalizes both synthetic texts that are unlike real ones and real texts that the synthetic set fails to cover.
Evaluation Results. In Table 2, Guided Diffusion achieves the best score on every fidelity metric in all four dataset–backbone settings and the best utility in three of them; on ICLR26 with Dream-7B, its macro-F1 is comparable with DP-SGD+LoRA (within standard deviation).
_Downstream Utility._ On ICLR26 with LLaDA-8B-Instruct, Guided Diffusion reaches 24.85% macro-F1, compared with 13.34% for PE and 20.12% for DP-SGD+LoRA. With Dream-7B, its macro-F1 (28.68%) is 16.19 points above PE and 0.55 points below DP-SGD+LoRA. PE’s macro-F1 stays near 13% with both backbones. On PubMed, Guided Diffusion improves causal next-token accuracy over PE by 10.53 points with LLaDA-8B-Instruct (32.68% vs. 22.15%) and by 12.43 points with Dream-7B (32.89% vs. 20.46%), and over DP-SGD+LoRA by more than 7 points with both backbones. The performance of Guided Diffusion also exceeds the results of PE’s official implementation, where PE and DP-SGD-FT ([53]) on GPT-2 Large use their default generators (27.90% and 31.10%; Table 4). Our result approaches the non-private upper bound obtained by fine-tuning on the real private corpus (34.60%).
_Distributional Fidelity._ Guided Diffusion produces synthetic text that is closer to the private data. It attains the lowest FD and unigram JSD and the highest Distinct-2 and MAUVE in every setting. The largest gap is in FD: with LLaDA-8B-Instruct, Guided Diffusion reduces FD from 0.78 (PE) and 0.60 (DP-SGD+LoRA) to 0.40 on ICLR26, and from 0.55 and 0.90 to 0.32 on PubMed. The higher Distinct-2 shows that this fidelity does not come from collapsing to a few prototypical texts.
_Computational Efficiency._ On eight A100 GPUs, Guided Diffusion generates the 2,000 ICLR26 samples in 2.74 hours. End-to-end PE with the same backbone takes 4.19 hours for a single private-selection iteration and 12.20 hours for the four iterations used in Table 2, so Guided Diffusion is faster. The difference is structural: PE repeatedly queries the generator for candidate variants and selects among them with a DP histogram, whereas our framework aligns each sample within a single denoising run. In our implementation, DLM forward passes account for 47.6% of the runtime, encoder scoring for 29.0%, and candidate construction for 17.0%; Appendix § 16 gives the full profile, including batch size, memory, and encoder-call counts. This comparison holds for a shared DLM backbone; PE with an autoregressive generator that supports KV caching has a different cost profile, which we have not measured. We do not include DP-SGD-based methods here, since their cost is dominated by private fine-tuning with per-sample gradient clipping and PE reports DP fine-tuning to be more than slower than PE ([42]).
_Hyperparameter Sensitivity._ Guided Diffusion is insensitive to the privacy budget: on PubMed with LLaDA-8B-Instruct, its accuracy is 32.68% / 32.65% / 32.48% at , so the strict budget costs little utility, and it stays above every DP baseline at each budget (Table 4). Generation quality is insensitive to the candidate pool size and the weight ratio in Algorithm 1. See Appendix § 15 for more results.
Ablation Study. To verify that the gains observed in § 3 carry over to the downstream settings, we ablate each guidance mechanism on PubMed with LLaDA-8B-Instruct while keeping the same DP embedding pipeline. As shown in Table 3, removing either component substantially degrades downstream accuracy: without semantic-aware sampling, accuracy collapses from 32.68% to 18.01%, and without semantic-aware re-masking it drops to 26.73%. This confirms that both components remain essential in DP synthetic text generation.
| Method | Accuracy (%) |
| Guided Diffusion (Full) | 32.68 |
| w/o semantic-aware sampling | 18.01 |
| w/o semantic-aware re-masking | 26.73 |
Table 3. Ablation of semantic-aware sampling and re-masking on PubMed with LLaDA-8B-Instruct under .
Trustworthy Synthetic Text via DSI
Beyond DP, we extend our Guided Diffusion framework to incorporate Data-Specific Indistinguishability (DSI; Xiao et al., [41]), providing complementary trustworthiness guarantees. The overall pipeline is consistent with the DP synthetic data generation framework in Figure 3, with DP aggregation replaced by DSI-based aggregation.
DSI is designed to mitigate specific data risks with rigorous protection guarantees, including backdoor attacks, copyright infringement, and unintended memorization. Unlike DP, which enforces indistinguishability against the presence of an arbitrary single sample under worst-case sensitivity, DSI ensures that the output is statistically indistinguishable from the output produced by a “safe” reference dataset. In our context, this guarantees that synthetic data generated from protected embeddings is statistically close to data generated from references with potentially corrupted contributions removed. Formally, DSI transfers high-probability trustworthy guarantees from a designated set of safe references to the target output:
Definition 5.1 (DSI Gaussian Mechanism). Given a function , a target input , and reference datasets , the mechanism with satisfies -DSI if, for a chosen divergence ,
When is the Rényi divergence of order , recovers the of -Rényi DP per reference.
DSI permits anisotropic optimized for the geometry of , which our framework exploits to construct high-utility trustworthy aggregations. Detailed motivation, the safe-reference construction, and the convex optimization for are deferred to Appendix § 13.
Framework for Trustworthy Synthetic Text Generation with Guided Diffusion
To deploy DSI for trustworthy synthetic data generation, we consider a distributed data collection paradigm where the corpus is collected from multiple sources. This setup mimics collaborative environments where data is aggregated from diverse providers, some of whom may be untrusted.
Semantic Embedding: Similar to the DP pipeline, we first project the collected text into a continuous semantic embedding space using a pre-trained semantic encoder, and then partition the embeddings into distinct clusters (e.g., via K-Means). The number of clusters may be set to one, in which case aggregation reduces to a single average embedding over the collected samples. After clustering, DSI-protected groups are defined locally within each cluster/class partition: for cluster and source , denotes source ’s records inside cluster ; for labeled corpora, the same definition is applied within each class. These groups are disjoint, so removing one group changes only the corresponding cluster-level DSI release; the joint synthetic release therefore preserves the same DSI bound by parallel composition and post-processing.
DSI Aggregation: Within each cluster, we define reference sets by removing groups. For leave--groups-out DSI, each reference excludes any protected groups from the cluster. We then compute the aggregated cluster centroid using the Optimal DSI Gaussian Mechanism, which injects calibrated noise so that the released centroid is statistically indistinguishable from one computed from the corresponding safe reference, bounding the influence of corrupted groups on the aggregation.
Guided Diffusion Generation: Finally, these DSI-sanitized embeddings serve as the conditioning targets for Guided Diffusion. The model generates synthetic text that preserves the semantic utility of the original cluster while inheriting the DSI indistinguishability guarantee through post-processing.
Our DSI Guided Diffusion framework is applicable to a broad spectrum of trustworthiness risks, including copyright protection and memorization mitigation as studied in the original DSI formulation [41]. Empirical evaluations against backdoor attacks (data poisoning) and unintended memorization [6] are reported in Appendix § 17 and Appendix § 18, respectively. In both cases, the pipeline without DSI already shows no observed attack at the tested scale (0 of 60 triggered tests succeed; max ROUGE-L stays below the 0.2 memorization threshold), and DSI Guided Diffusion preserves these outcomes at and , respectively, with a small utility cost. DSI therefore adds a formal certificate rather than an observed improvement: finite-sample tests cannot exclude untested triggers or rarer memorization events, whereas the DSI bound transfers the high-probability properties of each safe reference to the actual release by post-processing.
Trustworthy Guarantee. Under analogous conditions of public pretrained and , class/cluster-conditional DSI aggregation via the Optimal DSI Gaussian Mechanism, and a fixed generation procedure that does not access the collected corpus, the synthetic dataset remains -indistinguishable from the leave-one-protected-group-out reference for every protected group . The guarantee transfers from the safe references to by DSI post-processing. The formal statement (Proposition I.2) and proof sketch are in Appendix § 19.
Related Work
Prior DP synthetic text generation falls into two streams: DP-SGD-based fine-tuning [48, 53] (Yu et al. 2021) and Private Evolution (PE) variants [22, 42] that filter LLM candidates via DP histograms; both inherit structural limits from per-sample clipping or rejection sampling. The closest concurrent work, Zhou et al. [57], reconstructs DP-protected images from privatized embeddings via diffusion-based image editing [32]; translating this “embedding-to-data” approach to text is non-trivial because text is discrete and no off-the-shelf guided text-diffusion pipeline exists. Our framework bridges this gap via DLMs and accommodates both DP and DSI [41]. A detailed elaboration is given in Appendix § 12.
Conclusion & Limitations
We propose a novel framework that converts semantic embeddings into high-fidelity text via Guided Diffusion, establishing a new paradigm for private synthetic text generation. Our approach naturally integrates Differential Privacy, ensuring precise semantic control while providing rigorous privacy guarantees with little loss in utility. The same framework extends to Data-Specific Indistinguishability, bounding the influence of corrupted sources and individual samples at a small utility cost. Our empirical results demonstrate the superiority of Guided Diffusion over prior DP synthetic text generation baselines across datasets and diffusion backbones.
Limitations and Future Work. Several extensions remain open. Semantic-aware re-masking evaluates one ablated embedding per token; sparse or batched ablations could amortize this cost for very long sequences. Our generation pipeline operates on fixed-length token blocks; extending to longer documents fits naturally with hierarchical or sliding-window schemes that we leave for future work. Empirically, we focus on English short-form generation; transferring to multilingual corpora, code, and structured outputs is a promising direction.
AI Use Statement
We used generative AI tools to assist with the writing of this paper, mainly to check grammar and to polish the wording and readability of author-written text. The tools were also used to format LaTeX tables and figures. All research ideas, the Guided Diffusion method and its technical components, the privacy analysis, the experimental design and implementation, and the interpretation of results were developed by the authors. We did not use generative AI to generate datasets or to develop the method or the conceptual framework. The authors reviewed and revised all AI-polished text. The authors take responsibility for all the final content of this work.
Ethics Statement
This work aims to reduce the privacy risks in synthetic text generation. The DP and DSI guarantees hold under the assumptions stated in Propositions I.1 and I.2; practitioners should verify these assumptions before deployment. DP bounds the influence of each individual record but does not prevent inferences about population-level properties of the private corpus. Our experiments use publicly available data: PubMed abstracts, Yahoo Answers Topics, IMDB and Yelp reviews, and ICLR 2023 and ICLR 2026 paper reviews publicly posted on OpenReview. We use the reviews only to evaluate synthetic data generation and make no attempt to identify reviewers. Our study involves no human-subject experiments. We discuss broader impacts in Appendix § 11.
Reproducibility Statement
We describe Guided Diffusion in § 3, with pseudocode in Algorithms 1 and 2, the DP synthetic text framework in § 4, and the DSI instantiation in § 5, with its experiments in Appendices § 17 and § 18. Appendix § 14 lists all hyperparameters, the backbone and embedding models, and the DP clustering configuration, and Appendix § 15 reports the sensitivity to the privacy budget, the number of synthetic samples, and the generation hyperparameters. All backbone DLMs, embedding models, and baseline implementations are publicly available, and § 4.2 describes the evaluation datasets and protocols; ICLR26 follows the setup and train/test split of the ICLR 2023 review dataset used by PE ([42]). We will release our code publicly.
Broader Impact
This paper presents work whose goal is to advance the field of privacy-preserving machine learning. Our framework for private synthetic text generation has significant potential to benefit society by enabling the safe utilization of sensitive datasets, such as medical records, financial reports, and private communications, that are currently inaccessible due to privacy regulations (e.g., GDPR). By reconciling high-fidelity generation with rigorous DP guarantees, our approach mitigates critical risks associated with Large Language Models, including data memorization and leakage from training corpora.
While advancements in generative AI carry inherent risks of misuse, such as the generation of misinformation or impersonating content, our research focuses specifically on defensive mechanisms that certify safety. The DP guarantee bounds the influence of any single record on the released synthetic data, providing a quantifiable foundation for responsible release of synthetic corpora derived from sensitive sources. We hope this work contributes to the responsible and transparent deployment of AI systems in sensitive domains.
Detailed Related Work
Guided Diffusion Models. Image diffusion models have achieved remarkable success in controllable synthesis, using embedding-based guidance to steer generation [36, 14, 34, 8, 32]. While extensive research optimizes this guidance for images [30, 45, 11, 4] (Ye et al. 2023), its application to DLMs remains underexplored. Unlike the continuous image domain, text is discrete, and the dominance of prompt-based AR LLMs has overshadowed the development of embedding-based control. To the best of our knowledge, we are the first to effectively guide DLMs in converting semantic embeddings into coherent, high-fidelity text.
Differentially Private Synthetic Data Generation. Prior research on generating synthetic data with DP guarantees spans both image and text domains [16], generally bifurcating into two primary streams: 1) DP-SGD-based methods [48, 20, 13, 53, 12, 9, 38] (Yu et al. 2021) and 2) Private Evolution (PE) variants [22, 42, 15, 54, 23].
DP-SGD approaches involve training/fine-tuning a generative model. While foundational, these methods suffer from significant computational overhead due to per-sample gradient clipping and often yield suboptimal utility when scaling to LLMs. In contrast, PE variants circumvent direct training by iteratively querying public LLMs to generate a massive pool of synthetic candidates. A DP selection mechanism, which adds Gaussian noise to distance histograms, is then employed to filter for samples that align with the private data distribution. While PE offers improved efficiency and utility-privacy tradeoffs compared to DP-SGD, it remains heavily dependent on the quality of manual sampling prompts and implicitly constrains the diversity and volume of synthetic samples due to the rejection-sampling bottleneck.
Recently, Zhou et al. [57] introduced a novel paradigm for image synthesis, demonstrating that high-fidelity images can be reconstructed from DP embeddings using diffusion-based image editing [32]. However, translating this “embedding-to-data” approach to the text domain is non-trivial due to the discrete nature of language and the absence of established Guided Diffusion pipelines for text. Our work bridges this gap: we propose an alternative approach that, for the first time, exploits the steerable properties of DLMs to efficiently convert semantic embeddings into high-utility synthetic text. Crucially, this framework naturally accommodates DP mechanisms, providing rigorous privacy guarantees alongside superior generation quality.
Background on DSI
Motivation for DSI. DP is rooted in Input-Independent Indistinguishability (III): the indistinguishability of the released leakage must hold for arbitrary pairs of neighboring datasets. III is necessary when the goal is to uniformly upper bound the additional knowledge an adversary gains from observing the leakage, but is overly conservative for trustworthiness concerns whose threat models are not worst-case over all neighbors.
DSI as Transferring Guarantees from Safe References. DSI (Xiao et al., [41]) relaxes III by establishing specific indistinguishability between the target output and a designated set of safe reference outputs. As a concrete example, consider an input dataset where each is contributed by source , and suppose is poisoning data from an unknown malicious source. The leave-one-out subset is then a safe reference: a model trained on it should be robust to the backdoor in with high probability. If we ensure indistinguishability between and each for — which naturally includes the unknown — then the trustworthy guarantee of transfers to . Crucially, only the specific pairs need to be indistinguishable, rather than arbitrary pairs.
Optimal DSI Gaussian Noise. Unlike the DP Gaussian mechanism, which adds isotropic noise calibrated to the global sensitivity, DSI permits anisotropic noise optimized for the specific geometry of . The optimal covariance minimizing the expected noise magnitude subject to the DSI constraints is the solution of a convex optimization over Lagrangian multipliers; we use this in our framework (§ 5).
Implementation Details
We list the concrete hyperparameter values used throughout our experiments. All settings are held fixed unless stated otherwise. All experiments are run on A100 GPUs.
Diffusion Backbone
We use LLaDA-8B-Instruct (Nie et al., [28]) and Dream-7B (Ye et al., [46]) as the backbone masked diffusion language models. For these models, every generation uses the settings below.
Sequence length: tokens. Note that LLaDA takes a semi-autoregressive approach, with default block size 32.
We use the default number of diffusion (denoising) steps: .
Denoise schedule : We take the default linear schedule used by LLaDA-8B-Instruct and Dream-7B.
Semantic-Aware Sampling (Algorithm 1)
Candidate pool factor: .
Top- filter size: .
Log-probability weight: .
Hamming-distance weight: .
We did not tune and extensively; Appendix § 15 shows that varying and has little effect on generation quality.
Semantic-Aware Re-Masking (Algorithm 2)
Denoise schedule : We take the same default linear schedule used by the backbone LLaDA-8B-Instruct and Dream-7B.
DP and DSI Aggregation
DP clustering: Chang & Kamath [7]’s implementation with default hyperparameters; the privacy budget is specified per experiment. The number of clusters is set to 100. For each class, the number of synthetic samples generated from each released centroid is fixed uniformly in advance. We do not use private cluster sizes.
DSI optimal Gaussian noise: computed via the convex optimization formulation described in Appendix § 13 (see also Xiao et al. [41]), with the security parameter specified per experiment ( in Appendix § 17, in Appendix § 18).
Embedding Models
We deliberately use two different semantic embedding models chosen to match the dominant semantic axis of each dataset. sentence-t5-base ([27]), trained on a broad mixture of sentence-level tasks, produces embeddings that emphasize topical and factual content; we therefore use it for PubMed (medical abstracts grouped by topic area) and Yahoo (10 explicit topic categories). stsb-roberta-base-v2 ([31]), fine-tuned on the STS-Benchmark sentence-similarity task that emphasizes evaluative and stylistic similarity, captures sentiment- and style-driven semantics more reliably, so we use it for the review-heavy datasets ICLR26, ICLR23, Yelp, and IMDB, where rating and stylistic register dominate the relevant signal.
Sensitivity Analyses
Privacy Budget. Table 4 varies with using LLaDA-8B-Instruct on PubMed. The accuracy of Guided Diffusion changes by at most 0.2 points, so the strict budget used in our main results costs little utility relative to looser budgets. At , Guided Diffusion is within 2 points of the non-private upper bound of fine-tuning on the real private corpus (34.60%). For comparison, the table also lists the baselines across budgets: PE and DP-SGD-FT ([53]) on GPT-2 Large from PE’s official implementation ([42]), which use their default generators, and DP-SGD+LoRA on Llama-3-8B ([18]). Guided Diffusion exceeds every DP baseline at each tested budget, and even PE without DP noise (, 28.60%). Scaling DP-SGD to Llama-3-8B with LoRA does not help: its accuracy stays within 24.4%–24.9% across budgets, below DP-SGD-FT on GPT-2 Large at .
Number of Synthetic Samples. Since the synthetic texts are a post-processing of the released DP centroids, generating more of them incurs no additional privacy cost. Table 5 shows that utility improves as the synthetic training set grows from 100 to 2,000 samples: macro-F1 rises from 13.13% to 24.85% on ICLR26 with LLaDA-8B-Instruct.
| Synthetic samples | Macro-F1 (%) |
| 100 | 13.13 |
| 500 | 20.46 |
| 1,000 | 20.83 |
| 2,000 | 24.85 |
Table 5. Effect of the number of synthetic samples on ICLR26 (LLaDA-8B-Instruct, ), reported as meanstd.
Generation Hyperparameters. Table 6 varies the candidate pool factor , the weight ratio of the sampling score, and the number of denoising steps . We report the cosine similarity between each generated text and its target embedding under the guidance encoder (Guide sim.) and under an independent MPNet encoder (MPNet sim.), together with perplexity (PPL). Pool size and weighting have little effect: across and , guide similarity stays within 0.691–0.693 and perplexity within 4.20–4.48. Halving the denoising steps to $T = 224 lowers guide similarity from 0.692 to 0.676 and nearly doubles perplexity, from 4.22 to 7.95. This degradation with fewer denoising steps is expected: it is a property of the DLM rather than of our guidance design. We therefore recommend using the default number of denoising steps of each DLM.
| Setting | Guide sim. | MPNet sim. | PPL |
| Default (, , ) | 0.6920 | 0.5396 | 4.218 |
| 0.6756 | 0.5335 | 7.948 | |
| 0.6929 | 0.5345 | 4.482 | |
| 0.6930 | 0.5405 | 4.352 | |
| 0.6919 | 0.5385 | 4.247 | |
| 0.6910 | 0.5415 | 4.203 |
Table 6. Effect of generation hyperparameters on ICLR26. Each row changes one hyperparameter from the default.
Runtime Profile
Table 7 profiles Guided Diffusion with LLaDA-8B-Instruct at batch size 16 in BF16 on a single A100-SXM4-40GB, the same configuration used for PE. One output uses one complete denoising trajectory of DLM forward passes; the encoder is called in batched form on the filtered candidates and on the token-ablated variants, which amounts to 897 logical encoder calls (executed as 10,527 internal micro-batches) per batch. Generating 2,000 samples takes 21.94 hours on one A100 and 2.74 hours on eight, versus 12.20 hours for four rounds of PE with the same backbone on the same eight GPUs.
| Measurement | Value |
| Wall-clock time per 16-sample batch | 631.86 s |
| Amortized time per sample | 39.49 s |
| 2,000 samples on 1 / 8 A100s | 21.94 h / 2.74 h |
| Peak allocated memory | 16.35 GiB |
| DLM forward passes per batch | 448 |
| Logical encoder calls / internal micro-batches per batch | 897 / 10,527 |
| Component | Share of time |
| DLM forward passes | 47.6% |
| Encoder forward passes | 29.0% |
| Candidate construction | 17.0% |
| Decoding / other | 3.7% / 2.7% |
Table 7. Runtime profile of Guided Diffusion (LLaDA-8B-Instruct, batch size 16, BF16, one A100-SXM4-40GB).
Evaluation: Backdoor-Resistant Synthetic Text Generation
We evaluate DSI Guided Diffusion against a semantic backdoor attack in synthetic text generation.
Experimental Setup. For dataset, we use ICLR23, the dataset of ICLR 2023 paper reviews that the authors of PE (Xie et al. [42]) collected from OpenReview, to aggregate semantic embeddings and generate synthetic reviews. The downstream utility is evaluated by training a classifier on 2,000 synthetic samples for review-rating prediction; accuracy is measured on the ICLR23 test split. Unlike the DP evaluation in § 4.2, which treats each dataset as a single private corpus, here the data is partitioned across sources, one of which may be malicious. For labeled data, aggregation is performed class-conditionally, and synthetic samples inherit the rating label of their source class/cluster centroid. For models, we employ LLaDA-8B-Instruct as the backbone DLM and stsb-roberta-base-v2 as the semantic embedding encoder; we fine-tune roberta-base (Liu et al. [25]) as the text classifier.
Threat Model. We consider a distributed setting where data is contributed from sources. Within each class/semantic cluster, the DSI protected groups are the source-specific subsets, matching the protected-group definition in Proposition 1.2. One source may be malicious and inject poisoned reviews to manipulate the review-rating classifier, while the other sources provide non-poisoned data. The attacker uses a semantic trigger: it generates paper reviews on the topic of medical images and labels them as “strong reject.” If the attack succeeds, reviews about medical images are classified as strong reject at inference time. The attack success rate (ASR) is measured on a held-out poisoned test set; the attacker injects 45 poisoned samples into the training corpus, and ASR is evaluated on 60 triggered test samples.
Defense Mechanism and Baselines. We apply DSI Guided Diffusion with leave-one-group-out aggregation. For each class/cluster centroid, we compute the optimal DSI Gaussian noise required to make the released centroid indistinguishable from one computed without any single source group in that class/cluster. We set the DSI security parameter to (see § 5). We compare four methods: (1) Orig-benign, training the classifier on the original benign dataset; (2) Orig-backdoor, training on the original dataset with the backdoor planted; (3) Synthetic-w/o-DSI, training on synthetic data generated by Guided Diffusion without DSI noise; and (4) Synthetic-w/-DSI, training on synthetic data generated using DSI Guided Diffusion.
Experimental Results. The results are summarized in Table 8. We evaluate clean accuracy on benign data (ACC) and backdoor attack success rate (ASR).
| Method | ACC (%) | ASR (%) |
| Orig-benign | 47.80 | 0 |
| Orig-backdoor | 48.82 | 100 |
| Synthetic-w/o-DSI | 39.85 | 0 |
| Synthetic-w/-DSI | 37.74 | 0 |
Table 8. Evaluation against a semantic backdoor attack. ASR is measured on 60 held-out triggered examples. DSI Guided Diffusion preserves the empirical 0% ASR while adding a formal indistinguishability certificate.
Vulnerability of Standard Training. As shown in the first two rows of Table 8, training on the original benign data yields an ACC of 47.80% and 0% ASR. When the backdoor is planted, the model remains useful (48.82% ACC) but becomes fully compromised, exhibiting 100% ASR. This confirms that standard training is highly susceptible to this semantic poisoning attack.
Empirical Robustness of Guided Diffusion. Guided Diffusion already provides strong empirical mitigation even without explicit DSI noise. Generating synthetic data from raw aggregated embeddings drops ASR to 0% on the 60 triggered test examples. This suggests that semantic embedding aggregation followed by Guided Diffusion reconstruction disrupts this topic-triggered backdoor pattern.
Certified Mitigation with DSI. While Guided Diffusion itself mitigates this attack empirically, DSI adds a formal indistinguishability certificate relative to the leave-one-group-out reference. With , DSI Guided Diffusion also maintains 0% ASR while achieving an ACC of 37.74%, a small utility drop relative to Synthetic-w/o-DSI (39.85%). For context, this accuracy is comparable to PE without DP noise (38.5% using Llama-2-7B), while additionally providing the DSI certificate.
Evaluation: Memorization-Mitigated Synthetic Text Generation
Beyond data poisoning, DSI is also designed to mitigate unintended memorization, where a generative model regenerates verbatim training samples [6]. We evaluate whether our DSI Guided Diffusion can certifiably bound such memorization on a sensitive review corpus.
Experimental Setup. We adopt the same backbone and embedding models as in Appendix § 17 and modify the threat model to target memorization. For dataset, we use 50 1-star Grocery reviews randomly drawn from Yelp [47].
Threat Model. An adversary attempts to extract training samples via prefix-based completion: for each training sample, we provide the first 100 tokens as a prefix and generate 10 completions, then measure the maximum ROUGE-L between any generated suffix and the true suffix. Values below 0.2 indicate low memorization.
ROUGE-L [21] is an F-score based on the longest common subsequence between two token sequences and is widely used to quantify near-verbatim overlap; in our setting, a higher ROUGE-L between a generated suffix and the true suffix indicates more verbatim or near-verbatim copying of the training sample. For each prefix we report the maximum ROUGE-L over the 10 generated completions, using 0.2 as a conservative cutoff below which generations are considered to share no significant verbatim content with the reference.
Defense Mechanism and Baselines. We apply DSI Guided Diffusion with leave-one-out aggregation at the individual sample level (there is a single cluster in this setting), calibrating the optimal DSI Gaussian noise so that the generated centroid is statistically indistinguishable from one computed without any single training sample. We set the security parameter to . We compare against Synthetic-w/o-DSI, which conditions Guided Diffusion on the average sample embedding without any DSI noise, isolating the empirical contribution of guided generation.
Experimental Results. Table 9 reports the max ROUGE-L value averaged over the samples (lower is better) and semantic alignment (cosine similarity to the target embedding; higher is better). The semantic alignment values reported here are higher than those in § 3 (e.g., 0.59 on IMDB) because the 1-star Grocery subset of Yelp is far more topically homogeneous than IMDB, which spans diverse movies, genres, and stylistic registers.
| Method | Max ROUGE-L ↓ | Semantic Alignment ↑ |
| Synthetic-w/o-DSI | ||
| Synthetic-w/-DSI |
Table 9. Memorization evaluation on Yelp 1-star Grocery reviews. Max ROUGE-L below 0.2 indicates low memorization. DSI Guided Diffusion provides formal indistinguishability with a small utility cost. Results are reported as mean.
Empirical Memorization Mitigation by Guided Diffusion. Both variants achieve max ROUGE-L well below the 0.2 memorization threshold (0.1446 for Synthetic-w/o-DSI and 0.1460 for Synthetic-w/-DSI), confirming that Guided Diffusion empirically suppresses verbatim copying of training samples even before any DSI noise is added. This mirrors the Appendix § 17 finding that the framework offers strong empirical robustness on its own.
Certified Memorization Mitigation by DSI. The cost of converting empirical robustness into a formal indistinguishability guarantee is small: applying DSI Gaussian noise at keeps max ROUGE-L small while keeping semantic alignment high (0.76 vs. 0.80 without DSI). Crucially, DSI extends the empirical guarantee to a formal indistinguishability bound: it certifies closeness to the leave-one-out centroid-generation pipeline while preserving low observed copying. It bounds the influence of individual samples under the specified DSI mechanism and reference construction.
Formal Guarantees and Proof Sketches
Proposition I.1 (DP Guarantee). Let denote the private corpus, where is an optional class label, and let denote the labeled synthetic dataset produced by the DP Guided Diffusion framework. Suppose the following conditions hold:
Public pretrained models. The embedding model and the diffusion language model are pretrained on a corpus disjoint from and are treated as public.
Data-independent configuration. The requested number of clusters , the per-class synthetic sample budget, the generation hyperparameters, and the choice of and are fixed before is accessed.
Class-conditional DP aggregation. The DP clustering algorithm [7] is run separately within each label , releasing noisy centroids (and, if the algorithm returns a data-dependent number of centroids, that number) that are -DP with respect to add/remove adjacency on the per-class subset . (For an unlabeled corpus, this reduces to a single class.) Nothing else derived from , in particular no raw cluster size or assignment, is released.
Fixed generation procedure. Guided Diffusion (Algorithms 1 and 2) is a randomized function of and the public models alone, and does not access . Each synthetic sample inherits the class label of its source per-class centroid.
Then the released synthetic dataset is -DP with respect to under add/remove adjacency. Under replace-one adjacency, where a single record may change its label, is -DP.
Proof sketch of Proposition I.1 (DP Guarantee). By condition (3), each per-class centroid release is -DP w.r.t. . Under add/remove adjacency a single record belongs to exactly one class, so the per-class subsets form a disjoint partition of and parallel composition yields -DP for the joint release w.r.t. ; under replace-one adjacency a record may leave one class and enter another, affecting two independent mechanisms, and basic composition gives . Conditions (1), (2), and (4) ensure that is a randomized function of and public model components only, with no further dependence on . Post-processing invariance (§ 2) preserves the -DP guarantee on the final synthetic data release.
Proposition I.2 (DSI Guarantee). Let denote the input corpus, where is an optional class label. After optional class-conditional clustering, let denote cluster within class ; for an unlabeled corpus, there is a single class. Let denote the records from source . We define DSI protected groups locally within each class/cluster partition:
The nonempty form a disjoint partition of the clustered corpus. For leave--groups-out DSI, let with , define
Let denote the synthetic dataset produced by DSI Guided Diffusion. Suppose the following conditions hold:
Public pretrained models. The embedding model and the diffusion language model are fixed public mechanisms independent of , and all hyperparameters, including the clustering configuration, are fixed before is accessed.
Class/cluster-conditional DSI aggregation. For each class/cluster pair , the DSI Gaussian mechanism releases a sanitized centroid
satisfying
for every leave--groups-out reference . The randomness used for different class/cluster releases is independent. When is Rényi divergence, we report the corresponding DSI security parameter .
Fixed generation procedure. The Guided Diffusion procedure is a randomized function of and the public models alone, and does not access . Each synthetic sample inherits the label of its source class/cluster centroid.
Then, for every protected group collection , the released synthetic dataset satisfies the -DSI indistinguishability bound with respect to the leave--groups-out reference . In particular, if any protected cluster-source group, or any protected groups within the same class/cluster partition, are corrupted, remains statistically close to the dataset that would have been produced from the corresponding safe reference.
Proof sketch of Proposition 1.2. By condition (2), each class/cluster centroid release satisfies the stated DSI bound with respect to its leave--groups-out references. Consider any protected group collection and the corresponding reference . Removing changes only the input to the cluster-level DSI mechanism; all other class/cluster releases have identical distributions under the target and reference inputs. Since the class/cluster releases use independent randomness, the divergence of the joint centroid release equals the divergence of the changed coordinate:
Thus the joint release of all DSI-sanitized centroids preserves the same bound by parallel composition over disjoint class/cluster partitions. Conditions (1) and (3) ensure that the final synthetic dataset is a randomized function only of the DSI-sanitized centroids and public model components, with no further dependence on . The post-processing property of DSI ([41]) therefore preserves the indistinguishability bound on the final synthetic text release.
References
- [1]Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp. 308–318, 2016.DOI
- [2]Borja Balle and Yu-Xiang Wang. Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In International conference on machine learning, pp. 394–403. PMLR, 2018.arxiv.org/abs/1805.06530
- [3]Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adversaries. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1138–1156. IEEE, 2022.arxiv.org/abs/2201.04845
- [4]Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Roni Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=pzpWBbnwiJ.arxiv.org/abs/2302.07121
- [5]California State Legislature. California Consumer Privacy Act of 2018. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180AB375, 2018.
- [6]Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pp. 2633–2650, 2021.arxiv.org/abs/2012.07805
- [7]Alisa Chang and Pritish Kamath. Differentially private clustering in google’s differential privacy library. https://github.com/google/differential-privacy/tree/main/learning/clustering, 2021. Accompanying article available at https://research.google/blog/practical-differentially-private-clustering/.
- [8]Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780–8794, 2021.
- [9]Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion models. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=ZPpQk7FJXF.arxiv.org/abs/2210.09929
- [10]Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference, pp. 265–284. Springer, 2006.
- [11]Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Reddy Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=PUiqjI4rzq7.arxiv.org/abs/2212.05032
- [12]Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L Smith, Olivia Wiles, and Borja Balle. Differentially private diffusion models generate useful synthetic images. arXiv preprint arXiv:2302.13861, 2023.arxiv.org/abs/2302.13861
- [13]Frederik Harder, Milad Jalali, Danica J. Sutherland, and Mijung Park. Pre-trained perceptual features improve differentially private image generation. Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL https://openreview.net/forum?id=R6W7zkMz0P.arxiv.org/abs/2205.12900
- [14]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020.arxiv.org/abs/2006.11239
- [15]Charlie Hou, Akshat Shrivastava, Hongyuan Zhan, Rylan Conway, Trang Le, Adithya Sagar, Giulia Fanti, and Daniel Lazar. PrE-text: Training language models on private federated data in the age of LLMs. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 19043–19061. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/v235/hou24c.html.arxiv.org/abs/2406.02958
- [16]Yuzheng Hu, Fan Wu, Qinbin Li, Yunhui Long, Gonzalo Munilla Garrido, Chang Ge, Bolin Ding, David Forsyth, Bo Li, and Dawn Song. Sok: Privacy-preserving data synthesis. In 2024 IEEE Symposium on Security and Privacy (SP), pp. 4696–4713. IEEE, 2024.arxiv.org/abs/2307.02106
- [17]Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. A diversity-promoting objective function for neural conversation models. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 110–119, San Diego, California, June 2016. Association for Computational Linguistics. doi: 10.18653/v1/N16-1014. URL https://aclanthology.org/N16-1014/.DOI
- [18]Qinbin Li, Junyuan Hong, Chulin Xie, Jeffrey Tan, Rachel Xin, Junyi Hou, Xavier Yin, Zhun Wang, Dan Hendrycks, Zhangyang Wang, et al. Llm-pbe: Assessing data privacy in large language models. Proceedings of the VLDB Endowment, 17(11):3201–3214, 2024.arxiv.org/abs/2408.12787
- [19]Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. Diffusion-lm improves controllable text generation. Advances in neural information processing systems, 35:4328–4343, 2022a.arxiv.org/abs/2205.14217
- [20]Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. Large language models can be strong differentially private learners. In International Conference on Learning Representations, 2022b. URL https://openreview.net/forum?id=bVUB31tATmz.arxiv.org/abs/2110.05679
- [21]Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pp. 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013/.
- [22]Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Differentially private synthetic data via foundation model APIs 1: Images. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=YEhQs8PO1O.arxiv.org/abs/2305.15560
- [23]Zinan Lin, Tadas Baltrusaitis, and Sergey Yekhanin. Differentially private synthetic data via APIs 3: Using simulators instead of foundation model. In Will Synthetic Data Finally Solve the Data Access Problem?, 2025. URL https://openreview.net/forum?id=7AvUO96Pmk.arxiv.org/abs/2502.05505
- [24]Pierre Lison, Ildikó Pilán, David Sanchez, Montserrat Batet, and Lilja Øvrelid. Anonymisation models for text data: State of the art, challenges and future directions. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4188–4203, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.323. URL https://aclanthology.org/2021.acl-long.323/.DOI
- [25]Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019.arxiv.org/abs/1907.11692
- [26]Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142–150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P11-1015.
- [27]Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the association for computational linguistics: ACL 2022, pp. 1864–1874, 2022.arxiv.org/abs/2108.08877
- [28]Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.09992, 2025.arxiv.org/abs/2502.09992
- [29]Krishna Pillutla, Swabha Swamyamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. MAUVE: measuring the gap between neural text and human text using divergence frontiers. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS ’21, Red Hook, NY, USA, 2021. Curran Associates Inc. ISBN 9781713845393.arxiv.org/abs/2102.01454
- [30]Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.arxiv.org/abs/2204.06125
- [31]Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL http://arxiv.org/abs/1908.10084.
- [32]Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695, 2022.arxiv.org/abs/2112.10752
- [33]Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136–130184, 2024.arxiv.org/abs/2406.07524
- [34]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=St1giarCHLP.arxiv.org/abs/2010.02502
- [35]Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. Mpnet: masked and permuted pre-training for language understanding. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA, 2020. Curran Associates Inc.arxiv.org/abs/2004.09297
- [36]Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1907.05600
- [37]Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuehang Yang, Hongli Yu, Xingwei Qu, et al. Seed diffusion: A large-scale diffusion language model with high-speed inference. arXiv preprint arXiv:2508.02193, 2025.arxiv.org/abs/2508.02193
- [38]Bowen Tan, Zheng Xu, Eric P. Xing, Zhiting Hu, and Shanshan Wu. Synthesizing privacy-preserving text data via finetuning *without* finetuning billion-scale LLMs. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=FCm4laCLiH.arxiv.org/abs/2503.12347
- [39]Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962, 2019.arxiv.org/abs/1908.08962
- [40]Paul Voigt and Axel Von dem Bussche. The eu general data protection regulation (gdpr). A practical guide, 1st ed., Cham: Springer International Publishing, 10(3152676):10–5555, 2017.
- [41]Hanshen Xiao, Zhen Yang, and G. Edward Suh. Trustworthy machine learning through data-specific indistinguishability. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=JRMoENZgNl.
- [42]Chulin Xie, Zinan Lin, Arturs Backurs, Sivakanth Gopi, Da Yu, Huseyin Inan, Harsha Nori, Haotian Jiang, Huishuai Zhang, Yin Tat Lee, Bo Li, and Sergey Yekhanin. Differentially private synthetic data via foundation model apis 2: text. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org, 2024.arxiv.org/abs/2403.01749
- [43]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengyuan Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025a.arxiv.org/abs/2505.09388
- [44]Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505.15809, 2025b.arxiv.org/abs/2505.15809
- [45]Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721, 2023.arxiv.org/abs/2308.06721
- [46]Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487, 2025.arxiv.org/abs/2508.15487
- [47]Yelp, Inc. Yelp Open Dataset. https://www.yelp.com/dataset, 2023.
- [48]Jinsung Yoon, James Jordon, and Mihaela van der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=S1zk9iRqF7.
- [49]Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.16933, 2025.arxiv.org/abs/2505.16933
- [53]Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. Synthetic text generation with differential privacy: A simple and practical recipe. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1321–1342, 2023.arxiv.org/abs/2210.14348
- [54]Jianqing Zhang, Yang Liu, JIE FU, Yang Hua, Tianyuan Zou, Jian Cao, and Qiang Yang. PCEvolve: Private contrastive evolution for synthetic dataset generation via few-shot private data and generative APIs. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=IKCfxWtTsu.arxiv.org/abs/2506.05407
- [56]Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015.arxiv.org/abs/1509.01626
- [57]Felix Zhou, Samson Zhou, Vahab Mirrokni, Alessandro Epasto, and Vincent Cohen-Addad. Private training & data generation by clustering embeddings. arXiv preprint arXiv:2506.16661, 2025.arxiv.org/abs/2506.16661
- [58]Zhanke Zhou, Jianing Zhu, Fengfei Yu, Xuan Li, Xiong Peng, Tongliang Liu, and Bo Han. Model inversion attacks: A survey of approaches and countermeasures. arXiv preprint arXiv:2411.10023, 2024.arxiv.org/abs/2411.10023

