betaFeedback

WorksComputer Science

Training-Free Character-Length Control in Summarization with Diffusion Language Models

Abstract

Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space. Autoregressive language models (LMs) struggle with precise character-length control: training-based methods require task-specific fine-tuning, while zero-shot prompting strategies achieve only approximate compliance. We propose DELTA (Diffusion Expected Length Targeting Algorithm), a training-free inference-time method for masked diffusion LMs that estimates expected character length at each denoising step and dynamically inserts or removes masked positions to steer toward a target length. DELTA requires no fine-tuning and is designed for off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to baselines while maintaining competitive summary quality in both automatic and human evaluation.

Introduction

Many summarization tasks require outputs that fit a precise character budget. In publishing workflows, for example, headlines and article previews are often produced for predefined display slots with fixed character capacities ([10]); similar constraints arise in notification previews, social media posts, subtitles, and UI text, where even a few extra characters can cause truncation or layout overflow. Existing approaches to length-controlled summarization operate within the constraints of autoregressive generation: training-based methods ([2] (Miculicich et al. 2023)) modify the model or its objective to encourage a target length, while zero-shot strategies ((Retkowski and Waibel, 2025; Xie and Lee, 2025)) rely on prompting techniques or post-hoc filtering. Both approaches operate under standard left-to-right autoregressive decoding, where previously emitted

tokens are not directly revised within the same generation pass. This makes it difficult to dynamically correct the decoded character length as generation proceeds.

Masked diffusion language models such as LLaDA ((Nie et al., 2025)) and Dream ((Ye et al., 2025)) offer a structurally different generation process. These models iteratively refine a buffer of masked tokens over multiple denoising steps, and are trained to generate across a range of output lengths. This means that occasional insertions or removals of mask tokens will not severely disrupt generation quality. We exploit this property with DELTA, an inference-time method that estimates expected character length from both committed tokens and the probability distributions over remaining masks, then adjusts the buffer to steer toward a target. DELTA requires no fine-tuning or architectural changes and is designed to apply to off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to all baselines while maintaining competitive summary quality.

Background and Related Work

Length-Controlled Summarization. Prior work on length-controlled summarization predominantly operates within the autoregressive framework and spans token-, word-, sentence-, and character-level constraints. Training-based approaches modify either the model or its objective: CTRLsum ([8]) conditions generation on coarse length bins, Miculicich et al. (Miculicich et al. (2023)) jointly train a length predictor and constrained decoder for precise token or sentence counts, and Belligoli et al. (Belligoli et al. [2]) upweight the EOS-token loss to enforce a character-length ceiling.

Training-free approaches instead steer pretrained autoregressive LMs at inference time. Retkowski and Waibel (2025) convert character targets to word targets using statistical ratios and filter multiple samples, while Xie and Lee (2025) append countdown markers to obtain near-exact word-level control. Instruction- and position-based methods likewise target explicit length constraints, including sentence compression with InstructCMP (Juseon-Do et al. [12]) and positional representations for length control (Wang et al. (2024)). Other inference-time approaches address precise length control in LLMs (Butcher et al. [3]), black-box length-constrained generation (Gu et al. [7]), and iterative controllable summarization (Ryu et al. (2026)).

Controllable generation has also been studied in diffusion language models, including methods that steer generation toward desired attributes (Li et al. (2022)). In contrast to these approaches, DELTA exploits masked-diffusion denoising to directly estimate and dynamically correct decoded character length during generation, requiring neither model training nor post-hoc regeneration.

Discrete Diffusion Language Models. D3PM (Austin et al. [1]) introduced discrete diffusion over token vocabularies, and MDLM (Sahoo et al. (2024)) refined the training objective with a Rao-Blackwellized loss that substantially narrows the likelihood gap with autoregressive models. These foundations enabled scaling to billion parameters: LLaDA (Nie et al. (2025)) trains an 8B masked diffusion model from scratch, competitive with LLaMA 3, while Dream (Ye et al. (2025)) initializes from a pretrained autoregressive checkpoint with context-adaptive noise rescheduling. Both generate output by iteratively unmasking tokens using confidence-based strategies originating in MaskGIT (Chang et al. [4]).

Variable-Length Generation in Diffusion LMs. Standard masked diffusion decoding requires choosing a fixed token-buffer size before denoising. This token-buffer length is distinct from a target decoded character length because tokens correspond to strings of varying character length. DreamOn (Wu et al. (2026)) introduces special expand/delete tokens that allow the model to adjust the token-buffer length during denoising, requiring fine-tuning. DAEDAL (Li et al. (2026)) and ρ\rho-EOS (Yang et al. (2026)) are training-free alternatives that adapt sequence length during open-ended generation. These methods address variable-length generation rather than generation toward a specified decoded character budget.

Overview of DELTA

Figure 1. Overview of DELTA. Top: Algorithm flowchart. Bottom: Example with target L∗=24L^* = 24.

Methodology: Delta

We describe DELTA (Diffusion Expected Length Targeting Algorithm), an inference-time method that steers masked diffusion language models toward a target character length. DELTA operates within the diffusion iterative denoising loop and requires no fine-tuning or architectural changes.

Preliminaries

Let VV denote the vocabulary and ℓ:V→N0\ell: V \to\mathbb{N}_0 the character-length function, which maps each token to the number of characters in its decoded string (with ℓ(v)=0\ell(v) = 0 for special tokens such as [MASK], [PAD], and [EOT]). Because tokens may decode to strings of different character lengths, fixing the number of output tokens does not in general fix the decoded character length. Given a prompt p=(p1,…,pn)\mathbf{p} = (p_1,\ldots,p_n), a masked diffusion LM generates by initializing an output buffer x=(x1,…,xm)\mathbf{x} = (x_1,\ldots,x_m) with all positions set to [MASK] and iteratively predicting tokens at masked positions until none remain. At each denoising step, the model produces a probability distribution qi∈Δ∣V∣\mathbf{q}_i \in\Delta^{|V|} over the vocabulary for each output position ii. These distributions, combined with the character-length function ℓ\ell, provide an estimate of the output’s character length at every point during generation, even before any tokens are committed.

Expected Character Length

We define the expected character length of the current output as L^=∑i=1mℓ^i\hat{L} = \sum_{i=1}^{m}\hat{\ell}_i, where ℓ^i\hat{\ell}_i depends on whether position ii has been committed (unmasked) or is still masked:

ℓ^i={ℓ(xi)if xi≠[MASK],∑v∈Vqi,v⋅ℓ(v)if xi=[MASK].(1)\hat{\ell}_i = \begin{cases} \ell(x_i) & \text{if } x_i \ne[\mathrm{MASK}],\\ \sum_{v \in V} q_{i,v} \cdot\ell(v) & \text{if } x_i = [\mathrm{MASK}]. \end{cases} \tag*{(1)}

For committed tokens, ℓ^i\hat{\ell}_i is the exact character length. For masked positions, ℓ^i\hat{\ell}_i is the probability-weighted expected character length under the model’s current prediction. We additionally mask out zero-length (special) tokens from logits before computing qi\mathbf{q}_i, ensuring they are never predicted.

Confidence Scoring

At each step, the algorithm must decide which masked positions to unmask. Following MaskGIT-style decoding (Chang et al., [4]), we assign a confidence score cic_i to each position and unmask those exceeding a threshold τ\tau. We use the maximum predicted probability as the confidence score:

ci=max⁡v∈Vqi,v.c_i = \max_{v \in V} q_{i,v}.

At each step, at least one position is always unmasked (the one with highest confidence), guaranteeing progress.

Length-Controlled Denoising

The algorithm combines three operations: length estimation, buffer adjustment, and token commitment. A schematic is shown in Algorithm 1.

Buffer adjustment. After computing L^\hat{L} and the confidence scores, the algorithm compares L^\hat{L} to the target L∗L^*. If L^>L∗\hat{L} > L^*, the buffer is too long and masked positions should be removed. We sort masked positions by ascending confidence (least confident first) and greedily remove masks until the expected length deficit is minimized:

nremove=arg⁡min⁡k ∣L^−∑j=1kℓ^σ(j)−L∗∣,(2)n_{\mathrm{remove}} = \underset{k}{\arg\min}\ \left|\hat{L} - \sum_{j=1}^{k}\hat{\ell}_{\sigma(j)} - L^*\right|, \tag*{(2)}

where σ\sigma is the sorted ordering. Removal is restricted to masks whose confidence falls below a threshold τlow\tau_{\mathrm{low}}, preventing deletion of positions the model is already confident about. If instead L^<L∗\hat{L} < L^*, new [MASK] tokens are inserted adjacent to the least-confident masked positions. The number of insertions is estimated as (L∗−L^)/ℓˉ(L^* - \hat{L})/\bar{\ell}, where ℓˉ\bar{\ell} is the current mean expected character length per token.

Token commitment. After any buffer adjustment, masked positions with confidence ci>τc_i > \tau are unmasked by replacing them with their argmax predictions. To prevent oscillation, where the algorithm alternates between inserting and removing masks without making progress, buffer adjustments are only permitted immediately after a commitment step. Since each commitment step unmasks at least

Algorithm 1 Length-Controlled Diffusion Decoding
Require: Diffusion LM θ\theta, prompt p\mathbf{p}, target length L∗L^*, initial buffer size mm, confidence threshold τ\tau
1: x←[p;MASKm;EOT]\mathbf{x} \leftarrow[\mathbf{p}; \mathrm{MASK}^m; \mathrm{EOT}]
2: can_adjust←true\mathit{can\_adjust} \leftarrow\mathrm{true}
3: while x\mathbf{x} contains MASK tokens do
4: q,c←ForwardPass(θ,x)\mathbf{q}, \mathbf{c} \leftarrow\mathrm{ForwardPass}(\theta, \mathbf{x})
5: L^←ExpectedCharLength(x,q)\hat{L} \leftarrow\mathrm{ExpectedCharLength}(\mathbf{x}, \mathbf{q})
6: if L^>L∗\hat{L} > L^* and can_adjust\mathit{can\_adjust} then
7: Remove lowest-confidence masks to reduce L^\hat{L} toward L∗L^*
8: can_adjust←false\mathit{can\_adjust} \leftarrow\mathrm{false}
9: else if L^<L∗\hat{L} < L^* and can_adjust\mathit{can\_adjust} then
10: Insert new masks at lowest-confidence positions
11: can_adjust←false\mathit{can\_adjust} \leftarrow\mathrm{false}
12: else
13: Unmask positions where ci>τc_i > \tau (at least one)
14: can_adjust←true\mathit{can\_adjust} \leftarrow\mathrm{true}
15: end if
16: end while
17: return x\mathbf{x}

Table 1.

MethodLength MAELength <10Quality R-1Quality R-LQuality BF1G-Eval RelG-Eval ConG-Eval CohG-Eval Flu
XSum
DELTA+LLaDA2.399.60.2860.2170.8823.414.214.614.66
DELTA+Dream2.399.70.3330.2550.8903.174.144.764.82
LA-SF-AR5.284.20.2730.2040.8813.564.384.794.89
GPT-4o-mini13.148.40.2640.1940.8803.834.734.964.99
Lead-char-snap15.760.50.1660.1220.8501.804.723.083.31
TextRank14.550.10.1780.1300.8451.744.794.074.59
CNN/DailyMail
DELTA+LLaDA2.499.60.3630.2590.8764.044.594.584.49
DELTA+Dream3.099.30.4090.2810.8803.974.724.754.73
LA-SF-AR10.359.70.3990.2540.8794.294.614.944.96
GPT-4o-mini96.611.10.3930.2450.8774.534.864.995.00
BART-CNN104.75.70.4410.3060.8823.694.834.604.61
Lead-char-snap59.421.40.3630.2300.8682.854.924.224.40
TextRank17.332.40.3560.2350.8662.404.803.384.07
arXiv
DELTA+LLaDA2.898.40.4430.2340.8373.714.354.434.39
DELTA+Dream3.198.30.4310.2360.8533.094.423.603.78
LA-SF-AR44.922.90.4660.2500.8453.984.284.784.98
GPT-4o-mini112.23.70.4320.2170.8394.354.654.975.00
Lead-char-snap96.57.00.3510.1830.8372.224.994.114.27
TextRank46.912.00.4060.2120.8372.524.553.003.19

Table 1 (PDF p. 4). Results on XSum, CNN/DailyMail, and arXiv, ordered by increasing reference summary length. MAE is mean absolute character-length error; <10 is the percentage of outputs within 10 characters of the target. R-1/R-L = ROUGE-1/L; BF1 = BERTScore F1. G-Eval scores (1–5 Likert) use GPT-5-mini as judge (Liu et al., 2023). Best values per column within each dataset are bolded. BART-CNN is included only for CNN/DailyMail as it was fine-tuned on that dataset.

one token, this guarantees that the sequence state changes between consecutive adjustments, breaking potential cycles. The algorithm terminates when no masked positions remain or a maximum iteration count is reached.

Evaluation

We evaluate on three datasets: CNN/DailyMail [9], a news summarization benchmark; XSum (Narayan et al., 2018), which targets single-sentence summaries; and arXiv [6], which contains scientific paper abstracts. For each reference summary, we set the target character length to the character length of the reference. Dataset statistics are provided in Appendix § 8.

Baselines. We compare DELTA with LLaDA-8B-Instruct (Nie et al., 2025) and Dream-7B-Instruct (Ye et al., 2025) against: TextRank (Mihalcea and Tarau, 2004); Lead-char-snap, which selects the maximum number of initial source sentences that fit within the target character budget; BART fine-tuned on CNN/DailyMail [13]; GPT-4o-mini prompted with the target character count; and LA-SF-AR (Retkowski and Waibel, 2025) using Llama-3.1-8B-Instruct. Full baseline and hyperparameter details are in Appendix § 16. We additionally evaluate DAEDAL (Li et al., 2026), a training-free variable-length diffusion decoding method, using the same character-target prompts as the other generative methods. Results are reported in Appendix § 11. To isolate the contribution of dynamic buffer adjustment, we also evaluate a fixed-buffer ablation using the same diffusion models, prompts, confidence schedules, and initialization as DELTA, but with mask insertion and removal disabled. Results are reported in Appendix § 10.1

Metrics. We report mean absolute character-length error (MAE) and the percentage of outputs within 10 characters of the target, along with summary quality via ROUGE-1, ROUGE-L (Lin, 2004), and BERTScore F1 (Zhang et al., 2020). We additionally evaluate using LLM-as-a-judge following the G-Eval framework (Liu et al., 2023), which scores summaries on relevance (Rel), consistency (Con), coherence (Coh), and fluency (Flu) on a 1–5 Likert scale. We use GPT-5-mini as the judge; we additionally report GPT-4o-mini scores in the Appendix. All prompts are listed in Appendix § 15.

Human evaluation. We additionally conduct a blind human evaluation on 50 CNN/DailyMail examples comparing DELTA+Dream, LA-SF-AR, and GPT-4o-mini. Two annotators independently rate anonymized and randomly ordered outputs on relevance, consistency, coherence, and fluency using the same 1–5 scale as the G-Eval evaluation. Full results and agreement statistics are reported in Appendix § 9.

Runtime. We report wall-clock inference time per summary for DELTA and the non-API baselines. Values are measured as mean ±\pm standard deviation over generated summaries; full results are reported in Appendix § 12.

Results

Results are shown in Table 1. On all datasets, DELTA achieves the best length compliance by a wide margin. While baseline errors scale with summary length, DELTA maintains MAE below 3.1 characters even on arXiv, where references average 969 characters. LA-SF-AR, the strongest autoregressive baseline for length control, places 84.2% of outputs within 10 characters on XSum but only 22.9% on arXiv, whereas DELTA remains above 98% across all datasets.

On summary quality, DELTA remains competitive. On XSum, DELTA+Dream achieves the highest ROUGE and BERTScore values, while on arXiv it achieves the highest BERTScore. On CNN/DailyMail, DELTA+Dream is second only to the fine-tuned BART baseline on ROUGE and BERTScore. Autoregressive baselines generally receive higher GPT-5-mini G-Eval scores, but this advantage comes with substantially poorer character-length compliance. A complementary human evaluation on 50 CNN/DailyMail examples finds DELTA+Dream competitive with LA-SF-AR and GPT-4o-mini, with average ratings of 4.27, 4.32, and 4.31 respectively and 91.5% within-one-point annotator agreement; full results are reported in Appendix § 9. Additional analyses confirm that DELTA’s length-control gains come from dynamic buffer adjustment. Disabling mask insertion and removal in the fixed-buffer ablation substantially worsens character-length compliance while leaving summary quality broadly similar. Likewise, DAEDAL produces reasonable summaries but does not reliably satisfy specified character budgets. Full results are reported in Appendices § 10 and § 11.

Conclusion

We presented DELTA, a training-free inference-time method that steers masked diffusion language models toward exact character-length targets by estimating expected length from token distributions and dynamically adjusting the generation buffer. Applied to LLaDA and Dream, DELTA reduces character-length errors by an order of magnitude compared to strong baselines while maintaining competitive summary quality in both automatic and human evaluation. Future work could address inference speed, evaluate DELTA across languages and model scales, and extend the approach to other constrained generation tasks.

Limitations

DELTA has several limitations. First, inference is slow. Diffusion language models require multiple full-sequence forward passes during denoising and, unlike autoregressive models, cannot use standard KV caching due to their bidirectional attention ((Nie et al., 2025)). On XSum, DELTA requires 0.67–0.69 seconds per summary, compared with 1.43 seconds for LA-SF-AR. On CNN/DailyMail and arXiv, however, DELTA requires 8.59 and 20.75–22.80 seconds per summary, respectively, compared with 2.19 and 13.22 seconds for LA-SF-AR. Thus, the tighter character-length control comes with increased inference cost on longer inputs. Full runtime measurements are reported in Appendix § 12. Recent work on accelerating diffusion LM inference through approximate KV caching and guided decoding ([11]) may mitigate this cost.

Second, automatic quality evaluation remains sensitive to the choice of LLM judge. GPT-4o-mini and GPT-5-mini produce substantially different Consistency and Coherence scores for several methods, particularly extractive baselines, consistent with known limitations of LLM-based judges ([5]). Our human evaluation provides additional evidence that DELTA’s length control does not cause a substantial quality regression, but it is limited to 50 CNN/DailyMail examples and two annotators. Larger human evaluations across datasets would provide stronger evidence. Third, our evaluation is limited to English-language summarization and to two diffusion models in the 7–8B parameter range. Both LLaDA and Dream are predominantly trained on English data and perform poorly on other languages in our informal testing. The effectiveness of character-level length estimation also depends on the token-to-character mapping of the underlying tokenizer, which varies across languages and writing systems. Although DELTA operates on output distributions rather than model-specific internal components, its effectiveness on substantially smaller or larger diffusion models remains to be validated.

Finally, the hyperparameters τ\tau and τlow\tau_{\mathrm{low}} vary across datasets and models, and we selected them via grid search on validation sets. We do not provide a sensitivity analysis or adaptive method for setting these thresholds, which limits the plug-and-play applicability of DELTA to new domains without tuning.

Acknowledgments

This work was funded by the German Federal Ministry of Research, Technology and Space (BMFTR) under the GenKI4Media project (grant 01MK25004E).

Dataset Statistics

XSumCNN/DMarXiv
Test examples11,33411,4906,440
Avg src length (words)3766845,906
Avg ref length (words)2155163
Avg ref length (chars)125312969

Table 2. Dataset statistics for the test splits used in our evaluation.

Human Evaluation

We conduct a blind human evaluation on 50 CNN/DailyMail examples comparing DELTA+Dream, LA-SF-AR, and GPT-4o-mini. Two annotators independently rated anonymized and randomly ordered outputs on relevance, consistency, coherence, and fluency using a 1–5 Likert scale.

MethodRelConCohFluAvg.
DELTA+Dream4.054.334.314.404.27
LA-SF-AR4.184.364.324.424.32
GPT-4o-mini4.264.324.264.424.31

caption: Blind two-annotator human evaluation on 50 CNN/DailyMail examples. Within-one-point annotator agreement is 91.5% across dimensions.

All three methods receive similar ratings across the four dimensions, providing additional evidence that DELTA’s substantially tighter character-length control does not come at the cost of a substantial degradation in perceived summary quality.

Fixed-Buffer Ablation

To isolate the effect of DELTA’s dynamic buffer adjustment, we compare against fixed-buffer diffusion decoding using the same model, prompt, confidence schedule, and initialization as DELTA, but with mask insertion and removal disabled. As shown in Table 4, disabling mask insertion and removal substantially reduces character-length compliance while leaving summary quality broadly similar.

DatasetMethodMAE<10R-1R-LBF1RelConCohFlu
XSumFixed-buffer LLaDA15.5538.40.2850.2180.8833.293.963.354.54
XSumDELTA+LLaDA2.399.60.2860.2170.8823.333.963.384.53
XSumFixed-buffer Dream15.7438.10.3340.2590.8922.813.813.094.46
XSumDELTA+Dream2.399.70.3330.2550.8902.913.833.184.47
CNN/DMFixed-buffer LLaDA27.3523.90.4230.2820.8784.214.693.974.86
CNN/DMDELTA+LLaDA2.499.60.3630.2590.8764.194.663.944.84
CNN/DMFixed-buffer Dream26.2024.60.4330.2880.8814.024.713.954.89
CNN/DMDELTA+Dream3.099.30.4090.2810.8804.024.663.924.86
arXivFixed-buffer LLaDA113.944.40.4480.2350.8384.724.764.394.89
arXivDELTA+LLaDA2.898.40.4430.2340.8374.654.694.314.89
arXivFixed-buffer Dream92.318.10.4320.2380.8543.934.293.644.47
arXivDELTA+Dream3.198.30.4310.2360.8533.944.283.624.45

Table 4. Fixed-buffer ablation. Disabling DELTA’s insert/remove operations substantially reduces character-length compliance while leaving summary quality broadly similar. G-Eval scores use GPT-4o-mini.

DAEDAL Comparison

DAEDAL targets open-ended variable-length diffusion generation rather than generation toward a specified decoded character budget. We nevertheless evaluate DAEDAL with LLaDA using the same character-target prompts as the other generative methods.

The full comparison is reported in Table 5. DAEDAL produces reasonable summaries but does not reliably satisfy specified character budgets. Across all three datasets, DELTA reduces character-length error by a large margin, supporting the need for explicit decoded character-length estimation and correction.

DatasetMethodnnMAE<10R-1R-LBF1RelConCohFluTime
XSumDAEDAL (LLaDA)11,33474.57.70.2510.1960.8452.823.412.814.252.50
XSumDELTA+LLaDA11,3342.399.60.2860.2170.8823.333.963.384.530.69
CNN/DMDAEDAL (LLaDA)11,490192.33.40.3670.2480.8584.004.433.674.695.94
CNN/DMDELTA+LLaDA11,4902.499.60.3630.2590.8763.734.403.644.708.59
arXivDAEDAL (LLaDA)1006337.51.00.1060.0730.7962.642.922.342.90306.56
arXivDELTA+LLaDA6,4402.898.40.4430.2340.8374.654.694.314.8922.80

Table 5. Comparison with DAEDAL. Time is wall-clock seconds per summary. DAEDAL on arXiv is evaluated on a 100-example diagnostic subset because of its high runtime cost; DELTA+LLaDA uses the full test set. G-Eval scores use GPT-4o-mini.

Runtime Analysis

Runtime results are reported in Table 6.

MethodXSumCNN/DMarXiv
DELTA+LLaDA0.69±0.430.69 \pm0.438.59±7.918.59 \pm7.9122.80±22.9822.80 \pm22.98
DELTA+Dream0.67±0.390.67 \pm0.398.59±7.988.59 \pm7.9820.75±21.7220.75 \pm21.72
LA-SF-AR1.43±0.331.43 \pm0.332.19±0.642.19 \pm0.6413.22±8.0413.22 \pm8.04
DAEDAL (LLaDA)2.50±7.542.50 \pm7.545.94±4.225.94 \pm4.22306.56±233.80306.56 \pm233.80*
BART-CNN–0.37±0.060.37 \pm0.06–
Lead-char-snap<0.01< 0.01<0.01< 0.01<0.01< 0.01
TextRank<0.01< 0.01<0.01< 0.01<0.01< 0.01

caption: Wall-clock runtime in seconds per summary (mean ± standard deviation). *DAEDAL on arXiv is measured on a 100-example diagnostic subset because of its high runtime cost. GPT-4o-mini is omitted because concurrent API calls make per-sample latency not directly comparable.

Lead Baseline Diagnostic

To distinguish our character-budgeted Lead-char-snap baseline from the conventional Lead-3 baseline, we additionally evaluate standard Lead-3 on CNN/DailyMail. Table 7 shows that Lead-3 achieves stronger extractive-summary quality but substantially violates the target character budget.

MethodMean charsMAE<10R-1R-LBF1RelConCohFlu
Lead-char-snap252.559.421.40.3630.2300.8682.854.924.224.40
Standard Lead-3455.4174.82.60.4000.2500.8693.514.964.674.75
DELTA+Dream311.13.099.30.4090.2810.8803.974.724.754.73

Table 7. Standard Lead-3 diagnostic on CNN/DailyMail. Lead-3 achieves stronger extractive-summary quality than Lead-char-snap but substantially violates the target character budget. G-Eval scores use GPT-5-mini.

LLM Judge Comparison

We compare the original GPT-4o-mini G-Eval scores with GPT-5-mini. The two judges differ substantially on consistency and coherence, particularly for extractive baselines. GPT-5-mini assigns substantially higher consistency scores to Lead-char-snap, which is more consistent with its extractive nature. We therefore use GPT-5-mini as the primary judge in the main paper; the complete comparison is reported in Table 8.

DatasetMethodJudgeRelConCohFlu
XSumDELTA+LLaDAGPT-4o-mini3.333.983.384.54
XSumGPT-5-mini3.414.214.614.66
XSumDELTA+DreamGPT-4o-mini2.903.843.184.48
XSumGPT-5-mini3.174.144.764.82
XSumLA-SF-ARGPT-4o-mini3.494.023.554.64
XSumGPT-5-mini3.564.384.794.89
XSumGPT-4o-miniGPT-4o-mini3.854.333.874.85
XSumGPT-5-mini3.834.734.964.99
XSumLead-char-snapGPT-4o-mini1.953.541.773.94
XSumGPT-5-mini1.804.723.083.31
XSumTextRankGPT-4o-mini1.703.691.694.15
XSumGPT-5-mini1.744.794.074.59
CNN/DMDELTA+LLaDAGPT-4o-mini4.194.663.944.84
CNN/DMGPT-5-mini4.044.594.584.49
CNN/DMDELTA+DreamGPT-4o-mini4.004.653.914.85
CNN/DMGPT-5-mini3.974.724.754.73
CNN/DMLA-SF-ARGPT-4o-mini4.484.754.144.97
CNN/DMGPT-5-mini4.294.614.944.96
CNN/DMGPT-4o-miniGPT-4o-mini4.794.894.344.98
CNN/DMGPT-5-mini4.534.864.995.00
CNN/DMLead-char-snapGPT-4o-mini3.004.423.074.50
CNN/DMGPT-5-mini2.854.924.224.40
CNN/DMTextRankGPT-4o-mini2.644.082.674.04
CNN/DMGPT-5-mini2.404.803.384.07
CNN/DMBART-CNNGPT-4o-mini3.844.633.784.74
CNN/DMGPT-5-mini3.694.834.604.61
arXivDELTA+LLaDAGPT-4o-mini4.644.694.334.89
arXivGPT-5-mini3.714.354.434.39
arXivDELTA+DreamGPT-4o-mini3.944.273.634.44
arXivGPT-5-mini3.094.423.603.78
arXivLA-SF-ARGPT-4o-mini4.874.834.614.98
arXivGPT-5-mini3.984.284.784.98
arXivGPT-4o-miniGPT-4o-mini4.984.964.915.00
arXivGPT-5-mini4.354.654.975.00
arXivLead-char-snapGPT-4o-mini3.984.763.784.58
arXivGPT-5-mini2.224.994.114.27
arXivTextRankGPT-4o-mini3.233.883.003.91
arXivGPT-5-mini2.524.553.003.19

Table 8. Comparison of GPT-4o-mini and GPT-5-mini as G-Eval judges.

Summarization Prompts

We use dataset-specific prompt templates for all generative methods (DELTA, LA-SF-AR, GPT-4o-mini, and DAEDAL). The {target_len} placeholder is replaced with the target character length and {source} with the source document.

CNN/DailyMail.

Summarize    the    following    article
in    approximately    {target_len}
characters:\n\n{source}

arXiv.

Write an abstract for the following
scientific paper in approximately
{target_len} characters:\n\n{source}

XSum.

Write a one-sentence summary of the
following article in approximately
{target_len} characters:\n\n{source}

G-Eval Evaluation Prompt

We use GPT-5-mini as the primary judge with the following system and user prompts. Scores range from 1 (poor) to 5 (excellent). We additionally evaluate the same outputs with GPT-4o-mini for the judge comparison reported in Appendix § 14.

System prompt.

You are an expert evaluator tasked with assessing the quality of text summaries.

Evaluate the provided summary against the source text based on the following criteria:

  • Relevance: Does the summary capture only the most important information from the source text?

  • Consistency: Is the summary factually consistent with the source text?

  • Coherence: Is the summary well-structured and logically organized?

  • Fluency: Is the summary written in clear, grammatical English?

Score each criterion on a scale from 1 to 5:

1: Poor – Fails to meet the criterion

2: Below Average

3: Average

4: Good

5: Excellent – Perfectly meets the criterion

Respond only in the following JSON format, where X is your score:

{"relevance": X, "consistency": X, "coherence": X, "fluency": X}

User prompt.

Evaluate the following summary against the source text based on the criteria defined above.

[Source Text]: ’{source}’

[Summary]: ’{summary}’

Respond only in the following JSON format, where X is your integer score for each criterion:

{"relevance": X, "consistency": X, "coherence": X, "fluency": X}

Implementation Details

TextRank. TF-IDF sentence scoring ((Mihalcea and Tarau, 2004)), truncated to the target character length at the nearest sentence boundary.

Lead-char-snap. Selects the maximum number of initial source sentences that fit within the target character budget. This avoids unnatural midsentence truncation while providing an extractive baseline with approximate character-length control.

GPT-4o-mini. Prompted with the target character count using the templates in Appendix § 15.

BART-CNN. BART-large fine-tuned on CNN/DailyMail (Lewis et al., [13]), beam size 4, length penalty 2.0.

LA-SF-AR. The zero-shot pipeline of Retkowski and Waibel (2025) using Llama-3.1-8B-Instruct with length approximation (μw=6.31\mu_w=6.31 chars/word), sample filtering (N=3N=3, temperature 0.7), and automated revisions (up to 3, ϵ=10%\epsilon=10\%).

DAEDAL. We evaluate DAEDAL (Li et al., 2026) with LLaDA using the same dataset-specific character-target prompts as the other generative methods. On arXiv, we evaluate DAEDAL on a 100-example diagnostic subset because of its high inference cost.

DELTA hyperparameters. On CNN/DailyMail and XSum: (τ,τlow)=(0.65,0.95)(\tau, \tau_{\mathrm{low}}) = (0.65, 0.95) for Dream, (0.65,0.90)(0.65, 0.90) for LLaDA. On arXiv: (0.90,0.82)(0.90, 0.82) for Dream, (0.71,0.71)(0.71, 0.71) for LLaDA. Selected via grid search on the validation set. The initial buffer size is determined using the average number of characters per token estimated from validation references. The maximum number of iterations is set to ⌊L∗/2⌋\lfloor L^*/2 \rfloor.

Reproducibility. We release our implementation, fixed-buffer ablation code, DAEDAL evaluation script, prompts, hyperparameters, random seeds, and evaluation scripts at https://github.com/DFKI-NLP/delta-diffusion.

Use of AI Assistants

Claude (Anthropic) was used to assist with writing and editing portions of the manuscript, generating figure components, and developing parts of the codebase. All outputs were reviewed and verified by the authors.

References

  1. [1]Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Ri​​anne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. In Proceedings of the 35th International Conference on Neural Information Processing Systems, NIPS ’21, Red Hook, NY, USA. Curran Associates Inc.arxiv.org/abs/2107.03006
  2. [2]Zeno Belligoli, Emmanouil Stergiadis, Eran Fainman, and Ilya Gusev. 2025. Controlling summarization length through eos token weighting. Preprint, arXiv:2506.05017.
  3. [3]Bradley Butcher, Michael O’Keefe, and James Titchener. 2025. Precise length control for large language models. Natural Language Processing Journal, 11:100143.DOI
  4. [4]Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. 2022. Maskgit: Masked generative image transformer. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).arxiv.org/abs/2202.04200
  5. [5]Khaoula Chehbouni, Mohammed Haddou, Jackie CK Cheung, and Golnoosh Farnadi. 2025. Neither valid nor reliable? investigating the use of LLMs as judges. In The Thirty-Ninth Annual Conference on Neural Information Processing Systems Position Paper Track.arxiv.org/abs/2508.18076
  6. [6]Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. A discourse-aware attention model for abstractive summarization of long documents. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 615–621, New Orleans, Louisiana. Association for Computational Linguistics.arxiv.org/abs/1804.05685
  7. [7]Yuxuan Gu, Wenjie Wang, Xiaocheng Feng, Weihong Zhong, Kun Zhu, Lei Huang, Ting Liu, Bing Qin, and Tat-Seng Chua. 2025. Length controlled generation for black-box LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16878–16895, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2412.14656
  8. [8]Junxian He, Wojciech Kryściński, Bryan McCann, Nazneen Rajani, and Caiming Xiong. 2020. Ctrlsum: Towards generic controllable text summarization. arXiv.arxiv.org/abs/2012.04281
  9. [9]Karl Moritz Hermann, Tomáš Kočiský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1, NIPS’15, page 1693–1701, Cambridge, MA, USA. MIT Press.arxiv.org/abs/1506.03340
  10. [10]Yuta Hitomi, Yuya Taguchi, Hideaki Tamori, Ko Kikuta, Jiro Nishitoba, Naoaki Okazaki, Kentaro Inui, and Manabu Okumura. 2019. A large-scale multi-length headline corpus for analyzing length-constrained headline generation model evaluation. In Proceedings of the 12th International Conference on Natural Language Generation, pages 333–343, Tokyo, Japan. Association for Computational Linguistics.
  11. [11]Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S. Abdelfattah, Jae-Sun Seo, Zhiru Zhang, and Udit Gupta. 2025. Flashldm: Accelerating diffusion language model inference via efficient kv caching and guided diffusion. arXiv preprint arXiv:2505.21472v2.
  12. [12]Juseon-Do, Jingun Kwon, Hidetaka Kamigaito, and Manabu Okumura. 2024. InstructCMP: Length control in sentence compression through instruction-based large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 8980–8996, Bangkok, Thailand. Association for Computational Linguistics.arxiv.org/abs/2406.11097
  13. [13]Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics.arxiv.org/abs/1910.13461

Paper details

vlnc:W_4zbahe52
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude

View statement
Looking for related papers…
Contents