ReAnchor: Reusing Rejected Verifier States for Diffusion Drafting

Introduction

Autoregressive large language models generate tokens sequentially, making decoding a major source of inference latency. Speculative decoding [17, 6] reduces this cost by using a lightweight drafter to propose multiple tokens and verifying them in parallel with the target model. Many existing methods use autoregressive drafters [21, 10, 19], whose token-by-token generation introduces sequential work into the drafting stage. Block diffusion drafters such as DFlash [7] remove this sequential dependence by predicting multiple draft tokens in a single forward pass.

This single-pass block generation comes at the cost of flexibility in how draft proposals are used and extended. Unlike autoregressive drafters, which can extend proposals one token at a time and stop when further drafting is unlikely to be worthwhile [14, 39], block diffusion drafters generate a fixed-length block determined by their training configuration. When verification encounters an early rejection, hidden states computed for the remaining draft positions are typically discarded. Conversely, when the entire block is accepted, no further proposals are available beyond its endpoint. Simply increasing the block length at inference time forces the drafter outside its training configuration and degrades acceptance performance [7]. Thus, block diffusion drafting leaves verifier computations underutilized when proposals are rejected early, while limiting draft coverage when the entire block is accepted.

Our observation is that verifier hidden states computed at discarded draft positions retain useful information for subsequent drafting Table 1. Although these states depend on rejected tokens and cannot be reused as valid target-model states, they can still serve as conditioning signals for the pretrained drafter. This suggests that otherwise discarded verifier computations can be leveraged for additional drafting without further target-model evaluations.

	GSM8K	HumanEval	MBPP	MATH-500
Agreement (%)	65.4	60.2	60.6	66.0

Building on this finding, we introduce ReAnchor, a training-free method that reuses verifier hidden states from the discarded post-rejection suffix to reanchor the same pretrained drafter at those later positions. The reanchored blocks are generated in parallel with the baseline draft, each preserving the drafter’s trained block length. To combine these blocks, ReAnchor matches their starting tokens to the baseline and attaches their continuations at the matched positions. By reanchoring at later positions Figure 1, these blocks provide alternative predictions for nearby tokens within the original block, while also extending draft coverage beyond its endpoint. All attached continuations are verified by the target model under their new prefixes, preserving lossless decoding.

With DFlash drafters, ReAnchor improves average acceptance length by 15.2% and decoding speed by 13.4% across four target models and eight benchmarks at $T = 0$. The largest average gains are observed on Gemma 4-12B, with improvements of 19.4% in acceptance length and 17.9% in decoding speed. These improvements also extend to three DFlash-based drafter variants, Domino, DSpark, and Draft-OPD, which employ different training and prediction mechanisms. Across 48 comparisons spanning two Qwen3 target models and eight benchmarks, ReAnchor improves average acceptance length by 11.8% and end-to-end decoding speed by 9.8% over their respective baselines. Component-wise ablations further show that both in-block refinement and block extension contribute to these gains.

Our contributions are summarized as follows:

• We introduce ReAnchor, a training-free method that reuses verifier hidden states as shifted anchors for block diffusion drafting.

• ReAnchor improves DFlash by refining predictions within the original block and extending draft coverage beyond it, achieving consistent gains in acceptance length and decoding speed across diverse settings.

• We demonstrate that ReAnchor can be directly applied to three DFlash-based variants, Domino, DSpark, and Draft-OPD, without additional training or variant-specific algorithmic modifications.

Preliminaries

We review DFlash-style block diffusion drafting and speculative decoding, introducing the notation and key concepts used throughout this work. Related work is discussed in Appendix § 9.

Block Diffusion Drafter

A block diffusion drafter generates multiple draft tokens in a single forward pass. The drafter receives two inputs: an anchor token, i.e., the bonus token that the autoregressive target model produces at the last verified position as a byproduct of its verification pass, and prefix context derived from the target model’s hidden states over the accepted prefix. Given these inputs, the drafter simultaneously

predicts a fixed number of tokens at the positions immediately following the anchor.

We now formalize this procedure. Let $p$ denote the target model and $q$ the diffusion drafter. We write $B$ for the block size: one anchor followed by $B - 1$ drafted tokens. All positions are absolute; $i$ denotes the anchor position. The anchor token $a_i = x_i \sim p(\cdot\mid x_{<i})$ is the target-generated bonus token, where $x_{<i}$ is the accepted prefix. The target model’s hidden states $\mathbf{H}_{<i} = (\mathbf{h}_j)_{j<i}$ are projected into the drafter’s hidden space by a learned linear projection $\Pi_\phi$ to form the prefix context $\mathbf{C}_{<i} = \Pi_\phi(\mathbf{H}_{<i})$. The drafter takes the anchor followed by $B - 1$ mask tokens $\mathbf{m}_{i+1:i+B-1}$ and produces a distribution over each masked position:

The superscript $[i]$ identifies the anchor.

Speculative Decoding

Given a draft block $\hat{x}_{i+1:i+B-1}^{[i]}$, the target model $p$ evaluates all proposed positions in a single forward pass. Verification then proceeds from left to right, accepting draft tokens as long as they match tokens independently sampled from the target distribution at each position. At the first mismatch, verification stops and accepts the target-sampled token.

Unlike standard probabilistic verification, which uses importance ratios and residual sampling, this sample-wise verification does not require evaluating proposal probabilities. It preserves the target distribution because each accepted token is sampled from $p$ under its verified prefix.

Let $\tau$ denote the average number of tokens accepted per round, including the target-generated token at the stopping position. The average decoding latency per accepted token is

The drafter therefore aims to generate low-cost proposals that match the target’s sampled trajectory for as many consecutive positions as possible.

Method

Revisiting Diffusion Drafting

Block diffusion drafting removes the sequential bottleneck of autoregressive drafters, but the fixed-length parallel block introduces constraints that do not arise in the sequential setting. We now revisit this formulation and examine where its fixed-block design leaves room for improvement. We present two observations that together motivate ReAnchor: (i) the native block endpoint is frequently the binding constraint on acceptance length and no proposal exists beyond it, yet naively extending the block degrades acceptance; (ii) verifier states computed over rejected draft suffixes retain substantial predictive information and can condition additional drafts without retraining.

Block-End Saturation and Naive Extension. The native draft length frequently limits the accepted prefix, but simply increasing the proposal length at inference time does not resolve this issue. Figure 2 shows the acceptance-length distribution

on representative math and code tasks for drafters trained with $B = 16$. Both distributions exhibit a clear concentration at the maximum length. All drafted tokens are accepted in 16.4% of rounds on MATH-500 and 12.7% on HumanEval. In these rounds, every drafted position is consumed and no further proposal is available. Full-block acceptance does not establish that the next token would also be accepted, but the concentration at the final bin shows that the block endpoint is the binding constraint in a substantial fraction of decoding rounds.

A direct solution is to increase the proposal length at inference time. We test this by evaluating drafters trained with $B = 16$ at $B = 24$ and $B = 32$ on Qwen3-8B across MATH-500, GSM8K, HumanEval, and MBPP. Despite the additional positions, the average accepted length decreases by 0.3 and 1.1 respectively compared to the native block size. The longer block forces the drafter to predict at offsets beyond its training range from the same anchor, under a masked input configuration it was not trained on. Extending coverage therefore motivates while preserving the native block size. We next examine whether the rejected portion of the preceding verification pass provides the conditioning information needed at these later starting points.

Predictive Information after Rejection. A rejection at position $r$ prevents the remaining draft suffix from being accepted, but does not render every subsequent draft token incorrect. Table 1 reports the fraction of draft tokens after $r$ that match the target model’s top-1 prediction. Across the evaluated tasks, 60–66% of these agree with despite following a rejected prefix. This indicates that the suffix is invalidated as a contiguous sequence rather than on a per-token basis, and that the discarded positions still carry substantial predictive information. Since the target model has already computed hidden states at these positions during verification, these states are available as conditioning signals for subsequent drafting at no additional target-model cost. Motivated by these observations, the next subsection introduces ReAnchor, which reuses these rejected-prefix verifier states as shifted anchors for the same pretrained drafter to generate additional fixed-length proposals. without retraining the drafter.

ReAnchor

ReAnchor constructs draft proposals by combining the standard baseline draft with additional shifted blocks conditioned on verifier hidden states discarded after rejection in the preceding round (Figure 3). These drafts are generated in parallel using the same pretrained drafter at its trained block length. Each shifted block is aligned with the baseline by matching its starting token, and its continuation is attached at the matched position. The resulting paths are merged into a token trie and jointly verified by the target model. This construction provides alternative predictions within the original block while extending draft coverage beyond its endpoint.

Parallel drafting from discarded verifier states. Section 3.1 shows that verifier states from rejected prefixes remain useful for drafting. ReAnchor reuses a subset of these states as conditioning points for the same pretrained drafter. The resulting shifted blocks improve predictions within the original block and extend the draft beyond its endpoint without retraining the drafter.

The baseline drafter uses the anchor $a_i$ and $\mathbf{C}_{<i}$ to generate $\hat{\mathbf{x}}^{[i]}_{i+1:i+B-1}$. Let $i'$ and $r$ denote the anchor and the first rejected position of the preceding round; the token accepted at $r$ becomes the current anchor, so $i=r$. We retain the otherwise discarded verifier states $\tilde{\mathbf{h}}_j$ over the rejected suffix $r \le j \le i' + B - 1$, and let $\tilde{x}_j$ denote the token decoded from $\tilde{\mathbf{h}}_{j-1}$ through the target LM head. Let

denote the selected offsets from the current anchor. For each $d \in\mathcal{D}$, we define the shifted anchor $\tilde{a}_{i+d} = \tilde{x}_{i+d}$ and its context $\tilde{\mathbf{C}}_{<i+d} = \Pi_{\phi}(\tilde{\mathbf{H}}_{<i+d})$, where $\tilde{\mathbf{H}}_{<i+d} = (\mathbf{H}_{<r},\tilde{\mathbf{h}}_r,\ldots,\tilde{\mathbf{h}}_{i+d-1})$, mirroring the baseline pair $(a_i,\mathbf{C}_{<i})$. The same pretrained drafter takes $\tilde{a}_{i+d}$ followed by $B-1$ masked positions and predicts them in parallel. The baseline and shifted blocks are

each consisting of one anchor and $B-1$ proposed tokens. The entry $g(d)_k$, $k \in\{0,\ldots,B-1\}$, nominally corresponds to absolute position $i+d+k$.

The $1+|\mathcal{D}|$ drafting calls are independent and can be batched together. Each block attends only to its own verifier context and masked input, and reusing states from the preceding verification pass requires no additional target evaluation. The subset $\mathcal{D}$ controls the number of additional drafting calls; in this work, we use every available offset, $\mathcal{D}=\{1,\ldots,i^\prime+B-i\}$. When the preceding round accepts its entire block, no states are retained, $\mathcal{D}$ is empty, and only the baseline draft is generated.

Token matching and continuation attachment. ReAnchor produces one baseline block and several shifted blocks, which must be combined into a single proposal set. A shifted block $g(d)$ is drafted from a reused anchor at offset $d$, so it would naturally be appended to the baseline at that position. However, correcting the rejected token changes the continuation that follows it, so the tokens drafted in $g(d)$ may now appear at a different position in the new baseline $g(0)$, or not at all. Rather than fixing $g(d)$ at its original offset $d$, we therefore search the baseline for a position at which $g(d)$ aligns and attach its continuation there.

We say that $g(d)$ has a positive overlap at baseline position $t$ if its anchor matches the baseline token there, i.e., $g(0)_t=g(d)_0$. We attach $g(d)$ at the earliest such position $t^\star$, so that its continuation follows the baseline prefix $g(0)_{0:t^\star}$ (Algorithm 1). This gives every matched shifted block a unique, deterministic attachment point without a secondary ranking rule, and in practice it selects a path with high accepted length (Appendix § 8.§ 1).

The overlapping prefix is represented only once in the concatenated path, and shifted blocks with no positive-overlap match are discarded; each shifted block therefore contributes at most one attached path. We collect the baseline and at most one attachment per shifted block into $\mathcal{Y}$ and merge them into a token trie. Since a token match only identifies a candidate attachment rather than validating its continuation, we jointly verify the trie in a single target forward pass using tree attention (Miao et al. [25]), so that each continuation is evaluated under its attached prefix.

Experiments

Experimental setup

Models and Evaluation. We evaluate ReAnchor with Qwen3 (Yang et al. [35]) models at 4B and 8B scales, Llama-3.1-8B (Graffatiori et al. (2024)) and Gemma-4-12B (Team et al. [33]). Experiments cover GSM8K (Cobbe et al. [11]), MATH-500 (Lightman et al. [22]), AIME25, HumanEval (Chen et al. [8]), MBPP (Austin et al. [4]), LiveCodeBench (Jain et al. [15]), MT-Bench (Zheng et al. [40]), and Alpaca (Taori et al. [32]). We report average acceptance length $\tau$ and end-to-end decoding speedup relative to autoregressive decoding with the target model. We evaluate on 100 examples per dataset, sampled with random seed 0, using the full set for AIME25 (30) and MT-Bench (80). In all experiments, ReAnchor uses every retained rejected-suffix state as a shifted anchor.

Drafter Setup. Unless otherwise specified, our main experiments and ablations use pretrained DFlash (Chen et al. [7]) checkpoints. We additionally apply ReAnchor to three DFlash-based variants to assess its compatibility with different training and prediction mechanisms. Draft-OPD (Lei et al. [16]) post-trains DFlash through on-policy distillation with error-position replay. Domino (Huang et al. [13]) adds a GRU-based causal correction head. DSpark (Cheng et al. [9]) uses a lightweight Markov head for sequential token prediction. Further details on the drafter variants are provided in Appendix § 6.

Algorithm 1 ReAnchor Proposal Construction

Require: Target context $\mathrm{C}_{<i}$, anchor $a_i$, block size $B$

Require: Retained verifier states $\widetilde{\mathrm{H}}$, source tokens $\widetilde{x}$, selected offsets $\mathcal{D}$ (§ 3.2)

Ensure: Proposal set $\mathcal{Y}$ for target verification

1: $\mathrm{C}^{(0)} \leftarrow\mathrm{C}_{<i},\quad g(0)_0 \leftarrow a_i$

2: for all $d \in\mathcal{D}$ do

3: $\mathrm{C}^{(d)} \leftarrow\Pi_{\phi}(\widetilde{\mathrm{H}}_{<i+d}),$

$g(d)_0 \leftarrow\widetilde{x}_{i+d}$

4: end for

5: for all $d \in\{0\}\cup\mathcal{D}$ in parallel do

6: $\mathbf{z}_{i+d} \leftarrow(g(d)_0,\mathbf{m}_{i+d+1:i+d+B-1})$

7: $(q_j^{[i+d]})_{j=i+d+1}^{i+d+B-1}\leftarrow q(\mathrm{C}^{(d)},\mathbf{z}^{[i+d]})$

8: Generate $g(d)_{1:B-1}$ from these distributions using the chosen drafting rule.

9: end for

10: $\mathcal{Y}\leftarrow\{g(0)\}$

11: for all $d\in\mathcal{D}$ do

12: $\mathcal{T}\leftarrow\{t\in\{0,\ldots,B-1\}:g(0)_t=g(d)_0\}$

13: if $\mathcal{T}\ne\varnothing$ then

14: $t^\star\leftarrow\min\mathcal{T}$

15: $\mathcal{Y}\leftarrow\mathcal{Y}\cup\{g(0)_{0:t^\star-1},g(d)_{0:B-1}\}$

16: end if

17: end for

18: Merge identical token prefixes in $\mathcal{Y}$ into a trie.

19: return $\mathcal{Y}$

Latency Measurement. We measure end-to-end wall-clock decoding latency with batch size one on a single NVIDIA H100 GPU. All compared methods use the same target model, drafter checkpoint, hardware, and generation settings. We exclude model loading and tokenization time while including measured latencies for both drafting and target verification for end-to-end latency evaluation. GPU execution is synchronized before timing, and we report results after warm-up runs. More details of the setup are provided in Appendix § 7.

Main Results

Acceptance length and decoding speed. At $T = 0$, ReAnchor improves both average acceptance length $\tau$ and end-to-end speedup in all 32 target–benchmark comparisons, achieving average gains of 15.2% and 13.4%, respectively (Table 2). The largest relative gains occur on Gemma 4-12B, with improvements of 19.4% in $\tau$ and 17.9% in speedup, despite its shorter baseline accepted prefixes than Qwen3. At $T = 1$, the improvements remain substantial, with $\tau$ increasing in 31 of 32 comparisons and speedup in 27, averaging 11.8% and 6.9%, respectively. The smaller gains under stochastic sampling may stem from increased token mismatches between independently sampled draft and target continuations, reducing the benefit of additional proposals relative to their verification cost. Overall, these results demonstrate the effectiveness of reanchoring across different target models and decoding settings.

Target model	Drafting method	GSM8K	MATH	AIME25	HumanEval	MBPP	LiveCode Bench	MT-Bench	Alpaca
Temperature = 0
Qwen3-4B	DFlash	6.24 / 4.19×	7.37 / 5.10×	6.36 / 4.35×	6.51 / 4.39×	5.81 / 4.13×	5.87 / 4.19×	3.46 / 2.44×	2.89 / 2.01×
Qwen3-4B	+ ReAnchor	7.51 / 5.03×	8.45 / 5.78×	7.20 / 4.90×	7.50 / 5.00×	6.78 / 4.78×	6.84 / 4.76×	3.89 / 2.68×	3.17 / 2.17×
Qwen3-8B	DFlash	6.34 / 4.39×	7.28 / 5.05×	6.38 / 4.41×	6.49 / 4.43×	5.71 / 4.06×	5.94 / 4.10×	3.42 / 2.40×	2.87 / 1.99×
Qwen3-8B	+ ReAnchor	7.45 / 5.03×	8.31 / 5.67×	7.11 / 4.72×	7.52 / 5.05×	6.68 / 4.69×	7.24 / 4.83×	3.82 / 2.57×	3.16 / 2.14×
Gemma 4-12B	DFlash	3.88 / 3.27×	4.97 / 4.14×	4.56 / 3.80×	4.48 / 3.70×	4.12 / 3.37×	4.07 / 3.27×	2.96 / 2.37×	2.77 / 2.16×
Gemma 4-12B	+ ReAnchor	4.79 / 3.99×	6.04 / 4.94×	5.30 / 4.39×	5.75 / 4.69×	5.04 / 4.05×	4.74 / 3.69×	3.36 / 2.65×	3.13 / 2.48×
Llama-3.1-8B	DFlash	4.42 / 2.71×	3.85 / 2.43×	3.82 / 2.43×	5.03 / 3.26×	5.02 / 3.13×	4.02 / 2.51×	3.85 / 2.42×	3.70 / 2.32×
Llama-3.1-8B	+ ReAnchor	4.98 / 3.11×	4.25 / 2.65×	4.25 / 2.63×	5.60 / 3.58×	5.65 / 3.52×	4.44 / 2.69×	4.28 / 2.62×	4.17 / 2.59×
Temperature = 1
Qwen3-4B	DFlash	4.90 / 3.54×	4.81 / 3.32×	3.70 / 2.62×	5.00 / 3.60×	4.56 / 3.26×	4.68 / 3.34×	2.66 / 1.86×	2.30 / 1.63×
Qwen3-4B	+ ReAnchor	5.64 / 3.80×	5.53 / 3.73×	4.24 / 2.83×	5.69 / 3.90×	5.16 / 3.49×	4.93 / 3.18×	2.86 / 1.94×	2.46 / 1.67×
Qwen3-8B	DFlash	4.62 / 3.36×	4.68 / 3.32×	3.57 / 2.53×	4.42 / 3.20×	4.03 / 2.93×	4.72 / 3.37×	2.43 / 1.75×	2.23 / 1.58×
Qwen3-8B	+ ReAnchor	5.35 / 3.79×	5.11 / 3.46×	3.94 / 2.71×	4.88 / 3.34×	4.52 / 3.20×	5.42 / 3.73×	2.63 / 1.77×	2.38 / 1.63×
Gemma 4-12B	DFlash	4.39 / 3.76×	4.88 / 4.12×	4.17 / 3.45×	4.19 / 3.46×	3.79 / 3.12×	3.62 / 3.00×	2.86 / 2.22×	2.59 / 1.81×
Gemma 4-12B	+ ReAnchor	5.45 / 4.48×	5.84 / 4.71×	4.92 / 3.92×	5.24 / 4.18×	4.70 / 3.72×	4.22 / 3.24×	3.22 / 2.38×	2.94 / 2.04×
Llama-3.1-8B	DFlash	2.92 / 1.80×	2.31 / 1.52×	1.93 / 1.32×	3.58 / 2.19×	3.53 / 2.31×	2.68 / 1.68×	2.38 / 1.47×	2.43 / 1.49×
Llama-3.1-8B	+ ReAnchor	2.97 / 1.76×	2.40 / 1.49×	1.93 / 1.18×	3.87 / 2.34×	3.86 / 2.49×	2.95 / 1.73×	2.48 / 1.46×	2.63 / 1.57×

Compatibility with drafter variants. We further evaluate ReAnchor on three DFlash-based variants, Domino, DSpark, and Draft-OPD, with different training and prediction mechanisms. At $T = 0$, ReAnchor improves both $\tau$ and end-to-end speedup in all 48 comparisons, with average gains of 11.8% and 9.8%, respectively (Table 3). At $T = 1$, the improvements remain consistent, increasing $\tau$ in all 48 comparisons and speedup in 45, with average gains of 11.5% and 8.1%. These results demonstrate that ReAnchor generalizes across different drafting mechanisms without additional training or variant-specific modifications.

Target model	Drafting method	GSM8K	MATH	AIME25	HumanEval	MBPP	LiveCode Bench	MT-Bench	Alpaca
Temperature = 0
Qwen3-4B	Domino	9.32 / 5.82×	8.20 / 5.34×	6.49 / 4.19×	6.84 / 4.50×	6.57 / 4.42×	5.82 / 3.83×	4.06 / 2.71×	3.56 / 2.33×
Qwen3-4B	+ ReAnchor	11.01 / 6.84×	9.31 / 5.94×	7.46 / 4.72×	7.73 / 4.99×	7.75 / 5.09×	6.75 / 4.29×	4.48 / 2.93×	3.90 / 2.50×
Qwen3-4B	DSpark	6.42 / 4.38×	6.20 / 4.27×	5.52 / 3.85×	5.70 / 4.10×	5.49 / 4.07×	5.18 / 3.67×	3.98 / 2.90×	3.61 / 2.46×
Qwen3-4B	+ ReAnchor	6.81 / 4.72×	6.47 / 4.41×	5.87 / 4.05×	6.11 / 4.35×	5.93 / 4.36×	5.62 / 4.00×	4.30 / 3.11×	3.96 / 2.65×
Qwen3-4B	Draft-OPD	7.10 / 4.82×	8.27 / 5.76×	6.76 / 4.75×	6.66 / 4.73×	6.23 / 4.48×	6.10 / 4.34×	3.54 / 2.57×	2.94 / 2.08×
Qwen3-4B	+ ReAnchor	8.35 / 5.66×	9.46 / 6.50×	7.78 / 5.33×	7.64 / 5.33×	7.27 / 5.17×	7.09 / 4.90×	3.91 / 2.78×	3.20 / 2.22×
Qwen3-8B	Domino	9.60 / 6.32×	8.39 / 5.38×	6.60 / 4.20×	7.20 / 4.72×	6.69 / 4.37×	6.22 / 4.09×	4.06 / 2.56×	3.56 / 2.28×
Qwen3-8B	+ ReAnchor	11.01 / 7.08×	9.50 / 6.00×	7.46 / 4.62×	8.32 / 5.29×	7.87 / 5.06×	6.95 / 4.22×	4.60 / 2.80×	4.02 / 2.50×
Qwen3-8B	DSpark	6.54 / 4.73×	6.30 / 4.45×	5.52 / 3.75×	5.94 / 4.14×	5.56 / 3.94×	5.47 / 3.68×	3.96 / 2.71×	3.71 / 2.54×
Qwen3-8B	+ ReAnchor	6.95 / 5.07×	6.59 / 4.60×	5.94 / 3.97×	6.29 / 4.42×	5.98 / 4.18×	5.87 / 3.97×	4.29 / 2.89×	4.03 / 2.73×
Qwen3-8B	Draft-OPD	7.23 / 5.24×	8.31 / 5.70×	6.93 / 4.81×	6.67 / 4.71×	6.23 / 4.45×	6.11 / 4.24×	3.52 / 2.45×	2.92 / 1.94×
Qwen3-8B	+ ReAnchor	8.46 / 5.99×	9.33 / 6.27×	7.88 / 5.33×	7.71 / 5.32×	7.18 / 5.01×	7.19 / 4.77×	3.87 / 2.61×	3.21 / 2.13×
Temperature = 1
Qwen3-4B	Domino	7.46 / 5.53×	5.66 / 4.22×	4.18 / 3.15×	5.59 / 4.17×	5.21 / 3.89×	4.77 / 3.14×	3.06 / 2.36×	2.72 / 2.07×
Qwen3-4B	+ ReAnchor	8.63 / 6.16×	6.47 / 4.68×	4.79 / 3.49×	6.44 / 4.67×	6.04 / 4.38×	5.52 / 3.50×	3.32 / 2.46×	3.01 / 2.21×
Qwen3-4B	DSpark	5.20 / 4.15×	4.58 / 3.61×	3.76 / 3.04×	4.53 / 3.62×	4.42 / 3.30×	4.80 / 3.43×	3.09 / 2.58×	3.04 / 2.49×
Qwen3-4B	+ ReAnchor	5.78 / 4.55×	4.99 / 3.99×	3.99 / 3.15×	5.02 / 3.89×	4.91 / 3.67×	5.25 / 3.92×	3.36 / 2.73×	3.23 / 2.54×
Qwen3-4B	Draft-OPD	5.49 / 4.63×	5.40 / 4.58×	3.95 / 3.38×	5.16 / 4.32×	4.83 / 4.10×	4.79 / 3.57×	2.76 / 2.39×	2.39 / 2.04×
Qwen3-4B	+ ReAnchor	6.48 / 5.25×	6.04 / 4.93×	4.64 / 3.81×	6.00 / 4.82×	5.58 / 4.55×	5.41 / 3.86×	2.94 / 2.44×	2.56 / 2.10×
Qwen3-8B	Domino	7.13 / 4.56×	5.88 / 3.71×	4.20 / 2.69×	5.38 / 3.43×	4.83 / 3.09×	4.93 / 3.05×	2.98 / 1.89×	2.72 / 1.71×
Qwen3-8B	+ ReAnchor	8.09 / 5.02×	6.66 / 4.04×	4.81 / 2.94×	5.98 / 3.67×	5.70 / 3.52×	5.82 / 3.41×	3.27 / 1.98×	3.01 / 1.80×
Qwen3-8B	DSpark	5.42 / 4.07×	4.78 / 3.45×	4.00 / 2.93×	4.92 / 3.68×	4.74 / 3.65×	4.60 / 3.21×	3.32 / 2.37×	3.09 / 2.14×
Qwen3-8B	+ ReAnchor	5.97 / 4.43×	5.14 / 3.59×	4.05 / 2.92×	5.31 / 3.99×	5.27 / 3.95×	4.85 / 3.16×	3.64 / 2.60×	3.34 / 2.33×
Qwen3-8B	Draft-OPD	5.17 / 3.74×	5.06 / 3.55×	3.86 / 2.82×	4.71 / 3.42×	4.39 / 3.16×	5.02 / 3.55×	2.52 / 1.81×	2.27 / 1.60×
Qwen3-8B	+ ReAnchor	6.24 / 4.35×	5.76 / 3.94×	4.24 / 2.95×	5.17 / 3.61×	4.96 / 3.43×	5.77 / 3.83×	2.73 / 1.86×	2.37 / 1.60×

Ablation Study

We analyze where the gains of ReAnchor come from: which draft positions benefit, how much is due to reusing rejected states versus token matching, and how the attachment rule affects performance. All ablations use Qwen3-4B with the DFlash drafter at $T = 0$.

In-Block Refinement and Block Extension. We separate the gains of ReAnchor into improvements within the native draft block and extensions beyond its endpoint. We consider two partial variants while keeping the same shifted anchors and drafter evaluations. In-block only uses predictions from shifted anchors only at positions up to $i+B-1$ and discards all predictions beyond the native block. Extension only retains the original DFlash predictions within $i+1:i+B-1$ and uses ReAnchor only to provide additional predictions after $i+B-1$. The full method uses both.

Table 4 shows that both components improve over the DFlash baseline. In-block refinement increases acceptance by providing later positions with predictions conditioned from closer verifier states. Block extension provides additional accepted tokens in rounds where verification reaches the native endpoint. Combining the two gives the largest improvement in both acceptance length and decoding speed, showing that ReAnchor benefits from improving existing draft positions as well as extending the draft beyond its original length.

Method	In-block	Beyond	$\Delta\tau$
DFlash	×	×	−
In-block only	✓	×	+0.54
Extension only	×	✓	+0.31

Analysis of token alignment. We next examine how often a shifted block $g(d)$ remains aligned with its original offset $d$ after a rejection. Correcting the rejected token can alter the continuation that follows it, so the anchor $g(d)_0$ may appear at a different position of the new baseline draft $g(0)$. Table 5 reports two statistics over shifted blocks whose anchor matches at least one position $t$, i.e., $g(0)_t = g(d)_0$. Shifted only counts blocks that do not match at $t = d$ but match at some $t \ne d$; Multiple counts blocks that match at more than one $t$. The two categories may overlap.

Task	Shifted only	Multiple
GSM8K	50.7	35.8
MATH-500	51.2	40.6
HumanEval	49.0	34.7
MBPP	45.5	38.0

Across the evaluated tasks, 45.5–51.2% of matched shifted blocks fall into the shifted-only category, indicating that enforcing the original offset would discard a substantial fraction of otherwise usable continuations. In addition, 34.7–40.6% admit multiple candidate attachment positions, showing that alignment is frequently ambiguous rather than fixed. These results motivate searching over baseline positions when attaching shifted blocks. We next quantify the effect of this design choice by comparing fixed-offset attachment with the full token-matching strategy of ReAnchor.

Ablation of attachment-site matching. We compare our default earliest-overlap rule with fixed-offset, random-match, and longest-prefix attachment on Qwen3-4B with DFlash (Table 6). All matching rules clearly outperform fixed-offset attachment ($7.51$–$7.56$ vs. $7.24$ in $\tau$), while differing from one another by at most 0.05 in $\tau$ and $0.02\times$ in speedup. Re-alignment itself thus matters more than the specific attachment site. We adopt the earliest rule because it matches the best results while

Attachment rule	Acceptance $\tau$	Speedup
Original offset	7.24	4.96×
Random match	7.51	5.16×
Earliest match	7.56	5.16×
Longest-prefix match	7.51	5.14×

being the simplest: a single-token comparison that yields a unique, deterministic attachment point.

Component-wise ablation of ReAnchor. We isolate the contributions of shifted drafting and token matching with a cumulative ablation (Table 7). Starting from DFlash, which uses only the native anchor and draft block, we first add shifted blocks drafted from the retained rejected-suffix states, attaching each block to the baseline at its original offset $t = d$. We then replace this fixed attachment with token matching, which attaches each block at the earliest positive-overlap position. All configurations share the same offsets $\mathcal{D}$, drafter evaluations, and verification rule; only the proposal assembly differs. Shifted drafting alone raises $\tau$ on every benchmark, from 5.38 to 5.87 on average, showing that rejected verifier states provide useful conditioning. Token matching further raises it to 6.22, accounting for roughly 40% of the total gain, as many shifted continuations become usable only at a position other than their original offset (Table 5).

Configuration	MATH	GSM8K	MBPP	HumanEval	MT-Bench	Alpaca
DFlash	7.37	6.24	5.81	6.51	3.46	2.89
+ Fixed-offset ReAnchor	8.03	6.89	6.49	7.04	3.75	3.04
+ ReAnchor	8.45	7.51	6.78	7.50	3.89	3.17

Conclusion

We introduced ReAnchor, a training-free method that reuses verifier hidden states discarded after rejection to reanchor block diffusion drafting at later positions. By drafting multiple fixed-length blocks in parallel and attaching their continuations through token matching, ReAnchor refines predictions within the original block and extends draft coverage beyond its endpoint, while preserving lossless decoding. It consistently improves acceptance length and decoding speed across target models and benchmarks, and applies directly to DFlash-based drafter variants, including Domino, DSpark, and Draft-OPD. These results show that otherwise discarded verifier computations can be effectively reused to improve speculative decoding.

Limitations

Like speculative decoding in general, ReAnchor is limited by how well the drafter aligns with the target model. Shifted blocks are drafted from verifier states conditioned on rejected tokens, so their benefit depends on how much the post-rejection drafts still agree with the target’s continuation. The gains therefore vary across tasks and decoding settings. At $T = 0$, the average speedup gain is 9.9% on MT-Bench and Alpaca, compared with 14.5% on math and code tasks. Under stochastic sampling ($T = 1$), independently sampled drafts agree less often with the target, and ReAnchor is slower than the baseline on four Llama-3.1-8B tasks and on LiveCodeBench with Qwen3-4B, where the additional verification cost outweighs the extra accepted tokens. In addition, ReAnchor has no retained states to reuse after a round in which the entire block is accepted, so it cannot extend the draft in consecutive full-acceptance rounds. Finally, all latency measurements use batch size one, where verifying additional trie tokens is nearly free; at larger batch sizes, where verification becomes compute-bound, the extra candidates may reduce the net speedup.

Variants of DFlash

This section provides brief descriptions and pseudocode for the baseline methods used in our experiments. We summarize the core inference procedures of DSpark, Domino and the post-training update of Draft-OPD.

DSpark. DSpark [9] combines parallel drafting with a low-rank Markov correction conditioned on the preceding sampled token. A calibrated confidence head estimates conditional acceptance probabilities, whose cumulative products give prefix-survival estimates. A hardware-aware scheduler selects the verification length by balancing expected token progress against profiled verification cost. Let $K \in\{0,\ldots,B-1\}$ be the selected verification length. For the default Markov variant, $\mathbf{h}^{q,[i]}_j$ and $\mathbf{o}^{[i]}_j$ denote the backbone feature and base logits for predicted token position $j$, following DSpark’s shifted input–output alignment. $E_M$ and $W_M$ parameterize the Markov correction, and $\operatorname{Conf}_{j-i}$ includes position-wise calibration. We summarize scheduling at the block level: SchedulePrefix uses the hardware profile and current verification workload, extends verification prefixes while estimated throughput improves, and stops at the first non-improving extension.

Algorithm 2 DSpark: drafting and confidence-scheduled verification

1. Input: $p$; $q$; $C_{<i}$; $x_i$; $B-1$; hardware profile and current scheduler state.

2. Compute $(\mathbf{h}^{q,[i]}_j,\mathbf{o}^{[i]}_j)_{j=i+1}^{i+B-1}$ in parallel.

3. Set $\hat{x}^{[i]}_i \leftarrow x_i$.

4. for $j=i+1,\ldots,i+B-1$ do

5. $\mathbf{q}^{[i]}_j \leftarrow\operatorname{softmax}(\mathbf{o}^{[i]}_j+W_ME_M(\hat{x}^{[i]}_{j-1}))$.

6. $\mathbf{c}^{[i]}_j \leftarrow\operatorname{Conf}_{j-i}(\mathbf{h}^{q,[i]}_j,E_M(\hat{x}^{[i]}_{j-1}))$.

7. Sample $\hat{x}^{[i]}_j \sim\operatorname{Cat}(\mathbf{q}^{[i]}_j)$.

8. end for

9. For $j=i+1,\ldots,i+B-1$, compute $\mathbf{a}^{[i]}_j \leftarrow\prod_{t=i+1}^{j}\mathbf{c}^{[i]}_t$.

10. $K \leftarrow\operatorname{SchedulePrefix}((\mathbf{a}^{[i]}_j)_{j=i+1}^{i+B-1};$ hardware profile, scheduler state$)$.

11. Verify only $\hat{x}^{[i]}_{i+1:i+K}$ with $p$.

12. return the accepted prefix and a target-issued token.

Domino. Domino [13] adds a lightweight causal correction head to a parallel DFlash backbone. A GRU summarizes the anchor and preceding draft tokens, and a low-rank head combines this state with the current backbone feature to correct the base logits. The backbone and full LM head are evaluated once per block; only correction and sampling are sequential. Let $\mathbf{h}^{q,[i]}_j$ be the prediction-aligned drafter feature and $\mathbf{o}^{[i]}_j=W_{\mathrm{LM}}\mathbf{h}^{q,[i]}_j$ the base logits, where $W_{\mathrm{LM}}$ is the frozen target LM head. The sequence $\mathbf{E}^{[i]}_{i:j-1}$ contains embeddings of $x_i$ and the previously sampled tokens $\hat{x}^{[i]}_{i+1:j-1}$; $\operatorname{GRU}(\mathbf{E}^{[i]}_{i:j-1})$ denotes the incrementally maintained recurrent state. The matrices $W_1,W_2$ form the low-rank correction head.

Algorithm 3 Domino: one inference round

1. Input: Target $p$; drafter $q$; context $C_{<i}$; anchor $a_i$; draft length $B-1$.

2. Compute $(\mathbf{h}^{q,[i]}_j,\mathbf{o}^{[i]}_j)_{j=i+1}^{i+B-1}$ in one parallel backbone and LM-head pass.

3. for $j=i+1,\ldots,i+B-1$ do

4. $\mathbf{s}^{[i]}_{j-1}\leftarrow\operatorname{GRU}(\mathbf{E}^{[i]}_{i:j-1})$.

5. $\Delta\mathbf{o}^{[i]}_j\leftarrow W_2\operatorname{SiLU}\left(W_1[\mathbf{h}^{q,[i]}_j;\mathbf{s}^{[i]}_{j-1}]\right)$.

6. $\mathbf{q}^{[i]}_j\leftarrow\operatorname{softmax}(\mathbf{o}^{[i]}_j+\Delta\mathbf{o}^{[i]}_j)$.

7. Sample $\hat{x}^{[i]}_j\sim\operatorname{Cat}(\mathbf{q}^{[i]}_j)$.

8. end for

9. Verify $\hat{x}^{[i]}_{i+1:i+B-1}$ with $p$ using $(\mathbf{q}^{[i]}_j)$.

10. return the accepted prefix and a target-issued token.

Draft-OPD. Draft-OPD [16] post-trains the drafter through on-policy distillation: the target supervises proposals generated by the current draft policy, rather than only fixed target-generated sequences. Target-assisted speculative decoding keeps the overall rollout stable while recording draft-block starting positions, proposals, and verification outcomes. Training returns to these saved block-start contexts to score the drafter’s proposals, including discarded suffixes. Let $\mathcal{I}_{\mathrm{acc}}$ and $\mathcal{I}_{\mathrm{rej}}$ collect accepted and discarded positions across the replayed blocks. ForwardKL computes the mean of $D_{\mathrm{KL}}(p^{[i]}_j\|q^{[i]}_j)$ over accepted positions. ReverseKL computes the weighted mean of $D_{\mathrm{KL}}(q^{[i]}_j\|p^{[i]}_j)$ over discarded positions, using block-position weights $w_{i,j}=\gamma^{j-i-1}$, where $0<\gamma\le1$, and normalizing by the sum of the discarded-position weights. Empty groups contribute zero. Inference remains unchanged.

Experiment Details

All implementations for DFlash and its variants are based on the publicly available implementations. We implement inference in PyTorch with Hugging Face Transformers and run both the target and draft models in bfloat16 using PyTorch SDPA. Due to the nature of ReAnchor that constructs a Algorithm 4 Draft-OPD: one on-policy training update of a DFlash drafter

1. Input: Prompts $\mathcal{X}$; frozen target $p$; current drafter $q$; $B-1$; learning rate $\eta$; decay factor $\gamma$.

2. Run target-assisted speculative rollouts with $q$; record block starts, contexts, proposals, and verification outcomes.

3. for each recorded draft block starting at $i$ do

4. Recompute $(q_j^{[i]})_{j=i+1}^{i+B-1}$ in parallel from the cached context and original anchor $(C_{<i}, x_i)$.

5. Score the recorded proposal with $p$ at each position $j$: $p_j^{[i]} \gets p(\cdot\mid x_{\leq i}, \hat{x}_{i+1:j-1}^{[i]})$.

6. end for

7. Collect $I_{\mathrm{acc}}$ and $I_{\mathrm{rej}}$; the latter includes the first rejection and all subsequent positions in its block.

8. $L_{\mathrm{acc}} \gets\operatorname{ForwardKL}(I_{\mathrm{acc}})$.

9. $L_{\mathrm{rej}} \gets\operatorname{ReverseKL}(I_{\mathrm{rej}};\gamma)$.

10. $\theta\gets\theta-\eta\nabla[(L_{\mathrm{acc}}+L_{\mathrm{rej}})/2]$.

Keep $p$ and the sampled proposals fixed.

trie for every decoding round, shifted proposals are not processed in Python but passed in a native C++17 backend, compiled with g++ -O3 as a shared library. The backend aligns each proposal at its longest matching prefix, merges the resulting paths into a token trie, and prepares node, position, and ancestor-attention data; Python transfers these inputs to the GPU for joint verification in one target-model pass. End-to-end latency includes per-round trie construction and preparation of verifier inputs, but excludes model loading, tokenization, and compilation of the C++ backend.

Additional experiments

Winning rate of earliest attachment selection

We evaluate the earliest positive-overlap rule with Qwen3-4B as the target and DFlash as the drafter. We use greedy decoding ($T = 0$) with a 128-token generation cap and evaluate 100 randomly shuffled examples per task (seed 0) from GSM8K, HumanEval, and MBPP. For each verifier round and shifted offset, we jointly verify paths from all positive-overlap sites together with the baseline path. A group is eligible if at least one attached path ties the maximum accepted draft length among all verified paths. The earliest site counts as a win when its path ties that maximum; ties are included. The table reports the number of eligible groups, the number won by the earliest site, and the corresponding fraction.

# Tested draft groups	# Earliest wins	Win rate
25,304	23,875	94.35%

Related Work

Speculative Decoding. Autoregressive LLMs achieve strong performance but are often noted for their slow decoding speed. While generation must proceed sequentially, verification of multiple tokens can be done in parallel through batched computation [17, 6]. Speculative decoding exploits this property, using a smaller model (the drafter) to propose tokens and the large model to verify them. Spending more computation on the drafter raises acceptance but slows drafting, so the drafter must align with the verifier under a tight compute budget. SpecInfer [25] introduced the tree attention mechanism, which is used broadly by subsequent methods as an almost plug-and-play component. Following Stern et al. [31], Medusa [5] brings the drafter into the verifier model using multiple decoding heads, since the computation over the prefix has already been done during verification. EAGLE [20] instead uses a feature-level autoregressive drafter so that the draft carries causality, and its successors add dynamic draft trees [19] and multi-layer feature fusion [21].

Diffusion Language Model. Masked diffusion language models generate tokens in any order and in parallel, committing multiple positions per forward pass. Capability has scaled from D3PM [3] through continuous-time masked formulations [24, 29, 30] to billion-parameter models such as LLaDA [26] and Dream [36]. Despite this progress, dLLMs have not replaced autoregressive models. Committing more tokens per step degrades quality, since tokens decoded in the same step do not condition on one another [23], and the arbitrary decoding order with bidirectional attention prevents KV cache reuse [34]. Block diffusion [2] interpolates between the two, using bidirectional attention within a block and causal attention across blocks. This recovers cache reuse at block granularity and yields a block-parallel proposal structure that closely mirrors speculative decoding [17, 6].

Speculative Decoding with Diffusion Draft Model. Several methods supply drafts through parallel masked prediction, either by sampling candidate paths from a diffusion model and scoring them before committing, or by adapting a small independent model to predict multiple positions at once [18, 1]. DFlash [7] instead conditions the drafter on the target’s hidden states, following EAGLE [20]. Subsequent methods weaken the conditional independence inside a block by imposing causal structure among drafted positions [13, 9]. D²SD [37] truncates the draft and re-runs the drafter from the accepted prefix, so that later positions are proposed under stronger conditioning. Without retraining the drafter, another line reallocates the verification budget, expanding the per-position marginals into a candidate tree verified in one pass. DDTree [28] maximizes expected acceptance length under a fixed node count, and later work makes the budget adaptive by folding verification cost into the objective [38, 27].