betaFeedback

WorksComputer Science

Beyond the Point Mass: Acceptance-Derived Weights and Distributional Losses for Speculative Decoding

Abstract

In speculative decoding, a lightweight draft model proposes tokens that the target model verifies in parallel, speeding up generation without changing the target distribution. D-PACE (Wu et al., 2026) trains such drafters with a loss that weights each position by its contribution to the expected accepted length. Both its weights and its cross-entropy (CE) loss use only the draft's probability of the recorded target token, treating the target as a point mass. Under rejection sampling, however, a sampled draft token is accepted with probability equal to the overlap of the draft and target distributions, one minus their total variation (TV). We derive D-PAL (Dynamic Position-wise Acceptance Loss) from the exact expected accepted length, using this overlap as the acceptance signal. This yields overlap-based position weights with log acceptance (LA) as the per-position loss, and the objective reduces to D-PACE for a point-mass target. Since the weights are detached, any per-position loss can replace LA in practice. We show that forward KL divergence and Rényi divergence of order 1/2 match or exceed LA, and that Rényi-1/2 provably bounds the true acceptance rate from above and below. Replacing CE with Rényi-1/2 improves acceptance length under both D-PACE's weights and ours. On six benchmarks with a Qwen3-4B target, D-PAL raises mean acceptance length over D-PACE from 4.50 to 4.57 at T=0 and from 4.20 to 4.28 at T=1, and consistently achieves higher throughput at both temperatures.

Introduction

Speculative decoding accelerates autoregressive generation without changing the target model’s output distribution [21, 6]: a lightweight draft model proposes tokens, and the target model verifies them in one parallel pass. DFlash drafts a whole block in one forward pass [7]. Since verification stops at the first rejection, early block positions matter most. DFlash trains with cross-entropy (CE) and a fixed position decay, and DSpark and H-Spec add a total variation (TV) term to CE under the same decay [9, 19]. LK Losses optimizes acceptance directly, with the negative log acceptance or an adaptive mix of Kullback–Leibler (KL) divergence and TV, but still uses fixed decay [33]. D-PACE (Dynamic Position-Aware Cross-Entropy) instead derives position weights from a surrogate of the expected accepted length [41]. SpecForge [35], Speculators [36], and TorchSpec [38] use it to train DFlash drafters.

D-PACE treats the target as a point mass on the recorded token yt∗y_t^*: it estimates acceptance by the draft’s confidence qt(yt∗)q_t(y_t^*) and fits that token with CE. Under standard rejection sampling, however, acceptance depends on the full target distribution ptp_t through the overlap $\tilde{\alpha}_t = \sum_v \min(p_{t,v}, q_{t,v}) = 1−TV⁡(pt,qt)1-\operatorname{TV}(p_t,q_t) (Leviathan et al., [21]). When the target is uncertain, confidence can underestimate this overlap by a wide margin. For example, if pt=qtp_t=q_t is uniform over kk tokens, the verifier always accepts a sampled draft token (α^t=1\hat{\alpha}_t=1), yet qt(yt∗)=1/kq_t(y_t^*)=1/k. CE likewise fits only the recorded token.

We propose D-PAL (Dynamic Position-wise Acceptance Loss), which we derive by differentiating the expected accepted length with the true acceptance signal 1−TV⁡(pt,qt)1-\operatorname{TV}(p_t,q_t). The derivation yields overlap-based position weights paired with a log acceptance (LA) loss, −log⁡α^t-\log\hat{\alpha}_t, as the local loss, which reduces to D-PACE when ptp_t is a point mass. Because D-PAL detaches the weights, we can swap LA for any local loss in practice. We write D-PAL[X] for D-PAL with local loss X. We show overlap bounds the acceptance of the argmax proposals that deployed DFlash uses (Proposition 1). Replacing CE with a distributional loss, Rényi divergence of order 1/21/2, raises mean acceptance length under both confidence and overlap weights. In our experiments, we show that LA is not the best local loss. It is not differentiable where a draft probability equals the target probability, and its gradient with respect to the draft logits does not shrink near qt=ptq_t=p_t, whereas the KL and Rényi gradients vanish there. Forward KL and Rényi-1/2 match or exceed LA, and Rényi-1/2, unlike KL, bounds overlap from both sides (Lemma 1). We also test TV-based losses, where pure TV collapses and CE+TV falls short, and analyze their gradients.

Our contributions are:

  1. Derivation from expected accepted length. Differentiating the exact expected accepted length with the overlap as the acceptance signal gives overlap-based weights paired with LA. Proposition 1 connects the overlap to the argmax verification used in deployment. We show D-PAL[LA] improves on D-PACE, raising mean acceptance length on Qwen3-4B from 4.50 to 4.54 at T=0T=0 and from 4.20 to 4.24 at T=1T=1 (Table 1).

MethodGSM8K SRGSM8K τ\tauMATH-500 SRMATH-500 τ\tauHumanEval SRHumanEval τ\tauMBPP SRMBPP τ\tauMT-Bench SRMT-Bench τ\tauAlpaca SRAlpaca τ\tauMean SRMean τ\tau
Baselines — Temperature T=0T = 0
DFlash3.204.503.945.463.334.673.064.281.973.151.772.712.884.13
D-PACE3.504.964.356.063.605.093.344.652.053.361.832.873.114.50
D-PAL weights
D-PAL[LA]3.505.034.326.063.655.153.384.702.043.411.842.903.124.54
D-PAL[KL]3.505.064.436.113.685.193.384.712.083.421.872.923.164.57
D-PAL[Rényi]3.535.074.396.103.665.173.314.682.083.441.862.923.144.56
D-PAL[TV]0.751.010.761.020.771.040.771.030.771.050.771.070.771.04
D-PAL[CE+TV]3.394.894.245.863.504.923.274.541.943.231.772.863.024.39
D-PAL[KL+TV]3.495.034.336.053.645.143.334.682.043.401.852.883.114.53
Baselines — Temperature T=1T = 1
DFlash2.994.293.254.803.074.402.854.091.843.041.712.712.623.89
D-PACE3.244.763.545.313.304.753.034.411.903.181.752.822.794.20
D-PAL weights
D-PAL[LA]3.284.783.525.263.374.813.054.481.913.261.762.852.824.24
D-PAL[KL]3.284.813.495.273.364.823.054.471.913.251.782.852.814.25
D-PAL[Rényi]3.344.843.525.353.374.823.104.481.943.281.812.892.854.28
D-PAL[TV]0.751.010.761.020.771.040.771.030.771.050.771.060.761.04
D-PAL[CE+TV]3.184.643.485.193.234.612.964.321.853.111.692.782.734.11
D-PAL[KL+TV]3.254.773.585.343.374.883.074.491.913.301.772.842.834.27

Table 1. D-PAL[Rényi], D-PAL[LA], and D-PAL[KL] beat D-PACE; D-PAL[TV] and D-PAL[CE+TV] fall short. Qwen3-4B target, three-layer DFlash drafts, B=16B = 16, 100K training examples, ρ=0.3\rho= 0.3 for D-PAL and 0.5 for D-PACE. SR is speedup over autoregressive decoding, τ\tau is acceptance length, and bold marks the best value.

  1. Local loss beyond log acceptance. The derivation points to LA, but LA is not differentiable where the draft matches the target. Keeping the same weights, we replace it with forward KL and with Rényi-1/2, and both reach the same or higher acceptance length (Table 1). We also show Rényi-1/2 bounds overlap from above and below (Lemma 1).

  1. Consistent gains over D-PACE and DFlash. In experiments across temperatures, block sizes, draft depths, target models at the 4B and 8B scale, and thinking modes, we show that D-PAL consistently achieves longer acceptance lengths and higher throughputs than both D-PACE (up to 5%) and DFlash (up to 11%).

Background

Speculative decoding with DFlash

In speculative decoding (Leviathan et al., [21]; Chen et al., [6]), the target model scores a block of BB draft proposals in one forward pass and accepts a contiguous prefix of length τ\tau.

DFlash (Chen et al., [7]) drafts with a small block-diffusion model that projects hidden states from a few target layers into the keys and values of every draft layer. A block of BB positions starts with an anchor, the last verified token, followed by D=B−1D=B-1 mask tokens. In one forward pass, the drafter predicts distributions q1,…,qDq_1,\ldots,q_D for all masked positions. Each qtq_t thus depends on the prefix but not on earlier proposals in the block, while the target’s ptp_t conditions on all tokens before tt.

In training, DFlash samples anchor positions in responses generated by the frozen target and predicts the next DD recorded tokens y1∗,…,yD∗y_1^*,\ldots,y_D^* from each anchor context. Since a proposal counts only if every earlier one is accepted, DFlash fits yt∗y_t^* with CE and weights positions by a fixed exponential decay,

LDFlash=∑t=1De−(t−1)/γ[−log⁡qt(yt∗)],(1)\mathcal{L}_{\mathrm{DFlash}}=\sum_{t=1}^{D}e^{-(t-1)/\gamma}\left[-\log q_t(y_t^*)\right], \tag*{(1)}

with γ=7\gamma=7 for B=16B=16. The frozen target output head also gives the full target distribution ptp_t at each position, but DFlash does not use it.

Expected accepted length

Let L=τ−1L=\tau-1 count the accepted draft tokens, and let αt=Pr⁡(L≥t∣L≥t−1)\alpha_t=\Pr(L\ge t\mid L\ge t-1) be the conditional acceptance probability at position tt. The tail-sum identity gives the exact expected accepted length [23, 41]

E[L]=∑k=1D∏i=1kαi.(2)\mathbb{E}[L]=\sum_{k=1}^{D}\prod_{i=1}^{k}\alpha_i. \tag*{(2)}

Each αt\alpha_t appears in every accepted prefix ending at k≥tk\ge t. For positive αt\alpha_t, its relative contribution and the gradient of E[L]\mathbb{E}[L] are

wtraw=αt∂E[L]∂αt=∑k=tD∏i=1i≠tkαi,∇E[L]=∑t=1Dwtraw∇log⁡αt.(3)w_t^{\mathrm{raw}}=\alpha_t\frac{\partial\mathbb{E}[L]}{\partial\alpha_t}=\sum_{k=t}^{D}\prod_{\substack{i=1\\i\ne t}}^{k}\alpha_i,\qquad\nabla\mathbb{E}[L]=\sum_{t=1}^{D}w_t^{\mathrm{raw}}\nabla\log\alpha_t. \tag*{(3)}

Every term is positive, so wtraww_t^{\mathrm{raw}} decreases with tt.

Position weights and local losses

Equation (3) suggests drafter objectives for a block of the form ∑t=1Dwtℓt\sum_{t=1}^{D}w_t\ell_t, with weights computed on target-generated training contexts. DFlash [7] uses a fixed exponential decay e−(t−1)/γe^{-(t-1)/\gamma} to emphasize early positions (Equation (1)). D-PACE [41] instead uses the draft’s confidence qt(yt⋆)q_t(y_t^\star) as a proxy for αt\alpha_t, smooths it to q~t=ρ+(1−ρ)qt(yt⋆)\widetilde{q}_t=\rho+(1-\rho)q_t(y_t^\star) with ρ∈(0,1)\rho\in(0,1), and builds detached weights paired with CE:

wtPACE=∑k=tD∏i=1kq~i,LD-PACE=∑t=1DwtPACE[−log⁡qt(yt⋆)].(4)w_t^{\mathrm{PACE}}=\sum_{k=t}^{D}\prod_{i=1}^{k}\widetilde{q}_i,\qquad\mathcal{L}_{\mathrm{D\text{-}PACE}}=\sum_{t=1}^{D}w_t^{\mathrm{PACE}}[-\log q_t(y_t^\star)]. \tag*{(4)}

With ρ>0\rho>0, an early low-confidence position cannot zero out all later weights.

D-PAL: Acceptance in the Weights, Fit in the Loss

Acceptance-derived weights from overlap

D-PACE’s unsmoothed confidence is overlap with a point mass on the recorded target token: ∑vmin⁡{δy⋆,v,qv}=q(y⋆)\sum_v\min\{\delta_{y^\star,v},q_v\}=q(y^\star). Standard rejection sampling instead compares the draft with the full target distribution. At a fixed context with target and draft distributions pt,qtp_t,q_t, the verifier accepts y∼qty\sim q_t with probability min⁡{1,pt(y)/qt(y)}\min\{1,p_t(y)/q_t(y)\}. Averaging over yy gives the one-token acceptance [21, 6]

α^t=∑vmin⁡{pt,v,qt,v}=1−TV⁡(pt,qt),TV⁡(p,q):=12∥p−q∥1.(5)\widehat{\alpha}_t=\sum_v\min\{p_{t,v},q_{t,v}\}=1-\operatorname{TV}(p_t,q_t),\qquad\operatorname{TV}(p,q):=\frac{1}{2}\lVert p-q\rVert_1. \tag*{(5)}

D-PAL applies the weighting rule of Equation (3) to this overlap. On target-generated sequences, the product of overlaps through position kk approximates the chance that sampled proposals survive to kk. As in D-PACE, we smooth only inside the products, st=ρ+(1−ρ)α^ts_t=\rho+(1-\rho)\widehat{\alpha}_t, recompute the weights from the current draft, and detach them:

wt=∑k=tD∏i=1ksi,LD-PAL=∑t=1Dwtℓt,(6)w_t=\sum_{k=t}^{D}\prod_{i=1}^{k}s_i,\qquad\mathcal{L}_{\mathrm{D\text{-}PAL}}=\sum_{t=1}^{D}w_t\ell_t, \tag*{(6)}

where ℓt\ell_t is the local loss chosen in Section 3.2. Equivalently,

wt=(∏i≤tsi)(1+∑k>t∏t<i≤ksi).(7)w_t=\left(\prod_{i\le t}s_i\right)\left(1+\sum_{k>t}\prod_{t<i\le k}s_i\right). \tag*{(7)}

the smoothed acceptance through tt times the continuation value after tt (Figure 1).

D-PACE and D-PAL decomposition into acceptance through $t$, local loss at $t$, and continuation after $t$

Figure 1. D-PAL uses the full target distribution for position weighting and, with KL or Rényi, for local fitting. Top: D-PACE weights by confidence and fits one token with CE; faded bars are target mass CE ignores. Bottom: D-PAL weights by overlap and, with a distributional loss such as KL or Rényi, fits qtq_t to all of ptp_t. Each weighted term wtℓtw_t\ell_t is the product of the acceptance through tt (left), the local loss at tt (center), and the continuation value after tt (right).

The SGLang DFlash backend proposes the draft argmax y^t=arg⁡max⁡vqt,v\hat{y}_t = \arg\max_v q_{t,v} at both temperatures, so proposals are never sampled from qtq_t. Overlap still bounds the acceptance of this proposal.

Proposition 1. (i) Let Δt\Delta_t be the gap between the largest and second-largest target probabilities. If TV⁡(pt,qt)<Δt/2\operatorname{TV}(p_t,q_t) < \Delta_t/2, then y^t=arg⁡max⁡vpt,v\hat{y}_t = \arg\max_v p_{t,v}. (ii) pt(y^t)≥max⁡vqt,v−TV⁡(pt,qt)p_t(\hat{y}_t) \ge\max_v q_{t,v} - \operatorname{TV}(p_t,q_t).

At T=0T = 0, the verifier accepts y^t\hat{y}_t if it matches the target argmax, which (i) guarantees whenever the target margin exceeds 2TV⁡2\operatorname{TV}. At T=1T = 1, SGLang’s target-only verification accepts it with probability pt(y^t)p_t(\hat{y}_t) (SGLang Contributors [34]), which (ii) bounds from below (proof in Appendix § 8.§ 2).

Log acceptance loss and beyond

D-PAL[LA]. Equation (3) pairs each weight with ∇log⁡αt\nabla\log\alpha_t, so gradient ascent on E[L]\mathbb{E}[L] matches gradient descent on the log-acceptance (LA) loss ℓt=−log⁡α^t\ell_t = -\log\hat{\alpha}_t, which gives D-PAL[LA]. For a point-mass target, α^t=qt(yt∗)\hat{\alpha}_t = q_t(y_t^*), so LA becomes CE, as LK Losses notes (Samarin et al. [33]), and D-PAL[LA] reduces to D-PACE. At ρ=0\rho= 0, gradient descent on D-PAL[LA] follows gradient ascent on the overlap-based prefix-sum surrogate (Appendix § 8.§ 3); as in D-PACE, we keep the floor ρ>0\rho> 0 as a regularizer.

However, LA is not differentiable where a draft probability equals the corresponding target probability. Its gradient is the TV gradient scaled by 1/α^t1/\hat{\alpha}_t. With respect to the draft logits, the TV gradient is 1qt⊙(σt−Eqt[σt])\frac{1}{q_t} \odot(\sigma_t - \mathbb{E}_{q_t}[\sigma_t]) with σt,v=sign⁡(qt,v−pt,v)\sigma_{t,v} = \operatorname{sign}(q_{t,v} - p_{t,v}) (Samarin et al. [33]). It does not vanish as qtq_t approaches ptp_t, and changes sign where qt,vq_{t,v} crosses pt,vp_{t,v}. On the other hand, KL and Rényi logit gradients vanish at qt=ptq_t = p_t (Appendix § 8.§ 1). Figure 2 shows a binary example. Because wtw_t is detached, any local loss can replace LA without changing the weight formula; in the following, we consider D-PAL[KL] and D-PAL[Rényi] as smooth alternatives to D-PAL[LA].

Plot comparing LA, KL, and Rényi losses as a function of draft parameter $x$

Figure 2. Near q=pq=p, LA’s slope stays near ±1\pm1 and flips sign, while KL and Rényi-1/2 flatten out. Binary example with target p=(0.5,0.5)p=(0.5,0.5) and draft q=(x,1−x)q=(x,1-x); slopes are with respect to xx, and all three losses are zero at x=0.5x=0.5.

D-PAL[KL]. Cross-entropy, the local loss of D-PACE, is smooth but fits one sampled token only. If y∼pty \sim p_t, then Ey[−log⁡qt(y)]=H(pt)+KL⁡(pt∥qt)\mathbb{E}_y[-\log q_t(y)] = H(p_t) + \operatorname{KL}(p_t\Vert q_t), and the logit gradient qt−eyq_t - e_y has mean qt−ptq_t - p_t. Up to the constant H(pt)H(p_t), CE is thus a one-sample estimate of forward KL, with label noise at every position. Our labels are sampled at positive temperature, so CE also fits a sharper target than the T=1T = 1 distribution ptp_t that sets acceptance. D-PAL[KL] fits this forward KL directly instead.

D-PAL[R ényi]. KL fits the full distribution, but it bounds overlap only from below and can be arbitrarily large while overlap is close to one. D-PAL[Rényi] instead replaces the minimum in overlap with a geometric mean, the Bhattacharyya coefficient BC⁡(p,q)=∑vpvqv\operatorname{BC}(p,q)=\sum_v\sqrt{p_vq_v}, whose scaled negative logarithm is the Rényi divergence of order 1/21/2 [30]:

D1/2(p∥q)=−2log⁡BC⁡(p,q),(8)D_{1/2}(p\Vert q)=-2\log\operatorname{BC}(p,q), \tag*{(8)}
LD-PAL[Reˊnyi]=∑t=1DwtD1/2(pt∥qt).(9)\mathcal{L}_{\mathrm{D\text{-}PAL}[\mathrm{Rényi}]}=\sum_{t=1}^{D}w_tD_{1/2}(p_t\Vert q_t). \tag*{(9)}

Its logit gradient is qt−rtq_t-r_t with rt,v∝pt,vqt,vr_{t,v}\propto\sqrt{p_{t,v}q_{t,v}} (Appendix § 8.§ 1), which varies continuously with the target–draft mismatch. The Bhattacharyya coefficient bounds overlap from above and below.

Lemma 1 (Overlap bounds). For categorical distributions p,qp,q, α(p,q)=1−TV⁡(p,q)\alpha(p,q)=1-\operatorname{TV}(p,q) satisfies

1−1−e−D1/2(p∥q)≤α(p,q)≤e−D1/2(p∥q)/2.(10)1-\sqrt{1-e^{-D_{1/2}(p\Vert q)}}\leq\alpha(p,q)\leq e^{-D_{1/2}(p\Vert q)/2}. \tag*{(10)}

This bound is the classical inequality between TV and the Bhattacharyya coefficient [12] (proof in Appendix § 8.§ 2). Taking negative logarithms gives D1/2/2≤−log⁡α^t≤−log⁡(1−1−e−D1/2)D_{1/2}/2\leq-\log\hat{\alpha}_t\leq-\log(1-\sqrt{1-e^{-D_{1/2}}}). D-PAL[Rényi] minimizes the smooth quantity D1/2D_{1/2} that controls both bounds, so driving it to zero drives LA to zero. This lower bound on overlap is never below the Bretagnolle–Huber bound from KL [39, 5] (Appendix § 8.§ 1).

D-PAL Algorithm

Algorithm 1 shows how to compute the D-PAL loss. Any of the previously discussed losses (LA, KL, Rényi, etc.) can be plugged in for ℓt\ell_t.

Algorithm 1: D-PAL loss computation

Input: target pp, draft qq, valid mask mm, smoothing ρ\rho, and chosen local loss ℓt\ell_t.

for each nonempty valid block bb:

Select proposals with mb,t=1m_{b,t}=1 in order; index them by t=1,…,Dbt=1,\ldots,D_b.

α^t←∑vmin⁡(pt,v,qt,v);st←ρ+(1−ρ)α^t\hat{\alpha}_t\leftarrow\sum_v\min(p_{t,v},q_{t,v});\quad s_t\leftarrow\rho+(1-\rho)\hat{\alpha}_t\quad for all tt

R~0←1;wDb+1←0\widetilde{R}_0\leftarrow1;\quad w_{D_b+1}\leftarrow0

for t=1,…,Dbt=1,\ldots,D_b: R~t←stR~t−1\widetilde{R}_t\leftarrow s_t\widetilde{R}_{t-1} ▷ Smoothed prefix product

for t=Db,…,1t=D_b,\ldots,1: wt←wt+1+R~tw_t\leftarrow w_{t+1}+\widetilde{R}_t ▷ Position weights

Detach w1:Dbw_{1:D_b}.

Lb←∑t=1Dbwtℓt\mathcal{L}_b\leftarrow\sum_{t=1}^{D_b}w_t\ell_t

return Mean of Lb\mathcal{L}_b within each sequence, then across sequences.

Experiments

Experimental setup

Models and training. We train DFlash drafters in SpecForge with frozen Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct targets [7, 22, 42, 13]. Within each comparison, methods share target-generated data, optimizer, schedule, and decoding (Appendix § 7). Table 1 uses the Qwen3-4B-Instruct-100K dataset released by the DFlash authors. Following D-PACE, we use three-layer drafts and block size B=16B = 16 [7, 41]. Request grouping uses these same checkpoints. For the ablations and other targets, we use SpecForge to sample a 36K-example ShareGPT subset; ablations use Qwen3-4B and B=16B = 16 unless varied.

Evaluation. We evaluate on GSM8K [10] and MATH-500 [15, 27] for math, HumanEval and MBPP [8, 3] for code, and first-turn MT-Bench (Zheng et al., 2023) and Alpaca [37] for chat. We run the six-benchmark evaluations with SGLang DFlash on one NVIDIA L40S, one request at a time, at T=0T = 0 and T=1T = 1. Our two primary evaluation metrics are speedup ratio (SR) (throughput divided by autoregressive throughput) and acceptance length τ\tau. Six-benchmark means weight benchmarks equally, and every result averages three evaluation seeds. Across seeds, the standard deviation of each mean τ\tau is 0 at T=0T = 0 (greedy decoding) and at most 0.02 at T=1T = 1.

Objectives. DFlash and D-PACE are the baselines. All D-PAL objectives share the weight rule of Equation (6) and differ only in the local loss. We compare D-PAL[LA], D-PAL[KL], and D-PAL[Rényi] with three losses that contain an explicit TV term: D-PAL[TV], D-PAL[CE+TV], and D-PAL[KL+TV] (see Appendix § 7.§ 2 for details). Table 1 uses ρ=0.3\rho= 0.3 for D-PAL, the best Qwen3-4B value in Table 7, and ρ=0.5\rho= 0.5 for D-PACE, the default that D-PACE reports as stable over ρ∈[0.3,0.7]\rho\in[0.3, 0.7] [41]. Ablations use ρ=0.5\rho= 0.5 unless varied.

ρ\rhoQwen3-4B, B=8B=8, T=0T=0Qwen3-4B, B=8B=8, T=1T=1Qwen3-4B, B=16B=16, T=0T=0Qwen3-4B, B=16B=16, T=1T=1Llama-3.1-8B, B=10B=10, T=0T=0Llama-3.1-8B, B=10B=10, T=1T=1
01.511.501.031.031.601.51
0.33.513.373.593.433.292.69
0.53.483.343.583.413.312.73
0.73.433.303.503.353.232.65
0.93.373.253.363.243.092.58

Table 7. Without the floor, D-PAL[Rényi] collapses; ρ∈{0.3,0.5}\rho\in\{0.3, 0.5\} works best. Mean τ\tau over six benchmarks as the floor ρ\rho varies.

D-PAL beats D-PACE

Table 1 compares all objectives on Qwen3-4B. D-PAL[Rényi] improves on D-PACE at both temperatures: mean τ\tau rises from 4.50 to 4.56 at T=0T = 0 and from 4.20 to 4.28 at T=1T = 1, and mean SR from 3.11 to 3.14 and from 2.79 to 2.85. The gain holds across benchmarks, since D-PAL[Rényi] has the higher τ\tau on all 12 benchmark–temperature pairs and the higher SR on 10. The other two D-PAL objectives, D-PAL[LA] and D-PAL[KL], also beat D-PACE on both metrics at both temperatures. When we batch requests with the same checkpoints, D-PAL[Rényi] leads in τ\tau in all ten settings and in throughput in nine (Table 2). In the separate 36K ablations at D-PACE’s own ρ=0.5\rho= 0.5, D-PAL[Rényi] still leads D-PACE by 0.08 in mean τ\tau at T=0T = 0 and 0.05 at T=1T = 1 (Table 3 and Table 7).

MethodK=1K=1 TPSK=1K=1 τ\tauK=2K=2 TPSK=2K=2 τ\tauK=4K=4 TPSK=4K=4 τ\tauK=8K=8 TPSK=8K=8 τ\tauK=16K=16 TPSK=16K=16 τ\tau
DFlash310.244.82423.175.48607.935.46990.605.491628.155.49
D-PACE320.136.06466.276.06669.646.041120.196.061714.126.06
D-PAL[Rényi]339.736.10477.466.13686.756.101094.146.131797.826.11
DFlash252.704.82342.244.86487.594.83788.094.821155.274.83
D-PACE274.515.21362.025.25515.485.23786.065.291215.915.27
D-PAL[Rényi]276.685.30370.045.32530.425.33826.425.351263.315.30

Table 2. D-PAL[Rényi] has the highest τ\tau at every batch size and the highest throughput in 9 of 10 settings. 128 MATH-500 prompts on Qwen3-4B with the 100K checkpoints; KK is the inference batch size (number of requests) and TPS is output tokens per second. Temperature T=0T=0 for the first three rows and T=1T=1 for the last three rows.

LossSignalρ\rhoT=0T=0T=1T=1
CEConfidence0.53.503.36
RényiConfidence0.53.583.42
CEOverlap0.33.503.36
RényiOverlap0.33.593.43

Table 3. Rényi beats CE under both confidence and overlap weights. Each pair fixes the weight signal and floor and changes only the loss. Confidence with CE is D-PACE; overlap gives D-PAL[CE] and D-PAL[Rényi]. Mean τ\tau over six benchmarks.

LA is not the best local loss. Under the same D-PAL weights, KL and Rényi reach the same or higher mean τ\tau than LA at both temperatures (Table 1). KL leads by 0.01 at T=0T = 0 and Rényi by 0.03 at T=1T = 1; Rényi is our default because it also bounds overlap from above (Lemma 1).

Pure TV and CE+TV fall short. D-PAL[TV] collapses to τ=1.04\tau= 1.04, consistent with the small TV gradient at initialization that KL Losses reports (Samari et al., 2026). Adding CE restores training, but D-PAL[CE+TV], a DSpark- and H-Spec-style mixture, trails D-PAL[KL] by 0.18 at T=0T = 0 and 0.14 at T=1T = 1 and falls below D-PACE. Its fixed TV term keeps the nondifferentiable points at qt,v=pt,vq_{t,v} = p_{t,v} (§ 3.2), and the toy example in Figure 3 (Appendix § 7.§ 2) shows that its CE gradient can raise TV to first order. D-PAL[KL+TV], using the adaptive mixture of losses proposed in LK Losses, shifts weight to KL at low-acceptance positions and stays within 0.04 of D-PAL[LA], D-PAL[KL], and D-PAL[Rényi].

Figure 3. A CE+TV descent step can raise TV. Negative weighted gradients of TV (a=0.9a = 0.9) and CE (b=0.1b = 0.1) at one parameter point of the three-token example, on one shared scale and rotated 25∘25^\circ for display. The two gradients have a negative inner product, and the mixture step −gΣ-g_\Sigma raises TV to first order (gTTgΣ<0g_T^{\mathsf{T}}g_\Sigma< 0).

Ablations separate the loss from the weights

Distribution fitting beats cross-entropy. Table 3 changes only the local loss, from CE to Rényi, under each weight signal. The switch raises mean τ\tau by 0.08 at T=0T = 0 and 0.06 at T=1T = 1 under D-PACE’s confidence weights (ρ=0.5\rho= 0.5), and by 0.09 and 0.07 under overlap weights (ρ=0.3\rho= 0.3). With Rényi at the same floor ρ=0.5\rho= 0.5, the two weight signals give similar mean τ\tau (3.58 for both at T=0T = 0; 3.42 and 3.41 at T=1T = 1; Table 3 and Table 7). We keep overlap as the weight signal because it matches rejection-sampling acceptance, reduces to D-PACE’s weights for a point-mass target, and bounds argmax acceptance (Proposition 1).

D-PAL weights beat fixed decay on baseline losses. In Table 4, we compare DFlash’s static weight decay against D-PAL’s weight schedule across four losses from the literature: CE, used in Chen et al. [7]; LA and KL+TV, used in Samarin et al. [33]; and CE+TV, used in Cheng et al. [9]. At T=0T = 0, D-PAL weights raise mean τ\tau over DFlash’s schedule by 0.26 for CE, 0.29 for LA, 0.33 for KL+TV, and 0.08 for CE+TV. Gains are smaller at T=1T = 1 but hold for every loss.

Local lossT=0T=0 StaticT=0T=0 D-PALT=1T=1 StaticT=1T=1 D-PAL
CE3.243.503.113.36
LA3.273.563.153.40
CE+TV3.403.483.273.34
KL+TV3.203.533.083.37

Table 4. D-PAL weights beat fixed decay for all losses from baselines. KL+TV follows the adaptive mixture of LK Losses with per-position λt\lambda_t. Static uses DFlash’s decay e−(t−1)/7e^{-(t-1)/7}; D-PAL uses ρ=0.3\rho=0.3 (0.5 for CE). Mean τ\tau over six benchmarks.

D-PAL[Rényi] leads at every block size and depth. Larger blocks help up to B=12B = 12: the mean τ\tau of D-PAL[Rényi] rises from B=8B = 8 to B=12B = 12 and levels off at B=16B = 16 (Table 5). Deeper drafts help both methods, and with five layers D-PAL[Rényi] still leads D-PACE by 0.06 at T=0T = 0 and 0.07 at $T = 1 ({xref}Table 6`).

D-PACED-PAL[Rényi]
BBT=0T=0T=1T=1T=0T=0T=1T=1
83.413.283.483.34
123.513.363.573.42
163.503.363.583.41

Table 5. D-PAL[Rényi] has the highest τ\tau at every block size. Mean τ\tau over six benchmarks; three-layer drafts.

The floor prevents collapse. Without the floor (ρ=0\rho=0), mean τ\tau falls to 1.03–1.60, likely because an early low-overlap position drives every later prefix product toward zero (Table 7). Any floor in {0.3, 0.5} works: the two stay within 0.04 of each other on every configuration.

Gains hold on other targets and inference settings

Gains transfer to 8B targets. D-PAL[Rényi] has the highest mean SR and τ\tau on both 8B targets at both temperatures (Table 8). The gain is largest on Llama, where D-PAL[Rényi] leads D-PACE by 0.13 at T=0T=0 and 0.12 at T=1T=1, while D-PACE falls below Static[CE+TV].

MethodMetricLlama-3.1-8B, B=10B=10, T=0T=0Llama-3.1-8B, B=10B=10, T=1T=1Qwen3-8B, B=16B=16, T=0T=0Qwen3-8B, B=16B=16, T=1T=1
D-PACESR2.471.772.592.37
τ\tau3.182.613.543.34
D-PAL[Rényi]SR2.571.842.652.40
τ\tau3.312.733.633.39

Table 8. D-PAL[Rényi] leads on both 8B targets in SR and τ\tau. Means over six benchmarks; three-layer drafts.

Gains hold in thinking mode. With thinking enabled, all three D-PAL losses match or improve D-PACE in mean τ\tau; D-PAL[Rényi] leads at T=0T=0 and D-PAL[KL] at T=1T=1 (Table 9).

MethodGSM8KMATH-500HumanEvalMBPPMT-BenchAlpacaMean
Temperature T=0T = 0
DFlash3.643.533.923.553.123.043.47
D-PACE3.943.824.143.963.283.173.72
D-PAL[LA]3.963.914.213.853.473.223.77
D-PAL[KL]3.983.904.203.963.463.233.79
D-PAL[Rényi]4.093.874.054.023.523.313.81
Temperature T=1T = 1
DFlash3.213.103.083.012.692.622.95
D-PACE3.423.343.293.202.842.733.14
D-PAL[LA]3.443.353.293.182.862.733.14
D-PAL[KL]3.473.383.333.232.872.793.18
D-PAL[Rényi]3.473.373.313.212.872.773.17

Table 9. With thinking enabled, D-PAL[Rényi] leads mean τ\tau at T=0T = 0 and D-PAL[KL] at T=1T = 1. DFlash is the reference checkpoint (protocol in Appendix § 7.§ 1).

Related Work

Parallel drafters. Medusa and DFlash propose every block position in parallel ([4, 7]). Their per-position marginals ignore tokens drafted earlier in the block, which LiLiCorr calls the marginals problem ([31]). EAGLE, Hydra, Domino, and DSpark condition later proposals on earlier tokens ([24, 26, 2, 17, 9]). LiLiCorr reranks adjacent top-kk candidates with a head co-trained with DFlash, DFlash2 adds a predecessor-conditioned selector, and DFlow carries verifier

sad representations between rounds [31, 45] (Inco AI 2026). H-Spec replaces the drafter-side KV cache with a hybrid Mamba–attention drafter trained with CE+TV and fixed decay [19]. These methods change what the drafter conditions on; orthogonally, D-PAL changes the objective that trains the marginals and can be combined with them.

D-PACED-PAL[Rényi]
LayersT=0T=0T=1T=1T=0T=0T=1T=1
33.503.363.583.41
53.753.563.813.63

Table 6. Deeper drafts help both methods; D-PAL[Rényi] leads at both depths. Mean τ\tau over six benchmarks.

Acceptance-aware position weighting. Several methods adapt position weights to acceptance. D-PACE weights CE by a surrogate of expected length built from draft confidence [41], following EAGLE-2’s observation that draft confidence tracks acceptance [25]. PARD2 instead uses the product of the target’s probabilities of the preceding recorded tokens [1]. Other methods tie supervision to where the draft first fails: GRIFFIN propagates top-kk alignment masks [16], Spec-AUF keeps CE through the first predicted failure [43], and Whalen et al. [40] study first-error supervision with an expected-length chain reward. VAT keeps full weight up to the first simulated rejection, restarts the decay there, and trains a verification head [14]. It also finds that weighting by the exact derivative of expected length, computed from confidence, trails D-PACE’s weights. D-PAL weights keep D-PACE’s weighting rule and floor and use overlap, 1−TV⁡(p,q)1 - \operatorname{TV}(p, q), in place of confidence.

Local losses for acceptance. Other works change the local loss. DistillSpec finds that the best divergence depends on the task and decoding strategy (Zhou et al., 2024), and DART fits target-to-draft KL with fixed decay [28]. Draft-QPD and AdaFlash train on draft-generated contexts with forward and reverse KL or clipped reverse KL [20, 29]. Closer to our setting, Bebop optimizes a normalized expected length built from TV acceptance [23]. LK Losses minimizes the negative log acceptance, the same term as our LA, or an adaptive KL–TV mixture, and notes that the negative log acceptance reduces CE for a point-mass target [33]. It keeps fixed position decay and leaves learnable or data-dependent weighting across draft heads to future work. DSpark and Uno mix CE or distillation with TV terms [9, 32]. D-PAL puts acceptance into the weights, so the loss can be a smooth distributional fit (§ 4).

Conclusion

We derived D-PAL from the expected accepted length in self-speculative decoding by replacing the point-mass confidence from D-PACE [41] with the full-distribution overlap. The derivation gives acceptance-derived position weights with log acceptance as the local loss. Furthermore, we show that other losses, such as KL and Rényi-1/2, match or exceed the derived log acceptance in average acceptance, while enjoying theoretical guarantees and smoothness, and show they can be combined with the detached D-PAL weight schedule. Experiments across six benchmarks show up to 5% longer acceptance lengths and higher throughput than D-PACE, and up to 11% higher than DFlash [7].

Limitations and future work. D-PAL weights assume that proposals within a block do not condition on each other, as in DFlash; drafters such as DSpark or EAGLE would need weights that model this dependence [9, 24, 26]. D-PAL weights could also train the parallel logits of DFlash2 or LiLiCorr [31] (Inco AI 2026).

Reproducibility Statement

Appendix § 7 gives training and evaluation settings. § 2.2–§ 3 and Algorithm 1 specify the objectives, with supporting derivations in Appendices § 8.§ 1 and § 8.§ 3. We provide the training code at https://github.com/Lucas-TY/D-PAL-SpecForge.

AI Use Statement

Generative AI tools assisted with drafting and revising text. Target language models generated the training responses as described in § 4. The authors are responsible for the final manuscript.

Experimental Settings and Loss Definitions

Training and evaluation

Table 1 uses the Qwen3-4B-Instruct-100K dataset released by the DFlash authors, which D-PACE also uses for its main Qwen3-4B experiments [7, 41]. Of its 99,987 rows, 99,171 are valid training sequences. We randomly sample ShareGPT prompts with SpecForge for the roughly 36K-example runs. All training responses in this paper are generated by the frozen target at temperature 0.7; Qwen3 uses non-thinking mode. Methods share data within each comparison (settings in Table 10). Results in Table 1–Table 2 average evaluation seeds 980406, 980407, and 980408. Across evaluation seeds, the standard deviation of each six-benchmark mean τ\tau is 0 at T=0T = 0 and at most 0.02 at T=1T = 1. Every configuration uses training seed 42.

Training
Optimizer / epochsAdamW / 6
Learning rate / schedule6×10−46 \times10^{-4}; cosine decay, 4% warm-up
Global batch / seed8 (Table 1 and Table 2), 4 (other runs); seed 42
Sequence / anchor limits3072 tokens / 512 anchors
Default block sizeQwen: B=16B = 16; Llama: B=10B = 10
Loss temperature1 (target and draft)
Evaluation
BackendSGLang 0.5.19 DFLASH
Evaluation temperaturesT=0T = 0, T=1T = 1
Generation limit2048 new tokens

Table 10. Training and evaluation settings. Each block includes one anchor token excluded from the loss. Both draft depths use the same three target feature layers.

Compute. We estimate that running all training experiments on NVIDIA H200 GPUs would require approximately 840 GPU-hours, including 138 GPU-hours for the eight 100K runs. We run all evaluations on NVIDIA L40S GPUs.

Target and draft distributions. The frozen target output head maps cached final hidden states to full-vocabulary logits; softmax gives ptp_t. At each recorded position, the draft predicts qtq_t from the anchor context and a masked block. It does not see the full prefix available to the target at later positions. Both loss softmaxes use temperature one; we generate labels at temperature 0.7.

Inference protocol. SGLang 0.5.19 DFLASH uses argmax draft proposals at both temperatures. At T=0T = 0, verification retains the target-argmax prefix; at T=1T = 1, it applies target-only stochastic acceptance and rejection correction, which accepts the deterministic proposal y^t\hat{y}_t with probability pt(y^t)p_t(\hat{y}_t) [34]. We disable top-kk and top-pp truncation and set SGLang’s single-token and accumulated acceptance thresholds to one. We measure τ\tau through verification; overlap serves as a training proxy.

The grouped-request experiment in Table 2 uses 128 fixed MATH-500 prompts and one L40S. All inference batch sizes, including K=1K = 1, use grouped-request inference, so its throughput is not directly comparable with the single-request results in Table 1. It evaluates the 100K checkpoints of DFlash, D-PACE, and D-PAL[Rényi] from Table 1.

Evaluation subsets. We use 128 examples each from GSM8K, MATH-500, and MBPP, 164 from HumanEval, and 80 first-turn MT-Bench prompts, with subset seed 42. Alpaca uses a fixed list of 128 single-turn prompts, supplied as alpaca_128.jsonl. We use the same prompts across methods and temperatures within each target.

Thinking-mode comparison. Table 9 uses a separate set of three-layer, B=16B = 16 checkpoints on the same six benchmark subsets, with thinking enabled. The D-PACE and D-PAL checkpoints use ρ=0.3\rho= 0.3. The context limit is 16384 tokens and the generation limit is 8192; both reasoning and final-answer tokens count toward the metrics. Each run uses one L40S and one active request. We retain responses that reach the generation limit.

Local losses and position weights

Table 11 lists the local fitting terms, with recorded target token yt∗y_t^* and training overlap α^t=1−TV⁡(pt,qt)\hat{\alpha}_t = 1 - \operatorname{TV}(p_t, q_t). All D-PAL objectives use the detached weights in Equation (6) and differ only in their local loss. Divergences and overlap use the full vocabulary.

ObjectiveLocal loss ℓt\ell_t
Cross-entropyℓCE,t=−log⁡qt(yt∗)\ell_{\mathrm{CE},t} = -\log q_t(y_t^*)
D-PAL[LA]−log⁡[1−TV⁡(pt,qt)]-\log[1 - \operatorname{TV}(p_t, q_t)]
D-PAL[KL]KL⁡(pt∥qt)\operatorname{KL}(p_t \Vert q_t)
D-PAL[Rényi]D1/2(pt∥qt)D_{1/2}(p_t \Vert q_t)
D-PAL[TV]TV⁡(pt,qt)\operatorname{TV}(p_t, q_t)
D-PAL[CE+TV]0.1ℓCE,t+0.9TV⁡(pt,qt)0.1\ell_{\mathrm{CE},t} + 0.9\operatorname{TV}(p_t, q_t)
D-PAL[KL+TV]λtKL⁡(pt∥qt)+(1−λt)TV⁡(pt,qt)\lambda_t\operatorname{KL}(p_t \Vert q_t) + (1 - \lambda_t)\operatorname{TV}(p_t, q_t)

Table 11. Local losses compared in this paper. D-PAL[KL+TV] computes λt=exp⁡(−3α^t)\lambda_t = \exp(-3\hat{\alpha}_t) for each block and position and detaches it during optimization.

DSpark’s 0.1LCE+0.9∥p−q∥10.1L_{\mathrm{CE}} + 0.9\lVert p - q\rVert_1 has twice the TV coefficient of our CE+TV loss. D-PAL[KL+TV] computes λt\lambda_t at each position, whereas LK Losses aggregates acceptance across sequence and batch before setting its hybrid coefficient (Samarin et al., [33]). Static weights use exp⁡(−(t−1)/γ)\exp(-(t - 1)/\gamma), with γ=4,5,6,7\gamma= 4, 5, 6, 7 for B=8,10,12,16B = 8, 10, 12, 16. D-PACE smooths confidence with ρ=0.5\rho= 0.5. D-PAL smooths α^t\hat{\alpha}_t with ρ=0.3\rho= 0.3 in Table 1 and with ρ=0.5\rho= 0.5 in the ablations unless varied; D-PAL[CE] uses ρ=0.3\rho= 0.3. We average blocks within sequences, then sequences, without normalizing by the weight sum.

Implementation. The two weight scans cost O(Db)O(D_b); vocabulary sums cost O(DbV)O(D_bV). We compute the Rényi loss in 32-bit floating point (FP32) using log-softmax and logsumexp.

Gradient conflict in TV mixtures. A fixed positive TV coefficient leaves a mixture nondifferentiable, and the auxiliary gradient can oppose the TV gradient when the terms share parameters (Du et al., [11]; Yu et al., [44]). Let gT=∇θTV⁡g_T = \nabla_\theta\operatorname{TV} and gF=∇θLFg_F = \nabla_\theta L_F. At differentiable points, a stochastic gradient descent step of size η\eta on a TV +bLF+_b L_F changes TV by

δTV⁡=−η(a∥gT∥22+bgT⊤gF)+o(η),(11)\delta\operatorname{TV} = -\eta\left(a\lVert g_T\rVert_2^2 + b g_T^\top g_F\right) + o(\eta), \tag*{(11)}

so the mixed step raises TV to first order when a∥gT∥22+bgT⊤gF<0a\lVert g_T\rVert_2^2 + b g_T^\top g_F < 0. For target p=(0.45,0.35,0.20)p = (0.45, 0.35, 0.20) and draft q=(0.55,0.01,0.44)q = (0.55, 0.01, 0.44), vary two draft logits and fix the third at zero. With hard-target CE for token 1 and (a,b)=(0.9,0.1)(a,b) = (0.9, 0.1), gT⊤(0.1gCE+0.9gT)<0g_T^\top(0.1g_{\mathrm{CE}} + 0.9g_T) < 0: descent on the mixed loss raises TV to first order (Figure 3). KL and TV gradients can also oppose, which can affect D-PAL[KL+TV]. For p=(0.4,0.5,0.1)p = (0.4, 0.5, 0.1) and q(θ)=softmax⁡(log⁡(0.1,0.6,0.3)+θ(0,1,−1))q(\theta) = \operatorname{softmax}(\log(0.1, 0.6, 0.3) + \theta(0, 1, -1)), dTV⁡/dθ=0.03d\operatorname{TV}/d\theta= 0.03 and dKL⁡/dθ=−0.10d\operatorname{KL}/d\theta= -0.10 at

θ=0\theta= 0: the KL and TV gradients oppose. With detached position weights, Equation (11) applies to the weighted sum of these gradients over positions.

Additional Derivations

Rényi fitting gradient

Fix pp and let q=softmax⁡(z)q = \operatorname{softmax}(z). Using ∂qj/∂zv=qj(1j=v−qv)\partial q_j/\partial z_v = q_j(\mathbf{1}_{j=v} - q_v),

∂BC⁡(p,q)∂zv=12(pvqv−qvBC⁡(p,q)),∂D1/2∂zv=−2BC⁡(p,q)∂BC⁡(p,q)∂zv=qv−rv.\frac{\partial\operatorname{BC}(p,q)}{\partial z_v} = \frac{1}{2}\left(\sqrt{p_vq_v} - q_v\operatorname{BC}(p,q)\right), \qquad\frac{\partial D_{1/2}}{\partial z_v} = -\frac{2}{\operatorname{BC}(p,q)}\frac{\partial\operatorname{BC}(p,q)}{\partial z_v} = q_v-r_v.

Here rv=pvqv/BC⁡(p,q)r_v = \sqrt{p_vq_v}/\operatorname{BC}(p,q) is the normalized geometric mean.

Comparison with KL. Rényi divergence is nondecreasing in order, so D1/2(p∥q)≤KL⁡(p∥q)D_{1/2}(p\mathbin{\|}q) \le\operatorname{KL}(p\mathbin{\|}q) [39]. Bretagnolle–Huber gives α(p,q)≥1−1−e−KL⁡(p∥q)\alpha(p,q) \ge1-\sqrt{1-e^{-\operatorname{KL}(p\mathbin{\|}q)}} [5]. For the same p,qp,q, Equation (10) gives an overlap lower bound at least as high as the KL bound.

Proofs of Proposition 1 and Lemma 1

Proof of Lemma 1. The upper bound follows from min⁡{pj,qj}≤pjqj\min\{p_j,q_j\} \le\sqrt{p_jq_j}. For the lower bound, set uj=pju_j=\sqrt{p_j} and vj=qjv_j=\sqrt{q_j}. Both have unit norm, and uTv=BC⁡(p,q)u^{\mathsf{T}}v=\operatorname{BC}(p,q):

2TV⁡(p,q)=∑j∣(uj−vj)(uj+vj)∣≤∥u−v∥2∥u+v∥2=21−BC⁡(p,q)2=21−e−D1/2(p∥q).\begin{aligned} 2\operatorname{TV}(p,q)=\sum_j\left|(u_j-v_j)(u_j+v_j)\right| \\ &\le\lVert u-v\rVert_2\lVert u+v\rVert_2=2\sqrt{1-\operatorname{BC}(p,q)^2}=2\sqrt{1-e^{-D_{1/2}(p\mathbin{\|}q)}}. \end{aligned}

Hence α(p,q)=1−TV⁡(p,q)≥1−1−e−D1/2(p∥q)\alpha(p,q)=1-\operatorname{TV}(p,q)\ge1-\sqrt{1-e^{-D_{1/2}(p\mathbin{\|}q)}}.

Proof of Proposition 1. Write p=ptp=p_t, q=qtq=q_t, Δ=Δt\Delta=\Delta_t, y^=arg⁡max⁡vqv\hat{y}=\arg\max_v q_v, and v⋆=arg⁡max⁡vpvv^\star=\arg\max_v p_v.

(i) Suppose y^=j≠v⋆\hat{y}=j\ne v^\star, so qj≥qv⋆q_j\ge q_{v^\star}. Then

2TV⁡(p,q)≥(pv⋆−qv⋆)+(qj−pj)=(pv⋆−pj)+(qj−qv⋆)≥Δ,2\operatorname{TV}(p,q)\ge(p_{v^\star}-q_{v^\star})+(q_j-p_j)=(p_{v^\star}-p_j)+(q_j-q_{v^\star})\ge\Delta,

which contradicts TV⁡(p,q)<Δ/2\operatorname{TV}(p,q)<\Delta/2. (ii) Since TV⁡(p,q)=∑v(qv−pv)+≥qy^−py^\operatorname{TV}(p,q)=\sum_v(q_v-p_v)_+\ge q_{\hat{y}}-p_{\hat{y}}, we have py^≥max⁡vqv−TV⁡(p,q)p_{\hat{y}}\ge\max_v q_v-\operatorname{TV}(p,q).

TV gradients and position weights

TV coefficients. Fix the target model and target-generated training contexts. Differentiating the prefix sum of training overlaps α^t=1−TV⁡(pt,qt)\hat{\alpha}_t=1-\operatorname{TV}(p_t,q_t) gives

∇θ∑k=1D∏i=1kα^i=∑t=1D[∑k=tD∏1≤i≤ki≠tα^i]∇θα^t.(12)\nabla_\theta\sum_{k=1}^{D}\prod_{i=1}^{k}\hat{\alpha}_i=\sum_{t=1}^{D}\left[\sum_{k=t}^{D}\prod_{\substack{1\le i\le k\\i\ne t}}\hat{\alpha}_i\right]\nabla_\theta\hat{\alpha}_t. \tag*{(12)}

Multiplying this coefficient by α^t\hat{\alpha}_t gives the relative-contribution form in Equation (3), evaluated at training overlaps.

Gradient of the smoothed prefix sum. For positive st=ρ+(1−ρ)α^ts_t=\rho+(1-\rho)\hat{\alpha}_t, direct differentiation of the smoothed prefix sum gives

∇θ∑k=1D∏i=1ksi=∑t=1D1−ρstwt∇θα^t.\nabla_\theta\sum_{k=1}^{D}\prod_{i=1}^{k}s_i=\sum_{t=1}^{D}\frac{1-\rho}{s_t}w_t\nabla_\theta\hat{\alpha}_t.

At ρ=0\rho=0, st=α^ts_t=\hat{\alpha}_t and gradient descent on D-PAL[LA] follows this direction; for ρ>0\rho>0, the factor (1−ρ)/st(1-\rho)/s_t replaces 1/α^t1/\hat{\alpha}_t, and we treat the floor as a regularizer, as D-PACE does.

References

  1. [1]Zihao An, Taichi Liu, Ziqiong Liu, Dong Li, Ruofeng Liu, and Emad Barsoum. PARD-2: Target-aligned parallel draft model for dual-mode speculative decoding. arXiv preprint arXiv:2605.08632, 2026. URL https://arxiv.org/abs/2605.08632.
  2. [2]Zachary Ankner, Rishab Parthasarathy, Aniruddha Nrusimha, Christopher Rinard, Jonathan Ragan-Kelley, and William Brandon. Hydra: Sequentially-dependent draft heads for Medusa decoding. arXiv preprint arXiv:2402.05109, 2024. URL https://arxiv.org/abs/2402.05109.
  3. [3]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. URL https://arxiv.org/abs/2108.07732.
  4. [4]Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. Medusa: Simple LLM inference acceleration framework with multiple decoding heads. arXiv preprint arXiv:2401.10774, 2024. URL https://arxiv.org/abs/2401.10774.
  5. [5]Clément L. Canonne. A short note on an inequality between KL and TV. arXiv preprint arXiv:2202.07198, 2023. URL https://arxiv.org/abs/2202.07198.
  6. [6]Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318, 2023. URL https://arxiv.org/abs/2302.01318.
  7. [7]Jian Chen, Yesheng Liang, and Zhijian Liu. DFlash: Block diffusion for flash speculative decoding. arXiv preprint arXiv:2602.06036, 2026. URL https://arxiv.org/abs/2602.06036.
  8. [8]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. URL https://arxiv.org/abs/2107.03374.
  9. [9]Xin Cheng, Xingkai Yu, Chenze Shao, Jiashi Li, Yunfan Xiong, Yi Qian, Jiaq i Zhu, Shirong Ma, Xiaokang Zhang, Jiasheng Ye, Qinyu Chen, Chengqi Deng, Jiping Yu, Damai Dai, Zhengyan Zhang, Yixuan Wei, Yixuan Tan, Wenkai Yang, Runxin Xu, Yu Wu, Zhean Xu, Xuanyu Wang, Muyang Chen, Rui Tian, Xiao Bi, Zhewen Hao, Shaoyuan Chen, Huanqi Cao, Wentao Zhang, Anyi Xu, Huishui Zhang, Dongyan Zhao, and Wenfeng Liang. DSpark: Confidence-scheduled speculative decoding with semi-autoregressive generation. arXiv preprint arXiv:2607.05147, 2026. URL https://arxiv.org/abs/2607.05147.
  10. [10]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. URL https://arxiv.org/abs/2110.14168.
  11. [11]Yunshu Du, Wojciech M. Czarnecki, Siddhant M. Jayakumar, Mehrdad Farajtbar, Razvan Pascanu, and Balaji Lakshminarayanan. Adapting auxiliary losses using gradient similarity. arXiv preprint arXiv:1812.02224, 2018. URL https://arxiv.org/abs/1812.02224.
  12. [12]Christopher A. Fuchs and Jeroen van de Graaf. Cryptographic distinguishability measures for quantum-mechanical states. IEEE Transactions on Information Theory, 45(4):1216–1227, 1999. doi: 10.1109/18.761271. URL https://arxiv.org/abs/quant-ph/9712042.
  13. [13]Aaron Grattafiori, Abhinanyu Dubey, Abhinav Jauhri, et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. URL https://arxiv.org/abs/2407.21783.
  14. [14]Geonmo Gu, Byeongho Heo, HeeJae Jun, Yoohoon Kang, Sangmin Lee, Sangdoo Yun, and Dongyoon Han. Verification-aware training for speculative decoding. arXiv preprint arXiv:2608.30135, 2026. URL https://arxiv.org/abs/2608.30135.
  15. [15]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems Datasets and Benchmarks Track, 2021. URL https://arxiv.org/abs/2103.03874.
  16. [16]Shijing Hu, Jingyang Li, Xingyu Xie, Zhihui Lu, Kim-Chuan Toh, and Pan Zhou. GRIFFIN: Effective token alignment for faster speculative decoding. arXiv preprint arXiv:2502.11018, 2025. URL https://arxiv.org/abs/2502.11018.
  17. [17]Jianuo Huang, Yaojie Zhang, Qituan Zhang, Hao Lin, Hanlin Xu, and Linfeng Zhang. Domino: Decoupling causal modeling from autoregressive drafting in speculative decoding. arXiv preprint arXiv:2605.29707, 2026. URL https://arxiv.org/abs/2605.29707.
  18. [19]Weifan Jiang, Krishna Teja Chitty-Venkata, Megan Flynn, Reed Meyerson, Zhenting Qi, Tianyu Wu, Eldar Kurtic, Minlan Yu, and Alexandre Marques. H-Spec: Parallel speculative decoding without a drafter-side KV cache. arXiv preprint arXiv:2609.24197, 2026. URL https://arxiv.org/abs/2609.24197.
  19. [20]Haodi Lei, Yafu Li, Haoran Zhang, Shunkai Zhang, Qianjia Cheng, Xiaoye Qu, Ganqu Cui, Bowen Zhou, Ning Ding, Yun Luo, and Yu Cheng. Draft-OPD: On-policy distillation for speculative draft models. arXiv preprint arXiv:2605.29343, 2026. URL https://arxiv.org/abs/2605.29343.
  20. [21]Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, 2023. URL https://arxiv.org/abs/2211.17192.
  21. [22]Shenggui Li, Chao Wang, Yikai Zhu, Yubo Wang, Fan Yin, Shuai Shi, Yefei Chen, Xiaomin Dong, Qiaoling Chen, Jin Pan, Ji Li, Laixin Xie, Yineng Zhang, Lei Yu, Yonggang Wen, Ivor Tsang, and Tianwei Zhang. SpecForge: A flexible and efficient open-source training framework for speculative decoding. arXiv preprint arXiv:2603.18567, 2026a. URL https://arxiv.org/abs/2603.18567.
  22. [23]Yucheng Li, Huiqiang Jiang, Yang Xu, Jianxin Yang, Yi Zhang, et al. Breaking entropy bounds: Accelerating RL training via MTP with rejection sampling. arXiv preprint arXiv:2606.12370, 2026b. URL https://arxiv.org/abs/2606.12370.
  23. [24]Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. EAGLE: Speculative sampling requires rethinking feature uncertainty. arXiv preprint arXiv:2401.15077, 2024a. URL https://arxiv.org/abs/2401.15077.
  24. [25]Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. EAGLE-2: Faster inference of language models with dynamic draft trees. arXiv preprint arXiv:2406.16858, 2024b. URL https://arxiv.org/abs/2406.16858.
  25. [26]Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. EAGLE-3: Scaling up inference acceleration of large language models via training-time test. arXiv preprint arXiv:2503.01840, 2025. URL https://arxiv.org/abs/2503.01840.
  26. [27]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2305.20050.
  27. [28]Fuliang Liu, Xue Li, Ketai Zhao, Yinxi Gao, Ziyan Zhou, Zhonghui Zhang, Zhibin Wang, Wanchun Dou, Sheng Zhong, and Chen Tian. DART: Diffusion-inspired speculative decoding for fast LLM inference. arXiv preprint arXiv:2601.19278, 2026. URL https://arxiv.org/abs/2601.19278.
  28. [29]Yu-Yang Qian, Hao-Cong Wu, Chen Chen, Jiacheng Sun, Zhenhua Dong, Peng Zhao, and Zhi-Hua Zhou. AdaFlash: Adaptive speculative decoding via on-policy distilled diffusion drafters. arXiv preprint arXiv:2607.19223, 2026. URL https://arxiv.org/abs/2607.19223.
  29. [30]Alfréd Rényi. On measures of entropy and information. In Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, 1961.
  30. [31]Matan Rusanovsky, Yoav Miron, Roy Uziel, Omer Belhasin, Ran Zilberstein, Maor Ashkenazi, and Michael Elad. LiLiCorr: Lightweight likelihood correlation of parallel drafts for speculative decoding. arXiv preprint arXiv:2608.20530, 2026. URL https://arxiv.org/abs/2608.20530.
  31. [32]Subham Sekhar Sahoo, Lingjie Chen, Khiem Pham, Jonathan Geuter, Chaitanya Dwivedi, Varad Pimpalkhute, Yash Akhauri, Alexander Moreno, Mikhail Yurochkin, Zhenting Wang, Mostafa Elhoushi, Nolan Dey, Shane Bergsma, Joel Hestness, John Thickstun, Eric Xing, and Zhengzhong Liu. Unlocking lossless speedups in LLMs via discrete diffusion. arXiv preprint arXiv:2609.04010, 2026. URL https://arxiv.org/abs/2609.04010.
  32. [33]Alexander Samarin, Sergei Krutikov, Anton Shevtsov, Sergei Skortsov, Filipp Fišin, and Alexander Golubev. LK Losses: Direct acceptance rate optimization for speculative decoding. arXiv preprint arXiv:2602.23881, 2026. URL https://arxiv.org/abs/2602.23881.
  33. [34]SGLang Contributors. SGLang v0.5.19 DFlash speculative verification. https://github.com/sgl-project/sglang/blob/v0.5.19/python/sglang/srt/speculative/dflash_utils.py, 2026. Source code, lines 775–995.
  34. [35]SpecForge Contributors. SpecForge unified training recipe catalog. Software documentation, 2026. URL https://github.com/sgl-project/SpecForge/blob/main/examples/configs/README.md. Accessed September 23, 2026.
  35. [36]Speculators Contributors. Speculators training command reference. Software documentation, 2026. URL https://docs.vllm.ai/projects/speculators/en/stable/cli/train/. Accessed September 23, 2026.
  36. [37]Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models, 2023. URL https://crfm.stanford.edu/2023/03/13/alpaca.html.
  37. [38]TorchSpec Contributors. TorchSpec DFlash training guide. Software documentation, 2026. URL https://github.com/lightsеekorg/TorchSpec/blob/main/docs/dflash/README.md. Accessed September 23, 2026.
  38. [39]Tim van Erven and Peter Harremoës. Rényi divergence and Kullback-Leibler divergence. IEEE Transactions on Information Theory, 60(7):3797–3820, 2014. doi: 10.1109/TIT.2014.2320500. URL https://arxiv.org/abs/1206.2459.
  39. [40]Lexington Whalen, Yuki Ito, and Ryo Sakamoto. Teaching diffusion to speculate left-to-right. arXiv preprint arXiv:2606.11552, 2026. URL https://arxiv.org/abs/2606.11552.
  40. [41]Tianyu Wu, Yu Yao, Zhenting Qi, Han Zheng, Zhuohan Wang, Haoran Ma, Lawrence Liao, Himabindu Lakkaraju, Ju Li, and Yilun Du. D-PACE: Dynamic position-aware cross-entropy for parallel speculative drafting. arXiv preprint arXiv:2605.18810, 2026. URL https://arxiv.org/abs/2605.18810.
  41. [42]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. URL https://arxiv.org/abs/2505.09388.
  42. [43]Tianjian Yang and Meng Li. Spec-AUF: Accept-until-fail training under train-inference misalignment for masked block drafters. arXiv preprint arXiv:2607.01893, 2026. URL https://arxiv.org/abs/2607.01893.
  43. [44]Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. In Advances in Neural Information Processing Systems, volume 33, pages 5824–5836, 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/3fe78a8acf5fda99de95303940a2420c-Paper.pdf.arxiv.org/abs/2001.06782
  44. [45]Yaohjie Zhang, Linfeng Zhang, Bin Cui, and Xupeng Miao. DFlow: Enabling verifier information flow in block diffusion speculative decoding. arXiv preprint arXiv:2609.06498, 2026. URL https://arxiv.org/abs/2609.06498.

Paper details

vlnc:W_bz3dadw2
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Generative AI tools

View statement
Looking for related papers…
Contents