betaFeedback

WorksComputer Science

TextSeal2: Distortion-Free LLM Watermarking with Copula Mixing and Memoryless Races

Abstract

We present TextSeal2, a distortion-free watermark for large language models that keeps its signal under speculative decoding: tokens that replace rejected drafts remain watermarked with the same keys, and the acceptance rate is unchanged. It also preserves output diversity and adds no measurable inference latency. Its predecessor, TextSeal, restores diversity in the Gumbel-max scheme by routing each token to one of two keys via a coin flip. This is suboptimal because the detector averages the scores from both keys without knowing which key was used. TextSeal2 instead merges two or more keys into a single virtual key and couples it to fresh noise through AutoMark's Gaussian copula. Any number of keys, including publicly shared keys, can be merged this way without diluting detection. Under speculative decoding, TextSeal2 verifies keyed drafts with one more exponential race, run on the leftovers of the draft races. By memorylessness, this race reproduces block verification, so the output follows the target model exactly. We evaluate TextSeal2 on three LLMs. At equal output token diversity, TextSeal2 detects 8 to 22 percentage points more 100-token answers than TextSeal at a false-positive rate of 10−310^{-3}, and is comparable to textGrain while requiring no post-logits optimization. Under speculative decoding, it detects 18 and 22 percentage points more 200-token answers than TextSeal and textGrain, respectively, at the same acceptance rate.

Introduction

Identifying AI-generated text helps prevent misuse, from fabricated news ([34, 5]) to model distillation by malicious actors ([28]). An LLM watermark embeds a keyed statistical signal at generation time: each token is sampled from the model’s next-token distribution through a coupling with pseudo-random values derived from a secret key and the preceding tokens. The detector recomputes these values from the key and the text and tests whether the tokens are statistically aligned with them. The EU AI Act requires providers to mark AI-generated content ([8]), and watermarking is deployed at scale: SynthID-Text runs in Gemini ([7]), Anthropic marks Claude outputs ([2]), OpenAI deploys textGrain ([24]), and we released TextSeal ([29]).

We argue that a deployed watermark should meet four requirements: it should (i) preserve the model’s output distribution; (ii) keep answers diverse under a fixed watermark key; (iii) add minimal latency to decoding; and (iv) preserve the acceptance rate of speculative decoding ([20, 3, 30]). Schemes whose next-token distribution, averaged over all possible keys, equals the model’s are called distortion-free. The earliest such scheme applies the Gumbel-max trick to keyed pseudo-random values ([1]); later ones achieve the same property with other samplers ([4, 16, 32, 19]). Distortionary schemes such as the green list ([17]) instead shift token probabilities, trading quality for detectability; we focus on distortion-free schemes.

Every distortion-free watermark couples the sampled token to key-derived randomness ([23, 33, 26, 31, 13, 12]). If the key fully determines the token, detection is strongest but repeated generations with the same context never vary; if the token is independent of the key, sampling is unchanged but carries no signal. Put differently, watermarking removes a fraction of each token’s entropy, which is the budget formulation of textGrain ([24]). Schemes differ in how they trade diversity against detectability. TextSeal routes each token to one of two keys by a coin flip ([29]); textGrain samples from a regularized optimal-transport coupling solved at every decoding step ([24]); SynthID-Text selects among randomly sampled candidates by tournament ([7]); and AutoMark blends its Gumbel values with freshly drawn noise through a Gaussian copula ([12]). Other couplings rely on reweighting ([16, 32]), inverse-transform sampling ([19]), or maximal couplings ([33, 26, 31]). Dual-key constructions such as TextSeal also support tampering detection ([9]) and public detection, with one key public and the other private ([27]). Among these, textGrain detects the most watermarked text at a given diversity, but its optimal transport adds post-logits latency and its speculative variant lowers acceptance, violating our requirements (iii)(iii) and (iv)(iv) presented above.

We present TextSeal2, which retains TextSeal’s two keys, meets all four requirements, and matches the state of the art in detectability. Our starting point is a diagnosis of the gap between TextSeal, textGrain, and AutoMark. In TextSeal, each token is generated under one key, but the detector does not know which key, and must average the scores from both keys. This halves the signal and reduces the signal-to-noise ratio by 2\sqrt{2}. AutoMark and textGrain do not suffer from this loss because they use a single key, which the detector scores on every token, and draw their diversity from fresh randomness.

TextSeal2 makes three contributions to reach the state of the art (Table 1, Figure 1):

MethodSpeculative decodingAcceptance keptExtra post-logits work
xx✓\checkmark
SynthID-Text (tournament)89‾±3\underline{89}\pm350±250\pm2✓\checkmarkO(L∣A∣)O(L\lvert A\rvert)
TextSeal (coin flip)81±181\pm171±271\pm2✓\checkmarknone
AutoMark copula91±3\mathbf{91}\pm360±260\pm2✓\checkmarknone
textGrain89‾±4\underline{89}\pm467±467\pm4✓\checkmarkSinkhorn, O(NitBm)O(N_{\mathrm{it}}Bm)
textGrain, own speculative-decoding recipe−-84‾±6\underline{84}\pm6×\times (−5%-5\% tokens/round)Sinkhorn, O(NitBm)O(N_{\mathrm{it}}Bm)
TextSeal2 (ours)91±291\pm289±189\pm1✓\checkmarknone

Table 1. Detection on Qwen3.5-9B (ELI5, temperature 1, top-p 0.95), at fixed diversity: % of 200-token answers detected at a false-positive rate (FPR) of 10−310^{-3}. Gray: half-width of the 95% bootstrap interval; bold: best; underlined: second best, per column. Speculative decoding: Qwen3.5-0.8B drafts 8 tokens per round, with block verification; TextSeal2 uses the residual race, the baselines use block verification’s native stopping rule with the last token of each round drawn under an independent key, and textGrain’s own recipe verifies the watermarked distributions. Acceptance is kept when tokens per round do not drop below those of unwatermarked block verification (2.74; paired over 480 prompts, Figure 11 in Appendix § 13). NitN_{\mathrm{it}}: Sinkhorn iterations on a B×mB \times m table (1–3 ms per token, Appendix § 9); LL: tournament layers over candidates AA. TextSeal2 equals AutoMark’s copula in distribution; their gap under speculative decoding is the residual race.

Figure 1. Diversity–detection trade-off on Qwen3.5-9B at 100 tokens at FPR=10−310^{-3}, (a) without and (b) with speculative decoding (8 drafts per round, block verification). In (a), TextSeal2 coincides with AutoMark’s single-key copula as expected since it behaves like a single-key copula. In (b), TextSeal2 uses the residual race, and the baselines use block verification’s native stopping rule with the last token of each round drawn under an independent key. Diversity is measured by Self-BLEU (lower is more diverse).

  • Copula mixing (subsection 3.1). We merge the two keys’ Gaussian values into a single virtual key and apply AutoMark’s copula on it. Each key retains its role (e.g., one can be published to allow for public detection), and additional keys can be merged without diluting detection.

  • The residual race for speculative decoding (subsection 3.2). Gumbel-max sampling, used for the keyed drafts, is an exponential race. By memorylessness, the losers’ leftovers are fresh exponentials. We use this property to run one more race on them, and with block verification (Sun et al., [30]), we decide the first draft token to be rejected, and the bonus token. The output is exactly distortion-free, acceptance is that of block verification (Proposition 1), and detection gains 28 to 33 percentage points over drawing the last token of each round under an independent key.

  • A calibrated strength parameter (subsection 3.3). The coupling strength ρ\rho plays the role of textGrain’s entropy budget β\beta: the entropy it removes from a token of positive entropy increases strictly with ρ\rho (Theorem 1). No distortion-free scheme can obtain more detection evidence per token than the entropy it removes, and the copula comes close to this bound (Proposition 3).

Background & Related work

Gumbel-max watermarking (Aaronson and Kirchner, [1]). At step tt, the model outputs a distribution p⃗(t)=(pv(t))v∈V\vec{p}^{(t)} = (p_v^{(t)})_{v\in\mathcal{V}} over the vocabulary V\mathcal{V}, after temperature scaling and top-pp truncation (we drop the superscript tt when no confusion arises). A pseudo-random function (PRF) seeded by the secret key KK and the watermark context window w=(xt−k,…,xt−1)w = (x_{t-k}, \ldots, x_{t-1}) – the last kk tokens – assigns to each candidate token vv a value Rv=PRF⁡(v,w,K)∈[0,1]R_v = \operatorname{PRF}(v,w,K) \in[0,1], and the watermarked sampler emits:

xt=arg⁡max⁡v∈VRv1/pv=arg⁡min⁡v∈VEvpv,Ev=−ln⁡Rv.(1)x_t = \arg\max_{v\in\mathcal{V}} R_v^{1/p_v} = \arg\min_{v\in\mathcal{V}} \frac{E_v}{p_v}, \qquad E_v = -\ln R_v. \tag*{(1)}

The right-hand form is an exponential race: if the RvR_v are i.i.d. uniform, then the EvE_v are i.i.d. Exp⁡(1)\operatorname{Exp}(1), each Ev/pvE_v/p_v is Exp⁡(pv)\operatorname{Exp}(p_v), and the arg min is distributed exactly according to p⃗\vec{p}. In this race, candidate vv runs a lane of length EvE_v at speed pvp_v, and the first to finish is emitted (Figure 2). For detection, each observed token xix_i receives the score si=−ln⁡(1−Rxi)s_i = -\ln(1-R_{x_i}), where RxiR_{x_i} is recomputed from the key and the token’s window. Under the null hypothesis H0\mathcal{H}_0 (unwatermarked text, or the wrong key), the scores are Exp⁡(1)\operatorname{Exp}(1) and their sum follows a Gamma distribution, giving a simple sum test; watermarked text, whose tokens tend to win their races with unusually large RR, produces stochastically larger scores.

Figure 2. TextSeal2 generates one token (subsection 3.1). (a) The model’s forward pass gives the probabilities pvp_v, and the last k=3k=3 tokens form the window w\mathbf{w}. (b) The keys’ values at w\mathbf{w} are merged into the virtual key Gv⋆G_v^\star, which the copula (4) blends with the noise ηv\eta_v into R~v\widetilde{R}_v; the Gumbel-max choice is arg min⁡v(−ln⁡R~v)/pv\operatorname*{arg\,min}_v(-\ln\widetilde{R}_v)/p_v. (c) The Gumbel-max choice can be seen as a race (1): candidate vv runs a lane of length Ev=−ln⁡R~vE_v=-\ln\widetilde{R}_v at speed pvp_v, and the token that finishes first is chosen.

Repeated windows. A sampler is single-token distortion-free if, in expectation over the key, every token retains its model probability given the prefix (Dhathatri et al., 2024). It is single-sequence distortion-free if the distribution of the entire answer is likewise preserved. Windows repeat often in code and long reasoning traces. Reusing a window’s keyed values re-runs the same race, so the text is single-token but not single-sequence distortion-free. Repeated context masking draws fresh values at the later occurrences of a window, i.e. plain sampling there (Dhathatri et al., 2024). AutoMark’s ResidualRace instead continues the window’s race from its leftovers (Gloaguen et al., [12]). Both make the text single-sequence distortion-free (Appendix § 6.§ 5). At detection, each (window, token) pair is scored once, to avoid biasing the detection test (Fernandez et al., [10]).

TextSeal Sander et al., [29]. Plain Gumbel-max is deterministic given the key, so TextSeal randomises the race (1) with two keys k(1)k^{(1)} and k(2)k^{(2)}, producing values Rv(1)R_v^{(1)} and Rv(2)R_v^{(2)}: at each step it uses k(1)k^{(1)} with probability 1−α1-\alpha and k(2)k^{(2)} with probability α\alpha. The detector does not know which key generated a given token, so it fuses the two per-key scores, si=(1−α)si(1)+αsi(2)s_i=(1-\alpha)s_i^{(1)}+\alpha s_i^{(2)} with si(j)=−ln⁡(1−Ri(j))s_i^{(j)}=-\ln(1-R_i^{(j)}). Optionally, each score is weighted by the entropy of the next-token distribution under an auxiliary language model: low-entropy tokens are nearly forced by the context and carry little watermark signal, so down-weighting them improves detection; the weighted sum is then tested against a Gamma distribution with matched mean and variance. The two keys also serve speculative decoding and public detection. Under speculative decoding, TextSeal routes each draft to one of its two keys as usual and generates a replacement token with the other key. For public detection, kpub=k(1)k_{\mathrm{pub}}=k^{(1)} is published so anyone can run a weaker test, while kpriv=k(2)k_{\mathrm{priv}}=k^{(2)} stays secret for full verification Milis et al., [27].

Gaussian copula. AutoMark Gloaguen et al., [12] randomises differently: it draws independent noise ηv∼N(0,1)\eta_v\sim\mathcal{N}(0,1) at every position and replaces RvR_v by

R~v=Φ(ρΦ−1(Rv)+1−ρ2 ηv)(2)\widetilde{R}_v=\Phi\left(\rho\Phi^{-1}(R_v)+\sqrt{1-\rho^2}\,\eta_v\right) \tag*{(2)}

where Φ\Phi is the standard normal CDF. The parameter ρ∈[0,1]\rho\in[0,1] interpolates between unwatermarked sampling (ρ=0\rho=0, no dependence on the key) and plain Gumbel-max (ρ=1\rho=1, deterministic given the key), while R~v\widetilde{R}_v remains uniform – hence the race distortion-free – for every ρ\rho.

textGrain Li et al., [24]. At each decoding step, it uses the key and the window to partition the vocabulary into BB blocks and to draw a B×mB\times m table of keyed Gumbel costs. Sinkhorn iterations then solve an entropy-regularised optimal-transport problem between the block probabilities and mm columns, calibrated so that the key removes a prescribed fraction β\beta of the token’s entropy; a keyed column is selected, and the block and token are sampled from the resulting coupling with freshly drawn randomness. Only the key-only computations can overlap the model’s forward pass: the Sinkhorn iterations require the next-token distribution, and the following step requires the sampled token, so this work lies on the critical path of decoding. Better kernels can shorten it but not remove it (1–3 ms per token in our implementation, 0.2–0.4 ms in a minimal version; Appendix § 9).

Speculative decoding. Write p⃗=PT(⋅∣x<t)\vec{p}=P_T(\cdot\mid x_{<t}) for the target model’s distribution and q⃗=PD(⋅∣x<t)\vec{q}=P_D(\cdot\mid x_{<t}) for the (smaller) draft model’s. In each round, the drafter proposes a block of tokens, and the target evaluates the whole block in one forward pass. Token-level verification tests the drafts in order. A draft token x∼q⃗x\sim\vec{q} is accepted with probability min⁡(1,px/qx)\min(1,p_x/q_x), the raw acceptance test on the unwatermarked distributions p⃗\vec{p} and q⃗\vec{q}. At the first rejection, the target emits a replacement token from the residual distribution:

rv=(pv−qv)+TV⁡(p⃗,q⃗),TV⁡(p⃗,q⃗)=∑v(pv−qv)+(3)r_v=\frac{(p_v-q_v)_+}{\operatorname{TV}(\vec{p},\vec{q})},\qquad \operatorname{TV}(\vec{p},\vec{q})=\sum_v(p_v-q_v)_+ \tag*{(3)}

and discards the later drafts; if every draft is accepted, it emits a bonus token from its next distribution. The output follows p⃗\vec{p} at acceptance rate 1−TV⁡(p⃗,q⃗)1-\operatorname{TV}(\vec{p},\vec{q}) Leviathan et al., [20]; Chen et al. [3]. Block verification Sun et al., [30] instead decides on all the block jointly, so it can keep a longer prefix even when a shorter one fails its per-token test (closed forms in Appendix § 6.§ 3). Sun et al. [30] prove that repeating such rounds also generates the target’s law, and that after any number of rounds block verification has decoded at least as many tokens in expectation as any other verification procedure with this guarantee, so it is never worse than token-level verification. They report wall-clock speedups of 5–8% over token-level verification across tasks and datasets.

The TextSeal2 watermark

TextSeal2 retains TextSeal’s two keys, Gumbel-max sampling (1), and detector. It changes how the keys enter the sampler (subsection 3.1), how speculative decoding chooses the tokens it emits (subsection 3.2), and how the strength is calibrated (subsection 3.3). It handles repeated windows as in § 2, and under speculative decoding every window raced in a round counts as used, whether or not its position is kept (Appendix § 6.§ 5). Figure 2 shows one generation step, Figure 3 one round of speculative decoding, and Figure 4 the detection of one text; Algorithm 1 is the pseudocode.

Figure 3. Block verification checks the drafts against the target as a block, keeps those before a stop position II, and emits one token yy at II. (a) The drafter proposes “the mat” with keyed races. (b) One target pass then gives the probability πi,v\pi_{i,v} that I=iI = i and y=vy = v. One more race on the leftovers picks II and yy: “floor” wins, so the round keeps “the” and emits “floor” at I=2I = 2. TextSeal2 preserves the target’s distribution and block verification’s acceptance rate (Proposition 1). Algorithm 1 in the appendix gives more details.

Figure 4. TextSeal2 detects the text “The cat sat on the floor and fell asleep in the sun.” (subsection 3.4). (a) Each token xix_i and its window wiw_i; the highlighted row is the token of Figure 2. (b) For “floor”, the key’s values are merged into Gxi⋆G_{x_i}^\star as in (4), without the noise, and give the score si⋆s_i^\star; key k(1)k^{(1)} alone gives the public score si(1)s_i^{(1)}. (c) The n=10n=10 scores sum to S=23.9S=23.9, and the Gamma⁡(n,1)\operatorname{Gamma}(n,1) tail beyond SS (box) is p=4.6⋅10−4p=4.6\cdot10^{-4}.

Copula mixing of the two keys

For each key j∈{1,2}j \in\{1,2\}, map its values to Gaussian space by Gv(j)=Φ−1(Rv(j))G_v^{(j)} = \Phi^{-1}(R_v^{(j)}), which is standard normal when Rv(j)R_v^{(j)} is uniform. Instead of running the exponential race on R⃗(1)\vec{R}^{(1)} or R⃗(2)\vec{R}^{(2)}, chosen at random as in TextSeal, TextSeal2 runs the race (1) on the merged values

R~v=Φ(ρGv⋆+1−ρ2 ηv),where Gv⋆=Gv(1)+Gv(2)2 and ηv∼i.i.d.N(0,1).(4)\widetilde{R}_v = \Phi\left(\rho G_v^\star+ \sqrt{1-\rho^2}\,\eta_v\right), \qquad\text{where } G_v^\star= \frac{G_v^{(1)} + G_v^{(2)}}{\sqrt{2}} \text{ and } \eta_v \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,1). \tag*{(4)}

Since the two keys are independent, Gv⋆G_v^\star is standard normal; it depends only on (v,w,k(1),k(2))(v,\mathbf{w},k^{(1)},k^{(2)}) and is independent across candidate tokens vv (and across positions whose window has not been used before; see § 2). Equation (4) is therefore exactly AutoMark’s copula (2) applied to a virtual key with values Φ(Gv⋆)\Phi(G_v^\star). In particular, the R~v\widetilde{R}_v are i.i.d. uniform and the race is distortion-free for every ρ\rho (Lemma 2). Detection uses the matched score si⋆=−ln⁡Φ(−Gxi⋆)s_i^\star=-\ln\Phi(-G_{x_i}^\star) – the usual score −ln⁡(1−R)-\ln(1-R) of the virtual key – which is Exp⁡(1)\operatorname{Exp}(1)-distributed under H0H_0. The two-key scheme thus has exactly the diversity and detection power of AutoMark’s single-key copula at the same ρ\rho. The construction extends to MM keys: G⋆=M−1/2∑k=1MG(k)G^\star=M^{-1/2}\sum_{k=1}^{M}G^{(k)} remains a single standard normal per candidate, so the full detector loses no signal as keys are added, e.g. one publishable key per verifier. (Each key’s correlation with the race input, and hence a single-key test’s power, shrinks as ρ/M\rho/\sqrt{M}.)

Why the coin flip loses signal. With the coin flip at α=1/2\alpha=1/2, each token is generated under one key, but the detector does not know which and must score 12si(1)+12si(2)\frac{1}{2}s_i^{(1)}+\frac{1}{2}s_i^{(2)}. Only the key that generated the token carries signal, so averaging halves the expected excess of the score over its null mean of 1. The two scores are independent under the null, so averaging also halves the variance – hence the standard deviation falls only by 2\sqrt{2}. The signal-to-noise ratio is therefore reduced by a factor 2\sqrt{2}. Copula mixing has no such loss: every token is generated from G⋆G^\star, which the detector scores.

Optional public detection. By default, detection uses both keys through si⋆s_i^\star. kpub=k(1)k_{\mathrm{pub}}=k^{(1)} can be published for anyone to compute single-key scores si(1)=−ln⁡(1−Rxi(1))s_i^{(1)}=-\ln(1-R_{x_i}^{(1)}). This test is weaker: G(1)G^{(1)} has correlation only ρ/2\rho/\sqrt{2} with the race input ρG⋆+1−ρ2 η\rho G^\star+\sqrt{1-\rho^2}\,\eta, while kpriv=k(2)k_{\mathrm{priv}}=k^{(2)} supports full-strength

verification and exposes manipulations of the public signal [27]. SAC-Copula [21] and AutoMark’s Gaussian direction mix keyed Gaussians, without a public key.

Speculative decoding through memoryless races

Speculative decoding poses a specific problem for watermarking: the token that verification emits after the kept drafts must follow a residual distribution, such as (3), and must itself carry the watermark. Prior approaches each give up something. Re-running the race with weights r⃗\vec{r} on the same keyed values distorts the output, because the draft’s win shifts the remaining values (Remark 1). Replacing from fresh noise [15] or from a second key [29, 7] keeps the acceptance rate, but the replaced tokens, about a third of the text in our experiments, then lose the watermark or split it across keys. Verifying between the watermarked distributions keeps the watermark but accepts fewer drafts [15, 24, 7] (Proposition 2). He et al. [14] let the detector guess which key made each token, and Liu et al. [25] use several drafts per position. TextSeal2 keeps both the keys and the acceptance of block verification, with two races on the same keyed values.

Keyed drafts and their leftovers. A round has NN drafts at positions i=1,…,Ni = 1, \ldots, N and a bonus position N+1N + 1, with drafter laws q⃗i\vec{q}_i and target laws p⃗i\vec{p}_i from one target pass. Position ii reads the lane lengths Ei,v=−ln⁡R~v(wi)E_{i,v} = -\ln\widetilde{R}_v(w_i) from (4) at its window wiw_i, which ends with di−1d_{i-1} for i≥2i \ge2, and the drafter runs the race at its own speeds:

di=arg⁡min⁡vEi,vqi,v,τi=Ei,diqi,di,i=1,…,N.(5)d_i = \arg\min_v \frac{E_{i,v}}{q_{i,v}}, \qquad\tau_i = \frac{E_{i,d_i}}{q_{i,d_i}}, \qquad i = 1,\ldots,N. \tag*{(5)}

When did_i finishes, lane vv has covered the distance τiqi,v\tau_i q_{i,v}, and Ei,v−τiqi,vE_{i,v} - \tau_i q_{i,v} is its leftover. By memorylessness, the leftovers are fresh (proof in Appendix § 6.§ 2):

Lemma 1 (Leftovers of an exponential race; AutoMark). If the EvE_v are i.i.d. Exp⁡(1)\operatorname{Exp}(1) and q⃗\vec{q} is a probability vector, the race x=arg⁡min⁡vEv/qvx = \arg\min_v E_v/q_v, with time τ\tau, emits vv with probability qvq_v, and given (x,τ)(x,\tau) the leftovers Ev−τqvE_v - \tau q_v, v≠xv \ne x, are again i.i.d. Exp⁡(1)\operatorname{Exp}(1).

Block verification and the relay race. Given the drafts, block verification (§ 2) stops at a position II, keeps d<I=(d1,…,dI−1)d_{<I} = (d_1,\ldots,d_{I-1}), and emits a token yy at position II, with law πi,v=P(I=i)ri,v\pi_{i,v} = \mathbb{P}(I = i)r_{i,v}. Here r⃗i\vec{r}_i is a residual law at position ii, with r⃗N+1=p⃗N+1\vec{r}_{N+1} = \vec{p}_{N+1}, and P(I=i)\mathbb{P}(I = i) depends on all the drafts, since a

later success can override an earlier failure; Appendix § 6.§ 3 gives both in closed form. TextSeal2 draws this outcome with one more race, on the leftovers of all positions:

(I,y)=arg⁡min⁡(i,v): πi,v>0Ei,v−τiqi,vπi,v(6)(I,y)=\underset{(i,v):\,\pi_{i,v}>0}{\arg\min}\frac{E_{i,v}-\tau_iq_{i,v}}{\pi_{i,v}} \tag*{(6)}

with τN+1qN+1,v=0\tau_{N+1}q_{N+1,v}=0 at the bonus position, which has no draft race. Each lane resumes where its draft race left it and runs its leftover at speed πi,v\pi_{i,v}, as in a relay (Figure 3), and the round emits d<Iyd_{<I}y without drawing an accept coin. We call (6) the residual race: a single race on the leftovers of the draft races picks the stopping position and the last token together. Kept drafts carry the signal of their draft races and the last token that of the relay, so the detector is unchanged: it sums the scores and does not need to locate the ends of rounds. AutoMark’s ResidualRace continues the race of a repeated window, and exponential races (Li and Anantharam [22]) also couple drafter and target in unwatermarked speculative sampling (Kobus and Gündüz [18]; Daliri et al. [6]); to our knowledge, racing block verification’s outcome on the leftovers of keyed drafts is new.

Proposition 1 (Speculative decoding with the residual race). Suppose that every round reads i.i.d. uniform keyed values at its windows, independent across positions and of earlier rounds, and independent copula noise. (i) Given the drafts, (I,y)(I,y) has law π⃗\vec{\pi}, so the number of tokens per round, and hence acceptance, is that of block verification. (ii) The generated text follows the target’s law. (iii) The window rule (Appendix § 6.§ 5) meets this condition: the text is distortion-free over keys.

This proposition says that the residual race preserves the output distribution and the acceptance of block verification. Appendix § 6.§ 4 proves Proposition 1 in three steps: memorylessness makes the leftovers fresh given the drafts, block verification gives each draft’s own lane zero mass (Lemma 3), and Sun et al. [30] turn block verification’s law into the target’s. It also derives the lane length of each emitted token, which is what the detector reads, and shows that racing on the raw lane lengths instead of the leftovers is biased (Remark 1). Appendix § 6.6 proves that no scheme with a single draft accepts more often while keeping the output distribution (Proposition 2), a bound that the residual race attains for N=1N=1, and compares it with the optimality of block verification for N>1N>1. Remark 2 gives a coin-free token-level variant, and Algorithm 1 one round in pseudocode.

Controlling the entropy removed

Under a fixed key, the diversity that remains is exactly the entropy the key does not remove: whatever randomness the independent noise still contributes. textGrain calibrates its strength directly in these terms, removing a prescribed fraction ρ\rho of each token’s entropy (Li et al. [24]). In TextSeal2, the coupling strength ρ\rho plays the same role. For a token of positive entropy, the entropy removed is a strictly increasing function of ρ\rho, from none at ρ=0\rho=0 to the token’s full entropy at ρ=1\rho=1 (Theorem 1), so any target budget can be met by a unique ρ\rho, token by token if desired. In practice this calibration is unnecessary: a fixed ρ\rho removes nearly the same fraction of a token’s entropy – ρ≈0.64\rho\approx0.64, 0.810.81 and 0.930.93 remove about 30%, 50% and 70% respectively, across a range of many-token distribution shapes (Appendix § 8). We therefore use a single constant ρ\rho throughout, with a per-token ρt\rho_t available when an exact budget guarantee is required.

Two results in Appendix § 7 indicate that little is lost by this simplicity. First, no distortion-free coupling can extract more detection evidence per token, measured as a Kullback–Leibler divergence, than the entropy it removes (Proposition 3). Second, the copula uses this budget near-optimally: on 5,169 real Qwen3.5-9B next-token distributions, it attains on average 97% of the optimal per-token signal-to-noise ratio in the weak-watermark limit with a Gaussian score, and 91–100% at ρ=0.5\rho=0.5, 0.80.8 and 0.950.95 on synthetic distributions. A per-token optimisation such as textGrain’s can therefore gain only a few percent in signal-to-noise ratio, which is consistent with the empirical parity in § 4.

Detection and localization

The detector scores each token xix_i at its window, with both keys through the virtual key (private score) or with the public key k(1)k^{(1)} alone (public score, subsection 3.1):

si⋆=−ln⁡Φ(−Gxi⋆),si(1)=−ln⁡(1−Rxi(1)).(7)s_i^\star=-\ln\Phi\left(-G_{x_i}^\star\right),\qquad s_i^{(1)}=-\ln\left(1-R_{x_i}^{(1)}\right). \tag*{(7)}

Both scores are Exp⁡(1)\operatorname{Exp}(1) under H0\mathcal{H}_0 for every ρt\rho_t, since they do not involve the coupling strength, so the detection machinery of TextSeal carries over unchanged ([29]). The sampler can also watermark existing text post hoc, by rephrasing it ([11]). Keys with windows of different lengths can be merged into G⋆G^\star at no cost to the full detector, and because edits break long windows first, the decay of the per-key scores with window length estimates the edit rate (Appendix § 11).

Detection tests. The private test sums the scores of the nn distinct (window, token) pairs, each at its first occurrence (§ 2, Figure 4):

S=∑i=1nsi⋆∼H0Gamma⁡(n,1),p=1−Fn(S),(8)S=\sum_{i=1}^{n}s_i^\star\mathrel{\mathop{\sim}^{\mathcal{H}_0}}\operatorname{Gamma}(n,1),\qquad p=1-F_n(S), \tag*{(8)}

where FnF_n is the CDF of Gamma⁡(n,1)\operatorname{Gamma}(n,1); the public test is the same with si(1)s_i^{(1)}. Two optional refinements come from TextSeal ([29]). Entropy weighting down-weights low-entropy tokens, which carry little signal, using the next-token entropy HiH_i of an auxiliary model:

Sw=∑i=1nwisi⋆,where wi=0.1+0.9Hi−Hmin⁡Hmax⁡−Hmin⁡.(9)S_w=\sum_{i=1}^{n}w_i s_i^\star,\qquad\text{where }w_i=0.1+0.9\frac{H_i-H_{\min}}{H_{\max}-H_{\min}}. \tag*{(9)}

Under H0\mathcal{H}_0, SwS_w has mean ∑iwi\sum_i w_i and variance ∑iwi2\sum_i w_i^2, and the test compares it with the Gamma law of matched moments, of shape (∑iwi)2/∑iwi2(\sum_i w_i)^2/\sum_i w_i^2 and scale ∑iwi2/∑iwi\sum_i w_i^2/\sum_i w_i. The weights do not use the key, so the test is valid for any auxiliary model. Localization finds watermarked spans in documents that mix human and generated text: it tests windows of dyadic lengths L0,2L0,4L0,…L_0,2L_0,4L_0,\ldots, slid at half-length offsets, about 4n/L04n/L_0 candidates, Bonferroni-corrected, and greedily extracts the most significant ones.

Experiments

We compare detection at equal diversity, first without and then with speculative decoding (subsection 4.3). Unless stated otherwise, all differences below are in percentage points of detected answers.

Setup

Generation. We generate answers to ELI5 questions with Llama-3.1-8B-Instruct, Qwen3.5-9B, and Qwen3.5-27B, at temperature 1 and top-pp 0.95, up to 200 new tokens, with watermark window size

Without speculative decoding

k=3k = 3. For speculative decoding, Qwen3.5-0.8B drafts N=8N = 8 tokens per round for Qwen3.5-9B, and every scheme verifies the drafts by block verification [30]; Appendix § 11 repeats the comparison with N=4N = 4 and 1616. All schemes, TextSeal2 included, follow the repeated-window rules of Appendix § 6.§ 5 (one use per window at generation; each (window, token) pair scored once); under speculative decoding, every window that a round reads counts as used, kept or not.

Metrics. Detection is the fraction of watermarked answers declared watermarked at a nominal false-positive rate of 10−310^{-3}. Because no closed-form null is exact in practice, each detector’s threshold is the empirical 10−310^{-3} quantile of its p-values on 10,000 unwatermarked answers scored under 10 keys (10510^5 null p-values; calibration in Figure 9). Detection is measured on the first 100 or 200 tokens, with key-only (unweighted) scores unless stated otherwise. We generate 3000 answers per configuration on Qwen3.5-9B, with or without speculative decoding, and 1000 on the other models. Diversity is measured as in TextSeal, by Self-BLEU among answers to the same question under a fixed key (mean BLEU over all pairs of answers; lower is more diverse), on 100 questions with 32 answers each, or 16 under speculative decoding. Measuring diversity by the share of distinct 4-grams instead leaves the conclusions unchanged, except that textGrain is then slightly ahead of the copulas without speculative decoding (Appendix § 11). To compare at equal diversity, each method’s detection is interpolated along its diversity–detection curve at TextSeal’s Self-BLEU; 95% intervals come from a bootstrap that resamples questions jointly across methods, so intervals for differences are paired.

Figure 9. Calibration: false-positive rate against the nominal p-value threshold for the detectors of Table 1 without speculative decoding, on 10 keys and 10,000 unwatermarked Qwen3.5-9B answers; grey: 95% binomial interval around the diagonal. At 10−310^{-3} and above, every detector is within a factor 1.3 of nominal.

Methods. We evaluate TextSeal with α∈{0.5,0.7,0.9}\alpha\in\{0.5, 0.7, 0.9\} (plus 0.8 and 0.95 on Qwen3.5-9B); AutoMark’s copula and TextSeal2 with ρ∈{0.7,0.8,0.9,0.95}\rho\in\{0.7, 0.8, 0.9, 0.95\} (on Qwen3.5-9B also 0.5, 0.6, 0.975, 0.99 and 0.995, plus 0.85, 0.925 and 0.999 under speculative decoding), using a 53-bit PRF (Appendix § 10), with TextSeal2 scored by s∗s^{*} (or s(1)s^{(1)} for the public-key test); our reimplementation of textGrain (B=m=256B = m = 256, β∈{0.25,0.5,0.75}\beta\in\{0.25, 0.5, 0.75\}); and, on Qwen3.5-9B, SynthID-Text with 2–30 tournament layers (2–20 under speculative decoding), using its official keyed coins and detection test (fused over its two keys under speculative decoding, Appendix § 13). Under speculative decoding, TextSeal2 uses the residual race (subsection 3.2). The baselines keep block verification’s native stopping rule on the unwatermarked laws p^\hat{p} and q^\hat{q}, as TextSeal and SynthID-Text do with token-level verification [29, 7]: each draft comes from the scheme’s sampler on q⃗i\vec{q}_i, and the last token of the round from the same sampler under an independent key on the residual law r⃗I\vec{r}_I, or from the scheme’s normal step on p⃗N+1\vec{p}_{N+1} when it is the bonus token. TextSeal routes drafts and bonus tokens between its two keys as usual and uses the other key for the replacement token; textGrain, SynthID-Text and AutoMark’s copula, which has no speculative mode of its own, use their first key for drafts and bonus tokens and a second key for replacement tokens. TextSeal2 on its own recipe instead runs block verification between the watermarked distributions, as in its report [24].

Without speculative decoding

The copula dominates the coin flip. Interpolated at TextSeal’s Self-BLEU, TextSeal2 detects 10, 22 and 8 percentage points more 100-token answers than the coin flip [29] on Llama-3.1-8B, Qwen3.5-9B and Qwen3.5-27B respectively (paired 95% intervals [+4, +15], [+15, +25] and [+3, +12]; Figure 1a and Figure 7). Against textGrain, TextSeal2 is ahead on Llama-3.1-8B (+11 [+4, +17]), level on Qwen3.5-9B (+1 [−5, +8]), and behind on Qwen3.5-27B (−5 [−10, 0]). AutoMark’s copula, which coincides with TextSeal2 in distribution, stays within 3 points throughout, confirming that the gain over TextSeal comes from removing the coin flip rather than from implementation differences. With the public key alone, TextSeal2 still detects 27% / 67% of 100 / 200-token answers at ρ=0.9\rho= 0.9; merging four keys detects as much as merging two, so extra keys cost nothing (Appendix § 11).

Figure 7. Diversity (Self-BLEU; lower is more diverse) vs. detection of 100-token answers without speculative decoding, on three models (32 answers to each of 100 questions; copulas with the 53-bit PRF). Interpolated at TextSeal’s Self-BLEU, TextSeal2 gains 10, 22 and 8 percentage points over the coin flip on Llama-3.1-8B, Qwen3.5-9B and Qwen3.5-27B respectively, and AutoMark’s copula overlaps TextSeal2 throughout.

Entropy weighting. TextSeal’s entropy-weighted test improves every method by 4–23 points at 200 tokens, without changing the ranking (Figure 5, Table 6). On Qwen3.5-9B, TextSeal2 detects 78% / 97% of 100 / 200-token answers, against 74% / 97% for textGrain; under speculative decoding, 67% / 95%, against 48% / 87% for TextSeal and 36% / 76% for textGrain (Self-BLEU 18.9–20.1).

Figure 5. Entropy weighting improves detection for every watermark, including under speculative decoding. % of 100-token answers detected at a false-positive rate of 10−310^{-3} with key-only scores (light bars) and with TextSeal’s entropy-weighted test (full bars), at comparable Self-BLEU. The gains are 4–18 percentage points at 100 tokens and 4–23 at 200 tokens (Table 6); without speculative decoding, TextSeal2 and AutoMark’s copula coincide in distribution and differ only within their 95% intervals, as expected.

ModelMethodSelf-BLEUKey-onlyEntropy-weighted
Llama-3.1-8BTextSeal23.953 / 8258 / 91
Llama-3.1-8BtextGrain24.155 / 8667 / 94
Llama-3.1-8BAutoMark copula23.963 / 9071 / 95
Llama-3.1-8BTextSeal223.964 / 9071 / 95
Qwen3.5-9BTextSeal20.642 / 8259 / 93
Qwen3.5-9BtextGrain20.965 / 9174 / 97
Qwen3.5-9BSynthID-Text (10 layers)20.954 / 9272 / 98
Qwen3.5-9BAutoMark copula20.964 / 9377 / 97
Qwen3.5-9BTextSeal220.763 / 9178 / 97
Qwen3.5-27BTextSeal26.521 / 5332 / 76
Qwen3.5-27BtextGrain27.237 / 7144 / 84
Qwen3.5-27BAutoMark copula28.237 / 7348 / 87
Qwen3.5-27BTextSeal228.536 / 7147 / 87
Qwen3.5-9B, spec. dec.TextSeal20.131 / 7148 / 87
Qwen3.5-9B, spec. dec.textGrain, second key18.924 / 5936 / 76
Qwen3.5-9B, spec. dec.textGrain, own recipe*20.659 / 8969 / 96
Qwen3.5-9B, spec. dec.SynthID-Text (10 layers)18.414 / 4123 / 63
Qwen3.5-9B, spec. dec.AutoMark copula, second key19.023 / 5536 / 78
Qwen3.5-9B, spec. dec.TextSeal2, residual race19.155 / 8767 / 95

Table 6. Key-only vs. entropy-weighted detection (TextSeal’s weighted Gamma test): % of 100 / 200-token answers detected at a false-positive rate of 10−310^{-3}, with thresholds calibrated per test over 10 keys and 10,000 unwatermarked answers. First 1000 answers per configuration without speculative decoding; last block: all 3000 answers with speculative decoding (8 drafts, block verification). Strengths: α=0.5\alpha= 0.5 (TextSeal), β=0.5\beta= 0.5 (textGrain), ρ=0.9\rho= 0.9 (copulas, 53-bit PRF). For SynthID-Text, the key-only test is its official binomial test and the weighted test is the same Gamma test applied to twice its mean coin score. Self-BLEU over 32 answers to each of 100 questions. Weighting improves every method by 4–23 percentage points at 200 tokens, and the two copulas remain level without speculative decoding, as they must in distribution.

Robustness and cost. Under random word substitutions, TextSeal2 retains more detections than TextSeal and as many as textGrain, while paraphrasing by an LLM removes almost every watermark, as expected for token-level schemes (Table 7). The gains over the coin flip persist at temperature 0.7 (Appendix § 11). TextSeal2’s decoding step is within 0.6% of TextSeal’s once key-only work overlaps the forward pass (Table 5), and its post-logits work is one pass over the vocabulary (Appendix § 9).

BatchPlain samplingTextSealTextSeal2TextSeal, overlappedTextSeal2, overlapped
150.4450.8551.0850.6950.87
852.3952.5452.6452.5252.79
6456.4658.4361.3456.8156.93

Table 5. Table 5 Full decoding step (ms, median), Qwen3.5-9B; overlapped: key-only work on a side CUDA stream.

MethodClean10% subst.20% subst.30% var.Paraphr.
TextSeal (coin flip, α=0.5\alpha= 0.5)824616283
textGrain (β=0.5\beta= 0.5)916324485
AutoMark’s copula (ρ=0.9\rho= 0.9)926626485
TextSeal2 (ρ=0.9\rho= 0.9)926427466
TextSeal2 (ρ=0.95\rho= 0.95)947131517

Table 7. Robustness: % of 1000 200-token Qwen3.5-9B answers still detected after each attack, at the clean thresholds (10−310^{-3} false-positive rate, 10 keys; paraphraser: Llama-3.2-3B-Instruct).

Code and long generations. On the 500 MBPP test problems (Table 12), TextSeal2 detects 55% of answers at 256 tokens, against 39% / 46% / 50% for TextSeal/ textGrain / SynthID-Text (AutoMark’s copula: 59%). In long thinking traces (up to 2048 tokens; Table 13), 16% of positions on ELI5 and 34% on MBPP repeat an earlier window. There, continuing each window’s race from its leftovers instead of using it once detects 3.8 [+2.1, +5.6] points more at 256 tokens on ELI5 and 7.6 [+2.4, +12.8] points more at 512 tokens on MBPP, and raises median zz-scores, without measurable distortion (Table 13).

MethodDetected (128 / 256 / 512 tokens)
TextSeal (coin flip)30 / 39 / 42
textGrain38 / 46 / 48
SynthID-Text42 / 50 / 50
AutoMark copula54 / 59 / 59
TextSeal2, one use per window47 / 55 / 60

Table 12. Code: Qwen3.5-9B writing Python solutions to the 500 MBPP test problems, up to 512 new tokens (most answers end earlier, so detection saturates after 256 tokens). % of answers detected at 128 / 256 / 512 tokens, thresholds at a 1% false-positive rate over 10 keys and 2922 unwatermarked answers (all 974 MBPP problems, 3 seeds); same settings as on ELI5 (α=0.5\alpha= 0.5, ρ=0.9\rho= 0.9, β=0.5\beta= 0.5, 10 SynthID layers; the copulas use the 53-bit PRF), whose Qwen3.5-9B Self-BLEU is 20.6–20.9 for all five; one answer per problem, so code diversity is not measured. 11% of positions are a window’s second occurrence in the text.

Repeated windowsELI5MBPP
Detected @256Median z @256 / 1024 / 2048Detected @512Median z @1024 / 2048
One use per window93.06.0 / 15.9 / 22.853.24.8 / 5.6
Leftover continuation96.86.5 / 16.9 / 23.960.85.2 / 5.7
Δ\Delta leftover continuation+3.8 [+2.1, +5.6]+0.5 / +0.9 / +1.1+7.6 [+2.4, +12.8]+0.3 / +0.4

Table 13. Repeated windows in long generations: Qwen3.5-9B with thinking traces, up to 2048 tokens (ELI5: 1024 prompts, MBPP: 500 problems; ρ=0.9\rho= 0.9, 53-bit PRF). A window repeats at 16% of positions on ELI5 and 34% on MBPP. Detection at a false-positive rate of 10−310^{-3} (thresholds over 10 keys and 2048 / 1000 unwatermarked traces) and median z-score of the key-only test; both rules use the same detector. Δ\Delta: paired difference against one use per window, with 95% bootstrap interval over prompts. Distortion check, on 480 traces generated with one key per answer: the mean of −ln⁡px−H(p⃗)-\ln p_x - H(\vec{p}) at a window’s 2nd / 3rd+ occurrences is +3 [−6, +13] / −3 [−8, +2] millinats with leftover continuation, and −3 [−12, +7] / +2 [−3, +6] millinats without any watermark, so leftover continuation introduces no measurable distortion.

With speculative decoding

With block verification and 8 drafts, unwatermarked decoding emits 2.74 tokens per round, and every scheme but one keeps this number: paired over 480 prompts, TextSeal2’s residual race differs by +0.02 [−0.01, +0.05] tokens per round, as Proposition 1 predicts, and the baselines by at most 0.04 (Figure 11a in Appendix § 13). The exception is textGrain’s own recipe, which verifies the watermarked distributions and emits 5% fewer tokens per round (−0.14 [−0.17, −0.11]). Interpolated at TextSeal’s Self-BLEU, TextSeal2 detects 59% / 89% of 100 / 200-token answers, against 31% / 71% for TextSeal, 31% / 67% for textGrain, 25% / 60% for AutoMark’s copula, and 17% / 50% for SynthID-Text; all paired intervals against TextSeal2 exclude zero (Table 1, Figure 1b). textGrain’s own recipe comes closest (52% / 84%, paired differences +7 [−1, +15] / +5 [−1, +11] in favour of TextSeal2), at the cost of acceptance. With 4 and 16 drafts the ranking is the same, and TextSeal2 leads every baseline that keeps acceptance by 15–41 points at 200 tokens (Table 9); its lead over textGrain’s own recipe is significant with 4 drafts (+6 [+2, +10]) but not with 16 (+1 [−1, +7]). A second drafter–target pair from the Llama family gives the same ranking (Table 10). On AIME math problems, with answers of up to 8192 tokens, TextSeal2 also detects the most answers (41% at 2048 tokens, against 38% for textGrain’s own recipe and 7–22% for the other baselines), and accuracy is unchanged (Table 11); sweeping strengths, TextSeal2 also leads at equal diversity, with 30% detected at 2048 tokens at TextSeal’s Self-BLEU against 23% for the coin flip and 9–21% for the other baselines (Figure 8).

MethodN=4N = 4 Tokens / roundN=4N = 4 DetectedN=8N = 8 Tokens / roundN=8N = 8 DetectedN=16N = 16 Tokens / roundN=16N = 16 Detected
No watermark2.44–2.74–2.85–
TextSeal (coin flip)2.4634 / 732.7731 / 712.8730 / 68
textGrain2.4530 / 652.7531 / 672.8428 / 62
textGrain, own recipe2.3648 / 822.6152 / 842.6851 / 85
SynthID-Text2.4419 / 482.7517 / 502.8517 / 46
AutoMark copula2.4424 / 572.7325 / 602.8323 / 57
TextSeal2, residual race2.4458 / 882.7459 / 892.8553 / 86

Table 9. Number of drafts per round: Qwen3.5-0.8B drafting NN tokens for Qwen3.5-9B, block verification, every configuration generated separately for each NN (3000 answers for detection, 100 questions × 16 answers for Self-BLEU). Tokens per round: emitted tokens over target passes, at α=0.5\alpha= 0.5, β=0.5\beta= 0.5, 10 layers and ρ=0.9\rho= 0.9. Detected: % of 100 / 200-token answers at a false-positive rate of 10−310^{-3}, interpolated at TextSeal’s Self-BLEU for the same NN (19.8 / 20.1 / 19.6). Paired over 480 prompts, only textGrain’s own recipe lowers the tokens per round (−0.08 / −0.14 / −0.18); the distortion tests of Figure 11 detect the raw-lane-length control at every NN, and no other scheme on the tests that apply to it, except that at N=16N = 16 the residual race and textGrain’s own recipe both sit 7 millinats below the same unwatermarked run in the all-token test (intervals [−13, −1]), which points to noise in that shared reference.

MethodTokens per round (N=4N = 4 / 88)Detected, N=4N = 4Detected, N=8N = 8
No watermark3.20 / 4.08––
TextSeal (coin flip, other key for the last token)3.21 / 4.0936 / 7237 / 62
textGrain, other key for the last token3.21 / 4.1420 / 5118 / 43
textGrain, own recipe (lowers acceptance)3.11 / 3.8855 / 8350 / 79
TextSeal2, residual race3.20 / 4.1163 / 8755 / 81

Table 10. Second drafter–target pair: Llama-3.2-1B-Instruct drafting N=4N = 4 or 8 tokens per round for Llama-3.1-8B-Instruct, with block verification (ELI5, 240 prompts per row, α=0.5\alpha= 0.5, ρ=0.9\rho= 0.9, β=0.5\beta= 0.5, TextSeal’s 13-bit PRF for every scheme). Tokens per round: emitted tokens over target passes. Detected: % of 100 / 200-token answers at a false-positive rate of 10−310^{-3}, thresholds over 10 keys and 10,000 unwatermarked Llama-3.1-8B-Instruct answers; 95% Wilson intervals are about ±6 points.

MethodTokens / roundAccuracy (%)Self-BLEUFinished (%)25651210242048
No watermark5.4271.144.868––––
TextSeal (coin flip)5.4369.246.166151322
textGrain5.3969.345.86612615
textGrain, own recipe5.3069.746.868382238
SynthID-Text5.3969.245.66824919
AutoMark copula5.3870.245.8681247
TextSeal2, residual race5.3970.646.5684112441

Table 11. AIME under speculative decoding: Qwen3.5-0.8B drafting 8 tokens per round for Qwen3.5-9B with block verification, 16 answers to each of 100 AIME problems (thinking disabled, prompt asking for a \boxed{} answer, temperature 1, top-pp 0.95, up to 8192 tokens), at α=0.5\alpha= 0.5, β=0.5\beta= 0.5, 10 layers and ρ=0.9\rho= 0.9. Tokens per round: emitted tokens over target passes; only textGrain’s own recipe lowers it (−0.11-0.11 [−0.14-0.14, −0.09-0.09], paired by problem with unwatermarked decoding). Accuracy: last \boxed{} answer; paired by problem, every scheme is within 2 points of unwatermarked decoding, all 95% intervals containing zero. Self-BLEU: over each problem’s 16 answers, first 1024 tokens. Finished: answers that end before 8192 tokens. Detected: % of answers detected at a false-positive rate of 10−310^{-3} on their first 256 / 512 / 1024 / 2048 tokens, with each method’s detector; thresholds are the empirical 10−310^{-3} quantile over the 1600 unwatermarked answers scored under 10 keys (1.6⋅1041.6 \cdot10^4 null pp-values; realised false-positive rates 0.05–0.15%); 95% Wilson intervals are at most ±2.4 points. Strengths are defaults, not diversity-matched; Self-BLEU varies by at most 2 points.

Figure 8. Diversity (Self-BLEU over each problem’s 16 answers, first 1024 tokens; lower is more diverse) vs. detection at a false-positive rate of 10−310^{-3} on AIME under speculative decoding, on the first 1024 (a) and 2048 (b) tokens; same setup as Table 11, sweeping each method’s strength. Interpolated at TextSeal’s Self-BLEU (46.1), TextSeal2 detects 30% of answers at 2048 tokens, against 23% for the coin flip, 21% for SynthID-Text, 17% for textGrain and 9% for AutoMark’s copula; paired by problem, its lead is significant over all but the coin flip (+7.4 [−0.2-0.2, +19.5]). Error bars: 95% Wilson intervals.

Figure 11. Is speculative decoding with a watermark still distortion-free? Paired differences from unwatermarked block verification (8 drafts per round) on the same 480 prompts, with one key per answer (95% paired bootstrap intervals). (a) Tokens per round. (b) Mean of −ln⁡rI,y−H(r⃗I)-\ln r_{I,y} - H(\vec{r}_I) at the last token yy of each round, which has expectation zero when it follows block verification’s residual law r⃗I\vec{r}_I (not shown for textGrain’s own recipe, whose last token follows the residual of the watermarked distributions). (c) Mean of −ln⁡px−H(p⃗)-\ln p_x - H(\vec{p}) over all tokens, which has expectation zero for any distortion-free scheme. Only the control (racing on raw lane lengths, Remark 1) deviates in (c), and only textGrain’s own recipe lowers tokens per round.

Memorylessness is what makes this work. AutoMark’s copula coincides with TextSeal2 in distribution, so the entire gap between them in Figure 1b (25% / 60% against 59% / 89%) comes from how the last token of each round is drawn: under an independent key, or by the residual race on the draft races’ leftovers. Independent keys waste signal twice over. The last tokens (37% of the text) carry only the second key’s signal, except at the bonus position (5% of rounds), and under the draft key they score 0.84 on average – below the null mean of 1, because a token drawn from block verification’s residual law lost its draft race – actively diluting the fused score. The residual race instead keeps the keyed values in play: last tokens score 1.34 under s∗s^*, against 1.42 for kept drafts (Appendix § 13).

Distortion. Proposition 1 guarantees the exactness of the residual race under an ideal PRF; we check the implementation. Generating with one key per answer (independent draws over keys) and reading the laws p⃗\vec{p} and q⃗\vec{q} used at generation, we test whether −ln⁡px−H(p⃗)-\ln p_x-H(\vec{p}) over all tokens and −ln⁡rI,y−H(r⃗I)-\ln r_{I,y}-H(\vec{r}_I) at the last token of each round have mean zero, as they must without distortion. TextSeal2 and the baselines are indistinguishable from unwatermarked block verification on both tests (textGrain’s own recipe on the all-token test, the only one that applies to it), whereas racing on the raw lane lengths fails the all-token test, by +11.7 [+5.8, +17.8] millinats per token (Figure 11).

Conclusion

TextSeal2 satisfies the four requirements we set out for a deployed watermark: it is exactly distortion-free, preserves diversity under a fixed key, stays within 0.9% of plain sampling when key-only work overlaps the forward pass, and maintains the acceptance rate of speculative decoding. Its detection is comparable to that of textGrain overall – ahead on Llama-3.1-8B, level on Qwen3.5-9B, and slightly behind on Qwen3.5-27B at 100 tokens – without textGrain’s per-token optimisation. Our analysis attributes the previous gap to TextSeal’s coin-flip routing rather than to the absence of optimal transport: merging the two keys into one virtual key removes the 2\sqrt{2} loss in signal-to-noise ratio, while still allowing one key to be published for public detection. For speculative decoding, drawing the drafts by keyed races and choosing block verification’s outcome by a race on their leftovers carries the watermark without lowering acceptance.

Limitations. Our evaluation covers three domains (open-ended ELI5 answers, MBPP code, and AIME math problems under speculative decoding) and three open-weight models, at temperatures 0.7–1. With block verification and 8 drafts, speculative decoding costs 2–4 percentage points of detection relative to standard decoding, and our speculative-decoding implementation is unbatched, so we do not measure wall-clock speedups. We do not analyse attacks that exploit knowledge of the public key beyond the private verification it enables ([27]). Finally, textGrain’s code is not public, so we use our reimplementation (Appendix § 13); its deployed parameters may differ.

Proofs

This appendix proves the results of § 3. Throughout, keyed values at an unused window are i.i.d. uniform (ideal PRF), and the copula noise η\eta and the values drawn afresh at used windows, the only other randomness, are independent of them.

Copula mixing

Lemma 2 (Distortion-free for every coupling). Let Rv(1),Rv(2)R_v^{(1)}, R_v^{(2)} be i.i.d. uniform and ηv\eta_v i.i.d. N(0,1)\mathcal{N}(0,1), independent of them. For every ρ∈[0,1]\rho\in[0,1], the values R~v=Φ(ρGv⋆+1−ρ2ηv)\widetilde{R}_v = \Phi\left(\rho G_v^\star+ \sqrt{1-\rho^2}\eta_v\right) of (4) are i.i.d. uniform, so the race (1) on them emits vv with probability pvp_v.

Proof. Each Gv(j)=Φ−1(Rv(j))G_v^{(j)} = \Phi^{-1}(R_v^{(j)}) is N(0,1)\mathcal{N}(0,1), and the two keys are independent, so Gv⋆G_v^\star is N(0,1)\mathcal{N}(0,1). Zv=ρGv⋆+1−ρ2ηvZ_v = \rho G_v^\star+ \sqrt{1-\rho^2}\eta_v is a sum of independent centred Gaussians of variance ρ2+(1−ρ2)=1\rho^2 + (1-\rho^2)=1. Distinct candidates use disjoint keyed values and noise, so the ZvZ_v are i.i.d. N(0,1)\mathcal{N}(0,1), the R~v=Φ(Zv)\widetilde{R}_v = \Phi(Z_v) are i.i.d. uniform, and the race emits vv with probability pvp_v (§ 2).

Exponential races and their leftovers

Lemma 1 (restated). If the EvE_v are i.i.d. Exp⁡(1)\operatorname{Exp}(1) and q⃗\vec{q} is a probability vector, the race x=arg min⁡vEv/qvx = \operatorname*{arg\,min}_v E_v/q_v, with time τ\tau, emits vv with probability qvq_v, and given (x,τ)(x,\tau) the leftovers Ev−τqvE_v-\tau q_v, v≠xv \ne x, are again i.i.d. Exp⁡(1)\operatorname{Exp}(1).

Proof. Each clock Tv=Ev/qvT_v=E_v/q_v rings after an Exp⁡(qv)\operatorname{Exp}(q_v) time, with density qve−qvtq_v e^{-q_vt}. The winner xx is the first clock to ring, and τ\tau is its time. We compute the joint law of the winner, its time, and the losing clocks’ leftovers, and check that it factorises.

Fix a token xx, a time t≥0t \ge0, and thresholds bv≥0b_v \ge0 for v≠xv \ne x. The event “xx wins at time tt and every other leftover exceeds bvb_v” requires Ex/qx∈dtE_x/q_x \in dt and Ev>tqv+bvE_v > tq_v+b_v for v≠xv \ne x. The clocks are independent, so this probability is one density term times ∣V∣−1|V|-1 tail terms:

P(x wins, τ∈dt, Ev−τqv>bv ∀v≠x)=qxe−qxt dt∏v≠xe−(qvt+bv)=qxe−t⏟τ dt∏v≠xe−bv⏟leftovers,\mathbb{P}\left(x\text{ wins},\ \tau\in dt,\ E_v-\tau q_v>b_v\ \forall v\ne x\right)=q_xe^{-q_xt}\,dt\prod_{v\ne x}e^{-(q_vt+b_v)}=q_x\underbrace{e^{-t}}_{\tau}\,dt\prod_{v\ne x}\underbrace{e^{-b_v}}_{\text{leftovers}},

using ∑vqv=1\sum_v q_v=1.

The right side factorises in xx, tt, and the bvb_v. Reading them off: xx has law q⃗\vec{q}, τ\tau has density e−te^{-t} on t≥0t \ge0 and is hence Exp⁡(1)\operatorname{Exp}(1), and each leftover Ev−τqvE_v-\tau q_v has tail e−bve^{-b_v}, so the leftovers are i.i.d. Exp⁡(1)\operatorname{Exp}(1), independent of (τ,x)(\tau,x).

Tokens with qv=0q_v=0 never win, and their EvE_v are left untouched.

Block verification

Fix a round with drafts d1,…,dNd_1,\ldots,d_N, drafter laws q⃗1,…,q⃗N\vec{q}_1,\ldots,\vec{q}_N and target laws p⃗1,…,p⃗N+1\vec{p}_1,\ldots,\vec{p}_{N+1}. For i≤Ni \le N, block verification (Sun et al. [30]) uses

c0=1,ci=min⁡(1,ci−1pi,diqi,di),ui,v=(ci−1pi,v−qi,v)+,zi=∑vui,v,ri,v=ui,vzi,(10)c_0=1,\quad c_i=\min\left(1,c_{i-1}\frac{p_{i,d_i}}{q_{i,d_i}}\right),\quad u_{i,v}=(c_{i-1}p_{i,v}-q_{i,v})_+,\quad z_i=\sum_v u_{i,v},\quad r_{i,v}=\frac{u_{i,v}}{z_i}, \tag*{(10)}

with r⃗N+1=p⃗N+1\vec{r}_{N+1}=\vec{p}_{N+1}. By the retention identity of Sun et al. [30], cic_i is the probability that the round keeps d1,…,did_1,\ldots,d_i, given its first ii drafts and averaged over the later ones. It drops when the target likes a draft less than the drafter does and recovers when later drafts are likelier under the target, so a likely later draft can rescue an unlikely earlier one. With ci−1=1c_{i-1}=1, r⃗i\vec{r}_i is the residual (3) of token-level verification. The success probabilities are

h1=1,hi=zizi+1−ci−1(2≤i≤N),hN+1=cN,h_1=1,\qquad h_i=\frac{z_i}{z_i+1-c_{i-1}}\quad(2\le i\le N),\qquad h_{N+1}=c_N,

with hi=1h_i=1 when ziz_i and 1−ci−11-c_{i-1} both vanish. Native block verification tests each position i≥2i\ge2 with an independent uniform, which succeeds with probability hih_i, stops at the last success, or at position 1 if every test fails, and draws y∼r⃗Iy\sim\vec{r}_I. Given the drafts, its outcome therefore has the law

πi,v=P(I=i)ri,v,P(I=i)=hi∏j>i(1−hj),(11)\pi_{i,v}=\mathbb{P}(I=i)r_{i,v},\qquad\mathbb{P}(I=i)=h_i\prod_{j>i}(1-h_j), \tag*{(11)}

whose total is 1−∏j=1N+1(1−hj)=11-\prod_{j=1}^{N+1}(1-h_j)=1, since h1=1h_1=1. Sun et al. [30] index by the number of kept drafts: their pip_i, hiblockh_i^{\mathrm{block}} and τ\tau are our cic_i, hi+1h_{i+1} and I−1I-1, and their residual after τ\tau kept drafts is r⃗τ+1\vec{r}_{\tau+1}. For N=1N=1 it is token-level verification: P(I=1)=1−min⁡(1,p1,d1/q1,d1)\mathbb{P}(I=1)=1-\min(1,p_{1,d_1}/q_{1,d_1}), and r⃗1\vec{r}_1 is the residual (3).

Lemma 3 (Zero mass on the draft lanes). For every i≤Ni\le N, πi,di=0\pi_{i,d_i}=0, and P(I=i)=0\mathbb{P}(I=i)=0 if zi=0z_i=0.

Proof. If ci=1c_i=1, then hi+1=1h_{i+1}=1: for i<Ni<N because 1−ci=01-c_i=0, and for i=Ni=N because hN+1=cNh_{N+1}=c_N. The factor 1−hi+11-h_{i+1} of P(I=i)\mathbb{P}(I=i) then vanishes. If ri,di>0r_{i,d_i}>0, then ci−1pi,di>qi,dic_{i-1}p_{i,d_i}>q_{i,d_i}, so ci=1c_i=1 and P(I=i)=0\mathbb{P}(I=i)=0; otherwise ri,di=0r_{i,d_i}=0. Either way, πi,di=0\pi_{i,d_i}=0. If zi=0z_i=0 and ci−1<1c_{i-1}<1, then i≥2i\ge2 and hi=0h_i=0. If zi=0z_i=0 and ci−1=1c_{i-1}=1, then pi,v≤qi,vp_{i,v}\le q_{i,v} for every vv, so p⃗i=q⃗i\vec{p}_i=\vec{q}_i, ci=1c_i=1, and P(I=i)=0\mathbb{P}(I=i)=0 as above. □

So r⃗i\vec{r}_i is needed only when zi>0z_i>0.

Exactness of the residual race

Proposition 1 (restated). Suppose that every round reads i.i.d. uniform keyed values at its windows, independent across positions and of earlier rounds, and independent copula noise. (i) Given the drafts, (I,y)(I,y) has law π⃗\vec{\pi}, so the number of tokens per round, and hence acceptance, is that of block verification. (ii) The generated text follows the target’s law. (iii) The window rule (Appendix § 6.§ 5) meets this condition: the text is distortion-free over keys.

Proof. Fix everything before the round, and write E⃗i=(Ei,v)v∈V\vec{E}_i=(E_{i,v})_{v\in\mathcal{V}} for the lane lengths of position ii.

(i) Leftovers given the drafts. Fix a sequence δ=(δ1,…,δN)\delta=(\delta_1,\ldots,\delta_N) of drafts. The windows wi\mathbf{w}_i and the laws q⃗i\vec{q}_i, p⃗i\vec{p}_i and π⃗\vec{\pi} are then fixed, and by assumption and Lemma 2, E⃗1,…,E⃗N+1\vec{E}_1,\ldots,\vec{E}_{N+1} are independent families of i.i.d. Exp⁡(1)\operatorname{Exp}(1) variables. The event {d=δ}\{d=\delta\} is the intersection of the events {arg minvEi,v/qi,v=δi}\{\mathop{\rm arg\,min}_v E_{i,v}/q_{i,v}=\delta_i\}, i≤Ni\le N, each of which involves only E⃗i\vec{E}_i. So given d=δd=\delta, the families stay independent, E⃗N+1\vec{E}_{N+1} keeps its law, and each E⃗i\vec{E}_i, i≤Ni\le N, has the law of i.i.d. Exp⁡(1)\operatorname{Exp}(1) variables conditioned on δi\delta_i winning the race with weights q⃗i\vec{q}_i. By Lemma 1, the leftovers Ei,v−τiqi,vE_{i,v}-\tau_iq_{i,v}, v≠δiv\ne\delta_i, are then i.i.d. Exp⁡(1)\operatorname{Exp}(1) and independent of τi\tau_i; tokens with qi,v=0q_{i,v}=0 keep Ei,vE_{i,v} as their leftover. Hence, given the drafts, the leftovers of all lanes but the drafts’ own, together with E⃗N+1\vec{E}_{N+1}, are i.i.d. Exp⁡(1)\operatorname{Exp}(1) and independent of τ1,…,τN\tau_1,\ldots,\tau_N.

(i) Relay. A draft’s own lane has leftover Ei,di−τiqi,di=0E_{i,d_i}-\tau_iq_{i,d_i}=0, but πi,di=0\pi_{i,d_i}=0 by Lemma 3, so (6) reads only the leftovers above. It is then the race of Lemma 1 over the outcomes with πi,v>0\pi_{i,v}>0, with these leftovers as values and the weights π⃗\vec{\pi}, which sum to 1. So P(I=i,y=v∣d)=πi,v\mathbb{P}(I=i,y=v\mid d)=\pi_{i,v}, the law of block verification given the drafts, and the proof of Lemma 1 shows that the winning time TT is Exp⁡(1)\operatorname{Exp}(1) and independent of the winner. By Lemma 1, di∼q⃗id_i\sim\vec{q}_i given d<id_{<i}, so (d,I,y)(d,I,y) has the law of a round of native block verification; in particular, acceptance is kept.

(ii) Output. Given everything before it, each round emits d<Iyd_{<I}y with the law of a native round, so by induction over rounds the text has the law of speculative decoding with block verification, which is the target’s law ([30 Theorem 1]).

(iii) Window rule. Let FF contain the emitted prefix and all randomness used to generate it. Under the rule of Appendix § 6.§ 5, position ii of a round reads either the keyed values of a window that was raced neither before the round nor at an earlier position of the round, or freshly drawn values. Keyed values at a window that no race has read are i.i.d. uniform and independent of the values at other windows and of F\mathcal{F}, and the copula noise is drawn anew at every position. So the values read at positions 1,…,N+11,\ldots,N+1 are i.i.d. uniform and independent of F\mathcal{F}, as (i) and (ii) require. □

What the detector reads. A kept draft did_i, i<Ii<I, has lane length Ei,di=τiqi,diE_{i,d_i}=\tau_iq_{i,d_i}, where τi\tau_i is Exp⁡(1)\operatorname{Exp}(1) and independent of (I,y)(I,y) given the drafts, because the relay reads only leftovers. Kept drafts therefore carry the signal of their draft races. The last token has lane length EI,y=τIqI,y+πI,yTE_{I,y}=\tau_Iq_{I,y}+\pi_{I,y}T, with T∼Exp⁡(1)T\sim\operatorname{Exp}(1) the winning time of the relay, and carries the signal of the relay, which includes the choice of where to stop. Every emitted token is selected by a race on its own keyed values, so the detector is unchanged: it sums the scores and needs no round boundaries.

Remark 1 (Racing on raw lane lengths is biased). Racing on the raw lane lengths Ei,vE_{i,v} instead of the leftovers uses Ei,v=τiqi,v+(Ei,v−τiqi,v)E_{i,v}=\tau_iq_{i,v}+(E_{i,v}-\tau_iq_{i,v}). The extra term τiqi,v\tau_iq_{i,v} penalises the tokens the drafter liked and the positions whose draft race ended late, while the bonus position has no such term. With one draft, q⃗1=(0.6,0.3,0.1)\vec{q}_1=(0.6,0.3,0.1), p⃗1=(0.2,0.4,0.4)\vec{p}_1=(0.2,0.4,0.4) and d1d_1 the first token, the residual race stops at position 1 with probability 2/32/3 and then emits the third token with probability r1,3=3/4r_{1,3}=3/4. On the raw lane lengths, the race stops there with probability 85/144≈0.5985/144\approx0.59 and emits the third token with probability 15/17≈0.8815/17\approx0.88. The same bias affects the single-position rule that re-runs the race with weights r⃗1\vec{r}_1 on the raw values after a rejection by a fresh coin: it picks the third token 86% of the time.

Remark 2 (Token-level verification). Token-level verification keeps draft ii with probability ai=min⁡(1,pi,di/qi,di)a_i=\min(1,p_{i,d_i}/q_{i,d_i}) and, at the first rejection, emits a token from the residual r⃗itok\vec{r}^{\mathrm{tok}}_i of (3) at that position. Given the drafts, its outcome has the law

πi,vtok=(∏j<iaj)(1−ai)ri,vtok(i≤N),πN+1,vtok=(∏j≤Naj)pN+1,v.\pi^{\mathrm{tok}}_{i,v}=\left(\prod_{j<i}a_j\right)(1-a_i)r^{\mathrm{tok}}_{i,v}\quad(i\le N),\qquad\pi^{\mathrm{tok}}_{N+1,v}=\left(\prod_{j\le N}a_j\right)p_{N+1,v}.

If ai=1a_i=1, the factor 1−ai1-a_i vanishes; otherwise pi,di<qi,dip_{i,d_i}<q_{i,d_i} and ri,ditok=0r^{\mathrm{tok}}_{i,d_i}=0. Either way, this law also vanishes on the draft lanes. The proof of Proposition 1 then applies with π⃗tok\vec{\pi}^{\mathrm{tok}} in place of π⃗\vec{\pi}, and with the validity of token-level verification (Leviathan et al., [20]; Chen et al., [3]) in place of that of block verification: the race (6) on π⃗tok\vec{\pi}^{\mathrm{tok}} keeps token-level verification’s law without a coin.

Repeated windows

The rules. At generation, a key’s values at a given window serve at most one emitted token per answer, and a position whose window was already used draws its values afresh, which is plain sampling (Appendix § 13). Under speculative decoding, a window counts as used once a round races it, whether or not its position is kept. The positions after II are discarded, but their keyed values still shaped the outcome: the relay reads the leftovers of every position, and P(I=i)\mathbb{P}(I=i) depends on the later drafts. A position whose window was already raced, earlier in the text or earlier in the same round, therefore uses freshly drawn values, so the positions of a round have independent values. A round races N+1N+1 windows but emits only II tokens, so more positions fall back to fresh values than without speculative decoding. Without speculative decoding, TextSeal2 can instead continue a window’s race at each later visit, as the ResidualRace of AutoMark does (Gloaguen et al., [12]): it keeps the leftovers of the window’s last race and gives the token that won it a freshly drawn exponential. At detection, each (window, token) pair is scored once, at its first occurrence.

The whole text is distortion-free. Reusing a window’s keyed values re-runs the same race: each token still follows the model given its prefix, averaged over keys, but tokens at repeated windows become dependent, so the law of the whole answer is not the model’s. Under the rules, each position reads keyed values that no earlier race has read, or freshly drawn values, which are independent of the prefix and of the randomness that produced it. Each token then follows the model given its prefix, and the chain rule over positions gives the model’s law for the whole answer; under speculative decoding, this is part (iii) of Proposition 1, whose proof uses exactly this independence. With leftover continuation, given the earlier visits of a window, the leftovers of its losing tokens are again i.i.d. Exp(1) by Lemma 1, and the winner’s fresh value is independent of them, so every visit is distortion-free given the past.

The detection tests stay valid. Under H0H_0, the text does not depend on the key, so with an ideal PRF the scores (7) of distinct (window, token) pairs are i.i.d. Exp(1) given the text, while a repeated pair only repeats its score. Scoring each pair once leaves nn distinct pairs, so SS in (8) follows Gamma(n,1)(n,1) exactly, and distinct tokens at the same window still give independent scores. The score of a pair depends only on the key, the window and the token, so it is the same at every occurrence, under either generation rule. When cropping or editing changes which occurrence of a pair comes first, the detector therefore reads the same score for that pair.

Acceptance

For N=1N=1, it keeps the draft with probability 1−TV⁡(p⃗1,q⃗1)1-\operatorname{TV}(\vec{p}_1,\vec{q}_1), and no scheme with a single draft at a position does better while preserving the output distribution.

Proposition 2 (Acceptance bound). At one position, let a scheme draft DD and emit OO, and let q~\tilde{q} and p~\tilde{p} be their laws given the key, with E[q~]=q⃗\mathbb{E}[\tilde{q}]=\vec{q} and E[p~]=p⃗\mathbb{E}[\tilde{p}]=\vec{p} over keys. Then P(accept)≤1−E[TV⁡(p~,q~)]≤1−TV⁡(p⃗,q⃗)\mathbb{P}(\mathrm{accept})\leq1-\mathbb{E}[\operatorname{TV}(\tilde{p},\tilde{q})]\leq1-\operatorname{TV}(\vec{p},\vec{q}), and the second inequality is strict iff for some vv, p~v−q~v\tilde{p}_v-\tilde{q}_v takes both signs with positive probability. Verifying on watermarked laws (accepting DD with probability min⁡(1,p~D/q~D)\min(1,\tilde{p}_D/\tilde{q}_D)) attains the first bound; the raw test with an independent coin, and the residual race with N=1N=1, attain the second.

Proof. We adapt the sum-over-min step and the expected-TV step from the proof of Hu and Huang [15], with the second step proved there through the dual form of TV. Fix the key, and take probabilities given it. An accepted draft is the emitted token, and (D,O)(D,O) is a coupling of q~\tilde{q} and p~\tilde{p}, so

P(accept)≤P(O=D)=∑vP(D=O=v)≤∑vmin⁡(p~v,q~v)=1−TV⁡(p~,q~).\mathbb{P}(\mathrm{accept})\leq\mathbb{P}(O=D)=\sum_v\mathbb{P}(D=O=v)\leq\sum_v\min(\tilde{p}_v,\tilde{q}_v)=1-\operatorname{TV}(\tilde{p},\tilde{q}).

Averaging over keys gives the first inequality.

For the second, let Yv=p~v−q~vY_v=\tilde{p}_v-\tilde{q}_v, so that TV⁡(p~,q~)=∑v(Yv)+\operatorname{TV}(\tilde{p},\tilde{q})=\sum_v(Y_v)_+ and E[Yv]=pv−qv\mathbb{E}[Y_v]=p_v-q_v. Since y↦y+y\mapsto y_+ is convex, Jensen’s inequality gives E[(Yv)+]≥(pv−qv)+\mathbb{E}[(Y_v)_+]\geq(p_v-q_v)_+ for each vv. Summing over vv gives the second inequality, which is strict iff one of these terms is. If YvY_v takes both signs with positive probability, then (Yv)+>Yv(Y_v)_+>Y_v on {Yv<0}\{Y_v<0\} and (Yv)+>0(Y_v)_+>0 on {Yv>0}\{Y_v>0\}, so E[(Yv)+]>max⁡(E[Yv],0)\mathbb{E}[(Y_v)_+]>\max(\mathbb{E}[Y_v],0) and the term is strict. Otherwise (Yv)+(Y_v)_+ is almost surely YvY_v or almost surely 00, and the term is an equality.

At a fixed key, verifying on the watermarked laws is speculative sampling with draft law q~\tilde{q} and target law p~\tilde{p}, which accepts with probability ∑vq~vmin⁡(1,p~v/q~v)=1−TV⁡(p~,q~)\sum_v\tilde{q}_v\min(1,\tilde{p}_v/\tilde{q}_v)=1-\operatorname{TV}(\tilde{p},\tilde{q}). The raw accept test with an independent coin accepts with probability ∑vqvmin⁡(1,pv/qv)=1−TV⁡(p⃗,q⃗)\sum_v q_v\min(1,p_v/q_v)=1-\operatorname{TV}(\vec{p},\vec{q}), since the draft follows q⃗\vec{q} over keys. Given the draft, the residual race with N=1N=1 keeps it with probability P(I=2)=c1=min⁡(1,pD/qD)\mathbb{P}(I=2)=c_1=\min(1,p_D/q_D) by Proposition 1, the same as the raw test. □

For N>1N>1, block verification decodes at least as many tokens in expectation as any valid draft verification algorithm [30 Theorem 2], and the residual race inherits it (Proposition 1).

Entropy removed

Let πρ(⋅∣G⋆)\pi_{\rho}(\cdot\mid G^\star) be the law of the emitted token XX given the keyed Gaussians, over the newly drawn noise. As in textGrain Li et al., [24], the entropy the key removes is

Δ(p⃗;p⃗)=I(G⋆;X)=H(p⃗)−EG⋆[H(πρ(⋅∣G⋆))].(12)\Delta(\vec{p};\vec{p})=I(G^\star;X)=H(\vec{p})-\mathbb{E}_{G^\star}\left[H\left(\pi_{\rho}(\cdot\mid G^\star)\right)\right]. \tag*{(12)}

Theorem 1 (The coupling sets the entropy removed). *If H(p⃗)>0H(\vec{p})>0, then p⃗↦Δ(p⃗;p⃗)\vec{p}\mapsto\Delta(\vec{p};\vec{p}) is continuous and strictly increasing on [0,1][0,1], from Δ(0⃗;p⃗)=0\Delta(\vec{0};\vec{p})=0 to Δ(1⃗;p⃗)=H(p⃗)\Delta(\vec{1};\vec{p})=H(\vec{p}). Hence for every target $\delta\in

[0,H(p⃗)][0, H(\vec{p})] there is a unique ρ∗(δ;p⃗)\rho^*(\delta;\vec{p}) with Δ(ρ∗;p⃗)=δ\Delta(\rho^*;\vec{p}) = \delta, and running the race with ρt=ρ∗(δt;p⃗(t))\rho_t = \rho^*(\delta_t;\vec{p}^{(t)}) removes exactly δt\delta_t at every previously unused window, keeps the token law p⃗(t)\vec{p}^{(t)}, and leaves the detector unchanged.

Under speculative decoding, ρt\rho_t may depend on the draft distribution q⃗(t)\vec{q}^{(t)} but not on the target distribution p⃗(t)\vec{p}^{(t)}: the drafter fixes its race inputs R~v\widetilde{R}_v, and hence ρt\rho_t, before the target computes p⃗(t)\vec{p}^{(t)}.

Proof. Write A=G∗A = G^*, Z=ρA+1−ρ2 ηZ = \rho A + \sqrt{1-\rho^2}\,\eta, and Δ(ρ)=I(A;X)\Delta(\rho) = I(A;X).

Step 1: endpoints. At ρ=0\rho= 0, Z=ηZ = \eta is independent of AA, so XX depends only on η\eta and Δ=0\Delta= 0. At ρ=1\rho= 1, Z=AZ = A, so XX is a function of AA and Δ=H(X)=H(p⃗)\Delta= H(X) = H(\vec{p}), since X∼p⃗X \sim\vec{p} (Lemma 2).

Step 2: monotone. Take 0<ρ′<ρ≤10 < \rho' < \rho\le1 and let c=ρ′/ρc = \rho'/\rho. Draw an independent ξ∼N(0,I)\xi\sim\mathcal{N}(0,I) and form the degraded key A′=cA+1−c2 ξA' = cA + \sqrt{1-c^2}\,\xi, which is standard Gaussian. Run the race at coupling ρ\rho on A′A', giving X~\widetilde{X}. Since

ρA′+1−ρ2 η=ρ′A+ρ2−ρ′2 ξ+1−ρ2 η,\rho A' + \sqrt{1-\rho^2}\,\eta= \rho' A + \sqrt{\rho^2-\rho'^2}\,\xi+ \sqrt{1-\rho^2}\,\eta,

and the last two terms form a Gaussian of variance (ρ2−ρ′2)+(1−ρ2)=1−ρ′2(\rho^2-\rho'^2) + (1-\rho^2) = 1-\rho'^2 independent of AA, the pair (A,X~)(A,\widetilde{X}) has the law of (A,X)(A,X) at coupling ρ′\rho', while (A′,X~)(A',\widetilde{X}) has the law at coupling ρ\rho. As A→A′→X~A \to A' \to\widetilde{X} is a Markov chain, data processing gives Δ(ρ′)=I(A;X~)≤I(A′;X~)=Δ(ρ)\Delta(\rho') = I(A;\widetilde{X}) \le I(A';\widetilde{X}) = \Delta(\rho).

Step 3: strict. The chain rule gives I(A′;X~)−I(A;X~)=I(A′;X~∣A)I(A';\widetilde{X}) - I(A;\widetilde{X}) = I(A';\widetilde{X}\mid A). It vanishes only if the law of X~\widetilde{X} given (A,A′)(A,A') depends on AA alone. Given AA, A′A' still has a density that is positive everywhere, so this law would have to be constant in A′A'. It is not: for a token vv with pv>0p_v > 0, sending Av′→+∞A'_v \to+\infty drives the race input Φ(ρAv′+1−ρ2 ηv)\Phi(\rho A'_v + \sqrt{1-\rho^2}\,\eta_v) to 11, so vv wins almost surely, and since H(p⃗)>0H(\vec{p}) > 0 the same holds for a second token ww with pw>0p_w > 0. Hence I(A′;X~∣A)>0I(A';\widetilde{X}\mid A) > 0 and the inequality is strict.

Step 4: continuous. Write Δ(ρ)=H(p⃗)−EA[H(πρ(⋅∣A))]\Delta(\rho) = H(\vec{p}) - \mathbb{E}_A[H(\pi_\rho(\cdot\mid A))], with H(p⃗)H(\vec{p}) fixed. For almost every AA, the race inputs move continuously with ρ\rho, and the argmax is almost surely unique, so the conditional law πρ(⋅∣A)\pi_\rho(\cdot\mid A) moves continuously. Entropy is continuous and bounded by ln⁡∣V∣\ln|\mathcal{V}|, so dominated convergence makes the expectation, hence Δ\Delta, continuous.

Conclusion. ρ↦Δ(ρ)\rho\mapsto\Delta(\rho) is a continuous strictly increasing bijection from [0,1][0,1] onto [0,H(p⃗)][0,H(\vec{p})]. Every δ\delta in range is therefore hit by exactly one ρ∗(δ;p⃗)\rho^*(\delta;\vec{p}). A prefix-dependent ρt\rho_t keeps the token law by Lemma 2, applied given the prefix, and the scores do not involve ρt\rho_t, so the detector is unchanged.

Near-optimality of the copula race

Setup. Fix a next-token distribution p⃗\vec{p} and a detection score ss applied to the emitted token’s keyed Gaussian, with mean μ0\mu_0 and standard deviation σ0\sigma_0 under the null (unwatermarked sampling). A coupling π(x∣g)\pi(x\mid g) between the keyed Gaussians gg and the emitted token xx is assessed by its per-token signal-to-noise ratio SNR=(E[s(gX)]−μ0)/σ0\mathrm{SNR} = (\mathbb{E}[s(g_X)]-\mu_0)/\sigma_0 and its cost Δ=I(g;X)\Delta= I(g;X), the entropy removed ((12)).

An upper bound for every scheme.

Proposition 3 (No free evidence). For any distortion-free coupling of keyed Gaussians GG and token XX, let QQ be the law of the emitted token’s Gaussian GXG_X. Then KL(Q∥N(0,1))≤I(G;X)=Δ\mathrm{KL}(Q\mathbin{\|}\mathcal{N}(0,1)) \le I(G;X) = \Delta, and the inequality is strict whenever Δ>0\Delta> 0.

The left-hand side is the expected log-likelihood ratio per token, i.e. the rate at which detection evidence accumulates, so each nat of entropy removed yields less than one nat of evidence.

Proof. Under π\pi, the KL to the product of its marginals is the mutual information: KL(π∥PG⊗p⃗)=I(G;X)=Δ\mathrm{KL}(\pi\mathbin{\|}P_G\otimes\vec{p}) = I(G;X) = \Delta. The map (g,x)↦gx(g,x)\mapsto g_x reads off the emitted token’s Gaussian. It sends PG⊗p⃗P_G\otimes\vec{p} to N(0,1)\mathcal{N}(0,1), since every coordinate of GG is standard normal, and π\pi to QQ. Data processing gives the bound.

For strictness, suppose equality holds with Δ>0\Delta> 0. Equality in data processing requires the likelihood ratio to be a function of the statistic: π(x∣g)/px=fx(gx)\pi(x\mid g)/p_x = f_x(g_x) for some fxf_x. Summing over xx gives ∑xpxfx(gx)=1\sum_x p_x f_x(g_x)=1 for all gg. This is a sum of functions of separate independent coordinates, so each summand is constant: the variance of the sum is the sum of the variances, and it is zero. Hence each fxf_x is constant, and Eg π(x∣g)=px\mathbb{E}_g\,\pi(x\mid g)=p_x makes it 1. Then π(x∣g)=px\pi(x\mid g)=p_x and Δ=0\Delta=0.

The best coupling. Maximising E[s(gX)]\mathbb{E}[s(gX)] subject to X∼p⃗X\sim\vec p and I(g;X)≤ΔI(g;X)\le\Delta has the Lagrangian Eπ[s]−λ KL⁡(π∥Pg⊗p⃗)\mathbb{E}_{\pi}[s]-\lambda\,\operatorname{KL}(\pi\Vert P_g\otimes\vec p), maximised by

π(x∣g)∝axes(gx)/λ,a set so that X∼p⃗,\pi(x\mid g)\propto a_x e^{s(g_x)/\lambda},\qquad a\text{ set so that }X\sim\vec p,

an exponential tilt, i.e. entropic optimal transport, which textGrain approximates per token.

Weak-watermark limit. Write π(x∣g)=px(1+ϵψx(g))\pi(x\mid g)=p_x(1+\epsilon\psi_x(g)) with Egψx=0\mathbb{E}_g\psi_x=0 and ∑xpxψx=0\sum_xp_x\psi_x=0. The first constraint keeps the token marginal at p⃗\vec p; the second keeps π(⋅∣g)\pi(\cdot\mid g) normalized. To second order, the cost is Δ≈12ϵ2∑xpxEψx2\Delta\approx\frac{1}{2}\epsilon^2\sum_xp_x\mathbb{E}\psi_x^2. The signal is ϵ∑xpxE[s(gx)ψx]\epsilon\sum_xp_x\mathbb{E}[s(g_x)\psi_x]. The best perturbation is ψx=sˉ(gx)−∑vpvsˉ(gv)\psi_x=\bar{s}(g_x)-\sum_vp_v\bar{s}(g_v), with sˉ=s−μ0\bar{s}=s-\mu_0, and it gives, for every score,

SNR⁡≤2Δ(1−∑vpv2).\operatorname{SNR}\le\sqrt{2\Delta\left(1-\sum_vp_v^2\right)}.

Copula efficiency. For the copula, Δ≈12J(p⃗)ρ2\Delta\approx\frac{1}{2}J(\vec p)\rho^2 with J(p⃗)=∑vpve(1/pv)2/(1−pv)J(\vec p)=\sum_vp_v e(1/p_v)^2/(1-p_v), where e(n)e(n) is the mean maximum of nn standard normals, extended to real nn as the mean of the law with CDF Φn\Phi^n. Its SNR is ρ Cov⁡(s(g),g)∑vpve(1/pv)/σ0\rho\,\operatorname{Cov}(s(g),g)\sum_vp_ve(1/p_v)/\sigma_0. Its efficiency, the ratio of its SNR to the bound, is

η(p⃗)=corr⁡(s(g),g)⋅∑vpve(1/pv)J(p⃗)(1−∑vpv2)≤1\eta(\vec p)=\operatorname{corr}(s(g),g)\cdot\frac{\sum_vp_ve(1/p_v)}{\sqrt{J(\vec p)(1-\sum_vp_v^2)}}\le1

by Cauchy–Schwarz, with equality iff ss is increasing and affine in gg and p⃗\vec p is uniform on its support. Extending e(n)e(n) to non-integer nn is accurate: an exact computation from the score identity ∂P(X=x)/∂μv=E[1{X=x}ηv]\partial P(X=x)/\partial\mu_v=\mathbb{E}[\mathbb{1}\{X=x\}\eta_v] agrees with η(p⃗)\eta(\vec p) to within 0.001 on uniform, Zipf, and peaked distributions.

Numbers. For 8 equiprobable tokens, the copula achieves KL⁡/Δ=0.85\operatorname{KL}/\Delta=0.85, 0.79 and 0.71 at ρ=0.5\rho=0.5, 0.8 and 0.95 respectively, against a weak-limit ceiling factor of 1−∑vpv2=0.881-\sum_vp_v^2=0.88. Over 5,169 positions of 60 Qwen3.5-9B answers with at least two candidates (after top-pp), the weak-limit efficiency η\eta with the Gaussian score averages 0.966 (0.971 when weighted by each position’s capacity 1−∑vpv21-\sum_vp_v^2), with a 10th percentile of 0.94. Rescoring generated Qwen3.5-9B answers under the copula and TextSeal2 with the Gaussian score instead of TextSeal’s score changes detection by at most 2 percentage points. At finite strength, Table 2 compares the copula with the optimal coupling at equal Δ\Delta (computed by Sinkhorn on 4000 sampled keys, a discretisation that favours the optimum): the copula attains 91–100% of the optimal signal-to-noise ratio at ρ=0.5\rho=0.5, 0.8 and 0.95.

Next-token distributionGaussian scoreTextSeal score
uniform over 8 tokens0.998 / 0.998 / 1.0000.915 / 0.962 / 0.992
Zipf(1.1) over 24 tokens0.982 / 0.988 / 0.9930.908 / 0.960 / 0.989
top token 0.5, Zipf tail0.966 / 0.978 / 0.9870.911 / 0.959 / 0.990
top token 0.8, Zipf tail0.929 / 0.938 / 0.9420.908 / 0.952 / 0.973

Table 2. Copula race vs the best distortion-free coupling at equal entropy removed: ratio of per-token signal-to-noise ratios, for the Gaussian score s=gs=g and TextSeal’s score s=−(1−u)s=-(1-u), at couplings ρ=0.5/0.8/0.95\rho=0.5/0.8/0.95 (the fraction of each token’s entropy removed by the key is then about 0.16 / 0.45 / 0.72).

Entropy removed by a constant coupling

Table 3 reports the fraction of entropy removed, Δ(p⃗;p⃗)/H(p⃗)\Delta(\vec p;\vec p)/H(\vec p), computed by quadrature over the keyed Gaussians (640 sampled keys per value; exact for two tokens), for synthetic next-token distributions: uniform over nn tokens, a flat Zipf(0.8) law over 100 tokens, and a top token of probability pmax⁡p_{\max} with a Zipf(1.1) tail over 99 further tokens. The fraction depends only weakly on the shape: a fixed ρ\rho removes 0.47–0.49 of the entropy at ρ=0.8\rho= 0.8 and 0.63–0.65 at ρ=0.9\rho= 0.9 on the many-token shapes, and about 0.1 less in the extreme case of two equiprobable tokens. A single ρ\rho therefore removes a nearly shape-independent fraction of the entropy, as subsection 3.3 uses.

ShapeHH (nats)ρ=0.5\rho= 0.50.70.70.80.80.90.90.950.95
uniform over 20.690.1250.2720.3850.5500.677
uniform over 82.080.1470.3080.4260.5900.712
uniform over 644.160.1730.3530.4770.6400.754
flat Zipf(0.8)4.070.1730.3540.4790.6410.750
pmax⁡=0.5p_{\max}=0.52.420.1700.3500.4750.6380.750
pmax⁡=0.95p_{\max}=0.950.370.1830.3700.4930.6520.757

Table 3. Fraction of each token’s entropy removed at a fixed coupling ρ\rho, on synthetic distributions.

These fractions are characterised on synthetic shapes; on real Qwen3.5-9B distributions we instead measure the copula’s efficiency directly (Appendix § 7), which is the quantity that enters detection.

Algorithm and latency

One round in pseudocode

Algorithm 1 implements one round of speculative decoding with the residual race (subsection 3.2); its last line gives decoding without speculation.

Algorithm 1 One round of TextSeal2 with speculative decoding (NN drafts)

Require: prefix xx, keys k(1)k^{(1)}, k(2)k^{(2)}, coupling ρ\rho, drafter and target, number of drafts NN, set W\mathcal{W} of windows raced so far

  1. for i=1,…,N+1i = 1,\ldots,N+1 do

  2. wi←w_i \leftarrow last kk tokens of (x,d<i)(x,d_{<i})

  3. if wi∈Ww_i \in\mathcal{W} then

  4. R~v←\widetilde{R}_v \leftarrow freshly drawn U(0,1)\mathrm{U}(0,1) values ▷ window already raced

  5. else

  6. Gv⋆←(Φ−1(Rv(1))+Φ−1(Rv(2)))/2G^\star_v \leftarrow\left(\Phi^{-1}(R^{(1)}_v)+\Phi^{-1}(R^{(2)}_v)\right)/\sqrt{2} at wiw_i; R~v←Φ(ρGv⋆+1−ρ2ηv)\widetilde{R}_v \leftarrow\Phi(\rho G^\star_v+\sqrt{1-\rho^2}\eta_v) ▷ key-only

  7. end if

  8. Ei,v←−ln⁡R~vE_{i,v}\leftarrow-\ln\widetilde{R}_v; W←W∪{wi}\mathcal{W}\leftarrow\mathcal{W}\cup\{w_i\}

  9. if i≤Ni\leq N then

  10. q⃗i←\vec{q}_i\leftarrow drafter’s law after (x,d<i)(x,d_{<i})

  11. di←arg⁡min⁡vEi,v/qi,vd_i\leftarrow\arg\min_v E_{i,v}/q_{i,v}; τi←Ei,di/qi,di\tau_i\leftarrow E_{i,d_i}/q_{i,d_i} ▷ draft race ((5))

  12. end if

  13. end for

  14. p⃗1,…,p⃗N+1←\vec{p}_1,\ldots,\vec{p}_{N+1}\leftarrow target’s laws after (x,d<i)(x,d_{<i}), in one forward pass

  15. πi,v←P(I=i)ri,v\pi_{i,v}\leftarrow\mathbb{P}(I=i)r_{i,v} from ((11)); πi,di←0\pi_{i,d_i}\leftarrow0 for i≤Ni\leq N ▷ block verification

  16. (I,y)←arg⁡min⁡(i,v):πi,v>0(Ei,v−τiqi,v)/πi,v(I,y)\leftarrow\arg\min_{(i,v):\pi_{i,v}>0}(E_{i,v}-\tau_iq_{i,v})/\pi_{i,v}, with τN+1qN+1,v=0\tau_{N+1}q_{N+1,v}=0 ▷ residual race ((6))

  17. return d<Iyd_{<I}y

Without speculative decoding: return arg⁡min⁡vEv/pv\arg\min_v E_v/p_v at the window of the next position.

In Algorithm 1, the masses πi,di\pi_{i,d_i} vanish in exact arithmetic (Lemma 3) but are set to 0 explicitly: the leftover of a draft’s own lane is 0, so a rounding residue in its mass would make it win the race. The key-only work of position ii needs the draft di−1d_{i-1}, so it runs alongside the drafter’s forward pass; after the target pass, block verification’s law and the relay are linear in the round’s (N+1)∣V∣(N+1)|\mathcal{V}| values.

Post-logits latency

Only work performed after the logits are known can add latency; key-only work (PRF evaluation, forming G⋆G^{\star} and η\eta, and textGrain’s keyed cost table) depends only on the key and the window, and can therefore run concurrently with the model’s forward pass. Table 4 accordingly times the post-logits work of each sampler on the Qwen3.5 vocabulary, with every method except our full textGrain implementation captured in a CUDA graph. TextSeal2 costs the same as TextSeal: a single pass over the vocabulary, cheaper than the top-pp filtering that every method (watermarked or not) requires. For textGrain we time three variants: a minimal version with only the core post-logits work (block masses, a fixed number of Sinkhorn iterations, keyed column selection, and two sampling draws), without calibrating the regularisation to the entropy budget; our implementation of its Algorithm A.1 with 10–30 Sinkhorn iterations; and the same implementation with the regularisation calibrated per token so that the budget is met. Any faithful implementation does at least the minimal version’s work. At batch size 1, our implementation of Algorithm A.1 takes 1.1–3.0 ms per step (2–6% of a 50 ms Qwen3.5-9B decoding step, Table 5), and 2.3–4.1 ms with calibration. Even the minimal version (0.2–0.4 ms at batch 1) does 8–22 times TextSeal2’s post-logits work (Table 4).

BatchTop-p (all)TextSealTextSeal2textGrain minimalA.1, 10 / 20 / 30 it.A.1 calibrated, β=0.25/0.5\beta= 0.25/0.5
10.160.0200.0180.22–0.401.1 / 2.1 / 3.02.3 / 4.1
80.720.0300.0250.28–0.501.4 / 2.6 / 3.82.4 / 4.5
642.140.150.120.98–1.243.8 / 6.7 / 9.52.7 / 4.8

Table 4. Table 4 Post-logits latency per decoding step (ms, median of 200 CUDA-graph replays, or 50 eager runs for calibrated textGrain, whose calibration loop has data-dependent control flow; one H200, vocabulary of 248,320 tokens, synthetic high-entropy logits). Top-pp filtering is shared by all methods. The minimal textGrain version uses 5–20 Sinkhorn iterations on the 256 × 256 kernel without calibrating λ\lambda.

Comparison with TextSeal. TextSeal2 needs both keys’ PRF values and their Gaussian transforms at every token, whereas TextSeal needs a single key’s PRF values. Table 5 times full Qwen3.5-9B decoding steps (Hugging Face forward pass, one H200, real logits). If the extra key-only work is run sequentially after the forward pass, it costs 0.2 ms at batch size 1 and 2.9 ms (5%) at batch size 64; overlapped on a side CUDA stream during the forward pass, TextSeal2 is within 0.3 ms (0.6%) of TextSeal and within 0.9% of plain unwatermarked sampling.

PRF resolution

TextSeal’s PRF returns (h mod 8191)/8191(h \bmod8191)/8191, i.e. values quantised to 13 bits. The largest attainable value is then 8190/8191, so Ev=−ln⁡RvE_v = -\ln R_v is bounded away from zero. A rare token can win the race only if its EvE_v is extremely small relative to pvp_v, and is therefore emitted slightly too rarely: the quantisation measurably distorts the entropy. Simulating the race at all 8059 positions of 60 Qwen3.5-9B answers, the mean of −ln⁡px−H(pˉ)-\ln p_x - H(\bar{p}) is −0.03±0.12-0.03 \pm0.12 millinats per token with continuous (unquantised) uniform values, against −1.00±0.14-1.00 \pm0.14 millinats with the 13-bit PRF. The bias concentrates at high-entropy positions: it is −0.3-0.3 millinats at positions with 2–10 candidates after top-pp (52% of positions), but −42-42 millinats at positions with 101–1000 candidates (0.9%). TextSeal2 therefore uses a 53-bit PRF. With the 53-bit PRF (splitmix64; uniformity checked by a Kolmogorov–Smirnov test), TextSeal2 under speculative decoding passes both distortion tests of Figure 11. The 13-bit PRF also induces a slight correlation between the two keys’ values (−0.011±0.001-0.011 \pm0.001 on random windows and tokens, and −0.019±0.002-0.019 \pm0.002 on unwatermarked answers, against −0.0006-0.0006 for splitmix64), so that G⋆G^\star has variance 0.980.98–0.990.99 rather than 1. With 13 bits, TextSeal2’s mean per-token score was 1.2–1.6% below the single-key copula’s on the two Qwen models at ρ=0.9\rho= 0.9; with the 53-bit PRF the two agree on all three models at ρ=0.8\rho= 0.8 and 0.9 (differences of per-answer mean scores from −0.006-0.006 to +0.011+0.011, every 95% interval containing zero), as they must, since the two samplers coincide in distribution.

Additional results

Four keys. With TextSeal’s (13-bit) PRF, merging four keys detects 63% / 91% of 100 / 200-token answers, against 60% / 92% for two keys: additional keys (e.g. one publishable key per verifier) cost the full detector no detection power, as subsection 3.1 predicts.

Temperature 0.7. At the lower entropies induced by temperature 0.7, TextSeal2 detects 22% / 56% of 100 / 200-token answers, against 9% / 37% for TextSeal at a similar Self-BLEU (26.3 vs. 25.4); textGrain detects 22% / 55% at a Self-BLEU of 27.3. (For this experiment, Self-BLEU uses 16 answers to each of 25 questions, and thresholds are calibrated on 6000 unwatermarked answers.)

Keys with different windows. The virtual key can merge keys seeded by windows of different lengths. Let FtF_t be the set of keys whose window at position tt has not been used before in the answer; then Gt⋆=∑j∈FtGj,v/∣Ft∣G^\star_t = \sum_{j\in F_t} G_{j,v}/\sqrt{|F_t|} is still standard normal, so the race remains distortion-free with any such mix. This enables edit-rate estimation without access to the original text: under token substitutions at rate ee, a window of length LL and the token it seeds survive intact with probability about (1−e)L+1(1-e)^{L+1} ([27]), so the mean excess score of the length-LL key decays geometrically in LL, at a rate that identifies ee; the least-squares slope of the log excess against L+1L+1 estimates ln⁡(1−e)\ln(1-e). This application needs merging. Under TextSeal’s coin-flip routing, each of MM keys generates only a 1/M1/M share of the tokens, and detection collapses as keys are added: a ladder of window lengths 1,…,101,\ldots,10 detects just 22% of 400-token Llama-3.1-8B-Instruct answers. Merged into G⋆G^\star, each key instead retains a ρ/M\rho/\sqrt{M} correlation with every token’s race, and the same ladder detects 98%; on Qwen3.5-9B, multi-window variants match two-key TextSeal2 at equal Self-BLEU on clean text (Table 8). Pooled over a corpus, the resulting estimate of ee is unbiased up to 20% token substitutions; for a single 200-token answer it remains informative up to about 5%. The cost is robustness: long windows are broken by edits first, and under 8–12% substitutions the ladder’s detection falls by 15–27 percentage points relative to fixed short windows.

WindowsCleane=0.05e = 0.05e=0.12e = 0.12e=0.2e = 0.2Estimate at e=0.05e = 0.05Estimate at e=0.12e = 0.12
(3, 3)68 / 92835827––
(2, 10)70 / 947431100.045 / 0.0430.116 / 0.087
ladder67 / 94753390.045 / 0.0370.117 / 0.059

Table 8. Keys with different window lengths merged in the virtual key (Qwen3.5-9B, ELI5, 200 tokens; ρ=0.9\rho= 0.9 for the robustness and estimation columns; 53-bit PRF). Rows: standard two-key TextSeal2 with windows (3, 3); two windows (2, 10); and a six-key ladder (2, 3, 4, 6, 8, 10). Clean: % of 100 / 200-token answers detected at a false-positive rate of 10−310^{-3}, interpolated at Self-BLEU 21.5. Substitution columns: % of 200-token answers still detected, at the clean thresholds, after a fraction ee of tokens is replaced uniformly at random. Estimate columns: edit rate fitted from the decay of per-key scores with window length, reported as (pooled fit over 1000 answers) / (median of per-answer fits).

Repeated windows in long answers. In long-form prose, repeated windows are rare: on 1024-token Qwen3.5-9B answers (prompts requesting at least 800 words; 300 answers per scheme; thresholds calibrated on 2000 unwatermarked long answers), only 2.3% of positions are a window’s second occurrence. Every scheme (TextSeal, TextSeal2 with one use per window, and AutoMark’s copula) then detects 98–100% of answers within 256 tokens and 100% from 512 tokens (at a 1% false-positive rate),

Experimental details

so the repeated-window rule does not matter there. Windows repeat far more often in code (11% of positions, Table 12) and in thinking traces (16–34%, Table 13), where leftover continuation helps.

Experimental details

Generation. Prompts are ELI5 questions (a version without math questions, 267,901 rows), taken in order and formatted with each model’s chat template with thinking disabled; decoding uses temperature 1, top-pp 0.95, and at most 200 new tokens, with watermark window size k=3k = 3. The copulas (TextSeal2 and AutoMark’s) use the 53-bit PRF of Appendix § 10 in Table 1, Figure 1, Figure 7, Figure 6 and Figure 10, and Table 6, Table 12 and Table 13, as well as in every speculative-decoding experiment on Qwen3.5-9B; the remaining experiments (robustness, four keys, and the Llama drafter–target pair of Table 10) use TextSeal’s 13-bit PRF, which slightly understates TextSeal2 (Appendix § 10). Detection runs use 3000 answers per configuration on Qwen3.5-9B (rows 0–2999, one answer each), with or without speculative decoding, and 1000 on the other models (rows 0–999); null calibration uses 10,000 unwatermarked answers from the same model (rows 0–9999; rows 100,000–109,999 for the speculative-decoding experiments, with the target model). Key values follow TextSeal’s derivation: k(1)=42k^{(1)} = 42 and k(2)=42+12345k^{(2)} = 42 + 12345; textGrain uses B=m=256B = m = 256 and keys 42 and 4242. For null calibration, adding 105j10^{5}j to every key, j=0,…,9j = 0, \ldots, 9, gives ten keys, j=0j = 0 being the generation key.

Figure 6. The diversity–detection trade-off of Figure 1 at 200 tokens instead of 100; same protocol and conventions, including the speculative panel (b).

Figure 10. Figure 1 with diversity measured by the share of distinct 4-grams (higher is more diverse), including the speculative panel (b).

Speculative decoding. In each round, Qwen3.5-0.8B proposes NN drafts, one at a time, from its own distribution (after temperature and top-pp) through the scheme’s sampler; Qwen3.5-9B then evaluates p⃗1,…,p⃗N+1\vec{p}_1, \ldots, \vec{p}_{N+1} in a single forward pass, and the round emits the kept drafts and one more token, by block verification [30] with the closed forms of Appendix § 6.§ 3. A draft that is an end-of-sequence token ends the round’s drafting, so the round then has fewer than NN drafts. TextSeal2 follows Algorithm 1: drafts by the race (5), then the residual race (6), with the copula noise η\eta drawn once per position and read by both races. The unwatermarked reference is native block verification with freshly drawn randomness. N=8N = 8 is the main setting; Table 9 reports N=4N = 4 and 16, for which every configuration was generated again. During generation, each answer tracks the (window, key) pairs whose values have been used, and every pair that a round reads counts as used, kept or not, for every scheme (Appendix § 6.§ 5). When a window has already been used, TextSeal and SynthID-Text fall back to their other key and then to fresh randomness, and textGrain and the copulas directly to fresh randomness (for TextSeal2, freshly drawn lane lengths, on which the draft race and the residual race still run). Both models keep a key–value cache that is rolled back to the kept prefix after each round; on 40 random rounds, the logits after a rollback match a recomputation without cache as closely as those of a cache without rollback (bfloat16 noise).

Detection. Under speculative decoding with 8 drafts, the last token of each round makes up 36–37% of the text (first 1000 answers at α=0.5\alpha= 0.5 and ρ=0.9\rho= 0.9). Scored under the draft key, AutoMark’s last tokens, drawn under an independent key except at the bonus position (5% of rounds), average 0.84 (below the null mean of 1, because a token drawn from block verification’s residual law lost its draft race) against 1.42 for its kept drafts; TextSeal2’s last tokens average 1.34 under s∗s^*, against 1.42 for its kept drafts (all ±0.004\pm0.004). Positions whose window was already used get fresh values: 9% of TextSeal2’s tokens, against 6% of AutoMark’s. For every detector, each (window, token) pair is scored once, at its first occurrence. Entropy-weighted test. Each scored position tt is assigned the entropy ete_t of the next-token distribution of an auxiliary model (here the generating model itself, the target model under speculative decoding, evaluated on the answer under examination) and the weight wt=0.1+0.9min⁡(1,max⁡(0,(et−emin⁡)/(emax⁡−emin⁡)))w_t = 0.1 + 0.9 \min(1,\max(0,(e_t-e_{\min})/(e_{\max}-e_{\min}))), where emin⁡e_{\min} and emax⁡e_{\max} range over the scored positions of the text. With per-position scores sts_t of null mean 11 and null variance θ\theta (θ=1\theta= 1 for single-key scores, θ=a2+(1−a)2\theta= a^2 + (1-a)^2 for two-key fused scores as(1)+(1−a)s(2)a s^{(1)} + (1-a) s^{(2)}), the weighted sum ∑twtst\sum_t w_t s_t is compared with the Gamma distribution of the same mean μ=∑twt\mu= \sum_t w_t and variance σ2=θ∑twt2\sigma^2 = \theta\sum_t w_t^2 (shape μ2/σ2\mu^2/\sigma^2, scale σ2/μ\sigma^2/\mu). Since all decision thresholds are empirical (below), this moment matching affects power but not validity. The threshold for each detector and answer length is the empirical 10−310^{-3} quantile of its pp-values on the 10,000 unwatermarked answers scored under 10 keys. Keys are interchangeable on unwatermarked text, so this pools 10510^5 null pp-values, about 100 of which fall below the threshold; a single key would give only about 10, and hence a much noisier threshold. At the nominal 10−310^{-3} level, realised false-positive rates are 0.08–0.13% for the detectors of Figure 9 and 0.07–0.15% for those used under speculative decoding (two-key fusions with the measured shares, and the Llama pair), so the Gamma and binomial tail approximations are already accurate, and the empirical thresholds absorb the remaining deviation. Under speculative decoding, TextSeal, textGrain, SynthID-Text and AutoMark’s copula fuse both keys’ scores, weighted by their measured shares of emitted tokens (as TextSeal does for its coin flip); textGrain’s own recipe uses a single key; TextSeal2 uses s∗s^* (private) and s(1)s^{(1)} (public) exactly as without speculative decoding (subsection 3.1).

Diversity. The Self-BLEU of a question is the mean sentence-level BLEU (sacrebleu) over all ordered pairs of its answers generated under the fixed key, decoded from their first 200 tokens. This is TextSeal’s two-generation diversity estimator averaged over all answer pairs; we report the mean over questions. Intervals for values interpolated at TextSeal’s Self-BLEU resample the 100 questions jointly for all methods (and detection outcomes binomially within questions), so that question-level variation common to all methods cancels in paired comparisons. Figure 10 repeats Figure 1 with diversity measured by the share of distinct 4-grams among a question’s answers. The copulas still dominate the coin flip, and TextSeal2 still leads under speculative decoding; without speculative decoding, textGrain is slightly ahead of the copulas under this metric (66% vs. 62% detected at TextSeal’s distinct-4 level), whereas it is level with TextSeal2 under Self-BLEU.

Distortion tests. Answers used for acceptance and distortion measurements are generated with a fresh key 107+7919i10^7 + 7919i for prompt ii (textGrain: this key and the key plus 4200), so that answers are independent draws over keys, the regime in which distortion-freeness is defined; 480 prompts per scheme and draft length. The tests read the laws used at generation: for every emitted token, pxp_x and H(p⃗)H(\vec{p}) come from the target pass of its round, and for the last token of each round, r⃗I\vec{r}_I is block verification’s residual law at its position, computed from these p⃗\vec{p} and q⃗\vec{q}. All intervals are 95% bootstrap intervals over answers. Figure 11 reports the acceptance and distortion measurements for N=8N = 8, and Table 9 summarises them for N=4N = 4 and 16. As a simulation control, the decisions were run on fixed toy laws with fresh values for every round (N=3N = 3 drafts). The outcome (d,I,y)(d, I, y) of the residual race follows block verification’s law (χ2\chi^2 test over 6⋅1046 \cdot10^4 rounds and 5 tokens, p=0.15p = 0.15), whereas racing on the raw lane lengths (Remark 1) does not (χ2=3194\chi^2 = 3194 on 401 degrees of freedom). For the baselines (6⋅1036 \cdot10^3 rounds over 12 tokens), the first emitted token follows the target’s law and the number of kept drafts follows native block verification (p≥0.13p \ge0.13), except that textGrain’s own recipe keeps fewer drafts, as it verifies the watermarked laws.

Compute. Each generation job uses one H200 GPU. Our speculative-decoding implementation (Hugging Face models with a key–value cache) is unbatched and unoptimised: with N=8N = 8 it takes about 0.11 s per token on one GPU, three times slower than plain decoding with the target, because the drafter is called once per draft; this affects only experimental throughput, not any reported rate, since block verification is exact. Under it, the residual race’s decision takes 0.33 ms per token, against 0.38 ms for native block verification, and computing the keyed lane lengths of the drafted positions takes 2.4 ms per token more than sampling the drafts without a key; this key-only work can run concurrently with the drafter’s forward pass (Appendix § 9).

textGrain reimplementation. The textGrain report Li et al., [24] releases no code, so we implemented its Algorithms 1 and A.1 from the text: a keyed partition of the vocabulary into B=256B = 256 blocks derived from the key and the window; a 256×256256 \times256 table of standardised keyed Gumbel costs with m=256m = 256 columns (the report fixes neither BB nor mm); the log-domain Sinkhorn updates with marginal correction and the regularisation update of its Eq. (A.4); selection of a keyed column; and independent sampling of the block and then of the token within it. Detection follows its Section 3.2 (Gamma(nn, 1) null), except that, as for every method in this paper, we score the first occurrence of each (window, token) pair rather than each window’s first occurrence, which adds about 1% more scored positions. We validated the GPU kernels against a direct (unoptimised) implementation of the algorithm on random inputs, and verified on toy distributions that the emitted-token distribution is unbiased and that the entropy removed respects the budget β\beta. The speculative recipe implements its Appendix B with block verification between the watermarked distributions. OpenAI’s implementation and hyperparameters may differ from ours, and our textGrain numbers carry that caveat.

Reproducibility. Generation and evaluation scripts, including fixed seeds, key derivations, prompt lists (ELI5 rows; MBPP problems) and threshold calibration, together with the per-answer outputs underlying every reported number, will be released with the paper.

Broader impact. Watermarks give providers, platforms and the public one signal for tracing AI-generated text. They are not robust to paraphrasing (Table 7), can in principle be spoofed or removed by determined adversaries, and should therefore be combined with complementary provenance signals rather than treated as proof of origin. Publishing a detection key enables independent public verification, but also gives adversaries a target signal to attack; retaining a private key allows such manipulation to be detected Milis et al., [27].

References

  1. [1]Scott Aaronson and Hendrik Kirchner. Watermarking GPT outputs, 2023.
  2. [2]Anthropic. How Claude marks AI-generated content, 2026. Accessed: 2026-08-13.
  3. [3]Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. In arXiv preprint arXiv:2302.01318, 2023.arxiv.org/abs/2302.01318
  4. [4]Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. Cryptology ePrint Archive, 2023.arxiv.org/abs/2306.09194
  5. [5]Evan Crothers, Nathalie Japkowicz, and Herna Viktor. Machine generated text: A comprehensive survey of threat models and detection methods. arXiv preprint arXiv:2210.07321, 2022.arxiv.org/abs/2210.07321
  6. [6]Majid Daliri, Christopher Musco, and Ananda Theertha Suresh. Coupling without communication and drafter-invariant speculative decoding, 2024.arxiv.org/abs/2408.07978
  7. [7]Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanfordth, Tatiana Matejovicova, et al. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818–823, 2024.DOI
  8. [8]European Union. Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (AI Act), 2024.
  9. [9]Xiaoyan Feng, Yanjun Zhang, He Zhang, Leo Yu Zhang, and Shirui Pan. Tracing provenance and detecting tampering with complementary llm watermarks. arXiv preprint arXiv:2608.12713, 2026.arxiv.org/abs/2608.12713
  10. [10]Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chapellier, and Teddy Furon. Three bricks to consolidate watermarks for large language models. 2023 IEEE International Workshop on Information Forensics and Security (WIFS), 2023.arxiv.org/abs/2308.00113
  11. [11]Pierre Fernandez, Tom Sander, Hady Elsahar, Hongyan Chang, Tomáš Souček, Valeriu Lacatusu, Tuan Tran, Sylvestre-Alvise Rebuffi, and Alexandre Mourachko. How good is post-hoc watermarking with language model rephrasing? arXiv preprint arXiv:2512.16904, 2025.arxiv.org/abs/2512.16904
  12. [12]Thibaud Gloaguen, Robin Staab, and Martin Vechev. AutoMark: Enabling autoresearch to discover better LLM watermarks, 2026.
  13. [13]Haiyun He, Yepeng Liu, Ziqiao Wang, Yongyi Mao, and Yuheng Bu. Theoretically grounded framework for LLM watermarking: A distribution-adaptive approach. In Advances in Neural Information Processing Systems, pages 114528–114564, 2025.arxiv.org/abs/2410.02890
  14. [14]Weiqing He, Xiang Li, Li Shen, Weijie Su, and Qi Long. Improving the trade-off between watermark strength and speculative sampling efficiency for language models. In ICLR, 2026. arXiv:2602.01428.arxiv.org/abs/2602.01428
  15. [15]Zhengmian Hu and Heng Huang. Inevitable trade-off between watermark strength and speculative sampling efficiency for language models, 2024.arxiv.org/abs/2410.20418
  16. [16]Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. Unbiased watermark for large language models. arXiv preprint arXiv:2310.10669, 2023.arxiv.org/abs/2310.10669
  17. [17]John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. arXiv preprint arXiv:2301.10226, 2023.arxiv.org/abs/2301.10226
  18. [18]Szymon Kobus and Deniz Gündüz. Speculative sampling via exponential races, 2025.arxiv.org/abs/2504.15475
  19. [19]Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. arXiv preprint arXiv:2307.15593, 2023.arxiv.org/abs/2307.15593
  20. [20]Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In ICML, 2023.arxiv.org/abs/2211.17192
  21. [21]Baixin Li and Haiyun He. SAC-Copula: Quality-preserving watermarking for diffusion language models via smooth correlated Gumbel fields, 2026.arxiv.org/abs/2608.20839
  22. [22]Cheuk Ting Li and Venkat Anantharam. A unified framework for one-shot achievability via the Poisson matching lemma, 2018.
  23. [23]Xiang Li, Feng Ruan, Huiyuan Wang, Qi Long, and Weijie J. Su. A statistical framework of watermarks for large language models: Pivot, detection efficiency and optimal rules. The Annals of Statistics, 53(1): 322–351, 2025.DOI
  24. [24]Xiang Li, Garrett Wen, Xiaohong Chen, Qi Long, Arzav Jain, Florent Joly, Mike Lam, Qingquan Song, and Weijie Su. textGrain: Entropy-calibrated watermarking for language model text. Technical report accompanying OpenAI’s October 5th blog post, OpenAI, 2026.
  25. [25]Yanxiao Liu, Sicheng Wan, Zhan Gao, and Deniz Gündüz. Watermarkable multi-draft speculative sampling via Poisson processes, 2026.arxiv.org/abs/2609.21858
  26. [26]Carol Xuan Long, Dor Tsur, Claudio Mayrink Verdun, Hsiang Hsu, Haim Permuter, and Flavio P. Calmon. Optimized couplings for watermarking large language models. In IEEE International Symposium on Information Theory (ISIT), pages 1–6, 2025.arxiv.org/abs/2505.08878
  27. [27]Georgios Milis, Tom Sander, Tomáš Souček, Heng Huang, and Pierre Fernandez. Could LLM watermark detection be public? arXiv preprint arXiv:2610.12106, 2026.arxiv.org/abs/2610.12106
  28. [28]Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. Watermarking makes language models radioactive. NeurIPS, 2024.arxiv.org/abs/2402.14904
  29. [29]Tom Sander, Hongyan Chang, Sylvestre-Alvise Rebuffi, Tomáš Souček, Tuan Tran, Valeriu Lacatusu, Alexandre Mourachko, Surya Parimi, Christophe Ropers, Rashel Moritz, Vanessa Stark, Hady Elsahar, and Pierre Fernandez. TextSeal: A localized LLM watermark for provenance & distillation protection, 2026.arxiv.org/abs/2605.12456
  30. [30]Ziteng Sun, Uri Mendlovic, Yaniv Levithan, Asaf Aharoni, Jae Hun Ro, Ahmad Beirami, and Ananda Theertha Suresh. Block verification accelerates speculative decoding. In International Conference on Learning Representations, 2025.arxiv.org/abs/2403.10444
  31. [31]Dor Tsur, Carol Long, Claudio Mayrink Verdun, Sajani Vithana, Hsiang Hsu, Richard Chen, Haim Permuter, and Flavio Calmon. HeavyWater and SimplexWater: Distortion-free LLM watermarks for low-entropy distributions. In Advances in Neural Information Processing Systems, 2025.DOI
  32. [32]Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. DiPmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710, 2023.
  33. [33]Yangxinyu Xie, Xiang Li, Tanwi Mallick, Weijie Su, and Ruixun Zhang. Debiasing watermarks for large language models via maximal coupling. Journal of the American Statistical Association, 120(551):1424–1436, 2025.arxiv.org/abs/2411.11203
  34. [34]Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems, 32, 2019.arxiv.org/abs/1905.12616

Paper details

vlnc:W_qw6qq4vw
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 9

downloads · 2

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents