WorksComputer Science
Reconciling Privacy with Provenance for Language Model Watermarks
Abstract
Watermarking has great potential for regulating the use of language models (LMs), but its deployment has been marred by controversy, as many users argue that fine-grained tracking of LM usage violates their privacy. This tension reduces the effectiveness of watermarks, as users engage in evasion strategies such as paraphrasing that can remove much of the watermark signal. We show that provenance and privacy can be reconciled if we shift the goal of watermark design from tracking individuals to estimating aggregate usage. We develop locally differentially private watermarks which provably limit the evidence that any individual text can contribute to detection, while still enabling accurate estimation of population frequencies. We introduce the Red-Green Sequence (RGS) watermark, a private and computationally efficient watermark that achieves accurate frequency estimation at high privacy levels across various models, sample sizes and generation entropies in settings that reflect realistic deployment scenarios. On C4 language modeling tasks, we attain MSE with mean absolute bias at a low of 1, while in more challenging settings such as an essay writing task with only 100 text samples, we still find errors on the order of MSE with mean absolute bias at .
Introduction
Watermarking has emerged as a promising approach to enable content provenance through injecting recoverable signals into each piece of model generated text ([23, 27]). Among provenance techniques, watermarking is unique in providing a false positive guarantee. If the detection key is kept secret, the false positive rate of identifying a piece of text as being LM generated can be precisely controlled via the generation mechanism ([48]).
While provable guarantees for tracking individual texts are powerful for the targeted monitoring and regulation of AI-generated content, they have also faced significant pushback from users in recent times ([42, 16]). A major deployment by Anthropic ([2]) has led to accusations of surveillance and concentration of power ([6, 7]), amidst increasing concerns that steganographic techniques may be used to spy upon users ([41, 5]).
The tension between the desire for accurate tracking by regulators and users’ desire for privacy has also led to adversarial dynamics ([35]), where users actively modify their text to evade watermark detection. Even simple strategies, like paraphrasing by another language model, can erase most of the signal from the watermarks ([26, 47]). This leads
Correspondence to: [email protected]. Code is available at github.com/elisel yhan/dp-watermarks. to a paradoxical situation where the precision and accuracy of a watermark lead to user behavior that sabotages watermark efficacy.
Therefore, we ask if we can achieve many of the provenance goals of watermarking without violating individual privacy. We answer the question in the affirmative: we observe that many of the regulatory and monitoring goals of watermarking only require population level tracking [33, 24, 40]. In these cases, it is possible to design privacy-preserving watermarks that enable accurate estimation of the prevalence while protecting individuals.
We develop the Red-Green Sequence (RGS) watermark, which is a locally differentially private watermark that provably limits the evidence that any individual text can contribute to detection [12, 13], while enabling efficient estimation of the prevalence of watermarked texts in a population of texts.
Our watermark builds upon standard red-green style language model watermarks [23], but carefully modifies them to be effective in the local differential privacy setting. We find that sequence-level watermarks, which are unappealing in the nonprivate setting, are highly efficient in injecting watermarks in private, low- settings.
To obtain good estimates of aggregate watermark frequencies from these watermarks, we identify several interesting statistical problems in the bias-variance trade-offs of moment estimators for privacy-preserving watermarking, where the natural variability of watermark entropy across texts leads to different sampling and estimation choices compared to standard, randomized-response style estimators.
We implement this watermark on a range of scenarios, finding our frequency estimates to be highly accurate across a range of settings. In a standard C4 language modeling task, we can attain a low MSE of 0.0019 with 0.0038 mean absolute bias at a low of 1, while in more challenging settings like an essay writing task with very few text samples, we still find errors on the order of 0.0113 MSE with 0.004 mean absolute bias at . These results suggest that privacy-preserving watermarking is possible and effective.
Related Work
Language model watermarks inject recoverable statistical signal into the generated text to enable detection of the presence of the watermark from the text. Prominent token-level watermarks work by reweighting token-level probabilities with a secret key [23, 28, 34] or coupling sampling with key randomness [27, 10] (Aaronson 2022; Li et al. 2026a). Among these, the closest to our work is Kirchenbauer et al. [23], who use the idea of slightly upweighting pseudorandomly chosen “green” tokens that can later be detected. Our approach similarly uses a colored biasing scheme, but our goals and techniques differ significantly from theirs in that we aim to precisely control the signal rather than maximize it.
Our watermark preserves privacy by making the key-conditional watermarked text distribution close to the one without the watermark. While distortion-free watermarks [27, 18, 46] achieve a similar goal, their watermarks only preserve the base model distribution in expectation over the sampling randomness in the secret key. Such equality in marginal distributions does not imply that watermarked and unwatermarked text distributions remain close conditional on a fixed key. In fact, the separation in conditional distributions is what enables high-power detection, which is at odds with privacy. Another related objective is cryptographic undetectability ((Christ et al., 2024; Christ and Gunn, 2024)), which ensures the marginal distributions of watermarked and unwatermarked texts are computationally indistinguishable to anyone without the secret key, but not conditional on the key. While Kim et al. [22] constructs watermarks under key-conditional -divergence bounds, which can cover differential privacy constraints, their goal is to recover multi-bit messages from individual texts rather than estimate population watermark prevalence. On the other hand, Li et al. (2026b) estimates the proportion of watermarked content within individual texts rather than across a population of texts, and does not provide privacy guarantees.
Our watermark guarantees privacy formally through local differential privacy (DP) ([12, 13]), and the watermark method is inspired by randomized response ([45]), which enables efficient estimation of the prevalence of sensitive attributes while protecting individual respondents, achieving local DP guarantees. The privacy–utility tradeoff under local DP has been studied by Duchi et al. [11]; Kairouz et al. [21], who establish information-theoretic optimality of randomized response in several settings. Related variants of randomized response have also been used for frequency estimation in a fixed population under local DP ([3, 44]). Our setting departs from randomized response and generic local DP estimation by incorporating two important constraints. First, our private mechanism must enable signal to be recoverable from the output text alone, without requiring a separate report from each individual. Second, while randomized response allows for modification of distributions for groups with and without the sensitive attribute, our watermark must modify only the text distributions of the watermarked group, as the unwatermarked group’s generation process is beyond watermark control.
Problem Setup
Watermarks for language models
The process of watermarking involves modifying the standard language model decoding process using a secret key , so that correlations between the choice of the key and text output can be used to detect whether a piece of text was generated by an LM decoder that uses .
More formally, let be a prompt, and be the base model’s distribution over sequences given . We sometimes suppress the dependence on . A watermark consists of a decoder , which is an algorithm that maps a secret key and distribution to a sequence stochastically, i.e. it specifies a new distribution over completions, usually different from the base model distribution , and samples a sequence from the new distribution. The decoder is designed to allow detectors to detect the presence of the watermark in a text sequence given the secret key .
For example, suppose consists of only single-token sequences. Then, the Soft Red List watermark of Kirchenbauer et al. [23] has the following decoder:
where is the set of green tokens obtained by partitioning the token space into green and red tokens via a hash function with key , and is a parameter that controls the strength of the watermark. The decoder biases the watermark towards outputting more green sequences than usual. Generalizing to longer sequences, the watermark applies the same bias towards green tokens at each token position, so a text with many green tokens is strong evidence for the watermark. Consequently, an LM user who is unwilling to disclose their LM usage might paraphrase the watermarked text before publishing, in the hope of reducing the number of green tokens and evading detection.
Importantly, watermarks are often designed to allow for high-powered detection from a single piece of text. In the case of a red-green list watermark, any moderately long piece of text will carry an exponentially increasing watermark signal, and many watermarking papers show extremely low p-values for high-entropy sequences [23, 27].
The core claim of our work is that we can instead consider watermarks that more precisely control the watermarking signal, where individual texts still contribute enough signal to accurately estimate the aggregate prevalence of watermarked texts in a population, but are too weak to reliably identify any individual. To illustrate how we construct this estimator, we first illustrate a similar tension in survey design and how it can be addressed.
From individual detection to aggregate estimation: randomized response
Consider the scenario of reporting a sensitive attribute in response to a survey question, such as “have you ever used illegal drugs?” It has long been known that survey respondents can often alter their answers to avoid the possibility that their true, sensitive status may later be revealed [43]. To address this, survey designers have developed techniques that incentivize respondents to answer truthfully while maintaining the integrity of the survey [29]. One such technique is randomized response (RR) [45], which allows respondents to reveal sensitive information to interviewers with plausible deniability.
Consider a fixed pool of respondents whose sensitive attributes are , with indicating that respondent has the sensitive attribute. Suppose that the true prevalence of the sensitive attribute in the pool is .
In a generalized randomized response (RR) mechanism, each respondent engages independently with a randomization device with , and outputs a noisy answer with these probabilities:
Standard symmetric RR sets . RR provides plausible deniability to the respondents: if they had reported a sensitive answer , they can still claim that it was due to randomization since there is probability that they are forced to report even if their true status is .
Such plausible deniability can be formalized by the notion of local differential privacy (DP) [12, 13]. Consider a randomized mechanism which maps a private bit to a discrete output . The mechanism is -locally differentially private [13] if
This condition bounds the likelihood ratio of observing any output under the two possible values of the sensitive bit, thus limiting the evidence that the output can reveal about the sensitive bit. The reporting mechanism in equation 2 satisfies local -DP, where the privacy parameter
While privacy is preserved by adding noise to each person’s true bit, aggregating the noisy bits still allows us to recover information about the mean of the true bits in the pool of respondents, . This is because randomization is precisely controlled by the probabilities . Writing , we observe that . Correspondingly, the moment estimator
is unbiased and consistent for [45].
Increasing widens the separation between the reporting probabilities with private bits and , which makes each noisy output more informative about the private bit, which increases estimation accuracy but reduces privacy. For binary attributes, it has been shown that the randomized response mechanism with the following reporting probabilities attains optimal information utility under -local DP [20]:
This choice maximizes the separation and minimizes the variance of the moment estimator in equation (4) among all binary -local DP mechanisms, which also minimizes the MSE since there is no bias.
Locally Differentially Private Watermarks
We have illustrated how randomized response can preserve individual privacy and produce efficient estimation of aggregate prevalence in the survey setting. Inspired by this, we design a watermark that is locally differentially private, and an efficient estimator for the prevalence of watermarked texts in a mixture of texts. First, we formalize the notion of local differential privacy for watermarks by directly adapting equation (3):
Definition 1. For some , a watermark decoder is -locally differentially private with respect to the base model distribution , if for any key and any :
Denote as the prior distribution of keys, and the posterior distribution of keys given observed text . Let be the shorthand for . Definition 1 then implies:
Applying the same argument in the other direction gives . Viewing the watermark as introducing an association of the text with the secret key (Li et al., 2025), this condition limits the strength of such association that can be inferred by an observer with the key, and thus provides a natural notion of privacy for arbitrary distributions of non-watermarked text.
The Red-Green-Sequence (RGS) Watermark
We now present a simple construction for a watermark that satisfies -local DP, which we call the Red-Green-Sequence (RGS). RGS closely follows the classic token-level red-green list construction of Kirchenbauer et al. [23], but involves a few subtle changes that arise from our desire to fulfill local differential privacy.
Algorithm
The biggest difference between RGS and Kirchenbauer et al. [23] is that RGS operates at the sequence level, i.e. RGS only applies watermark bias to complete sequences, rather than at every token. For Kirchenbauer et al. [23], who operate without privacy constraints, watermarking at the sequence level is computationally expensive and unnatural. Identifying the set of potential green and red sequences requires a large Monte Carlo sample and provides little benefit, as injecting small distortions at each token generation step already provides exponentially strong detection signal.
In contrast, for differentially private watermarking, we have the opposite problem. Token-level watermarks lead to hard-to-control privacy leakage, as the information that we leak is a sequence of red-green labels whose length equals the length of the sequence, which is a high-dimensional bit string that is difficult to keep private. This challenge is analogous to the observation in locally differentially private frequency estimation that the optimal estimator is equivalent to a one-bit randomized response-style estimator, as that is the minimal information that needs to be leaked to enable frequency estimation ([11, 20]). Because of this, sequence-level watermarking is far more natural in our setting, and we encode exactly one red-green bit for each sequence.
We begin with a naive implementation of this sequence-level approach: given a prompt, we first draw candidate completions in parallel from the underlying base model, partition them into green and red bins with a keyed hash function, and reweight them appropriately to select the final response. The full pseudocode is presented in Algorithm 1.
Algorithm 1 RGS generation
Require: Prompt , candidate budget , privacy budget , keyed hash with parameter
Draw .
Deduplicate to ; set .
; .
.
if then
Set for all .
else
For each , set
end if
Draw .
return .
Comparing Algorithm 1 to the earlier token-level red-green watermark in equation (1), we see that one major difference is in the definition of the reweighting term , which is not a fixed offset determined by , but a term that is set to precisely attain our target privacy budget on either the green or red list probability ratio. With this choice of reweighting, we attain our local DP guarantee.
Proposition 1 (RGS privacy). RGS is locally -DP with respect to the base model.
The proof follows directly from our choice of reweighting coefficients. We defer the proof to
Appendix B.1.2.
One major drawback of sequence-level watermarks is that the requirement of drawing parallel samples for one completion is highly wasteful (compared to token-level ones, which are essentially free). However, another difference between standard watermarking and our setting is that the watermarking strength per sequence is very low, and thus the difference in distribution before and after watermarking is very small. This enables highly efficient rejection sampling algorithms for our setting, which downweight all red completions by a multiplicative factor of . At , the empirical mean number of samples needed by the rejection sampler per watermarked response is around 1.5 on real data, see § 7.4.
Algorithm 2 RGS via rejection sampling (RGS-reject)
Require: Prompt , candidate budget , privacy budget , keyed hash with parameter
Initialize counter
while do
Sample a sequence completion independently
Increment the counter
if then
return .
else
With probability , return
end if
end while
return .
We show formally that our nominal privacy level and the expected number of samples needed by the rejection sampler are directly linked. The proof is given in Appendix § 11.1.3. The stronger the privacy, the fewer samples needed.
Fact 1 (RGS-reject expected sample count). The expected number of samples needed for the rejection sampler in Algorithm 2 is at most .
As this is a rejection sampler, there is a natural bound on the probability ratio between the original distribution and our sampler, giving a local DP guarantee. The proof of the following privacy guarantee is given in Appendix § 11.1.4.
Proposition 2 (RGS-reject privacy). The rejection sampler in Algorithm 2 is -DP.
We remark that the two samplers use different reweighting rules. The rejection sampler upweights the probability of green candidates relative to red ones by a fixed multiplicative factor of . The parallel sampler instead uses the empirical green fraction from the parallel samples to calculate the largest reweighted green probability allowed by the privacy constraints. To see the difference between the weighting rules and to remove the finite budget effect for later analyses, we first state convergence properties of the samplers.
Fact 2 (Finite-sample convergence). For any key , denote as the probability of sampling a green sequence from the underlying base model distribution. Let and denote the probabilities that the parallel and rejection samplers return a green sequence with candidate budget . Then, with ,
The proof is deferred to Appendix § 11.1.5. In the infinite budget limit, because , the parallel sampler can achieve a larger green probability, while the rejection sampler achieves much better computational efficiency by stopping early.
Connection to Randomized Response
We might view RGS as a type of randomized response (RR) for watermarking, since both methods privatize the true attribute via a randomized one-bit report. While broadly this is correct, there are two major conceptual differences between RR and our watermarking setting that lead to complexities. First, while RR can randomize for both the group with the sensitive attribute and the group without, RGS can only randomize for the watermarked group, since the unwatermarked group is not generated by the watermark decoder.
Additionally, RGS uses a keyed hash function where a fixed key always assigns the same color to the same sequence, so an analyst can recover the report from the observed text. The construction of the keyed hash function and associated assumptions are discussed in the next section and Appendix § 10. With the key fixed, the noise in the report now comes from the randomness in sampling the underlying LM. This comes with an important consequence: the extent to which the randomization can be controlled depends on the inherent properties of the underlying base model distribution. The higher the entropy, the better control we have over the randomization, and the more accurate the prevalence estimate can be. We discuss the implications on estimation and design choices in detail in the next section.
Estimation
Having established privacy, we turn to the statistical objective of estimating watermark prevalence. Let be a mixture of watermarked and unwatermarked texts, with indicating whether is watermarked. We define as the true prevalence of watermarked texts in the mixture. Finally, writing , we assume that are mutually independent conditional on and the shared key .
Denote the index sets of watermarked and unwatermarked texts as and . For each , define as the reference distribution for the text. For , may be any distribution over text sequences and need not come from a language model. For , is the underlying base model distribution without the watermark decoder . For example, if the text is generated by prompting the watermarked LM with a prompt , then is the model distribution . Throughout, and the distributions are fixed, and we suppress the conditioning on these quantities.
Moment Estimator
The similarity of RGS to randomized response naturally motivates the use of a moment estimator for prevalence estimation. To construct this estimator, we first derive the expected fraction of green texts with and without the watermark.
For each and any fixed key , denote the probability of sampling a green sequence under the unwatermarked distribution as
For watermarked texts , the probability of sampling a green sequence is upweighted, which we define as
Let and . Under the conditional independence assumption, the indicators conditional on and are independent Bernoulli random variables with success probabilities for and for respectively. Therefore,
Note that for a fixed , the probabilities and are generally not knowable without assumptions about the underlying distributions . To enable estimation, RGS uses a coloring scheme such that each has a known expectation over random keys and then uses as a plug-in for the unknown ’s.
Ideally, we would pick uniformly at random, and any sequence would be equally likely to be green or red. However, to enable later detection, we must fix and have a deterministic coloring as a function of this key. To balance these two competing goals, we implement coloring by thresholding the output of a keyed hash, modeled as a pseudorandom function (PRF) [15]. We can then sample the key once uniformly at random, and reuse it across all texts. The details of the construction can be found in Appendix § 10. As a consequence of this construction, we adopt the following assumption for our statistical analysis:
Assumption 1 (Ideal coloring). For every finite collection of distinct sequences fixed independently of the random key , the indicators
are independent Bernoulli random variables.
In the formal PRF definition, these are only computationally indistinguishable [15], but we omit the negligible distinguishing advantage from our notation.
Under this idealization, we can characterize both the key-averaged green probability and its variation across keys. The following lemma also accounts for dependence between and arising from the use of the same key. Its proof is given in Appendix § 11.2.1.
Lemma 1. Let and be fixed distributions over independent of the key , and write
Under Assumption 1,
Lemma 1 links the second moments of with the diversity of the distributions . We can see that is the probability that two independent draws from yield the same sequence, which is a measure of (the lack of) entropy of , while is the probability that independent draws from and produce the same sequence, which measures the (lack of) “diversity” of the underlying distributions across individuals.
The variance identity in the lemma suggests that if the underlying distributions have high entropy, and thus low , then the green probability would be closer to on average. In a non-atomic idealization with , the green probability converges in probability to the null frequency . Therefore, with in both the parallel and the rejection samplers, the corresponding watermarked green probability is
In the ideal case where for every , we have that is an unbiased estimator of conditional on and , exactly mirroring the moment estimator for randomized response. We use the clipped version of this estimator:
On the other hand, when the underlying distributions do not have perfect entropy, our have nonzero variance over the choice of , and there is no guarantee that the average unwatermarked green frequency will converge to , resulting in a biased estimator. We now characterize this estimator error that arises from finite watermark entropy and decompose it into bias and variance terms.
Proposition 3 (Key-conditional MSE). Fix any key . Fix , and suppose that are conditionally independent. Let , . Assume , so that both and are nonempty. Define
Then,
While the variance term vanishes as , the bias terms and are not guaranteed to vanish, since they depend on the fixed key and the underlying distributions of the watermarked and unwatermarked texts. We defer the proof to Appendix § 11.2.2.
Null bias and distributional diversity
The term measures the bias in the green sequence fraction marginalized across all the unwatermarked examples. This bias can be small even when every individual text distribution has low entropy, provided the distributions are sufficiently diverse across examples.
To see this, observe that Lemma 1 implies
In this case, we can see that the per-prompt entropies can be low (and thus can be high), but the variance is dominated by the cross terms for , which can be small as long as each low-entropy distribution is mutually distinct from another. To illustrate this, consider an example where the distributions all put their mass on a single sequence, but they are all different across . Then even with a fixed , because the sequences are all different, the colors would average out. Indeed, in this example for all , but for all .
Note that the variance term can also be written as , where , is the average text distribution across all unwatermarked texts. When the pool contains many diverse text distributions that spread the mass of over many distinct sequences, the unwatermarked bias term would concentrate around 0 more tightly.
Bias terms for watermarked texts
Unlike which can go away under a set of ’s that are each low-entropy but collectively diverse, the bias term for the watermarked texts could be persistent even with diverse distributions across prompts. Consider the previous example where the ’s are all concentrated on a single sequence that differs across ’s. For a fixed key, for each we have or under both samplers, and the corresponding is either 1 or 0, whose average need not equal .
More generally, both samplers bias the model towards producing green sequences by taking the original base model green probability and upweighting it via a concave function. Jensen’s inequality then implies a non-positive bias in expectation over .
Fact 3 (Key-averaged bias). Under Assumption 1, both samplers have .
Proof. Let denote the mapping from base green probability to output green probability in the limit :
Therefore, . For either sampler, a finite candidate budget satisfies . Since is concave and , Jensen’s inequality gives
Averaging over yields . For a fixed key, however, the average base green probability may differ from , and can have either sign.
Bias control
Despite persistent bias, there exist design choices that can help us mitigate this bias through a bias-variance tradeoff. We show that adjusting the hash function to have a larger null green fraction can reduce the bias at the expense of variance, improving total estimation error.
Consider the quantity , which is the nominal separation in green probability between the watermarked and unwatermarked text, under the assumption that for every single prompt distribution . As previously mentioned, this coincides with the randomized response regime, where is chosen to be the largest value that respects local -DP, which minimizes the variance of the estimator in the absence of bias. In contrast, in our watermark setting, a larger nominal separation might lead to a larger bias that could offset the benefits from decreasing variance. The following proposition illustrates such a bias-variance tradeoff, in the case with sufficiently many pieces of text (i.e. and ) with sufficient diversity across individuals .
We remark that in the following theoretical analyses, we study the infinite sample budget limit to isolate the effect of the underlying text distributions on estimation error. For both samplers, by Fact 2 the finite- green probability converges to . Empirically, the effect of plateaus very quickly, see Figure 9 and Figure 10 in the Appendix.

Figure 9. MAPE versus candidate budget for rejection sampling. Left to right: .

Figure 10. MAPE versus candidate budget for parallel sampling. Left to right: .
Proposition 4 (Key-averaged MSE). Assume for both samplers, and
Then
where the remainder as and .
The proof is deferred to Appendix § 11.2.4. On a high level, this says that when there is enough diversity across different text distributions, only the sampling noise and the squared watermarked bias contribute to the error.
As illustrated in Fact 3, the nominal separation cannot be realized fully when the prompt distributions do not have perfect entropy, inducing a downward bias that need not vanish with sample size. This motivates us to mitigate this bias by reducing the nominal separation, which can be done by increasing . The following proposition formalizes this monotonicity over sampler-specific ranges.
Proposition 5 (Bias mitigation by tuning ). Consider either sampler in the limit . For each , let be the nominal watermarked green probability as in equation (8), and write . Consider the normalized key-averaged bias
where the key expectation is taken under ideal coloring with green probability . Then:
For the rejection sampler, is non-increasing on .
For the parallel sampler, is non-increasing on , where .
The proof is deferred to Appendix § 11.2.5. For the parallel sampler, the nominal separation is maximized at , while for the rejection sampler it is maximized at . This
result suggests that increasing beyond the RR-optimal values could improve bias at some cost of variance. This tradeoff is particularly relevant at large sample sizes, where sampling noise diminishes but systematic bias can persist. Consistent with this observation, we find empirically that the that works best at larger sample sizes is often substantially higher than the value that maximizes separation, see § 7.3.
Experiments
We now evaluate the empirical effectiveness of the RGS watermark at estimating the prevalence of watermarked texts in a mixture of watermarked and unwatermarked texts. To assess our estimator in both high-entropy and low-entropy settings, we consider a general language modeling task as well as a much harder coding and essay writing task inspired by real-world instruction-following tasks.
Models Throughout the experiment, we consider modern models and instruction-tuned models, consisting of Qwen3-1.7B, Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct.
Shared watermark parameters and setup For computational reasons, we use the RGS rejection sampler in Algorithm 2 to generate watermarked completions, with a maximum of candidates, and the keyed hash is implemented as a 128-bit BLAKE2b hash [39] of the generated token sequence. Unless explicitly varied, we use throughout this section. Across RGS estimation experiments, including the ablations (see Appendix § 12), we average over 20 randomly sampled keys from the space of 128-bit keys, and 50 Monte Carlo mixture repetitions per key.
Internet text (C4)
We use prompts derived from the Colossal Clean Crawled Corpus (C4), a collection of cleaned web text [37]. We use the first 500 prompts from the processed C4 dataset in Pan et al. [36] as our prompt set. For each prompt, we independently sample 144 base model completions using vLLM with temperature=1 and top-p=1. For each prompt, 64 completions enter the unwatermarked source pool, while the remaining 80 are divided into 4 disjoint blocks of candidates for the rejection sampler. We truncate each candidate to its first generated tokens before hashing and RGS selection, and use the same prefix for detection.
For each , we construct 50 Monte Carlo mixtures of total size with watermarked and unwatermarked texts sampled uniformly without replacement from the watermarked and unwatermarked source pools. For each mixture, we test RGS generation and estimation with 3 different privacy budgets , and repeat the same test using a set of 20 randomly sampled keys.
Accurate frequency estimates with private watermarks We find that differentially private watermarks provide highly accurate estimates of frequency across a range of , , and values. Figure 1 shows our estimates of the frequency as we vary the true frequency across a wide range. Our estimates show low bias across the entire range, with average absolute biases of 0.0038, 0.0037, and 0.001 for respectively. The 5-95% percentile bands are generally tight, with average half-widths of 0.072, 0.040, and 0.031 for .

Figure 1. RGS prevalence estimates on C4. Left to right: . Markers show means and shading shows 5th–95th percentiles over keys and repetitions.
These findings hold across a range of values, and our estimator remains accurate even at a fairly stringent value. The main challenge of the estimator is maintaining low relative error at small true , but such an estimator may be inherently difficult to construct given the sampling noise inherent to randomized response style frequency estimates.
Sensitivity to watermark and experiment parameters To assess how the choices of parameters affect estimator performance, we conduct ablation studies on this C4 experiment. We vary the mixture sample size , privacy budget , temperature and generation length, sampler type, candidate budget , and number of distinct prompts in the mixture. The results are presented in Appendix § 12.
Instruction following
We then test RGS on a much more challenging setting, such as a small number of students sharing the same homework prompt, where the underlying distributions are low-entropy, and the prompts are highly homogeneous. Throughout this section, we use . Our setting consists of two datasets: essay writing and coding.
Essay writing We randomly select 20 prompts from the OUTFOX dataset consisting of argumentative essay prompts [25]. Each mixture we test consists of texts sampled from the RGS decoder, and from the same base model, all independently sampled under the same prompt.
Coding We randomly select 20 problems from the LiveCodeBench dataset [19] and construct the same one-prompt mixture as the essay setting. We enable the thinking mode for Qwen3-1.7B and Qwen3-4B and allow them to emit at most 256 thinking tokens before the final answer, which reduces entropy and more accurately reflects realistic LLM use. For watermark generation and detection, we use the first generated tokens, excluding thinking tokens.
Accurate frequency estimates for instruction following settings We find that even in this extreme stress test where there is only a single prompt, and only 100 total samples, we are able to obtain accurate frequency estimates at (Figure 2). The essay setting shows low bias and low overall error across the range of , likely due to the higher per-prompt entropy. In contrast, the coding setting shows higher bias and variance, with visually distinguishable deviation from the unbiased line at low values. Despite this, the estimates have relatively low bias, and it is promising that even in extreme cases with very low diversity and entropy, we are able to obtain such estimates.

Figure 2. RGS prevalence estimates on OUTFOX (left) and LiveCodeBench (right), with . Markers show means and shading shows 5th–95th percentiles over keys and repetitions.
Bias control
We empirically test the effectiveness of the bias-mitigation strategy by tuning proposed in § 6.4. We follow the same C4 mixture construction procedure described above, using an expanded set of 1,000 prompts. We sample from Llama-3.1-8B-Instruct with temperature=0.2, and truncate each completion to tokens, reflecting low per-prompt entropy but high prompt diversity. We find that adjusting can significantly improve estimation error at large via the bias variance tradeoff. Figure 3 shows the mean absolute percentage error averaged over all mixtures and keys, for and , at . In this setting, the value of that optimizes the separation is . We observe that at small sample sizes, smaller generally achieves lower error, but gradually gets taken over by bigger values of . In particular, for and , the errors plateau quickly across 's. This corroborates our theory that bias persists, and at large sample sizes where bias dominates the total error, a larger value achieves a better tradeoff.

Figure 3. MAPE versus mixture size for different . Left to right: .
Computational efficiency
We show empirically that our rejection sampler is very computationally efficient, especially in small regimes, which is consistent with Fact 1. In Figure 4, we plot the mean number of candidates examined until a response is accepted for the rejection sampler, for on C4, OUTFOX, and LiveCodeBench. We use a rejection sampling cap of and . Each point averages the cost equally over 20 randomly sampled keys. Across models, we see that only an average of fewer than 1.5 samples are needed from the underlying base model at small of 0.5 and 1, while larger values require only around 2 samples.

Figure 4. Mean candidates examined by the rejection sampler versus . Left to right: C4, OUTFOX, LiveCodeBench.
Discussion
In this work, we focused on the core challenges behind locally differentially private watermarking, and showed that careful construction of a sequence level, randomized-response style watermark can balance privacy and efficacy. However, there remains important future work in this direction, such as imbuing mild levels of robustness, since currently the sequence-level hashing used by RGS is sensitive to minor edits. Possible mitigations include using locality sensitive hashing [17] or incorporating error correcting codes [8]. While a token-level watermark like that of Kirchenbauer et al. [23] is more edit-robust and computationally efficient, it remains an important problem to control the total privacy leakage of such token-level watermarks, especially over long, autoregressive sequences. Preliminary investigation by the authors using the optimal DP composition theorem [21] over the text sequence shows some promise for frequency estimation, but the accuracy under a fixed sequence-level privacy budget requires further improvement.
Another direction for future work is mitigating privacy leakage across time from repeated inference, as currently our watermark does not guarantee privacy over unlimited queries by a single user. Although memoized randomized response can protect a sensitive attribute across repeated queries by reusing a previously randomized report [14], they apply randomization to both groups with and without the sensitive attribute. In the watermark setting, since the generation process of the unwatermarked group is outside our control, reusing the same noisy report could introduce a persistent shift in detection signal away from the unwatermarked baseline, revealing the watermark status over time.
Acknowledgements
We thank members of the Hashimoto group for useful discussions. ELH was supported by a Stanford Graduate Fellowship. TH was supported by a grant by HAI, gifts from Open Philanthropy, Amazon, Schmidt Sciences, Hudson River Trading, the Tianqiao and Chrissy Chen Foundation and a grant under the NSF CAREER IIS-2338866, ONR N00014-24-1-2609, and DARPA Cooperative Agreement HR00112520013. This work does not necessarily reflect the position or policy of the government and no official endorsement should be inferred.
Keyed hash construction
We construct the coloring scheme described in § 6 as follows: first, we use a keyed hash to generate a pseudorandom value for each sequence, then use a deterministic threshold to map that value to a green or red color. Specifically, we use keyed BLAKE2b [39] with a 128-bit digest. Let denote the integer representation of the digest of an injectively encoded token sequence . Here, is the keyed function modeled as a PRF. The coloring function is its thresholded output, with the threshold chosen to obtain the nominal green probability :
If is modeled as a uniformly random function, distinct sequences receive independent colors, each with green probability
We suppress this small rounding error in the analysis and use as the ideal green probability.
Proofs
Sampler proofs
Feasible reweighting under privacy
Lemma 2 (Feasible reweighting under privacy). Under Algorithm 1, with , a choice of satisfies
if and only if
where
Proof. For , we have that if , and if . Therefore, is equivalent to satisfying both of these inequalities:
Their intersection is the stated interval .
Proof of Proposition 1
Proof. Fix any key , and let be the base model distribution given the prompt. The RGS decoder chooses , which lies in the feasible interval in Lemma 2. Therefore, we have
Conditional on the candidate set , for any that appears in the pool, let be the number of appearances of in . For sequences not in the set, . Then, we have
Taking expectations over and using gives the -local DP guarantee:
Proof of Fact 1
Proof. Fix any key . In each round of the rejection sampling procedure, the probability of accepting a completion is
Since the ’s are sampled independently from the base model across all rounds, the acceptance indicators are independent random variables. Let be the number of rounds until acceptance, so . The sample count needed by the rejection sampler is thus , with .
Proof of Proposition 2
Proof. Fix any prompt and key, and suppress their dependence in the notation. Let denote the base model distribution given the prompt. For a candidate , let be the event that it gets accepted by the rejection sampler. Since , we also have . Thus, by Bayes’ rule, the distribution of the accepted candidate satisfies, for ,
The same bounds hold when , since in that case.
The finite candidate budget only moves the watermarked output distribution toward the base . If round is reached, its fresh candidate has distribution identical to the base and is returned by the rejection sampler regardless of color. Writing as the probability of reaching the final round, the watermarked distribution is therefore
This mixture only moves each likelihood ratio toward , so
Proof of Fact 2
Proof. For the parallel sampler, the empirical green fraction in Algorithm 1 converges almost surely to by the strong law of large numbers. Conditional on the candidates and , the output is green with probability . Since is continuous and bounded, by the bounded convergence theorem.
For the rejection sampler, the probability of accepting a completion on each attempt before the final one is . Since each attempt is independent, the cumulative green probability is
Estimation proofs
Proof of Lemma 1
Proof. Use the shorthand . Under Assumption 1, the 's are independent random variables. Since , we have
Furthermore, independence gives
Proof of Proposition 3
Proof. With the unclipped moment estimator , we have the decomposition
Denote . Then
Note that and are fixed given . The only random quantity is , the sampling noise, with
Then, the conditional MSE is
By conditional independence, we have
Since projection onto can only decrease the MSE, combining the above we have
Lemma: Variance of
Lemma 3 (Variance of ). Consider both samplers in the limit . Let . Under Assumption 1, we have
Proof. For either sampler, write , where
Both maps are -Lipschitz on . For the rejection sampler, this follows from
For the parallel sampler, is continuous and piecewise linear with slopes and .
Write , so that , and is a function of :
Under Assumption , the are independent variables. Suppose is the variable obtained by replacing by an i.i.d. copy . By the Lipschitz bounds, this changes by at most , i.e. .
Note that this change is only nonzero if , and since they are independent variables, this event has probability . Therefore, by the Efron-Stein inequality [4], we have
Proof of Proposition 4
Proof. Taking expectation over the key in equation (10), we can decompose the error to
We bound the last three terms of the equation above. Since , we have
Then, we bound the cross term using Cauchy-Schwarz:
Finally, we bound using Lemma :
Substituting the above into equation (12) gives the desired inequality with remainder . Every term of is proportional to , or , so under the assumption.
Proof of Proposition 5
Proof. Write and without loss of generality, suppose . We consider both samplers.
Rejection Sampler. Consider the rejection sampler in the limit , so that , where
Then, for any , we have
We use the shorthand , which implies . Then, the unnormalized bias is
Since both the numerator and the denominator in the integrand depend on , which is a Bernoulli random variable with mean , we use the leave-one-out trick to tease out the dependence of the whole expectation on .
Denote . Then, . Note that conditional on , . Then,
where
Consolidating, the magnitude of the normalized bias is
Note that is nonincreasing. Also , where each . Since a variable stochastically dominates a variable for , by Corollaries 4.2.7 and 4.2.6 of Roch [38] we have that is nonincreasing in . Overall, since the multipliers and are nonnegative, the magnitude of the normalized bias is nonincreasing in .
Parallel Sampler. Consider the parallel sampler in the limit , so that , where
Similar to the rejection sampler proof, we use the shorthand , which implies . Under the ideal coloring assumption, the are independent variables, with .
Let . Note that is a piecewise linear function, which can be rewritten as
We consider . Using the identity in equation (13), we can write the unnormalized bias as
To isolate the dependence on , define the nonincreasing function
Then , so
The interchange of expectation and summation is valid because all summands are nonnegative.
Since both factors in the integrand depend on , we use the leave-one-out trick to isolate this dependence. Denote . Then , and conditional on , . Therefore,
Consolidating, the magnitude of the normalized bias is
where the last equality comes from the fact that .
Note that is nonincreasing. By the same argument as the rejection sampler case, we have that is nonincreasing in and thus is nonincreasing in .
Ablations
We conduct ablations on the C4 task described in § 7.1. Unless varied, we use rejection sampling with , , , a candidate cap of , and temperature 1. Error plots report mean absolute percentage error (MAPE), obtained by averaging over repeated estimates with 20 randomly sampled keys, where is the clipped estimator in equation (9).
Effect of and
We compare the performance of our watermark estimator on C4 data with the clipped RR moment estimator (equation (4) with clipping onto ) under an ideal randomized response experiment described as follows. For each mixture sample size and prevalence , we fix watermarked examples and draw independent noisy reports and . We compute , estimate prevalence using , and compute MAPE over 1,000 simulations per condition. This RR benchmark matches the nominal green probabilities of RGS rejection sampling at in the limit of high entropy and infinite candidate budget.
Figure 5 plots MAPE as a function of at a fixed . We observe that the errors decrease with increasing , consistent with the looser privacy requirements. Figure 6 plots MAPE as a function of sample size at a fixed , which shows strong scaling. Across both figures we see that in this high-entropy C4 task, the RGS watermark behaves very similarly to the moment estimator in an ideal randomized response mechanism.

Figure 5. MAPE versus privacy budget , with the ideal RR benchmark dashed. Left to right: .
Effect of temperature and generation length
Higher temperature and longer generation lengths both increase the entropy of the text distribution. Figure 7 varies both temperature and the number of generated prefix tokens used for hashing and selection for Llama-3.1-8B-Instruct. Increasing temperature substantially reduces error for short prefixes, while longer prefixes attain lower error even at low temperatures. These trends agree with the theoretical intuition that higher entropy leads to lower error.

Figure 7. MAPE versus temperature for Llama-3.1-8B-Instruct with prefix lengths . Left to right: .

Figure 6. MAPE versus mixture size , with the ideal RR benchmark dashed. Left to right: .
Parallel versus rejection sampling
Figure 8 evaluates the parallel sampler in Algorithm 1 using the same C4 setup as Figure 1. Mean estimates remain close to the true prevalence across all four models, and the percentile bands narrow as increases. Compared with the rejection sampler, the parallel sampler has narrower bands, most visible at . This is consistent with its larger nominal green probability as described in § 4.1.

Figure 8. RGS parallel-sampling prevalence estimates on C4. Left to right: . Markers show means and shading shows 5th–95th percentiles over keys and repetitions.
Effect of candidate budget
Figure 9 and Figure 10 show that both samplers benefit substantially from increasing the candidate budget from to , with little further improvement at or 32. A small budget limits the desired probability of returning a green sequence, as the rejection sampler may reach the cap before encountering a green sequence, and the parallel sampler may have too few green candidates to realize the nominal separation. As a result, a very small will introduce downward bias.
Effect of prompt diversity
We fix the mixture sample size at and vary the number of distinct C4 prompts in the mixture over , allocating an equal number of responses to each prompt. We use Qwen3-4B at temperature 1 with , rejection RGS with , , and . Figure 11 shows that increasing prompt diversity consistently reduces estimation error when the total sample size is fixed, consistent with theoretical findings in § 6.

Figure 11. MAPE versus prompt count for Qwen3-4B, with and . Left to right: .
Examples of RGS-watermarked text
Setting. We present example responses generated by the RGS watermark for five C4 prompts with Llama-3.1-8B-Instruct. We use the rejection sampler with , , and temperature 1. Each prompt is supplied as raw text without a chat template. The four completions presented for each prompt share the same key, with a different key used for each prompt. We display only the first 50 tokens of the completions.
| C4 prompts with four RGS-selected completions each |
| Prompt 0. Cluster comprises IBM’s Opteron-based eServer 325 server and systems management software and storage devices that can run Linux and Windows operating systems |
| 1. . IBM has introduced a high-performance computing cluster, called eServer Cluster 1600, which includes the Linux operating system and the Viuke dual-core processor, a x86 chip from Sun Microsystems technology provider: Advanced Micro Devices (AMD). In this |
| 2. . IBM’s "open" cluster, which comprises the eServer 325 (beta), IBM Systems Director software, and several storage subsystems, is generating interest among customers wanting a high-performance computing setup that accommodates a mix of operating systems, including |
| 3. . Oracle has announced Oracle9i Release 2 Standard Edition, which includes a standard edition of Oracle9i Database Server, Oracle9i Application Server, Oracle Portal and Applications. Release 2 adds improved high availability, the ability to integrate business processes |
| 4. . IBM is pushing the M@L cluster high-end servers into swift and fast implementation paths, highlighting Linux and Windows combinations. The recommendation is for IBM eServer cluster to ease and accelerate Highly-Scalable and optimized cluster solutions for employment by businesses |
| Prompt 1. Belying expectations, Prasar Bharti has earned only Rs 58.19 crore (Rs 581.9 million) as revenue during the |
| 1. 2006-07 financial year. The figure is significantly lower than the Rs 750 crore (Rs 7.5 billion) expected by the organisation as an immediate step towards achieving its annual targets. The shortfall as estimated appears to be mainly due |
| 2. first quarter of FY2023-24, till September 30th, which is just 38% of the total collection of Rs147.120 crore (Rs 1.4712 billion), projected for the entire year. The numbers were shared |
| 3. current fiscal as against the budgeted estimate of Rs 1,061 crore (Rs 10.61 billion). According to the Prasar Bharti’s Annual Report 2010-11, the public sector undertaking (PSU) has failed to |
| 4. April-June quarter, less than Rs 172 crore (Rs 1.72 billion) in the same period during the previous year. According to the revenue figures released by the ministry of information and broadcasting, the significant drop in revenue is partly due |
| Prompt 2. Marketers are employed in the public sector as well as the private sector. Most people think of marketing as a strictly private sector activity, but |
| 1. in reality, many public sector jobs also involve marketing. Marketing is the functional process for which teams are tasked with the planning, creation, and distribution of ideas, goods, services, and values to develop cohort, create Brand awareness, and to achieve [ |
| 2. public sector employees also market products and services. Additionally, many marketers work in non-profit organizations and professional associations. After obtaining employment, they may assume roles which help to bring their product or services to their customers, then, develop and execute marketing strategies aimed |
| 3. many government services and public institutions employ marketing professionals to convey information to the public. Overall, there are a wide range of careers to be found in both public and private sectors. The common term "marketer" generally refers to any marketing professional. Other |
Table 1.
it is also a vital part in the public sector. While government and non-profit organisations may not have sales objectives in the same way as private companies, the need for customer awareness and a positive image translates to a need for effective marketing. In the public
Prompt 3. MIDDLETOWN, NJ - The Middletown Township Public Library and the Township of Middletown are pleased to announce the
launch of "Park ...Then", an innovative new program that combines public fitness classes, environmental education, and relaxed social gatherings at several of the Township’s parks. The program patterned after the successful "Park Fit" program, will feature a series of
launch of a "Fugitive" online public record search tool for the township. The new tool, which is available on the Township of Middletown Municipal Government website, allows users to search for fugitives based on name, race, sex
celebration of their second year of progress in making Middletown a more inclusive and welcoming community. Over the past year, the Township has made a concerted effort to engage in planning and community outreach to identify goals and objectives for inclusion and diversity, with
Middletown-Sayreville-Holmdel-Matawan community wide resource fair, taking place on September 21, from 10:00 a.m. to 2:00 p.m. at the Middletown Township
Prompt 4. An eagerly awaited report from the World Health Organization (WHO) states that processed meats such as bacon and sausages cause cancer, and red meat
, such as beef, may be carcinogenic to humans. The report, from the WHO’s International Agency for Research on Cancer (IARC), recognizes that a clear link between processed meat consumption and cancer has been demonstrated as a causal link. A class
increases cancer risk. Photo:Getty Australia’s biggest meal kit delivery service, Marley Spoon, says it is taking the World Health Organization’s recent cancer warnings on processed and red meat seriously. The WHO says that eating processed meat, such as bacon
may also carry a cancer risk. A team of 22 experts from 10 countries conducted a review of more than 800 scientific studies on the relationship between diet and cancer. The report’s findings are as follows: –Consuming 50 grams of
consumption is also linked to an increased risk of cancer. The WHO’s International Agency for Research on Cancer (IARC) classified processed meat as "carcinogenic to humans", group 1. This is the highest level of classification, assigned to substances
References
- [2]ANTHROPIC (2026). How claude’s text watermark works. https://www.anthropic.com/news/claude-text-watermark. Accessed: 2026-09-26.
- [3]BASSILY, R. and SMITH, A. (2015). Local, private, efficient protocols for succinct histograms. In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing. STOC ’15, Association for Computing Machinery, New York, NY, USA. URL https://doi.org/10.1145/2746539.2746632
- [4]BOUCHERON, S., LUGOSI, G. and BOUSQUET, O. (2004). Concentration inequalities. In Advanced Lectures on Machine Learning: ML Summer Schools 2003, Canberra, Australia, February 2 - 14, 2003, Tübingen, Germany, August 4 - 16, 2003, Revised Lectures (O. Bousquet, U. von Luxburg and G. Rätsch, eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 208–240. URL https://doi.org/10.1007/978-3-540-28650-9_9
- [5]BRAND (2026). Spymarks, not watermarks. https://brand.io/article/spymarks/. Accessed: 2026-09-26.
- [6]BRICKNER-WOOD, B. (2026). The mark of the machine. URL https://www.newyorker.com/culture/infinite-scroll/the-mark-of-the-machine
- [7]BROWN, A. (2026). Does claude’s watermark solve the ai transparency problem? URL https://www.techtarget.com/news/Does-Claudes-watermark-solve-the-AI-transparency-problem/856999/
- [8]CHRIST, M. and GUNN, S. (2024). Pseudorandom Error-Correcting Codes. ArXiv:2402.09370 [cs.CR]. URL http://arxiv.org/abs/2402.09370
- [10]DATHATHRI, S., SEE, A., GHAISAS, S., HUANG, P.-S., MCADAM, R. E., WELBL, J., BACHANI, V., KASKASOLI, A., STANFORTH, R., MATEJOVICOVA, T., HAYES, J., YVES, N., MEREY, M. A., BROWN-COHEN, J., BUNEL, R., BALLE, B., TAYLAN. CEMGIL, AHMED, Z., STACPOOLE, K., SHUMAILOV, I., BAETU, C., GOWAL, S., HASSABIS, D. and KOHLI, P. (2024). Scalable watermarking for identifying large language model outputs. Nature 634 818 – 823. URL https://api.semanticscholar.org/CorpusID:273547059
- [11]DUCHI, J. C., JORDAN, M. I. and WAINWRIGHT, M. J. (2013b). Local privacy and statistical minimax rates. In 2013 IEEE 54th Annual Symposium on Foundations of Computer Science.
- [12]DWORK, C., MCSHERRY, F., NISSIM, K. and SMITH, A. D. (2006). Calibrating noise to sensitivity in private data analysis. In Theory of Cryptography, Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006, Proceedings (S. Halevi and T. Rabin, eds.), vol. 3876 of Lecture Notes in Computer Science. Springer. URL https://doi.org/10.1007/11681878_14
- [13]DWORK, C. and ROTH, A. (2014). The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9 211–407. URL https://doi.org/10.1561/0400000042
- [14]ERLINGSSON, Ú., PIHUR, V. and KOROLOVA, A. (2014). Rappor: Randomized aggregatable privacy-preserving ordinal response. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. CCS ’14, Association for Computing Machinery, New York, NY, USA. URL https://doi.org/10.1145/2660267.2660348
- [15]GOLDREICH, O., GOLDWASSER, S. and MICALI, S. (1986). How to construct random functions. J. ACM 33 792–807. URL https://doi.org/10.1145/6490.6503
- [16]HAYNES, K. (2026). URL https://www.inc.com/kevin-haynes/anthropics-new-ai-watermark-sparks-backlash-from-claude-subscribers/91391704
- [17]HOU, A., ZHANG, J., HE, T., WANG, Y., CHUANG, Y.-S., WANG, H., SHEN, L., VAN DURME, B., KHASHABI, D. and TSVETKOV, Y. (2024). SemStamp: A semantic watermark with paraphrastic robustness for text generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (K. Duh, H. Gomez and S. Bethard, eds.). Association for Computational Linguistics, Mexico City, Mexico. URL https://aclanthology.org/2024.naacl-long.226/arxiv.org/abs/2310.03991
- [18]HU, Z., CHEN, L., WU, X., WU, Y., ZHANG, H. and HUANG, H. (2024). Unbiased watermark for large language models. In The Twelfth International Conference on Learning Representations. URL https://openreview.net/forum?id=uWVC5FVidcarxiv.org/abs/2310.10669
- [19]JAIN, N., HAN, K., GU, A., LI, W.-D., YAN, F., ZHANG, T., WANG, S., SOLAR-LEZAMA, A., SEN, K. and STOICA, I. (2024). Livecodebench: Holistic and contamination free evaluation of large language models for code. URL https://arxiv.org/abs/2403.07974
- [20]Kairouz, P., Oh, S. and Viswanath, P. (2014). Extremal mechanisms for local differential privacy. CoRR abs/1407.1338. URL http://arxiv.org/abs/1407.1338
- [21]Kairouz, P., Oh, S. and Viswanath, P. (2015). The composition theorem for differential privacy. In Proceedings of the 32nd International Conference on Machine Learning (F. Bach and D. Blei, eds.), vol. 37 of Proceedings of Machine Learning Research. PMLR, Lille, France. URL https://proceedings.mlr.press/v37/kairouz15.html
- [22]Kim, B. D., Zhang, W., Ruan, W., Varshney, L. R. and Alabi, D. (2026). Simplemark: Fast multi-bit text watermarking under f-divergence constraints. URL https://arxiv.org/abs/2610.05712
- [23]Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I. and Goldstein, T. (2023). A watermark for large language models. In Proceedings of the 40th International Conference on Machine Learning (A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato and J. Scarlett, eds.), vol. 202 of Proceedings of Machine Learning Research. PMLR. URL https://proceedings.mlr.press/v202/kirchenbauer23a.htmlarxiv.org/abs/2301.10226
- [24]Kobak, D., González-Márquez, R., Ágnes Horvát, E. and Lause, J. (2025). Delving into llm-assisted writing in biomedical publications through excess vocabulary. Science Advances 11 eadt3813. URL https://www.science.org/doi/abs/10.1126/sciadv.adt3813DOI
- [25]Koike, R., Kaneko, M. and Okazaki, N. (2024). Outfox: Llm-generated essay detection through in-context learning with adversarially generated examples. URL https://arxiv.org/abs/2307.11729
- [26]Krishna, K., Song, Y., Karpinska, M., Wieting, J. and Iyyer, M. (2023). Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. In Advances in Neural Information Processing Systems (A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt and S. Levine, eds.), vol. 36. Curran Associates, Inc. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/575c450013d0e999e4b0ecf82bd1afaa4-Paper-Conference.pdfarxiv.org/abs/2303.13408
- [27]Kuditipudi, R., Thickstun, J., Hashimoto, T. and Liang, P. (2024). Robust distortion-free watermarks for language models. URL https://arxiv.org/abs/2307.15593
- [28]Lee, T., Hong, S., Ahn, J., Hong, I., Lee, H., Yun, S., Shin, J. and Kim, G. (2024). Who wrote this code? watermarking for code generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (L.-W. Ku, A. Martins and V. Srikumar, eds.). Association for Computational Linguistics, Bangkok, Thailand. URL https://aclanthology.org/2024.acl-long.268/
- [29]Lensvelt-Mulders, G., Hox, J., Heijden, P. G. and Maas, C. (2005). Meta-analysis of randomized response research: Thirty-five years of validation. Sociological Methods & Research 33 319–348.
- [33]Liang, W., Izzo, Z., Zhang, Y., Lepp, H., Cao, H., Zhao, X., Chen, L., Ye, H., Liu, S., Huang, Z., McFarland, D. A. and Zou, J. Y. (2026). Monitoring ai-modified content at scale: A case study on the impact of chatgpt on ai conference peer reviews. URL https://arxiv.org/abs/2403.07183
- [34]Liu, Y. and Bu, Y. (2024). Adaptive text watermark for large language models. URL https://arxiv.org/abs/2401.13927
- [35]Liu, Y., Zhao, X., Song, D., Wornell, G. W. and Bu, Y. (2026). Position: LLM watermarking should align stakeholders’ incentives for practical adoption. In Findings of the Association for Computational Linguistics: ACL 2026 (M. Liakata, V. P. Moreira, J. Zhang and D. Jurgens, eds.). Association for Computational Linguistics, San Diego, California, United States. URL https://aclanthology.org/2026.findings-acl.1290/
- [36]Pan, L., Liu, A., He, Z., Gao, Z., Zhao, X., Lu, Y., Zhou, B., Liu, S., Hu, X., Wen, L., King, I. and Yu, P. S. (2024). MarkLLM: An open-source toolkit for LLM watermarking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (D. I. Hernandez Farias, T. Hope and M. Li, eds.). Association for Computational Linguistics, Miami, Florida, USA. URL https://aclanthology.org/2024.emnlp-demo.7/arxiv.org/abs/2405.10051
- [37]Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W. and Liu, P. J. (2019). Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR abs/1910.10683. URL http://arxiv.org/abs/1910.10683
- [38]Roch, S. (2024). Modern Discrete Probability: An Essential Toolkit. Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press.
- [39]Saarinen, M.-J. and Aumasson, J.-P. (2015). The blake2 cryptographic hash and message authentication code (mac). RFC 7693, RFC Editor, USA.
- [40]Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R. and Gal, Y. (2024). Ai models collapse when trained on recursively generated data. Nature 631 755–759.DOI
- [41]SitePoint (2026). URL https://www.sitepoint.com/claude-steganographic-request-marking/
- [42]Spirlet, T. (2026). Claude users are canceling their subscriptions, citing anthropic’s new ai watermark. URL https://www.businessinsider.com/claude-users-cancel-subscriptions-citing-anthropic-new-ai-watermark-2026-8
- [43]Tourangeau, R. and Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin 133 859–883.pubmed.ncbi.nlm.nih.gov/17723033
- [44]WANG, T., BLOCKI, J., LI, N. and JHA, S. (2017). Locally differentially private protocols for frequency estimation. In 26th USENIX Security Symposium (USENIX Security 17). USENIX Association, Vancouver, BC. URL https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/wang-tianhao
- [45]WARNER, S. L. (1965). Randomized response: a survey technique for eliminating evasive answer bias. Journal of the American Statistical Association 60 309 63–6. URL https://api.semanticscholar.org/CorpusID:35435339pubmed.ncbi.nlm.nih.gov/12261830
- [46]WU, Y., HU, Z., GUO, J., ZHANG, H. and HUANG, H. (2024). A resilient and accessible distribution-preserving watermark for large language models. URL https://arxiv.org/abs/2310.07710
- [47]ZHANG, H., EDELMAN, B. L., FRANCATI, D., VENTURI, D., ATENIESE, G. and BARAK, B. (2025). Watermarks in the sand: Impossibility of strong watermarking for generative models. URL https://arxiv.org/abs/2311.04378
- [48]ZHAO, X., ANANTH, P., LI, L. and WANG, Y.-X. (2023). Provable robust watermarking for ai-generated text. URL https://arxiv.org/abs/2306.17439