WorksComputer Science
Constructing Budgeted KGW Evasion through Token-Event Evidence Replacement
Abstract
KGW watermarks bias sampling to accumulate keyed detection evidence. We construct budgeted evasion through token-event evidence replacement using ordinary phrase rewrites. Our construction jointly selects compositions of LLM-proposed alternatives from an independent model and measures global token-edit distance on complete outputs. At the largest attainable cost within the budget, exact coverage search maximizes removal of original scored token pairs. It uses a known tokenizer without the key, detector feedback or queries to the watermarked model. Exact accounting separates source evidence removed, evidence created and length effects, with initial strength setting the required reduction. Separate signed-event analysis found that mean suppression arose mainly from removing watermark-enriched source events. In controlled, occurrence-counted, one-token-context KGW tests on short passages, 39 of 82 qualified, initially detected sources yielded rewrites with terminal detector score Z < 4 that passed blinded source-relative model review, at a mean actual token-edit cost of 16.0% under a 20% allowance. To examine whether this initial requirement can be inferred, a separate historical 95-passage study estimated initial detector score Z0 from generation traces with correlation 0.793 and an in-sample association beyond length. This aggregate information did not establish improved allocation, and the separate no-trace stopping test did not reliably exploit cheaper reviewed crossings in the candidate bank. Comparisons did not establish an advantage over simpler selection and ordinary constrained rewriting. The construction makes budgeted evidence replacement executable while exposing limits in signed reduction and recognition of sufficient editing.
Introduction
LLM watermarking embeds statistical evidence during generation and permits comparatively inexpensive detection [10, 5]. In soft KGW, a secret key and token context select a green vocabulary subset. A positive logit bias favors that subset during sampling; detection reconstructs membership and tests the resulting excess with a normalized score. The signal depends on tokenizer IDs and their contexts, whose boundaries can differ from words and propositions. This creates a constrained editing problem: can ordinary phrase replacements reduce detection evidence while preserving the source’s claims and changing only a limited number of tokens?
An edit budget limits how much of this evidence an editor can replace. In the one-token-context setting studied here, each scored event is a token paired with its predecessor. A token substitution can affect its own event and the successor’s context. A phrase rewrite can change more events through retokenization, but those token changes also consume the budget. The signed effect remains hidden: removing green-enriched source events can reduce evidence, while newly created events can reinforce or offset that gain. Even a favorable change must overcome the source’s initial distance from
Preprint. the threshold. Meaning-preserving alternatives, structural coverage and sufficient signed reduction therefore impose distinct requirements.
Robustness analysis and probability-guided paraphrasing already study editing attacks [12, 3, 8] (Zhao et al. 2024). Color-aware substitution also studies low-edit evasion using queries to the watermarked model [17]. Our question concerns how ordinary local rewrites operate under verified whole-output token budgets. We examine three possible limits: whether the available rewrites replace enough source events, whether their hidden signs yield sufficient reduction, and whether the editor can recognize how much editing is needed. A finite bank of local rewrites makes these limits separately measurable.
Our executable construction generates ordinary phrase alternatives and jointly selects complete compositions. It measures token-edit cost on each complete output and selects the composition that removes the most source events at the largest attainable cost within each allowance. Selection uses no source-model probabilities, generation traces or detector feedback; an independent LLM supplies the rewrite proposals. Joint evaluation accounts for token boundaries and events shared across replacements. Initial strength determines whether the resulting reduction is sufficient. Enumeration implements the confirmed policy; a verified prefix-sharing dynamic program reproduces its selections on the same materialized candidate bank.
The spending rule permits comparisons between selectors at the same actual cost; finding the cheapest successful rewrite requires a separate stopping decision. Original events are enriched by watermarked generation, which motivates their removal as a structural objective. The signs of the new events remain unknown, so this objective supplies no guarantee of crossing the detector threshold.
Figure 1 illustrates how this construction changes accumulated evidence through localized wording changes. One phrase changes from “exposed grave misconduct” to “revealed serious misconduct.” The complete rewrite combines three such regions and passes blinded source-relative model review. Its token-edit distance is 14 for a 72-token source, about 19.4%: the intervention is local in the prose but uses nearly the full 20% allowance. The final score falls from 6.290 to 3.973, just below the threshold of four. Some earlier prefixes remain above four, so the example establishes terminal escape only. This retrospectively selected illustration was exceptional: it was the only quality-qualified success with an initial score above 6 at this allowance.
We evaluated whether the construction yields reviewed escapes across generated passages and whether probability weighting improves its selections. In the fixed evaluation, coverage and a surprisal-weighted rule each produced 39/82 escapes that passed blinded source-relative model review, among passages that passed the source-quality gate and were initially detected. Mean actual cost was 15.96% under a 20% allowance. These outcomes establish feasible budgeted construction in the tested setting. Comparisons within the same candidate set and a matched-cost ordinary-rewriting pilot did not demonstrate an additional method benefit.
The study makes three contributions. First, it supplies an executable construction that combines independent local proposals under verified whole-output token budgets with measured detector and quality outcomes. Second, signed-event accounting connects that construction to initial headroom and token-budget limits, while complete candidate enumeration separates missed choices from absent crossing opportunities. Third, it examines estimation of the initial evidence requirement and recognition of sufficient editing. A historical study estimates initial detector score from generation observations without the key. Follow-up inference on the confirmation bank tests whether estimated source evidence improves matched-cost selection. The bank contains cheaper reviewed crossings that a separate no-trace stopping rule does not reliably exploit; added allocation value from the historical estimate beyond length remains unresolved. These contributions concern the original one-token-context KGW setting; human fidelity and transfer require further validation.
Background and related work
Embedding and detection. A generative text watermark couples output choices to a designated statistical signal, which a corresponding detector tests in a candidate passage. Research balances detectability, language quality, computational cost, and editing robustness. KGW uses context-keyed vocabulary partitions [10], while Unigram-Watermark fixes the vocabulary grouping [Zhao et al., 2024]. Distortion-free constructions couple sampling to keyed randomness while preserving the text distribution up to a specified generation budget [13]. SynthID-Text provides production-oriented context for sampling-time embedding and efficient detection [5]. These families have different event representations and scoring rules. A positive test concerns the watermark-associated process under its assumptions; it certifies neither factual correctness nor personal authorship.

Figure 1. Sparse rewriting changes the accumulation of detector evidence. (a) Exact original-CUDA prefix scores for the successful illustration and its frozen selected rewrite end above and below the terminal threshold, respectively. Each curve uses its own token-prefix length; equal counts need not align in content. Numbered gray windows group paired phrase regions; blue/orange strips and table ranges locate them on each token path. (b) Bold words mark wording changes within the larger selected spans, which include unchanged context. These locations are not independent causal score contributions. The output has some prefixes above four, so this example establishes terminal escape only. The retrospectively selected output had global token-edit distance 14 for a 72-token source, yielded 70 output tokens, and passed relative and polished model review. Coverage and weighting selected the same output, reproduced in Appendix § 12.§ 6.
The KGW signal. Soft KGW promotes a context-selected green subset through a positive logit bias and detects excess green events relative to nominal fraction [10]. Membership is defined on tokenizer IDs, whose boundaries can differ from words and propositions. Our setting uses ordered predecessor/current-token pairs with occurrence counting. Repeated identities share a keyed state, which affects statistical dependence. § 3 defines the exact statistic and information boundary; the analysis and experiments concern that historical adjacent-token configuration.
Editing robustness. The edited-token and successor-context effect appears in KGW, and subsequent work provides editing-robustness analysis [10] (Zhao et al. 2024). Character-level perturbation work explicitly analyzes attack range through retokenization and studies both zero-feedback corruption and reference-detector-guided genetic search [19]. Our support statements specialize local counting with occurrence-multiset cancellation, length change and signed margin units; the constructor admits ordinary phrase replacements. Shared counting conventions are required for constant comparisons. Cryptographic constructions also provide edit-distance robustness under computational and alphabet-size assumptions [6]. Our historical KGW result does not extend to those constructions. The experiments use a pinned MarkLLM implementation [14].
Reliability experiments study human and model paraphrasing, surviving source fragments and detection of watermarked spans within longer mixed documents [11]. Their longer-text and localized-detection findings delimit the inference from a short-source terminal crossing in our controlled setting. Surviving evidence can remain detectable under another observation length or detector policy.
Rewriting and selection information. DIPPER studies contextual paragraph paraphrasing [12]. SIRA targets tokens through self-information [3], BIRA applies negative logit bias to a surprisal-derived suppression set [8], and Smoothing uses target-API probabilities to identify low-confidence positions for reference-model resampling [1]. These studies establish probability-guided rewriting as prior work. Our final selector uses token-event coverage without source-model probabilities; the historical trace-assisted initial-strength estimate is a separate diagnostic.
Methodological comparison with SCTS. Low-edit evasion has direct precedent in Self Color Testing-based Substitution (SCTS), which also operates without the key. It queries the watermarked model to infer relative colors from sampled frequencies, then scans for substitutions inferred to change green to red. Its sequential rule uses a successor-context heuristic and evaluates normalized word-edit budgets [17]. Our constructor makes no such queries: it enumerates complete compositions of independently proposed phrases and verifies global token cost before maximizing removed-event coverage at the selected dose. This comparison separates the information used for selection, the search rule and the unit of cost; it supplies no matched performance ranking.
Historical initial-strength inference estimates the required reduction; additive-utility DP optimizes supplied candidate effects under the companion’s state assumptions. It does not reveal their hidden signs or certify crossing. The confirmed constructor uses neither this DP nor the initial-score estimate. The newly verified coverage DP implements its structural policy (§ 5); sufficient-dose recognition remains a separate question.
Watermark stealing learns approximate watermark rules from queried outputs [9]; De-mark probes watermark strength and color assignments through model queries [2]. Published rates under different access, cost units and fidelity criteria do not establish a matched advantage.
Quality-constrained removal. Watermarks in the Sand derives removal under quality-oracle and efficiently mixing perturbation assumptions [18]. Sandcastles in the Storm examines practical mixing and quality-oracle failures, including disagreements with human review [7]. These findings motivate explicitly separating detector crossing from reviewed source preservation. Our model-only judgments and finite local bank establish neither human fidelity nor the assumptions required for unrestricted removal.
Setting and concrete interventions
The editor receives source prose, a pinned tokenizer and the controlled watermark family. It measures token changes and event incidence without querying the watermarked generator or detector. It does not access the generator’s weights, key, colors, detector scores or original generation trace. Public-model probabilities are available only to the labeled comparator. Appendix § 11 records the complete access table.
Let and be the canonical tokenizations of the complete source and output. Actual edit cost is
Insertions, deletions and substitutions each cost one. Edited words and source-character sites can differ from this minimum distance because retokenization and global alignment affect the complete output.
In the original context-width-one KGW configuration, the fixed key assigns each adjacent-token identity a state . Repeated identities share their state. With occurrence count , centered state and , the detector computes
The first token supplies context; scores require . We use , threshold and the original CUDA partition implementation. Repeated identities induce dependence, so this occurrence-counted does not itself establish an independent-event null calibration.
Reductions are source minus output: and . An initially positive source has ; strict escape requires . Equality is excluded even though the implementation’s positive rule is . Quality-qualified escape additionally requires the blinded source-relative review in § 6. Review follows sealed selection and supplies no editing feedback. Human fidelity and external factual grounding remain separate outcomes.
Evidence replacement and crossing capacity
Signed evidence determines the effect of a rewrite
The schematic below illustrates how a replacement changes adjacent-token events. Its colors represent hypothetical evaluator states, unavailable to the editor; they are not inferred states for the measured example.

Figure 1 (PDF p. 5). Source : ; output : . ordinary rewording; unchanged context. Schematic states, : G: , R: . Editor sees token paths. Evaluator supplies colors. Other events cancel.
A rewrite changes event multiplicities. Write and cancel common occurrences, using , to define
These are net occurrence counts: removing an identity and recreating it elsewhere cancels in the terminal statistic. The corresponding evidence is
Substitution into Eq. (2) gives
These identities require no randomness or independence assumptions. At unchanged length, and . For initial margin , strict escape occurs exactly when
Thus the same reduction can escape one source and leave a higher-headroom source detectable. The tokenizer determines which events change; their keyed states determine whether the created evidence offsets the removed evidence.
A necessary token budget
A substitution can change its own event and the successor’s context, as established in KGW and editing-robustness analysis [10] (Zhao et al. 2024). Accounting for global occurrence cancellation gives the following length-sensitive bound.
Proposition 1 (Support and historical margin capacity). For complete token paths at global token Levenshtein distance ,
For fixed binary event states, , , and ,
An initially positive source therefore requires for strict escape.
The proof and general- ceiling appear in Appendices A and A.5. This necessary bound grants every changed event its most favorable state. It guarantees neither a faithful candidate nor favorable realized signs. For an initially detected source in this configuration, , so the one-sided length condition holds automatically. A two-sided version also certifies robustness to subsequent token edits within its stated domain (Appendix § 10.§ 3).
Source enrichment and estimated starting strength
Watermarked sampling enriches the source in green events. Prior color-agnostic editing analysis already distinguishes enriched retained fragments from nominal new fragments [17]. Under the additional approximation that destroyed events have common positive centered mean and created evidence has conditional mean zero,
Here is the editor’s permitted information, including its chosen candidate. This conditional reference motivates coverage selection; it supplies neither a universal bound nor calibrated prediction. Created identities can reuse observed contexts, and the active modulo-reseed implementation does not justify secure-PRF independence.
Initial-strength estimation addresses the required reduction in Eq. (7). The historical diagnostic in § 7.4 asks whether richer observations reveal this headroom. Let include generation-time logits and known sampling processors, which are unavailable to the final constructor. Write . If , linearity gives
The implemented approximates these conditional means. This trace-assisted setting uses no key but has a richer observation contract.
At unchanged length with homogeneous enrichment, nominal created evidence, and approximately constant efficiency , the mean-response reference becomes
This connects aggregate strength to the amount of evidence that editing must replace. Candidate-specific source yield, created states and semantic constraints can change the realized requirement. Moreover, estimating the sum along the source and comparing two edits require different posterior directions: shared evidence cancels in candidate contrasts. At equal output length and fixed candidate-effect estimates, the initial-score term is common to all candidates and cannot change their ranking. It can inform the separate question of whether the predicted reduction is sufficient. The construction below does not use as a stopping rule. We evaluate the historical estimate and a separate stopping attempt under their respective access contracts. Joint search first constructs the attainable coverage choices at each actual cost; recognizing a sufficient choice additionally requires information about its signed effect.
Constructing and selecting local rewrites
Constructing the candidate language
Algorithm 1 constructs ordinary rewrites and selects among their complete combinations under a token-edit budget. The procedure first chooses phrase locations, requests alternatives and applies explicit mechanical checks. It then measures complete outputs so that shared token boundaries and interactions between replacements enter the selection. A fixed rule chooses at most eight disjoint source-character spans using punctuation, word-count limits and distributed quantiles. GPT-5.6 Luna at low reasoning effort receives the whole source and two requests per span: a small wording change and a grammatical reordering/equivalent phrase. Instructions preserve propositions, roles, referents, quantities, qualifications, modality and negation; the provider receives no tokenizer objective, key or score.
Mechanical admission checks each single-site complete output for canonical round-trip, token distance between one and eight, absolute net length change at most , lexical anchors and ordinary formatting. Malformed or missing responses remain failures without automatic retries. These literal checks precede global preservation review. Exact rules and the language appear in Appendices B.3 and B.4.
Let mean that decoding the canonical encoding of the complete composed prose reproduces that prose exactly. For , feasible compositions satisfy
Joint selection under actual token budgets
To compare selectors at equal actual editing cost, we first identify the largest attainable token-edit count within each allowance. Among outputs with that count, coverage selection removes the most original event occurrences:
Here counts edited sites, lists candidate IDs in source order, and is the text hash; ties use the retained deduplication representative. The largest-attainable-dose rule was frozen for matched-cost selector comparisons; it supplies no minimum-cost crossing guarantee. Let retain the winner of the same coverage ordering at each attained actual cost. The returned output is ; this map separates joint selection from the dose decision.
Algorithm 1 Budgeted token-event coverage construction
Require: Source prose with scorable, round-tripping token path ; pinned tokenizer; rewrite provider; frozen nomination and admission rules.
Ensure: Sealed outputs for and structural metadata.
Count source adjacent-token occurrences ; set .
Nominate, request and mechanically admit alternatives (§ 5.1).
Initialize seen-text hashes and cost-winner map .
for each keep/alternative configuration in deterministic order do
Compose complete prose ; set ; if then continue.
Add to ; canonically encode as .
Measure minimum token distance and .
if then continue.
Measure complete-output event-multiset differences using Eq. (3).
Retain in if its tuple in Eq. (15) is smaller, or is empty.
end for
Read all four from the completed map using Eq. (14).
return sealed and metadata for each budget; the identity is available at .
Sites follow source order; each site lists keep first, then alternatives by candidate ID. Text deduplication retains the first path. Keyed scoring and blinded quality review occur after selection, without feedback.
Exact implementations and computational tradeoffs
Equation (10) motivates this objective: removing more watermark-enriched source events can reduce expected evidence under the stated assumptions on their enrichment and the introduced events. Exactness applies to coverage in the finite actual-cost slice; semantic preservation and crossing remain measured outcomes.
Complete-output enumeration measures shared edges, boundary retokenization, recreated identities and global alignment directly. A follow-up prefix-sharing dynamic program (DP) computes the same on the materialized token paths. Its state retains an exact minimum-distance column and capped source-event overlap; bottom-up tables choose the best descendant at each actual cost. It reproduced all 420 allowance selections and 1,521 attained-cost choices (Appendix § 10.§ 7). This verifies the coverage solver on the supplied bank; every terminal is examined and upstream enumeration remains necessary for this implementation. Graph construction and storage make its computational benefit conditional (Appendix § 11.§ 2). The historical additive-utility DP has a different objective and is retained in the technical companion.
There are at most configurations, measured using a tokenizer and bit-parallel edit distance, with no local language model or partition computation. An isolated replay using source prose and recorded proposal responses reproduced all 420 coverage outputs on 105 sources with networking and model/provider/detector imports disabled. This verifies the local constructor’s access boundary; fresh proposal generation remains a separate provider operation. Median local selection took 8.92 ms per source, excluding provider and review inference (Appendix § 11). The same-bank surprisal comparator and its uncalibrated rank utility are specified in Appendix § 11.§ 4.
Experimental design
Cohort and detector. We generated 128 watermarked continuations from 16 prompt groups crossed with eight fresh keys. We edited these generated continuations. The saved prompts came from a locally processed artifact described as derived from the Colossal Clean Crawled Corpus (C4) [16]; its raw upstream split and URL lineage remain unresolved. Qwen2.5-1.5B-Instruct [15] used float32 CUDA, temperature 0.8, top-, top-, repetition penalty 1.1, a four-gram repetition constraint and a NumPy-PCG64 draw. Generation and detection shared the original left-window KGW with , , logit bias and terminal threshold four. Scoring preserved original CUDA partitions. Appendix § 11 records pinned revisions and hashes; Appendix § 15 gives executed prompts and provider metadata.
Blinded source review preceded rewriting and keyed outcome evaluation. Enrollment required a completed paragraph, at least 40 canonical tokens and inherited language/coherence defect . It retained 105 sources, including 82 initially above , with median qualified length . Sixteen same-model unwatermarked passages supplied smoke controls. Every selector used the same source-specific attained cost at each allowance; detector scores did not determine dose.
Quality and outcomes. The frozen rubric separates benign variation , inherited source defects , introduced factual/logical risk , and introduced fluency damage . Relative quality requires ; polished quality further requires , and absolute quality requires . For confirmation, GPT-6.1 Sonnet with medium reasoning reviewed prose blinded to scores and states. Reuse required exact source/output hashes; missing judgments remained unreviewed. These are model-only source-preservation judgments, without human or independent factual validation.
The prospective primary compared weighted and coverage selection in relative-quality strict escape at the 20% allowance. Secondary controls used deterministic hash ordering and weighting from a public 3B checkpoint. The primary weighted rule used the public generating checkpoint. Neither probability comparator received generation traces or keyed states. Continuous original summaries use all 105 sources; cost–outcome curves use the same 82 positives for both coordinates. Weighted outputs have complete reviews at all four doses. Later completion of 25 missing coverage crossing reviews determines that policy’s qualified-escape curve, while noncrossing outputs without judgments remain unreviewed (Appendix § 12.§ 1).
Inference and follow-ups. We retained within-document pairing and 10,000 crossed prompt/key bootstrap replicates, seed 130913, over 16 prompt/eight key clusters, using independent axis resampling as in the pigeonhole bootstrap [Owen, 2007]. Intervals are pointwise; Appendix § 11.§ 6 specifies resampling and empty-draw handling. Candidates, doses and repeated events do not increase the source sample size. Retrospective completion of 39 hash labels preserved 66 existing judgments and did not change the primary.
Subsequent audits enumerated the frozen bank, reviewed minimum-cost witnesses and tested blind stopping with training/calibration excluding held-out prompt and key groups. A solver check compared coverage DP with enumeration; a separate reconstructed-observation analysis tested source-evidence selection at matched actual cost. These analyses reuse previously analyzed sources. Historical strength estimation, nomination and the ordinary-rewriting development comparison use separate cohorts and access contracts (Table 3); none enlarges the confirmation sample. Detailed protocols and missingness are retained in Appendices § 11, § 13 and § 14.
| Study | Sources; clusters | Information and role |
| Confirmation | 105 of 128; 16/8 | 82 initially detected; final coverage uses tokenizer-only structure. |
| Starting score | 95; 12/8 | Historical CUDA generation observations; aggregate inference. |
| Source nomination | 64; 8/8 | Historical MPS generation observations; candidate-supply diagnostic in the companion. |
| Ordinary rewriting | 16; 10/8 | Exposed development pairs; matched actual cost and mixed model reviewers. |
| Stopping | Same 105; 16/8 | Exposed bank; other-key training scores, without generation traces. |
Table 3. Cohort roles and information access. Prompt/key counts denote clusters, not additional observations. Development analyses are exploratory and are not pooled with confirmation.
Results
Budgeted construction produces reviewed terminal escapes
We first test whether composing ordinary local rewrites can cross the terminal threshold while meeting the token budget and source-preservation rubric. At the 20% allowance, coverage selection achieved 39/82 relative-quality-qualified escapes (47.6%, crossed 95% interval [28.6%, 66.7%]), at mean actual cost 15.94 tokens (15.96%) among initially positive qualified sources. Of its 40 raw crossings, 39 passed relative review and 34 passed polished review. Thus the construction yielded reviewed escapes in this setting, while the stricter fluency criterion reduced their number. The small raw-to-relative gap describes admitted outputs under model review; it does not measure human fidelity or the effect of quality constraints on candidate supply.
Figure 2 distinguishes continuous suppression from threshold crossing across editing allowances. Mean rose from 0.436 to 1.373, but a reduction produces escape only when it exceeds the source’s initial score excess. Most of the observed increase in qualified crossings occurred by the 15% allowance: the count rose from 13 at 5% to 37 at 15%, then to 39 at 20%. The final spending increase therefore added few crossings in aggregate, without establishing noninferiority of the smaller allowance. These are nonnested fixed-dose selections, so the curve supplies neither individual monotonicity nor an optimal cost frontier. Appendix § 12.§ 1 retains all operating points, uncertainty, the separate weighted curve and retrospective completion of 25 coverage reviews.
Figure 2. Frozen coverage-policy operating points under 5%, 10%, 15% and 20% allowances, in increasing horizontal order. Both axes in both panels use the same 82 initially detected qualified sources. Actual cost is mean global token distance divided by source token count. Panel (a) separates detector-only, relative-quality and polished-quality escapes; panel (b) retains all score changes in the mean. Error bars are pointwise crossed prompt/key 95% intervals for relative-quality escape and mean (16 prompt/eight key clusters). Outputs are nonnested fixed-dose selections, and no dose was chosen by detector score. Retrospective review completed 25 previously unreviewed crossing outputs under the unchanged rubric; all crossing identities now have judgments.
Probability weighting did not improve the prospective primary endpoint. Weighted and coverage selection each achieved 39/82 qualified escapes at identical actual costs; their paired difference was zero percentage points with 95% interval [, ]. This supports the feasibility of the tokenizer-only selection stage within the shared construction, without establishing equivalence or an advantage over ordinary constrained rewriting. We examine those attribution limits below.
Evidence replacement and initial margin account for the outcomes
To identify what produces suppression, we separate removed source evidence, created evidence and the length boundary. This mechanism analysis retains frozen weighted-policy outputs across all 105 qualified sources, separately from the coverage dose curve. At the largest dose, removed source evidence contributed about 6.14 margin units on average, compared with 0.20 from created states and a small adverse length contribution. The complete decomposition is retained in Eq. (43). Destroyed occurrences were 78.92% green, whereas created occurrences were 49.02% green; mean absolute length change was only 1.44 tokens (1.64%). Replacing enriched source evidence therefore accounted for most of the mean reduction. These evaluator-side summaries contain correlated events and do not establish a blind predictor of individual signed effects.
Individual crossings also depend on which evidence is removed and introduced. Figure 3 tracks the remaining margin in two retrospectively chosen outputs with similar initial margins. The successful rewrite destroyed fewer events, yet removed more source evidence; its created events further reduced the margin. In the still-detected rewrite, created evidence offset half of the source contribution before the length penalty. Their net centered-evidence removal was 10 versus 3. The first output ended only slightly below the threshold, whereas the second retained a substantial positive margin. Both passed relative and polished model review. Their different sources and costs make this an accounting illustration, without a matched causal or per-edit attribution claim.

Figure 3. Remaining detector margin for two retrospectively chosen weighted selections at the 20% allowance. Both panels use the same margin scale; strict terminal escape requires a final value below the dashed zero line. Starting at , source removal changes the margin by (blue), created evidence by (orange), and length by (gray), giving (black). Numbers above intermediate bars are signed changes; initial and final bars show levels. These are accounting stages of complete outputs, not sequential edit scores or separate per-edit effects. Both outputs passed relative and polished model review. The successful output is the passage in Figure 1(a). The different sources and costs preclude a matched causal comparison.
Initial headroom determines how much reduction is required. Among weighted outputs, qualified escape counts were 28/37 for , 10/28 for , and 1/17 for . These descriptive strata place 38/39 successes at , connecting the observed yield to source strength. Neither this association nor the favorable-sign budget bound guarantees a crossing for an individual source. Appendix § 12.§ 4 reports the full event, budget and post-edit stability diagnostics, including the opening example’s lack of a positive certified repair radius.
The candidate bank separates selection limits from dose opportunities
The next question is whether better choices within the admitted bank can improve these outcomes. At fixed actual cost, weighting added mean [−0.0380, 0.0883] over coverage; retrospective constant and uniform-choice controls also left added selection value unestablished. Choice was restricted: among 82 positive sources at the largest dose, only six attained-cost menus contained both crossing and noncrossing outputs. The null comparisons neither identify the effects of shared proposal and admission stages nor establish selector equivalence. Full policy outcomes, turnover and control intervals appear in Appendix § 12.§ 2.
A separate comparison tests the broader construction against ordinary localized rewriting. On 16 previously analyzed development sources, the historical weighted constructive reference added mean [−0.486, 0.509] at identical actual token cost; both arms escaped on 7/10 initially detected sources. All selected outputs passed relative review, with mixed cached Sol/new Astra reviewers under a common rubric. Unequal proposal supply and inference cost, source reuse and the wide interval leave both superiority and equivalence unresolved. Appendix § 12.§ 3 retains the competence gate and the separate null all-cost-objective test.
Complete enumeration shows where a different dose could matter even when fixed-cost selection has limited opportunity. The actual-key audit scored 35,548 eligible unique outputs, reproduced 1,680 original endpoints and partitioned the 82 positives in Table 1. Of the 42 coverage failures, two could cross through another output at the same actual cost, fourteen had crossings only at a lower cost, and twenty-six had no crossing in the allowed bank. Improving ranking only at the largest attained cost cannot recover the fourteen lower-cost-only opportunities. The last category identifies a limit of this frozen candidate language for the realized key, without implying universal editing impossibility.
| Source outcome | Count |
| Escaped under the original coverage selection | 40 |
| Additional escape available at the same actual cost | 2 |
| Additional escape available only at a lower actual cost | 14 |
| No escaping output in the full allowed candidate bank | 26 |
| Total initially detected qualified sources | 82 |
Table 1. Mutually exclusive outcomes for the 82 initially detected qualified sources at the 20% allowance. Counts use raw strict , the actual key and the frozen candidate language. The first row includes 39 reviewed quality-qualified escapes; no quality qualification is imputed to oracle alternatives at the same actual cost. Subsequent review qualified a minimum-cost witness for each of the 56 bank-crossing sources, as reported below. The source sets were checked directly against saved records.
Quality review confirms that cheaper crossings can preserve the source under the rubric. All 56 passed relative quality and 51 were polished. Their costs therefore attain both raw and relative-quality crossing minima within this bank. On the same 39 frozen coverage successes, mean cost was 17.31% for the policy and 5.43% for reviewed minima. This actual-key-discovered gap establishes unused cost-saving opportunities, without changing the operational 39/82 result. Many witnesses crossed narrowly: median slack was , with 40/56 between 3.8 and 4. Four. These small slacks make prediction error consequential for a rule seeking to identify the cheaper crossings without detector access. This motivates the final analysis.
Estimating headroom and recognizing sufficient editing
Information about initial strength. For the evidence-based stopping formulation studied here, remaining margin combines initial headroom with a candidate’s signed effect. Estimating initial headroom is one route to reducing this uncertainty. On the separate historical 95-source CUDA cohort, trace-assisted correlated 0.793 with measured , with mean error +0.089 and error standard deviation 0.769. An in-sample regression on an intercept and had ; adding raised it to 0.630 (Figure 4). The generation observations therefore contained aggregate strength information beyond length in this exposed cohort, with no held-out calibration or plaintext prediction claim.
Figure 4. Historical trace-assisted starting-strength diagnostic on 95 sources (12 prompt/eight key clusters), separate from the closing confirmation. Panel (a) compares approximate key-free estimates with saved actual-key scores; the dashed line is equality. Panel (b) residualizes both quantities on an intercept and in the same cohort; dashed axes indicate zero. Each point is one source. The association is descriptive, without held-out calibration or demonstrated allocation benefit beyond length. Stored generation observations are inputs to this diagnostic only, and are absent from the final constructor.
This association did not establish added allocation value: retrospective length-first and estimate-first allocation each yielded 32 crossings, compared with 33 for saved triage at slightly different expenditures (Appendix § 13). The final constructor uses no generation traces. The following stopping experiment evaluates a different estimator and cohort, so its outcome does not test the historical estimator.
From initial strength to candidate choice. A follow-up reconstructed generator observations for the confirmation bank to test whether inferred source evidence improves selection. With the bank, enumerator and actual edit costs fixed, maximizing predicted destroyed evidence added mean with crossed 95% interval across 104 resolved sources; one source used coverage fallback. This richer information did not establish an advantage over coverage. Two newly selected crossings lacked quality reviews. Appendix § 13.§ 4 retains the operational results and quality bounds. Estimating aggregate strength and predicting differences between rewrites therefore remain distinct empirical tasks; exact search does not ensure an accurate signed-effect objective.
Recognizing a sufficient dose without traces. Returning to the 105-source confirmation bank, a blind path keeping one coverage-selected output per attainable cost retained raw crossings for 50/56 bank-crossing sources, attaining the full-bank minimum cost for 31. These counts establish substantial retained opportunity, without implying identity with the reviewed witnesses. A controller estimated remaining evidence from length and public-model mean surprisal, using held-out prompt/key groups, other-key training scores and an empirical upper-residual allowance. Its information differs from both the original constructor’s and the historical trace-assisted estimator’s.
The primary uncertainty-adjusted rule selected exactly the original 20%-allowance coverage outputs for all 105 sources. Only one of the 82 positives met its predicted-target condition; 81 returned the fixed-dose fallback. Unchanged yield thus reflects fallback, with only 1/82 positive-source target declarations. Removing the uncertainty allowance saved 4.41 cost percentage points (15.96% → 11.55%; 95% interval [2.24, 6.90]), but raw crossings fell from 40 to 25. The cheaper outputs exposed a cost–crossing tradeoff, without demonstrating preserved quality and robustness (Appendix § 14).
Starting-score error and intervention-conditioned signed effects both contributed to remaining-score error; neither dominated uniformly. The narrow slack of many cheap witnesses leaves little tolerance for their combined error. The tested controller did not convert retained candidate opportunities into reliable savings. Reliable additional cost savings over the fixed-dose rule remain unresolved. The reviewed witnesses nevertheless establish that the largest-dose rule does not minimize actual-key crossing cost within this bank. This separates the demonstrated ability to construct budgeted escapes from the unresolved ability to recognize the cheapest sufficient edit without detector access.
Discussion and limitations
Complete-output measurement makes the editing constraint verifiable; the weighted-policy analysis connects suppression to removed and created evidence. The bank audit separates cheaper candidates’ availability from their recognition: the tested stopping failure supplies no cost lower bound for every key-blind method. Coverage DP shares computation while preserving selections. Improving signed prediction or candidate supply requires separate evidence.
The constructor can operate after target-model access ends, given the source, known tokenizer and independent rewrite proposals. Observability through other records or subsequent examination of edited text was not evaluated.
Further evaluation should distinguish the contributions of proposal generation, admission, selection and dose. The present controls leave those shared stages unseparated and competitive advantage unresolved. Initial-strength estimation and candidate comparison require different information. The development result supports aggregate strength inference under its trace contract; candidate-difference prediction, plaintext-only inference and added allocation value beyond length require separate evidence. These distinctions constrain which improvements the existing results can justify.
Quality and scope also delimit the result. The reviewer assesses preservation of the source’s claims; it can accept an output that retains an unsupported source assertion. The exploratory Dolly pilot had no sources supported by the supplied context, leaving the escape rate conditional on such sources undefined (Appendix § 12). The confirmation covers short sources and the original occurrence-counted KGW setting. Other generators, context widths, seeding rules, repeat handling and prefix/interval detectors require separate evaluation. Sixteen prompt and eight key clusters limit precision, and 16 unwatermarked smoke controls cannot validate a rare-tail false-positive rate. Local timing excludes proposal inference and quality review. Fresh, component-matched comparisons and blinded human review would address these remaining efficacy and fidelity boundaries.
Conclusion
Jointly composing ordinary local rewrites produced 39/82 terminal KGW escapes that passed source-relative model review, at mean actual token cost 15.96% under a 20% allowance. Evidence accounting relates these outcomes to removed source states, created states and initial headroom. Enumeration and a verified prefix-sharing DP implement the same finite coverage rule. Controls established no advantage for elaborate selection or over ordinary constrained rewriting. Reviewed cheaper crossings demonstrate further candidate-bank feasibility. Development estimation provides partial headroom information from generation observations; the separate no-trace stopping test leaves sufficient-dose recognition unresolved. Actual cost, continuous reduction, crossing and preservation together delimit the construction’s demonstrated value.
Evidence accounting, capacity and joint-composition proofs
Occurrence identities and exact score changes
The event histogram counts occurrences of the fixed ordered identity . Define , and . Since , summing over identities gives
The equality holds with repeated identities and with identities moved between positions. It relies on occurrence-counting with fixed state per identity; a repeat-deduplicated detector has a different histogram and statistic. For , subtracting the two centered margins directly proves Eq. (5). For the normalized score, write
which proves Eq. (6). The strict crossing criterion follows by writing and preserving the strict inequality.
Proof of the support and capacity proposition
For scorable endpoints, fixed binary event states, and ,
The following proof establishes this general ceiling and the historical corollary stated in the main text.
Proof of Proposition 1. For nonnegative counts , . Coordinatewise application proves a directed multiset triangle inequality , and the same inequality holds for with the arguments reversed. It remains valid when paths contain repeated identities or an intermediate path is empty.
An internal substitution destroys at most two adjacent events and creates at most two. A deletion destroys at most two and creates at most one; an insertion destroys at most one and creates at most two. Endpoint operations and identity cancellation can only decrease these counts. Consider a shortest token edit script with substitutions, deletions and insertions, where . The directed triangle inequalities along this script give
Since , we have , and , proving Eq. (8). The support inequalities also hold for empty endpoints if event count is defined as . The identity uses nonempty endpoints, as in the scorable domain of the score equations.
Every destroyed centered state is at most , and the negative of every created centered state is at most . Hence . Applying the first two support bounds to this weighted sum, followed by Eq. (5), gives Eq. (19)`. This is a relaxation: states of shared or constrained partitions need not simultaneously attain all favorable extremes.
At and , the right side simplifies to
If , square-root concavity and imply . If , both the absolute-length penalty and square-root change are negative. Thus . Strict escape requires and therefore , yielding the stated integer floor. ▫
The support factor two is sharp in token space when the vocabulary contains at least distinct IDs. Start with distinct tokens and replace each even-positioned token by a distinct fresh ID. The changed positions are internal and pairwise nonadjacent, their original incident edges disappear, and all new identities are fresh. Consequently . The proved bound also establishes that the endpoints have minimum distance at least , while the displayed substitutions supply a script of cost . This witness establishes geometric sharpness without asserting a semantic or fluency property.
For general and , the same corollary follows whenever
Indeed, the positive-length boundary term is at most , which under this condition is at most . For negative the boundary term is already adverse. This sufficient condition makes explicit why a uniform margin constant depends on the threshold, green fraction and scored-length domain. Context-width, seeding-by-position and repeat-handling changes require corresponding analyses.
For a strictly detected source, and imply for and . The domain in Eq. (22) is therefore automatic for such sources; in the historical configuration, detection requires . The one-sided ceiling consequently applies to every strictly detected source in this fixed-state adjacent-event setting, with a scorable output. The length assumptions below remain explicit for the two-sided repair statement because an escaped path can have negative margin.
A two-sided margin and strict repair certificate
Corollary 1 (Sufficient post-edit token stability). For the historical , detector with fixed adjacent-token states, two paths , with at least four scored events each satisfy
If is an escaped output of margin and scored length , every further canonical-token edit of cost at most
retains strict escape.
Proof. Apply Eq. (9) to both ordered pairs and . For a repair to of distance at most , the resulting margin is at most . The first radius constraint ensures this expression is strictly negative. The second retains at least four scored events, since a distance- edit changes token count by at most .
The ceiling-minus-one preserves strictness. For example, certifies a one-token repair; two edits can reach equality and are uncertified. In the historical setting the strict condition can be checked without floating-point threshold comparisons. For a proposed radius , let . Then is equivalent to or when . This is a sufficient, evaluator-side decision certificate. Character- or word-level repairs need their own canonical token distance, and the certificate supplies no guarantee of semantic preservation.
Conditional uncertainty and the crossing objective
Conditional on editor information , let be the vector of centered keyed states, and . For a fixed candidate with incidence vector , its gain has
The final margin instead has
The unknown baseline and candidate gain are correlated because they involve shared keyed states. Controlling gain variance consequently does not by itself control the probability that . Likewise, two outputs with different lengths can order differently under expected , expected margin and crossing probability. The paper’s measured construction optimizes its declared structural objective; these conditional equations characterize the information gap without asserting a calibrated chance-optimal policy.
Repeated identities also affect uncertainty directly. If distinct identities were conditionally independent with nominal Bernoulli variance , an event appearing times would contribute to the variance of its occurrence-weighted sum. Shared partition constraints and posterior observations can add covariance terms. Candidate appearances and repeated occurrences therefore cannot be treated as independent experimental units. The empirical intervals use the recorded document, prompt and key structure.
Aggregate prediction and candidate contrasts. For candidates , fixed by , write , and . Their margin-gain contrast has
For posterior-mean error , aggregate starting-score error is , whereas the error in the predicted margin-gain contrast is . If the two directions are not collinear, zero error in the former need not imply zero error in the latter. Shared states cancel in the contrast, so absolute prediction error can also coexist with an accurate candidate ordering. Under the homogeneous reset reference, its mean is approximately ; matching support and length removes this common structural signal.
At equal output lengths, greater margin reduction and smaller final induce the same ordering. With different lengths, define . Then
These identities locate the information required for a selection decision. They do not establish that the historical aggregate estimator or the final surprisal utility estimates these directions reliably.
Position relative to prior editing bounds. The own-event/next-event locality and token-edit robustness arguments precede this study [10] (Zhao et al. 2024). Our exposition fixes the scored count at , carries occurrence multiplicity and net length through the signed margin, and applies the resulting ceiling to the controlled editing problem. Numerical constants for a normalized cannot be compared directly with an unnormalized centered score or a convention using all tokens in centering. We make no claim of the first editing-robustness theorem or a generally tighter security bound.
Equivalent crossing scales and conditional editing references
On the score scale, the crossing requirement in Eq. (7) is
The reduction measures the intervention’s effect, while measures the source’s initial distance from crossing. This distinction motivates reporting both continuous suppression and strict escape at a measured edit cost. The observed score and its estimate have separate roles: keyed evaluation supplies , and a blind estimate can only approximate the required headroom.
The required fraction is explicit when source evidence is positive. Writing , and gives
Here is a realized signed evidence fraction and can lie outside . It differs from the structural fraction : removed source states and created states determine how much evidence a given support actually replaces. This identity relates source strength to the required intervention without assuming a homogeneous source or nominal created states.
A schematic equal-support contrast. Consider an internal substitution with no identity collisions. It has , and . At , destroying two green events gives . If the two created events are red, then and ; if both are green, then and . This mathematical schematic isolates the role of signed conversion at identical edit cost, support and length. The measured examples in § 7 provide separate, retrospectively selected illustrations with their recorded quality labels.
Structural efficiency and signed conversion. For and , let and . Then
Structural efficiency satisfies , whereas the realized contrast determines conversion of that support into score reduction. The quantities on the right are evaluator diagnostics apart from , , . This decomposition explains how improved coverage can coexist with weaker signed reduction and why additional structural search has diminishing room when approaches two.
The mean-cost reference in Eq. (12) assumes approximately constant and nominal created evidence. It predicts a mean response, with separate assumptions from the necessary favorable-sign ceiling. All finite computations retain exact square roots. The technical companion preserves the normalization expansions and their remainder bounds.
Why complete compositions require joint measurement
In the distinct-identity schematic of Figure 5, use and analogous subscripts. The isolated and composed gains are

Figure 5. Schematic of a joint-composition interaction, with distinct token identities and unchanged tokenization. The isolated edits each have and ; the joint path has and . Adding isolated counts would remove the source edge twice and omit the new joint edge . No colors, measured detector effect or semantic quality are assigned. Appendix A.6 derives the signed correction.
Consequently,
The correction removes a double-counted source edge and substitutes the actual composed edge for the two isolated middle edges. It can have either sign. The equal-length score correction divides this expression by . This motivates whole-output measurement without establishing a solver advantage.
Three representation requirements also matter. Canonical tokenization must be verified on complete prose because separately encoded fragments can merge across boundaries. Actual cost is minimum Levenshtein distance; a nonminimal alignment’s cost cannot identify an exact-cost slice. Finally, coverage is
It depends on capped occurrence overlap. Last-token state suffices for an additive edge utility, but generic exact coverage also needs overlap history. Enumeration measures all three quantities directly in the small final language. The technical companion retains the alternative additive-utility DP, sufficient state conditions and estimator-error bound. That formulation was not the executed Algorithm 1.
Verified coverage DP on canonical output paths
The follow-up solver compiles the admitted canonical paths into a prefix trie. At prefix , it retains the exact Levenshtein column for , represented by bit vectors, and capped counts . The prefix identifies its length and last token. Appending token uses the ordinary minimum-distance recurrence and increments only when the new edge has not exhausted its source multiplicity. At a terminal, actual cost is and .
Let be the best descendant terminal at actual cost , ordered by the tuple in Eq. (15). The bottom-up recurrence is
where contains admissible outputs ending at with actual cost ; unavailable entries are omitted. The root table is .
Exactness follows by induction. The empty prefix initializes the distance column and zero overlap. Each emission preserves the exact minimum distance and capped event intersection, including repeated identities. Consequently, terminal costs and coverages equal independent whole-output measurements. Taking the minimum over each subtree preserves the best complete output at every actual cost, including all deterministic ties. Reading therefore reproduces the enumeration policy. Unlike an arbitrary alignment cost, identifies the required minimum-distance slice.
The implementation uses depth-first rollback for counts and visits every terminal. It merges common prefixes, without suffix-state compression or pruning based on predicted watermark states. Agreement on the supplied bank does not establish an efficient compact representation of all possible prose rewrites. Canonical paths are materialized before compilation; candidate generation and its upstream enumeration remain outside this solver comparison.
Construction, evaluation and reproducibility
Cohorts, configuration and information access
The confirmation contains 128 attempted generations, 105 prospectively qualified sources and 1,680 policy-dose records. Its locally processed C4-derived prompts have the lineage limitation stated in § 6; they do not establish an official C4 test-set or native attack-reproduction claim. Table 3 separates confirmation from development analyses. Table 2 specifies the final editor’s information; historical inference and stopping remain their own observation contracts.
| Object | Blind editor | Evaluator |
| Source prose, pinned tokenizer, literal alternatives | Yes | Yes |
| Full-output , , , | Yes | Yes |
| Public-model source surprisals, when enabled | Yes | Yes |
| True key, event states, source/output , | No | Yes |
| Original generation trace | No | Diagnostic only |
| Global source-relative quality judgment | After selection | Yes |
Table 2. Information boundary for the final construction. Public-model surprisals are enabled only for the labeled weighted comparator. Quality review and keyed scoring measure outcomes after selection.
All 128 source reviews were available. Among the 23 excluded sources, nine failed only minimum length, eight only paragraph completeness, and two only the inherited-language criterion. Two failed both length and completeness, one length and language, and one completeness and language. No source failed all three conditions. The criteria were , a completed paragraph and ; qualification did not use detector scores.
The source-model revision was 989aa7980e4cf806f80c7fef2b1adb7bc71aa306; the tokenizer JSON SHA-256 was c0382117ea329cdf097041132f6d735924b697924d6f6fc3945713e96ce87539. Generation and replay used the kernel in § 6. Detection retained original-CUDA partitions; CPU or MPS substitution would define another evaluator. Provenance receipts retain input hashes and plotted coordinates. The later hash-review overlay added 39 labels and preserved 66 existing labels without changing texts, scores or the primary hypothesis.
Coverage solver agreement and local efficiency
The bounded follow-up retained all 105 sources and 35,548 canonical outputs. Both solvers recomputed global cost and net event counts from token paths; cached metrics were used only for auditing. All candidate , , , , values, 1,521 attained-cost winners and 420 original allowance selections agreed. Identical output hashes preserve the existing detector and quality outcomes; this check adds no new efficacy observations.
Timing used Python 3.11.10 on arm64 macOS, one warmup and five alternating-order repetitions per source. Inputs were already loaded; canonical-tokenization checks, proposal generation and review were excluded. The reference used bit-parallel edit distance and occurrence counters. Prefix sharing reduced token-state transitions from 4,665,462 to 997,338, but added graph construction and table-management work. Table 4 distinguishes cached solving from build-inclusive execution. Ratios above one favor DP.
| Measurement | Cached graph | Build included |
| Median paired speedup | 1.090× | 0.878× |
| Sources faster than enumeration | 61/105 | 38/105 |
| Sum of source median times (s) | 1.348 | 1.868 |
| Aggregate speedup | 1.678× | 1.211× |
Table 4. Coverage-DP efficiency on the materialized bank. Aggregate speedup uses sums of per-source median times; it differs from a median paired ratio.
Enumeration’s summed source-median time was 2.262 seconds. DP reduced aggregate workload time while slowing the median source when construction was included. These conditional measurements differ from the full local pipeline timing in § 5. On eight prospectively size-stratified sources, median incremental Python allocation peaks were 48.6 KiB for enumeration, 69.9 KiB for cached DP solving and 918.2 KiB for build-plus-DP; the latter peaked at 43.45 MiB. Cached solving excludes retained graph memory, and these allocations are not process RSS. Runtime repetitions characterize this local implementation, without increasing the statistical sample for evasion claims.
Candidate nomination and mechanical admission
The fixed nomination rule split each source at punctuation and retained units with at least six words. A unit longer than eighteen words was clipped to its first twelve words. If more than eight units remained, spatially distributed quantiles were selected in source order. The resulting disjoint character spans need not be complete syntactic clauses; trailing punctuation stayed in the source. Although historical metadata included density ranks, the standalone constructor reproduced its requests with zero-valued source features.
Admission preserved named lexical anchor counts, quantities, written numbers, modal and negation words, temporal markers and bracket counts. It rejected new non-ASCII letters and nonordinary control characters and required a changed replacement. The single-site complete output needed global token distance between one and eight, net length change at most , at most sixty-four replacement tokens and exact encode/decode round-trip. Missing or malformed responses remained failures without automatic retries. Global semantic preservation was measured separately after composition and selection.
For sites with admitted alternatives each, the literal language is
Here denotes the source path together with its prose. Complete compositions undergo the canonical round-trip and resource checks in Eq. (13), even when their individual alternatives passed admission. This captures retokenization at replacement boundaries.
Probability comparator and constant-weight control
The weighted comparator used the same language and actual-cost slices. Let be public-model source-token surprisal and . Its source-event utility was
Absent-source identities received zero utility. The rule minimized
at , followed by site-count, candidate-ID and text-hash ties. This rank utility has no established calibration as posterior centered evidence; zero utility does not infer a red state. Its normalization, aggregation and ties also differ from coverage, so the primary comparison changes complete selection policies.
The retrospective constant-weight control isolated the source weights while preserving that denominator and tie rule. Assigning positive constant to source identities and defining gives
This counts uncapped output occurrences of source identities. Its difference from coverage can therefore reflect length, surplus occurrences and ties.
Quality measurement and exact-identity reuse
The frozen source gate and relative/polished thresholds are specified in § 6. Appendix § 15 reproduces the executed ABC-v2 instruction, including every score level. Reviewer requests contained anonymous prose pairs and no keys, states, scores or policy labels. Exact source/output identities alone permitted reuse. Unreviewed outputs remained unavailable until an explicitly recorded completion.
Ten controls tested the policy: identity, paraphrase, inherited defect, benign emphasis and small readable fluency damage were accepted; quantity, negation, role, modality and causation changes were rejected. All ten decisions matched expectation. These checks establish instruction compliance for the controls, without human agreement or independent factual validation. The absolute level adds to polished quality; all quality levels remain source-relative.
Clustered uncertainty
Confirmation intervals use 10,000 crossed prompt/key bootstrap draws with seed 130913. Each draw independently resamples the observed prompt and key labels with replacement, retaining the number of clusters on each axis. A document receives the product of its sampled prompt and key multiplicities. This is the pigeonhole resampling construction [Owen, 2007]. Weighted means use document outcomes or within-document policy differences, with the same draws across compared outcomes and doses. Draws with zero total weight are omitted and their number is recorded; they are not replaced. The 2.5th and 97.5th percentiles of the remaining draws give pointwise 95% intervals. All 10,000 draws were nonempty for the displayed confirmation dose curves. The ordinary-rewriting comparison uses its own clusters and effective draw counts (Appendix § 12.§ 3). Sixteen prompt and eight key clusters limit confirmation precision; candidates, repeated events and doses do not enlarge the independent sample.
Finite-bank and follow-up protocols
The retrospective audit enumerated 37,036 configurations and retained 35,548 eligible unique outputs, reproducing canonical IDs, costs, incidence, hashes and all 1,680 original endpoints. Blind controls and structural subgroups were sealed before additional actual-key scoring. Original-CUDA scoring loaded no language model. Previously examined sources retain exploratory status.
At each original actual-cost slice, uniform choice was its exact arithmetic expectation over unique outputs, and the evaluator oracle minimized realized . Singleton or all-tied slices received tie-aware percentile 0.5. Choice-rich slices had at least five outputs; variable- slices were specified before scoring. At-most coverage maximized across all costs under the cap, using lower for coverage ties. Source summaries preceded crossed prompt/key bootstrap inference; candidate counts did not enlarge the sample.
Minimum-cost witness review and the separate stopping protocol appear in Appendix § 14. Those analyses reused exposed outcomes; the stopping fits excluded held-out prompt and key groups and used other-key training scores. No quality labels or test-key states entered their selection.
Standalone replay and resource scope
A standalone worker with the pinned Rust tokenizer replayed sanitized source text and recorded proposal responses. It reproduced all 1,018 request bodies, 803 admitted patches and 420 coverage outputs on 105 sources, with zero archived-run reads, heavy-model imports, network calls or loaded model weights. Five timing passes supplied 2,100 identical output checks, without new efficacy observations. Tokenizer loading took 91.949 ms; selection median and 95th percentile were 8.917 and 335.475 ms. Absolute peak RSS was 108.2 MiB, without subtracting interpreter memory. These one-host measurements exclude provider proposals and review inference. Comparative end-to-end speed remains unmeasured.
Complete outcomes, controls and measured examples
Outcome definitions and dose summaries
For policy output at allowance , let contain qualified sources with and let . Define
For relative and polished quality indicators and , joint rates and continuous suppression are
Quality failure leaves the denominator unchanged. For the same selected outputs, ; this does not bound other candidates’ quality-qualified success. Negative reductions remain in . Mean actual cost is an operating point under the cap, not a shared individual cost. Table 5 uses the same 82 positives for both coordinates.
| Policy | Cap | Cost | Raw | Relative | Polished | 95% CI | |
| Coverage | 5% | 4.33% | 13 | 13 | 13 | 0.436 | [0.293, 0.581] |
| 10% | 9.31% | 25 | 24 | 22 | 0.930 | [0.716, 1.142] | |
| 15% | 13.41% | 39 | 37 | 32 | 1.255 | [1.012, 1.515] | |
| 20% | 15.96% | 40 | 39 | 34 | 1.373 | [1.089, 1.660] | |
| Weighted | 5% | 4.33% | 16 | 16 | 15 | 0.496 | [0.360, 0.634] |
| 10% | 9.31% | 29 | 27 | 24 | 0.977 | [0.791, 1.171] | |
| 15% | 13.41% | 39 | 37 | 31 | 1.228 | [0.986, 1.465] | |
| 20% | 15.96% | 40 | 39 | 34 | 1.359 | [1.066, 1.649] |
Table 5. Dose outcomes on 82 initially detected qualified sources. Cost is mean actual ; escape columns are strict counts out of 82. Intervals are pointwise crossed prompt/key 95% intervals for mean (16/eight clusters). Four allowances use separately selected, nonnested outputs. Coverage includes retrospective completion of its 25 missing crossing reviews.
All 420 weighted outputs have reviews. Coverage’s 25 formerly unreviewed crossings comprised 3, 10 and 12 distinct pairs at the three lower allowances. The frozen pairs received the unchanged Sol-medium ABC-v2 rubric after the ten controls passed: all 25 passed relative quality and 24 were polished. Existing labels, texts and scores were unchanged. Only exact identities gained labels. Every coverage crossing is now reviewed; some noncrossing lower-dose outputs remain unreviewed, leaving overall quality-pass rates incomplete.
The original all-105 weighted mean costs were 4.12/9.20/13.38/16.04%, with mean = 0.441/0.930/1.184/1.333. These include 23 initially nonpositive sources. At p20, mean cost was 14.50 tokens; counting excluded initially positive attempts as operational nonsuccesses gives 39/91, compared with 39/82 qualified positives. Coverage and weighted successes overlap on 38 sources, with coverage-only s088 and weighted-only s062. Their success-conditioned mean costs are 17.31% and 17.32%.
From p15 to p20, coverage gained five qualified escapes and lost three; weighted gained four and lost two. Scores worsened for 16/82 sources under each policy and for 21/105 qualified sources under weighted selection. The p20 positive-subgroup actual-cost interquartile range was 13.93–19.22%, with mean 15.96%. Neither an aggregate curve nor its mean cost establishes monotonic improvement or a common realized edit count.
Selection controls and finite-bank opportunity
Table 6 preserves the complete largest-dose outcomes. The weighted 39/82 rate has crossed 95% interval [27.8%, 67.3%]; the corresponding coverage interval is [28.6%, 66.7%]. Among the 82 positive sources, 26 attained-cost menus contained one unique output, 22 varied in destroyed-event count, and six contained both crossing and noncrossing outputs. These counts describe the available selection opportunity; they do not establish equivalence between selectors.
Table 6: Fully reviewed 20%-allowance selections on the same candidate bank and identical actual-cost slices. Mean and relative-quality passes use all 105 qualified sources; escape columns use the 82 initially positive qualified sources. Joint means strict and relative quality. Polished joint additionally requires . The hash quality column includes retrospective completion of 39 labels.
| Policy | Mean | Relative /105 | Raw /82 | Joint /82 | Polished /82 |
| Weighted | 1.333 | 101 | 40 | 39 | 34 |
| Coverage | 1.313 | 100 | 40 | 39 | 34 |
| Fixed hash | 1.305 | 102 | 38 | 37 | 32 |
| 3B weighting | 1.334 | 101 | 39 | 38 | 33 |
Table 6.
The primary complete-rule comparison is in § 7. Table 7 separates the retrospective weight ablation, uniform-choice comparison and actual-key oracle. They use the fixed protocol in Appendix § 11 and the same attained-cost slices.
Table 7: Retrospective mean extra with pointwise crossed 95% intervals across 105 source-paired observations. Uniform is exact within-source expectation over unique outputs; the minimum- oracle uses hidden states. Candidate rows and doses do not increase the independent source count.
| Allowance | Weighted − constant | Coverage − uniform | Oracle − coverage |
| 5% | 0.058 [−0.014, 0.144] | 0.019 [−0.065, 0.103] | 0.288 [0.183, 0.410] |
| 10% | 0.063 [−0.027, 0.168] | 0.056 [−0.028, 0.151] | 0.365 [0.253, 0.488] |
| 15% | −0.0003 [−0.091, 0.082] | 0.087 [0.004, 0.175] | 0.191 [0.107, 0.311] |
| 20% | 0.013 [−0.054, 0.087] | 0.019 [−0.035, 0.072] | 0.139 [0.077, 0.223] |
Table 7.
At p20, constant-minus-coverage added 0.0066 [−0.0039, 0.0340]; together with the weight contrast it reproduces the original 0.0196 policy difference. Constant weights also gave 40 raw and 39 reviewed joint escapes. Coverage-minus-fixed-hash gain was 0.0086 [−0.0655, 0.0876], with joint difference +2.44 percentage points [−5.13, +10.98]. The hash control uses one deterministic ordering. These secondary comparisons establish neither equivalence nor separate effects of the shared proposal, admission and dose stages.
Restricted candidate choice limits what these comparisons can identify. The p15 pointwise gain in Table 7 is exploratory among several doses; small choice-rich subgroups do not replace the primary null. The technical companion retains menu sizes, selection turnover, uniform-choice percentiles and subgroup intervals.
Let . For the realized key, nested feasibility gives
Identity ensures nonempty feasibility. Same-cost oracle raw counts were 23/40/47/42 of 82 across doses, while all-cost counts were 26/49/53/56. Only all-cost capacity must be nondecreasing. No quality pass is imputed for unreviewed oracle outputs. Blind at-most coverage gave 41/82 raw escapes, mean versus 14.50, and extra mean [−0.0328, 0.1232]. Its current qualified count remains 36–40: one known failure and four exact outputs unreviewed even after later completions.
Objective correction and ordinary-rewriting comparison
The all-cost reset rule minimized with identity fallback in the same 105-source bank. At p20 its extra mean over at-most coverage was 0.007025 [−0.034336, 0.061242], whose upper endpoint is below the tested 0.1-Z reference. This retrospective null concerns that objective and bank, without ruling out other estimators.
The ordinary-rewriting pilot retained 16 exposed development sources and the frozen weighted reference with its sealed proposal-depth repair. Both arms had mean actual and cost 17.222%. Fourteen pairs met the pre-existing combined dose/quality gate; all 16 remained in analysis. All selected outputs passed relative review. The reference used eleven cached Sol and five new Astra judgments, and the baseline used sixteen new Astra judgments under the same rubric. This was neither common-reviewer nor human confirmation.
The baseline requested sixteen independent literal-edit plans per source from Luna at low reasoning, capped at eight self-nominated disjoint edits and 1,024 output tokens per call. It assembled coherent plans through sixteen fixed hash-priority greedy restarts per cap. Selection used actual cost and output hashes, with no , source probabilities, quality labels or scores. Both arms used the largest positive actual cost available to both under the allowance, returning identity and retaining the source if none existed. Canonical round-trip, global cost, ordinary formatting and 5% length constraints applied, without the reference’s specialized factual-anchor guard. Of 256 primary baseline calls, 207 were valid and 49 failed; the reference used 198 historical/repair calls. Matching final cost did not match supply or inference expenditure.
The extra mean was 0.010634 [−0.486268, 0.508954] over ten prompt/eight key clusters. Each arm escaped on 7/10 positives: six shared, one unique to each and two under neither. Mean margin difference was 0.102729 = 0.250000 − 0.187500 + 0.040229, separating destroyed, created and length contributions. Only sealed primary matched outputs were scored. Of 10,000 bootstrap draws, 9,997 mean-effect and 9,965 positive-crossing resamples were nonempty. Unequal supply, mixed reviewers, source reuse and the wide interval leave superiority and equivalence unresolved; this was not a native SIRA/BIRA comparison.
Signed conversion, initial headroom and repair
For the frozen weighted outputs at the largest dose, the mean decomposition across all 105 qualified sources is
The 2,230 destroyed occurrences had pooled green rate 78.92%, and the 2,197 created occurrences had rate 49.02%. These correlated occurrences are descriptive evaluator evidence, not independent inferential units.
The main witnesses illustrate why source strength alone cannot predict a rewrite. For analyst reference , their reset residual is
Table 8 retains both source-yield and created-state departures. These are different-source, retrospectively chosen illustrations, not a matched causal comparison or blind prediction. Both reset-reference scores remain above threshold, while the realized signed effects separate escape from detection. Figure 3 gives their complete margin accounting; its numerical inputs are also retained in the technical companion.
| Quantity | s029 | s001 |
| Source/output scored events , | 71, 69 | 132, 128 |
| Source centered evidence | 26.5 | 33 |
| Destroyed-event count | 21 | 32 |
| Homogeneous source reference | 7.838 | 8.000 |
| Selected-source excess | +0.662 | −2.000 |
| Created contribution | +1.500 | −3.000 |
| Actual evidence reduction | 10.000 | 3.000 |
| Final under reset reference | 4.493 | 4.419 |
| Measured final | 3.973 | 5.303 |
Table 8. Evaluator reset diagnostics for the two main-text witnesses. Both complete outputs passed relative and polished review; source, cost and key state differ. The homogeneous reference uses actual source evidence.
The first output replaced 37.74% of centered source evidence against a 37.31% requirement; the second replaced 9.09% against 31.43%. Figure 6 retains all sources, including negative gains. The budget floor excluded 20/82 at the allowed 5% dose and 22 at its attained cost, and none at the three larger doses. Its median was 3.5 edits; median attained p20 cost exceeded the individual floor by twelve. Passing this necessary bound was therefore permissive at larger allowances.

Figure 6. Evaluator-side variation in frozen weighted p20 selections. Panel (a) uses all 105 qualified sources and distinguishes raw strict escape from other outcomes; destroyed-event count does not uniquely determine signed reduction. Panel (b) uses 82 initially positive sources and places achieved against required headroom . Strict escapes lie above the dashed equality line: 39 passed relative quality and one did not. Blue circles remained at or above four. Measured evaluator scores are not editor inputs. All observations, including negative gains, are retained; event occurrences and candidate outputs are not independent inference units.
Among the 39 weighted joint successes, the two-sided certificate in Appendix § 10.§ 3 certified one additional token edit for 32, two for 18 and three for seven. Seven had radius zero, including the opening example. Median successful was 3.151; 29 were below 3.5 and fifteen below three. These evaluator-side certificates concern detector decisions, without semantic guarantees.
Quality examples and the grounding boundary
One raw escape changed “decision on dropping charges” to “decision to drop charges.” The reviewer marked introduced risk: an unresolved question could become an apparently settled decision. Its score fell 5.683 → 3.674, but it failed joint quality. The accepted complete witness below includes “exposed grave misconduct” becoming “revealed serious misconduct.” These are complete-output judgments; individual displayed phrases have no independent quality or causal score attribution.
The grounding pilot used context-bearing records from Dolly [4]. Our generated-source cohort yielded zero grounded sources, leaving grounded-source-conditioned escape undefined. Its historical Q3 labels are separate from confirmation ABC-v2. The technical companion retains attempt counts and detector outcomes; these do not establish transfer to grounded-source rewriting.
Complete source and constructed output
Both coverage and weighted p20 selected the following s029 output. Bold intervals are the three source-character replacements, including retained context. Global cost was for , with , , .
Source, .
After two weeks following the establishment of a Special Committee tasked with investigating violent and unruly events in the Canadian parliament in November last month, their comprehensive report exposed grave misconduct that deeply disturbed the democratic process, highlighting instances of intimidation, disorderly conduct, and blatant disregard for protocol, thereby prompting calls for enhanced regulations to ensure parliamentary conduct adheres to high ethical standards.
Output, .
Two weeks after the establishment of a Special Committee tasked with investigating violent and unruly events in the Canadian parliament in November last month, their comprehensive report revealed serious misconduct that profoundly disrupted the democratic process, highlighting instances of intimidation, disorderly conduct, and blatant disregard for protocol, thus prompting demands for stronger regulations to ensure that parliamentary conduct meets high ethical standards.
The reviewer passed relative and polished quality. The inherited awkward date expression remains; historical factual accuracy was unmeasured. Terminal margin was approximately , giving sufficient repair radius zero despite the larger score reduction.
Initial-strength estimation and information limits
Historical cohort and approximate estimator
The 95-source cohort used original-CUDA generation and scoring over twelve prompt/eight key clusters. It first supplied an independent supply-expansion replication and was subsequently reused for strength estimation, allocation and trace ablation. Those later analyses are exposed-cohort diagnostics, separate from the 105 qualified confirmation sources. The planned 96th cell, tc4_p8_k7, was excluded during site preparation because the saved generation had retokenization_exact=false. The pre-existing canonical check preceded strength estimation; exclusion did not depend on its error.
Starting-strength inference used stored generation-time pre-bias logits and known sampling processors. Context-row inference retained 400 scenarios with a base importance-sampling batch parameter of 3,000. Difficult rows used 1,500 auxiliary prior draws, further base-sized proposal batches and, when needed, a Gibbs-informed batch of 6,000. MCMC or prior fallbacks handled unresolved rows, so 3,000 was not a fixed total sample count per row. Stored logits were truncated and the partition prior was approximate. The implementation formed centered source estimate and stored for . Rechecking all 95 saved rows reproduced this arithmetic exactly, without regenerating detector states.
The main constructor uses neither these observations nor the estimator. An earlier 20-document source-state study and the historical nomination, supply and prefix diagnostics are retained in the technical companion with their separate access and quality limitations.
Aggregate association and uncertainty
The measured correlation was 0.792855, mean estimate-minus-measurement error 0.089280, and population error standard deviation 0.769175. The crossed prompt/key interval for mean error was . An in-sample regression on an intercept and had ; adding gave 0.629916, with partial correlation 0.518891 after length residualization. Figure 4 displays these unchanged source rows. This descriptive adjustment establishes neither unbiasedness nor held-out calibration, plaintext inference or improved allocation beyond length.
For the ideal conditional model, source-count vector and covariance give
Weak enough dependence can yield score uncertainty when source evidence variance is , while stable positive enrichment gives mean score proportional to . Repeated identities and partition conditioning affect covariance; a diagonal approximation is not a general lower bound. The exchangeable working prior differs from the active finite modulo-reseed implementation. Thus Eq. (11) defines an ideal mean but only an operational approximation for the implemented estimator. The aggregate and candidate-contrast directions in Appendix § 10.§ 4 explain why this association does not validate a replacement ranker.
Allocation control and interpretation
To distinguish strength-informed allocation from concentration on short sources, the decisive control ordered saved 30%-dose outputs under the recorded cap: length-first and estimate-first each yielded 32 crossings, versus 33 for triage at slightly different expenditures. Random concentration averaged about 20.8. These outcomes identify concentration and length as alternative explanations and leave added Z0-based allocation value unresolved. The historical reviewer’s identity gate failed, so these are raw descriptive outcomes, not quality-qualified or uniform-20% confirmation rates. The technical companion (§ 3) retains the preliminary allocation sequence, expenditure differences, rules and prefix plots. A temporary low prefix does not imply terminal escape, and full-trace conditional prefix estimates are not online forecasts.
Reconstructed same-bank source-evidence selection
The historical estimates did not match the confirmation sources. A subsequent repair replayed preserved prompts and generated token paths using the pinned float32 CUDA generator. On 238 archived historical rows, replay reproduced all stored top-1500 logit values exactly and the top-20 sets agreed. Closing-source replays were repeatable, but their original logits were not archived, so direct historical equality was untestable there. This diagnostic requires generator access beyond the confirmed constructor’s information contract.
The historical approximate inference procedure retained 400 scenarios, with base importance batch parameter 3,000. It resolved 104 sources; one canonical versus generated token-path mismatch triggered coverage fallback. All 7,855 context records had positive importance-sampling effective size; the minimum was 173.20. These computational checks do not establish posterior calibration. The independent-context approximation and historical observation handling remain.
Before reading keyed outcomes, the selector maximized at each coverage reference’s exact actual cost, retaining coverage’s deterministic ties. Two secondary controls also fixed output length, or fixed , , , . The candidate bank and enumeration solver stayed unchanged; the tested utility estimated preserved source evidence only. The primary was the resolved-source contrast at the 20% allowance.
| Comparison | Extra | 95% interval | |
| Resolved, primary | 104 | 0.0314 | [−0.0372, 0.0988] |
| Operational, with fallback | 105 | 0.0311 | [−0.0369, 0.0981] |
| Initially detected | 82 | 0.0070 | [−0.0672, 0.0667] |
| Same output length | 104 | 0.0117 | [−0.0603, 0.0728] |
| Same | 104 | 0.0209 | [−0.0269, 0.0772] |
Table 9. Reconstructed evidence selection minus coverage at matched actual cost. Intervals are crossed prompt/key 95% intervals; controls are secondary.
The primary changed 22 outputs: 13 improved , six worsened it and three tied. Both policies produced 40 raw crossings among 82 positives, with different crossing identities. The inferred selector had 37 reviewed relative-quality passes, one failure and two unreviewed crossings, giving 37–39 possible qualified crossings against coverage’s 39. Missing reviews remain unknown. The intervals use 10,000 crossed replicates over sixteen prompt/eight key clusters and condition on the sealed estimates; they exclude inference-resampling uncertainty.
The exact extra mean margin reduction decomposed as : destroyed-source evidence, favorable created evidence and the length boundary. Every component’s interval included zero. The source-only selector did not model created-state effects. A descriptive reconstructed check gave correlation 0.82845, mean error −0.02027 and RMSE 0.65207 on 104 sources. This reused-cohort diagnostic and the uncertain selection increment distinguish aggregate estimation from candidate contrasts; they establish neither held-out stopping calibration nor a new quality advantage.
Reviewed minimum costs and key-blind stopping
Reviewing the minimum-cost witnesses
The retrospective minimum-cost audit used the 35,548 previously scored outputs from the 105 qualified sources. For each of the 56 initially detected sources with a crossing in the allowed bank, we identified the smallest actual token cost admitting . Eight witnesses reused passing exact-identity reviews; 48 deterministic representatives received blinded GPT-6.1 Sol-medium review under the unchanged rubric after all ten control pairs passed. The reviewer received prose and preservation instructions, without detector scores, colors or selection status. Witness discovery itself used actual-key outcomes.
All 56 witnesses passed relative quality; 51 had and five had ; all had . Their polished status describes these selected witnesses and does not determine the minimum cost of every polished alternative in the bank. The quality-constrained minimum equals the raw minimum because a passing output attains the raw lower bound. This is a finite-bank certificate under the recorded model rubric. Other oracle outputs retain their own review status, and these retrospectively discovered witnesses do not constitute an operational attack rate.
Conditional on these 56 sources, median minimum cost was 5.70%. This conditional median is distinct from the same-source mean-cost comparison on 39 coverage successes in § 7.
Frozen paths, prediction and stopping
The stopping study retained all 105 sources and the existing candidate bank, with and . One candidate at each attainable actual cost formed a coverage path, selected by maximal and the original deterministic ties. A secondary path used the existing public-model surprisal utility. Identity outputs were included. Paths, features and rules were frozen before comparison, but the bank and evaluator outcomes had already been examined. This was a retrospective reuse of existing outputs and scores.
For candidate , let count created occurrences whose event identity also occurs in the canonical source. A homogeneous-source reference predicts remaining evidence through
The length model S0 fitted a common enrichment , giving . Model S1 used , where is mean cached public-model source surprisal standardized using training sources. Fitting minimized source-balanced squared baseline-score error, with a fixed penalty for S1. This scalar plaintext feature differs from the historical trace-assisted estimator in § 7.4.
Four prompt groups crossed with four key groups defined 16 outer folds. Training excluded both held-out groups. The empirical 90th percentile of inner held-out source-wise maximum residuals supplied a nonnegative correction for each outer fold and path type. The rule selected the cheapest path output with , for prespecified targets . Absence of such an output returned the original coverage-p20 output as an unassured best effort. Mean-only controls set . A fixed-dose control chose the cheapest training allowance whose raw yield was within five percentage points of training p20 yield; every outer fold selected 15%.
These controllers used detector scores from other training prompt/key groups. They were test-key-blind, while the original tokenizer-only constructor required no such score training. Quality labels and test-source detection status were excluded from selection. The empirical envelope supplies no distribution-free or per-declaration success guarantee. Whole-path coverage, declaration frequency and success among declared outputs are separate measurements.
Starting-score and signed-effect errors
The prediction error can be separated exactly using evaluator-side and :
Here propagates starting-score error, while remains after replacing the estimate with the true starting score. It includes source heterogeneity, created and reused event states, and error in the homogeneous-reset reference. It is not an estimate of irreducible cryptographic noise. The components are correlated, requiring the cross term in .
Across all 105 sources, baseline RMSE was 0.792 for S0 and 0.780 for S1; S1’s mean coverage-path correction was 1.41. The empirical envelope covered the complete path for 96/105 sources and 74/82 positives; this is path coverage, distinct from correctness conditional on declaring a target attained. These small predictive differences did not produce a useful primary cost saving. A separate posthoc check averaged squared candidate errors within source before aggregation. On the 82 positive-source nonidentity coverage paths, S1’s total squared error was 0.551 = 0.334 + 0.258 − 0.041, corresponding to propagated-starting-score RMS 0.578 and exact-starting-score residual RMS 0.508. At p20 the respective RMS values were 0.520 and 0.677. The component differences have crossed intervals spanning zero; neither dominates uniformly. All 35,548 stored candidate identities satisfied Eq. (47) to numerical precision. The true-score substitution is an analyst diagnostic with no operational access claim.
Quality, slack and uncertainty of cheaper policies
Table 10 records the secondary operating points on the same 82 positive sources. The primary envelope policy selected identical text hashes to coverage-p20 for all 105 sources. Overall, three sources met the predicted-target condition and 102 used fallback. Only one positive source satisfied its predicted-target condition; the remaining 81 used fallback. Its unchanged raw yield therefore does not imply successful certification of all returned outputs. The mean-only target-3.5 result remains secondary after its raw count matched the reference. The table includes later review completion only where source/output identities match.
Table 10: Retrospective stopping outcomes on 82 initially positive sources, including exact-identity coverage-review completion. Cost is mean actual . Quality ranges bound qualified crossings by treating unreviewed crossings as failing or passing; they are not confidence intervals. The last column counts raw outputs satisfying the sufficient any-one-token certificate described below.
| Policy | Cost (%) | Qualified | Certificate | ||
| Coverage-p20 | 15.96 | 40 | 39 | 30 | 34 |
| S1 envelope, target 4 | 15.96 | 40 | 39 | 30 | 34 |
| Training-selected fixed dose | 13.41 | 39 | 37 | 23 | 26 |
| S1 mean, target 4 | 11.55 | 25 | 15–24 | 15 | 18 |
| S1 mean, target 3.5 | 14.35 | 40 | 28–39 | 24 | 27 |
Table 10.
The mean-only rules still have nine and eleven unreviewed crossing outputs; their ranges are missing-outcome bounds. The fixed 15% dose has 37 qualified crossings, including 32 polished, at lower cost but with two fewer qualified crossings and fewer certified outputs than p20. These descriptive differences establish neither noninferiority nor reliable per-source stopping. The technical companion retains review-completion and fitting counts. Exact joins changed no text, score, cost or certificate.
For this historical detector, and sufficiently certify that any additional canonical-token edit of cost at most one retains strict escape. This follows from the maximum two-unit centered-evidence increase and worst-case one-event contraction. The certificate concerns the detector decision; a word-level repair can cost multiple tokens and its quality requires separate assessment. Failure of this sufficient condition does not establish instability. Observed capacity at a stronger realized-score threshold likewise supplies no bound on all possible rules that might confidently declare .
This evidence-plus-contraction certificate yields the 34 coverage/raw-positive outputs reported in Table 10. The main-text count of 32/39 instead uses the two-sided margin bound on weighted, relative-quality successes in its domain (Appendix § 12.§ 4). The policies, populations and sufficient conditions differ, so the two counts are not competing estimates of one rate.
Comparisons used 10,000 crossed prompt/key bootstrap replicates with seed 130913 and source-level pairing. There were 16 prompt and eight key clusters; intervals condition on the fitted retrospective exercise and omit refitting and reviewer uncertainty. Repeated reuse of this cohort limits generalization despite the held-out fitting design. The study ended at its prespecified gate without establishing a cheaper policy preserving crossing, reviewed quality and robustness. The original confirmation result remains unchanged.
Executed prompts and service provenance
Proposal and review services
The confirmation used the OpenAI Responses API with the exact requested and returned model identifiers in Table 11. Each request set store=false and service_tier=default. Automatic retries were disabled. The saved response records contain no immutable model-snapshot identifier or system fingerprint, so the recorded aliases and execution dates do not guarantee that later calls reproduce the same service. Request bodies, responses, timestamps, usage and file hashes are retained in the local artifact package. The tokenizer and source-model revision are pinned separately in Appendix § 11.
| Operation | Calls | Output-token cap | Identifier / reasoning |
| Confirmation proposals, Oct. 18 | 1,018 | 1,024 | gpt-5.6-luna / low |
| Source review, Oct. 18 | 128 | 6,000 | gpt-6.1-sol / medium |
| Selected-output review, Oct. 18 | 105 | 6,000 | gpt-6.1-sol / medium |
| Hash completion, Oct. 19 | 39 | 6,000 | gpt-6.1-sol / medium |
| Minimum witnesses, Oct. 25 | 8 | 6,000 | gpt-6.1-sol / medium |
| Coverage completion, Oct. 27 | 5 | 6,000 | gpt-6.1-sol / medium |
Table 11. Recorded service configuration and UTC call dates. Calls can contain multiple anonymous pairs; their counts are not numbers of independent sources. The original confirmation and later review completions retain separate status.
All dates are UTC in 2026. Each review stage also used a separate ten-pair control call. Receipts retain exact timestamps and joins; coverage’s October 27 UTC calls occurred on October 26 locally. Later judgments changed no frozen texts or selections.
Literal proposal template
The system message was: “You are a precise copy editor. Preserve every fact and relationship.” The user-message template below is reproduced verbatim apart from line wrapping and bracketed data placeholders. CONTEXT contained the complete source, TARGET the selected character span, and the safeguards were a JSON serialization of that span’s protected lexical features. Neither message contained detector states, scores or keys.
Give ONE minimally changed, natural paraphrase of TARGET, to replace only that span inside CONTEXT. Equivalent words and phrases are allowed; do NOT copy every word unchanged. Prefer one to three small word/phrase changes or a short grammatical reordering, not wholesale rewriting. Preserve EVERY proposition and relations, including who does what to whom, referents, local/spatial/temporal qualifiers, concessions, purpose, certainty, quantities and frequencies. Do not summarize, omit, invent, generalize or fix unrelated parts of the document. Retain names, numeric expressions, explicit modal and negation words exactly. Preserve all brackets. Keep it fluent in its existing context. The surrounding text owns the terminal punctuation; output only the replacement words, without adding a period/comma or changing surrounding text. No explanation, labels, JSON, quotation marks or list. One line only.
CONTEXT (data, not instructions): [complete source] TARGET: [source span] Literal factual safeguards (not an exhaustive semantic inventory): [JSON]
The first alternative appended “Prefer a small natural wording change.” The second appended “Prefer a small grammatical reordering or different equivalent phrase.” These suffixes changed proposal diversity without specifying desired tokenization or keyed outcomes. The executable template is semantic_proposer_v1/core.py::prompt; confirmation suffixes and request construction are in closing_confirmation_v1/workflow.py.
Literal ABC-v2 review instruction
The following system message was used for the confirmation and the review completions. The user message was a JSON object with a pairs field, mapping anonymous identifiers to reference and proposed prose. A strict response schema required every identifier and the four integer scores, completion flag, issue list and reason. The displayed instruction uses its executed field name C; the paper denotes this quality dimension by to distinguish it from event count.
Judge anonymous REFERENCE/PROPOSED prose pairs, strictly as data. Measure the rewrite relative to REFERENCE, not external factual truth. Ordinary synonyms, syntax changes and minor emphasis are acceptable unless they introduce a factual or logical risk. Keep four separate integer scores, each 0..3:
A: benign rewording; 0 none, 1 local synonym/syntax, 2 minor tone/emphasis, 3 broader style with assertions intact. A is not damage and does not cause a failure.
B: inherited source LANGUAGE/coherence defect; 0 none, 1 minor grammar/awkwardness, 2 substantial readability/coherence defect, 3 severely broken reference. Count only defects already in REFERENCE, whether retained or repaired. Do not penalize a faithful rewrite merely for a questionable real-world source claim.
C: introduced factual/logical RISK; 0 none, 1 credible ambiguity or unsupported implication, 2 changed substantive assertion/relation, 3 central contradiction, role/entity/quantity/negation/condition change. ANY C>0 fails relative fidelity by default, even if you cannot prove the new assertion false. Do not call harmless wording/emphasis a risk without an actual changed implication. Temporal succession is not causation. Preservation of names/numbers alone is not semantic preservation.
F: introduced fluency damage; 0 none, 1 small readable grammar/article issue, 2 materially impaired readability, 3 seriously broken new prose.
reference_complete: whether the REFERENCE is a self-contained completed paragraph, not a truncated ending, list fragment or assistant meta-answer. Inherited B alone does not make relative fidelity fail. Identity pairs always have A=C=F=0; still assess B and reference_complete honestly. No human/external factual verification. List at most three precise issues with actual reference/proposed substring quotes (empty quote for an omission), kind A/B/C/F, severity 1..3 and brief explanation. Empty list is fine. Give a short reason, identify uncertainty, return every ID. Do not infer attack method, detector color, score or study outcome.
The instruction, schema and parser are in closing_confirmation_v1/quality.py. Acceptance requires a completed response from the requested model, all identifiers, valid scores, verbatim issue quotations and zero introduced changes for identities. These checks establish neither human agreement nor external factual validity.
Prompt inventory
The confirmation used the first sixteen entries of the previously frozen confirmation split, after document-group separation from development. Table 12 describes the topics of these saved prefixes. This is a descriptive inventory of the controlled sample, without a claim of population-representative topic sampling. The source lineage limitation in § 6 remains unchanged. The common instruction requested one self-contained explanatory paragraph of about 110–140 words, staying on the prefix topic, with no heading, list or assistant address. The complete instruction and prefixes are retained in pilot_prompts.json.
| ID | Topic | ID | Topic |
| p000 | US healthcare costs | p008 | Biblical fathers |
| p001 | Himalayan religious community | p009 | Casting a musical |
| p002 | Church and public-policy debates | p010 | Surface properties and boiling |
| p003 | Parliamentary misconduct | p011 | Alleged attack on a football mascot |
| p004 | Social-media data security | p012 | Diplomatic criminal charges |
| p005 | Television drama adaptation | p013 | Winter television viewing |
| p006 | Air pollution and genetic damage | p014 | US charitable giving |
| p007 | Employer pension funds | p015 | Insurance-rule benefits |
Table 12. Topics of the sixteen frozen continuation prefixes, in source order. These labels describe the prompts; they do not verify their assertions.
Artifact and dual-use boundaries
The research archive at https://github.com/phunterlau/bpe_watermark retains templates, proposal/review receipts, canonical paths, selected-output identities and reconstruction checks. The technical companion is supplied as ancillary material with this article. Budgeted rewriting supports robustness assessment and can obstruct machine-text provenance. Controlled-key, model-relative evaluation supplies no human-quality guarantee, proprietary-detector transfer result or deployment validation.
References
- [1]Hongyan Chang, Hamed Hassani, and Reza Shokri. Watermark smoothing attacks against language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 4915–4941, 2025. doi: 10.18653/v1/2025.findings-emnlp.264. URL https://aclanthology.org/2025.findings-emnlp.264/.DOI
- [2]Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. De-mark: Watermark removal in large language models. In International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 9316–9333, 2025. URL https://proceedings.mlr.press/v267/chen25bbq.html.arxiv.org/abs/2410.13808
- [3]Yixin Cheng, Hongcheng Guo, Yangming Li, and Leonid Siga. Revealing weaknesses in text watermarking through self-information rewrite attacks. In International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 9982–10009, 2025. URL https://proceedings.mlr.press/v267/cheng25c.html.arxiv.org/abs/2505.05190
- [4]Databricks. Databricks dolly 15k. Dataset card, 2023. URL https://huggingface.co/datasets/databricks/databricks-dolly-15k.DOI
- [5]Sumanth Dathathri, Abigail See, Sumedh Ghaisas, et al. Scalable watermarking for identifying large language model outputs. Nature, 634:818–823, 2024. doi: 10.1038/s41586-024-08025-4. URL https://www.nature.com/articles/s41586-024-08025-4.DOI
- [6]Noah Golowich and Ankur Moitra. Edit distance robust watermarks via indexing pseudorandom codes. In Advances in Neural Information Processing Systems, volume 37, 2024. URL https://proceedings.neurips.cc/paper_files/paper/2024/hash/24c53bfa5b53fc2cf05644f5a7a26bb0-Abstract-Conference.html.arxiv.org/abs/2406.02633
- [7]Fabrice Harel-Canada, Boran Erol, Connor Choi, Jason Liu, Gary Jiarui Song, Nanyun Peng, and Amit Sahai. Sandcastles in the storm: Revisiting the (im)possibility of strong watermarking. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29698–29735, 2025. doi: 10.18653/v1/2025.acl-long.1436. URL https://aclanthology.org/2025.acl-long.1436/.
- [8]Jeongyeon Hwang, Sangdon Park, and Jungseul Ok. LLM watermark evasion via bias inversion. In International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 49078–49107, 2026. URL https://proceedings.mlr.press/v306/hwang26e.html.arxiv.org/abs/2509.23019
- [9]Nikola Jovanovic, Robin Staab, and Martin Vechev. Watermark stealing in large language models. In International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 22570–22593, 2024. URL https://proceedings.mlr.press/v235/jovanovic24a.html.arxiv.org/abs/2402.19361
- [10]John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 17061–17084, 2023. URL https://proceedings.mlr.press/v202/kirchenbauer23a.html.arxiv.org/abs/2301.10226
- [11]John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. In International Conference on Learning Representations, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/hash/d78e4316e1714fb0f20be66f8044c-Abstract-Conference.html.arxiv.org/abs/2306.04634
- [12]Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. In Advances in Neural Information Processing Systems, 2023. URL https://arxiv.org/abs/2303.13408.
- [13]Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortion-free watermarks for language models. Transactions on Machine Learning Research, 2024. URL https://arxiv.org/abs/2307.15593v3. Art B. Owen. The pigeonhole bootstrap. The Annals of Applied Statistics, 1(2):386–411, 2007. doi: 10.1214/07-AOAS122. URL https://arxiv.org/abs/0712.1111.
- [14]Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Yuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. MarkLLM: An open-source toolkit for LLM watermarking. In Proceedings of the Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 61–71, 2024. doi: 10.18653/v1/2024.emnlp-demo.7. URL https://aclanthology.org/2024.emnlp-demo.7/.DOI
- [15]Qwen Team. Qwen2.5 technical report, 2025. URL https://arxiv.org/abs/2412.15115v2. arXiv:2412.15115, version 2.
- [16]Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1–67, 2020. URL https://jmlr.org/papers/v21/20-074.html.arxiv.org/abs/1910.10683
- [17]Qilong Wu and Varun Chandrasekaran. Bypassing LLM watermarks with color-aware substitutions. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8549–8581, 2024. doi: 10.18653/v1/2024.acl-long.464. URL https://aclanthology.org/2024.acl-long.464/.DOI
- [18]Hanlin Zhang, Benjamin L. Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak. Watermarks in the sand: Impossibility of strong watermarking for language models. In International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 58851–58880, 2024. URL https://proceedings.mlr.press/v235/zhang24o.html.arxiv.org/abs/2311.04378
- [19]Zhaoxi Zhang, Xiaomei Zhang, Yanjun Zhang, He Zhang, Shirui Pan, Bo Liu, Asif Qumer Gill, and Leo Yu Zhang. Character-level perturbations disrupt LLM watermarks. In Network and Distributed System Security Symposium, 2026. doi: 10.14722/ndss.2026.230138. URL https://www.ndss-symposium.org/ndss-paper/character-level-perturbations-disrupt-llm-watermarks/.DOI

