WorksComputer Science
Conserved Verdicts, Drifting Commitments: Identifying Harmful Overthinking
Abstract
A reasoning model that has reached a correct answer can lose it by continuing to think. Deciding whether to stop means comparing two probabilities at the same point of a chain: that the answer the model would give if stopped now is correct (its commitment), and that the answer it will finish with is correct (its verdict). We show that the verdict alone, which is what final-correctness probes estimate, cannot make this comparison. As a forecast of a fixed outcome it has no predictable drift, whether continuing helps or hurts. The commitment-verdict gap equals the expected remaining change in answer quality and the accuracy gained by stopping. A second identity splits any accuracy-versus-budget curve into answer change within chains and change in which chains are still running. We measure both on MATH-500. At all 379 points where DeepSeek-R1-Distill-Qwen-1.5B holds a correct answer, selected without regard to how its chain ends, continuing loses 3.6 points on average (95% CI [2.2, 5.2]), and the loss concentrates in a few problems. In six reasoning models from three families, on MATH-500 and OlympiadBench and with budgets up to 16,384 tokens, declining accuracy curves come from which chains remain, while the chains still reasoning keep or improve their answers. An oracle stopping rule gains at most +1.8 and +2.5 points at 1.5B and 7B. The framework says what to measure before building a stopping rule.
Introduction
Reasoning models solve hard problems by thinking longer, and spending more inference-time computation is now a main way to improve them ([42, 37, 33]). Longer is not always better. A model can reach a correct answer and then talk itself out of it ([43]), and accuracy is often reported to fall at long budgets ([51]). Early-exit methods try to save tokens and keep such answers ([47, 29, 13]). All of them face one question at every point of a chain: is it better to stop now, or to keep reasoning?
The question compares two numbers at the same point. The commitment is the probability that the answer the model would give if stopped now is correct. The verdict is the probability that the answer it will end with, if it keeps reasoning, is correct. Stopping helps exactly when the commitment is the larger of the two.
Two common measurements answer different questions (Figure 1). First, probes and value models trained to predict whether a chain will end correct estimate the verdict. However accurate, the verdict cannot decide when to stop on its own. Two chains can both have a 50% chance of ending correct while their current answers are right 75% and 25% of the time. Stopping helps the first chain and hurts the second. Second, accuracy-versus-budget curves usually average over the chains still running at each budget. Easy problems tend to finish early and correctly, so the chains left at long budgets are harder ones. Accuracy among them can fall even when no chain’s answer gets worse.
Figure 1. Two comparisons determine what further reasoning buys. Illustrative numbers (C: correct; W: wrong). Left: two chains have the same 50% chance of ending correct, yet the answer available now is right 75% of the time in one and 25% in the other, so stopping helps one and hurts the other. Right: two correct chains finish early; accuracy among the chains still running falls from 75% to 50% although no answer changes. Comparing the same chains over time separates answer change from selection.
This paper. We make both comparisons exact and then measure them.
The verdict is conserved. For a fixed way of continuing, an exact forecast of the final outcome cannot predict its own next revision; it is a martingale. A stable final-correctness forecast is therefore no evidence that reasoning is harmless (Prop. 1, Cor. 2).
The gap is the value of stopping. The commitment–verdict gap equals the expected remaining change in the commitment, and equals the accuracy gained by stopping (Eq. (2), Cor. 3).
Budget curves mix two effects. An exact identity splits any change in a survivor-averaged curve into answer change within chains and change in which chains remain (Prop. 4).
These results use standard probability: the tower property, the Doob decomposition and telescoping. The commitment–verdict gap is the familiar comparison of stopping and continuing values in optimal stopping (Chow et al., [4]). What we add is which quantities answer the overthinking question, why two common practices measure something else, and the measurements that answer it. On MATH-500 (Hendrycks et al., [17]) and OlympiadBench (He et al., [15]) we find:
Losing a correct answer is real, rare, and concentrated. At all 379 points where the 1.5B model holds a correct answer, chosen without looking at how its chain ends, fresh continuations lose 3.6 points on average. In 348 of the 379 points the model never loses the answer; a few problems account for most losses. A misleading hint inserted mid-reasoning makes the stop-now answer wrong in most cases, yet continuing restores it: the commitment collapses while the verdict holds (Sec. 4.2).
Declining budget curves come from selection. In six models from three families, on two benchmarks, at several numeric precisions and with budgets up to 16,384 tokens, chains still reasoning keep or improve their answers; the decline comes from which chains remain (§ 4.3).
Early stopping has little room to gain. An oracle that stops each chain at its best checkpoint gains +1.8 points at 1.5B and +2.5 at 7B. None of the replayed early-exit rules improves accuracy at lower token cost (Sec. 4.4).
Commitment, Verdict, and Drift
Consider one reasoning chain on one problem, continued by a fixed protocol: the model keeps sampling until it answers or reaches its token budget. Let denote everything generated up to step , and whether the final answer is correct. A step can be a paragraph of reasoning or a fixed number of tokens.
Two readouts. The commitment is the probability that the answer elicited at step , by a fixed stop-and-answer procedure, is correct. The verdict is , the probability that the final answer will be correct. When the chain ends at step , the current answer is the final answer, so . The formal assumptions (a Markov reasoning state and a terminal answer read from the final state) are in App. § 12.
Drift. Split each change in the commitment into an expected part and a surprise:
The drift is positive when continuing is expected to improve the current answer and negative when it is expected to erode it. The surprise averages to zero. This split is unique (Thm. 7), so drift is a well-defined property of the chain and the elicitation procedure.
What Identifies the Value of Continuing
Proposition 1 (Conserved verdict). (i) The verdict is a martingale: for every , under any fixed continuation protocol.
(ii) If the chain ends at with (automatic under a token budget) and , then
In particular at every step if and only if the drift is zero throughout.
Proof sketch. Part (i) is the tower property. For (ii), add up the expected changes from to the end and use . App. § 12 gives the full proof.
Part (i) is why a final-correctness forecast cannot reveal harm. An accurate forecast already averages over what continuing will do, so its next revision is unpredictable whether continuing helps or hurts. Part (ii) says the gap between the two readouts is exactly the drift still to come: if continuing will on balance erode the current answer, the commitment sits above the verdict by that amount.
Corollary 2 (A verdict trajectory does not decide whether to stop). Two chains can share the same verdict at every step and still have opposite signs of .
Proof. Let the chain end at and reveal only then. Set , and . Both versions have , but stopping at step 1 gains . With this is the left panel of Figure 1. □
Corollary 3 (Stopping advantage). Let be a stopping rule that uses only , and the correctness of the answer it returns, with . Then
Proof. Apply the tower property to and , then Eq. (2) at . □
A stopping rule gains accuracy only from the states where it stops and where remaining drift is negative. Its best possible gain is the total expected loss at such states, which we measure directly in Sec. 4.2. If continuing also has a cost priced at per unit, the value of continuing is , so an efficiency-driven exit can be right even when continuing would improve accuracy. App. § 13 writes the gap as a Poisson equation for Markov reasoning states, and App. § 14 turns the drift into training objectives.
From one chain to a population. Accuracy-versus-budget curves average a correctness indicator , the correctness of the forced answer at budget , over the chains still running. Write .
Proposition 4 (Survivorship decomposition). For budgets ,
Proof. Add and subtract . □
The within-cohort term compares the same chains at two budgets: it is the rate of wrong-to-correct minus correct-to-wrong changes. The composition term is the effect of chains finishing. Only the first measures what longer reasoning does to answers.
Two measurements. The results translate into two experiments. At a fixed prefix, compare the answer elicited now with fresh continuations from the same prefix; their difference estimates . Across a population, compute the decomposition on fixed cohorts.
Experiments
Setup
The primary configurations are DeepSeek-R1-Distill-Qwen-1.5B in FP16 on 367 MATH-500 problems (levels 3–5) and DeepSeek-R1-Distill-Qwen-7B on 262 problems (levels 4–5) [6, 17]. The 7B model is loaded in 4-bit NF4 [8]; an 8-bit run [7] gives the same results (§ 4.3). Each problem gets one chain sampled at temperature 0.6 and top-p 0.95, up to 4096 tokens. Every 512 tokens, a separate branch closes the thinking block and asks for a boxed answer, decoded greedily for at most 48 tokens [33]; the chain itself continues unchanged. Answers are scored by normalized string matching and symbolic verification. A chain is early-correct/final-wrong if an answer at or before 2048 tokens is correct and its final answer is wrong. Intervals are 95% bootstrap intervals over whole chains or problems unless noted. App. § 16 gives full details, and Table 8 in App. O marks which analyses were fixed in advance in a dated plan, which are follow-ups, and which were added during review.
| Analysis | Specified | Reported in |
| Early-correct/final-wrong prevalence (Gate 1a), 7B escalation, predictable commitment drift, stopping-rule replays | before results | Secs. 4.2, 4.4; App. § 16 |
| Eight-sample resampling, temperature 0.9, 256-token grid, greedy vs. sampled forcing, Qwen3-1.7B, self-consistency | follow-up, before its run | § 4.3; App. § 16.§ 1 |
| R1-Distill-Llama-8B and Phi-4-mini-reasoning audits | follow-up, before its run | Table 1 |
| Survivorship decomposition; matched-prefix continuations; probe refits; cumulative-history ablation | follow-up after primary results; continuation and refit designs fixed before their runs | Secs. 4.2–§ 4.3; Apps. § 18, § 19 |
| Outcome-blind continuations, generic doubt, misleading hint, 16k budget, 1.5B in NF4, 7B in 8-bit, OlympiadBench, Qwen3.5-2B, problem concentration, difficulty match, oracle headroom | during review, before its run | Secs. 4.2–4.4; this appendix |
Table 8. When each analysis was specified.
How often does continuing lose a correct answer?
Outcome-blind continuations. We take every 1.5B chain that has a correct answer at a checkpoint at or before 2048 tokens, at its first such checkpoint. This selects 379 points across all five MATH levels, and the rule never looks at how a chain ends. From each point we sample four fresh continuations with the original prefix, temperature and remaining budget, and score them as in the primary runs. The immediate answer is correct at every point, so the drop in accuracy from continuing estimates .
Continuing loses 3.6 points on average (95% CI [2.2, 5.2]; levels 3–5: 4.2, [2.5, 6.2]). The loss is concentrated (Figure 2, left): in 348 of 379 points all four continuations keep the answer, while 13 points (3.4%) lose it in at least two of four. This is in line with the realized rate of early-correct/final-wrong chains: eight fresh chains for each of 367 problems give 100/2936 such chains (3.4%, problem-clustered CI [2.4%, 4.5%]).
Figure 2. Continuing rarely loses a correct answer, and the losses concentrate. Left: at each of the 379 points where the 1.5B model holds a correct answer, how many of four fresh continuations keep it. Right: continuation accuracy from the 12 prefixes whose original chains ended wrong, and from all other points. Gray: the eight-draw matched study; blue: four new draws per point. Bars show 95% intervals.
Losses sit in particular problems. The 12 points whose original chains ended wrong keep their answer in only 47.9% of new continuations, against 98.0% for all other points; an earlier eight-draw matched study gave 50.0% against 97.9% (Figure 2, right; App. § 18). The repeated-sample data show the same concentration at the level of problems. If every problem had the same transition rate, the number of early-correct/final-wrong chains per problem would be binomial. The variance is 2.6 times larger, and 5 problems show the transition in at least five of eight fresh chains, where the binomial predicts none. The ten most affected problems produce 40% of all such chains.
Measurement choices. The count of early-correct/final-wrong chains depends on how often answers are elicited. On a matched set of 240 chains it is 5, 5 and 7 at 1024-, 512- and 256-token spacing, so we report rates at a fixed spacing. The forcing string matters little: where a truncated chain also answers without it, the two answers agree on correctness in 85.7% of cases, and greedy versus sampled forcing changes accuracy by +0.7 points ([−0.3, +1.8]). Across final-answer definitions and at temperature 0.9 the rate stays between 1% and 4% (App. § 16).
A misleading hint separates the two readouts. To push the model toward abandoning a correct answer, we insert a wrong answer into the reasoning at 90 of these points: “Wait, I made an error earlier. Rechecking carefully, the correct answer should be ,” where is a wrong answer the model itself gave on that problem, or the integer next to the gold answer. The hint moves the commitment sharply: an answer forced right after it is correct in only 32.2% of cases and equals in 64.4%. The verdict barely moves: continuations end correct in 93.6% of cases, against 94.4% at the same points without the hint (paired drop 0.8 points, [−1.7, 3.6]), and end at in only 5.0%. Here the right decision is to keep reasoning. A rule that reads only the current answer would stop at the worst moment, and a final-correctness forecast would not register the abandonment at all (App. O).
Does the declining budget curve reflect worse answers?
Both primary configurations show the familiar decline: accuracy among unfinished chains falls after about 2000 tokens (Figure 3, gray). Applying Prop. 4 over the four 512-token intervals from 2048 to 4096 tokens separates its sources. The chains still reasoning improve, by +8.5 points at 1.5B ([+2.1, +14.5]) and +20.0 at 7B ([+12.6, +27.9]). The decline comes from composition: −16.8 and −24.4 points, as chains that finish early are disproportionately correct. Fixed cohorts followed across all budgets are flat or rising (Figure 3, colored).
Figure 3. Answers improve beneath a declining accuracy curve. R1-Distill-Qwen 1.5B, levels 3–5 (left), and 7B, levels 4–5 (right). Gray: accuracy among chains still running at each budget. Colored: fixed groups of chains followed across budgets; their final-step changes are −0.7 [−4.0, +2.6] () and +2.1 [−1.0, +6.2] () points.
Across models and settings. Table 1 repeats the decomposition for four more reasoning models (Qwen3-1.7B, R1-Distill-Llama-3.1-8B, Phi-4-mini-reasoning and the 2026 Qwen3.5-2B ([34] with thinking enabled), on OlympiadBench, at other precisions, and with a longer budget. In every configuration the within-cohort term is positive and the composition term negative; wherever survivor accuracy falls, composition is the cause. Holding difficulty fixed separates the two primary configurations: at levels 4–5 the 1.5B model shows the same early-correct/final-wrong rate as the 7B model (9/262 = 3.4% against 9/262). Precision does not move the rate: 3.1% for the 1.5B model in NF4, and 3.1% ([1.6, 5.9]) for the 7B model in 8-bit. The rate is 3.8% ([2.3, 6.3]) on OlympiadBench, where final accuracy is only 48.8%, and 1.9% ([0.9, 3.9]) for Qwen3.5-2B. With a 16,384-token budget, the share of chains stopped by the cap falls from 44% to 11%, and the rate is 3.3% ([1.9, 5.6]) against 2.5% on the same chains truncated at 4096 tokens. Majority-vote curves show the same structure (within-cohort +12.0 points, composition −12.0; App. H.1).
| Model | Setting | Survivor change | Within-cohort | Composition | |
| R1-Qwen 1.5B | FP16, levels 3–5 | 367 | −8.3 | +8.5 [+2.1, +14.5] | −16.8 |
| R1-Qwen 1.5B | FP16, levels 4–5 | 262 | −6.3 | +7.3 [+0.1, +14.2] | −13.5 |
| R1-Qwen 1.5B | NF4, levels 4–5 | 262 | −2.2 | +15.0 [+8.4, +22.1] | −17.2 |
| R1-Qwen 7B | NF4, levels 4–5 | 262 | −4.4 | +20.0 [+12.6, +27.9] | −24.4 |
| R1-Qwen 7B | 8-bit, levels 4–5 | 262 | −4.1 | +17.9 [+10.0, +26.0] | −22.0 |
| Qwen3 1.7B | FP16, levels 3–5 | 367 | −2.6 | +7.6 [+1.6, +13.7] | −10.2 |
| R1-Llama 8B | NF4, levels 3–5 | 367 | −9.3 | +16.6 [+9.3, +24.1] | −25.9 |
| Phi-4-mini 3.8B | FP16, levels 3–5 | 367 | −7.5 | +9.1 [+1.8, +16.6] | −16.6 |
| Qwen3.5 2B | FP16, levels 3–5 | 367 | +7.1 | +8.7 [+5.6, +11.9] | −1.7 |
| R1-Qwen 1.5B | FP16, OlympiadBench | 367 | −4.4 | +7.3 [+3.1, +11.8] | −11.7 |
| R1-Qwen 1.5B | FP16, 16k cap† | 367 | −28.6 | +7.9 [−4.4, +21.8] | −36.5 |
Table 1. The same pattern across models, benchmarks, difficulty, precision and budget. MATH500 unless marked. Accuracy changes in percentage points, summed over 512-token intervals from 2048 to 4096 tokens (†from 4096 to 16,384 tokens); brackets are 95% chain-bootstrap intervals.
Accuracy by natural length. A forcing-free check gives the same picture. Sorting chains by how long they ran, accuracy falls by −9.1 (1.5B) and −8.6 (7B) points per 512 tokens. Among chains that ended on their own, the slopes shrink to −2.4 ([−4.5, −0.6]) and −1.1 ([−2.8, +0.3]); the longest bins are filled with chains stopped by the token cap (41% and 37% of all chains).
How much can early stopping gain?
Corollary 3 bounds what any stopping rule can gain by the expected loss at the states where it stops. We replay stopping rules on held-out chains, using only information available at each checkpoint, and charge each forced-answer branch 48 tokens (Table 2). An oracle that knows the correct answer and stops each chain at its best checkpoint gains +1.8 points at 1.5B ([+0.0, +4.5]) and +2.5 at 7B ([+0.0, +6.2]) while saving 39% of tokens. No real rule can do better, so in this regime early stopping is worth about two points of accuracy at most, and its practical value is the saved tokens.
| Policy | R1-Distill-Qwen 1.5B Accuracy | R1-Distill-Qwen 1.5B Tokens | R1-Distill-Qwen 7B Accuracy | R1-Distill-Qwen 7B Tokens |
| Full budget | 77.3 | 2808 | 77.5 | 3015 |
| Oracle stopping (gold labels) | 79.1 | 1713 | 80.0 | 1836 |
| Drift alarm (ours) | 74.5 | 2624 | 73.8 | 2767 |
| Checkpoint vote | 71.8 | 3087 | 65.0 | 3315 |
| Answer agreement | 65.5 | 1625 | 61.3 | 1535 |
Table 2. Stopping replays on held-out chains: accuracy (%) and mean charged tokens. There are 110/80 evaluation chains at 1.5B/7B. App. N reports all 48 configurations from the published method families.
None of the 44 implementable configurations at 1.5B (21 at 7B), which include fixed truncation and the answer-convergence, self-consistency and adaptive-sampling rules from the literature, improves accuracy at lower charged token cost. Rules that save tokens change accuracy by between −0.9 and −45.5 points at 1.5B. Our drift alarm, which stops when a learned critic predicts accumulated erosion, does not improve on them either (App. N). The census in Sec. 4.2 explains why: the states worth stopping at are few and concentrated in particular problems, so even perfect detection would add about two points.
Related Work
Overthinking and adaptive compute. Work on overthinking studies wasted computation on easy problems [3, 38], budget-dependent accuracy [51], answer abandonment [43] and agentic over-reasoning [5]. Early-exit methods use answer convergence, redundancy detection, attention probes, cost-aware stopping and adaptive sampling [47, 29, 31, 20, 39, 32, 25, 10, 46, 2, 18]. Our decomposition shows that survivor-averaged budget curves overstate within-chain harm, and our headroom measurement bounds what such methods can add in accuracy.
Value estimation and optimal stopping. Process reward models and value probes are often trained on Monte Carlo rollout labels: the fraction of continuations from a prefix that end correct [40, 28]. That fraction estimates the verdict. Deciding whether to stop also needs the commitment from the same prefix, the stop-now value of optimal stopping [4].
Martingale views of reasoning. Li et al. [24] use the Doob decomposition and stopping theory for path selection, Kim et al. [22] study coherence under answer resampling, Farmer et al. [12] characterize elicited probability transformations, and He et al. [16] test self-reported belief revisions against a martingale rationality criterion. We separate the forecast of the final outcome from the quality of the answer available now, and connect their gap to the value of continuing.
Discussion and Limitations
Deciding whether more reasoning is worth it requires comparing the answer available now with the answer continuing would produce, on the same chain. Final-correctness forecasts and survivor-averaged budget curves each miss one side of this comparison. Measured directly, continuing rarely loses a correct answer in the models we study; the losses concentrate in a few problems, and the declining curves reported in the literature come largely from which chains remain. The same comparison applies to other ways of spending computation, such as self-revision rounds or multi-agent exchanges (App. M).
Limitations. Our experiments cover mathematical reasoning and models up to 8B parameters. Commitment is defined by a specific stop-and-answer procedure; the robustness checks in App. § 16 vary it. The training objectives of App. § 14 are tested only on synthetic data.
Reproducibility Statement
All experiments run on single consumer GPUs (12 and 16 GB); models and datasets are public. Seeds (0 for the primary audits; the resampling and ablation studies draw from disjoint seed streams recorded in the released code), prompts and forcing strings, splits, hyperparameters, and the pre-committed gates are in App. § 16. The matched follow-up and refit variant have separate frozen settings, raw outputs, and checks (Apps. § 18–§ 19); their seed streams start at 20260907. Analyses added during review (App. O) have their own dated plan, fixed before generation; their seed streams start at 20261106. Quantitative tables are checked against result files. Internal dated plans are distinguished from external registration. The new follow-up code and results accompany the primary audit artifacts.
AI Use Statement
An AI coding assistant was used throughout this project: for drafting and editing prose, writing and debugging experiment and analysis code, scaffolding statistical analyses, and assisting with mathematical reformulation and proof checking. The authors take responsibility for the final text, theoretical claims, proofs, code, analyses, and reported results, including any errors.
A Notation Glossary
| , , | complete reasoning state; its filtration; observed tap |
| , | commitment belief (answer-if-stopped-now correct); its probe |
| , , | verdict ; its state form, equal to under Assumption 1 (App. § 13); its probe |
| belief increment = predictable drift + innovation | |
| tap-conditional drift (what a critic can see) | |
| , | accumulated drift (Doob compensator); its estimate |
| drift critic (sensor); compensated belief | |
| Signal A / B | forced-answer correctness (probe-independent) / probe readout |
| overthink-wrong, stay-correct | correct at a checkpoint then wrong / then right at the end |
Table 9.
B Full Audit Matrix
C Extended Related Work
Probes, values, estimation. Hidden states encode answer correctness [47] and linearly represent belief-state geometry [36]—the license for our readouts: commitments can drift, whereas exact verdicts on nested histories cannot; lossy instantaneous taps and fitted probes need not inherit that null (App. I). Process reward models and MC/TD variants estimate step values [28, 40, 49]; ours is orthogonal.
The training objective is adversarial conditional-moment estimation [9, 23] in the predictability-minimization family [35, 14, 48]; post-hoc self-revision alone is unreliable [19, 30].
Proofs
The results of § 2–§ 3 use the following assumption on the reasoning process.
Assumption 5. The sampling noise is i.i.d. and independent of , so is a time-homogeneous Markov chain with operator ; and terminal correctness is a function of the state in which the chain absorbs. The readout is a measurable function of alone (lossy taps: App. N); a critic receiving the step index augments the state to , for which all results hold.
Theorem 6 (Ballistic bias, diffusive variance). If and a.s., then with , , and a martingale with increments bounded by , so (Azuma–Hoeffding). If, for a fixed sign , a.s. for every , then with probability at least , for ,
This is a finite-horizon statement. On the probability scale the persistence assumption entails in expectation; it is not an asymptotic growth law for bounded beliefs. Log-odds drift requires its own decomposition.
Theorem 6. Summing (1), has and increments bounded by , so Azuma–Hoeffding gives the tail bound. is the triangle inequality. The one-sided inequality applied to gives the displayed signed bound. Taking expectations under gives .
Ensembling: innovation and shared drift. Averaging independent belief processes reduces the standard deviation of their averaged innovation by a factor of under identical variances, but leaves common expected drift intact. This concerns belief averaging, not nonlinear voting over terminal answers; vote-curve survivorship is audited in App. § 16.§ 1.
Theorem 7 (Doob). Any integrable adapted decomposes uniquely as with a martingale and predictable, ; necessarily .
Uniqueness (App. § 12) makes the construction canonical: with “systematic drift” meaning the predictable part, there is exactly one such object.
Definition 8 (Commitment conservation). The commitment is conserved (dynamically calibrated) iff , equivalently . This is a dynamics condition, not standard probabilistic calibration, a rationality axiom, or a correctness guarantee: even a constant wrong readout satisfies it, whereas useful computation can increase commitment.
At certainty, on , strictly if a decrease has positive conditional probability. A conserved bounded commitment cannot leave 1. A sampled correct answer need not imply certainty; deterministic elicitation at a fully specified state does (App. § 12).
Theorem 7. Existence by the displayed construction. Uniqueness: if with both decompositions as stated, is a predictable martingale with , so .
Proposition 1. (i) by the tower property, since ; is bounded, so all conditional expectations exist. No further hypothesis is used—in particular none on the dynamics, the tap, or the stopping time—so a chain that overthinks has a conserved verdict exactly as one that does not. Its Doob decomposition (Thm. 7) is , so by uniqueness. (ii) For , ; summing from to and letting , the commitment side converges by conditional dominated convergence (the chain is frozen after , so pointwise; and give ) and the drift side is dominated by (as for and ), integrable because . This gives , i.e. (2). If a.s. for all then is a martingale, so by uniqueness; conversely makes the right side vanish. Writing for a measurable requires additionally that be sufficient for with respect to (Assumption 5), which is used nowhere above and is an assumption about the observation, not the process.
Theorem 9 (Convergence and budget-invariance). A calibrated belief () converges a.s. and in and, for every stopping time , satisfies . Thus its expected conclusion is budget-invariant; a biased chain instead adds .
This concerns the mean, not the full stopped distribution; interior beliefs can decrease on individual paths. Calibration also yields a polarization bound (Thm. 10); adverse drift produces an interior optimal budget in the synthetic model. Proofs are in App. § 12.
Theorem 9. Convergence: Doob’s martingale convergence theorem for bounded martingales; by uniform integrability. Budget-invariance: optional stopping for bounded martingales (). For the biased case apply optional stopping to .
Certainty at the boundary. If , then on we have . The inequality is strict wherever . If , the nonnegative variable has conditional expectation zero and therefore vanishes a.s. Iteration makes 1 absorbing for a bounded martingale. The same argument applies to 0. Thus a single stochastic correct elicitation is not proof of certainty, but an exact binary correctness readout cannot undergo a positive-probability correct-to-wrong transition with zero adverse drift at every preceding correct state. With lossy taps or learned probabilities, this implication cannot be inferred from one observed label alone.
Theorem 10 (Polarization / step count). Assume calibration, , and relative innovation non-degeneracy, for all (the right-hand side vanishes at ), so absorbing boundaries remain admissible. Then . For and , after steps the belief has exited to within of a definitive verdict with probability , and bounded threshold stopping preserves the mean by Thm. 9. More information per step (larger ) means fewer steps; the dependence on the target is logarithmic.
Theorem 10. Let . Calibration gives for every , using . Iterating, . On , , so Markov’s inequality gives for . Validity of threshold stopping is Thm. 9.
Accuracy is not posterior polarization. For a calibrated posterior on a fixed binary hypothesis, the Bayes decision accuracy is , which can increase while is constant. For an answer-if-stopped-now success probability , expected accuracy is instead . These are different readouts. In particular, a martingale decomposition alone does not justify an accuracy expansion of the form , or an interior optimal budget. The interior peak in Sec. N.3 is a property of the specified synthetic decision model, measured numerically. Equation (3) is the corresponding exact statement for operational commitment accuracy.
Proposition 15. since the cross term vanishes by the tower property ( is -measurable). A penalty with locally quadratic charges both terms at the same second-order rate. Consecutive-step KL between readout distributions is, to second order, at interior probabilities, the Fisher-weighted squared increment ; the weight is -measurable, so with the same weight on both components. This is a local expansion away from the endpoints, not a global comparison between KL and squared increments.
Lemma 16. using the tower property (), maximized at over any class containing ; for general the display equals , whose supremum over is .
Theorem 18. (i) Sum Lemma 16 over (the critic receives the step index, so the sum is attained term-wise); under Assumption 5, . (ii) If then for every , so the inner objective is , maximized at with value 0.
Proposition 11 (Graceful degradation). Under Assumption 5, if , then for any stopping time with , , and conditionally .
Proof. has conditional mean under Assumption 5. Write with a martingale (, increments equal to and bounded by 2) and predictable with and . Optional stopping for bounded-increment martingales at with gives , and gives ; conditioning on throughout gives the second form.
The Verdict in Operator Form
Harmonic reference and accumulated drift. The common terminal boundary in Prop. 1(ii) makes the gap equal to accumulated remaining expected change. In Markov form, the same relationship becomes a Poisson equation. Under Assumption 5, the exact value function satisfies : it is harmonic for the fixed continuation policy. Therefore
In a finite absorbing chain, is the unique bounded harmonic extension of terminal correctness and , with the transient-state matrix (see below). Thus the gap weights signed local drift by expected future state visits; positive and negative changes can offset. In particular, . The expected occupation of future states determines how much each local drift contributes. A harmful step followed by sufficiently beneficial ones can still leave positive continuation value. The Poisson equation thus connects stepwise answer dynamics to the value of the entire remaining path.
Let (Assumption 5) and .
Proposition 12 (Operator form). (a) : drift is the generator applied to the readout. (b) The chain is calibrated iff , i.e. the readout is harmonic: . (c) Assume the terminal structure is absorbing: from every interior state the boundary (an emitted answer, carrying ) is reached with probability one. Then is the unique bounded solution of the discrete Dirichlet problem with boundary data , and : the verdict is the harmonic extension of terminal correctness, i.e. the exact value function.
Under Assumption 5 this state function agrees with the filtration-conditional verdict of Prop. 1, a.s., which is what licenses writing as a function of the state at all. (d) , where is the unnormalized pre-absorption occupancy measure (no horizon factor is needed with this normalization): the training objective of Sec. N.2 is a squared generator residual, minimized exactly on harmonic readouts on the occupied states. It is not the usual Dirichlet form .
Proof. (a), (b) are definitional; (d) sums pointwise. For (c), define . Almost-sure absorption makes this bounded candidate well-defined; the Markov property gives in the interior and its boundary values are . Uniqueness holds because if is bounded, harmonic, and zero on the boundary, then by dominated convergence under a.s. absorption. Without absorption uniqueness genuinely fails: is bounded, harmonic, and vanishes on the boundary.
Fundamental-matrix form (finite case). If the interior state space is finite (e.g. a finite discretization, or the state augmented with a step index under a hard budget , which makes the interior block nilpotent) and the chain is absorbing, write . Then , hence , is invertible, and
Kemeny & Snell, [21]: is the fundamental matrix (the Green’s operator of ), the expected visits to before absorption, and the absorption-distribution map. On infinite state spaces sub-stochasticity does not imply ; the probabilistic representation of Prop. 12(c) is the correct general statement, and (5) has a probabilistic interpretation through visit counts. Almost-sure absorption alone does not ensure finite expected duration or a bounded inverse on an infinite state space.
Poisson representation and horizon sensitivity. Writing on the transient states, the common terminal boundary gives , hence . Consequently
Expected remaining duration therefore controls how local drift translates into a commitment–verdict gap. In the finite absorbing case ; if is diagonalizable, convergence is bounded by a constant times . Diagonalizability of an arbitrary Markov operator alone is insufficient: other unit-modulus eigenvalues can prevent convergence.
On log-odds “closed forms.” The compensated belief estimates the martingale component (Thm. 7); it differs from the verdict by the expected total accumulated drift , while the raw belief differs from by expected future drift alone (2). No identity of the form holds: expectations do not commute with , and probability-scale drift is not a log-odds increment. If a log-odds compensation is wanted, define the log-odds drift for and compensate inside the logit; we use the probability scale throughout.
The Variance-Blind Objective: Formal Statements
Selective improvement. The sign of distinguishes improvement from erosion, and the sum of future signed drifts determines the value of the remaining computation. This makes the choice of a training target consequential. A penalty on all answer movement, , suppresses both predictable change and stochastic exploration. A penalty on isolates predictable change but also charges beneficial improvement. A one-sided target, with , instead isolates expected erosion. The rest of this appendix derives the corresponding variational objectives. The mathematical distinction is useful for process supervision: rewarding better answers and preserving available correct answers require a signed quality signal.
This appendix states and discusses the results summarized in Sec. N.2. In the critic formulas, denotes the available input (the tap in App. N), with time included when supplied. Equality requires a sufficient input; otherwise the objectives target only the predictable component visible to that input.
Proposition 13 (One-sided erosion objective). For a square-integrable increment , let and . Then
where the supremum is over square-integrable, measurable functions. The value is zero iff a.s.; thus it penalizes adverse predictable movement without directly penalizing beneficial predictable movement or residual variance.
Proof. Conditioning gives . Pointwise, the concave quadratic is maximized over at , with value . This maximizer is square-integrable by conditional Jensen. A restricted critic class containing zero has a nonnegative supremum no larger than this value, with equality when it contains the maximizer.
This is a derived alternative to the symmetric objective, not a reported training experiment. Under a sufficient tap, its zero set is the submartingale condition , which by Cor. 3 excludes an accuracy-improving stopping rule under the same protocol. Estimator error, representation shift, and task loss still matter during optimization.
Proposition 14 (Readout error propagates into measured drift). Let be adapted with square-integrable errors, and define . Then
If a.s. at every step, the pointwise drift error is at most . The same bound holds after projection onto a tap.
Proof. Subtract the conditional increment definitions and apply conditional expectation contraction in , followed by the triangle inequality. The uniform bound follows directly; another conditional expectation is also a contraction.
Neither AUC nor binned ECE supplies such a uniform error bound. Consequently, predictable increments of a learned probe certify predictability of that probe, not automatically of the underlying behavioral success probability.
The objective, as used in the main text. Penalizing the variational value of Sec. N.2 gives the label-free, variance-blind objective
With a fixed readout and a critic class containing , the inner maximum is exactly : it vanishes iff the chain is state-calibrated, while admissible zero-mean innovations incur no direct penalty.
Proposition 15 (Locally quadratic penalties charge both components). Since is -measurable, . Any penalty that is locally quadratic in the per-step belief change—squared-increment regularizers and, to a second order, KL-type proximity penalties between consecutive readout distributions—charges innovation on the same footing as drift. It therefore penalizes useful unpredictable updates as well as predictable movement; the effect of optimization on either component depends on the model and task loss.
We penalize only the predictable part. As a conditional expectation, it admits a variational characterization—the classical projection, used here as an adversarial conditional-moment estimator ([9, 23]).
Lemma 16 (Variational identity). Let and let be a function class containing . Then
attained at . For a capacity-limited , the supremum equals , the representable share of the drift energy: guarantees hold against drift representable by the critic class.
Definition 17 (Drift-penalized training). With a frozen readout and a critic (state and step index),
The critic forecasts the next belief move; the reasoner is trained to make its own updates unforecastable. The penalty uses no gold labels.
Theorem 18 (Identification of drift energy and variance-blindness). (i) Provided contains the map , the inner maximum of (7) equals ; it is zero iff the chain is state-calibrated (), and under Assumption 5 equals with zero iff . (ii) Variance-blindness: if with arbitrary, the optimal critic is and the penalty vanishes. The objective does not directly penalize zero-mean innovations; this orthogonality does not guarantee that optimizing a shared model preserves accuracy, information acquisition, or innovation variance.
Algorithm 1 (App. G) instantiates (7) with a two-timescale update. Freezing the head fixes its weights but does not fix its input representation or calibration. Equation (6) descends from predictability minimization and adversarial conditional-moment estimation (lineage in § 5); what is new is the semantics—the value held fixed, the dynamics regularized, the variance-blindness guarantee (ii), and the critic’s second job at inference (Sec. N.2). Sec. N.3 validates the mechanism where ground truth is available; fine-tuning a full reasoning model with (7) is future work (App. G).
Deployment Notes and Future Training Work
Serving (Level 1). One critic forward pass per step; maintain ; alarm when ; answer from the last safe checkpoint (or force an answer at the trigger point). The compensated belief is the drift-purged reading used for answer selection. is set from the erosion distribution of healthy (stay-correct) held-out chains.
Level 2 (state steering). Let be the Jacobian row of the scalar critic. Then , with and both , is the Gauss–Newton step for to first order; it vanishes as . For a linear critic does not depend on the current hidden state, so the update follows a time-indexed fixed direction with state-dependent magnitude—the sense in which static activation steering ([26]) is the degenerate case. A critic constant in has and produces no steering at all. Variance-gate the intervention with the critic’s uncertainty head; fall back to Level 1, which does not modify hidden states.
Implementation choices for language-model training (Alg. 1). The -step differentiates the penalty through generated text. Sampled tokens are discrete, so the pathwise gradient is unavailable as written and a concrete estimator must be chosen: policy-gradient/REINFORCE on the sampled trajectory, a teacher-forced fixed-token surrogate, a straight-through relaxation, or differentiable latent reasoning. Our synthetic study differentiates a continuous belief directly and so does not settle this; the reference implementation we release adopts the teacher-forced surrogate (sample a rollout without gradient, re-forward it with gradient, and propagate only through the readout’s dependence on the hidden states). We therefore state Alg. 1 as conceptual for autoregressive models.
A frozen head is not a frozen ruler. Freezing the readout’s parameters prevents direct updates to the readout weights, but updating changes the representation that feeds the readout, so its calibration and semantics can drift even with fixed weights. Any real fine-tuning run should therefore monitor readout calibration on held-out problems during training, or freeze the tapped representation below the intervention point; we do not consider this solved.
Feedback trap. Never fit the critic on corrected trajectories: the controller removes the signal the critic estimates. Audit rollouts must stay open-loop.
Algorithm 1 Drift-penalized training (two-timescale; conceptual—see App. § 15 for the gradient estimator a language model requires)
train readout on base-model rollouts; freeze ;
while not converged do
roll out uncorrected chains;
for to do
end for
\hfill
end while
return ; retain as the inference-time drift sensor
Algorithm 2 Drift audit / critic fit (frozen model )
roll out uncorrected chains on ; record
fit (equivalent to (6))
return ; report held-out and chain-clustered uncertainty
Training with gold anchors (future work). Eq. (7) enforces martingale (self-consistency) calibration; it is silent about which verdict is conserved. When verified pairs exist, a complementary phase teacher-forces the gold answer and supervises the belief toward the harmonic extension of the verified boundary with a proper scoring rule—supervising calibration, not token-likelihood. Critic data must remain free-running (the guard above); gold labels are load-bearing only at the boundary. Fine-tuning a full reasoning model with the combined objective, and the accuracy-vs-budget flattening it predicts, is the natural next experiment; we did not run it. Expected pitfalls, for those who do: (i) the frozen-ruler guard matters most under fine-tuning, since the readout would otherwise drift with the model; (ii) interaction with preference-tuned policies is untested—a KL-to-reference term and the drift penalty regularize different objects and should compose, but the two-timescale inner maximization adds instability risk; (iii) rationale style may shift (the penalty discourages predictable belief movement, not predictable text); and (iv) with positive average predicted drift at these scales, the penalty’s measurable effect may be small until adverse-drift regimes (longer budgets, weaker models, distribution shift) are targeted deliberately.
Experimental Details
Internal pre-registration. Gates, thresholds, operating points, label definitions, and interpretation rules were written down and dated before the corresponding real-data results were computed, and are reproduced in App. § 16; the dated plan itself ships with the supplement. The original gates were fixed before any real-data run, and the secondary analyses before any Exp. 1/2 number was unblinded. The record is an internal, dated plan rather than an entry in an external registry; we use “pre-registered” in that sense throughout. Its content is auditable against the reported outcomes: Gate 1(a) failed and triggered its committed fallback, Gate 2 was met at no operating point, and rule (ix) suppressed a detection claim—all reported as such.
Historical cross-protocol branching diagnostic. The earlier study uses the same 12 wrong-ending 1.5B chains at their first correct checkpoint (; all five levels), but samples 16 immediate answers and 16 continuations with 4096 additional tokens and no terminal forcing. Its continuation success probability is not the primary-policy verdict . It gives , and gap ; 76% of 192 continuations are capped without an answer, versus 7% of 480 in the earlier 30-prefix control arm. Low success therefore largely reflects nonresolution. The post-hoc partition (, gap ) is descriptive. These historical results are retained below but are not substituted for the matched follow-up in Sec. 4.2.
Figure 4. The commitment–verdict gap on a simulated binary-evidence chain with adverse predictable drift (illustrative). The verdict is the exact posterior of a binary terminal outcome, revealed at termination. The commitment adds a nonnegative offset whose conditional expectation decreases, giving adverse predictable drift. The shaded gap equals and vanishes when both readouts equal . It need not shrink monotonically on an individual path.
Figure 5. Left: synthetic overthinking under decaying evidence and a persistent lean; compensation recovers the calibrated plateau. Right: the symmetric penalty destroys evidence gain and accuracy, while the variance-blind penalty leaves both intact across .
| problem | checkpoint | gap | ||
| 326 | 2048 | 1.00 | 0.00 | +1.00 |
| 101 | 1536 | 1.00 | 0.00 | +1.00 |
| 493 | 1536 | 1.00 | 0.19 | +0.81 |
| 128 | 512 | 0.81 | 0.00 | +0.81 |
| 274 | 2048 | 0.94 | 0.12 | +0.81 |
| 120 | 1536 | 1.00 | 0.25 | +0.75 |
| 126 | 1024 | 0.62 | 0.00 | +0.62 |
| 398 | 512 | 0.94 | 0.31 | +0.62 |
| 381 | 512 | 0.56 | 0.00 | +0.56 |
| 9 | 2048 | 0.38 | 0.00 | +0.38 |
| 382 | 2048 | 0.38 | 0.06 | +0.31 |
| 238 | 1536 | 0.94 | 1.00 | −0.06 |
Table 10 (PDF p. 19).
Experiment 0: late-onset drift and the configuration sweep. A pre-registered late-onset test (Figure 10) makes the penalty load-bearing: a lean beginning only after the supervised horizon survives task supervision, is removed by the drift penalty (85.4% accuracy with gain intact), and defeats the symmetric penalty, which instead saturates the belief. Across 19 synthetic configurations, median compensation recovery is 98.6%; the main failure mode is belief saturation, where probability-scale drift becomes hard to observe.
Figure 10. Late-onset lean: the lean acts only after the supervised horizon (; task loss at ), so task supervision cannot see it. Left: accuracy vs. step; without a penalty the late lean erodes accuracy after the horizon; the drift penalty restores the calibrated trajectory; the symmetric penalty flattens the gain as well. Right: residual lean by objective.
Hardware and stack. One RTX 4070 Ti (12 GB), Windows 11, PyTorch cu126, Hugging Face transformers (SDPA attention), fp16. Primary-audit seed 0; follow-up studies use their recorded, disjoint seed streams.
Rollouts. MATH-500 test split; prompt = problem + “Please reason step by step, and put your final answer within \boxed{}.” via the chat template (the R1-distill template opens <think> automatically); temperature 0.6, top-p 0.95, max 4096 new tokens, one chain per problem, batch 8.
Forcing. At checkpoints (truncating mid-think): append \n</think>\n\n**Final Answer:** \boxed{ and decode greedily ( tokens, parsed with the permissive brace-closing rule, which accepts a continuation that never closes); if the chain had already closed its think block, the forcing string is appended without the closing tag. The natural termination is its own checkpoint (the chain’s own boxed answer where present). Verification: normalized string match with math-verify fallback; unparseable incorrect.
States and steps. Teacher-forced prefill; hidden states at layers 14/21/27 (of 28) at paragraph boundaries (\n\n splits, segments tokens merged forward, steps/chain) and at every checkpoint position. Layer 21 is primary; 14/27 form the robustness panel.
Probes and critic. Commitment probe: logistic regression on checkpoint states with label = forced-answer correctness at that checkpoint; selected by inner CV over ; temperature-scaled on one fifth of the probe split; 15-bin ECE reported. Verdict probe: identical protocol, label = final-chain correctness. Critic: LayerNorm 2256 GELU MLP, dropout 0.1, mean and log-variance heads, Gaussian NLL, AdamW (, wd ), batch 1024, early stopping; input ; targets on uncorrected rollouts of the critic split only. Splits: 40/30/30 problem-level, stratified by (level, final correctness).
Labels. overthink-wrong: correct at some checkpoint tokens (strictly before the final answer) and wrong at the final answer; stay-correct: the same but correct finally; chains ending before the first checkpoint have no early checkpoint and can receive neither label; late-bloomer: no early correct checkpoint, correct finally; early-unresolved/wrong: no early correct checkpoint, wrong finally. The latter is stored as never_correct in the archived code and shown as “never-correct” in legacy figures. It does not imply correctness nowhere: the implemented label does not inspect transient correctness after token 2048.
Conventions. The theory’s abstract “step” is instantiated as a paragraph segment of the think text (typically 40–90 per chain); checkpoints are token-indexed, and the critic receives the token position, so the two clocks are linked through . “Mid-chain” is the first paragraph step at or after 50% of the chain’s generated length (and at the last step). Chains with fewer than four paragraph steps are excluded from step-level analyses.
Probe health at the pre-registered tap. Rule (viii) is not triggered at either scale, on its pre-registered AUC condition alone: the verdict probe’s final-correctness AUC is 0.747 at 1.5B and 0.735 at 7B, neither above 0.75. Separately—and as a post-hoc judgment, not part of the dated rules—the 1.5B verdict probe is poorly calibrated (ECE 0.138 against 0.074 for the commitment probe), so we do not read its predictable fraction as evidence about the tap.
Probe-shift and difficulty controls. Drift energy near vs. far from probe-training checkpoints is comparable (1.5 /1.6 ; 7B 7.0 /7.1 ), which does not reveal a large checkpoint-proximity effect; on levels 1–2 (5B), both overthink rate and matched-position drift energy fall (1.5% vs. 2.7%; 1.5 vs. 1.7 ).
Mean predicted drift by group. 1.5B: +0.0144 overthink-wrong, +0.0052 stay-correct, +0.0035 late-bloomer, +0.0087 early-unresolved/wrong; 7B: +0.0228, +0.0154, +0.0203, +0.0240.
Per-group outcomes at the fixed operating point (Figure 6). Alarm at the pre-registered (90th stay-correct percentile, critic split), eval levels 3–5: overthink-wrong 0.0% (0.0% strict), stay-correct 97.5%, late-bloomer 83.3%, early-unresolved/wrong 0.0%; fire rates 0% (overthink-wrong) vs. 13% (stay-correct); mean charged tokens 2624 vs. 2808 at full budget.
Figure 6. Per-overthink-wrong-chain outcomes at the pre-registered operating point (1.5B left, 7B right; annotated, eval events per scale). Anecdotal at this prevalence—deferred here per the interpretation rules; fire rates 0% / 50% on overthink-wrong.
Exp 2 policies. All replays are nonanticipating; answers only at recorded checkpoints (the last checkpoint at or before the trigger). th percentiles of on stay-correct critic-split chains; belief-drop thresholds for both raw and compensated ; fixed stops at every checkpoint. Tokens are charged by rule (vi) below.
Escalation outcome. Gate 1(a) failed at 1.5B (10 overthink-wrong chains on levels 3–5, minimum 30); the escalation to 7B-NF4 on levels 4–5—the pre-registered fallback of Gate 1(a)—was executed with identical protocol and returned 9 overthink-wrong of 262. Gate 0 passed (Sec. N.3); Gate 2 was met at no at either scale (the alarm loses accuracy at both). Both audits are reported in full. The rule-(ix) predictor comparison is unpowered—the evaluation split holds 2/2 overthink-wrong chains (1.5B/7B) against 79/50 stay-correct, below the pre-committed minimum of three positives for reporting AUC—so no AUC is reported and the raw paired-bootstrap differences are released with the result files in the supplement. The 1.5B stopping-policy replay (full 77.3% at 2808 tokens; alarm 74.5% at 2624; vote 71.8%; agreement 65.5% at 1625) mirrors the 7B replay of Sec. N.5 (Pareto fronts: Figure 8 at 7B, Figure 9 at 1.5B); the 1.5B commitment-probe and compensator trajectories by group are in Figure 7.
Figure 7. 1.5B belief and compensator trajectories by group (eval, levels 3–5; illustrative—two eval overthink-wrong chains).
Figure 8. Stopping-policy replays at 7B (levels 4–5 eval): every family sits below the full-budget point; the monotone cohort curve separately explains why global truncation has no favorable operating point.
Figure 9. 1.5B stopping-policy replay (levels 3–5 eval), analogous to Figure 8.
Gates (pre-committed). Gate 0: sensor slope in [0.9, 1.1], , no false drift at , detection floor reported. Gate 1: (a) overthink-wrong chains on levels 3–5, with the committed fallback “if not, escalate to 7B-4bit on levels 4–5 with identical protocol before concluding absence”; (b) and within 0.02 of the PCA-32 baseline or better; (c) onset localization consistent with the flip. Gate 2: at some , recovery of of the overthinking loss on overthink-wrong chains; stay-correct accuracy change pt with fire rate ; tokens saved at matched-or-better overall accuracy.
Secondary pre-registered analyses (fixed before any Exp. 1/2 number was unblinded). The original gates above are unchanged. The following analyses, definitions, and decision rules were fixed while the rollouts were running and before any real-data result was computed. (i) Operating point. The reported alarm operating point is fixed in advance as the 90th percentile of on stay-correct critic-split chains; other percentiles appear only in the Pareto sweep. (ii) Strict labels. Because a single forced answer is a noisy draw, all headline quantities are additionally reported under a strict label: overthink-wrong-strict requires correctness at two consecutive checkpoints $\le 2048 with the same normalized answer string, and a wrong final answer; stay-correct-strict analogously.
The population and the conclusions must survive this tightening. Per-checkpoint answer consistency (fraction of checkpoints agreeing with the chain’s modal answer) is reported by group. (iii) Baseline gauntlet (Exp 1). The predictor comparison additionally includes: a supervised MLP (critic architecture) classifying overthink-wrong vs. stay-correct from ; a length-only predictor; a SignalA-only predictor (forced-answer pattern up to mid-chain); an entrenchment score in the style of He et al. [16] (per-chain regression of on ); and critic ablations (no step-index input; input only). Paired bootstrap confidence intervals are reported for AUC() − AUC(probe slope) and AUC() − AUC(supervised MLP). (iv) Prediction vs. postdiction. All predictor rows are also reported restricted to overthink-wrong chains whose forced answer is still correct at mid-chain. (v) Policy gauntlet (Exp 2). Added causal-replay policies: checkpoint majority vote over the eight forced answers; answer-agreement stopping (two consecutive checkpoints with the same normalized answer); verdict-probe level and drop stopping; the fixed stop at 2048 called out against Gate 2’s region; and alarm vs. raw-drop detection compared at matched stay-correct fire rate. (vi) Token accounting. Tokens charged = trigger position plus 48 tokens per forced-answer branch executed by the policy (a fixed allocation, not measured decoded length); fixed stops and stop-policies are charged one branch per executed forcing, majority voting is charged all eight, and full budget is charged none, including when its terminal answer required forcing. Critic evaluation and branch prefill are not charged. These are archived token allocations, not measured wall-clock time or FLOPs. (vii) Onset null. Onset localization is reported against a within-chain permutation null (random step position), with grid resolution 512 tokens noted. (viii) Placebo normalization and failure criterion. The placebo is assessed as the predictable fraction against a probe-matched shuffled control at comparable increment variance. If the verdict probe’s normalized predictable fraction exceeds its shuffled control while its final-correctness AUC exceeds 0.75, we will report this as evidence against the tapped-state harmonicity approximation, not as probe estimation error. (ix) Interpretation rules. If AUC() is statistically indistinguishable from the supervised classifier, probe-slope, or length-only baselines (paired bootstrap), we will report that mid-chain states generically predict failure and that the compensator’s contribution is semantic (interpretation, onset localization, the readout–verdict gap) rather than raw detection power. If the drift alarm does not Pareto-dominate checkpoint-majority voting and verdict-probe stopping under accounting (vi), we will report the alarm as one member of a family of effective stopping rules rather than as superior. (x) Resampling study. If compute allows, additional chains are sampled for level-3–5 problems to measure label stability under resampling, the per-problem accuracy-vs-budget estimand, a sibling-chain shared-drift test, and a token-matched self-consistency point for the Pareto curve.
Preliminary AIME-2024 audit (post-hoc). As a hardest-regime probe we ran chains per problem on all 30 AIME-2024 problems at 1.5B with the identical protocol (120 chains; final accuracy 17.5%; 88% hit the 4096-token cap). No chain is overthink-wrong (0 of 120) and the capped-cohort curve rises monotonically (0.9% → 10.4%, ), but this configuration is overwhelmingly competence-limited: 99 chains are never correct, and only 12 of 120 chains ever reach a correct early checkpoint and are therefore eligible for an overthink-wrong transition at all. On that opportunity set the count is 0/12, whose 95% upper bound is ≈25%. The zero is a low-powered null, not evidence that overthinking is absent where folklore expects it; budgets beyond 4096 remain untested.
Forcing-operator validation (post-hoc). To test whether the budget-forced answer matches a natural halt, we truncated 160 stratified (chain, checkpoint) pairs at 1.5B, closed the think block without the forcing string, and let the model continue greedily for up to 220 tokens. Most natural continuations (69.4%) emitted no boxed answer at all—absent the forcing string the model tends to resume reasoning rather than commit, which is why the operator is needed in the first place. Where a natural answer did appear, it matched the forced answer string in 81.6% of cases and agreed on correctness in 85.7%; the residual disagreement is consistent with genuine indeterminacy of mid-chain commitment under two different halting prompts. We therefore read as commitment under the standard budget-forcing operator (the object our conventions define), with agreement measured only among the minority of prefixes that answer under both protocols; this does not bound unconditional operator error.
Checkpoint-grid robustness (post-hoc). Recomputing labels on a coarser 1024-token grid (a strict subset of recorded forcings): overthink-wrong counts move at 1.5B and at 7B. Because the grids are nested and the overthink-wrong label requires a correct answer at some checkpoint, refining the grid can add chains but never remove one: the count is non-decreasing in density, so the ≈3% figure is specific to the 512-token grid—a ceiling for coarser grids and a floor for finer ones, not a one-sided bound.
The quota resample is not a prevalence estimate. We do not pool the quota-selected resample set (item x) into a prevalence estimate; the pooled estimate is the separate study below. That set was selected by outcome-dependent quotas on the original chain’s label, so it is a census of the 10 overthink-wrong problems and is enriched ≈4.6× for the outcome being estimated; that design supports label stability, not prevalence. Post-stratifying by the known population strata gives 3.2% [1.0%, 6.4%] for a fresh chain (against 4.7% unweighted on the enriched sample), which merely recovers the independent single-chain rate 10/367 and so adds little information—the interval is wide because the overthink-wrong stratum contains only 10 problems.
Resampling study (pre-registered item x; results). fresh chains per problem on 80 level-3–5 problems (1.5B, identical protocol). Label stability: 3/10 overthink-wrong problems reproduce the label in at least one sibling; 38/40 stay-correct problems remain stay-correct. The sibling-chain shared-drift test finds no problem-level adverse-drift signal (the original chain’s mid-chain erosion does not predict sibling overthinking; Pearson ), and self-consistency over the five chains (76.2% at 15771 tokens) does not beat one full-budget chain here. This quota sample is too small to settle whether flips are chain-idiosyncratic; the larger study below, on the complete problem set, finds a clear problem-level propensity (23.8% of overthink-wrong siblings against 2.1% of stay-correct siblings) alongside substantial chain noise, and supersedes this reading.
resampling (pre-registered). Eight fresh chains per problem, from eight disjoint seed streams, on all 367 level-3–5 problems at 1.5B, under the audit protocol otherwise unchanged (, top- 0.95, 4096-token cap, 512-token forcing grid, identical labels): 2936 chains, of which 100 overthink-wrong, 1989 stay-correct, 249 late-bloomer and 598 early-unresolved/wrong. Unlike item (x) the problem set is complete rather than quota-selected, so pooling is legitimate: the rate 3.4% (problem-clustered 95% CI [2.4%, 4.5%]) is the repeated-sample prevalence in Sec. 4.2, which also reports the single-chain counts. As the dated plan specifies, it supersedes the single-chain 10/367 as the headline. Sibling propensity and the non-monotonicity screen are documented in this appendix.
Exp 0 details. Evidence separation , , ( in Figure 11); the drift arm adds to each log-odds increment (probability-scale drift ); slope/ are measured against the exact by quadrature, not raw . Overthinking variant: initial evidence scale 1.0 decaying as with the per-step Bayes gain, lean persistent. Objective panel: , learnable (gain, lean), fixed representational noise (hence the learned gain above the noise-free Bayes gain 0.35), scarce-label task loss (96 labeled problems), penalty on unlabeled rollouts, , critic 5 inner steps per reasoner step.
Figure 11. Bayesian evidence accumulation (belief = posterior on a fixed candidate). Left: honest updates (): chains diffuse to 0/1, the mean belief (= verdict) is conserved (max deviation 0.010). Right: the same evidence with a constant lean: the mean bends toward a confidently wrong value, ballistically.
Post-hoc robustness analyses (not pre-registered)
The analyses in this subsection were run after unblinding, on the frozen rollout and state caches. They are sensitivity checks on the pre-registered conclusions, not new pre-registered claims.
Survivorship decomposition: empirical terms (Prop. 4). Conditioning follows the published curve’s convention (: checkpoint recorded, on the grid, matching Prop. 4; capped chains included at the cap); the reproduced aggregate curve matches the published one to machine precision and the identity is exact per step. Region totals over 2048 4096 (chain-level bootstrap, 2000 resamples): 1.5B marginal change = within [, ] + composition [, ]; 7B [, ] [, ] points. within-cohort term is significantly positive at both scales—chains still reasoning keep improving—so survivorship more than explains the entire marginal decline.
| 1.5B | 7B | |||||||
| within | comp | within | comp | |||||
| 512 | 367 | 30.0 | – | – | 262 | 25.6 | – | – |
| 1024 | 358 | 50.6 | +21.5 | -0.9 | 257 | 44.7 | +20.2 | -1.1 |
| 1536 | 319 | 56.1 | +11.0 | -5.4 | 236 | 58.1 | +16.9 | -3.6 |
| 2048 | 270 | 56.7 | +6.7 | -6.1 | 197 | 50.8 | -2.5 | -4.8 |
| 2560 | 222 | 51.8 | +3.6 | -8.5 | 166 | 51.8 | +7.2 | -6.2 |
| 3072 | 192 | 50.0 | +2.1 | -3.9 | 146 | 51.4 | +2.7 | -3.2 |
| 3584 | 172 | 50.6 | +3.5 | -2.9 | 125 | 53.6 | +8.0 | -5.8 |
| 4096 | 151 | 48.3 | -0.7 | -1.6 | 97 | 46.4 | +2.1 | -9.3 |
Table 11.
Label-definition robustness. Relaxing the strict pre-registered overthink-wrong definition: dropping the -token requirement gives 13/367 (3.5%) at 1.5B and 9/262 (3.4%) at 7B—at 7B identical to the base count: every early-correct-then-wrong chain was already correct by token 2048. Requiring two consecutive correct checkpoints gives 8 (2.2%) and 3 (1.1%). Counting chains with any correctwrong forced-answer flip anywhere gives 40 (10.9%) and 26 (9.9%)—but most flipped chains recover by the natural end. Prevalence stays within 1–4% under every final-outcome definition.
Cumulative-history-summary ablation. This follow-up was specified before its own results, but after the original audit. Fixed-window taps can forget information: a pair tap at no longer explicitly includes . Moreover, multilayer_tap.py enriches only the critic in its pair variant, leaving both probes on single states. We therefore refit both probes and both critics with at layer 21. The splits, seed, calibration fraction, hyperparameters, and shuffled-label diagnostic are unchanged. A single-state control reproduces 0.325/0.418 and 0.300/0.359.
This construction is history-dependent but not necessarily nested. For example, histories and give different but identical . Hence the construction does not guarantee . The filtration is nested; conditioning on alone is a different operation. A compressed recurrent state likewise requires a sufficiency argument before inheriting the verdict martingale.
The verdict diagnostic decreases from 0.418 to 0.362 at 1.5B and from 0.359 to 0.336 at 7B, while the commitment diagnostic also falls (0.325 to 0.295; 0.300 to 0.255). The planned positive commitment-minus-verdict gap is not obtained (−0.067/−0.081). These results show only that this augmentation does not resolve the control discrepancy; they cannot distinguish observation loss from estimation error. Nor does the tower property require this ordering for fitted, variance-normalized diagnostics.
There are additional confounds. The doubled input dimension raises the 7B commitment shuffled-label diagnostic from 0.063 to 0.106 (42% of the measured value), limiting comparisons across feature sets. The implementation computes running means over checkpoint states for probe fitting but over paragraph-step states for the increment series, so the history-summary sampling rules differ between training and application. A conclusive history control would require consistent observation sequences, an explicitly nested conditioning history, and separate assessment of estimation error. The archived nested_tap filenames and numerical macros retain their original names for traceability, not as a claim of nested information.
Critic capacity. Replacing the MLP critic with a single linear layer (identical protocol and splits): at 1.5B the linear critic retains genuine held-out skill on increment prediction ( vs. the MLP’s 0.26), so the predictable-drift signal is not an expressivity artifact of the MLP. The energy-ratio metric is not meaningful for the linear critic (its shuffled-label null inflates), and at 7B the 3585-dimensional linear map overfits its 4.3k training rows (; high estimator variance)—the 7B half of this ablation is uninformative, and the 1.5B half carries the conclusion.
State-richness series under the variational score. Commitment and verdict variance-normalized scores as the tap is enriched (identical protocol and splits; single = pre-registered layer, +history = one-step hidden-state history, 3 layers = L14/21/27 concatenation), reported as rather than , with the score difference from a paired chain-level bootstrap (one shared resample index for both readouts, 2000 resamples):
| 1.5B | 7B | |||
| tap | c/v | gap [95% CI] | c/v | gap [95% CI] |
| single layer | 0.284/0.249 | +0.036 [+0.007,+0.063] | 0.221/0.244 | −0.023 [−0.050,+0.007] |
| + history | 0.232/0.231 | +0.001 [−0.025,+0.025] | 0.172/0.227 | −0.055 [−0.082,−0.027] |
| 3 layers | 0.278/0.296 | −0.018 [−0.056,+0.021] | 0.290/0.292 | −0.002 [−0.035,+0.032] |
| 3 layers + history | 0.299/0.285 | +0.014 [−0.023,+0.052] | 0.016/−2.096 | +2.112 [+1.832,+2.467] |
Table 12.
A positive commitment-minus-verdict gap is what an idealized, accurately estimated verdict control would suggest; the tower property alone does not imply this ordering for fitted, variance-normalized scores. The gap is significant only at the 1.5B single-layer tap; elsewhere the gap straddles zero or, at the 7B one-step-history tap, has a CI excluding zero in the reversed direction (one of eight post-hoc cells, uncorrected). The 7B three-layer-plus-history row is degenerate: its critic diverges at 21,505 input dimensions and yields a strongly negative variational value. A labeled post-hoc rerun at a smaller learning rate gives gap +0.015 [−0.020, +0.052]. Bootstrap intervals are conditional on the fitted models; they do not include probe or critic refit uncertainty, and enrichment need not improve a finite-sample estimate.
Randomized refits are diagnostics, not calibrated hypothesis tests. The archived analysis used 200 refits per scheme at the primary taps and 50 at richer taps: a global training-step shuffle and a circular shift of the concatenated training target vector. The latter is not a within-chain shift. Refits used at most 80 epochs, versus 200 for the observed critic. Neither scheme establishes exchangeability for the heterogeneous, nonstationary chains, so its exceedance ranks are not calibrated permutation -values; we make no significance claim from them. The released implementation also provides a separately versioned within-chain shift with matched fit settings; that correction has not been run for the results reported here. Even within-chain shifts require additional stationarity or exchangeability assumptions for inferential use. The positive held-out estimates and their chain-bootstrap intervals, rather than these ranks, support the reported probe-increment predictability.
The older statistic has a positive estimation-noise floor: its ratio to the shuffled control varies across refits. The population variational identity avoids that particular inflation, but does not make the empirical estimate a guaranteed bound or require every randomized-refit value to be negative. All archived numeric diagnostics are retained in the result files.
Base-family audits: R1-Distill-Llama 8B and Phi-4-mini-reasoning (endpoints fixed in the dated plan before either run). The Qwen3 audit varies training recipe but not base lineage, so we repeated the audit on two reasoning models with non-Qwen bases: DeepSeek-R1-Distill-Llama-8B ([6]), on a Llama-3.1 base, in 4-bit NF4 as in the 7B escalation; and the 3.8B Phi-4-mini-reasoning ([44]), on a Phi-4 base, in fp16. Both models use an explicit thinking block, which the forcing operator requires. The pipeline is otherwise unmodified—one chain per problem on the same 367 level-3–5 MATH-500 problems, temperature 0.6, 4096-token cap, the same 512-token forcing grid and labels. Neither audit is floored or ceilinged (final accuracy 78.7% and 82.6%). Labels: 8 overthink-wrong (5 strict), 252 stay-correct, 37 late-bloomer, 70 early-unresolved/wrong for Llama; 8 (5), 272, 31, 56 for Phi. Both headline findings reproduce outside the Qwen lineage. Prevalence is 2.2% [1.1%, 4.2%] and 2.2% [1.1%, 4.2%], just below the ≈3% of the two primary Qwen-based R1-distills. The marginal decline over 2048 → 4096 (−9.3 and −7.5 points) again decomposes into a positive within-cohort term (+16.6 [+9.3, +24.1]; +9.1 [+1.8, +16.6]) against composition (−25.9 [−31.8, −20.9]; −16.6 [−22.7, −11.1]). Deviations are recorded in results_llama8b/ and results_phi4mini/; the generation batch was reduced from the pipeline default because a larger batch exceeded the 12 GB card—on this platform the driver spills to host memory rather than raising an error, so the reduction was triggered by a throughput floor—and each run was interrupted and resumed through the pipeline’s own resume logic (on (problem, checkpoint) for forcing, which decodes greedily and is therefore deterministic; rollout resumption re-seeds as the Qwen3 run did). Every record was generated exactly once. These runs measure the two pre-registered endpoints only: they extract no hidden states, so no probe, critic or stopping replay is fitted for them and they do not appear in Table 7. They are two additional families, not a scaling study.
| Policy | R1-Distill-Qwen 1.5B Accuracy | R1-Distill-Qwen 1.5B Tokens | R1-Distill-Qwen 7B Accuracy | R1-Distill-Qwen 7B Tokens |
| Full budget | 77.3 | 2808 | 77.5 | 3015 |
| Drift alarm | 74.5 | 2624 | 73.8 | 2767 |
| Checkpoint vote | 71.8 | 3087 | 65.0 | 3315 |
| Answer agreement | 65.5 | 1625 | 61.3 | 1535 |
Table 7. Held-out stopping replay: accuracy (%) and mean charged tokens, on 110/80 evaluation chains at 1.5B/7B with only two early-correct/final-wrong chains each. Small differences do not establish a policy ranking; App. § 16.§ 1 gives the full 48-configuration gauntlet.
Third-model audit: Qwen3-1.7B (endpoints fixed in the dated plan before the run; post-hoc relative to the original audit). Qwen3 shares a base family with our R1-distills but is independently trained and not R1-distilled, so this varies training recipe and distillation status, not base lineage; we do not read it as evidence of cross-family generality. The full pipeline—one chain per problem on the same 367 level-3–5 MATH-500 problems, temperature 0.6, 4096-token cap, the same 512-token forcing grid and labels—applied to Qwen3-1.7B (Yang et al. [45]). The audit is informative rather than floored or ceilinged (final accuracy 78.2%, 63% at 2048). Labels: 11 overthink-wrong (5 strict), 253 stay-correct, 34 late-bloomer, 69 early-unresolved/wrong. Both headline findings reproduce: prevalence 3.0% [1.7%, 5.3%] brackets the ≈3% of the two R1-distills, and the marginal decline over 2048 → 4096 (−2.6 points) again decomposes into a positive within-cohort term (+7.6 [+1.6, +13.7]) against composition (−10.2 [−14.3, −6.1]). Deviations, recorded in results_qwen3/qwen3_audit.json: generation batch 6 rather than 8 (VRAM headroom; throughput only), and the rollout was interrupted by external process kills three times and resumed via the pipeline’s resume logic, so the sampling stream is not bit-identical to one uninterrupted seed-0 run—all 367 chains are complete, deduplicated, and generated once.
A forcing-free view: length–accuracy curves are censoring-dominated (post-hoc). The decomposition above reads forced answers. As a check that the forcing operator is not manufacturing the result, we repeat the exercise with no forcing in the analysis: bin each chain by its natural termination length and score it by its own unforced final answer. The familiar declining curve appears, steeply (−9.1 points per 512-token bin at 1.5B, −8.6 at 7B). Two findings follow, and the first does not match the mechanism our forced decomposition isolates. (i) Stratifying by MATH level removes little of the decline—pooled within-level slopes are −8.4 / −8.4, so difficulty composition accounts for only ≈8% / ≈3% of it. (ii) It is almost entirely censoring: restricting to chains that actually terminated () collapses the slope to −2.4 [−4.5, −0.6] and −1.1 [−2.8, +0.3]. The long-length bins are populated by chains that hit the cap without ever committing to an answer (41% / 37% of chains). Correspondingly, the surviving pool degrades by −20.2 / −23.7 points over 2048 → 4096 under this natural-answer scoring—a same-direction, larger effect than the forced moves, though a different estimand (it carries no “what would it answer mid-chain” content). Caveat: for capped chains the “final” answer must itself be elicited by forcing, so this analysis is strictly forcing-free only on the terminated subset—which is exactly where the effect is smallest. Reading: aggregate accuracy-vs-length plots are dominated by which chains occupy which bin—here through censoring, in Figure 3 through termination-induced composition. Both are selection; neither is within-chain erosion. The forced decomposition remains the stronger design, since it conditions on chain identity and needs no difficulty proxy at all.
Decoding-matched forcing (pre-registered). Chains are sampled (, top- 0.95) but forced answers are decoded greedily, so the readout is taken under a different rule than the process it describes. We re-forced every checkpoint of the 240 chains above under the chain’s own decoding rule, 3 draws each, and compared paired on identical (chain, checkpoint) pairs. The difference is negligible: greedy exceeds sampled forced-answer correctness by +0.7 points, 95% problem-clustered CI [−0.3, +1.8] over 120 problems, with per-checkpoint deltas between −0.3 and +2.4 points. overthink-wrong counts are 5 under greedy and 6/7/4 across the three sampled draws. We recorded in advance the prediction that greedy forcing would inflate the overthink-wrong count, making the reported prevalence an upper bound in that respect; that prediction is not borne out—the sampled counts are if anything marginally higher, and the spread across draws exceeds the greedy-sampled difference. The defensible reading is that the operator’s decoding rule does not materially affect the measurement, not that either rule is conservative.
Dense-grid test (pre-registered). We recorded in advance the prediction that a finer forcing grid weakly increases the overthink-wrong count, because each extra checkpoint is another chance to catch a transient correct answer—the same mechanism that made the coarse-grid recount lower the count. Re-forcing the chains of the previous paragraph on a 256-token grid (a strict superset of the 512 and 1024 grids) confirms it: 5 overthink-wrong chains at 1024 tokens, 5 at 512, and 7 of 240 at 256. The crosstab isolates the mechanism exactly: all 5 chains labeled overthink-wrong on the 512 grid keep the label, and the increase comes entirely from 2 chains that the coarser grid had recorded as never-correct—the finer grid finds a brief correct answer between 512-token checkpoints that is subsequently lost. Prevalence measured this way is therefore non-decreasing in checkpoint density (unchanged from 1024 to 512, up at 256), so the 512-grid figure is a floor relative to finer nested grids and a ceiling relative to coarser ones—grid-specific rather than conservative.
Decoding-temperature ablation (pre-registered). On a stratified 120-problem subset with chains per problem, we repeat the audit at the pipeline default () and at , everything else identical. Prevalence does not rise with temperature: 3 of 240 chains are overthink-wrong at 0.9 (1.2%, [0.4%, 3.6%]) against 5 of 240 at 0.6 (2.1%, [0.9%, 4.8%])—if anything lower, though the intervals overlap and this subset is small. The structural finding reproduces at both settings: the marginal curve peaks near 1.5–2k tokens and declines to the cap (56% 37% at 0.6; 60% 48% at 0.9) while the final-step within-cohort delta stays non-negative—+1.1 points [−4.2, +6.4] at 0.6 and +8.4 [+3.3, +14.8] at 0.9, the latter significantly positive. Per the pre-registration this is a sensitivity report, not a gate.
Probe time-input ablation (post-hoc). The probes receive the tapped state only, while the critic additionally receives normalized token position, which raises the question whether the verdict probe’s residual predictable movement is an omitted-time artifact. The tested time-input augmentations do not explain it away. Normalized position is linearly decodable from the tap itself (– out-of-sample), so the probes were never time-blind; refitting both probes with position added (and with a scaled polynomial in position, to rule out the feature being crushed by the shared penalty) moves the verdict fraction by +0.011/+0.004 at 1.5B and at 7B, all smaller than the 0.014–0.037 standard deviation of the critic refit itself; and a critic given time alone reaches a fraction of only 0.001–0.006. Probe AUC/ECE reproduce the published values exactly in the no-time arm.
Head-to-head with published stopping rules (post-hoc). We replay named methods over the recorded chains under a single effective-token accounting: budget forcing / fixed truncation (Muenninghoff et al., 2025), self-consistency (Wang et al., [41]), Adaptive-Consistency (Aggarwal et al., [2]), ESC (Li et al., [27]), First-Finish Search (Aggarwal et al., [1]), answer-consistency (Liu & Wang, [29]), Dynasor-CoT (Fu et al., [13]), ES-CoT (Mao et al., [31]), our alarm, and an oracle bound. At 1.5B each method is labeled pure replay (28/48), approximate replay (16, the deviation recorded per row), or not implementable (4); the 7B replay has 28 rows, 21 of them implementable. Confidence- and entropy-gated exits (e.g. Kadavath et al., [20]) fall in the last class: we cached hidden states and answer strings but not token logits, so replaying them would require new generation, and we do not approximate them. Two step-level rules (ES-CoT, answer-consistency) never fire at their published hyperparameters because our 512-token forcing grid is coarser than a reasoning step; this is a limitation of our cache, not evidence against those methods, and grid-rescaled variants are reported as our rescaling. Outcome: 0 methods beat the full budget at lower cost; the accuracy gains that survive bootstrap are all multi-sample and cost 3–5× the tokens. Against a token-matched fixed truncation our alarm is within +0.3 points—i.e. at equal compute it is not better than truncating, which we regard as the sharpest negative result about our own controller.
Self-consistency inherits the survivorship structure. Majority-vote and best-of- budget curves over the resampled MATH chains ( plus the original) and AIME chains (), with endpoints pre-registered in the supplement: the vote curve never declines within cohorts beyond noise anywhere on the grid, and over 2048 4096 its pooled within-cohort term is +12.0 points [+4.6, +20.2] against composition −12.0 [−17.7, −7.3] (problem-level bootstrap, 2000 resamples); best-of- shows the same structure (+5.4 within, −9.3 composition). Thus changing-cohort selection also affects vote curves; this does not establish shared adverse drift or exclude benefits from voting at other operating points. Caveats: 80+30 problems give wide intervals, and the AIME split is nearly degenerate (one problem leaves the survivor set by the cap).
Alarm-threshold sensitivity. Sweeping the alarm’s calibration percentile over {50, 60, 70, 75, 80, 85, 90, 95, 99} with everything else fixed: the accuracy-vs-tokens trade-off is monotone at both scales and no threshold improves accuracy over the full budget. The sweep’s nominal best (99th percentile: +0.0 points at 1.5B, +0.0 at 7B) is degenerate—it ties the full budget by (almost) never firing—so the pre-registered 90th-percentile operating point is not beaten by any non-trivial alternative, and the absence of an observed accuracy-improving alarm operating point persists across this threshold sweep.
Synthetic mechanism validation across regimes. One-factor-at-a-time around the Exp 0 base configuration (drift strength, innovation noise, horizon, drift onset, critic training-sample size; 19 grid points × 10 seeds): Doob compensation recovers a median 98.6% of injected accuracy loss (range 36.9–99.5%), robust down to 500 critic-training problems (Figure 12). The one failure mode is low innovation noise (, recovery 36.9%): beliefs saturate near 0/1, where probability-scale drift vanishes and the compensator cannot flip wrong-side saturated paths—the same saturation pathology that defeats symmetric magnitude penalties (Sec. N.3), here appearing as a limit of compensation itself.
Figure 12. Recovered fraction of injected accuracy loss under Doob compensation, varying one factor at a time around the Exp 0 base configuration (marked); error bars are s.e. over 10 seeds. Recovery is high across all factors except very low innovation noise, where beliefs saturate.
I Observation Histories, Loss Hazards, and Stopping Gain
These consequences of conditional expectation clarify what the measured conservation property can mean; they are not proposed as new projection or optional-stopping theorems.
Proposition 19 (An observation-calibrated verdict can drift). Let , , and . The observation fields need not be nested. Write and , where . Then
If , the conditional drift is zero. Without nesting, exact observation-level calibration alone does not imply this conclusion.
Proof. The first identity substitutes the two conditional expectations and uses . For the second, by . Conditional expectation contracts norms. Nesting gives the final claim by the tower property. ∎
In particular, is a filtration and is a bounded martingale, even if . Retained observation history is therefore another sufficient route to an exact verdict null; full-state sufficiency is not necessary. This does not make a learned history probe exact, nor does it identify why the real-model control is nonzero.
Exact four-path control. Draw and a signal with . At the observation is ; at it is constant; at it is . The full history retains throughout. Conditional on , the full verdict is and the compressed verdict is ; for they are and . Every compressed value is exactly . Its first conditional increment is or , while its second has zero conditional mean given the current constant observation. Averaged over the two steps, predictable squared energy is , compared with zero for the full verdict. Retaining at restores the null. All four path probabilities are enumerated with exact rational arithmetic. This is a possibility result about measurement, not evidence that forgetting causes the LLM verdict residual.
Proposition 20 (Binary commitments and loss hazards). Let be the exact correctness of the fixed greedy stopping operator, adapted to . Set and . Then
Proof. Condition on or in . Only the latter case permits negative drift; squaring gives the two identities.
Thus the one-sided objective penalizes squared loss hazard at correct states, whereas the symmetric objective also penalizes repair at incorrect states. A binary martingale is constant almost surely; a submartingale can repair errors while making the correct set absorbing. These statements do not supply an observable correctness oracle or a tested training method.
Corollary 21 (Adverse energy bounds stopping gain). Under Cor. 3, define . For any admissible ,
Proof. The stopping identity bounds the gain by . Cauchy–Schwarz on the product measure of paths and pre-termination time indices bounds this by . Its total measure is ; ensures integrability. The gain is also at most one.
For binary commitments, equals the expected number of correct-to-wrong transitions. The reported early-correct/final-wrong frequency is not this count: it omits later onset and losses subsequently repaired, so it cannot be inserted into this bound.
Protocol-Matched Continuation Follow-Up
Design and selection. This post-hoc follow-up was specified in an internal plan after the primary results were known, with the sample fixed before collecting the new outcomes. All 12 historical wrong-ending 1.5B chains with a correct nonterminal checkpoint at or before 2048 are included, across all five MATH levels. For each first-correct checkpoint, one unique right-ending chain with a correct answer at that same checkpoint is selected, with an exact difficulty match. A seeded shuffle followed by minimum level-distance matching gives the 12 pairs in Table 4; no new outcome enters selection. All 24 immediate greedy answers reproduce the archived correct label.
Table 4: All matched pairs. IDs index MATH-500; the first ID is the historical wrong-ending case and the second its right-ending control. Every immediate answer is correct. Brackets are per-prefix 95% Wilson intervals for continuation success, not intervals for population prevalence.
| Case/control | Level | Checkpoint | Case correct | Interval | Control correct | Interval |
| 9/109 | 5 | 2048 | 1/8 | [0.02, 0.47] | 8/8 | [0.68, 1.00] |
| 101/63 | 5 | 1536 | 6/8 | [0.41, 0.93] | 8/8 | [0.68, 1.00] |
| 120/57 | 2 | 1536 | 7/8 | [0.53, 0.98] | 8/8 | [0.68, 1.00] |
| 126/461 | 4 | 1024 | 2/8 | [0.07, 0.59] | 6/8 | [0.41, 0.93] |
| 128/471 | 3 | 512 | 0/8 | [0.00, 0.32] | 8/8 | [0.68, 1.00] |
| 238/459 | 5 | 1536 | 7/8 | [0.53, 0.98] | 8/8 | [0.68, 1.00] |
| 274/328 | 5 | 2048 | 8/8 | [0.68, 1.00] | 8/8 | [0.68, 1.00] |
| 326/35 | 2 | 2048 | 1/8 | [0.02, 0.47] | 8/8 | [0.68, 1.00] |
| 381/313 | 5 | 512 | 0/8 | [0.00, 0.32] | 8/8 | [0.68, 1.00] |
| 382/232 | 5 | 2048 | 2/8 | [0.07, 0.59] | 8/8 | [0.68, 1.00] |
| 398/73 | 4 | 512 | 6/8 | [0.41, 0.93] | 8/8 | [0.68, 1.00] |
| 493/323 | 5 | 1536 | 8/8 | [0.68, 1.00] | 8/8 | [0.68, 1.00] |
Table 4.
Protocol fidelity and runtime. The cached R1-Distill-Qwen-1.5B model is run in FP16 on the same 12 GB RTX 4070 Ti. Prompt tokens are reconstructed with the original chat template; continuations start from the original generated prefix token IDs, without decoding and retokenizing that prefix. At checkpoint , each of eight continuations has new-token allowance, temperature 0.6, and top- 0.95, ending at EOS or the cap. Batches contain four draws; a deterministic per-prefix/per-batch seed derived from 20260907 permits exact resumption within the recorded runtime. The frozen manifest records input and source hashes, model revision, sampling configuration, and selected IDs. Runtime is PyTorch 2.13.0+cu126 and Transformers 5.15.1; cross-version bitwise reproduction of historical generation is not assumed.
Immediate forcing and terminal forcing reproduce the original text-based operator: close the thought if necessary, append the boxed-answer cue, and decode greedily for at most 48 tokens. At termination, a boxed answer after the closing thought is used if present; otherwise the forcing operator is applied. Forced outputs use the original brace-closing parser and symbolic verifier. A strictly balanced-box parser is also evaluated and changes no correctness label among the 192 continuations. No continuation lacks a scored answer. Terminal forcing is used in 85/96 case and 28/96 control draws; respectively 11/96 and 68/96 reach EOS. Thus the fixed-budget protocol still involves substantial censoring, but unlike the historical branching diagnostic it does not score a missing unforced answer as an automatic error. Raw tokens, answer strings, scoring flags, prompt/generation token counts, and timings are retained. Generation of the continuations and their terminal forcing uses 405,150 tokens including EOS, excluding the immediate preflight checks; this is not a matched FLOP or wall-clock policy comparison.
Estimates and uncertainty. For arm and pair , let be terminal correctness of continuation . The fixed immediate answer is correct at every selected prefix, so
Thus prefixes have equal weight; the 192 draws are not 192 independent problems. The contrast is descriptive of these selected cohorts, not a randomized treatment effect. Each arm has 12 prefixes and 96 new continuations. Immediate correctness is one at every prefix; continuation success is 48/96 in cases and 94/96 in controls. The equal-prefix mean gaps are 50.0 (95% prefix-bootstrap interval [28.1, 71.9]) and 2.1 ([0.0, 6.2]) percentage points. The case-minus-control contrast is 47.9 points ([27.1, 69.8]). The 5000-resample paired bootstrap draws entire matched pairs and recomputes the two means; it summarizes pair-level sampling variation rather than providing an exact population interval. Per-prefix Wilson intervals in Table 4 expose the limited precision of eight Monte Carlo draws. Conditional plug-in Monte Carlo standard errors of the two arm means are 3.43 and 1.36 percentage points, but can understate uncertainty at all-success/all-failure prefixes; the Wilson intervals should not be replaced by zero uncertainty there.
Specifically, NumPy’s default random generator with seed 20260907 draws 5000 vectors of 12 pair indices with replacement. The same indices are used for both arms, and the reported limits are the 2.5th and 97.5th percentiles. Draws are not independently resampled inside prefixes. These descriptive intervals therefore do not separately propagate all within-prefix Monte Carlo uncertainty or confer coverage for an outcome-independent population. For an arm with success estimates , the reported conditional plug-in Monte Carlo standard error is . The per-prefix Wilson intervals use eight Bernoulli trials and , with denoting the standard normal distribution function. These summaries quantify different sources of variation and are not interchangeable.
Scope of the inference. At a specified prefix, the matched success gap estimates expected remaining adverse drift by Eq. (2). It does not identify the mechanism of a particular flip. Because the cohorts were selected using archived final outcomes, this follow-up does not preserve the 3.4% prevalence, estimate loss at an outcome-independent sample of prefixes, or validate a deployable stopping policy. Difficulty and checkpoint matching do not remove all differences between problems. The historical cross-protocol diagnostic in App. H is preserved separately; its mean continuation success is not pooled with this experiment.
Whole-Problem Refit Sensitivity
This post-hoc diagnostic tests sensitivity to the fitted measurement pipeline, not a new behavioral intervention. Each model uses its original layer-21 states and problem-disjoint outer probe/critic/evaluation split. Logistic regularization is fixed at , rather than retuned by the original cross-validation grid. Within the probe role, the original 20% calibration problems remain disjoint from probe fitting. A fixed 15% of eligible critic-role problems form a disjoint validation set, replacing the original step-level validation partition. Ten seeds, 20260907–20260916, independently resample whole problems with replacement within each development role and final-label stratum, then refit both probes and both critics. The evaluation problems never enter fitting, calibration, early stopping, or model selection.
Probe fitting uses logistic regression (maximum 2000 iterations) and temperature calibration. The critic retains the two-layer 256-unit MLP, dropout 0.1, AdamW with learning rate and weight decay , batch size 1024, maximum 200 epochs, and validation-NLL early stopping with patience 12. Dense paragraph increments and normalized token position follow the primary pipeline. Every fitted pipeline is evaluated on the same held-out problems using ; the two readouts use identical evaluation steps. Complete development resampling lists, probe AUC/ECE/Brier scores, temperatures, critic validation diagnostics, and per-step held-out predictions and targets are saved.
Commitment exceeds verdict in 4/10 and 1/10 refits, respectively. The mean gaps are −0.088 and −0.029; Table 5 retains all fits, including the negative 1.5B commitment score in refit 2. That probe has temperature 101.02, ECE 0.020, and increment variance , while its critic predicts mean increment 0.0137 against observed mean 0.00017 and performs worse than the zero predictor in the variational score. This is an observed sensitivity, not a reason to discard the run or evidence for a unique cause. A small ECE does not guarantee drift estimation quality. With ten development refits and fixed evaluation problems, these results are not a reliable 95% interval for full-pipeline uncertainty. They qualify the original single-fit ordering without replacing its numerical results or resolving the real-model verdict null.
Table 5: All ten refits of the diagnostic variant. C/V denote commitment/verdict scores ; gap is C minus V. Ranges across these refits are sensitivity summaries, not confidence intervals.
| Refit | 1.5B C | 1.5B V | 1.5B Gap | 7B C | 7B V | 7B Gap |
| 1 | 0.2201 | 0.2332 | -0.0130 | 0.2254 | 0.2342 | -0.0088 |
| 2 | -0.6384 | 0.2512 | -0.8896 | 0.1837 | 0.2427 | -0.0590 |
| 3 | 0.2372 | 0.2553 | -0.0180 | 0.2403 | 0.2161 | 0.0242 |
| 4 | 0.3045 | 0.2790 | 0.0255 | 0.2193 | 0.2546 | -0.0353 |
| 5 | 0.2449 | 0.2539 | -0.0090 | 0.2161 | 0.2437 | -0.0276 |
| 6 | 0.2194 | 0.1964 | 0.0230 | 0.1857 | 0.1862 | -0.0005 |
| 7 | 0.2702 | 0.2460 | 0.0242 | 0.1804 | 0.2774 | -0.0970 |
| 8 | 0.1921 | 0.2532 | -0.0611 | 0.1857 | 0.2227 | -0.0370 |
| 9 | 0.2999 | 0.1953 | 0.1047 | 0.2368 | 0.2458 | -0.0090 |
| 10 | 0.1708 | 0.2347 | -0.0639 | 0.2095 | 0.2484 | -0.0389 |
Table 5.
Supplement Guide and Reproduction Boundaries
The unified supplementary_code.zip contains the primary audit and the separately versioned follow-up, together with archived manuscript sources and audit figures. App. M adds no experimental results. The root README gives an artifact index and commands. The original result files and numerical macros are retained; new measurements are isolated under results/research_20260907/ and paper/followup_numbers.tex. Internal plans record when designs were fixed, not external registration or independent replication.
| Claim or appendix | Evidence and verification route |
| Primary audits; App. H | Original results/, results7b/, results_qwen3/, results_llama8b/ and results_phi4mini/ JSONs; code/make_numbers.py regenerates the historical macro file. |
| Observation control; App. I | code/research_exact_control.py enumerates the four paths with rational arithmetic; exact_control.json stores the 29 checks. |
| Matched continuations; App. J | matched_manifest.json, matched_runtime.json, matched_immediate.jsonl, matched_draws.jsonl, and matched_summary.json; all are in the follow-up results directory. |
| Refits; App. K | refit_config.json, role lists, 20 individual JSON/NPZ output pairs, and refit_summary.json; NPZ files contain held-out targets, critic predictions, and problem IDs. |
Table 1 (PDF p. 34).
Model-free verification. The archive provides a standard-library SHA-256 integrity check. From a fresh extraction, run:
python code/verify_supplement.py
Adding --evidence requires NumPy and recomputes the matched success rates, paired-bootstrap intervals, per-prefix Wilson intervals, token totals, parser-label agreement, and all saved refit scores. It also checks frozen source hashes, problem-role separation, numerical follow-up macros, and the exact finite-state construction. Checks run in a temporary copy and leave the packaged evidence unchanged. They verify arithmetic and consistency, not the validity of the original answer verifier or independent replication of model behavior.
Additional assets and computation requirements. The archived 1.5B rollout and forced-answer files needed for the matched manifest are included. Model weights, dataset caches, and the large primary hidden-state arrays are not. New continuations require the recorded model revision, dataset, generation runtime, and GPU resources. Repeating the refits requires the original layer-21 state arrays and index files for both models; saved held-out predictions suffice only for score recomputation, not refitting. The historical README documents the extraction pipeline. Different runtime versions can change generated trajectories and therefore all downstream fitted quantities. Independent reruns must use a new output location or version, preserving the frozen evidence.
Evidence that remains absent. The matched experiment does not sample prefixes independently of archived outcomes, and the refit diagnostic is neither an original-tuning replication nor a full-pipeline confidence interval. Neither experiment supplies a held-out deployment benefit or identifies the cause of the verdict residual. The historical cross-protocol branching study and randomized-refit ranks remain archived diagnostics with their original limitations; they are not silently replaced by the new follow-up.
Beyond Single-Chain Reasoning: Scope and Testable Extensions
The central distinction is between forecasting a fixed protocol’s eventual success and evaluating the artifact that could be delivered now. The unit of computation need not be a token or a single-model reasoning segment. This section makes that extension explicit, without claiming empirical validation outside the settings in § 4.
A protocol-level formulation. Fix a finite horizon and a complete stochastic protocol, including its history-dependent choices of models, tools, messages and updates. Let be its nested full-history filtration, the score obtained by a specified deliver-now operator, and its current commitment. Fix the scoring rule and evaluation distribution. Let be the terminal score and require a common terminal boundary , with the terminal evaluation included in . Define
The tower property and conditional telescoping give
For a history-measurable stopping time , the same argument yields . Early termination can be represented by freezing the score thereafter. These are applications of the existing identities to a bounded score and a joint protocol, not new mechanisms or guarantees of improvement. They need no independence among agents or Markov assumption; a state-based Poisson equation additionally requires a sufficient Markov state. If the stopping readout and terminal score do not satisfy the common-boundary condition, a terminal mismatch term must be retained.
Multi-step reasoning and iterative self-revision. For a single task, let the current artifact be the answer after each reasoning segment or completed critique–revision round. Self-Refine, for example, iteratively generates feedback and revises an output ([30]). The commitment is the current artifact’s externally scored quality; the verdict forecasts the final artifact under the remaining refinement protocol. More elaborate feedback, increased confidence, or a changed answer does not alone identify their difference. A direct audit samples prefixes without selecting on their eventual outcomes, scores the immediate artifacts, and branches fresh continuations under the same protocol. This estimates remaining improvement or erosion at those states, while repeated runs quantify sampling variability. The theory permits beneficial revision and does not prescribe zero drift: positive drift is precisely the improvement that refinement seeks.
Multi-agent collaboration and debate. Multiagent debate uses several model instances to exchange and revise reasoning before producing a common answer ([11]). Here contains the joint interaction history, including relevant private states and tool observations, and the deliver-now operator specifies how the current messages or candidate answers become one system output. Equation (8) concerns that system output, not the average confidence or agreement of its members. A transcript-only probe may be lossy; even an exact forecast conditioned on one agent’s local information need not be a martingale relative to the larger joint filtration. Thus the information set must be specified before interpreting predictable updates as a control failure.
Communication can change both dependence and expected answer quality. The independent-ensemble innovation scaling discussed in App. § 12 therefore does not automatically extend to interacting agents. Neither consensus nor correlated errors alone determine the sign of remaining drift. A useful audit fixes the final-answer aggregation rule, measures quality and agreement separately at each round, and resamples continuations from the same joint state. Comparisons among communication protocols require their own verdicts and compute accounting. If only unfinished problem instances remain in a later-round average, the survivor decomposition applies to that changing problem cohort; replacing an agent inside a fixed problem is not itself this population-selection effect.
Recursive improvement of the system itself. Improving an answer is distinct from modifying the machinery that generates answers. The Darwin Gödel Machine, for example, modifies agent code and evaluates the resulting agents on coding benchmarks ([50]). To formulate an analogous audit, the state must include the evolving code, parameters, memory, selection history and update procedure. The current artifact is now the system version that would be deployed if development stopped, and commitment measures its performance under a fixed evaluation distribution. A fixed meta-protocol may contain adaptive, even self-modifying, updates: “fixed” specifies the overall probability law, not frozen model parameters.
Under that specification and the common terminal boundary, Eq. (8) still concerns the forecast of one terminal evaluation. It neither fixes the performance of intermediate versions nor rules out recursive capability improvement. Externally changing the update procedure or evaluation target changes the comparison; a verdict for the old protocol cannot simply be carried over. A frozen probe also need not retain its meaning when the underlying representations change (App. § 15). A credible empirical extension would separately evaluate successive deployable versions on held-out tasks, account for selection and development compute, and use replicated update trajectories where feasible. Our present experiments establish none of these system-improvement claims.
Implication for LLM development. Across these settings, the framework supplies a common comparison target: the externally evaluated artifact available now versus the artifact expected after additional computation. It separates measurement from mechanism and decision quality. This can inform how future evaluations, process supervision and adaptive-compute methods are designed, but it does not establish that our present estimators or stopping heuristics transfer successfully to these systems.
N Probe Diagnostics, Synthetic Validation, and Stopping Replays
This appendix collects the learned-sensor and policy experiments. The main-text behavioral results use scored answers directly; the analyses here evaluate hidden-state measurement and cached-checkpoint control.
Readout estimation
Probe estimation and observation loss. Identification specifies the exact quantity; estimation must recover it. For a lossy tap , a critic observes only ; conditional Jensen gives . This population bound concerns the same readout, not an arbitrary learned substitute. If , its full-filtration drift differs by , at most when uniformly (Prop. 14). AUC and binned calibration error do not establish that bound.
The commitment probe predicts current forced-answer correctness; the verdict probe predicts terminal correctness. An exact verdict conditioned on a nested observation history is a martingale, even without full-state sufficiency. Current-state taps need not retain that history: even can have predictable increments (App. I). Thus calibration alone does not establish the verdict null. Our control remains unresolved (Sec. N.6); matched behavioral continuations provide a probe-independent check on loss.
Variational objectives and critic implementation
Training implication: stability is not improvement. The decomposition also distinguishes which changes a regularizer suppresses. A quadratic increment penalty charges both predictable change and innovation: (Prop. 15). Because is an projection, with , and penalizing this variational value—requiring no new labels—can hedge the readout, blind to innovation variance—vanishes iff the chain is state-calibrated (). This symmetric objective penalizes beneficial predictable improvement too. For erosion alone, a sign-constrained variant targets , where (App. § 14); we derive, but do not experimentally evaluate, this variant. Suppressing movement, drift, and erosion are different objectives (Apps. § 14–§ 15); only synthetic validation is provided (Sec. N.3).
The critic as a sensor. Regress frozen-model probe increments on and accumulate . With a sufficient tap and uniform critic error , compensating the exact readout by
| R1-Qwen 1.5B | R1-Qwen 7B | |
| Commitment score | 0.284 [0.263, 0.305] | 0.221 [0.200, 0.245] |
| Verdict score | 0.249 | 0.244 |
| Commitment minus verdict | 0.036 [+0.007, +0.063] | −0.023 [−0.050, +0.007] |
Table 6. Primary probe scores. Brackets are conditional chain-bootstrap 95% intervals. These scores characterize learned readout increments.
gives (Prop. 11, App. § 12). This is a readout-error bound, not a guarantee of improved answers or of accurate learned probes. Our diagnostic drift alarm stops when , using the last checkpoint answer; is pre-registered as the 90th percentile of over stay-correct critic-split chains. Accumulated past drift is a heuristic signal, not the remaining-drift oracle above.
Minimal audit recipe. (i) Elicit answers at fixed intermediate budgets. (ii) Measure correct-to-wrong prevalence with repeated samples within problems. (iii) Separate within-cohort and composition changes. (iv) Test a sensor or stopping rule where behavioral checks show expected loss at candidate states; aggregate deterioration is not required. The first three steps assess measurement; the fourth assesses control (App. § 15).
Experiment 0: apparatus calibration with known injected drift
A binary Gaussian-evidence process supplies known drift: its unperturbed posterior on a fixed hypothesis is a martingale, while a constant log-odds lean injects probability-scale drift. With evidence separation 0.35, noise SD 1, and 60 steps, the critic recovers it (slope 1.04, at ), reports no false drift at zero, and has matched-sample detection floor 0.05. Under decaying evidence, accuracy loses 15.8 points after peaking at ; compensation restores it to 85.3%. The variance-blind penalty preserves gain and accuracy over the tested values, whereas collapses both. This calibrates the apparatus, not real-model decision accuracy; the 19-configuration sweep and late-onset test are in App. § 16.
Primary probe measurements
Predictable probe increments, with positive mean predicted drift. Here and refer to the probe increments. For any fixed critic , the population value is a lower bound on state-conditional drift energy. Its held-out sample estimate is not itself a guaranteed finite-sample bound. We report the normalized score : 0.284 [0.263, 0.305] at 1.5B and 0.221 [0.200, 0.245] at 7B, with chain-bootstrap intervals conditional on the fitted probes and critics. This variance normalization is not a bounded fraction of increment energy. Mean predicted increments are positive in all four label groups, with stay-correct CIs excluding zero ([+0.0032, +0.0079]/[+0.0111, +0.0202]). The critic also predicts positive mean drift for overthink-wrong chains and cannot resolve their rare erosion. Evidence for aggregate net improvement comes from the behavioral with-choice term (§ 4.3); it does not exclude adverse drift in particular states. Randomized-refit diagnostics and their inferential limitations are in App. § 16.§ 1.
Held-out stopping-policy replays
We fix thresholds on the critic split and replay policies using only information available at each decision, on held-out evaluation chains ( at 1.5B/7B). Table 7 compares the drift alarm, two-consecutive-answer agreement, checkpoint voting, and full budget. Charged tokens are generated length plus 48 per forcing branch, with no terminal forcing charged to the archived full-budget baseline; prefill and critic costs are excluded. This is not a wall-clock or FLOP comparison.
None of the implementable configurations (44/48 at 1.5B; 21/28 at 7B), or alarm thresholds from the 50th to 99th healthy-chain percentile, improves accuracy at lower charged token cost. Token-saving rules lose −0.9 to −45.5 points at 1.5B and −1.3 to −47.5 at 7B; only 4/21 and 3/16 intervals reach zero. The alarm fires on 13%/14% of stay-correct chains and catches none/one of the two evaluation overthink-wrong chains. App. § 16.§ 1 documents native-method deviations and the full replay matrix. These results assess our cached-checkpoint protocol, not the cited methods under their native execution.
The negative result distinguishes a quantity’s theoretical relevance from our ability to estimate and act on it. It establishes no general impossibility of beneficial early exit.
Validity: the unresolved verdict control, and robustness
The tower-property control remains unresolved. Score ordering changes across scales and taps (Table 6). Ten whole-problem bootstrap refits per model, using disjoint fitting, calibration and validation problems and a fixed evaluation split, give commitment above verdict in only 4/10 and 1/10 runs. This is tuning sensitivity, not a confidence interval (App. § 19). Verdict AUCs (0.747/0.735) also miss the prespecified 0.75 bar. An exact verdict on nonnested observations can drift (App. I); adding a cumulative history summary leaves negative gaps (), but does not guarantee nested information. Neither test separates observation loss from estimation error (App. § 16.§ 1).
Sensitivity of the behavioral findings. Absent the forcing string, 69% of 160 truncated prefixes emit no answer at all—the reason an operator is needed; where an unforced continuation does answer, it agrees with the forced answer on correctness in 85.7% of cases, and greedy forcing exceeds sampled by +0.7 points ([, +1.8]). The overthink-wrong count is grid-dependent (5/5/7 at 1024/512/256 tokens, matched ), so is specific to the 512-token grid. Prevalence stays at 1–4% under every final-outcome definition and at temperature 0.9, the within-cohort term stays positive, and a linear critic retains the drift signal; on AIME-2024 (120 chains) no chain is overthink-wrong, though only 12 are ever correct early (App. § 16). Table 3 and App. § 16.§ 1 give each check with its interval.
| Pre-registered | outcome |
| Gate 0: apparatus calibration | passed (slope 1.04, 0.82; matched floor 0.05) |
| Gate 1(a): 30 overthink-wrong chains | failed (10/367) committed 7B escalation: 9/262 |
| Gate 2: recovery at some | not met: no tested alarm recovers accuracy; this does not exclude local adverse drift |
| Verdict rule (viii) | not triggered: verdict AUC 0.747/0.735 does not clear 0.75 (App. § 16) |
| Robustness | outcome |
| Forcing operator | of 160 truncations, 69% yield no natural answer; on the rest, agreement 85.7%; greedy vs. sampled +0.7 pts [, +1.8] |
| Checkpoint grid | density inflates overthink-wrong: 5 (1024) = 5 (512) 7 (256) of 240 |
| Label definitions | prevalence 1–4% under all final-outcome variants |
| Critic capacity | linear critic keeps the signal at 1.5B ( 0.16 vs. 0.26) |
| Alarm threshold | no percentile in beats full budget (95th: [, +0.0] pts) |
| Tap richness | ordering holds only at the 1.5B primary tap; unresolved overall |
| Self-consistency | vote curves inherit survivorship (+12.0 within / comp.) |
| Third model | Qwen3-1.7B (non-R1-distilled, Qwen lineage): overthink-wrong 3.0% [1.7%, 5.3%]; survivorship reproduces |
| Other base families | R1-Distill-Llama 8B: overthink-wrong 2.2% [1.1%, 4.2%], within +16.6 / comp. ; Phi-4-mini: 2.2% [1.1%, 4.2%], +9.1 / |
| Decoding temperature | at 0.9: overthink-wrong 1.2% (vs. 2.1%); within-cohort still positive |
| Synthetic sweep | median recovery 98.6% over 19 configs; failure mode: belief saturation |
Table 3. Full pre-registered gate and post-hoc robustness matrix; quantities 1.5B/7B. The pre-registered gates were fixed before the corresponding real-data runs; post-hoc analyses are distinguished below and in the text.
O Analyses Added During Review
When each analysis was specified. Table 8 lists every analysis by the time its design was fixed. “Before results” means written in a dated internal plan before any real-data result was computed; the plan is internal, not an external registry. All three categories fixed their design before generating their own data, and every outcome is reported.
Outcome-blind continuations. The state set contains every primary 1.5B chain with a correct non-terminal forced answer at or before 2048 tokens, at its first such checkpoint: 379 states across all five MATH levels (255 at levels 3–5). The rule reads no final outcomes; the 12 states of the matched study (App. § 18) are among them. The protocol copies the matched study: original prefix token ids, temperature 0.6, top- 0.95, the remaining budget , the post-think boxed answer if present and otherwise a greedy forced answer of at most 48 tokens. Four continuations were drawn per state. Before any continuation, the immediate greedy answer reproduced the archived correct answer at every state. Intervals resample problems. Continuations end correct in 96.4% of draws; the mean loss is 3.6 points ([2.2, 5.2]). By state, 348 keep the answer in all four draws, 31 lose it at least once, 13 in at least two, and 3 in all four. The strict balanced-box parser gives the same scores.
Generic doubt and misleading hint. Both manipulations use the same 96 states, a seeded random subset of the outcome-blind set, and are paired with the outcome-blind draws at those states. The generic doubt appends “Wait, I made a mistake in an earlier step, so this answer is incorrect. Let me redo the solution carefully.” A forced answer right after it is correct in 91.7% of states; continuations end correct in 95.8% against 94.8% without it (paired difference +1.0 points, [−1.0, +3.1]). The misleading hint appends “Wait, I made an error earlier. Rechecking carefully, the correct answer should be .” The wrong answer is the most frequent wrong answer the 1.5B model gave on that problem in our runs (60 states), otherwise the gold answer plus one for integer answers (30 states); 6 states have neither and are excluded. A forced answer right after the hint is correct in 32.2% of states and equals in 64.4%. Continuations end correct in 93.6% against 94.4% without the hint, and end at in 5.0%.
Problem concentration. With eight fresh chains per problem (367 problems), the number of early-correct/final-wrong chains per problem has 2.6 times the variance expected if all problems shared one rate (permutation test against that null, ). The intraclass correlation is 0.23: about a quarter of the variation lies between problems. The transitions occur in 57 problems, against 89 expected under a common rate, and the ten most affected problems produce 40% of them. Restricting to problems whose chains ever give a correct early answer leaves these figures essentially unchanged.
Difficulty and precision. On the 262 level-4–5 problems used for the 7B model, the 1.5B FP16 audit gives 9/262 = 3.4% ([1.8, 6.4]) early-correct/final-wrong chains, the same count as the 7B model, with final accuracy 69.8%. Rerunning the 1.5B model in NF4 on the same problems gives 8/262 = 3.1% ([1.6, 5.9]) and final accuracy 71.4%. The 7B model loaded in 8-bit (bitsandbytes LLM.int8) on its 262 problems, seed 0, gives 8/262 = 3.1% ([1.6, 5.9]) with final accuracy 78.6%; its decomposition is in Table 1.
Second benchmark and a 2026 model. OlympiadBench: from the text-only, English, open-ended competition-mathematics split, we kept problems with a single numerical or expression answer and no unit (558), and drew a seeded sample of 367 to match the MATH-500 primary set. The 1.5B FP16 model ran with the primary protocol (seed 20261108). It reaches final accuracy 48.8%, with 14/367 = 3.8% ([2.3, 6.3]) early-correct/final-wrong chains. Qwen3.5-2B ran in FP16 on the 367 level-3–5 MATH-500 problems with its chat template’s thinking mode enabled and otherwise the primary protocol (seed 0). It reaches final accuracy 66.2%, with 7/367 = 1.9% ([0.9, 3.9]). Decompositions are in Table 1.
Longer budget. Fresh 1.5B chains (seed 20261107) on the 367 level-3–5 problems ran to a 16,384-token cap with forced answers every 1024 tokens. Because generation is left to right, forced answers at checkpoints up to 4096 tokens are unaffected by the cap, so the same chains also give the 4096-token audit; a chain still running at 4096 tokens counts as capped there. At the two caps, 44% and 11% of chains are capped, final accuracy is 75.7% and 80.9%, and the early-correct/final-wrong rate is 2.5% [1.3, 4.6] and 3.3% [1.9, 5.6]. The coarser grid finds no more transitions than the 512-token grid (§ 4.3), so these rates are comparable with each other rather than with the primary rate. Over 4096 to 16,384 tokens the decomposition gives within-cohort [, ] and composition points.
Oracle headroom. The oracle knows the gold answer and stops each held-out chain at its cheapest correct checkpoint, otherwise running to the full budget. It is an upper bound for any rule that stops on the information in a chain, under the same answer procedure and token accounting.
References
- [1]Aradhye Agarwal, Ayan Sengupta, and Tanmoy Chakraborty. First finish search: Efficient test-time scaling in large language models. arXiv preprint arXiv:2505.18149, 2025.arxiv.org/abs/2505.18149
- [2]Pranjal Aggarwal, Aman Madaan, Yiming Yang, and Mausam. Let’s sample step by step: Adaptive-consistency for efficient reasoning and coding with LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 12375–12396, 2023.arxiv.org/abs/2305.11860
- [3]Xingyu Chen, Jiah ao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do NOT think that much for 2+3=? on the overthinking of o1-like LLMs. arXiv preprint arXiv:2412.21187, 2024.arxiv.org/abs/2412.21187
- [4]Yuan Shih Chow, Herbert Robbins, and David Siegmund. Great Expectations: The Theory of Optimal Stopping. Houghton Mifflin, Boston, 1971.
- [5]Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, Nicholas Thumiger, Aditya Desai, Ion Stoica, Ana Klimovic, Graham Neubig, and Joseph E. Gonzalez. The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks. arXiv preprint arXiv:2502.08235, 2025.arxiv.org/abs/2502.08235
- [6]DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shiqing Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025.DOI
- [7]Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. In Advances in Neural Information Processing Systems, 2022.arxiv.org/abs/2208.07339
- [8]Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems, 2023.arxiv.org/abs/2305.14314
- [9]Nishanth Dikkala, Greg Lewis, Lester Mackey, and Vasilis Syrgkanis. Minimax estimation of conditional moment models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.arxiv.org/abs/2006.07201
- [10]Zhe Dong, Fang Qin, and Manish Shah. When does learning to stop help? a cost-aware study of early exits in reasoning models. arXiv preprint arXiv:2606.30852, 2026.
- [11]Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. arXiv preprint arXiv:2305.14325, 2023. URL https://arxiv.org/abs/2305.14325.
- [12]Mike Farmer, Abhinav Kochar, and Yugyung Lee. Empirical characterization of inference-time elicited probability transformations in large language models. arXiv preprint arXiv:2603.19262, 2026.arxiv.org/abs/2603.19262
- [13]Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Yonghao Zhuang, Yian Ma, Aurick Qiao, Tajana Rosing, Ion Stoica, and Hao Zhang. Efficiently scaling LLM reasoning with certaindex. arXiv preprint arXiv:2412.20993, 2024.arxiv.org/abs/2412.20993
- [14]Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 17(59):1–35, 2016.arxiv.org/abs/1505.07818
- [15]Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, 2024.
- [16]Zhonghao He, Tianyi Qiu, Hirokazu Shirado, and Maarten Sap. Martingale score: An unsupervised metric for Bayesian rationality in LLM reasoning. In Advances in Neural Information Processing Systems (NeurIPS), 2025.arxiv.org/abs/2512.02914
- [17]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021.
- [18]Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang. Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503.00031, 2025.arxiv.org/abs/2503.00031
- [19]Jie Huang, Xinyun Chen, Swaroop Mishra, Huaiqiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2310.01798
- [20]Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022.arxiv.org/abs/2207.05221
- [21]John G. Kemeny and J. Laurie Snell. Finite Markov Chains. Undergraduate Texts in Mathematics. Springer-Verlag, New York, 1976. Reprint of the 1960 Van Nostrand edition, with a new appendix.
- [22]SongEun Kim, Seungyoo Lee, Edwin Fong, Hyungi Lee, and Juho Lee. From drift to coherence: Stabilizing beliefs in LLMs. arXiv preprint arXiv:2606.17832, 2026.arxiv.org/abs/2606.17832
- [23]Greg Lewis and Vasilis Syrgkanis. Adversarial generalized method of moments. arXiv preprint arXiv:1803.07164, 2018.arxiv.org/abs/1803.07164
- [24]Huayu Li, ZhengXiao He, Siyuan Tian, Jinghao Wen, and Ao Li. Martingale foresight sampling: A principled approach to inference-time LLM decoding. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3522–3533, 2026a. doi: 10.18653/v1/2026.eacl-long.162. URL https://aclanthology.org/2026.eacl-long.162/.DOI
- [25]Jiakai Li, Ke Qin, Rongzheng Wang, Yizhuo Ma, Qizhi Chen, Muquan Li, and Shuang Liang. Stop when further reasoning won’t help: Attention-state adaptive generation in reasoning models. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026b.arxiv.org/abs/2606.15070
- [26]Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2306.03341
- [27]Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang, and Kan Li. Escape sky-high cost: Early-stopping self-consistency for multi-step reasoning. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2401.10480
- [28]Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2305.20050
- [29]Xin Liu and Lu Wang. Answer convergence as a signal for early stopping in reasoning. arXiv preprint arXiv:2506.02536, 2025.
- [30]Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2303.17651
- [31]Minjia Mao, Bowen Yin, Yu Zhu, and Xiao Fang. Early stopping chain-of-thoughts in large language models. arXiv preprint arXiv:2509.14004, 2025.
- [32]Dehai Min, Giovanni Vaccarino, Huiyi Chen, Yongliang Wu, Gal Yona, and Lu Cheng. Stop when reasoning converges: Semantic-preserving early exit for reasoning models. arXiv preprint arXiv:2605.17672, 2026.arxiv.org/abs/2605.17672
- [33]Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025.arxiv.org/abs/2501.19393
- [34]Qwen Team. Qwen3.5-2B. Model card, https://huggingface.co/Qwen/Qwen3.5-2B, 2026.DOI
- [35]Jürgen Schmidhuber. Learning factorial codes by predictability minimization. Neural Computation, 4(6):863–879, 1992. doi: 10.1162/neco.1992.4.6.863.
- [36]Adam S. Shai, Sarah E. Marzen, Lucas Teixeira, Alexander Gietelink Oldenziel, and Paul M. Riech- ers. Transformers represent belief state geometry in their residual stream. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
- [37]Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024.
- [38]Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models. Transactions on Machine Learning Research (TMLR), 2025.
- [39]Renliang Sun, Wei Cheng, Dawei Li, Haifeng Chen, and Wei Wang. Stop when enough: Adaptive early-stopping for chain-of-thought reasoning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), pp. 27250–27268, 2026.
- [40]Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024.
- [41]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023.
- [42]Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
- [43]Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Fei Sun, Huawei Shen, and Xueqi Cheng. The evolution of thought: Tracking LLM overthinking via reasoning dynamics analysis. arXiv preprint arXiv:2508.17627, 2025.arxiv.org/abs/2508.17627
- [44]Haoran Xu, Baolin Peng, Hany Awadalla, Dongdong Chen, Yen-Chun Chen, Mei Gao, Young Jin Kim, Yunsheng Li, Liliang Ren, Yelong Shen, et al. Phi-4-mini-reasoning: Exploring the limits of small reasoning language models in math. arXiv preprint arXiv:2504.21233, 2025.arxiv.org/abs/2504.21233
- [45]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
- [46]Renos Zabounidis, Aditya Golatkar, Michael Kleinman, Alessandro Achille, Wei Xia, and Stefano Soatto. Re-FORC: Adaptive reward prediction for efficient chain-of-thought reasoning. In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2511.02130
- [47]Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He. Reasoning models know when they're right: Probing hidden states for self-verification. In Conference on Language Modeling (COLM), 2025a.arxiv.org/abs/2504.05419
- [48]Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. Mitigating unwanted biases with adversarial learning. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (AIES), pp. 335–340, 2018. doi: 10.1145/3278721.3278779.
- [49]Dan Zhang, Min Cai, Jonathan Light, Ziniu Hu, Yisong Yue, and Jie Tang. TDRM: Smooth reward models with temporal difference for LLM RL and inference. arXiv preprint arXiv:2509.15110, 2025b.arxiv.org/abs/2509.15110
- [50]Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel machine: Open-ended evolution of self-improving agents. arXiv preprint arXiv:2505.22954, 2025c. URL https://arxiv.org/abs/2505.22954.
- [51]Shu Zhou, Rui Ling, Junan Chen, Xin Wang, Tao Fan, and Hao Wang. When more thinking hurts: Overthinking in LLM test-time compute scaling. In Findings of the Association for Computational Linguistics: ACL 2026, pp. 23967–23977, 2026. doi: 10.18653/v1/2026.findings-acl.1199. URL https://aclanthology.org/2026.findings-acl.1199/.DOI











