betaFeedback

WorksComputer Science

Early Stopping in Large Reasoning Models: A Survey and Empirical Audit of Efficiency and Reliability

Abstract

Large reasoning models can improve accuracy by generating longer reasoning traces, but some of this computation may be unnecessary. Early-stopping methods aim to stop once enough reasoning is available to answer. Evaluating these methods requires accounting for the cost of deciding when to stop and the reliability of the resulting answer. We evaluate different methods across five reasoning models on mathematical and scientific reasoning, code generation, and factuality tasks by using a shared rollout replay protocol. We introduce Target-Integrated Token Efficiency (TITE) to measure efficiency relative to the oracle across accuracy targets. We also examine correctness at later exit positions, transfer across models and tasks, post-stop reasoning continuation, and retained justification quality. In our experiments, correctness can decrease at later positions beyond what the fitted decoding-noise model predicts. Reliability checks also reveal continued reasoning after forced termination and weaker supporting reasoning in some correct early-stopping outputs. The implementation can be found in https://github.com/hung20gg/audit-early-stopping.

Introduction

Test-time scaling, or System 2, has become an important direction in large language models (LLMs). Instead of answering directly, recent large reasoning models (LRMs) advance the chain-of-thought capability (Wei et al., 2022; Wang et al., 2023), spend more computation at inference time by generating long reasoning traces before producing the final answer. The introduction of OpenAI o1 (OpenAI et al., 2026) and DeepSeek-R1 [12] have made this paradigm more visible by demonstrating strong reasoning ability through reinforcement learning and long reasoning traces. However, this improvement comes with a major drawback: reasoning models often consume far more tokens than standard direct-answer models, as LRMs may explore alternative solution paths, reconsider intermediate conclusions, repeat verification steps, or continue reasoning after the information needed for a correct answer

as LRMs may explore alternative solution paths, reconsider intermediate conclusions, repeat verification steps, or continue reasoning after the information needed for a correct answer has already been obtained [4] (Sui et al. 2025).

Early-stopping methods aim to address this problem by identifying when the current reasoning prefix is sufficient for producing a final answer. During generation, these methods inspect signals such as transition tokens, answer convergence, confidence, entropy, semantic redundancy, or agreement among probe answers [24] (Yang et al. 2026; Wang et al. 2026b). Once a stopping condition is satisfied, the method typically inserts or induces an end-of-thinking token, such as </think>, and force the model to produce its final answer.

The basic intuition is that if a model is already ready to answer correctly, additional reasoning may be unnecessary. However, observing a correct answer at one reasoning position does not imply that correctness will remain stable as reasoning continues. Secondly, a stopping signal that performs well for one model or task may fail when transferred to another. In addition, forcing a model to stop reasoning does not guarantee a clean transition to answer production. The model can continue reasoning in the answer channel or produce the correct conclusion from a prefix that does not yet contain sufficient justification. This raises an important evaluation question: How much of the available reasoning computation can existing early-stopping methods actually save without sacrificing accuracy?

In addition, several efficient-reasoning methods assume that answer correctness improves or remains stable as reasoning increases. TALE [13] states this monotonicity assumption explicitly to support binary budget search, while Efficient Long CoT Reasoning uses binary cutting to find the shortest valid prefix (Wang et al., 2025b). Thought Calibration similarly expects stopping risk to be monotonic (Wu et al., 2025). S-GRPO randomly samples exit positions and gives higher rewards to earlier correct exits, but does not test whether correctness is monotonic across positions [6]. [2] (Song et al. 2026) have shown that answer correctness generally improves with reasoning depth, while also documenting cases in which additional reasoning changes a correct answer into an incorrect one. [3, 7] set up experiments to point out the existent of optimal reasoning paths and show that continued reasoning can change an earlier correct answer into an incorrect. This raises a further question: Is stopping readiness sufficiently stable for methods to rely on monotonic reasoning progress?

These considerations lead to the following research questions:

  • RQ1 (efficiency): How close and generalized are existing methods to hindsight-oracle token allocation after verifier costs are included? (5.1)

  • RQ2 (stability): Does correctness at later exits remain stable beyond stochastic decoding variation? (5.2)

  • RQ3 (reliability): Do forced exits terminate reasoning cleanly and retain adequate justification? (5.3)

Our work makes four main contributions:

  • We introduce TITE to measure oracle-relative efficiency.

  • We test the continuation-stability and non-monotonic relationship analysis using randomized exit positions, revealing a challenge for early-stopping.

  • We evaluate the generalization of different methods across multiple model families and domains.

  • We extend the evaluation beyond final correctness by measuring post-stop reasoning continuation and retained-chain justification quality.

Early Exit Framework

Scope and Definition

Early-stopping is a method that reduce inference cost by terminating an ongoing reasoning trace before its natural completion. Given a prompt xx, a reasoning model MM generates a reasoning trace rTr_T and a final answer yfully^{\mathrm{full}}:

(rT,yfull)∼M(x),(r_T, y^{\mathrm{full}}) \sim M(x),

where TT is the natural end of the reasoning process. An early-stopping method selects a stopping position t<Tt < T, add end-of-thinking intervention τ\tau, and produces an answer from that prefix:

y(t)∼M( ⋅∣x,rt∘τ).y^{(t)} \sim M(\,\cdot\mid x, r_t \circ\tau).

A method is considered prefix-preserving when the reasoning tokens generated before position tt are identical to the corresponding prefix of the natural reasoning trajectory.

Generic Early-stopping Mechanism

A generic early-stopping method mm is represented by

(Sm,πm,Gm,τm)(S_m, \pi_m, G_m, \tau_m)

where SmS_m is a checkpoint function, πm\pi_m is a verifier that estimates whether the current prefix is sufficient for final answering, GmG_m is a stopping rule from verifier and τm\tau_m is the stopping intervention, such as an end-of-thinking token.

At reasoning position tt, the checkpoint function observes the current prefix rtr_t:

sm,t=Sm(x,rt),sm,t∈0,1.s_{m,t} = S_m(x, r_t), \qquad s_{m,t} \in0,1.

If sm,t=0s_{m,t} = 0, the model continues reasoning. If sm,t=1s_{m,t} = 1, the verifier evaluates the current state:

um,t=πm(x,rt).u_{m,t} = \pi_m(x, r_t).

The verifier output um,tu_{m,t} may be a confidence score, entropy value, tentative answer, agreement score, redundancy estimate, or another measure of reasoning sufficiency. Some verifiers can be computed directly from the reasoning trace, while others require temporary probe generations. The stopping decision is decided by

dm,t=Gm(um,t,rt),dm,t∈0,1.d_{m,t} = G_m(u_{m,t}, r_t), \qquad d_{m,t} \in0,1.

If dm,t=0d_{m,t} = 0, the reasoning continues. Else, the method applies τm\tau_m and generates the final answer:

ym∼M( ⋅∣x,rt∘τm).y_m \sim M(\,\cdot\mid x, r_t \circ\tau_m).

Algorithm 1 summarizes this process.

Algorithm 1 Generic Early-stopping Mechanism
1: Initialize reasoning prefix $r \leftarrow \varnothing$
2: for $t = 1$ to $B$ do
3: Generate the next reasoning token
4: $z_t \sim M(\cdot \mid x,r)$
5: Update the reasoning prefix
6: $r \leftarrow r \circ z_t$
7: if $S_m(x,r) = 1$ then
8: Compute verifier output
9: $u_{m,t} \leftarrow \pi_m(x,r)$
10: if $G_m(u_{m,t},r) = 1$ then
11: Apply the stopping intervention
12: $r \leftarrow r \circ \tau_m$
13: Generate final answer
14: $y_m \sim M(\cdot \mid x,r)$
15: return $(r,y_m,t)$
16: end if
17: end if
18: end for
19: $y_m \sim M(\cdot \mid x,r)$
20: $y_m$

Conditions for a Clean Early-stopping Solution

We define a clean solution through three conditions:

Correctness. The final answer produced from the selected reasoning prefix should be correct. With reference answer gg, let VV be the scoring function. The correctness condition is V(ym,g)=1V(y_m,g) = 1.

Efficiency. The method should reduce complete inference cost relative to natural full reasoning. Let CmC_m denote the total cost of method mm and CfullC_{\mathrm{full}} denote the cost of natural full reasoning and its final answer. The efficiency condition is Cm<CfullC_m < C_{\mathrm{full}}.

Clean termination. After the stopping intervention is applied, the model should transition from reasoning to final-answer production without continuing unresolved derivation, search, verification, or self-correction in the answer channel.

Assumption

Assumption of ideal KV cache implementation We do not account for the computational overhead associated with reading, loading, storing, or managing the KV cache. All results assume an ideal KV-cache management implementation with zero overhead, enabled by mechanisms such as cache reuse and an optimal cache-storage strategy. We report the total number of generated tokens, including any additional tokens required to evaluate the stopping condition, which can be used to approximate the actual consumption of prefill and decode state.

Assumption of runtime optimization Wall-clock runtime depends on model architecture, hardware, kernel optimizations, and implementation choices such as batching and scheduling. We therefore report generated-token counts as a more implementation-independent measure of reasoning efficiency.

Failure Modes

Stopping before answer readiness. The verifier may accept a reasoning prefix before the model has derived sufficient information to produce a reliable answer. Forcing the transition to final answering at this point can prevent useful computation or later error correction, turning a potentially correct full-reasoning answer into an incorrect early-stopping answer.

This failure corresponds to the exit too early, reasoning collapse or noisy trajectories trap examined in Section 5.2. We investigate these three patterns in the exit-trap analysis to distinguish failures associated with insufficient reasoning from those associated with continued reasoning.

Post-stop reasoning leakage. Reasoning models are designed to finish a reasoning trace before generating the final answer. Appending an end-of-thinking token at an unusual position changes the model’s generation context and fails to terminate reasoning cleanly. The model may continue deriving equations, testing alternatives, verifying intermediate steps, or revising its conclusion in the final-answer channel (Wang et al., 2026a). This behavior violates the clean-termination condition.

Taxonomy

Early-stopping methods differ mainly in the information used to determine whether the current reasoning prefix is sufficient for answering. We therefore classify methods according to the evidence consumed by the stopping verifier πm\pi_m. This distinction is directly related to inference cost and task applicability. Whether the verifier itself is learned or manually specified is treated as a secondary property, since a learned verifier may still be applied to a new model or task without additional training. Hidden-state-based verifiers. These methods estimate stopping readiness from representations already available during model inference, most commonly hidden states associated with the current reasoning prefix. A lightweight classifier can be trained to predict whether the current state contains sufficient information to answer, as in SpecExit, Thought Calibration, OSP, and Lynx ([10] (Yang et al. 2025; Wu et al. 2025; Ömer Faruk Akgül et al. 2025)). TERMINATOR instead introduces a larger learned prediction module (Nagle et al., 2026). These approaches can make stopping decisions without generating a provisional answer, making their inference-time verification relatively inexpensive. Their stopping signal, however, depends on how well the learned representation transfers across models, tasks, and reasoning distributions.

Output-based verifiers. Within output-based verifiers, we further distinguish answer-based and answer-free methods according to whether verification requires generating a complete provisional task answer.

Answer-based verifiers. Answer-based methods explicitly generate provisional answers from intermediate reasoning prefixes and use agreement or convergence among these answers as evidence for early-stopping. Answer Convergence measures the stabilization of intermediate answers ([24]). DynaSor ([11]) similarly uses consistency among intermediate outcomes, while PUMA (Min et al., 2026) combines provisional-answer information with information from the preceding reasoning trajectory. Because these methods inspect the objective outputs, their stopping signal has a direct interpretation, however, they introduce substantial additional token costs, where a valid provisional answer may contain thousands of tokens or unable to have direct comparison (e.g: Free-text and Code generation).

Answer-free verifiers. Answer-free methods attempt to estimate reasoning sufficiency without producing a complete provisional answer. One family uses statistics available from the current generation, such as token probabilities, entropy, uncertainty, or transition-related signals. Statistical Early Stopping uses uncertainty-based statistics (Xie et al., 2025); DEER (Yang et al., 2026) and CoDE-Stop ([15]) evaluate short continuations around reasoning-transition points; and EAT (Wang et al., 2026b), ThinkBrake (Song et al., 2026) and EntroCut (Yan et al., 2026) use entropy-related signals.

A second family directly queries whether the current prefix is sufficient. DTSR (Xiang et al., 2026) and FlashThink ([18]), for example, use short sufficiency or verification probes rather than requiring a complete task answer. These approaches usually require more computation than a verifier operating only on existing hidden states, but substantially less than repeatedly generating full provisional answers.

Reasoning-path optimization. Unlike prefix-preserving early stopping, reasoning-path optimization modifies the generation trajectory to reduce unnecessary computation. NoWait suppresses reflection tokens (Wang et al., 2025a), while Token-Skip compresses CoT by skipping less important tokens (Xia et al., 2025). NCoTS searches for shorter reasoning paths ([23]), and TokenProbe uses token-level confidence signals to preserve informative reasoning during reinforcement learning (Zeng et al., 2026). Other methods intervene directly in hidden representations: Activation-Steered Compression shifts generation toward concise reasoning ([1]), Latent Reward Steering adaptively corrects fragile states ([19]), and constrained latent-space optimization redirects generation toward efficient reasoning trajectories ([16]). Answer Convergence also modifies the probability of emitting the end-of-thinking token to encourage earlier termination ([24]). Because these interventions alter the generation process rather than simply selecting a stopping point along an unchanged trajectory, we discuss them as related efficiency approaches but exclude them from our prefix-preserving early-stopping comparison.

Benchmark Design

Evaluation Methodology

Evaluating many early-stopping methods independently can be expensive, because each method may require a separate model roll-out. To reduce this evaluation cost, we use a replay-based protocol, applied for both Training-free and Training-based methods. For each input example, we first generate one full reasoning trace from the model and store the generated tokens together with available metadata. We then replay this token sequence and evaluate all early-stopping methods on the same observed prefix stream. Let D\mathcal{D} be the evaluation dataset and let MM be the reasoning model. We denote the set of early-stopping methods by

StudyMathSci. QAGen. QAComplex CodeFact.OracleExit DynamicsPost-stop AuditCoT Quality
[13]✓△△
(Wang et al., 2025b)✓△✓
(Wu et al., 2025)✓✓
[6]✓✓△
[24]✓✓✓✓✓
[11]✓△✓
(Yang et al., 2026)✓✓✓✓
(Song et al., 2026)✓✓✓✓
[2]✓✓✓
(Min et al., 2026)✓✓✓✓✓
[3]✓✓✓
(Xiang et al., 2026)✓✓
[15]✓✓△
(Yan et al., 2026)✓
(Nagle et al., 2026)✓✓△△△
Ours✓✓✓✓✓✓✓✓✓

Table 1. Table 1: Comparison with closely related work. Checkmarks denote evaluation components explicitly supported by the study. △ denotes related evidence that does not meet the exact criterion.

Notes. Math includes GSM8K, MATH and AIME; Sci. QA includes GPQA-Diamond. Gen. QA covers broad-domain question answering; Complex Code requires competitive-programming evaluation such as LiveCodeBench. Fact. requires a dedicated factuality or hallucination benchmark, such as TruthfulQA or HaluEval. Oracle means a ground-truth-informed hindsight analysis of correct intermediate answers; budget search or steps-to-solution estimates count only as related evidence. Exit Dynamics means empirical correctness or answer changes across reasoning positions. Post-stop Audit examines reasoning after forced termination, rather than hesitation markers in provisional answers. CoT Quality requires a manual or automated evaluation of the retained chain.

I=1,2,…,n.\mathcal{I} = 1,2,\ldots,n.

Each method m∈Im \in\mathcal{I} is defined by

(Sm,πm,Gm,vm),(S_m,\pi_m,G_m,v_m),

where vmv_m is the number of probe tokens required by the verifier. When vm=0v_m = 0, method mm does not use probe answers. The checkpoint condition is triggered by

S∪=⋁m∈ISm(x,rt).S_{\cup} = \bigvee_{m\in\mathcal{I}} S_m(x,r_t).

For each example (x,g)∈D(x,g) \in\mathcal{D}, we first generate a full reasoning trajectory:

(rT,yfull)∼M(x),(r_T,y^{\mathrm{full}}) \sim M(x),

During replay, each early-stopping method observes only the prefix rtr_t at time tt. Future tokens rt+1r_{t+1} and the full answer yfully^{\mathrm{full}} are not provided to the verifier. They are used only for offline analysis and scoring, where

yt,vprobe∼M(x,rt∘τ∘yt,<vprobe),y_{t,v}^{\mathrm{probe}} \sim M(x,r_t \circ\tau\circ y_{t,<v}^{\mathrm{probe}}),

And vv is the number of tokens allowed per generation. All candidate methods share this collection. Method mm receives only the first vmv_m probe answer token:

um,t←πm(x,rt,yt,vprobe).u_{m,t} \leftarrow\pi_m\left(x,r_t,y_{t,v}^{\mathrm{probe}}\right).

When method mm decides to stop at time tt, we append τ\tau to the reasoning prefix and generate its early-stopping answer:

ym∼M(x,rt∘τ).y_m \sim M(x,r_t \circ\tau).

The early-stopping answer ymy_m is generated separately from the shared probe answers because the probe answer is not a complete answer. ymy_m is then evaluated by the scoring function VV. After method mm stops, it is removed from the active method set for that sample.

Under identical prefixes, decoding settings, and verifier outputs, the replay protocol produces the same stopping decisions as online execution, provided the method does not modify the ongoing reasoning trajectory. Independent executions may differ because of stochastic generation.

Random stop positions

In addition to evaluating the exit positions selected by each method, we perform a randomized exit analysis to reduce bias introduced by the stopping policies themselves. This allows us to study whether non-monotonic reasoning behavior is a general property of the reasoning trajectory rather than an artifact of a particular method.

Overview of the replay-based benchmark

Figure 1. Overview of the replay-based benchmark. (I) The frozen reasoning model answers each question once, and the full trajectory is stored. (II) The stored trajectory is replayed. At each checkpoint, the prefix followed by </think> yields one shared probe, and state-based, answer-free and answer-based verifiers read either the hidden states or this probe. Each method’s stopping rule then decides to stop or to continue to the next checkpoint. (III) A method that stops generates its final answer, which is checked against the gold answer and compared with reference policies in accuracy and total token cost.

For each selected question, we sample exit positions along the recorded reasoning trajectory and generate eight independent answers at each sampled position using the same decoding configuration. The sampled questions are distributed across the benchmarks as follows: 30 from AIME26 and 128-196 from each MATH500, MMLU-Pro, TruthfulQA, and LiveCodeBench. An example of the randomized exit evaluation is shown in Figure 2

Random exit accuracy for MATH500 problem 272 using gpt-oss-20b

Figure 2. Random exit accuracy for MATH500 problem 272 using gpt-oss-20b. Token colors indicate accuracy, ranging from red (0) to green (1).

Adjustment from original implementation

Adjustment to Preserve KV Cache optimization For methods that score the output like DTSR, the original implementation ask for confident scoring via a new chat template. This implementation remove all the KVCache from previous generation. To prevent this, we change the evaluation prompt by adding the scoring prompt after reasoning chunk rtr_t, similar to its variant DTSR-1 (Xiang et al., 2026).

Adjustment for Coding benchmark Most probe-based methods temporarily stop and request an immediate final answer. This works well for mathematics and multiple-choice tasks, where answers are short. For coding tasks, however, answers require much longer output, with an average length of around 600 tokens, and the first few tokens are

normally solution template (Figure 3). We therefore increase the budget for probe answers by 5 times and reduce the threshold score for methods by 0.99 to ensure the probe answers have more information while not being too harsh for extra steps.

Since answer-based methods require complete provisional code solutions during verification which increase the cost substantially, we exclude them from the main coding comparison.

Experimental Setup and Evaluation Metrics

Experimental Setup

Models. We evaluate Gemma 4 E4B (Team et al., 2026), Qwen3.5-4B and Qwen3.5-9B (Qwen Team, 2026), R1-Qwen3-8B [8], and gpt-oss-20b at high reasoning effort (OpenAI et al., 2025).

Datasets. We evaluate on eight benchmarks spanning mathematical and scientific reasoning, code generation, and factuality or hallucination evaluation. For mathematical and scientific reasoning, we use AIME26 (math ai, 2026), MATH500 [21], GSM8K [5], GPQA-Diamond (Rein et al., 2023) and a subset of MMLU-Pro (Wang et al., 2024). For code generation, we use LiveCodeBench v6 Lite [17]. For factuality and hallucination evaluation, we use TruthfulQA [22] and the Genfig:3: Scoring for LiveCodeBench question 585: minimum-cost-for-cutting-cake-i (substring only). The two ■ markers indicate token limits of 20 and 100, respectively. Green highlights tokens that exceed the corresponding thresholds. The first 20 tokens of both solutions consist of imports and function declarations, while the scores only fluctuate in later sections

Probability score for DEER

Figure 3a. Probability score for DEER

Entropy score for EntroCut

Figure 3b. Entropy score for EntroCut

eral set of HaluEval [20]. We run 3 fold and collect the macro average for AIME26 and GPQA-Diamond and single pass for the rest. With methods that required training, we use MATH [14] without the duplication in MATH500 and AIME 1983–2024 (Veeraboina, 2024) with the same curation pipeline of using greedy to find the nearest valid exit position as the breakpoint and label the hidden position accordingly. For training-free methods, we use their default configuration and model check, and we use the original checkpoint for PUMA.

Early-stopping methods. We evaluate methods from the early-stopping categories described in § 2.6. Training-based methods include TERMINATOR and Thought Calibration that need to be retrained. Training-free methods include two group: probe-based method: Statistical Early Exit, DEER, CoDE-Stop, EntroCut, DTSR-1 and answer-based method: Answer-coverage, DynaSor, and PUMA. Even though PUMA was originally a training-based method, we reuse the original checkpoint.

Hybrid policies. We additionally evaluate a rule-based policy that combine multiple stopping methods by combines agreement among binary exit decisions.

At checkpoint θk\theta_k, let Γ(θk)\Gamma(\theta_k) denote the set of active methods. For exit-decision agreement, let em(θk)∈{0,1}e_m(\theta_k) \in\{0,1\} denote whether method mm recommends stopping. We define

Rexit(θk)=1∣Γ(θk)∣∑m∈Γ(θk)em(θk),R_{\mathrm{exit}}(\theta_k) = \frac{1}{|\Gamma(\theta_k)|}\sum_{m\in\Gamma(\theta_k)} e_m(\theta_k),

and exit when Rexit(θk)≥ρR_{\mathrm{exit}}(\theta_k) \ge\rho.

Algorithm 2 Replay-Based Evaluation for Early Exit Methods
1: Initialize result set $\mathcal{R} \leftarrow \emptyset$
2: for each example $(x,g) \in \mathcal{D}$ do
3: $(r_{1:T}, y^{\mathrm{full}}) \sim M(x)$
4: $\mathcal{A} \leftarrow \mathcal{I}$
5: for $t = 1$ to $T$ do
6: $\mathcal{C}_t \leftarrow \{m \in \mathcal{A} : S_m(x,r_{1:t}) = 1\}$
7: if $\mathcal{C}_t \ne \emptyset$ then
8: $v_{\max} \leftarrow \max_{m \in \mathcal{C}_t} v_m$
9: $y^{\mathrm{probe}}_{t,v_{\max}} \sim M(x,r_t \circ \tau)$
10: for each method $m \in \mathcal{C}_t$ do
11: $u_{m,t} \leftarrow \pi_m(x,r_{1:t},y^{\mathrm{probe}}_{t,v_m})$
12: if $G_m(u_{m,t},r_t) = 1$ then
13: $y_m \sim M(x,r_t \circ \tau)$
14: $a_m \leftarrow V(y_m,g)$
15: Add $(x,m,t,y_m,a_m)$ to $\mathcal{R}$
16: Remove $m$ from $\mathcal{A}$
17: end if
18: end for
19: end if
20: end for
21: $a_{\mathrm{full}} \leftarrow V(y^{\mathrm{full}},g)$
22: for each remaining method $m \in \mathcal{A}$ do
23: Add $(x,m,T,y^{\mathrm{full}},a_{\mathrm{full}})$ to $\mathcal{R}$
24: end for
25: end for
26: return $\mathcal{R}$

Generation settings and token budgets. The reasoning trace is capped at 32,000 tokens for all models. Probe generations used by verifier-based methods are capped at 2,048 tokens for code and 20 tokens for others, while final-answer generation following a stopping intervention is capped at 2,048 tokens. All use temperature of 0.6.

We evaluate fixed-budget reasoning using a predefined budget set B\mathcal{B}.

Evaluation References and Cost Accounting

We compare methods with no thinking (direct answer), natural full reasoning, fixed-budget forcing for b∈Bb \in\mathcal{B}, and a hindsight discrete-budget oracle. For each question ii, the oracle selects the smallest evaluated reasoning budget that produces a correct answer:

bi∗=min⁡{b∈B:V(yi(b),gi)=1},b_i^* = \min\left\{b \in\mathcal{B} : V\left(y_i^{(b)}, g_i\right) = 1\right\},

where gig_i is the ground-truth target. If no evaluated budget produces a correct answer, we set bi∗=∞b_i^* = \infty.

DatasetSamplesFormat
AIME2630Free-text
MATH500500Free-text
GSM8K1,319Free-text
GPQA-Diamond198MCQ
MMLU-Pro (subset)2,406MCQ
LiveCodeBench v6 Lite (wo tools)1,055Test cases
HaluEval (General)4,507Binary
TruthfulQA817MCQ

Table 2. Summary of evaluation datasets.

For a fixed reasoning trajectory and fixed candidate answer realizations, a hypothetical all-position oracle would consider every token position under the same stopping intervention. Our empirical oracle searches only the evaluated budget set B\mathcal{B} and selects the smallest successful reasoning budget in that set. Methods using other exit positions or independently sampled answers can exceed its observed accuracy. Throughout the experiments, “oracle” refers to this discrete-budget reference.

We also report total generated-token cost as:

Cm,i=Cm,ireason+Cm,ianswer+Cm,iprobe+Cm,itemp,C_{m,i} = C^{\mathrm{reason}}_{m,i} + C^{\mathrm{answer}}_{m,i} + C^{\mathrm{probe}}_{m,i} + C^{\mathrm{temp}}_{m,i},

where the four terms denote reasoning tokens, final-answer tokens, verifier or sufficiency-probe tokens, and temporary-answer tokens, respectively. Unless stated otherwise, all token-efficiency comparisons use this total generated-token cost.

Accuracy and Oracle-Relative Efficiency

For method mm, question ii, and repeated run rr, let

am,i=I[V(ym,i,r,gi)=1].a_{m,i} = \mathbb{I}\left[V(y_{m,i,r},g_i)=1\right].

For method mm, we calculate its average accuracy aˉm\bar{a}_m and average total generated-token cost Cˉm\bar{C}_m over the repeated evaluations. The corresponding quantities for no-thinking and the hindsight oracle are denoted by aˉ0\bar{a}_0, aˉH\bar{a}_H, and CˉH\bar{C}_H.

Target-Integrated Token Efficiency. Inspired by ROC-style evaluation, TITE integrates attainment across oracle-normalized accuracy targets and incorporates generated-token efficiency. We use TITE to summarize the accuracy improvement achieved by a method relative to the no-thinking and hindsight-oracle references, together with its relative generated-token cost.

We define the available accuracy gain as

Δa=aˉH−aˉ0.\Delta a = \bar{a}_H - \bar{a}_0.

For Δa>0\Delta a > 0, the target accuracy at normalized level qq is

a(q)=aˉ0+qΔa,q∈Q,a(q) = \bar{a}_0 + q\Delta a,\qquad q \in Q,

where Q=0.001,0.002,…,1.000Q = 0.001, 0.002, \ldots, 1.000. The fraction of accuracy targets GmG_m and its relative token-efficiency EmE_m are:

Gm=1∣Q∣∑q∈QI[aˉm≥a(q)],G_m = \frac{1}{\lvert Q\rvert}\sum_{q\in Q}\mathbb{I}[\bar{a}_m \ge a(q)],
Em=min⁡(1,CˉHCˉm).E_m = \min\left(1,\frac{\bar{C}_H}{\bar{C}_m}\right).

Finally,

TITEm=GmEm.\mathrm{TITE}_m = G_m E_m.

Preservation, Recovery, and Continuation Stability

Aggregate early-stopping accuracy does not show how stable correctness is along a reasoning trajectory. In particular, a correct answer at an earlier point may be followed by an incorrect answer after additional reasoning.

We evaluate this behavior in two stages. First, the continuation reasoning-error rate measures how often observed later attempts fail after a correct earlier attempt. Second, the random-exit monotonicity test checks whether these reversals are larger than what can be explained by stochastic decoding alone.

Continuation reasoning-error rate. Let α=(r,m,t)\alpha= (r,m,t) denote a correct anchor answer produced by method mm on reasoning trajectory rr at position tt. For this anchor, define the set of valid later attempts

Lα={j:rj=r, mj≠m, tj>t, j is valid}.L_\alpha= \{j : r_j = r,\ m_j \ne m,\ t_j > t,\ j\text{ is valid}\}.

Let cj=1c_j = 1 if later attempt jj is correct and cj=0c_j = 0 otherwise. For every correct anchor with at least one valid later attempt, its continuation error is

CERα=1∣Lα∣∑j∈Lα(1−cj).CER_\alpha= \frac{1}{\lvert L_\alpha\rvert}\sum_{j\in L_\alpha}(1-c_j).
CER‾=1R∑α=1RCERα.\overline{CER} = \frac{1}{\mathcal{R}}\sum_{\alpha=1}^{\mathcal{R}} CER_\alpha.

Random-exit monotonicity validation. The first test relies on exit position of methods, which can be biased due to the algorithm. This test generalize the exit position by selecting random exit position to show that regardless of methods, early-stopping itself show the non-monotonic behavior. For question ii, let

ti1<ti2<⋯<tiKit_{i1} < t_{i2} < \cdots< t_{iK_i}

be its valid sampled exits, and let pikp_{ik} denote the probability of producing a correct answer when exiting at tikt_{ik}. The monotone null hypothesis is

H0,i:pi1≤pi2≤⋯≤piKi.H_{0,i}: p_{i1} \le p_{i2} \le\cdots\le p_{iK_i}.

At each exit, repeated generations provide a binomial estimate p^ik\hat{p}_{ik}. We fit the best nondecreasing sequence p^ikiso\hat{p}^{\mathrm{iso}}_{ik} using binomial isotonic regression. For trajectory ii, the normalized likelihood discrepancy is

di=2Ki[ℓi(p^i)−ℓi(p^iiso)].d_i = \frac{2}{K_i}\left[\ell_i(\hat{\mathbf{p}}_i)-\ell_i(\hat{\mathbf{p}}^{\mathrm{iso}}_i)\right].

DoverallD_{\mathrm{overall}} is the average result across all trajectories. We additionally report an interpretable reversal-magnitude statistic. For question ii,

Ai=2Ki(Ki−1)∑a<bmax⁡(p^ia−p^ib,0),A_i = \frac{2}{K_i(K_i-1)}\sum_{a<b}\max(\hat{p}_{ia}-\hat{p}_{ib},0),

This measures the average positive drop from an earlier exit to a later exit. Since some positive drops are expected from decoding noise, we report the excess reversal in magnitude AexcessA_{\mathrm{excess}}

Aexcess=Aobs−E0[A∗],A_{\mathrm{excess}} = A_{\mathrm{obs}} - E_0[A^*],

where E0[A∗]E_0[A^*] is the expected value under the fitted monotone bootstrap.

Output Reliability

Post-stop reasoning continuation. We test whether the model continues reasoning after the stopping intervention using a lexical heuristic applied to the final-answer channel. We distinguish primary markers, including explicit thinking delimiters and model-specific serialization boundaries, from soft lexical markers. The latter consist of case-insensitive reasoning-continuation markers such as wait, hmm and actually.

Let

Lm,i,r=CountPostStopSignals⁡(ym,i,r)L_{m,i,r} = \operatorname{CountPostStopSignals}(y_{m,i,r})

denote the number of detected continuation markers. We define

ℓm,i,r=I[Lm,i,r>0].\ell_{m,i,r} = \mathbb{I}[L_{m,i,r} > 0].

The continuation rate for method mm is

Leakm=∑i,rℓm,i,rEm,i,r∑i,rEm,i,r.\mathrm{Leak}_m = \frac{\sum_{i,r} \ell_{m,i,r} E_{m,i,r}}{\sum_{i,r} E_{m,i,r}}.

We normalize each stopping position by the length of its corresponding full reasoning trace.

Candidate exits and stable frontier plot

Figure 4. Example candidate exit positions along a reasoning trajectory.

Retained reasoning-chain justification. A correct answer may still be produced from a reasoning prefix that does not contain enough information to justify the conclusion. We therefore use DeepSeek-4.1-Flash ([9]) as independent judge to score whether the retained reasoning supports the final answer.

For each evaluated output, the judge receives the question, reference information available for evaluation, the retained reasoning prefix, and the model’s final answer. It produces a justification score J(r1:t)∈[10,100]J(r_{1:t}) \in[10, 100] and a primary failure category.

The failure categories distinguish insufficient derivation, unsupported assertions, circular repetition, contradiction with the final answer, and other failures. Importantly, the category INSUFFICIENT_DERIVATION describes only what is missing from the observed reasoning trace and does not attribute the failure specifically to early-stopping.

For each model, benchmark, and method, we stratify the evaluation by answer correctness and report Jˉmcorrect\bar{J}_m^{\mathrm{correct}}, Jˉmincorrect\bar{J}_m^{\mathrm{incorrect}}. We select k=100k = 100 samples per correctness class. We compare these values with the corresponding full-reasoning reference:

ΔJmcorrect=Jˉmcorrect−JˉThinkingcorrect,\Delta J_m^{\mathrm{correct}} = \bar{J}_m^{\mathrm{correct}} - \bar{J}_{\mathrm{Thinking}}^{\mathrm{correct}},
ΔJmincorrect=Jˉmincorrect−JˉThinkingincorrect.\Delta J_m^{\mathrm{incorrect}} = \bar{J}_m^{\mathrm{incorrect}} - \bar{J}_{\mathrm{Thinking}}^{\mathrm{incorrect}}.

A negative ΔJ\Delta J indicates that the early-stopping output retains weaker justification than full reasoning on the same model and benchmark.

Experiments Result

RQ1: Efficiency, Generalization and Oracle Gap

Comparison with the discrete-budget oracle. Table 3, Table 4, and Table 5 report the accuracy and TITE across five models and three task domains. The discrete-budget oracle consistently achieves higher accuracy than natural full reasoning. On mathematical and scientific reasoning, gpt-oss-20b improves from 0.824 to 0.894, while Gemma4 E4B improves from 0.590 to 0.701. A similar pattern appears on code generation, where R1-Qwen3-8B improves from 0.382 to 0.514, and on hallucination benchmarks, where its accuracy increases from 0.540 to 0.730.

MethodGemma4 E4Bgpt-oss-20bQwen3.5-4BQwen3.5-9BR1-Qwen3-8B
Acc.TITEAcc.TITEAcc.TITEAcc.TITEAcc.TITE
Optimal bucket0.701–0.894–0.865–0.891–0.858–
Full Thinking0.590–0.824–0.802–0.821–0.749–
No thinking0.297–0.381–0.469–0.513–0.297–
Statistical EE0.5900.4030.8240.1910.8020.1510.8410.1760.7490.177
EntroCut0.5740.4040.7730.3800.7000.2690.7410.2070.7090.354
DEER0.5730.4010.7730.3810.6860.2230.7400.1930.7150.331
DTSR-10.5860.3970.7040.2810.5290.1260.5390.0520.5170.261
CoDE-Stop0.5710.4030.6010.2540.5320.1180.5900.0870.5040.276
DynaSor0.5630.3840.7090.2860.5190.0790.5990.0940.6230.276
PUMA0.5640.4040.7990.2780.6510.1080.7840.1650.720.269
Answer Convergence0.5770.3850.7410.2890.6560.1640.7210.1630.7050.227
TERMINATOR0.3020.0900.4500.0890.5720.0790.6330.0770.2860.012
Thought calibration0.4930.2400.6610.2160.3080.0190.5190.0420.5530.159
Exit agreement0.4580.4060.7820.3370.6970.2990.7470.2040.7060.341

Table 3. Average accuracy and TITE on the Mathematical & Science benchmarks.

MethodGemma4 E4Bgpt-oss-20bQwen3.5-4BQwen3.5-9BR1-Qwen3-8B
Acc.TITEAcc.TITEAcc.TITEAcc.TITEAcc.TITE
Optimal bucket0.443–0.624–0.518–0.624–0.514–
Full Thinking0.415–0.510–0.424–0.530–0.382–
No thinking0.278–0.385–0.254–0.356–0.191–
Statistical EE0.4160.3030.5140.0380.4260.0520.5350.0810.3790.048
EntroCut0.3970.2970.4780.0980.3230.0460.3850.0150.2670.025
DEER0.4130.2860.5280.0740.3700.0450.4450.0290.3650.026
DTSR-10.4060.2960.4640.1410.2900.0250.3420.0000.2470.051
CoDE-Stop0.3940.2710.4240.0730.3160.0570.3640.0070.2260.058
TERMINATOR0.3310.2320.4100.0510.3740.0270.4500.0290.2110.029
Thought calibration0.3870.4680.4580.1040.3230.0740.3880.0330.2950.044
Exit agreement0.4110.2820.4990.1290.3240.0740.4470.0470.2660.021

Table 4. Average accuracy and TITE on the LiveCodeBench.

MethodGemma4 E4B Acc.Gemma4 E4B TITEgpt-oss-20b Acc.gpt-oss-20b TITEQwen3.5-4B Acc.Qwen3.5-4B TITEQwen3.5-9B Acc.Qwen3.5-9B TITER1-Qwen3-8B Acc.R1-Qwen3-8B TITE
Optimal bucket0.609–0.703–0.683–0.743–0.730–
Full Thinking0.559–0.611–0.569–0.635–0.540–
No thinking0.525–0.536–0.524–0.587–0.494–
Statistical EE0.5600.1610.6090.0600.5730.0030.6350.0060.5400.045
EntroCut0.5560.1440.5950.0960.4590.0000.5850.0040.5630.114
DEER0.5590.1570.5940.0950.4650.0000.5810.0020.5650.121
DTSR-10.5610.1620.6000.0860.4900.0000.5470.0040.5410.114
CoDE-Stop0.5590.1570.5930.1010.4550.0000.5730.0050.5500.141
DynaSor0.5540.1370.6100.0680.5100.0000.5530.0000.5360.075
PUMA0.5500.1370.6050.1100.5170.0000.6050.0020.5590.101
Answer Convergence0.5600.1550.6090.0840.5330.0010.5950.0020.5410.054
TERMINATOR0.4580.0660.5490.0360.5310.0160.4310.0000.3350.010
Thought calibration0.5600.1580.6020.1190.4920.0440.4230.0000.5380.047
Exit agreement0.5610.2410.5980.0840.4980.0000.5850.0060.5220.079

Table 5. Average accuracy and TITE on the hallucination benchmarks.

Figure 5 illustrates the relationship between accuracy and total generated tokens, normalized to full reasoning. Across all five models, the oracle achieves higher accuracy at lower token cost than full reasoning, demonstrating the potential benefits of adaptive compute allocation. Statistical Early Exit does not have any significant differences with baseline. In contrast, more aggressive methods such as DTSR-1, CoDE-Stop, and some training-based approaches achieve lower token costs but often suffer substantial accuracy degradation. DEER, EntroCut, and PUMA exhibit more favorable accuracy-cost trade-offs in several models, although none consistently dominates across all settings. Current methods still struggle to identify reliable stopping positions while keeping verification costs low.

Figure 5. Accuracy versus total generated tokens across five reasoning models. The dashed line indicates full-reasoning accuracy, while the shaded region represents lower-cost operating points with at least the same accuracy. Stars indicate the discrete-budget oracle, and arrows illustrate the remaining oracle gap.

High accuracy does not guarantee token efficiency. On gpt-oss-20b mathematical reasoning, Statistical Early Exit achieves 0.824 accuracy but only 0.191 TITE. DEER and EntroCut obtain lower accuracies of 0.773, yet reach considerably higher TITE scores of 0.381 and 0.380, respectively. Similarly, on Qwen3.5-4B, Statistical Early Exit preserves full-reasoning accuracy at 0.802 with a TITE of 0.151, whereas EntroCut achieves 0.269 TITE at 0.700 accuracy. Answer-based approaches also face this trade-off. For example, PUMA reaches 0.799 accuracy on gpt-oss-20b mathematical reasoning, exceeding DEER, but achieves a lower TITE of 0.278. On R1-Qwen3-8B, Thought Calibration achieves higher accuracy than TERMINATOR across all three domains, but its TITE remains limited: 0.159 on mathematical and scientific reasoning, 0.044 on code generation, and 0.047 on hallucination benchmarks. On mathematical and scientific reasoning, TERMINATOR achieves 0.286 accuracy and 0.012 TITE, compared with 0.749 accuracy under full reasoning.

Efficiency is strongly dependent on the task domain. The effectiveness of early-stopping varies considerably across domains. Mathematical and scientific reasoning generally shows the strongest efficiency, with the best TITE scores ranging from 0.207 to 0.406 across the five models. In comparison, code generation remains more challenging. Except for Gemma4 E4B, the highest TITE scores are below 0.15, despite the substantial accuracy improvements demonstrated by the oracle. Hallucination benchmarks exhibit similarly weak efficiency, particularly on Qwen3.5-4B and Qwen3.5-9B, where the best TITE scores reach only 0.044 and 0.006, respectively. Several methods even perform below the no-thinking accuracy baseline, indicating that premature stopping can eliminate useful reasoning without producing meaningful computational benefits.

RQ2: Correctness generally improves with reasoning, but is not strictly monotonic

The continuation-error analysis shows that reaching a correct answer does not guarantee that subsequent reasoning will preserve it, and this behavior varies substantially across benchmarks and models. For gpt-oss-20b, continuation errors are particularly low on AIME26 (0.83%), GSM8K (1.36%), and MATH500 (3.62%), but increase on GPQA-Diamond (13.33%), TruthfulQA (10.05%), and LiveCodeBench (9.66%). Similarly, R1-Qwen3-8B exhibits low continuation-error rates on AIME26 (1.40%) and MATH500 (1.57%), but much higher rates on HaluEval (20.59%) and LiveCodeBench (17.90%). However, the pattern does not generalize uniformly across models. For Gemma4 E4B, GPQA-Diamond has the highest continuation-error rate (19.46%), while HaluEval has the lowest (3.13%).

Random-exit validation. We further evaluate randomly sampled exit positions with repeated generations with repeated generations as a separate validation. Although average performance increases with reasoning budget, individual trajectories exhibit more reversals than expected from decoding noise alone (Figure 7). Across the tested settings, the observed discrepancy exceeds the fitted monotone baseline, with the overall discrepancy is Doverall=0.5871D_{\mathrm{overall}} = 0.5871, and the average excess reversal after subtracting the amount expected from decoding noise is 1.1694%. The effect is especially large for gpt-oss-20b on TruthfulQA, where excess reversal reaches 3.55%.

Figure 7. (a) Averaged over trajectories. (b) Per trajectory, the observed monotonicity discrepancy DobsD_{\mathrm{obs}} exceeds the value expected under a monotone model with decoding noise DnullD_{\mathrm{null}} in every setting. The right column gives the excess reversal magnitude AexcessA_{\mathrm{excess}}.

BenchmarkGemma4 E4Bgpt-oss-20bQwen3.5-4BQwen3.5-9BR1-Qwen3-8B
AIME269.630.833.973.991.40
GPQA-Diamond19.4613.336.105.9517.34
GSM8K6.761.365.667.801.60
HaluEval3.136.576.804.9720.59
LiveCodeBench8.609.669.268.2817.90
MATH5005.223.622.943.741.57
MMLU-Pro11.086.0011.836.8813.06
TruthfulQA12.4810.0516.8712.9613.11

Table 6. Continuation reasoning-error rate (CER, %). Lower is better.

Exit-trap analysis. To understand how non-monotonic reasoning affects early stopping, we classify failed exits into three types of traps, demonstrated in Figure 6:

Visualization of random exit position and exit traps

Figure 6. Visualization of random exit position and exit traps

  • Exit too early: The method terminates reasoning before a reliable answer has been reached.

  • Reasoning collapse: The method continues beyond a correct exit point, after which the reasoning trajectory becomes incorrect and does not recover a correct answer at any later evaluated position.

  • Noisy: The reasoning trajectory frequently alternates between correct and incorrect answers. We classify a trajectory as noisy when it has at least ten correctness shifts.

Figure 8 shows that exiting too early is the dominant failure mode for most early-stopping methods on both gpt-oss-20b and Qwen3.5-4B, particularly for Terminator, Thought Calibration, DynaSor, and CoDE-Stop. In contrast, methods such as

Figure 8. Exit traps. (a, b) Share of each method’s exits that fail because of each trap. (c) Each point is one (method, trap) pair: trap prevalence among the method’s exits versus the failure rate on trapped traces. Dashed curves are iso-lines of total failure.

DEER, Answer Convergence, and EntroCut exhibit smaller premature-exit shares. Reasoning collapse and noisy trajectories contribute less to the overall failure distribution, but remain challenging for stopping methods.

Figure 8(c) further shows that trap prevalence and conditional failure rate capture different aspects of stopping difficulty. While premature exits account for a larger share of failures, reasoning collapse and noisy trajectories can exhibit high conditional failure rates despite their lower prevalence. The dashed iso-failure curves illustrate that a trap’s overall contribution depends on both its prevalence and the probability of failure when encountered.

RQ3: Reliability Beyond Correctness

Post-Stop Reasoning Continuation

We measure post-stop reasoning leakage by identifying two types of continuation: primary leakage, where explicit thinking tags reappear after the forced stop, and soft leakage, where continuation markers suggest that reasoning persists. Figure 9 reports their rates across normalized exit positions.

Figure 9. Post-stop reasoning leakage rate by normalized exit position RexitR_{\mathrm{exit}}, split into primary leakage and additional soft leakage. Bin width is 0.05.

Leakage varies substantially across models. Qwen3.5-4B exhibits the highest and most persistent leakage with nearly 50% at early and remaining substantial across much of the reasoning trajectory. Qwen3.5-9B also shows considerable leakage at early exits, but its rate generally declines as the exit position increases. R1-Qwen3-8B displays a sharp initial peak, followed by a rapid reduction in leakage. Gemma4 E4B shows a smaller early peak of approximately 20%, with leakage approaching zero at later positions. In contrast, gpt-oss-20b exhibits almost no post-stop continuation throughout the trajectory.

Across most models and exit positions, soft-only leakage accounts for a large portion of observed continuation, primary leakage nevertheless remains visible, especially for Qwen3.5-4B and R1-Qwen3-8B. Although leakage generally decreases with exit position for several models, Qwen3.5-4B exhibits a renewed increase toward later exits. Thus, allowing more reasoning before stopping does not necessarily eliminate post-stop continuation.

Retained Justification Quality

Most early-stopping method shows a negative justification delta relative to Full Thinking, and incorrect answers frequently suffer larger justification losses. DEER tends to have the smallest loss of the three training-free methods, while EntroCut and DTSR-1 trail it. The gap widens on TruthfulQA, where every method on every model loses at least 9.8 points.

TERMINATOR is the most variable method. It collapses on gpt-oss-20b (GPQA-Diamond −59.9, MMLU-Pro −41.4, TruthfulQA −60.1) and on Qwen3.5-4B TruthfulQA (−73.0), far beyond the loss of any other method on the same benchmark. On Qwen3.5-9B and R1-Qwen3-8B it does not collapse: its losses stay between +2.8 and −28.6.

Excluding TERMINATOR, correct-output losses are mostly 4–17 points on LiveCodeBench and 5–13 on MMLU-Pro, but reach 10–32 on TruthfulQA. GPQA-Diamond shows the strongest model dependence: losses are at most 6.3 on Qwen3.5, 15–19 on gpt-oss-20b, and 5.6–20.1 on R1-Qwen3-8B. TruthfulQA losses also differ across models for the same method (DTSR: −20.0 on Qwen3.5-4B, −31.8 on Qwen3.5-9B, −25.5 on R1-Qwen3-8B), so the loss depends jointly on method, benchmark, and model.

Failure modes and the base-model floor. INSUFFICIENT_DERIVATION and CONTRADICTS_FINAL_ANSWER dominate low-quality outputs for every method; no other category exceeds 9.3%. TERMINATOR skews furthest toward insufficient derivation (71.7%), consistent with its collapse cells, while DEER tracks Full Thinking most closely (38.8% vs. 36.1% insufficient derivation). Since Full Thinking is never early-stopped yet shows both failure modes at comparable rates (36.1%, 49.0%), a substantial share of the justification shortfall reflects limits of the underlying model rather than the stopping policy alone: better thresholds can reduce errors from premature termination, but cannot fix derivation gaps already present in full reasoning.

Findings

Accuracy and token cost remain difficult to balance. Accuracy and token cost remain difficult to balance. Many evaluated methods achieve low TITE scores relative to the discrete-budget oracle, particularly on coding and factuality tasks. Methods that preserve accuracy often incur substantial verification costs, while aggressive stopping can reduce accuracy. These comparisons depend on the evaluated budgets and sampled answers; they do not establish a universal gap to an all-position oracle.

Early stopping performs poorly on hallucination benchmarks. On TruthfulQA and HaluEval, existing methods achieve limited efficiency gains and frequently underperform direct answering. The highest TITE scores are only 0.044 on Qwen3.5-4B and 0.006 on Qwen3.5-9B, despite oracle accuracies of 0.683 and 0.743, respectively. On Qwen3.5-4B, DEER (0.465) and EntroCut (0.459) perform worse than the no-thinking baseline (0.524). These results suggest that current stopping signals struggle to identify answer readiness on factuality-oriented tasks, even when substantial oracle headroom exists.

Reasoning correctness is not strictly monotonic. A correct intermediate answer can become incorrect after additional reasoning. Randomized exit experiments confirm that these reversals exceed those expected from stochastic decoding noise alone.

Thus, additional reasoning does not guarantee improved correctness, challenging monotonicity assumptions used by some stopping methods.

Reliable stopping requires more than answer correctness. Stopping performance varies across models and tasks. Forced termination can also cause reasoning to continue in the final-answer channel, while some correct answers lack sufficient supporting reasoning. These findings highlight the need to evaluate stopping methods using total inference cost, correctness stability, termination behavior, and justification quality.

Conclusion

We benchmark ten early-stopping methods across five reasoning models and eight tasks using common replay evaluation and the oracle-relative TITE metric. Many evaluated policies achieve low TITE scores relative to the discrete-budget oracle, especially on code and factuality/hallucination benchmarks. Randomized exit tests reveal non-monotone correctness beyond decoding noise; reliability audits find post-stop reasoning and weaker retained justification. Future stopping rules should account for correctness stability and verifier cost alongside termination behavior.

ModelMethodGPQA-DiamondLiveCodeBenchMMLU-ProTruthfulQA
JcorrectJ_{\mathrm{correct}}JincorrectJ_{\mathrm{incorrect}}JcorrectJ_{\mathrm{correct}}JincorrectJ_{\mathrm{incorrect}}JcorrectJ_{\mathrm{correct}}JincorrectJ_{\mathrm{incorrect}}JcorrectJ_{\mathrm{correct}}JincorrectJ_{\mathrm{incorrect}}
gpt-oss-20bFull Thinking85.243.688.477.289.453.088.873.4
EntroCut-16.3-20.0-8.2-12.1-12.6-16.2-15.3-23.6
DTSR-1-15.0-20.4-9.9-13.4-11.2-21.3-10.0-18.2
DEER-18.9-19.7-5.6-8.8-10.3-19.2-9.8-26.4
TERMINATOR-59.9-29.2-8.3-13.6-41.4-38.0-60.1-50.7
Qwen3.5-4BFull Thinking93.939.490.252.595.874.490.861.1
EntroCut-5.1-3.8-9.5-17.8-8.4-16.0-14.0-14.7
DTSR-1-6.3-1.8-14.9-11.7-9.6-28.3-20.0-23.4
DEER-3.6-3.2-6.5-13.6-10.7-13.8-10.3-12.5
TERMINATOR-10.2-9.3-25.9-13.9-11.1-31.9-73.0-43.6
Qwen3.5-9BFull Thinking92.247.393.870.593.177.793.780.5
EntroCut-4.9-7.5-12.7-27.8-7.5-16.7-26.8-33.5
DTSR-1-1.6-14.7-17.1-24.0-5.7-38.0-31.8-43.7
DEER-4.5-6.7-7.2-17.8-7.5-26.7-27.0-37.3
TERMINATOR+2.8-1.1-28.4-23.1-8.4-32.1-27.6-38.1
R1-Qwen3-8BFull Thinking76.244.166.845.589.159.287.273.3
EntroCut-8.1-16.2-4.8-11.5-9.5-19.5-24.5-30.9
DTSR-1-20.1-20.4-9.7-15.1-5.2-17.3-25.5-34.8
DEER-5.6-14.7-3.9+2.8-5.9-19.0-23.5-30.2
TERMINATOR-13.7-9.9-28.6-21.3-2.1-7.4-15.0-35.8

Table 7. Justification quality by outcome correctness, relative to FULL THINKING on the same model and benchmark. Full Thinking rows report the mean judge score (10–100); other rows report the difference from Full Thinking.

MethodInsuff.Unsupp.Circ.Contra.Other
Full Thinking36.19.32.949.02.3
DEER38.83.31.454.41.8
EntroCut44.74.11.448.01.3
DTSR-152.21.50.944.60.7
TERMINATOR71.71.10.526.10.5

Table 8. Failure-mode distribution among low-quality outputs (J<50J < 50), pooled across all models and benchmarks. Percentages are of nbelown_{\mathrm{below}} for that method.

Limitations

Our main limitation is the computational cost of evaluating early-stoppings in detail. Each additional exit position requires a new answer continuation, and repeated sampling, requiring up to 512 answer continuations for a single trajectory. Within the available compute budget, the reported mono-tonicity validation covers only sub-sampled data and models.

Our cost measurements assume ideal KV-cache reuse and exclude the overhead of cache management, therefore do not directly establish reductions in latency, memory use, or energy consumption, as different architectures, quantization methods produce different results and favor different types of hardware. Those measurements would require deployment experiments that account for hardware, batching, and scheduling.

Finally, the reliability measures also have limits: lexical indicators can misidentify post-stop reasoning, and automated justification scores assess the visible trace without establishing whether it faithfully represents the model’s internal computation.

Acknowledgments

The authors gratefully acknowledge VinUniversity for its institutional support for this research.

References

References

  1. [1]Seyedamin Azizi, Erfan Baghaei Potr... and Massoud Pedram. 2025. Activation steering for chain-of-thought compression. Preprint, arXiv:2507.04742.arxiv.org/abs/2507.04742
  2. [2]Marthe Ballon, Brecht Verbeken, Vincent Ginis, and Andres Algaba. 2026. Probing the trajectories of reasoning traces in large language models. Preprint, arXiv:2601.23163.arxiv.org/abs/2601.23163
  3. [3]Simone Caldarella, Davide Talon, Rahaf Aljundi, Elisa Ricci, and Massimiliano Mancini. 2026. Thinking past the answer: Evaluating harmful overthinking in large reasoning models. Preprint, arXiv:2606.02835.arxiv.org/abs/2606.02835
  4. [4]Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2025. Do not think that much for 2+3=? on the overthinking of o1-like llms. Preprint, arXiv:2412.21187.arxiv.org/abs/2412.21187
  5. [5]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. Preprint, arXiv:2110.14168.arxiv.org/abs/2110.14168
  6. [6]Muzhi Dai, Chenxu Yang, and Qingyi Si. 2025. S-grpo: Early exit via reinforcement learning in reasoning models. Preprint, arXiv:2505.07686.arxiv.org/abs/2505.07686
  7. [7]Ayan Datta, Zixue Zhao, Mounika Marreddy, Bhuvanesh Verma, Radhika Mamidi, and Alexander Mehler. 2026. LLMs decide early and explain later: You can stop them early. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 51796–51822, Budapest, Hungary. Association for Computational Linguistics.DOI
  8. [8]DeepSeek-AI. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. Preprint, arXiv:2501.12948.DOI
  9. [9]DeepSeek-AI. 2026. Deepseek-v4.1-flash: Pushing the limits of kv cache compression.arxiv.org/abs/2609.19969
  10. [10]Mohammed Ehab, Aymane El Gadari, Vivek F. Farias, Adam Jozefiak, and Ciamac C. Moallemi. 2026. Os-pruner: Pruning chains-of-thought of reasoning models via optimal stopping. Preprint, arXiv:2607.11089.arxiv.org/abs/2607.11089
  11. [11]Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongmao Dai, Yonghao Zhuang, Yian Ma, Aurick Qiao, Tajana Rosing, Ion Stoica, and Hao Zhang. 2025. Efficiently scaling LLM reasoning programs with certaindex. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.DOI
  12. [12]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qiaohu Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Flash-think: An early exit method for efficient reasoning. Preprint, arXiv:2505.13949.
  13. [13]Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. 2025. Token-budget-aware LLM reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24842–24855, Vienna, Austria. Association for Computational Linguistics.arxiv.org/abs/2412.18547
  14. [14]Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS.
  15. [15]Parsa Hosseini, Sumit Nawathe, Mahdi Salmani, Meisam Razaviyayn, and Soheil Feizi. 2026. Early stopping for large reasoning models via confidence dynamics. Preprint, arXiv:2604.04930.arxiv.org/abs/2604.04930
  16. [16]Zhinan Hou, XingChen Li, and Keyou You. 2026. Efficient reasoning via constrained optimization in latent space. arXiv preprint arXiv:2609.34181.arxiv.org/abs/2609.34181
  17. [17]Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Live-codebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974.arxiv.org/abs/2403.07974
  18. [18]Guochao Jiang, Guofeng Quan, Zepeng Ding, Ziqin Luo, Dixuan Wang, and Zheng Hu. 2025. Flash-think: An early exit method for efficient reasoning. Preprint, arXiv:2505.13949.arxiv.org/abs/2505.13949
  19. [19]Jiakang Li, Guanyu Zhu, Can Jin, Chenxi Huang, Dexu Yu, Ronghao Chen, Yang Zhou, Hongwu Peng, Xuanqi Lan, Dimitris N. Metaxas, and Youhua Li. 2026. Latent reward steering: An adaptive inference-time framework that implicitly promotes cognitive behaviors in reasoning llms. arXiv preprint arXiv:2606.00726.arxiv.org/abs/2606.00726
  20. [20]Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. Halueval: A large-scale hallucination evaluation benchmark for large language models. Preprint, arXiv:2305.11747.arxiv.org/abs/2305.11747
  21. [21]Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. arXiv preprint arXiv:2305.20050.arxiv.org/abs/2305.20050
  22. [22]Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. Preprint, arXiv:2109.07958.arxiv.org/abs/2109.07958
  23. [23]Guoming Ling, Zhongzhan Huang, Yupei Lin, Junxin Li, Shanshan Zhong, Hefeng Wu, and Liang Lin. 2026. Neural chain-of-thought search: Searching the optimal reasoning path to enhance large language models. In Findings of the Association for Computational Linguistics: ACL 2026, pages 22900–22933, San Diego, California, United States. Association for Computational Linguistics.arxiv.org/abs/2601.11340
  24. [24]Xin Liu and Lu Wang. 2025. Answer convergence as a signal for early stopping in reasoning. Preprint, arXiv:2506.02536.arxiv.org/abs/2506.02536

Paper details

vlnc:W_njywpemz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents