Learning to Reason Efficiently with Interventional Step Credit

Introduction

In recent years, large reasoning models have achieved strong performance on complex reasoning tasks by generating long chains of thought (CoT) that verify intermediate results, reflect on earlier steps, and explore alternative solutions [12] (Wei et al. 2022). However, such traces often exhibit “overthinking”: they contain redundant computation in the form of repetitive derivations and unnecessary continuation, much of which substantially inflates inference cost while leaving the answer essentially unchanged [6] (Jiang et al. 2026).

A line of work seeks to suppress overthinking through reasoning budget control or length-aware reinforcement learning, encouraging models to answer both correctly and concisely [1, 3] (Luo et al. 2026; Hou et al. 2025; Liu et al. 2026). However, such trajectory-level control makes limited distinctions between necessary and redundant steps, potentially suppressing both and compromising accuracy (Nie et al., 2026). Recent work therefore introduces finer-grained supervision. Early-exit methods identify a stopping point through prefix sufficiency or the first occurrence of a correct answer [11] (Tan et al. 2025; Yue et al. 2026), but leave redundancy within the retained prefix unresolved. Recent advances in step-level supervision score individual steps using attention mass (Nie et al., 2026), generation entropy (Li et al., 2026b) and mean token log-probability (Wang et al., 2026). These signals characterize the model’s behavior at a step rather than its dependence on that step for answer prediction: a low score does not reliably establish that suppressing the step will preserve the answer.

To address this, we propose interventional step credit (ISC), which takes a causal perspective on step importance through targeted parameter interventions. Simply applying text-level step perturbations can obscure this influence, as the model may compensate for them when regenerating subsequent reasoning ((Tutek et al., 2025)). We therefore probe this dependence through step-targeted parameter interventions, without regenerating subsequent reasoning. Specifically, we use preference-based unlearning ((Zhang et al., 2024)) to suppress the target step’s generation under its preceding prefix. With the prefix and the base model’s candidate answer fixed, ISC measures the magnitude of the resulting change in answer confidence. As shown in Figure 1, ISC generally achieves a more favorable accuracy–pruning frontier than attention mass ((Nie et al., 2026)) or step entropy ((Li et al., 2026b)) in post-hoc CoT pruning, particularly at high step-deletion ratios.

Building on ISC, we propose IGPO, which distills intervention-derived step-credit labels into an online step-level predictor and uses the predicted credit to guide stepwise advantage rescaling. For correct trajectories, IGPO attenuates the positive advantage of low-ISC steps and shields high-ISC steps from negative advantage. This selective rescaling discourages reinforcing redundant reasoning while protecting important steps, enabling the model to produce shorter reasoning traces while maintaining strong task performance.

We evaluate the proposed method on two representative LLMs across nine benchmarks. Experimental results show that IGPO reduces average response length by up to 62.0% while improving macro-average accuracy by up to 8.4 pp over base models. IGPO also achieves the highest macro-average accuracy and efficiency score among the compared methods at two model scales. Further analysis shows that IGPO generates more concise CoT with more reliable computations and logical deductions than compared methods.

Our main contributions are as follows:

• We introduce ISC, a causal formulation of step importance grounded in targeted parameter-level interventions.

• We propose IGPO, which employs ISC to selectively rescale stepwise advantages, targeting redundant steps while protecting important steps.

• Experiments show that ISC improves the accuracy–length trade-off in post-hoc CoT pruning, while IGPO reduces reasoning costs and improves task performance.

Related Work

Efficient Reasoning. Existing methods are broadly categorized as training-free and training-based. Training-free methods reduce inference costs through budget control, early termination, or CoT pruning ([13, 9] (Yang et al. 2026; Huang et al. 2026a)). Training-based methods use trajectory-level policy optimization ([3, 1] (Liu et al. 2026; Wu et al. 2025; Huang et al. 2026b)) or finer-grained policy optimization ([11] (Li et al. 2026a; Nie et al. 2026; Xu et al. 2026)). Our work uses intervention-derived step credit to guide advantage rescaling and selectively discourage redundant reasoning.

Step Importance Assessment. Prior work assesses reasoning-step importance through answer position, prefix-conditioned performance, or model-internal signals. Answer-position methods use the first correct answer as a stopping boundary, leaving redundancy within the retained prefix unresolved ([8] (Jiang et al. 2026; Tan et al. 2025)). Prefix-based methods evaluate progress toward the ground-truth answer ([11] (Kazemnejad et al. 2025; Setlur et al. 2025; Yue et al. 2026; Li et al. 2026a; Wu et al. 2026)), but may undervalue preparatory steps whose contributions have not yet improved answer predictions. Recent advances introduce step scores based on attention, generation entropy, and token-level confidence ((Nie et al., 2026; Li et al., 2026b; Wang et al., 2026)). These signals characterize model behavior rather than directly measuring its dependence on a step for answer prediction. Most related to us, SWAP ((Li et al., 2026a)) and SPAE ((Wu et al., 2026)) use answer predictions to assess each step’s contribution to reasoning progress. ISC instead assesses how answer prediction depends on the target step through parameter intervention, while holding the prefix and the answer fixed. This directly captures the target step’s causal effect on the reasoning state, revealing the true value of preparatory steps.

Interventions on Reasoning Steps. Prior work primarily uses text-level interventions, either perturbing steps and regenerating the continuation ([4] (Lanham et al. 2023; Yu et al. 2025; Ruan et al. 2025)), or deleting text units and measuring prediction changes conditioned on the remaining CoT, as in SCPO (He et al., [15]), AttriCoT (Wei et al., 2026), and LIMOPro (Xiao et al., 2025). However, regenerated continuations may compensate for perturbations through correction or alternative reasoning, while retained downstream steps may still carry information supplied by the deleted step. Li et al. (2026c) intervene on steps by modifying hidden states. FUR (Tutek et al., 2025) uses parameter interventions to assess the parametric faithfulness of reasoning steps through prediction changes in a no-CoT setting. However, a mathematical reasoning step’s role often depends on previously accumulated information, which no-CoT evaluation may not fully capture. We follow the key idea of FUR but evaluate answer confidence under the fixed prefix preceding the target step, without regenerating subsequent reasoning. ISC thus assesses prefix-conditioned step influence rather than no-CoT parametric faithfulness.

Interventional Step Credit

We introduce interventional step credit (ISC) to assess the influence of a reasoning step through targeted parameter interventions. We formulate each reasoning step’s causal contribution to the answer (§ 3.1), then estimate ISC from intervention-induced changes in answer confidence (§ 3.2).

Causal Formulation of Step Importance

Given a question $q$, the model generates a response $o = (\tau, \mathcal{F})$, where $\tau= (s_1, \ldots, s_N)$ denotes a reasoning trajectory of $N$ steps and $\mathcal{F}$ denotes the final solution. For the $n$-th step $s_n$, we define its preceding reasoning prefix as $h_n = (q, s_1, \ldots, s_{n-1})$. We use $|o|$ to denote the number of tokens in the complete response. Given $h_n$, we examine the causal dependence of the model’s answer prediction on the computation underlying step $s_n$. Specifically, we assume that after processing the reasoning prefix $h_n$, the model has formed a latent prediction of the next reasoning step $\tilde{s}_n$ and an anticipated subsequent reasoning trajectory $\tilde{s}_{>n}$. The actual step $s_n$ is the verbalized realization of $\tilde{s}_n$, while the anticipated subsequent reasoning $\tilde{s}_{>n}$ supports the model’s answer prediction $\hat{a}_n$ under prefix $h_n$. We use two idealized causal structures to describe how $\tilde{s}_n$ influences answer prediction through subsequent reasoning.

In the first structure, $\tilde{s}_n$ is verbalized as $s_n$ and further shapes the model’s anticipated subsequent reasoning $\tilde{s}_{>n}$ and answer prediction $\hat{a}_n$:

In the second structure, $\tilde{s}_n$ is still verbalized as $s_n$, while the anticipated subsequent reasoning $\tilde{s}_{>n}$ and answer prediction $\hat{a}_n$ are driven primarily by the information in $h_n$:

From this causal perspective, we characterize step importance by whether the computation underlying a step mediates the model’s answer prediction. In the next subsection, we measure this influence through intervention and quantify it as Interventional Step Credit (ISC). Intervening on the computation underlying an important step should induce a substantial change in answer prediction, yielding high ISC.

ISC Estimation

Step-Targeted Parameter Intervention. We implement step-targeted parameter interventions by unlearning the entire step $s_n$ conditioned on $h_n$, using the NPO+KL algorithm employed in FUR (Tutek et al., 2025). The NPO term (Zhang et al., 2024) reduces the probability of generating $s_n$, thereby approximately suppressing its underlying latent computation $\tilde{s}_n$. The KL term regularizes predictions toward those of the base model on a retain set $\mathcal{D}_{\mathrm{RT}}$ drawn from other reasoning trajectories. We minimize the joint objective:

where $\lambda_{\mathrm{KL}}$ controls the strength of retain regularization. Both loss terms are detailed in Appendix A.3. For each step, we independently update the base parameters $\theta_0$ to obtain $\theta^{(-n)}$.

Measuring the Intervention Effect. For a target step $s_n$, we prompt the base model $\pi_{\theta_0}$ to answer directly given the reasoning prefix $h_n$, yielding $a_n$. Keeping both $h_n$ and $a_n$ fixed, we then evaluate the confidence assigned to $a_n$ by the base model $\pi_{\theta_0}$ and the intervened model $\pi_{\theta^{(-n)}}$. We avoid regenerating subsequent reasoning to isolate the intervention effect from changes in reasoning paths and contexts. Suppose that $a_n$ consists of $T_n tokens. We use its average log-probability as the model’s confidence in the answer:

where $a_{n,t}$ denotes the $t$-th token of $a_n$. We define the ISC of $s_n$ as the magnitude of the intervention-induced change in answer confidence:

IGPO: ISC-Guided Policy Optimization

IGPO turns ISC into step-level supervision for efficient reasoning. We construct binary ISC labels and distill them into a step-level predictor (§ 4.1). The predicted credit guides stepwise advantage rescaling during reinforcement learning (§ 4.2).

ISC Labeling and Distillation

To avoid costly online parameter interventions, we distill offline ISC labels into a step-level predictor.

ISC Labeling. For $K$ annotation trajectories, let $N_i$ be the number of steps in trajectory $i$ and $c_{i,n}$ the ISC of its $n$-th step. We pool ISC values from all steps across all trajectories into $C = \{c_{i,n}\}$ and label the global top $\rho$ fraction as high-ISC:

where $\rho\in(0, 1)$, $Q_{1-\rho}$ denotes the corresponding quantile, and $z_{i,n} = 1$ indicates a high-ISC step.

ISC Prediction. A causal LM with a shared classification head predicts $p_{i,n} = g_{\phi}(h_{i,n}, s_{i,n})$, the probability that a completed step $s_{i,n}$ is high-ISC given its prefix $h_{i,n}$. We train the predictor with binary cross-entropy, averaging the loss over steps within each trajectory and then across trajectories (Appendix § 8.§ 4). During policy optimization, we use $\hat{z}_{i,n} = \mathbb{I}[p_{i,n} \ge\eta]$ to identify high-ISC steps, where $\eta$ is the prediction threshold.

ISC-Guided Policy Optimization

We use predicted ISC to guide length-aware reinforcement learning. We adopt a trajectory-level reward that combines answer correctness with a relative length penalty and perform stepwise advantage rescaling according to the predicted ISC.

Trajectory-Level Reward and Advantage Estimation. Given a question $q$, the old policy $\pi_{\psi_{\mathrm{old}}}$ samples a group of $G$ rollouts $\{o_i\}_{i=1}^{G}$. Following Arora and Zanette [3], we define a trajectory-level reward that combines answer correctness with a relative length penalty:

where $\lvert o_i \rvert$ denotes the response length, $\operatorname{mean}(q)$ and $\operatorname{std}(q)$ denote the mean and standard deviation of the lengths of correct rollouts for question $q$, respectively, $\sigma(\cdot)$ is the sigmoid function, and $\alpha$ controls the strength of the length penalty. Based on these trajectory-level rewards, we estimate the advantage using RLOO:

The trajectory-level advantage $A_i$ is shared across all reasoning steps. To focus length optimization on low-ISC steps while protecting high-ISC steps, we introduce stepwise advantage rescaling.

ISC-Guided Advantage Rescaling. We apply stepwise advantage rescaling only to correct rollouts, using predicted ISC labels $\hat{z}_{i,n}$. Incorrect rollouts retain the original trajectory-level advantage $A_i$. For positive advantages, we attenuate the positive updates assigned to low-ISC steps to avoid reinforcing redundant reasoning. Conversely, for negative advantages, we suppress the negative updates on high-ISC steps to protect essential reasoning from excessive length penalties. We set $\tilde{A}_{i,n} = \gamma_{i,n}A_i$ with

Here, $\delta\in[0,1]$ scales positive advantages for low-ISC steps. Since $\gamma_{i,n} \ge0$, rescaling does not reverse their signs. Each token in $s_{i,n}$ receives $\tilde{A}_{i,n}$, yielding token-level advantages $\tilde{A}_{i,t}$ for the clipped policy objective (Appendix § 8.§ 5).

Experiments

Experimental Setup

Training. We train DeepSeek-R1-Distill-Qwen-1.5B (DS-1.5B) and DeepSeek-R1-Distill-Qwen-7B (DS-7B) ([12]) with IGPO on the DeepScaleR dataset ((Luo et al., 2025)) using the VERL framework ((Sheng et al., 2025)). We set $\alpha= 0.15$ for both models (Eq. (7)), with $\delta= 0.7$ for DS-1.5B and $\delta= 0.5$ for DS-7B.

Evaluation. We evaluate on six in-domain reasoning benchmarks: GSM8K ([10]), MATH500 ((Lightman et al., 2024)), AMC23 ((math-ai, 2025)), OlympiadBench ([14]), AIME2024, and AIME2025 ((Mathematical Association of America, 2024, 2025)). In addition, we report performance on three OOD benchmarks: LiveCodeBench-v6 ((Jain et al., 2025)), GPQA-Diamond ((Rein et al., 2024)), and MMLU ([16]), covering math, coding, and science tasks of varying difficulty. To obtain more reliable estimates, we evaluate each problem 128 times on AIME2024 and AIME2025 and four times on other benchmarks, and report the average performance. We report Pass@1 (Acc.), response length (Tok.), and compute the Accuracy–Efficiency Score (AES) ((Luo et al., 2026)) to summarize the accuracy–length trade-off.

Baselines. We compare against three categories of efficient reasoning methods: (1) adaptive reasoning methods, including AdaptThink ((Zhang et al., 2025)), AutoThink ((Tu et al., 2025)), and DEER ((Yang et al., 2026)), included for reference; (2) outcome-supervised RL methods, including TLMRE ([3]), ThinkPrune ((Hou et al., 2025)), DIET ([5]), ACPO ([7]), LASER ((Liu et al., 2026)) (LASER-D and LASER-DE), SmartThinker ((Hu et al., 2026)), and Step Entropy ((Li et al., 2026b)); and (3) process-supervised RL methods, including LC-R1 ([8]), VSRM-R++ ((Yue et al., 2026)), S-GRPO ([11]), DEPO ((Tan et al., 2025)), DECS ((Jiang et al., 2026)), SWAP ((Li et al., 2026a)), SPAE ((Wu et al., 2026)), and AttnPO ((Nie et al., 2026)).

ISC Annotation and Distillation. For each model, we generate 1,000 correct trajectories on DeepScaleR for ISC annotation and construct binary labels using $\rho= 0.4$ (Eq. (6)). We train an ISC predictor based on Qwen3-4B-Instruct-2507 ((Qwen Team, 2025)) on 900 trajectories and reserve 100 for validation. Implementation details are provided in Appendices § 8 and § 9.

ISC Evaluation

We evaluate ISC through post-hoc CoT pruning, answer-sensitivity analysis, and checks for non-target intervention effects.

Post-hoc CoT pruning. We assess whether ISC identifies steps that can be removed while preserving answer accuracy. We sample 50 correct trajectories generated by DS-7B on DeepScaleR-Preview. Following Li et al. 2026b, we remove increasing fractions of the lowest-scoring steps under ISC, AttnPO, and Step Entropy, and evaluate answer accuracy conditioned on the remaining CoT. We always mask the final-answer step to prevent direct answer reuse. Figure 1 shows that ISC better preserves accuracy under aggressive pruning, supporting its usefulness for identifying removable reasoning steps.

Discriminative Ability of ISC. We randomly sample 100 annotated trajectories and select one high-ISC and one low-ISC step from each. For each step, we compare four base-model and four intervened-model completions against four base-model reference completions under the same prefix. We define the excess flip rate as the post-unlearning flip rate minus the base model’s natural flip rate. As shown in Figure 3, high-ISC interventions yield much larger excess flip rates than low-ISC interventions at both model scales, demonstrating ISC’s ability to identify steps whose unlearning more strongly affects answer predictions.

Assessing Potential Confounds. To rule out potential confounding effects, we sample 25 unrelated problems answered correctly in all four sampled base-model responses. For each intervened model, we sample four responses per problem and average accuracy within each ISC group. Figure 4 shows similar accuracy between high-ISC and low-ISC interventions at both model scales. These results suggest that the larger answer changes after high-ISC interventions are not simply due to greater overall capability degradation.

ISC Predictor Evaluation. We train the ISC predictor for 10 epochs and select the checkpoint with the highest balanced accuracy on 100 held-out validation trajectories for policy optimization. Figure 5 reports [email protected] and AUROC, showing discrimination between high- and low-ISC steps on labels derived from both model scales.

Effectiveness of IGPO

Main Results. Table 1 compares IGPO with existing efficient reasoning methods across six mathematical reasoning benchmarks. IGPO achieves the highest macro-average accuracy and AES among the compared methods at both model scales. Relative to the base models, it reduces average response length by up to 62% while improving accuracy. Compared with AttnPO and TLMRE, IGPO achieves a better overall accuracy–efficiency trade-off on DS-1.5B. Its advantage is more pronounced on DS-7B, where it improves macro-average accuracy while substantially reducing average response length, producing shorter responses than both baselines on all six benchmarks. In addition, on the challenging AIME2025 benchmark, where existing baselines struggle to balance accuracy and reasoning cost, IGPO substantially improves this trade-off on DS-7B, achieving an AES of 0.97 versus 0.60 for the best baseline. Overall, these results suggest that our approach achieves a favorable trade-off between reasoning efficiency and task performance.

Methods	GSM8K Acc.	Tok.	AES	MATH500 Acc.	Tok.	AES	AMC2023 Acc.	Tok.	AES	Olympiad Acc.	Tok.	AES	AIME2024 Acc.	Tok.	AES	AIME2025 Acc.	Tok.	AES	Macro Avg Acc.	Tok.	AES
1.5B Model
DS-1.5B	78.9	1018	0.00	81.7	4670	0.00	67.5	7356	0.00	48.6	9066	0.00	27.6	12327	0.00	22.9	12086	0.00	54.5	7754	0.00
♡ AutoThink	83.0	568	0.60	84.0	2195	0.62	67.0	5059	0.28	51.6	5053	0.63	34.6	9514	0.99	21.8	7944	0.10	57.0	5056	0.54
♡ AdaptThink	83.1	480	0.69	82.0	1782	0.63	67.0	3510	0.49	50.8	3708	0.73	31.0	6670	0.83	23.0	7255	0.41	56.1	3901	0.63
♡ DEER‡	74.7	984	-0.23	67.8	2497	-0.38	60.0	5496	-0.30	32.0	5960	-1.37	23.3	9557	-0.55	10.0	9281	-2.58	44.6	5629	-0.90
♠ ThinkPrune	83.0	609	0.56	83.2	1938	0.64	73.2	3039	0.84	51.4	3535	0.78	27.1	5631	0.46	22.5	5167	0.49	56.7	3320	0.63
♠ DIET	–	–	–	83.0	3061	0.39	65.4	6425	-0.03	–	–	–	31.8	10578	0.60	–	–	–	–	–	–
♠ ACPO	81.3	572	0.53	81.0	1679	0.60	–	–	–	–	–	–	30.0	6670	0.72	–	–	–	–	–	–
♠ Laser-D	83.4	863	0.33	84.2	1872	0.69	75.3	2981	0.94	54.4	4700	0.84	34.2	5750	1.25	23.1	6928	0.45	59.1	3849	0.75
♠ Laser-DE	80.4	820	0.25	83.5	1949	0.65	73.3	3080	0.84	54.4	5151	0.79	35.0	5789	1.34	24.2	7323	0.57	58.5	4019	0.74
♠ SmartThinker	79.0	697	0.32	82.0	2516	0.47	71.3	4022	0.62	52.5	4982	0.69	26.7	8022	0.19	21.4	7369	0.05	55.5	4601	0.39
♣ LC-R1	82.7	841	0.32	82.5	2233	0.55	61.7	3947	0.03	48.1	4546	0.45	23.6	7122	-0.30	21.2	6434	0.10	53.3	4187	0.19
♣ VSRM-R++	–	–	–	81.7	2597	0.45	64.7	4119	0.23	54.8	4388	0.90	29.5	6958	0.64	22.9	6892	0.43	–	–	–
♣ DEPO	–	–	–	87.2	2762	0.61	74.2	4388	0.70	–	–	–	30.8	7732	0.72	24.8	7649	0.62	–	–	–
♣ DECS	–	–	–	84.4	1817	0.71	75.4	2988	0.94	56.1	3396	1.09	31.2	5550	0.94	23.8	4965	0.71	–	–	–
♣ SWAP	–	–	–	86.5	1597	0.84	78.3	2645	1.12	45.1	3646	0.24	33.9	4915	1.29	23.3	4627	0.67	–	–	–
♠ TLMRE	87.2	604	0.72	85.8	1915	0.74	79.1	3349	1.06	56.9	4270	1.04	37.3	7024	1.49	26.6	7117	0.90	62.2	4046	0.99
♣ AttnPO	87.0	393	0.92	86.0	1318	0.88	77.8	2638	1.10	57.1	3133	1.18	37.7	5713	1.64	25.4	5488	0.87	61.8	3114	1.10
♣ Ours	87.4	586	0.75	86.5	1662	0.82	81.5	2798	1.24	57.7	3359	1.19	37.7	6027	1.61	26.6	5481	1.03	62.9	3319	1.11
7B Model
DS-7B	87.5	676	0.00	90.8	3713	0.00	85.0	5963	0.00	64.4	7363	0.00	51.5	10485	0.00	37.1	11140	0.00	69.4	6557	0.00
♡ AutoThink	91.1	866	-0.16	91.2	2146	0.44	83.3	4645	0.12	65.5	5133	0.35	54.8	8051	0.42	36.2	8608	0.11	70.4	4908	0.21
♡ AdaptThink	91.0	309	0.66	92.0	1875	0.54	87.5	3287	0.54	65.5	5574	0.29	55.6	8599	0.42	36.2	9523	0.03	71.3	4861	0.41
♡ DEER‡	90.6	917	-0.25	89.8	2143	0.37	85.0	4451	0.25	52.6	5420	-0.65	49.2	9839	-0.16	36.7	7257	0.30	67.3	5005	-0.02
♠ DIET	–	–	–	92.1	3187	0.19	82.6	6075	-0.16	–	–	–	57.9	10124	0.41	–	–	–	–	–	–
♠ ACPO	88.3	413	0.42	91.6	1405	0.65	–	–	–	–	–	–	52.8	4520	0.64	–	–	–	–	–	–
♠ Laser-D	87.3	804	-0.20	92.2	1836	0.55	90.0	2694	0.72	66.7	3914	0.58	58.3	5379	0.88	38.0	6167	0.52	72.1	3466	0.51
♠ Laser-DE	82.2	789	-0.47	92.0	1658	0.59	89.1	2612	0.71	68.6	3643	0.62	55.8	4969	0.78	37.5	6077	0.49	70.6	3291	0.45
♠ SmartThinker	–	–	–	93.0	2753	0.33	90.0	4118	0.49	–	–	–	–	–	–	40.8	9277	0.47	–	–	–
♠ Step Entropy†	79.2	168	0.28	85.0	2439	0.03	–	–	–	–	–	–	57.1	6839	0.68	33.3	10750	-0.47	–	–	–
♣ LC-R1	88.1	450	0.36	90.4	1568	0.56	79.1	3453	0.07	64.1	4144	0.41	23.6	6904	-2.37	36.2	7150	0.24	63.6	3945	-0.12
♣ S-GRPO	93.8	906	-0.12	92.4	2522	0.45	87.5	3494	0.50	69.7	3914	0.71	56.0	7737	0.56	36.0	7908	0.14	72.6	4308	0.37
♣ VSRM-R++	–	–	–	89.8	2044	0.40	80.9	3704	0.14	66.1	5470	0.34	52.2	6773	0.39	36.4	6953	0.28	–	–	–
♣ DEPO	–	–	–	94.4	2318	0.50	90.5	3215	0.65	–	–	–	52.7	6580	0.44	39.2	7092	0.53	–	–	–
♣ DECS	–	–	–	93.0	1728	0.61	89.0	2772	0.68	70.3	3283	0.83	51.3	5277	0.48	36.4	5516	0.41	–	–	–
♣ SWAP	–	–	–	93.6	1573	0.67	85.6	2685	0.57	60.3	4760	0.03	56.5	5744	0.74	38.1	5598	0.58	–	–	–
♣ SPAE	–	–	–	–	–	–	92.5	4543	0.50	74.0	6509	0.56	59.4	9908	0.51	42.7	10687	0.50	–	–	–
♠ TLMRE	91.1	515	0.36	93.8	1700	0.64	89.9	2967	0.68	69.6	4454	0.64	58.5	7462	0.70	39.2	8301	0.43	73.7	4233	0.57
♣ AttnPO	92.4	446	0.51	92.8	1300	0.72	89.6	2303	0.78	68.7	2912	0.80	57.2	5214	0.83	38.1	5359	0.60	73.1	2922	0.71
♣ Ours	92.9	264	0.79	91.9	948	0.78	91.0	1731	0.92	68.5	2370	0.87	56.3	4356	0.86	42.6	5275	0.97	73.9	2491	0.87

OOD Generalization. Table 2 shows that IGPO’s efficiency gains extend beyond mathematical reasoning, reducing response length by 35.3–61.2% across all three OOD benchmarks and both model scales. These savings accompany improved performance on DS-1.5B and largely preserved performance on DS-7B, with a 1.3 pp decrease on GPQA.² All settings achieve positive AES, indicating favorable accuracy–efficiency trade-offs beyond the training domain.

Model	LiveCodeBench Acc.	Tok.	AES	GPQA Acc.	Tok.	AES	MMLU Acc.	Tok.	AES
1.5B Model
DS-1.5B	24.0	10286	0.00	27.9	9603	0.00	52.1	2650	0.00
Ours	28.6	6656	0.93	33.9	4889	1.05	52.9	1357	0.53
7B Model
DS-7B	48.1	8905	0.00	49.2	8153	0.00	71.6	2067	0.00
Ours	48.5	5436	0.41	47.9	4029	0.36	71.7	803	0.62

CoT Quality Evaluation

To assess the quality of IGPO’s shortened CoT, we use ReasonEval-7B (Xia et al., 2025) to evaluate CoT generated by IGPO and AttnPO on DS-7B across the six mathematical reasoning benchmarks. Following its scoring protocol, we report average validity and redundancy scores. Validity assesses the computational correctness and logical soundness of CoT steps. As shown in Figure 7, IGPO achieves higher validity than AttnPO on most benchmarks, with particularly clear gains on AIME2024 and AIME2025. Redundancy measures correct steps that do not advance the solution. IGPO achieves lower redundancy than AttnPO on all datasets. These results suggest that IGPO produces shorter, less redundant CoT while maintaining or improving step validity.

Ablation and Sensitivity Analysis

For computational efficiency, we conduct these analyses on DS-1.5B with a batch size of 64 and 8 rollouts per question, using the same configuration across variants. Unless otherwise specified, other hyperparameters match the main experiments.

Intervention Method. We replace step-targeted unlearning with simple perturbation following Zhao et al. (2026), while keeping the fraction of steps labeled important unchanged. Figure 6(a) shows that unlearning achieves similar accuracy but shorter responses, increasing AES from 0.80 to 0.90. These results suggest that unlearning-derived credit provides a more favorable accuracy–efficiency trade-off under the same training configuration.

Impact of Positive-Label Ratio. We vary the global high-ISC ratio $\rho$ over $\{30\%, 40\%, 50\%\}$ and perform RL accordingly. As shown in Figure 6(b), larger $\rho$ yields higher accuracy but longer responses, whereas smaller $\rho$ favors stronger compression. The default $\rho= 40\%$ provides an intermediate operating point between two objectives.

Case Study. Appendix § 10 compares reasoning traces from IGPO and AttnPO. IGPO avoids repeated geometric reasoning and self-verification in the first example, and computes the required product directly in the second without unnecessary detours or post-answer verification, using approximately 39–60% fewer CoT tokens while retaining the key derivations. These examples suggest that IGPO more reliably identifies redundant reasoning than AttnPO, leading to more concise responses.

Conclusion

We introduced ISC and IGPO for learning efficient reasoning from intervention-derived step credit. ISC assesses step influence through targeted parameter interventions under fixed reasoning prefixes, while IGPO distills this credit to guide step-wise advantage rescaling. Experiments across two model scales and nine benchmarks demonstrate favorable accuracy–efficiency trade-offs, with up to 62% token reduction on mathematical reasoning tasks. Further analysis suggests that IGPO generates more concise CoT traces with more reliable computations and logical deductions. Future work will explore reducing offline annotation costs and extending training to broader domains.

Limitations

This work has the following limitations: 1) Step-level parameter unlearning approximates interventions on internal computation, so interpreting ISC depends on the effectiveness and selectivity of unlearning; 2) Estimating ISC requires independent parameter interventions for individual steps, incurring a one-time offline annotation cost. Distillation avoids online interventions but does not eliminate this cost; 3) Our training focuses on mathematical reasoning with verifiable answers. Extending the framework to more domains and open-ended tasks without clear verification criteria requires further research.

Supplementary Method Details

Prompts

For rollout generation, we provide only the mathematical question. For answer prediction during ISC annotation, we append the following suffix to $h_n$:

**Final Answer**
\boxed{

We score the answer generated by the base model, averaging log-probabilities only over the content inside $\boxed{\ldots}$.

Step Segmentation

We use double-newline (\n\n) segmentation throughout annotation, predictor training, and policy optimization. Each delimiter belongs to the preceding step. The final answer-bearing step within the CoT and all text after </think> are excluded from both unlearning and advantage rescaling. These tokens retain the original trajectory-level advantage during RL.

Unlearning Objective

The NPO term (Zhang et al., 2024) in Eq. (3) is

The step log-likelihood sums token log-probabilities without length normalization. For retain regularization, each update samples one other correct trajectory. Let $\mathcal{R}$ index its response tokens, including the CoT and final answer, and let $x_t$ denote the context preceding token $y_t$. We use a token-averaged $k_3$ surrogate for KL regularization:

The implementation clips log-ratios and penalties for numerical stability. This regularizer is evaluated on fixed retain trajectories and combined with NPO in Eq. (3), with $\lambda_{\mathrm{KL}}=1$.

Predictor Objective

The shared linear classification head reads the final-layer hidden state at the last token of each step. For a batch $\mathcal{B}$, let $\mathcal{V}_i$ denote the valid annotated steps in trajectory $i$. The training objective is

Thus, each trajectory receives equal total weight.

Policy Objective

We optimize the RLOO advantages (Ahmadian et al., [2]) after ISC rescaling using a PPO clipped policy loss (Schulman et al., 2017). Let $w_{i,t}(\psi)=\pi_{\psi}(o_{i,t}\mid q,o_{i,<t})/\pi_{\psi_{\mathrm{old}}}(o_{i,t}\mid q,o_{i,<t})$ denote the probability ratio between the current and rollout policies. The token-level loss before dual clipping is

where $\epsilon=0.2$. For negative advantages, dual clipping caps this loss at $-c\widetilde{A}_{i,t}$ with $c=3.0$.

Experimental Settings

Training Hyperparameters

Table Table 3 summarizes the hyperparameters for ISC annotation, predictor training, and policy optimization.

Stage	Setting	Value
ISC annotation	Updated modules	LoRA (Hu et al., 2022) in down_proj
	High-ISC proportion $\rho$	Global 0.4
	LoRA rank	32
	Learning rate and update count	$5 \times10^{-4}$, 5 updates per target step
	Optimizer	AdamW
	NPO $\beta$ and retain weight $\lambda_{\mathrm{KL}}$	$\beta= 0.1$, $\lambda_{\mathrm{KL}} = 1.0$
	Retain input length limit	8,192 tokens
	Rollout temperature	0.6 (DS-1.5B), 1.0 (DS-7B)
	Maximum rollout length	16,384 tokens
ISC predictor	Backbone	Qwen3-4B-Instruct-2507
	Trainable parameters	Entire backbone and classification head
	Train/validation split	900/100 trajectories per model scale
	Training epochs	10
	Prediction threshold $\eta$	0.5
	Optimizer	AdamW
	Learning rate and batch size	$2 \times10^{-5}$, 32
	Maximum sequence length	17,408 tokens
Policy optimization	Advantage scaling $\delta$	0.7 (DS-1.5B), 0.5 (DS-7B)
	Length weight $\alpha$	0.15 for both model scales
	Rollout group size $G$	16
	PPO clipping threshold $\epsilon$ / dual-clip $c$	0.2 / 3.0
	KL regularization / entropy bonus	Disabled / disabled
	Optimizer	AdamW
	Learning rate	$2 \times10^{-6}$
	Training steps	1,200
	Batch size	128
	Maximum response length	16,384 tokens
	Rollout temperature	1.0

Evaluation Settings and AES

For mathematical reasoning evaluation, we sample 128 responses per question on AIME2024 and AIME2025 and four on the other benchmarks. We use a sampling temperature of 0.6, $top$-$p=0.95$, and a maximum generation length of 16,384 tokens.

Let $A$ and $L$ denote accuracy and average response length, with $A_b$ and $L_b$ the corresponding base-model values. We compute AES (Luo et al.,

2026. as

Macro averages assign equal weight to each benchmark, with AES computed separately for each benchmark before averaging.

Case Studies

Figure 8 and Figure 9 compare the 7B models on MATH500 and AIME2024. Red marks redundant passages, and green highlights derived results from which the model can advance toward the answer. These highlights are manually annotated.