betaFeedback

WorksComputer Science

Towards Better Legal Reasoning LLMs: Signal Balancing and Reward Scheduling

Abstract

In the legal domain, LLMs can improve reliability and interpretability by generating explicit reasoning traces. However, training LLMs to produce high-quality reasoning traces remains challenging. Existing supervised fine-tuning methods struggle when reasoning traces are significantly longer than final answers, as the learning signal for the answer becomes diluted. Meanwhile, reinforcement learning methods such as Group Relative Policy Optimization also have several limitations, including costly reward design, performance plateaus, and reward hacking. We therefore propose a two-stage training framework. In Stage I, JurisCoT-SFT employs a length-normalized dual objective to balance learning signals between reasoning traces and final answers. In Stage II, Lifecycle-Aware Backtrackable Policy Optimization dynamically evaluates and retires auxiliary reward signals based on their impact on final evaluation objectives, enabling efficient reward utilization. Our framework substantially improves legal reasoning performance on relevant tasks selected from multiple legal benchmarks, including ChJGBench, LawBench, and LegalBench. Experimental results demonstrate consistent improvements in both reasoning trace quality and final answer accuracy, while our models remain competitive with larger models.

Introduction

Recent advances in large language models (LLMs) have highlighted the importance of explicit reasoning, in which models generate intermediate thought processes before producing final answers, for enhancing reliability and interpretability across complex tasks. This form of structured reasoning is particularly crucial in the legal domain, where decisions must be grounded in legal principles and

statutory provisions and must be logically coherent [19].

We observe that high-quality legal reasoning traces are generally quite long, as they must encompass both necessary legal knowledge and a logically structured analysis, as shown in Figure 1. This requires a deep understanding of legal principles, including identifying relevant statutes and legal precedents, as well as the ability to accurately interpret and apply them [9] (Li et al. 2025). At the same time, legal reasoning must adhere to several critical logical constraints, including consistency between statutes and conclusions and internal logical coherence throughout the analysis [6]. Given the extensive domain-specific knowledge and rigorous logical reasoning required, together with the low tolerance for hallucinations in legal contexts [4], generating reliable legal reasoning traces remains significantly more challenging than in general domains.

Figure 1. An anonymized and abridged example of a legal reasoning trace.

Directly post-training a foundation model on legal-domain data is insufficient to address these challenges. During Supervised Fine-Tuning (SFT), reasoning traces are often several times as long as the final answers, causing token-level loss to bias optimization toward the traces and weaken the learning signal for final answers. Moreover, when reasoning traces contain flaws, such as misapplied precedents, SFT cannot provide targeted feedback to correct such errors. In reinforcement learning, methods such as Group Relative Policy Optimization (GRPO) have become mainstream alignment approaches. However, GRPO relies on outcome-level rewards that evaluate only final answers and ignore the legal soundness of intermediate reasoning. A straightforward solution is to introduce multiple rewards for different aspects of reasoning [2] (Wu 2025; Wang et al. 2025). However, this requires multiple manually designed reward functions, which are costly to construct, may introduce conflicting optimization objectives, gradually lose effectiveness during training, or even lead to reward hacking [18] (Wang et al. 2024).

These challenges motivate our two-stage training framework for legal reasoning models. In Stage I, we introduce JurisCoT-SFT, which not only injects domain knowledge into the model but also adopts a length-normalized dual objective to mitigate signal imbalance. In Stage II, we propose LABPO (Lifecycle-Aware Backtrackable Policy Optimization), a self-regulating reinforcement learning framework for automatically managing reward signals during policy optimization.

The central idea of LABPO is to explicitly separate reward signals into two roles: primary rewards, which directly correspond to the final evaluation objectives (e.g., F1 score and accuracy), and auxiliary rewards, which serve as stage-dependent training signals (e.g., output format and language style). Unlike conventional approaches that keep all reward components active throughout training, LABPO treats auxiliary rewards as adaptive optimization instruments whose usefulness evolves over the course of learning.

Each auxiliary signal remains active only while it remains beneficial to training; otherwise, it is retired, and the complete training state is rolled back to the best checkpoint observed during its active period.

We evaluate LLMs on selected tasks from three legal benchmarks: the statutory-law benchmarks ChJGBench [16] and LawBench (Fei et al., 2024), and the case-law benchmark LegalBench [12]. Using Qwen3-8B as the base model, our two-stage framework substantially improves legal reasoning performance across these benchmarks. On ChJGBench, it increases the average score from 0.6406 to 0.7786, achieving the highest score among all evaluated models and outperforming strong general-purpose LLMs such as Gemini-3-Pro-Preview. On LawBench, the resulting model achieves 0.848 on APAP, ranking second overall, and 0.666 on CPCP, ranking first. It also performs strongly on case-law reasoning tasks in LegalBench, attaining the highest LRCLRC accuracy of 0.964 and competitive performance on the numerical reasoning task SaraSara-NN. Despite having only 8 billion parameters, the resulting Qwen3-8B-based model remains competitive with much larger models and even outperforms them on some tasks. Applying the same framework to Llama-3.1-8B likewise yields substantial gains, increasing its ChJGBench average score from 0.3374 to 0.6452 and improving its performance across LawBench and LegalBench. Ablation studies further demonstrate that both stages of the framework contribute to the observed performance improvements.

Our contributions can be summarized as follows:

  1. We propose JurisCoT-SFT, which balances learning signals across reasoning traces and final answers, improving the model’s ability to internalize legal knowledge and produce higher-quality reasoning outputs.

  1. We introduce LABPO, which automatically manages the lifecycle of reward signals during policy optimization by dynamically evaluating and retiring auxiliary rewards based on their impact on final evaluation objectives.

  1. We train a lightweight LLM that performs competitively across multiple benchmarks, establishing a strong lightweight baseline for future research on legal LLMs.

Related Work

Traditional legal-domain LLMs typically follow a two-stage paradigm consisting of continued pretraining and domain-specific fine-tuning to enhance legal capabilities. Representative models include LawGPT (Zhou et al., 2024), HanFei [14], LawLLM (US) (Shu et al., 2024), InternLM-Law [8], and Wisdom-Interrogatory (Wu et al., 2026). Previous studies have also shown that explicitly generating intermediate reasoning traces before producing final answers can improve performance on legal reasoning tasks [17]. DISC-LawLLM (Yue et al., 2024) incorporates a legal syllogism framework into training data design. Recent work has further introduced reinforcement learning to improve reasoning quality in legal LLMs. LexPam (Zhang et al., 2025a) focuses on legal numerical reasoning using a two-stage GRPO approach, optimizing numerical precision while incorporating legal constraints into the reward function. SyLeR (Zhang et al., 2025b) uses supervised fine-tuning (SFT) as a warm-up stage, followed by Proximal Policy Optimization (PPO) to explore multiple reasoning paths, though its reward function mainly relies on general similarity metrics such as ROUGE, which may not fully capture the nuances of legal reasoning. LexPro [3] integrates SFT and GRPO with rule-based format and process rewards, but these rewards primarily target output format and reasoning structure rather than domain-specific legal correctness. LegalΔ\Delta [5] improves reasoning through chain-of-thought-guided information gain with a multidimensional reward mechanism. Despite recent advances, challenges remain in learning from long reasoning traces and in designing reward functions that effectively optimize reasoning performance. To address these

challenges, we propose a two-stage training framework for legal reasoning models.

Method

We introduce a two-stage training framework for signal rebalancing and adaptive reward scheduling, as illustrated in Figure 2.

Two-stage training framework with JurisCoT-SFT and LABPO

Figure 2. The framework has two stages. In Stage I, λ\lambda balances learning between reasoning traces and final answers. In Stage II, auxiliary reward signals are activated, evaluated, and retired, with checkpoint rollback when needed.

Stage I: JurisCoT-SFT

In the legal domain, reliable LLMs must ensure both procedural legitimacy and conclusion correctness. This requires a think-then-answer paradigm. However, using standard SFT presents a critical challenge: the severe length imbalance between the typically lengthy reasoning traces and the concise final answers. During training, this imbalance leads to undue emphasis on the "reasoning" part, weakening the learning signal for the shorter but crucial "answering" part. Given that even minor inaccuracies in the final answer can have high-stakes consequences, addressing this training imbalance is particularly important.

To address this issue, we propose JurisCoT-SFT, a variant of standard SFT that introduces a length-normalized dual objective. Formally, let a training instance be (x,t,a)(x,t,a), where xx denotes the input query, t=(t1,…,t∣t∣)t=(t_1,\ldots,t_{|t|}) represents the provided reasoning sequence, and a=(a1,…,a∣a∣)a=(a_1,\ldots,a_{|a|}) denotes the final answer. Given model parameters θ\theta, we optimize the next-token cross-entropy loss, with the reasoning and answer segments weighted separately and normalized by their respective lengths:

LJurisCoT-SFT(θ)=λ⋅1∣t∣∑i=1∣t∣(−log⁡pθ(ti∣x,t<i))+(1−λ)⋅1∣a∣∑j=1∣a∣(−log⁡pθ(aj∣x,t,a<j)),(1)\mathcal{L}_{\mathrm{JurisCoT\text{-}SFT}}(\theta)=\lambda\cdot\frac{1}{|t|}\sum_{i=1}^{|t|}\left(-\log p_{\theta}(t_i\mid x,t_{<i})\right)+(1-\lambda)\cdot\frac{1}{|a|}\sum_{j=1}^{|a|}\left(-\log p_{\theta}(a_j\mid x,t,a_{<j})\right), \tag*{(1)}
SignalDescription
S1S_1Tag Format Validation
S2S_2JSON Schema Validation
S3S_3Statute Citation Format Verification
S4S_4Language Style Validation
S5S_5Statute Authenticity Verification
S6S_6Statute-Conclusion Consistency Verification

caption: Auxiliary reward signals used during training. Detailed definitions are provided in Appendix § 9.

:::

where λ\lambda is a balancing coefficient that controls the relative contributions of the reasoning trace (tt) and the final answer (aa) to the training objective. A smaller λ\lambda places greater emphasis on the final-answer loss while retaining supervision from the reasoning trace.

Stage II: Lifecycle-Aware Backtrackable Policy Optimization

Although models fine-tuned with JurisCoT-SFT demonstrate strong legal knowledge and produce structured outputs, supervised fine-tuning cannot provide fine-grained supervision over intermediate reasoning steps or deliver differentiated feedback when reasoning traces contain errors.

To address these challenges, we propose LABPO, a reinforcement learning framework that automatically manages reward signals during policy optimization. The central idea of LABPO is to decompose reward signals into two functional roles. Primary rewards are aligned with the final evaluation objectives and remain active throughout training. Auxiliary rewards serve as stage-dependent training signals that may facilitate optimization during certain phases of learning but are not necessarily beneficial throughout the entire training trajectory. Instead of treating auxiliary rewards as fixed components of the reward function, LABPO assigns each auxiliary reward an independent lifecycle: after an initial monitoring period, it remains active as long as the running-best development score improves by at least ϵ\epsilon over the most recent HH evaluation intervals and is retired otherwise.

Let τ\tau denote the development-set evaluation index, with reward gates fixed between evaluations. For input xx and output yy, the training reward is defined as:

R(y;τ)=Rpri(y)+∑i=1mαi(τ)Riaux(y),(2)R(y;\tau) = R^{\mathrm{pri}}(y) + \sum_{i=1}^{m} \alpha_i(\tau) R_i^{\mathrm{aux}}(y), \tag*{(2)}

where RpriR^{\mathrm{pri}} denotes the primary reward, {Riaux}i=1m\{R_i^{\mathrm{aux}}\}_{i=1}^{m} denotes the set of auxiliary rewards, and αi(τ)∈{0,1}\alpha_i(\tau) \in\{0,1\} is a gating variable indicating whether the ii-th auxiliary reward is active.

Adaptive Auxiliary Signal Scheduling

To operationalize the lifecycle management of auxiliary rewards, we propose an Adaptive Auxiliary Signal Scheduling mechanism that automatically regulates the activation and retirement of auxiliary signals during training, thereby alleviating optimization plateaus and suppressing reward hacking. Pseudocode for this mechanism is provided in Appendix § 7.

Design Principles. Instead of keeping all auxiliary signals active throughout the entire training process, we allow the dynamic activation of auxiliary signals. To avoid interference between multiple auxiliary signals, only one auxiliary signal is active at any given time.

Periodic Evaluation. Whenever the retained-step counter reaches a multiple of nn, we evaluate the current policy πτ\pi_\tau on a held-out development set Ddev\mathcal{D}_{\mathrm{dev}}, which is independent of the test sets. The resulting performance score S(θτ)S(\theta_\tau) serves as the sole criterion for determining the lifecycle of the auxiliary signal.

Dynamic Update of Auxiliary Signal Weights. Let τa\tau_a denote the evaluation index at which the current auxiliary signal RaauxR^{\mathrm{aux}}_a is activated. At evaluation step τ\tau, we first define the running-best development score over the active interval:

Bτ=max⁡j∈{τa,…,τ}S(θj).(3)B_\tau= \max_{j \in\{\tau_a,\ldots,\tau\}} S(\theta_j). \tag*{(3)}

If multiple checkpoints attain the same best score BτB_\tau, we select the latest one:

ττ∗=max⁡{j∈{τa,…,τ}∣S(θj)=Bτ}.(4)\tau^*_\tau= \max\{j \in\{\tau_a,\ldots,\tau\} \mid S(\theta_j)=B_\tau\}. \tag*{(4)}

During the first HH evaluations after activation, the auxiliary signal remains active, defining a minimum active period before it becomes eligible for deactivation. Once at least HH evaluations have been collected, we measure the improvement in the running-best score over the most recent HH evaluation intervals:

Gτ=Bτ−Bτ−H,τ−τa≥H.(5)G_\tau= B_\tau- B_{\tau-H}, \qquad\tau-\tau_a \ge H. \tag*{(5)}

The weight of the active auxiliary signal is then updated as follows:

αa(τ+1)={0,τ−τa≥H∧Gτ<ϵ,1,otherwise.(6)\alpha_a(\tau+1)= \begin{cases} 0, & \tau-\tau_a \ge H \land G_\tau< \epsilon,\\ 1, & \text{otherwise.} \end{cases} \tag*{(6)}

The auxiliary signal is therefore retired only when the cumulative improvement in the running-best development score over the preceding HH evaluation intervals is less than ϵ\epsilon. Using the running-best score rather than the latest score makes the retirement decision less sensitive to short-term fluctuations during reinforcement learning.

Checkpoint Rollback and Mechanism Iteration. Let ω\omega, ζ\zeta, and ss denote the optimizer state, learning-rate scheduler state, and retained-step counter, respectively. Each checkpoint stores (θ,ω,ζ,s)(\theta,\omega,\zeta,s) together with its associated development score. When the current auxiliary signal is retired, the complete training state is restored to the highest-scoring checkpoint observed during that signal’s lifecycle:

(θ,ω,ζ,s)←(θτ⋆,ωτ⋆,ζτ⋆,sτ⋆).(7)(\theta,\omega,\zeta,s) \leftarrow(\theta_{\tau^\star},\omega_{\tau^\star},\zeta_{\tau^\star},s_{\tau^\star}). \tag*{(7)}

The stored development score is reused to initialize the next phase, while the auxiliary-reward gates and phase variable are updated separately and are not restored from the checkpoint. The mechanism then activates the next auxiliary signal in the predetermined order and starts a new evaluation-and-scheduling cycle. After all auxiliary signals have been retired, all auxiliary rewards are disabled, and training continues using only the primary reward. Training ends when the retained-step counter reaches the target budget NN. If the running-best development score improves by less than ϵ\epsilon over HH consecutive evaluation intervals before the budget is reached, the best checkpoint observed during this phase is restored and training terminates immediately.

Auxiliary Reward Signal Integration

The goal of LABPO is not to eliminate reward design, but to make reward signals more reusable and easier to deploy. Unlike conventional reinforcement learning approaches that require carefully crafted reward functions to remain consistently effective throughout training, LABPO allows researchers to explore multiple auxiliary reward signals more freely, including potentially noisy or unstable heuristic signals, without predesigning, tuning, or manually combining them in advance. Through dynamic evaluation during training, LABPO automatically identifies useful signals, retires those that cease to provide sufficient improvement, and rolls back the training state when necessary.

As shown in Table 1, we introduce six auxiliary reward signals based on prior work and common evaluation criteria for legal reasoning. The rationale for selecting these signals and their detailed definitions are provided in Appendix § 9. In all main experiments, auxiliary signals are activated sequentially from S1S_1 to S6S_6. To assess whether this ordering affects final performance, we conduct an auxiliary-signal ordering experiment with the six proposed signals, as detailed in Appendix § 8. The results show that, for the auxiliary signals considered in our framework, the optimization order has only a limited impact on final performance.

Experiments

In this section, we describe the experimental setup, report the main results, analyze the training dynamics of auxiliary signal scheduling, compare reasoning performance, and conduct ablation studies. We additionally present a controlled diagnostic experiment on reward-hacking behavior in Appendix § 10.

Experimental Setup

Training Data. We prepare training data for the LLMs in both statutory law and case law domains. For statutory law, we construct a large-scale legal training corpus consisting of {Query, Reasoning Trace, Answer} triplets. The query and answer are extracted from judicial documents, while the reasoning traces are generated by DeepSeek-R1. To ensure generation quality, we apply a variant of rejection sampling to filter generated traces (Appendix § 6). The dataset contains 2.73M instances (Table 12). For case law, we construct a corresponding dataset of triplets. The same rejection sampling procedure is applied to ensure the quality of the generated reasoning traces. The dataset contains 35,965 instances (Table 13).

ItemCount
Training instances (x,t,a)(x,t,a)35,965
Average token count per reasoning trace2,369.2
Average token count per answer3.5

Table 13. Statistics of the case-law training dataset, including the total number of training instances and the average token counts in reasoning traces and answers.

ItemCount
Training instances (x,t,a)(x,t,a)2,734,100
Distinct statutes (clause level)35,010
Causes of action>400>400
Criminal cases581,579
Civil cases1,625,584
Administrative cases526,937
Average token count per reasoning trace3,922.4
Average token count per answer902.2

Table 12. Statistics of the statutory-law training dataset, including the total number of training instances, the number of distinct statutes at the clause level, the distribution of legal cases, and the average token counts per reasoning trace and per answer.

Evaluation Data. We evaluate LLMs on selected tasks from three legal benchmarks: ChJGBench (Hu et al., [16]) and LawBench (Fei et al., 2024) for statutory law, and LegalBench (Guha et al., [12]) for case law. Task details are summarized in Table 14, and metric definitions are provided in Appendix § 16.

TaskTask Full NameLitigation TypeBenchmarkMetric
LawStatutory Provision ApplicationGeneralChJGBenchCase-level set F1
ISSDisputed Issues IdentificationGeneralChJGBenchLLM-as-a-judge
JUSJudicial JustificationGeneralChJGBenchLLM-as-a-judge
ChargeCriminal Charge PredictionCriminalChJGBenchDefendant-level set F1
FineMonetary FineCriminalChJGBenchNVS
RECRecidivism AssessmentCriminalChJGBenchBinary macro-F1
MITMitigating Circumstances IdentificationCriminalChJGBenchBinary macro-F1
Pen_MPenalty Duration PredictionCriminalChJGBenchNVS
SUPCivil Claim Support DeterminationCivilChJGBenchThree-class macro-F1
PAYPayment ObligationCivilChJGBenchNVS
DISAdministrative Disposition PredictionAdministrativeChJGBenchLLM-as-a-judge
APArticle PredictionCriminalLawBenchF1
CPCharge PredictionCriminalLawBenchF1
SPSentence PredictionCriminalLawBenchNLD
LRCLegal Reasoning Causality-LegalBenchAccuracy
NYSJENew York State Judicial Ethics-LegalBenchAccuracy
Sara-NSARA Numeric Reasoning-LegalBenchLog MSE

Table 14. Summary of evaluation tasks, including statutory law tasks and case law tasks, with their corresponding litigation types, benchmark sources, and evaluation metrics. In ChJGBench [16], NVS stands for Normalized Value Similarity.

ChJGBench is a benchmark comprising 3,462 real-world cases spanning criminal, civil, and administrative law. It evaluates three general dimensions shared by all cases: statutory provision application (LawLaw), disputed issues identification (ISSISS), and judicial justification (JUSJUS). Domain-specific tasks include criminal charge prediction (ChargeCharge), monetary fine (FineFine), recidivism assessment (RECREC), mitigating circumstances identification (MITMIT), penalty duration prediction (Pen_MPen\_M), civil claim support and payment obligation (SUPSUP, PAYPAY), and administrative disposition prediction (DISDIS).

LawBench is a public benchmark dataset for evaluating Chinese legal capabilities. We select three tasks: Article Prediction (APAP), Charge Prediction (CPCP), and Sentence Prediction (SPSP).

LegalBench is a benchmark for evaluating the legal reasoning abilities of LLMs using tasks derived from real legal datasets and expert-designed evaluations. We select three representative reasoning tasks for evaluation: Legal Reasoning Causality (LRCLRC), New York State Judicial Ethics (NYSJENYSJE), and SARA Numeric Reasoning (SaraSara-NN).

Baselines. We evaluate our model against a series of strong baselines across the benchmark test sets. These baselines include open-source general-purpose LLMs, such as Llama-3.1-405B [11], DeepSeek-R1 [13] and Qwen3-235B-A22B-Instruct (Yang et al., 2025); closed-source general-purpose LLMs, including Grok-4 (xAI, 2025), GPT-5.1 (OpenAI, 2025), Claude-Sonnet-4.5 [1] and Gemini-3-Pro-Preview [10]; and legal-domain-specific LLMs, including WisdomInterrogatory (Wu et al., 2026) and DISC-LawLLM-7B (Yue et al., 2024).

Implementation Details. We use Llama-3.1-8B and Qwen3-8B as the base models for subsequent fine-tuning. Since Qwen3-8B achieves better overall performance, we conduct all ablation studies using Qwen3-8B as the backbone. Detailed training hyperparameters are provided in Appendix § 17, and the computational overhead of LABPO is reported in Appendix § 18.

Main Results

Results on Statutory-Law Benchmarks

We report the main experimental results on the statutory-law benchmarks ChJGBench and LawBench in Table 2 and Table 3.

ModelsAvg.LawISSJUSCriminal CaseCivil CaseAdmin.
FineMITChargePen_MRECSUPPAYDIS
Specialized Legal LLMs
WisdomInterrogatory0.37280.1580.5820.3840.1350.4080.4360.6080.1080.6490.2440.326
DISC-LawLLM-7B0.44820.1810.7290.5460.0800.7270.4140.4660.3700.6700.0560.420
Open-source LLMs
Llama-3.1-405B0.60060.3420.7120.5750.4760.8850.9570.6020.5000.6820.6160.712
DeepSeek-R10.70800.4220.8190.7180.5630.9610.9690.7400.7150.7830.7530.787
Qwen3-235B-A22B-Instruct0.71360.4610.812<u>0.748</u><u>0.603</u><u>0.974</u>0.967<u>0.780</u><u>0.765</u>0.7780.7440.653
Proprietary LLMs
Grok-40.71660.4990.8180.7280.5010.9730.9760.7240.6680.7610.7090.799
GPT-5.10.67950.4220.7630.7070.5560.965<u>0.975</u>0.6910.6650.7640.7220.687
Claude-Sonnet-4.50.72930.5400.8110.7290.5770.9680.9700.7510.7050.7790.7430.764
Gemini-3-Pro-Preview0.74700.6440.8100.6830.6110.9730.9710.7860.7530.8310.7570.731
Our Methods
Llama-3.1-8B0.33740.0990.5920.3570.1020.5070.2210.5130.3180.5750.1170.254
with JurisCoT-SFT0.62120.4140.7220.5940.4930.8770.8620.7180.6560.7080.6480.587
with JurisCoT-SFT + LABPO0.64520.4370.7430.6210.5180.8910.8930.7340.6720.7260.6710.628
Qwen3-8B0.64060.3150.7910.6520.4590.9250.9600.7270.6730.7260.7080.624
with JurisCoT-SFT<u>0.7667</u><u>0.667</u><u>0.830</u><u>0.748</u>0.5740.9760.9750.7700.7470.8010.717<u>0.802</u>
with JurisCoT-SFT + LABPO0.77860.6890.8460.7510.5790.9710.9740.7660.784<u>0.822</u>0.7240.812

Table 2. Evaluation results of different models on ChJGBench. The best results among all models are shown in bold, and the second-best results are underlined.

ModelsAP ↑CP ↑SP ↑
Specialized Legal LLMs
WisdomInterrogatory0.3280.3510.804
DISC-LawLLM-7B0.0540.1270.670
Open-source LLMs
Llama-3.1-405B0.3060.3520.807
DeepSeek-R10.8310.6060.891
Qwen3-235B-A22B-Instruct0.8650.6070.857
Proprietary LLMs
Grok-40.8440.5530.885
GPT-5.10.7210.5610.883
Claude-Sonnet-4.50.8180.5500.775
Gemini-3-Pro-Preview0.7790.6370.822
Our Methods
Llama-3.1-8B0.0570.2230.707
with JurisCoT-SFT0.7560.2540.849
with JurisCoT-SFT + LABPO0.7710.2880.862
Qwen3-8B0.7920.5370.837
with JurisCoT-SFT0.8220.6130.843
with JurisCoT-SFT + LABPO0.8480.6660.854

Table 3. Evaluation results of different models on LawBench. AP, CP, and SP denote Article Prediction, Charge Prediction, and Sentence Prediction, respectively. The best results among all models are shown in bold, and the second-best results are underlined.

Specialized legal LLMs built on outdated base models show limited capability. Models designed specifically for the legal domain perform relatively poorly overall. On ChJGBench, WisdomInterrogatory and DISC-LawLLM-7B achieve average scores of only 0.3728 and 0.4482, respectively, far below most modern general-purpose LLMs and our trained models. Their weaknesses are particularly evident in decision-oriented tasks such as FineFine and PAYPAY, where both models obtain substantially lower scores than recent open-source and proprietary LLMs. A similar pattern is observed on LawBench, where both models substantially underperform most recent general-purpose LLMs on Article Prediction and Charge Prediction. These results suggest that legal-domain training alone may be insufficient when the underlying base models lack strong general reasoning and instruction-following capabilities.

General-purpose LLMs perform reasonably well overall but still exhibit limitations in legal knowledge. Modern general-purpose LLMs achieve relatively strong overall performance across both benchmarks, indicating that large-scale pretraining provides a useful foundation for legal reasoning. On ChJGBench, Gemini-3-Pro-Preview obtains the highest average score among general-purpose models at 0.7470. However, these models may still show weaker performance on tasks requiring precise grounding in legal knowledge. For example, Gemini-3-Pro-Preview achieves the highest LawLaw score among general-purpose models (0.644), but remains below our best-performing model (0.689).

Our two-stage training framework consistently improves lightweight base models. Applying our proposed two-stage framework consistently

improves both lightweight backbones. On ChJGBench, Llama-3.1-8B improves from 0.3374 to 0.6452 after JurisCoT-SFT and LABPO, while Qwen3-8B improves from 0.6406 to 0.7786. The final Qwen3-8B-based model achieves the best overall score among all evaluated models and obtains top performance on key metrics such as LawLaw (0.689), ISSISS (0.846), JUSJUS (0.751), RECREC (0.784), and DISDIS (0.812). A similar trend is observed on LawBench.

Results on Case Law Benchmark

We report the main experimental results for three case-law tasks from LegalBench in Table 4. The results show that our two-stage framework generalizes beyond statutory-law tasks and remains effective across the evaluated case-law tasks. In particular, Qwen3-8B with JurisCoT-SFT + LABPO achieves the best performance on LRCLRC, improving from 0.873 to 0.964 and outperforming all evaluated models. It also substantially reduces the numerical error on SaraSara-NN, from 8.093 to 1.753. For the NYSJENYSJE task, performance is limited by the model size, resulting in lower scores than those of larger proprietary models. However, it still demonstrates a significant improvement over the base model. Overall, these results demonstrate the effectiveness and cross-domain applicability of our training framework.

ModelsLRC ↑NYSJE ↑Sara-N ↓
Specialized Legal LLMs
WisdomInterrogatory0.4910.387–
DISC-LawLLM-7B0.7460.63019.330
Open-source LLMs
Llama-3.1-405B0.9090.7097.720
DeepSeek-R10.9270.7710.250
Qwen3-235B-A22B-Instruct0.8910.7919.618
Proprietary LLMs
Grok-40.9090.8055.802
GPT-5.10.9460.8367.051
Claude-Sonnet-4.50.8910.7470.994
Gemini-3-Pro-Preview0.8730.7647.677
Our Methods
Llama-3.1-8B0.7640.49321.088
with JurisCoT-SFT0.8550.5756.125
with JurisCoT-SFT + LABPO0.9090.6234.846
Qwen3-8B0.8730.5968.093
with JurisCoT-SFT0.9270.6273.949
with JurisCoT-SFT + LABPO0.9640.6881.753

Table 4. Evaluation results of different models on LegalBench. The best results among all models are shown in bold, and the second-best results are underlined.

Training Dynamics of Auxiliary Signal Scheduling

As shown in Figure 3, the auxiliary signal S5S_5 initially improves the development score but later reaches a plateau. When the running-best development score improves by less than ϵ\epsilon over H=4H = 4 consecutive evaluation intervals, S5S_5 is retired and the policy is restored to the highest-scoring checkpoint observed since S5S_5 was activated. The next auxiliary signal, S6S_6, is then activated, after which development performance resumes an upward trend and eventually exceeds the previous peak obtained under S5S_5. This trajectory indicates that an auxiliary reward may be useful during only part of the optimization process and may become ineffective as training progresses. Restoring the best checkpoint limits the performance degradation accumulated after the auxiliary signal ceases to provide sufficient improvement.

Figure 3. Training dynamics of auxiliary-signal scheduling in LABPO. Detailed trajectories for S1S_1–S4S_4 are omitted for clarity.

Reasoning Performance Comparison

To further investigate improvements in reasoning ability, we conduct a direct evaluation of reasoning traces. Using a win-rate framework, we separately compare the model trained with our method and the Qwen3-8B baseline against DeepSeek-R1. We evaluate reasoning traces along two general dimensions: Helpfulness and Factual Correctness. Gemini-3-Pro-Preview is employed as the judge. The details of this experiment can be found in Appendix § 11. The specifics of the alignment evaluation between LLMs and human annotators are available in Appendix § 13. The LLM evaluator shows strong alignment with aggregated human judgments, achieving an accuracy of 0.84 and a Cohen’s κ\kappa of 0.756.

As shown in Figure 4, our method demonstrates notable improvements in reasoning quality compared to the baseline Qwen3-8B.

Figure 4. Pairwise reasoning-trace win rates against DeepSeek-R1 on ChJGBench. Ours: our two-stage model; Baseline: Qwen3-8B. Purple, blue, and green denote a DeepSeek-R1 loss, tie, and win, respectively.

Compared to the baseline Qwen3-8B. In direct comparisons with the strong open-source model DeepSeek-R1, the win rate of our approach increases from 18.37% to 70.19% on Helpfulness and from 16.46% to 64.47% on Factual Correctness. Overall, the proposed training strategy enables an 8B-scale model to achieve reasoning performance on legal tasks that is competitive with or even superior to that of the 671B-parameter MoE model DeepSeek-R1.

Ablation Studies

Our ablation study on ChJGBench (Table 5) evaluates each component’s contribution; the trends on LegalBench and LawBench are consistent (see Appendix § 14).

No.SFT UsedSFT VariantRL UsedRetireRollbackAvg. ↑
1✓✓✓✓✓0.7786
2✓✓✓✓×0.7718
3✓✓●××0.7671
4✓✓○××0.7684
5✓✓×××0.7667
6✓×✓✓✓0.7687
7✓××××0.7399
8××✓✓✓0.6735
9×××××0.6406

Table 5. Ablation study of different training configurations. ✓ indicates the component is enabled and × indicates it is disabled. Row 3 reduces to GRPO with aggregated primary and auxiliary rewards, while Row 4 reduces to GRPO using only the primary reward.

(1) LABPO Mechanisms Improve Final RL Performance. Rows 1–4 analyze the effect of the Retire and Rollback mechanisms during reinforcement learning (RL). The full configuration (Row 1) achieves the best performance of 0.7786. Removing Rollback (Row 2) decreases the score to 0.7718, and removing both mechanisms (Row 3) further decreases it to 0.7671, indicating that both core mechanisms contribute to LABPO’s final performance. Moreover, Row 3 performs slightly worse than Row 4, which optimizes using only the primary reward. A possible reason is that different rewards may not share fully aligned optimization objectives, and aggregating multiple rewards can introduce noise or optimization conflicts.

(2) JurisCoT-SFT Improves the Effectiveness of Supervised Training. When JurisCoT-SFT is adopted (Row 5), the model achieves a score of 0.7667, outperforming naive SFT (Row 7, 0.7399). A further ablation study of the SFT objectives, together with a sensitivity analysis of λ\lambda, is provided in Appendix G.

(3) LABPO Improves Performance Across Initialization Settings. Comparisons of Rows 1/5, 6/7, and 8/9 show that LABPO consistently improves performance under different initializations, boosting JurisCoT-SFT, naive SFT, and base models from 0.7667 to 0.7786, 0.7399 to 0.7687, and 0.6406 to 0.6735, respectively. This indicates that the gains stem not only from JurisCoT-SFT initialization but also from the proposed RL strategy.

Conclusion

In this paper, we introduce a two-stage training framework to enhance the legal reasoning capabilities of LLMs. We propose JurisCoT-SFT, which balances learning signals between reasoning traces and final answers, and LABPO, which dynamically schedules auxiliary reward signals. Our approach achieves strong performance on statutory and case law benchmarks. The ablation results further confirm the contributions of both training stages.

Limitations

Despite the strong empirical performance and methodological innovations of our approach, several limitations should be acknowledged. First, although the auxiliary signals in the LABPO framework no longer require meticulous selection thanks to the backtracking mechanism, the model still depends on a predefined reward function. Fully automating the design of reward functions (e.g., enabling LLMs to autonomously design appropriate reward signals) remains an open challenge, and we leave this direction for future work. Second, due to resource constraints, we were unable to validate the effectiveness of our proposed two-stage approach on larger-scale models. Third, compared with simpler reinforcement learning methods such as GRPO, LABPO introduces additional computational overhead. The lifecycle management of auxiliary reward signals, periodic evaluation on the development set, and potential rollback operations require extra training iterations and evaluation costs. Although these mechanisms contribute to final performance, they may increase the overall computational budget. Designing more efficient reward scheduling or lightweight rollback strategies to reduce this overhead is an important direction for future research.

Ethics Statement

Given the sensitive nature of the legal domain, the application of artificial intelligence in this field necessitates rigorous ethical management. To address potential ethical concerns, we implement several safeguards. In particular, to prevent the leakage of private information (e.g., personal names), we replace sensitive information in the training datasets with neutral third-person references. This ensures that our research upholds the principles of privacy protection and responsible AI development.

Acknowledgements

This work was supported by the "Pioneer" and "Leading Goose" R&D Program of Zhejiang (2025C02037), Research Project of the Philosophy and Social Sciences Planning Research Platform (Laboratory) of Zhejiang Province, and Key R&D Program of Hangzhou (2025SZDA0254). It was also supported by the National Natural Science Foundation of China (62106039) and Alibaba Research Intern Program. Any opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the funding organizations.

We declare that AI-based writing tools were used only for limited language polishing and grammatical refinement in preparing this paper. The authors have carefully verified all AI-assisted text and take full responsibility for the content of the paper.

Attempt-Bounded Self-Verification

This section presents a method for improving the quality of training data by filtering reasoning traces, as illustrated in Figure 5. This method is called Attempt-Bounded Self-Verification. Specifically, given a question–answer pair (x,a)(x, a), we prompt a strong LLM to generate a reasoning trace tt. We then feed the pair (x,t)(x, t) into the model to be trained, which produces multiple candidate answers a^(k)\hat{a}^{(k)}. If any candidate answer matches the ground-truth answer aa, we regard the corresponding training triple (x,t,a)(x, t, a) as valid and retain it in the training set. We allow up to KK attempts; if none of the

Figure 5. An overview of the Attempt-Bounded Self-Verification framework, in which a query and its reasoning trace are fed into the target model, and the pair is retained only if the model can infer the correct answer within a bounded number of attempts.

generated candidates matches the reference answer, the training triple is discarded.

D′={(x,t,a)∈D∣∃k≤K, a^(k)∼pθ(⋅∣x,t), v(a^(k),a)=1}.(8)\mathcal{D}' = \left\{(x,t,a) \in\mathcal{D} \mid\exists k \leq K,\ \hat{a}^{(k)} \sim p_\theta(\cdot\mid x,t),\ v(\hat{a}^{(k)},a)=1\right\}. \tag*{(8)}

Here, a^(k)\hat{a}^{(k)} denotes the kk-th candidate answer sampled from the model conditioned on (x,t)(x,t), pθ(⋅∣x,t)p_\theta(\cdot\mid x,t) represents the conditional distribution defined by model parameters θ\theta, and v(a^(k),a)∈{0,1}v(\hat{a}^{(k)},a) \in\{0,1\} is a verification function indicating whether the candidate answer agrees with the ground-truth answer aa.

Pseudocode for Adaptive Auxiliary Signal Scheduling

Algorithm 1 summarizes the scheduling procedure. Each checkpoint stores the policy parameters, optimizer state, learning-rate scheduler state, retained-step counter, and associated development score; score⁡(C)\operatorname{score}(C) returns the score stored in checkpoint CC.

Effect of Auxiliary Signal Ordering

Because LABPO activates auxiliary signals sequentially, we examine its sensitivity to signal ordering. To isolate this factor, all runs start from the Qwen3-8B base checkpoint, bypass the JurisCoT-SFT stage, and are trained on the Chinese statutory-law dataset using LABPO. The six auxiliary signals defined in tbl:1 are kept fixed, with only their order varied across the following five settings:

  1. Forward order: S1→S2→S3→S4→S5→S6S_1 \rightarrow S_2 \rightarrow S_3 \rightarrow S_4 \rightarrow S_5 \rightarrow S_6

  2. Reverse order: S6→S5→S4→S3→S2→S1S_6 \rightarrow S_5 \rightarrow S_4 \rightarrow S_3 \rightarrow S_2 \rightarrow S_1

  3. Random order 1: S4→S3→S6→S1→S5→S2S_4 \rightarrow S_3 \rightarrow S_6 \rightarrow S_1 \rightarrow S_5 \rightarrow S_2

  4. Random order 2: S5→S1→S2→S4→S3→S6S_5 \rightarrow S_1 \rightarrow S_2 \rightarrow S_4 \rightarrow S_3 \rightarrow S_6

  5. Random order 3: S3→S4→S1→S6→S5→S2S_3 \rightarrow S_4 \rightarrow S_1 \rightarrow S_6 \rightarrow S_5 \rightarrow S_2

Table 6 summarizes the results for the five tested signal orders. The average scores range from 0.6671 to 0.6735, with a mean of 0.6703 and a sample standard deviation of 0.0027. The difference between the highest and lowest scores is 0.0064. These findings suggest limited performance variation across the tested signal orders in our experimental setting. This conclusion is specific to the auxiliary signals evaluated here and should not be interpreted as evidence that auxiliary signals beyond this set are similarly insensitive to ordering.

Signal OrderAvg. ↑
S1→S2→S3→S4→S5→S6S_1 \rightarrow S_2 \rightarrow S_3 \rightarrow S_4 \rightarrow S_5 \rightarrow S_60.6735
S6→S5→S4→S3→S2→S1S_6 \rightarrow S_5 \rightarrow S_4 \rightarrow S_3 \rightarrow S_2 \rightarrow S_10.6682
S4→S3→S6→S1→S5→S2S_4 \rightarrow S_3 \rightarrow S_6 \rightarrow S_1 \rightarrow S_5 \rightarrow S_20.6671
S5→S1→S2→S4→S3→S6S_5 \rightarrow S_1 \rightarrow S_2 \rightarrow S_4 \rightarrow S_3 \rightarrow S_60.6723
S3→S4→S1→S6→S5→S2S_3 \rightarrow S_4 \rightarrow S_1 \rightarrow S_6 \rightarrow S_5 \rightarrow S_20.6704

Table 6. Ablation study on the effect of auxiliary signal ordering using Qwen3-8B. Reported values are average performance scores computed using Eq. (22).

Reward Functions for Auxiliary Signals

Section 3.2.2 and tbl:1 introduce six auxiliary reward signals for legal reasoning. This appendix provides their detailed definitions and calculation formulas.

These signals broadly cover three categories of errors. Tag Format Validation and JSON Schema Validation address output structure and parsability. Statute Citation Format Verification and Language Style Validation address legal expression and presentation. Statute Authenticity Verification and Statute-Conclusion Consistency Verification address substantive legal reliability.

Tag Format Validation

Model outputs are expected to follow a strict structure consisting of exactly one <think>...</think> block followed by exactly one <answer>...</answer> block. The former contains the reasoning process, whereas the latter contains the final answer. Whitespace is permitted before, between, and after the two blocks, but no other content is allowed. Let YFmt\mathcal{Y}_{\mathrm{Fmt}} denote the set of outputs that satisfy this structure. The reward function is defined as:

RFmt(y)={1,y∈YFmt,0,otherwise.(9)R_{\mathrm{Fmt}}(y) = \begin{cases} 1, & y \in\mathcal{Y}_{\mathrm{Fmt}},\\ 0, & \text{otherwise}. \end{cases} \tag*{(9)}

JSON Schema Validation

The JSON Schema Validation reward is applied only to the payload extracted from the unique <answer>...</answer> block. Let a(y)a(y) denote this payload. We parse a(y)a(y) as a JSON object and validate it against the schema specified for the corresponding task. A reward of 1 is assigned if the payload is valid JSON and contains all required keys with the expected data types. A reward of 0.5 is assigned if it is valid JSON but fails to fully satisfy the schema because one or more required keys are missing or have incorrect data types. If no unique answer payload can be extracted, or if the

Algorithm 1 Adaptive Auxiliary Signal Scheduling for Legal Reasoning
Require: Training set $D_t$; policy $\pi_\theta$; optimizer state $\omega$; learning-rate scheduler state $\zeta$
Require: Development set $D_{\mathrm{dev}}$; score $S(\theta)$
Require: Auxiliary rewards $\{R_i^{\mathrm{aux}}\}_{i=1}^{m}$; activation order $\sigma=(\sigma_1,\ldots,\sigma_m)$
Require: Target retained-step budget $N$; evaluation interval $n$; minimum gain $\epsilon$; patience window $H$
Ensure: Final policy parameters $\theta$
1: $\alpha \gets 0$, $k \gets 1$, $a \gets \sigma_k$, $\alpha_a \gets 1$, and phase $\gets$ auxiliary
2: Set the retained-step counter $s \gets 0$ and the lifecycle-local evaluation index $r \gets 0$
3: $B_0 \gets S(\theta)$; save $C^{\mathrm{best}} \gets \operatorname{Checkpoint}(\theta,\omega,\zeta,s,B_0)$
4: while $s < N$ do
5: Sample $x \sim D_t$, roll out $y \sim \pi_\theta(\cdot \mid x)$, and update $\pi_\theta$ by GRPO using {xref}`Eq. 2`
6: $s \gets s+1$
7: if $s \bmod n=0$ then
8: $r \gets r+1$; compute $q_r \gets S(\theta)$
9: if $q_r \ge B_{r-1}$ then
10: $C^{\mathrm{best}} \gets \operatorname{Checkpoint}(\theta,\omega,\zeta,s,q_r)$
11: end if
12: $B_r \gets \max(B_{r-1},q_r)$
13: if $r \ge H$ and $B_r-B_{r-H}<\epsilon$ then
14: if phase = auxiliary then
15: Retire the active reward: $\alpha_a \gets 0$
16: Restore $(\theta,\omega,\zeta,s)$ from $C^{\mathrm{best}}$; keep $\alpha$ and phase unchanged
17: if $k<m$ then
18: $k \gets k+1$, $a \gets \sigma_k$, and $\alpha_a \gets 1$
19: else
20: $\alpha \gets 0$ and phase $\gets$ primary
21: end if
22: Reset the phase-local history: $r \gets 0$ and $B_0 \gets \operatorname{score}(C^{\mathrm{best}})$
23: else
24: Restore $(\theta,\omega,\zeta,s)$ from $C^{\mathrm{best}}$; keep $\alpha$ and phase unchanged
25: return $\theta$ ▷ Terminate after restoring the best checkpoint from the phase using only the primary reward
26: end if
27: end if
28: end if
29: end while
30: return $\theta$ ▷ The retained-step budget $N$ has been reached

extracted payload is not valid JSON, the reward is 0. The reward function is defined as:

RJSON(y)={1,valid and schema-complete a(y),0.5,valid but schema-incomplete a(y),0,missing or invalid a(y).(10)R_{\mathrm{JSON}}(y)= \begin{cases} 1, & \text{valid and schema-complete }a(y),\\ 0.5, & \text{valid but schema-incomplete }a(y),\\ 0, & \text{missing or invalid }a(y). \end{cases} \tag*{(10)}

Statute Citation Format Verification

Model outputs should include statute citations in a canonical format. If all statute citations are correctly formatted, the model receives a reward of 1. If the citations are missing or incorrect, the reward is 0. The reward function is defined as:

RCFmt(y)={1,correct citation format,0,incorrect format or missing.(11)R_{\mathrm{CFmt}}(y)= \begin{cases} 1, & \text{correct citation format},\\ 0, & \text{incorrect format or missing}. \end{cases} \tag*{(11)}

Language Style Validation

Model outputs are expected to adhere to a formal, neutral, and objective writing style. Because these stylistic attributes are difficult to assess reliably using rule-based methods alone, we employ Gemini-3-Pro-Preview as an LLM-based evaluator. Given a predefined evaluation prompt PStyleP_{\mathrm{Style}} specifying the assessment criteria, the evaluator assigns each response yy a style reward:

RStyle(y)=EStyle(y;PStyle)∈{0,0.5,1},(12)R_{\mathrm{Style}}(y)=E_{\mathrm{Style}}(y;P_{\mathrm{Style}})\in\{0,0.5,1\}, \tag*{(12)}

where EStyleE_{\mathrm{Style}} denotes the LLM-based style evaluator. The reward is assigned according to the following criteria:

  • A score of 1 is assigned when the response is formal, neutral, and objective.

  • A score of 0.5 is assigned when the response is generally formal, neutral, and objective, with only minor subjective or emotional expressions.

  • A score of 0 is assigned when the response is informal or contains substantial subjective or emotional language.

Statute Authenticity Verification

We verify the authenticity of every statute citation instance in the reasoning trace included in output 定位胆 yy, requiring each cited statute to correspond to an actual legal provision. This reward builds on the Non-Hallucinated Statute Rate introduced in prior work [15].

  1. For each statute citation instance, extract the generated statute using regular expressions or LLMs, denoted as Lgen={Sname,Snumber,Scontent}L_{\mathrm{gen}} = \{S_{\mathrm{name}}, S_{\mathrm{number}}, S_{\mathrm{content}}\}, where SnameS_{\mathrm{name}} is the name of the statute, SnumberS_{\mathrm{number}} is the statute number, and ScontentS_{\mathrm{content}} is the content of the statute.

  1. Embed the extracted content together with all statute texts in the reference database using SimBERT (Wang, 2022).

  1. Calculate the semantic similarity between the generated content and each real statute. Select the real statute with the highest similarity as the best match and denote it by Lbest={Sname′,Snumber′,Scontent′}L_{\mathrm{best}} = \{S'_{\mathrm{name}}, S'_{\mathrm{number}}, S'_{\mathrm{content}}\}. The rationale behind this process is that although large models may not generate entirely hallucination-free statutes, they generally maintain high semantic consistency with real statutes. The closer the model-generated statute is to a real statute, the higher its semantic similarity.

  1. Using rule-based comparisons, we determine whether the generated statute LgenL_{\mathrm{gen}} is consistent with the matched real statute LbestL_{\mathrm{best}}. Specifically, LgenL_{\mathrm{gen}} is considered non-hallucinated if the generated statute content occurs as a substring of the matched official statute text, Snumber=Snumber′S_{\mathrm{number}} = S'_{\mathrm{number}}, and Sname∈A(Sname′)S_{\mathrm{name}} \in A(S'_{\mathrm{name}}), where A(Sname′)A(S'_{\mathrm{name}}) denotes the predefined set of accepted names and aliases for the matched statute.

Let N_{\mathrm{cite}}(y) denote the total number of statute citation instances in the reasoning trace of output yy, and let Nnh(y)N_{\mathrm{nh}}(y) denote the number of non-hallucinated statute citation instances among them. We use the proportion of non-hallucinated statute citation instances as the reward score:

RStatReal(y)=Nnh(y)Ncite(y).(13)R_{\mathrm{StatReal}}(y) = \frac{N_{\mathrm{nh}}(y)}{N_{\mathrm{cite}}(y)}. \tag*{(13)}

If no statute citation instance is extracted, i.e., N_{\mathrm{cite}}(y) = 0, we set RStatReal(y)=0R_{\mathrm{StatReal}}(y) = 0. Otherwise, RStatReal(y)R_{\mathrm{StatReal}}(y) takes a value between 0 and 1. A higher value indicates that a larger proportion of the generated statute citation instances correspond to authentic legal provisions.

Statute-Conclusion Consistency Verification

We evaluate whether the statutes cited in each model response support its final conclusion. Because this relationship is difficult to assess reliably using rule-based methods alone, we employ Gemini-3-Pro-Preview as an LLM-based evaluator. Given a predefined evaluation prompt specifying the assessment criteria, the evaluator assigns each response a consistency reward according to the relationship between the cited statutes and the final conclusion:

RConsistency(y)={1,direct support,0.5,potential support,0,contradiction or irrelevance.(14)R_{\mathrm{Consistency}}(y) = \begin{cases} 1, & \text{direct support},\\ 0.5, & \text{potential support},\\ 0, & \text{contradiction or irrelevance}. \end{cases} \tag*{(14)}

A reward of 1 is assigned when the cited statutes directly support the conclusion, 0.5 when they provide potential support, and 0 when they contradict or are irrelevant to the conclusion.

Identifying Reward-Hacking Behavior

To examine whether continuously optimizing against a fixed auxiliary reward can induce reward-hacking behavior, we conduct a controlled diagnostic experiment.

Experimental Setup. We initialize both reinforcement-learning variants from the same JurisCoT-SFT checkpoint, which is also directly evaluated as the SFT baseline. During reinforcement-learning, we retain the complete set of primary rewards used in the main experiments and introduce S5S_5, Statute Authenticity Verification, as the only auxiliary reward.

We evaluate all three models on the same fixed subset of ChJBench and report four metrics. In addition to the Law and S5S_5 reward scores, we report Avg. Statute Citations per Response and Avg. Mentions per Distinct Correct and Relevant Statute to determine whether an increase in the S5S_5 reward score is driven by redundant citations or genuine improvements in legal-task performance.

The two reinforcement-learning variants use the same training data, sampling parameters, and primary rewards; their key distinction is how S5S_5 is scheduled. In conventional GRPO, S5S_5 remains active throughout training, whereas LABPO manages its lifecycle and retires it when its marginal contribution diminishes.

We compare three settings: (1) direct evaluation of JurisCoT-SFT without reinforcement learning; (2) JurisCoT-SFT followed by conventional GRPO using the primary rewards and S5S_5; and (3) JurisCoT-SFT followed by LABPO using the same rewards.

Results. As shown in Table 7, continuously optimizing against S5S_5 under conventional GRPO increases the S5S_5 reward from 0.542 to 0.934, while the task-level Law score changes only marginally, from 0.667 to 0.669. Meanwhile, the average number of statute citations per response increases from 3.18 to 12.46, and the average number of mentions per distinct correct and relevant statute increases from 1.21 to 4.87.

ModelS5S_5 Reward ScoreAvg. Statute Citations per ResponseAvg. Mentions per Distinct Correct and Relevant Statute
JurisCoT-SFT0.6670.5423.181.21
JurisCoT-SFT + GRPO0.6690.93412.464.87
JurisCoT-SFT + LABPO0.6810.7614.271.48

Table 7. Results of the controlled diagnostic experiment on reward hacking using a fixed subset of ChJGBench.

This pattern reveals a clear reward-hacking strategy. Because S5S_5 is computed over individual statute citation instances, repeatedly citing an authentic statute adds to both the number of non-hallucinated citation instances and the total number of citation instances while leaving the number of hallucinated citation instances unchanged. Consequently, the model can increase the ratio-based S5S_5 reward by diluting the contribution of hallucinated citations, without meaningfully improving its performance on statutory provision application.

In comparison, LABPO substantially limits this behavior: the average number of statute citations per response decreases to 4.27, and the average number of mentions per distinct correct and relevant statute decreases to 1.48, while the Law score increases to 0.681. Although its S5S_5 reward is lower than that achieved by conventional GRPO, LABPO performs better on the underlying legal task.

Reasoning Performance Comparison Details

  1. We use the test set described in § 4.1 to generate complete reasoning traces for three different models: Qwen3-8B, Qwen3-8B with JurisCoT-SFT + LABPO, and DeepSeek-R1, all under the same prompt template. The generation settings are as follows: temperature = 0.7, top_p = 0.95, and max_new_tokens = 8192.

  1. For each comparison, we provide Gemini-3-Pro-Preview with an input formatted as {query, ground-truth answer, reference reasoning trace, Model A reasoning trace, Model B reasoning trace}. The evaluator is asked to produce a win, tie, or loss judgment for Model A relative to Model B along two dimensions: Helpfulness and Factual Correctness. To reduce sampling variability, we set the evaluator temperature to 0.

  1. The definitions for these two evaluation dimensions are as follows: Helpfulness primarily evaluates whether the reasoning process is clear, logical, and contributes to solving the problem. Factual Correctness assesses whether the facts in the reasoning process are accurate and error-free. For Factual Correctness, Gemini-3-Pro-Preview uses the ground-truth answer and reference reasoning trace to determine which model’s reasoning trace better aligns with the reference information or contains fewer factual discrepancies.

  1. For any pairwise model comparison, the win rate for a specific dimension is defined as:

Win Rate=Number of wins in that dimensionTotal number of samples.(15)\text{Win Rate} = \frac{\text{Number of wins in that dimension}}{\text{Total number of samples}}. \tag*{(15)}

Ablation Study of Signal-Rebalancing Objectives for SFT

Experimental Setup. To better understand the source of the performance gains from JurisCoT-SFT at the SFT stage, we compare standard SFT, a signal-rebalancing baseline based on Answer-Token Upsampling, and JurisCoT-SFT under different values of λ\lambda. All experiments in this section exclude LABPO, allowing us to isolate the effects of the SFT objective. Across all variants, we use the same base model (Qwen3-8B), training data, batch size, learning-rate schedule, maximum sequence length, and number of training steps.

Standard SFT. For each training instance, let xx denote the input query, t=(t1,…,t∣t∣)t = (t_1, \ldots, t_{\lvert t\rvert}) the reasoning trace, and a=(a1,…,a∣a∣)a = (a_1, \ldots, a_{\lvert a\rvert}) the final answer. We define the unnormalized negative log-likelihood losses over the reasoning and answer tokens as follows:

Lt=∑i=1∣t∣−log⁡pθ(ti∣x,t<i).(16)\mathcal{L}_t = \sum_{i=1}^{\lvert t\rvert} -\log p_{\theta}(t_i \mid x, t_{<i}). \tag*{(16)}
La=∑j=1∣a∣−log⁡pθ(aj∣x,t,a<j).(17)\mathcal{L}_a = \sum_{j=1}^{\lvert a\rvert} -\log p_{\theta}(a_j \mid x, t, a_{<j}). \tag*{(17)}

The standard SFT objective minimizes the average token-level negative log-likelihood over the concatenated reasoning and answer sequence:

LSFT=Lt+La∣t∣+∣a∣.(18)\mathcal{L}_{\mathrm{SFT}} = \frac{\mathcal{L}_t + \mathcal{L}_a}{\lvert t\rvert+ \lvert a\rvert}. \tag*{(18)}

Answer-Token Upsampling. Because legal reasoning traces are often substantially longer than final answers, reasoning tokens may dominate the standard SFT objective, thereby attenuating the supervision provided by the final answer. We therefore consider an Answer-Token Upsampling baseline that increases the contribution of answer tokens by a fixed factor kk:

LAnsUp=Lt+kLa∣t∣+k∣a∣.(19)\mathcal{L}_{\mathrm{AnsUp}} = \frac{\mathcal{L}_t + k\mathcal{L}_a}{\lvert t\rvert+ k\lvert a\rvert}. \tag*{(19)}

We set k=4k = 4 because, on average, the reasoning traces in our data contain approximately four times as many tokens as the final answers. This baseline provides a simple heuristic for compensating for the attenuated answer-token signal. However, it requires kk to be selected a priori and cannot adapt to instance-specific variation in the relative lengths of the two segments.

JurisCoT-SFT. JurisCoT-SFT instead normalizes the reasoning and answer losses separately and combines them using a weighting coefficient λ\lambda:

LJurisCoT−SFT=λLt∣t∣+(1−λ)La∣a∣.(20)\mathcal{L}_{\mathrm{JurisCoT-SFT}} = \lambda\frac{\mathcal{L}_t}{\lvert t\rvert} + (1-\lambda)\frac{\mathcal{L}_a}{\lvert a\rvert}. \tag*{(20)}

We evaluate λ∈{0.25,0.50,0.75}\lambda\in\{0.25, 0.50, 0.75\}. The setting λ=0.50\lambda= 0.50 assigns equal aggregate weight to the reasoning trace and the final answer. Smaller values place greater emphasis on the final answer, whereas larger values place greater emphasis on the reasoning trace.

Results. Standard SFT achieves an average score of 0.7399. Answer-Token Upsampling with k=4k = 4 increases the score to 0.7428, an absolute improvement of 0.0029. This modest gain indicates that simply increasing the global weight of answer tokens provides only limited benefit.

By contrast, all three JurisCoT-SFT variants outperform both standard SFT and Answer-Token Upsampling. The best performance is achieved with

ObjectiveSettingAvg. ↑\uparrow
Standard SFT–0.7399
Answer-Token Upsamplingk=4k = 40.7428
JurisCoT-SFTλ=0.25\lambda= 0.250.7629
JurisCoT-SFTλ=0.50\lambda= 0.500.7667
JurisCoT-SFTλ=0.75\lambda= 0.750.7612

Table 8. Ablation study of SFT objectives without LABPO. All variants are trained with Qwen3-8B under the same experimental configuration. Avg. denotes the average score on ChJGBench.

λ=0.50\lambda= 0.50, yielding a score of 0.7667 and absolute improvements of 0.0268 over standard SFT and 0.0239 over Answer-Token Upsampling. These results suggest that separately normalizing the two segment-level losses and explicitly controlling their relative contributions is more effective in this setting than either standard token-level averaging or fixed answer-token upsampling.

One possible explanation for why λ=0.50\lambda= 0.50 performs best is the quality of the reasoning supervision. In our main experiments, reasoning traces are retained for training only after undergoing quality-control procedures such as Attempt-Bounded Self-Verification. Consequently, these traces provide relatively reliable supervision, making it reasonable to assign comparable aggregate weights to the reasoning and answer losses in the present setting. However, this result should not be interpreted as evidence that equal weighting is universally optimal. When reasoning traces are less reliable, smaller values of λ\lambda may be preferable because they reduce the influence of noisy intermediate reasoning while placing greater emphasis on the final answer. This ability to adjust the balance according to the quality of the reasoning supervision illustrates the flexibility of the parameterized JurisCoT-SFT objective.

Human Evaluation and Inter-Annotator Agreement

Human Evaluation Setup. We conduct a human evaluation to assess the reliability of the proposed reasoning evaluation protocol and to analyze inter-annotator consistency. Three annotators with formal legal training independently evaluate 500 queries sampled from the test set using a stratified sampling strategy. To ensure representativeness, the queries are stratified by (i) legal domain, including criminal, civil, and administrative cases, and (ii) case complexity, measured by the length of the factual description. Each query corresponds to one pairwise comparison instance. The 500 instances are evenly divided into two comparison settings: our proposed method versus DeepSeek-R1 and our proposed method versus Qwen3-8B, ensuring a balanced number of pairs for both baselines.

Annotation Protocol. For each query, annotators perform pairwise comparisons between two anonymized reasoning traces along two dimensions: Helpfulness and Factual Correctness. The order of the two candidates is randomized for each instance to avoid positional bias. All annotations are conducted with annotators blinded to the model identities associated with the reasoning traces.

Aggregation Rule. For each query and each evaluation dimension, annotators assign one of three labels: {A wins, Tie, B wins}\{A\ \text{wins},\ \text{Tie},\ B\ \text{wins}\}. Final human labels are determined using a conservative majority rule: a model is assigned a win label only if at least two out of three annotators explicitly prefer its reasoning trace. If neither model receives at least two win votes, the final label is determined as Tie. This aggregation strategy reduces the impact of subjective or ambiguous judgments and yields more reliable human evaluation results.

Agreement Metrics. We report a set of standard metrics to evaluate both human annotator consistency and human–LLM alignment.

For human inter-annotator agreement, we use Krippendorff’s α\alpha and Majority Agreement. Krippendorff’s α\alpha measures chance-corrected agreement among multiple annotators and is suitable for nominal labels. In our setting, the three comparison outcomes (A wins, Tie, B wins) are treated as nominal categories. Majority agreement measures the percentage of queries for which at least two of the three annotators assign the same label, indicating whether a clear dominant human preference exists and supporting the use of majority voting for label aggregation.

To assess the reliability of the LLM evaluator (Gemini-3-Pro-Preview), we compare its judgments with the aggregated human labels using Accuracy and Cohen’s κ\kappa. Accuracy measures the proportion of exact matches between the LLM prediction and the human consensus, while Cohen’s κ\kappa provides a chance-corrected estimate of agreement between the two.

Results. Table 9 summarizes human inter-annotator agreement and human–LLM alignment. Krippendorff’s α=0.615\alpha= 0.615 indicates a reasonably good level of agreement among the annotators, while the 96.0% majority agreement shows that at least two annotators selected the same label for most queries. The LLM evaluator achieves an accuracy of 0.84 against the aggregated human labels, with Cohen’s κ=0.756\kappa= 0.756 indicating substantial agreement under the same evaluation protocol. These results support the use of the LLM evaluator as an automatic judge for this task.

α\alphaMajorityLLM Acc.LLM κ\kappa
0.61596.0%0.840.756

Table 9. Human inter-annotator agreement and human–LLM alignment results on 500 queries.

Ablation Studies on LawBench and LegalBench

Table 10 and Table 11 report the ablation results on LawBench and LegalBench, respectively, using Qwen3-8B as the backbone model.

No.SFT UsedSFT VariantRL UsedRL RetireRL RollbackAvg. ↑\uparrow
1✓✓✓✓✓0.7893
2✓✓✓×✓0.7593
3✓×✓✓✓0.7651
4✓××××0.7484
5×××××0.7220

Table 10. Ablation study on LawBench. A ✓ indicates that the component is enabled, while × indicates that it is disabled. Average denotes the mean of all metrics in the range [0, 1].

No.SFT UsedSFT VarRL UsedRL RetRL RollAvg. ↑Sara-N ↓
1✓✓✓✓✓0.8261.753
2✓✓×××0.7773.949
3✓×✓✓✓0.7875.906
4✓××××0.7597.231
5×××××0.7358.093

Table 11. Ablation study on LegalBench. A ✓ indicates that the component is enabled, and × indicates that it is disabled. Abbreviations: Var = Variant, Ret = Retire, Roll = Rollback. Average denotes the mean of all accuracy metrics. Sara-N corresponds to the Log MSE metric and is therefore reported separately.

Training Dataset Statistics

Statutory Law Dataset

Table 12 provides detailed statistics of the statutory law training dataset.

Case Law Dataset

Table 13 provides detailed statistics of the case law training dataset.

Detailed Definitions of Evaluation Metrics

ChJGBench Evaluation Metrics

All diagnostic dimensions used in this work, together with their corresponding evaluation metrics, are adopted from ChJGBench [16]. We refer readers to the original benchmark for their detailed definitions.

To account for the unequal numbers of diagnostic dimensions across litigation domains, we introduce AVERAGE, a domain-balanced hierarchical macro-averaging metric. Directly averaging all dimension-level scores would give greater weight to domains evaluated with more dimensions. We therefore first average the dimension-level scores within each domain and then assign equal weight to the resulting three domain-level scores. The resulting AVERAGE scores are reported in Table 2 and Table 5.

Let Mshared={Law,ISS,JUS}M_{\mathrm{shared}} = \{\mathrm{Law}, \mathrm{ISS}, \mathrm{JUS}\}, and let MdM_d denote the domain-specific metrics. For the criminal, civil, and administrative domains, these metrics are {Fine,MIT,Charge,Pen_M,REC}\{\mathrm{Fine}, \mathrm{MIT}, \mathrm{Charge}, \mathrm{Pen\_M}, \mathrm{REC}\}, {SUP,PAY}\{\mathrm{SUP}, \mathrm{PAY}\}, and {DIS}\{\mathrm{DIS}\}, respectively. Each domain-level score averages its shared and domain-specific dimensions:

Sd=1∣Mshared∪Md∣∑m∈Mshared∪Mdsm.(21)S_d = \frac{1}{\lvert M_{\mathrm{shared}} \cup M_d\rvert}\sum_{m\in M_{\mathrm{shared}}\cup M_d} s_m. \tag*{(21)}

The final AVERAGE score assigns equal weight to the three domains:

AVERAGE=SCrim+SCiv+SAdm3.(22)\mathrm{AVERAGE} = \frac{S_{\mathrm{Crim}} + S_{\mathrm{Civ}} + S_{\mathrm{Adm}}}{3}. \tag*{(22)}

LawBench Evaluation Metrics

We select three representative Legal Judgment Prediction tasks from LawBench for evaluation. For classification-based tasks, namely Article Prediction and Charge Prediction, we use F1 as the evaluation metric.

For Sentence Prediction, we use the official LawBench Normalized Log-Distance (NLD) metric (Fei et al., 2024). Let NevalN_{\mathrm{eval}} be the number of fixed-term-imprisonment cases after excluding references involving the death penalty or life imprisonment, and let yiy_i be the nonnegative reference term in months. After converting Chinese numerals, the evaluator parses the first duration in months, or the first duration in years multiplied by 12 if no month duration is found, as the nonnegative integer y^i\hat{y}_i. It then computes

di={∣log⁡(y^i+1)−log⁡(yi+1)∣,y^i is parsed,log⁡216,otherwise,(23)d_i = \begin{cases} \left\lvert\log(\hat{y}_i+1)-\log(y_i+1)\right\rvert, & \hat{y}_i\text{ is parsed},\\ \log216, & \text{otherwise}, \end{cases} \tag*{(23)}
NLD=1−1Nevallog⁡216∑i=1Nevaldi.(24)\mathrm{NLD}=1-\frac{1}{N_{\mathrm{eval}}\log216}\sum_{i=1}^{N_{\mathrm{eval}}}d_i. \tag*{(24)}

If no duration is parsed, the response is an abstention and contributes an instance-level score of zero.

LegalBench Evaluation Metrics

For classification-based tasks in LegalBench, we adopt the Accuracy (Acc) metric to measure prediction correctness. Accuracy is defined as the proportion of correctly predicted samples among all evaluated instances:

Acc⁡=1N∑i=1NI(y^i=yi),(25)\operatorname{Acc} = \frac{1}{N} \sum_{i=1}^{N} \mathbb{I}(\hat{y}_i = y_i), \tag*{(25)}

where NN denotes the total number of samples, y^i\hat{y}_i is the predicted label, yiy_i is the ground-truth label, and I(⋅)\mathbb{I}(\cdot) is the indicator function that returns 1 when the prediction matches the ground truth and 0 otherwise.

For numerical reasoning tasks such as Sara-N, we employ Log Mean Squared Error (Log MSE) to evaluate prediction accuracy on continuous numerical outputs. Log MSE measures the squared difference between the logarithms of predicted and ground-truth values, which helps stabilize variance when values span multiple orders of magnitude:

LogMSE⁡=1N∑i=1N(log⁡(y^i)−log⁡(yi))2,(26)\operatorname{LogMSE} = \frac{1}{N} \sum_{i=1}^{N} \left(\log(\hat{y}_i) - \log(y_i)\right)^2, \tag*{(26)}

where NN denotes the number of samples, y^i\hat{y}_i is the predicted numerical value, and yiy_i is the corresponding ground-truth value.

Implementation Details

We adopt a two-stage training strategy. Training is conducted on 32 NVIDIA A100 GPUs with 80 GB of memory each.

In the first stage, we perform SFT on the Qwen3-8B and Llama-3.1-8B base models with a learning rate of 1×10−51 \times10^{-5} and a sequence length cutoff of 16,384. The per-device training batch size is set to 1, with 2 gradient accumulation steps. We use a cosine learning-rate scheduler with a warmup ratio of 0.1. The balancing coefficient is set to λ=0.5\lambda= 0.5.

In the second stage, we apply reinforcement learning with a learning rate of 1×10−61 \times10^{-6}, a global training batch size of 256, and a micro batch size of 128. In all main experiments, we set the target number of retained optimization steps to N=3000N = 3000, the minimum gain threshold to ϵ=0.0005\epsilon= 0.0005, the patience window to H=4H = 4, and the evaluation interval to n=100n = 100.

LABPO distinguishes retained steps from total executed steps. The retained-step counter ss determines progress toward NN and is restored with the complete lifecycle-best checkpoint, including the policy parameters, optimizer state, and learning-rate scheduler state. The total executed-step count is not restored and is used only for overhead accounting. Consequently, steps discarded by an auxiliary-signal rollback are re-executed during the subsequent training phase.

The primary reward remains active throughout reinforcement learning, while at most one auxiliary reward is active at any time. Development-set evaluation is performed after every n=100n = 100 retained-step increments. Each phase maintains a running-best development score and its corresponding checkpoint. Once r≥Hr \geq H and Br−Br−H<ϵB_r - B_{r-H} < \epsilon, the active auxiliary signal is retired, the lifecycle-best checkpoint is restored, and the next auxiliary signal is activated. The phase-local evaluation history is then reset. The stored score of the restored checkpoint initializes the new phase, avoiding an additional development-set evaluation.

After the last auxiliary signal has been retired, all auxiliary rewards are disabled and training continues using only the primary reward. The same monitoring criterion is applied during this final phase. If the criterion is satisfied, the phase-best checkpoint is restored and training terminates immediately without re-executing the discarded steps; otherwise, training continues until the retained-step counter reaches NN.

The development set is used strictly for model selection, including auxiliary-signal retirement, checkpoint rollback, and early-termination decisions. It is never used for gradient updates or for optimizing the policy or reward models. The training, development, and test sets are strictly disjoint, and all final results are evaluated on a completely held-out test set that is never accessed during training or model selection.

Computational Overhead Analysis

Compared with standard GRPO, LABPO introduces additional overhead from re-executing optimization steps discarded by auxiliary-signal rollback, periodic development-set evaluation, and checkpoint I/O.

Let NN denote the target number of retained optimization steps, UU the total number of executed optimization steps, tstept_{\mathrm{step}} the average time per optimization step, and tevalt_{\mathrm{eval}} the time per development-set evaluation. For auxiliary-signal rollback aa, let rar_a and ra∗r_a^{\ast} denote the retirement index and the lifecycle-best checkpoint index, respectively. Since retirement is checked after every evaluation, the restored checkpoint can be at most HH evaluation intervals before the retirement point. Therefore,

da=n(ra−ra∗)≤Hn,d_a = n(r_a-r_a^*) \le Hn,
Daux=∑a=1Aauxda≤AauxHn≤mHn,(27)D_{\mathrm{aux}} = \sum_{a=1}^{A_{\mathrm{aux}}} d_a \le A_{\mathrm{aux}}Hn \le mHn, \tag*{(27)}

where Aaux≤mA_{\mathrm{aux}} \le m is the number of auxiliary-signal rollbacks and DauxD_{\mathrm{aux}} is the total number of re-executed steps. A rollback during the final phase in which only the primary reward is used is not included in DauxD_{\mathrm{aux}}, because it is followed immediately by termination.

If training reaches NN retained steps, then U=N+DauxU = N + D_{\mathrm{aux}}. Early termination during the final phase can only reduce UU. The development set is evaluated once before training and whenever the retained-step counter reaches a multiple of nn, including repeated evaluations when discarded intervals are re-executed. Because the stored score of a restored checkpoint is reused at phase transitions,

U≤N+Daux≤N+mHn,U \le N + D_{\mathrm{aux}} \le N + mHn,
E(U)=1+⌈Un⌉≤1+⌈N+mHnn⌉,(28)E(U) = 1 + \left\lceil\frac{U}{n} \right\rceil\le1 + \left\lceil\frac{N+mHn}{n} \right\rceil, \tag*{(28)}

where E(U)E(U) is the total number of development-set evaluations.

The computational costs of standard GRPO and LABPO are

CGRPO=Ntstep,C_{\mathrm{GRPO}} = Nt_{\mathrm{step}},
CLABPO=Utstep+E(U)teval+Cio,(29)C_{\mathrm{LABPO}} = Ut_{\mathrm{step}} + E(U)t_{\mathrm{eval}} + C_{\mathrm{io}}, \tag*{(29)}

where CioC_{\mathrm{io}} denotes the scheduling-related cost of checkpoint saving, loading, and restoration.

LABPO performs at most mm auxiliary-signal restorations and one additional restoration during the final phase in which only the primary reward is used.

Under our configuration of N=3000N = 3000, m=6m = 6, H=4H = 4, and n=100n = 100, the worst-case bounds are

Daux≤2400,D_{\mathrm{aux}} \le2400,
U≤5400,U \le5400,
E(U)≤55,E(U) \le55,
CLABPO≤5400tstep+55teval+Cio.(30)C_{\mathrm{LABPO}} \le5400t_{\mathrm{step}} + 55t_{\mathrm{eval}} + C_{\mathrm{io}}. \tag*{(30)}

The corresponding relative overhead over standard GRPO is bounded by

Overhead=CLABPO−CGRPOCGRPO\mathrm{Overhead} = \frac{C_{\mathrm{LABPO}}-C_{\mathrm{GRPO}}}{C_{\mathrm{GRPO}}}
≤0.8+553000tevaltstep+Cio3000tstep.(31)\le0.8 + \frac{55}{3000}\frac{t_{\mathrm{eval}}}{t_{\mathrm{step}}} + \frac{C_{\mathrm{io}}}{3000t_{\mathrm{step}}}. \tag*{(31)}

Thus, auxiliary-signal rollback increases the optimization-step computation by at most 80%, corresponding to at most 1.8× the optimization-step cost of standard GRPO before evaluation and checkpoint I/O are included. This is a theoretical worst-case bound; the actual re-execution overhead Daux/ND_{\mathrm{aux}}/N is smaller when the lifecycle-best checkpoint is close to the retirement point.

Case Study

To assess the model’s accuracy in both reasoning traces and final answers on complex legal reasoning tasks, we examine the representative case shown in Figure 6. This example is drawn from a typical criminal case involving illegal business operations and comprises multiple subtasks, including charge identification, sentencing determination, application of legal provisions, and fine calculation. We compare the outputs produced by the model trained using our proposed algorithm with those generated by two open-source models, Qwen3-8B and DeepSeek-R1, across three dimensions: Charge and Sentence, Legal Provisions, and Fine.

Figure 6. Case study: legal reasoning accuracy and completeness in a judgment concerning illegal business operations.

The case shows that our model correctly identifies the defendant’s voluntary surrender and eligibility for probation, with only a minor two-month discrepancy between the predicted prison sentence (six months) and the reference sentence (eight months). However, it omits one legal provision governing probation supervision (Article 76 of the Criminal Law). In contrast, both Qwen3-8B and DeepSeek-R1 fail to recognize the possibility of probation and instead predict prison sentences of 12 and 18 months, respectively. Additionally, both models omit legal provisions related to probation, including Articles 72, 73, and 76. This incomplete coverage of applicable legal provisions is reflected in incomplete sentencing rationales: both models predict imprisonment without considering probation. This example illustrates the importance of comprehensively recalling applicable legal provisions for reasonable sentencing.

Regarding fine determination, Qwen3-8B predicts a fine of 50,000 CNY, and DeepSeek-R1 predicts a fine of 10,000 CNY; both differ substantially from the reference fine of 20,000 CNY. In contrast, our model matches the reference amount. Notably, before generating the final answer, our

model correctly identifies and applies Article 52 of the Criminal Law, providing a more comprehensive legal basis for fine determination than the reference answer. This suggests that reliable numerical decision-making in legal tasks depends not only on strong multi-step reasoning but also on the integration of domain-specific legal knowledge or structural constraints.

In summary, this example illustrates how general-purpose LLMs may exhibit inadequate sensitivity to procedural and conditional legal rules and weak numerical reasoning in legal contexts. In this case, the proposed training method mitigates these issues, improving accuracy across key judgment elements while maintaining interpretable reasoning traces.

References

  1. [1]Anthropic. 2025. Claude Sonnet 4.5 System Card.
  2. [2]Hua Cai, Shuang Zhao, Liang Zhang, Xuli Shen, Qing Xu, Weilin Shen, Zihao Wen, and Tianke Ban. 2025. Unilaw-r1: A large language model for legal reasoning with reinforcement learning and iterative inference. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 18117–18131, Suzhou, China. Association for Computational Linguistics.arxiv.org/abs/2510.10072
  3. [3]Haotian Chen, Yanyu Xu, Boyan Wang, Chaoyue Zhao, Xiaoyu Han, Fang Wang, Lizhen Cui, and Yonghui Xu. 2025. Lexpro-1.0 technical report. Preprint, arXiv:2503.06949.arxiv.org/abs/2503.06949
  4. [4]Matthew Dahl, Varun Magesh, Mirac Suzgun, and Daniel E Ho. 2024. Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis, 16(1):64–93.DOI
  5. [5]Xin Dai, Buqiang Xu, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Xiaoyuan Yi, Shuo Wang, and Ge Yu. 2026. LegalΔ: Enhancing legal reasoning in llms via reinforcement learning with chain-of-thought guided information gain. In ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 16912–16916.
  6. [6]Saoirse Connor Desai, Stian Reimers, and David A. Lagnado. 2016. Consistency and credibility in legal reasoning: A Bayesian network approach. In Proceedings of the 38th Annual Meeting of the Cognitive Science Society, Recognizing and Representing Events, CogSci 2016, Philadelphia, PA, USA, August 10–13, 2016. cognitivesciencesociety.org.
  7. [8]Zhiwei Fei, Songyang Zhang, Xiaoyu Shen, Dawei Zhu, Xiao Wang, Jidong Ge, and Vincent Ng. 2025. InternLM-law: An open-sourced Chinese legal large language model. In Proceedings of the 31st International Conference on Computational Linguistics, pages 9376–9392, Abu Dhabi, UAE. Association for Computational Linguistics.arxiv.org/abs/2406.14887
  8. [9]Michael J. Gerhardt. 2008. The Power of Precedent. Oxford University Press.DOI
  9. [10]Google DeepMind. 2026. Gemini 3 Pro Model Card. Model release: November 2025; last updated: May 2026.
  10. [11]Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravanakumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.arxiv.org/abs/2407.21783
  11. [12]Neel Guha, Julian Nyarko, Daniel Ho, Christopher Ré, Adam Chilton, Aditya K, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory Dickinson, Haggai Porat, Jason Hegland, and 21 others. 2023. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. In Advances in Neural Information Processing Systems, volume 36, pages 44123–44279. Curran Associates, Inc.
  12. [13]Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 175 others. 2025. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638.DOI
  13. [14]Wanwei He, Jiabao Wen, Lei Zhang, Hao Cheng, Bowen Qin, Yunshui Li, Feng Jiang, Junying Chen, Benyou Wang, and Min Yang. 2023. Hanfei-1.0. https://github.com/siat-nlp/HanFei.
  14. [15]Yinghao Hu, Leilei Gan, Wenyi Xiao, Kun Kuang, and Fei Wu. 2025a. Fine-tuning large language models for improving factuality in legal question answering. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4410–4427, Abu Dhabi, UAE. Association for Computational Linguistics.arxiv.org/abs/2501.06521
  15. [16]Yinghao Hu, Linggang Kong, Qian Xiao, Yuting Huang, Yunlong Zhang, Yujie Zhang, Fei Duan, Yiquan Wu, Kaisong Song, Jun Lin, Kun Kuang, and Fei Wu. 2026. Cjbenchmark: A comprehensive benchmark for chinese legal judgment generation with diagnostic evaluation.DOI
  16. [17]Yinghao Hu, Yaoyao Yu, Leilei Gan, Bin Wei, Kun Kuang, and Fei Wu. 2025b. Evaluating test-time scaling LLMs for legal reasoning: OpenAI o1, DeepSeek-r1, and beyond. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 13759–13781, Suzhou, China. Association for Computational Linguistics.DOI
  17. [18]Cassidy Laidlaw, Shivam Singhal, and Anca Dragan. 2025. Correlated proxies: A new definition and improved mitigation for reward hacking. In International Conference on Learning Representations, volume 2025, pages 88379–88421.arxiv.org/abs/2403.03185
  18. [19]Edward H Levi. 2013. An introduction to legal reasoning. University of Chicago Press.

Paper details

vlnc:W_6r7paenf
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents