betaFeedback

WorksComputer Science

What RL on Expert-Built Tasks Teaches Nemotron 3.5 Super

Abstract

We post-train NVIDIA's Nemotron 3.5 Super (120B total, 12B active parameters) with reinforcement learning (RL) on five expert-built task datasets, one at a time: real-world software engineering, complex constraints, long-horizon tool use, professional chart understanding and enterprise document reasoning. Each run trains a fresh rank-32 LoRA adapter with CISPO on Tinker, with no supervised stage, and each adapter is compared with the base model on the same tasks, harness and settings. On twelve headline benchmarks, eight built by other groups and four by Surge AI, the gains range from +4.5 points on IFBench to +13.8 on GDP_pdf, which rises from 10.4% to 24.2%; SWE-bench Pro V2 rises from 86.9% to 92.1%, Terminal-Bench 2.1 from 63.9% to 70.3%, MultiChallenge from 58.3% to 66.6%, AutomationBench from 14.8% to 27.5% and Chartography from 39.4% to 50.6%. The document adapter, trained without API tools, also gains +8.0 points on AutomationBench. In attempts read side by side with the base model's on tasks each adapter newly solves, the base model's failures recur in a few modes, and each adapter does more often what the base model missed: the tool-use adapter finds the data and the rules before acting, the coding adapter writes its own test or scratch files on SWE-bench Pro V2, the constraints adapter carries earlier turns into its answers, the chart adapter measures the marks in code, and the document adapter reads AutomationBench's spreadsheets.

Introduction

Reinforcement learning (RL) on tasks with verifiable or rubric-graded rewards is now a standard stage of post-training [7, 22]. Most published recipes train on a mixture of datasets, so they show what the mixture adds rather than what one dataset adds to a strong open model: on which benchmarks it helps, by how much, and through which changes in behavior.

We answer these questions for NVIDIA’s Nemotron 3.5 Super [35, 35], a hybrid Mamba-Transformer mixture-of-experts model with 120B total and 12B active parameters. We train it with RL on five of Surge AI’s expert-built task datasets, one at a time: Real-World SWE, Complex Constraints, Long-Horizon Tool Use, Professional Chart Understanding and Enterprise Document Reasoning (§ 3). Each run trains a fresh rank-32 LoRA adapter [17] with CISPO [34] on Tinker [46], with no supervised stage and no KL penalty. We call the five adapters RL-SWE, RL-Constraints, RL-ToolUse, RL-Charts and RL-Docs. Each is compared with the base model on the same tasks, in the same harness and with the same settings, and every gain carries a paired 95% task-bootstrap interval (§ 4).

Over the twelve headline benchmarks (Figure 1, § 5), eight built by other groups and four by Surge AI, the gains range from +4.5 points on IFBench to +13.8 on GDP.pdf, where RL-Docs lifts the score from 10.4% to 24.2%. RL-SWE lifts SWE-bench Pro V2 from 86.9% to 92.1%. RL-ToolUse raises AutomationBench’s pass rate from 14.8% to 27.5%. RL-Constraints, trained only on single-turn requests, gains +8.3 points on MultiChallenge, a multi-turn benchmark. RL-Docs, trained on questions about PDF documents with no API tools, also gains +8.0 points on AutomationBench.

Figure 1. Base and trained scores on the twelve headline benchmarks, grouped by the adapter’s training dataset. Bold numbers are gains in percentage points. Metrics as in Table 3.

To see what changed, we read base and trained attempts side by side on public-benchmark tasks each adapter newly solves, grouped base’s failures into modes and measured matching behaviors over all attempts (§ 6). On AutomationBench, base often acted without finding the task’s data or the rule governing it; RL-ToolUse looks first: the share of tasks where it never opens the inbox or Slack falls from 74.3% to 41.2%. On the coding benchmarks, base often dropped a requirement or ran checks that missed its error; on SWE-bench Pro V2, RL-SWE writes its own test or scratch files more often. On MultiChallenge, base answered the latest turn but dropped what an earlier turn set, which RL-Constraints carries into its answers; on ChartMuseum, base judged close calls by eye; RL-Charts measures the marks in code. RL-Docs reads AutomationBench’s spreadsheets more often.

Contributions.

  1. Five RL runs on one base model, one per expert-built dataset, with a shared recipe: CISPO on rank-32 LoRA adapters, with no supervised stage and no KL penalty (§ 3).

  1. A paired evaluation of each adapter against base on twelve headline benchmarks, on the same tasks and harness and with intervals, plus every other evaluation of the same checkpoints (§§§ 4–§ 5).

  1. A side-by-side analysis that names the leading base failure modes, measures matching behaviors over all attempts, and links each mode to the training reward (§ 6).

Related Work

RL post-training for agents and instruction following. Rule-based rewards drove large-scale reasoning RL in DeepSeek-R1 [7], and Tülu 3 [22] made RL with verifiable rewards a stage of an open post-training recipe. GRPO [42] introduced group-relative advantages; Dr. GRPO [25] removed the per-group standard-deviation scaling, as our runs do; CISPO [34] clips the importance weight instead of the update, and ScaleRL [21] builds a large-scale recipe on it. software-engineering agents, SWE-RL [Wei et al., 2025] learns from software-evolution data with a patch-similarity reward, and DeepSWE-Preview [27], SkyRL [3] and Golubev et al. [ [13] train with RL in executable repository environments, which SWE-Gym [37] and SWE-smith [Yang et al., 2025] build at scale. Schulman and Thinking Machines Lab [ [41] find that LoRA applied to every weight matrix, the MLP layers included, matches full fine-tuning for RL at small ranks. For the same base model, NVIDIA describes its own post-training [35].

Rubric and verifiable rewards. RL rewards have moved from exact answers to instruction following and open-ended tasks through verifiable constraints [39], checklists [Viswanathan et al., 2025] and instance-specific rubrics [14, 16, 29]. Four of our five rewards are graded by a model judge against a rubric or an expert answer, and one by hidden tests. The rubrics state individually gradable requirements, and two of the rewards add a bonus for meeting all of them (§ 3.1).

Transfer from post-training. Studies of RL across reasoning domains find that what transfers between domains depends on the domains [4] (Li et al. 2025). Prior work from our group reports transfer across benchmarks and harnesses from RL on expert-built agentic coding tasks [30], gains in software engineering from post-training on long-horizon office work [31, 40], and multi-turn gains from single-turn constraint training [29]. Here each run starts from Nemotron 3.5 Super and the dataset varies, so each adapter’s gains and behaviors can be read against its own training data.

Training

Five datasets

Each run trains on one of Surge AI’s expert-built task datasets (Table 1). No run uses reference trajectories, supervised fine-tuning or distillation.

Dataset (adapter)DomainTask formatReward and graderTraining harness
Real-World SWE (RL-SWE)Software engineeringA repository at a fixed commit and an expert-written change requestPass only if every hidden test passesOpenCode and mini-swe-agent ports, network blocked
Complex Constraints (RL-Constraints)Instruction followingA single-turn request, often with source material, and a binary rubric(c+w a)/(1+w)(c + w\,a)/(1 + w), w=0.5w = 0.5; GPT-6.1 Sol judgeOne response, no tools
Long-Horizon Tool Use (RL-ToolUse)Agentic tool useA request inside a simulated company, with a rubric of requirements and prohibitionsShare of requirements met, cut for broken prohibitions; GPT-6.1 Sol judgeThe company’s services as MCP tools, network blocked
Professional Chart Understanding (RL-Charts)Chart readingA professional chart, a value-reading question and an expert answerPass only if judged correct; GPT-6.1 Sol judgePython kernel and crop tool
Enterprise Document Reasoning (RL-Docs)Document reasoningA PDF, a question and a rubric whose criteria carry expected values(c+w a)/(1+w)(c + w\,a)/(1 + w), w=0.5w = 0.5; GPT-6.1 Sol judgePage-view, zoom, search and Python tools

Table 1. The five training datasets. cc is the share of a rollout’s rubric criteria met and aa is 1 when all are met. Every judge reads only the final visible answer or the final state, never the reasoning.

Real-World SWE. Each task pairs a real open-source repository at a fixed commit with an expert-written request for a feature, a bug fix or a behavior change, graded by hidden fail-to-pass and pass-to-pass tests in the format of SWE-bench [20]. A rollout earns the reward only if every hidden test passes in a fresh grading sandbox. The rollout groups ran in two harnesses, a port of OpenCode [1] and a port of mini-swe-agent [24] (Yang et al. 2024), and every task in a repository of SWE-bench Verified, SWE-bench Pro or DeepSWE was removed before training.

Complex Constraints. Each task is one professional or personal request with an expert-written rubric of binary criteria, in the format of the ComplexConstraints benchmark [29]. GPT-6.1 Sol grades all criteria of the visible answer in one call, and the reward adds a bonus for meeting every one, so most of its spread within a group comes from that bonus. The judge’s instructions say that a response’s own claims about meeting a criterion are not evidence.

Long-Horizon Tool Use. Each task places the agent inside a simulated company whose services, among them mail, chat, calendar, tickets and files, are exposed as Model Context Protocol (MCP) tools. A rubric lists what the request requires and what must not change. A GPT-6.1 Sol judge grades the services’ final state and the agent’s reply; the reward is the share of requirements met, and a change to something that must not change, or an unrequested side effect, drops it to zero or below.

Professional Chart Understanding. Each task pairs one real-world professional chart with a question that requires reading values off it and an expert answer, often an accepted range. The agent can run code in a persistent Python kernel and crop and enlarge regions of the chart. A rollout earns the reward only if a GPT-6.1 Sol judge, using the Chartography benchmark’s own prompt, marks its final answer correct; hedged answers, such as two alternative values, score zero. Charts matching a Chartography chart were removed before training.

Enterprise Document Reasoning. Each task is one PDF, a question and an expert-written rubric whose criteria carry their own expected values. The agent sees a first view of the document and can call page-view, zoom, search and Python tools. GPT-6.1 Sol grades one criterion per call with the GDP.pdf benchmark’s own prompt, and training tasks whose document matched a GDP.pdf document were removed before training.

Recipe

All five runs share one recipe; Appendix § 8 lists the settings that differ. Each run trains a fresh rank-32 LoRA adapter on the attention and MLP weights, the experts included, and the unembedding; the vision encoder is frozen in the chart and document runs. The loss is Tinker’s CISPO [34, 46]: a token-level policy gradient whose importance weight, the ratio of trainer to sampler probabilities, is clipped to [0,4][0, 4] and not differentiated. A rollout’s advantage is its reward minus the mean reward of its group, with no standard-deviation scaling [25], and groups whose rollouts all earn the same reward are dropped. There is no supervised stage, no KL penalty and no entropy bonus.

The optimizer is AdamW [26] with a peak learning rate of 1×10−41 \times10^{-4} (2×10−42 \times10^{-4} for RL-SWE) after a short warm-up. Rollouts sample at temperature 1.0 and top-pp 1.0, so that sampler probabilities match the policy the loss assumes. Sampling runs asynchronously: a group may start on sampler weights a few versions older than the trainer’s, up to the run’s staleness cap. Three runs add small shaping terms against overlong responses, cut-off turns or over-persistence.

Tinker served the base model from two deployments: RL-SWE and RL-ToolUse trained on an early-access deployment and the other three adapters on the deployment listed in Tinker’s documentation, and each adapter is compared with base on its own deployment.

Evaluation

Benchmarks. Table 2 lists the twelve headline benchmarks [6, 32, 45, 19, 16, 39, 8, 44, 29, 38, 11 b] (Shephard and Salimans 2026). Each adapter is evaluated on the benchmarks of its own domain. Eight come from other groups, and ComplexConstraints, HANDBOOK.md, Chartography and GDP.pdf were built by Surge AI. Hidden tests or programmatic checkers grade the three coding benchmarks, IFBench and AutomationBench, and LLM judges grade the other seven. § 5.2 adds every other evaluation of the same checkpoints.

Table 2: The twelve headline benchmarks. Tasks are those scored for both models, times attempts per task per model. T^{\mathrm{T}} marks the harness the adapter trained in. AdvancedIF’s prompts and rubrics were written by Surge experts for Meta.

BenchmarkAdapterTasksMetricGraderHarness
Built by other groups
SWE-bench Pro V2RL-SWE635 × 2Resolve rateHidden testsOpenCodeT^{\mathrm{T}}, no network
Terminal-Bench 2.1RL-SWE89 × 5Pass rateHidden testsOpenCodeT^{\mathrm{T}}, open internet
DeepSWERL-SWE112 × 4Pass rate of the committed changesHidden testsOpenCodeT^{\mathrm{T}}, context compaction
AdvancedIFRL-Constraints1,643 × 2Every rubric item met, mean of three subsetsClaude Opus 5.5One responseT^{\mathrm{T}} per turn
IFBenchRL-Constraints300 × 8Share of constraints met, strictProgrammatic checkersOne responseT^{\mathrm{T}}
MultiChallengeRL-Constraints266 × 8Mean of four axis pass ratesClaude Opus 5.5Conversation replayT^{\mathrm{T}}
AutomationBenchRL-ToolUse600 × 1Pass rate, every assertion metProgrammatic assertionsIts own agent loop and API tools
ChartMuseumRL-Charts1,000 × 2Accuracy, format-lenientGPT-6.1 SolPython and crop toolsT^{\mathrm{T}}
Built by Surge AI
ComplexConstraintsRL-Constraints75 × 8Every criterion metClaude Opus 5.5, per criterionOne responseT^{\mathrm{T}}
HANDBOOK.md v1.1RL-ToolUse65 × 5Mean share of criteria metGPT-6.1 Sol agent judgeOpenHands [Wang et al., 2025], MCP tools
ChartographyRL-Charts100 × 10AccuracyGPT-6.1 Sol, official scorerPython and crop toolsT^{\mathrm{T}}
GDP.pdfRL-Docs100 × 5Every criterion metGPT-6.1 Sol, per criterionDocument toolsT^{\mathrm{T}}

Table 2. The twelve headline benchmarks. Tasks are those scored for both models, times attempts per task per model. T^{\mathrm{T}} marks the harness the adapter trained in. AdvancedIF’s prompts and rubrics were written by Surge experts for Meta.

Paired protocol. Base and each adapter run the same tasks in the same harness, with the same prompts, caps and sampling settings; only the adapter differs, and base runs on the adapter’s own deployment. Every headline evaluation samples at temperature 1.0 and the model’s full reasoning effort, with top-pp 0.95, or 1.0 on AutomationBench and HANDBOOK.md as in training. Appendix § 9 gives the caps and the tasks scored.

Intervals. Every gain is trained minus base in percentage points. Its 95% interval is a paired task bootstrap [9, 33]: per task we average each model’s attempts and take the difference, then resample tasks with replacement 10,000 times. Gains are computed from unrounded rates. Appendix § 9 gives the details.

Results

Headline benchmarks

Table 3 gives the twelve headline results, and Figure 1 draws them.

BenchmarkAdapterBaseTrainedGain [95% CI]
SWE-bench Pro V2RL-SWE86.992.1+5.2 [+3.3, +7.1]
Terminal-Bench 2.1RL-SWE63.970.3+6.3 [+2.4, +10.6]
DeepSWERL-SWE27.932.6+4.7 [−0.7, +9.8]
ComplexConstraintsRL-Constraints27.335.5+8.2 [+2.3, +14.0]
AdvancedIFRL-Constraints50.658.3+7.7 [+5.8, +9.7]
IFBenchRL-Constraints72.376.8+4.5 [+2.4, +6.7]
MultiChallengeRL-Constraints58.366.6+8.3 [+5.5, +11.2]
AutomationBenchRL-ToolUse14.827.5+12.7 [+9.3, +16.2]
HANDBOOK.mdRL-ToolUse54.867.6+12.8 [+9.5, +16.2]
ChartographyRL-Charts39.450.6+11.2 [+7.5, +15.0]
ChartMuseumaRL-Charts79.3585.80+6.45 [+4.80, +8.10]
GDP.pdf, published rubricsRL-Docs10.424.2+13.8 [+7.8, +20.2]
corrected rubricsbRL-Docs12.431.0+18.6 [+12.2, +25.0]

Table 3. Headline results. Base and trained scores are in percent; the gain is trained minus base in percentage points, with its 95% paired task-bootstrap interval. Metrics as in Table 2.

Coding. RL-SWE gains +5.2 points on SWE-bench Pro V2, where base already resolves 86.9% of tasks, +6.3 on Terminal-Bench 2.1 and +4.7 on DeepSWE.

Instruction following. RL-Constraints gains on all four instruction benchmarks: +8.2 points on ComplexConstraints, +7.7 on AdvancedIF, +4.5 on IFBench and +8.3 on MultiChallenge. Its training requests were single-turn, yet MultiChallenge, a multi-turn benchmark, gains as much as ComplexConstraints, the benchmark closest to training. IFBench’s prompt-level loose accuracy, the metric its paper reports, also gains +4.5 [+2.4, +6.7].

Tool use. RL-ToolUse raises AutomationBench’s pass rate from 14.8% to 27.5%, and its outcome score, which is zero when any rule of the task is broken, from 31.9 to 49.5 (+17.6 [+14.3, +20.9]). AutomationBench’s six departments gain +5.0 to +23.0 points (Appendix § 10). On HANDBOOK.md, the mean share of each task’s criteria that RL-ToolUse meets rises from 54.8% to 67.6%.

Charts and documents. With its Python and crop tools, RL-Charts gains +11.2 points on Chartography and +6.45 on ChartMuseum (+7.35 with the official scorer). RL-Docs gains +13.8 on GDP.pdf with the published rubrics and +18.6 with the corrected rubrics; with the published rubrics, it gains +18.9 [+14.4, +23.7] points on the criteria that require specific content.

Other evaluations of the same checkpoints

Table 4 lists every other evaluation of the five checkpoints, among them Terminal-Bench 2.1 in Terminus 2 [15], SWE-bench Verified [5], CharXiv-RQ [Wang et al., 2024], ChartQAPro [28], OCR-Reasoning [18] and OCRBench v2 [10]. The three second runs reproduce their headline gains. On data from the training distribution, RL-SWE gains +8.6 points on the held-out SWE tasks at the training effort (medium), and RL-Constraints gains +11.6 on the Complex Constraints dev split at the training token cap. Without its tools, RL-Charts gains +1.75 points on ChartMuseum. RL-Docs gains +3.4 to +5.0 points on GDP.pdf without tools and +9.4 at temperature 0.6.

Table 4: Every other evaluation of the five checkpoints. “Setting” says what differs from the adapter’s headline evaluation. Base and trained scores are in percent; gains are in percentage points with 95% paired task-bootstrap intervals.

EvaluationSettingBaseTrainedGain [95% CI]
RL-SWE
SWE-bench Pro V2Second run, same settings86.691.8+5.2 [+3.3, +7.1]
Terminal-Bench 2.1Second run, same settings63.869.4+5.6 [+2.0, +9.4]
Terminal-Bench 2.1Another harness, Terminus 265.867.4+1.6 [−2.2, +5.4]
SWE-bench VerifiedAnother benchmark, same harness as SWE-bench Pro V273.372.8−0.5 [−2.7, +1.8]
Held-out SWE tasksRepositories kept out of training; training harnesses and grader35.437.8+2.4 [−0.2, +5.1]
Held-out SWE tasksAs above, at the training effort (medium)31.139.7+8.6 [+6.4, +10.8]
DeepSWEFive other harness, effort and limit settings; no base run–10.7–27.5–
RL-Constraints
Complex Constraints dev splitHeld-out split of the training data; training judge and token cap40.251.8+11.6 [+8.1, +15.3]
RL-Charts
ChartographySecond run, 3 attempts per task36.047.3+11.3 [+5.0, +17.7]
ChartographyaNo tools15.916.1+0.2 [−2.6, +2.8]
ChartMuseumNo tools70.5072.25+1.75 [+0.30, +3.20]
CharXiv-RQAnother benchmark, training tools84.0585.45+1.40 [−0.05, +2.85]
CharXiv-RQAnother benchmark, no tools77.0578.25+1.20 [−0.25, +2.70]
ChartQAProAnother benchmark, no tools68.7968.89+0.10 [−0.69, +0.92]
OCR-ReasoningAnother benchmark, no tools66.1865.90−0.28 [−1.68, +1.12]
OCRBench v2 (English)Another benchmark, no tools68.3268.57+0.25 [−0.29, +0.79]
RL-Docs
GDP.pdfTemperature 0.610.219.6+9.4 [+4.0, +15.0]
GDP.pdfbNo tools; four context and prompt settings7.0–10.410.4–14.2+3.4 to +5.0
AutomationBenchA domain RL-Docs never trained on13.521.5+8.0 [+5.2, +11.0]

Table 4. Every other evaluation of the five checkpoints. “Setting” says what differs from the adapter’s headline evaluation. Base and trained scores are in percent; gains are in percentage points with 95% paired task-bootstrap intervals.

The document adapter on AutomationBench

RL-Docs was trained on questions about PDF documents, with document tools and no API tools. We ran it on AutomationBench against base on its own deployment, in the same session. It gains +8.0 points in pass rate [+5.2, +11.0], +7.8 in outcome score [+4.7, +10.9] and +15.4 in the share of required checks met [+12.5, +18.2]. RL-ToolUse, trained on the tool-use data, gains more on each: +12.7, +17.6 and +20.6 [+17.7, +23.6]. The two differ most on the tasks’ rules. The share of tasks that break at least one rule rises from 34.7% to 40.0% with RL-Docs (+5.3 [+1.7, +9.0]) and falls from 37.3% to 32.5% with RL-ToolUse (−4.8 [−8.5, −1.2]). § 6 compares their behavior.

What Changed: A Side-by-Side Analysis

We paired base and trained attempts on the same tasks and read them side by side: tool calls, code, messages, final answers and, where saved, reasoning. The pairs come from public-benchmark tasks the trained model newly solves: AutomationBench tasks that RL-ToolUse or RL-Docs passed and base failed; DeepSWE, SWE-bench Pro V2 and Terminal-Bench 2.1 tasks on which every base attempt failed and RL-SWE passed at least once; IFBench and MultiChallenge items that base failed in every attempt and RL-Constraints passed in at least half; and ChartMuseum questions that base failed in both attempts and RL-Charts passed in both. Where a pool held more than 30 tasks, we read a random sample of 30; otherwise we read every pair. Claude Opus 5.5 agents described each base failure in a phrase and grouped the phrases into modes (Table 5). Other Claude Opus 5.5 agents, given the first reading’s codebook but not each pair’s mode, read every pair again and gave the same mode as before to 29 of the 30 RL-ToolUse pairs, 23 of the 30 RL-SWE pairs and all other pairs; a third agent re-read the data to settle each disagreement. Figure 2 gives the rates of Table 5 over all attempts.

Base failure mode (pairs)Behavior after RLLink to trainingRate over all attempts
RL-ToolUse on AutomationBench, 30 pairs
Data or target not found (12)Keeps looking: lists the Drive, queries by exact nameShare of requirements met; a stop meets noneNever reads Sheets or Drive: 24.4 → 7.2, −17.2 [−20.6, −14.0]
Rule or update kept elsewhere, not read (10)Reads every source first: the whole inbox, every tabA broken prohibition drops the reward to zero or belowNever opens the inbox or Slack: 74.3 → 41.2, −33.2 [−37.0, −29.5]
RL-SWE on DeepSWE, SWE-bench Pro V2 and Terminal-Bench 2.1, 30 pairs
Checks that missed the error (8)Tests the new behavior with a check of its ownPass only if all hidden tests passOwn test or scratch file, SWE-bench Pro V2: 14.3 → 32.6, +18.3 [+15.7, +20.9]
Lost requirement (13)Carries each requirement into the codeOne dropped requirement scores zero—
RL-Constraints on MultiChallenge and IFBench, 11 pairs
Earlier turn not carried into the answer (8, MultiChallenge)Carries earlier facts, instructions and versions forwardBonus only if every criterion is metAxes: inference memory +9.3 [+4.9, +13.9], retention +6.3 [+2.9, +10.3], version editing +14.4 [+6.3, +22.8]
Form rejected by the checker (3, IFBench)Avoids forms a strict reader could count against a ruleThe judge reads only the final answerUnrequested bold: 17.1 → 5.8, −11.3 [−14.3, −8.5]; line indentation met: 56.9 → 33.3, −23.6 [−36.1, −9.7]
RL-Charts on ChartMuseum, 24 pairs
Close call judged by eye (10)Measures the marks in codeAll or nothing against an expert answerNo measurement code, questions other than text reasoning: 67.9 → 44.4, −23.5 [−25.9, −21.2]
Encoding misread (7)Settles what each mark encodes firstAs above; a misread encoding scores zero—
RL-Docs on AutomationBench, 30 pairs
Source not found (11)Lists the Drive or queries further, then reads the sourceCriteria carry values from the sourceNever reads Sheets or Drive: 27.5 → 11.7, −15.7 [−19.1, −12.3]
Inbox not opened (6)Reads the inbox before actingAs aboveNever opens the inbox or Slack: 71.5 → 56.8, −14.7 [−18.2, −11.2]
One checked detail missed (11)Keeps the detail as the task gives itEach criterion graded on its own—

Table 5. The leading base failure modes in the pairs read (number of pairs; the rest are in the text), the adapter’s behavior on the same tasks, the part of the reward that favors it, and the matching rate over all attempts: base → trained in percent, change in points with its 95% interval (Figure 2; Sheets and Drive rates over spreadsheet-seeded tasks; MultiChallenge: axis pass-rate changes).

Figure 2. The rates of Table 5 over all attempts: base (gray) and the trained adapter (blue), with each paired change in points and its 95% task-bootstrap interval. Behavior rates are counted by rules on the tool calls, code and answers; the MultiChallenge rows are axis pass rates, and the IFBench line-indentation row is the share of attempts that meet that constraint. Spreadsheet tasks are the AutomationBench tasks seeded with a spreadsheet; RL-Docs is compared with base on its own deployment, and SWE-bench Pro V2 is measured on its second run (Table 4).

RL-ToolUse finds the data and the rules before acting

In 12 of the 30 pairs, base did not find where the task’s data or target lived and stopped or used something of its own; in 10 it acted without a rule or update kept in another message, tab or sheet; in 7 it did most of the job but dropped or botched one step, and 1 pair fit no mode. On spreadsheet-seeded tasks, the share that never reads Google Sheets or Drive falls from 24.4% to 7.2%; over all tasks, the share that never opens the inbox or Slack falls from 74.3% to 41.2%, and RL-ToolUse meets 67.9% of the required checks held only in those messages, against 26.6% (+41.2 [ +34.1, +48.4]).

Before. A finance task asked for statements to clients with outstanding balances and added: “Follow our standard statement distribution policy.” The base model read the invoices in Wave, searched the tool catalog for a policy and never opened the inbox, where the accounts-receivable manager’s message set a minimum balance below which no statement is sent. It decided to assume standard practice, wrote “I now have all the data I need.” and sent two statements, one for a balance under that minimum, which breaks one of the task’s rules.

After. RL-ToolUse listed the inbox, read the policy and sent one statement, for the balance above the minimum.

Link to training. Training rubrics list what must not change as well as what the request requires, and a broken prohibition drops a rollout’s reward to zero or below (§ 3.1), so a rollout that breaks a rule kept in another service scores below one that read the rule first.

RL-SWE checks its work against the requirement

In 13 of the 30 pairs, base dropped or misread a stated or clearly implied requirement (as in the lost-requirements mode of Mehta et al. [ [30]). In 8 it checked its work, but its checks missed the error: they covered only cases it had built, only printed output or only existing tests, or rested on its own assumption (as in that paper’s narrow-testing and weak-ground-truth modes). In 5 it broke behavior that had to keep working, and 4 failed for other reasons. On DeepSWE, both models create a test or scratch file of their own in most attempts (82.6% and 85.9%); on SWE-bench Pro V2, RL-SWE creates one in 32.6% of attempts against 14.3% for base, and runs the tests after its last code change in 91.9% against 83.9% (+8.0 [+5.7, +10.2]).

Before. An Element Web task asked discovery to expose delegated-authentication fields from the stable m.authentication entry, looked up through the SDK constant M_AUTHENTICATION. The base model read discoveryResult[M_AUTHENTICATION.name], ran the existing tests, type checker and linters, and reported the change verified. None of these checks exercised the new field. In this

SDK version .name is the unstable key org.matrix.msc2965.authentication, so the stable entry was never found and both hidden tests that expect it failed.

After. RL-SWE wrote the same lookup, then a temporary test that fed discovery the stable entry. The test failed, and a log line it added showed the unstable key. It read the SDK’s UnstableValue class, looked the entry up under both names and deleted the test once it passed.

Link to training. A rollout earns reward only if every hidden test passes, including fail-to-pass tests of behavior the repository’s own tests do not cover; one that tests the new behavior can catch such an error before submitting, and the group-relative advantage credits that testing with the fix.

RL-Constraints carries earlier turns into the answer

In all 8 MultiChallenge pairs, base answered the latest turn but dropped an earlier fact, instruction or version, which the benchmark’s inference-memory, instruction-retention and version-editing axes test; on the three inference-memory items, 18 of its 24 attempts mentioned in their reasoning an earlier fact the item turns on, and none passed; the three axes gain +9.3, +6.3 and +14.4 points. In the 3 IFBench pairs, base’s answers used forms the rule-based checker rejects, such as list numbers or bold markup, and RL-Constraints avoids such forms (unrequested bold 17.1% → 5.8%). The same caution lowers the share of attempts meeting the line-indentation constraint (each line indented further than the last) from 56.9% to 33.3%: in 48 of 72 attempts, against 30 for base, RL-Constraints puts those lines in a code block, whose unindented fences break the pattern.

Before. A MultiChallenge user mentions a hypertension diagnosis and a wish to lose weight in the first turn and later asks for a weekly plan to train with a partner who has asthma and wants to build muscle. Every base attempt plans muscle gain with asthma precautions, and none plans for the user’s weight loss; the 4 whose reasoning never returns to the first turn omit blood pressure too.

After. RL-Constraints passes 5 of 8 attempts, exactly those whose reasoning recalls the weight-loss goal, which no base attempt’s reasoning does. Its first passing plan warns against holding the breath, which can raise blood pressure, and adds walks for weight loss.

Link to training. Training requests are single-turn, often with pasted source material, and earn a bonus only when every criterion is met (§ 3.1), so an answer that drops one requirement loses it. A conversation spreads its requirements over earlier text in the same way.

RL-Charts measures the marks and settles what they encode

In 10 of the 24 pairs, the answer turned on a difference too fine to settle by eye; in 7 base read the right marks with the wrong meaning, such as an inverted color key; in 6 it answered from printed labels or outside knowledge instead of the marks, and in 1 it misread the question. Base ran measurement code in 13 of its 48 attempts on these questions and RL-Charts in 37. Over all ChartMuseum questions other than text reasoning, the attempts without measurement code fall from 67.9% to 44.4%, and in Chartography’s second run (Table 4) from 38.7% to 12.0% (−26.7 [−34.0, −19.7]).

Before. A ChartMuseum scatter plot of OECD countries sets weekly wage against weekly hours and asks which country in a given wage band works the most hours. South Korea’s bubble straddles the gridline at the band’s lower edge, its center just inside. In both attempts the base model judged the band by eye, from the full chart and one crop that missed South Korea, and answered New Zealand, which works fewer hours; the second attempt named New Zealand and Ireland as tied.

After. RL-Charts found the gridlines in code, detected the bubbles and converted nearby centers to data, putting South Korea inside the band and Israel below it; it answered South Korea both times.

Link to training. Training answers are graded all or nothing against an expert answer that is often an accepted range (§ 3.1). On close calls, rollouts that measure earn the reward more often than rollouts that estimate, and the group-relative advantage (§ 3.2) credits the measuring.

RL-Docs finds the data in a domain it never trained on

RL-Docs, trained with document tools and no API tools, passes 69 AutomationBench tasks that base fails. In 11 of the 30 pairs read, base never found the spreadsheet or record that held the data and used values of its own, guessed endpoints or stopped; in 6 it never opened the inbox that held the procedure or a rule; in 11 it missed one detail a check reads, such as a full name; and in 2 it stopped early. On spreadsheet-seeded tasks, the share that never reads Sheets or Drive falls from 27.5% to 11.7%; over all tasks, the share that never opens the inbox or Slack falls from 71.5% to 56.8%.

Before. A finance task asked for invoices to be converted to USD at “the most recent rates available” and logged to the accounts-payable tracker. The base model read the invoices and searched for a rate service. Its searches found a Drive endpoint for deleting files but none for listing them, and it concluded that without a listing it could not find the tracker. It used remembered rates and logged wrong amounts to a new spreadsheet; the Drive held the tracker and its dated rates.

After. RL-Docs searched for a Drive listing, found the tracker and reasoned that the rates might be in the spreadsheet; it logged and e-mailed the amounts at the latest dated rates.

Link to training. RL-Docs’ criteria carry their own expected values, found by paging, zooming or searching the document (§ 3.1); a remembered rate misses such a criterion, so finding the value scores higher. Training had no Drive and no spreadsheet files; looking the value up carried over.

A common thread

In most pairs, base acted before establishing what the task required: it missed the data or the rule (RL-ToolUse, RL-Docs); checked its work, but not against what was asked (RL-SWE); dropped what an earlier turn set (RL-Constraints); or judged by eye what it could measure (RL-Charts). Every reward grades the final answer or state against the requirements, so rollouts that find, measure or test what a requirement depends on score higher; each adapter takes that step more often. The habit carries over to new tools: RL-Docs, trained without API tools, reads AutomationBench’s spreadsheet more often.

Conclusion

Five LoRA adapters, each trained with RL on one expert-built dataset, raised Nemotron 3.5 Super’s headline scores in their domains by +4.5 to +13.8 points across twelve benchmarks, eight built by other groups and four by Surge AI. The document adapter also gained +8.0 points on AutomationBench, a domain it never trained on. Side by side, each adapter does more often what base missed, from reading the rules before acting to measuring chart marks in code; on this evidence, the common change is how the model gathers and checks what its tasks need.

Training Details

Table Table 6 lists the training settings that differ between the five runs.

RL-SWERL-ConstraintsRL-ToolUseRL-ChartsRL-Docs
Deploymentearly accessdocumentedearly accessdocumenteddocumented
Peak learning rate2×10−42 \times10^{-4}1×10−41 \times10^{-4}1×10−41 \times10^{-4}1×10−41 \times10^{-4}1×10−41 \times10^{-4}
Schedule to the reported checkpointconstantconstantconstant to mid-run, then linear decayconstantconstant
Staleness cap (sampler versions)4 (soft)32 (soft)82 (admission held)
Token cap32,768 per turn32,768, then 49,152 per response32,768 per turn32,768 per turn32,768 per turn
Context258,048one response258,048258,048258,048
Turn cap250one turn2005020
Episode timeout5 hnone3 hhang guard onlysoft notice, then drop
Reasoning effort, training (evaluation)medium (full)full (full)full (full)full (full)full (full)
Rollout harnessOpenCode and mini-swe-agent portsone responseMCP tools in a simulated companyPython kernel and crop tooldocument tools
Shapingnoneoverlong penalty; a small length term late in the runlong-turn charge; cut-off floor; two over-persistence termsnonecut-off floor

Table 6. Training settings that differ between the runs. Common to all: a fresh rank-32 LoRA adapter; CISPO with the importance weight clipped to [0, 4]; the group-mean advantage without standard-deviation scaling; no KL or entropy term; AdamW with β1=0.9\beta_1 = 0.9, β2=0.95\beta_2 = 0.95, ϵ=1×10−8\epsilon= 1 \times10^{-8}, weight decay 0 and gradient-norm clipping at 1.0; a learning rate that warms up over the first few updates; training rollouts at temperature 1.0 and top-pp 1.0. Soft caps bound only group starts; “admission held” pauses new groups while one lags past it; others drop late groups.

Deployments. The early-access deployment and the one listed in Tinker’s documentation are

  • nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16:peft:262144:ea and

  • nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16:peft:262144.

They serve different weights: on a fixed 8,319-token sequence, their log-probabilities differ by 0.40 nats per token on average.

Rewards and shaping. The tool-use reward is the share of required criteria met, less a term for failed content prohibitions; a broken world-state prohibition or an unrequested side effect sets R=0.25 c−0.25R = 0.25\,c - 0.25, and an attempt to reach the environment’s backend or to inject instructions sets R=−0.25R = -0.25. The constraints run penalizes long failed responses: a response at or below its group’s mean reward loses up to 0.25 of reward once it passes 32,768 tokens, ramping over 16,384 tokens; late in the run, before the reported checkpoint, a small group-relative length term was added. The tool-use run charges reasoning tokens in a turn beyond 6,144, by up to 0.1 of advantage over the next 10,240 tokens, trains a turn cut off unclosed at min⁡(A,0)−0.1\min(A,0) - 0.1, and weights the outcome of a forced ending by 0.5. From 0.3 of the way through the run it adds two terms against over-persistence: the turn that ignores the context wrap-up notice trains at min⁡(A,0)−0.2\min(A,0) - 0.2, and tool turns that start beyond 200,000 tokens of context are charged 0.1. The document run trains a turn cut at its token cap at min⁡(A,0)−0.1\min(A,0) - 0.1.

Evaluation Details and Statistics

Settings. Every headline evaluation samples at temperature 1.0 and the model’s full reasoning effort, with its earlier reasoning kept in context in multi-turn harnesses; attempts are sampled independently. The coding agents get 250 model calls per attempt on SWE-bench Pro V2 and Terminal-Bench 2.1 and 1,000 on DeepSWE, with 65,536 tokens per turn; Terminal-Bench 2.1 allows 3 hours of tool time with a 10-hour backstop. AutomationBench allows 50 agent steps and HANDBOOK.md 1,000 turns and 120 minutes, both with 32,768 tokens per turn. The chart benchmarks allow 50 turns and 258,048 tokens of context, the instruction-following benchmarks 131,072 tokens per response, and GDP.pdf 40 turns.

Tasks scored. SWE-bench Pro V2 is scored on 635 of its 642 tasks, because the reference patches of the others fail V2’s own tests; DeepSWE on 112 of 113, because one reference solution fails validation; and AdvancedIF on 1,643 of 1,645 items, because the judge could not grade two. On SWE-bench Verified, 15 of its 500 tasks are left out for both models: 9 need the internet at test time, 2 pass with an empty patch, 3 have a reference patch that fails validation and 1 was lost to an infrastructure failure.

Graders. The three judged instruction-following benchmarks use Claude Opus 5.5 with each benchmark’s own judge prompt; the ComplexConstraints harness grades one criterion per call. The chart benchmarks are graded by GPT-6.1 Sol through their official scoring code, which for ChartMuseum and CharXiv-RQ replaces the official judge model; ChartMuseum’s reported scores are format-lenient (Table 3, note a). HANDBOOK.md is graded by an agent judge on GPT-6.1 Sol; regrading the same trials with the benchmark’s release judge, GPT-5.6 Sol, gives a gain of +13.1 [+9.9, +16.6]. GDP.pdf is graded at high effort with its own per-criterion prompt.

Statistics. Each interval is a paired task bootstrap: per task, we average each model’s attempts and take trained minus base, then resample tasks with replacement 10,000 times with a fixed seed (numpy’s default generator, seed 0) and read the 2.5th and 97.5th percentiles. AutomationBench intervals are resampled within its six departments, and the means over AdvancedIF subsets and MultiChallenge axes within subset or axis. Every number is rounded once, half away from zero.

Overlap with the training data. We compared each headline benchmark and SWE-bench Verified with the training text of the run whose checkpoint it scores: prompts, rubrics and, for the tool-use run, the seeded databases, documents and files of every training task. Text is lower-cased and split into words. Two rules flag an item: the per-instance rule of Tülu 3 [22], under which more than half of the item’s tokens are covered by word 8-grams shared with one training instance, and a corpus rule, under which more than half of the item’s word 8-grams occur anywhere in the training text. We also count the items that share a word 13-gram with the training text [2]. As positive controls, 28 training passages planted into benchmark items all produced 13-gram matches with their source, and all 22 long passages were flagged. Table 7 gives the results. The corpus rule flags no item, and the per-instance rule flags one ChartMuseum question, which shares a generic question frame with a training prompt about a different chart.

BenchmarkAny 13-gramMax share (%)Notes
Terminal-Bench 2.100.0The benchmark’s canary string appears in no training prompt
SWE-bench Pro V201.0No SWE training task shares a repository with the three SWE benchmarks
DeepSWE00.4
SWE-bench Verified06.6
IFBench00.0
AdvancedIF15.1Without the 13-gram item: +7.8 [+5.8, +9.7]
MultiChallenge02.1
ComplexConstraints228.4Without the items whose prompt or rubric shares a 13-gram: +7.6 [+1.6, +13.7]; without the 3 items adjudicated as near-duplicates of training prompts: +7.1 [+1.0, +13.2]
AutomationBench00.9
HANDBOOK.md27 tasks4.613-grams are stock strings, such as field names and timestamps; without those tasks: +11.0 [+7.4, +14.9]
Chartography03.7Charts matching a benchmark chart were removed before training
ChartMuseum028.6Format-lenient gain without the flagged question, +6.51 [+4.85, +8.16]; without the 2 questions whose charts overlap training content, +6.46 [+4.86, +8.12]
GDP.pdfnone5.8Prompts and rubrics; training tasks sharing a document were removed before training

Table 7. Overlap between each benchmark and the training text of the checkpoint it scores: items sharing any word 13-gram, and the largest share of an item’s word 8-grams found in the training text. Gains in the last column are in points, with 95% intervals.

Tokens. On the eleven headline benchmarks with per-attempt token records, the trained checkpoints sample 1.23 (ChartMuseum) to 2.07 (SWE-bench Pro V2, second run) times as many tokens per attempt as base, in geometric means over tasks.

Breakdowns

Tables 8 and 9 split eight of the headline benchmarks by their own labels. Intervals are paired task bootstraps within each subgroup; B/W/T counts the tasks on which the trained model’s mean over its attempts is better than, worse than or tied with base’s.

SubgroupBaseTrainedGain [95% CI]B/W/T
AutomationBench (RL-ToolUse), pass rate, by department
HR10.033.0+23.0 [+15.0, +32.0]24/1/75
Sales7.022.0+15.0 [+7.0, +23.0]17/2/81
Operations24.037.0+13.0 [+4.0, +22.0]18/5/77
Finance17.027.0+10.0 [+1.0, +19.0]16/6/78
Marketing11.021.0+10.0 [+2.0, +18.0]14/4/82
Support20.025.0+5.0 [−2.0-2.0, +13.0]10/5/85
HANDBOOK.md (RL-ToolUse), mean share of criteria met, by business domain
HR49.069.0+20.0 [+14.3, +26.0]13/0/0
Logistics43.859.3+15.5 [+8.1, +22.0]11/1/0
Finance63.474.2+10.8 [+4.2, +17.4]8/3/1
Insurance57.768.2+10.5 [+2.3, +20.6]9/2/2
Medical billing59.267.1+7.9 [+1.9, +14.6]10/5/0
SWE-bench Pro V2 (RL-SWE), resolve rate, by repository language
Go87.993.1+5.1 [+2.4, +8.1]32/10/211
Python87.992.6+4.7 [+1.9, +7.7]24/8/203
TypeScript83.289.5+6.4 [+1.4, +11.8]17/7/86
JavaScript85.190.5+5.4 [−5.4-5.4, +16.2]7/3/27
Chartography (RL-Charts), accuracy, by professional field
STEM39.547.5+8.0 [+3.9, +12.5]29/8/24
Manufacturing and supply chain28.951.6+22.6 [+13.7, +32.1]14/1/4
Healthcare42.749.1+6.4 [−5.5-5.5, +21.8]4/2/5
Finance and investing56.771.1+14.4 [+4.4, +25.6]5/0/4

Table 8. Breakdowns of the tool-use, coding and chart headline benchmarks.

SubgroupBaseTrainedGain [95% CI]B/W/T
MultiChallenge, pass rate per attempt, by axis
Inference memory60.770.0+9.3 [+4.9, +13.9]46/16/47
Instruction retention74.881.2+6.3 [+2.9, +10.3]23/5/41
Reliable version editing45.059.4+14.4 [+6.3, +22.8]19/6/15
Self-coherence52.655.7+3.1 [−2.3-2.3, +8.3]18/10/20
AdvancedIF, every rubric item met, by subset
Carried context53.160.2+7.1 [+4.8, +9.6]168/80/488
Complex instruction following59.266.8+7.6 [+3.6, +11.7]107/61/234
System steerability39.548.0+8.5 [+5.1, +11.9]140/65/300
ComplexConstraints, every criterion met, by instruction type
Implicit requirements13.431.3+17.9 [+4.5, +33.0]6/1/7
Multistep instructions27.542.5+15.0 [+2.5, +27.5]10/3/7
Negative instructions31.333.5+2.3 [−6.3-6.3, +11.4]7/9/6
Conditional instructions32.933.6+0.7 [−11.2-11.2, +10.5]7/2/10
IFBench, share of constraints met (strict), by constraint family
Word-level57.065.6+8.6 [+4.3, +12.9]42/11/32
Formatting81.886.1+4.2 [−0.2-0.2, +9.0]23/14/59
Counting85.789.9+4.2 [−0.4-0.4, +8.8]21/12/29
Sentence-level72.174.7+2.6 [−3.8-3.8, +9.3]5/4/19
Ratios67.468.2+0.9 [−5.2-5.2, +6.4]9/7/28
Repeating41.741.70.0 [−4.2-4.2, +4.2]1/1/7
One-off tasks78.477.3−1.1-1.1 [−5.7-5.7, +3.4]2/3/6

Table 9. Breakdowns of the instruction-following headline benchmarks (RL-Constraints).

References

  1. [1]Anomaly. OpenCode: The open source AI coding agent, 2026. URL https://github.com/anomalyco/opencode. Software, version 1.17.11 (June 25, 2026); MIT license. Website: https://opencode.ai/. Accessed November 9, 2026.
  2. [2]Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. URL https://arxiv.org/abs/2005.14165.
  3. [3]Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, et al. SkyRL-v0: Train real-world long-horizon agents via reinforcement learning, 2025. URL https://novasky-ai.notion.site/skyrL-v0. NovaSky blog post (UC Berkeley Sky Computing Lab and Anyscale), released May 6, 2025. Code: https://github.com/NovaSky-AI/SkyRL. Accessed November 9, 2026.
  4. [4]Zhoujun Cheng, Shibo Hao, Tianyang Liu, Fan Zhou, Yutao Xie, Feng Yao, Yuexin Bian, Yonghao Zhuang, Nilabjo Dey, Yuheng Zha, et al. Revisiting reinforcement learning for LLM reasoning from a cross-domain perspective. arXiv preprint arXiv:2506.14965, 2025. URL https://arxiv.org/abs/2506.14965.
  5. [5]Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, et al. Introducing SWE-bench Verified, 2024. URL https://openai.com/index/introducing-swe-bench-verified/. OpenAI, August 13, 2024; updated February 24, 2025. Accessed November 9, 2026.
  6. [6]Soham Dan, Jeff Da, Sanyam Bhutani, Miguel Romero Calvo, Ivan Guevara, Fernando Carabedo, Jose Bernal, Julio Reyna, Luis Leandro Monsalve Mendez, Ying Liu, and Daniel Yue Zhang. SWE-Bench Pro V2: A cleaner, harder-to-game coding benchmark. arXiv preprint arXiv:2610.24206, 2026. URL https://arxiv.org/abs/2610.24206.
  7. [7]DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. URL https://arxiv.org/abs/2501.12948. Published in Nature 645, 633–638 (2025), doi: 10.1038/s41586-025-09422-z, as “DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning”.
  8. [8]Kaustubh Deshpande, Ved Sirdeshmukh, Johannes Baptist Mols, Lifeng Jin, Ed-Yeremai Hernandez-Cardona, Dean Lee, Jeremy Kritz, Willow E. Primack, Summer Yue, and Chen Xing. MultiChallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 18632–18702, Vienna, Austria, 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.findings-acl.958. URL https://aclanthology.org/2025.findings-acl.958/. arXiv:2501.17399.DOI
  9. [9]Bradley Efron. Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1): 1–26, 1979. doi: 10.1214/aos/1176344552.DOI
  10. [10]Ling Fu, Zhebin Kuang, Jiajun Song, Mingxin Huang, Biao Yang, Yuzhe Li, Linghao Zhu, Qidi Luo, Xinyu Wang, Hao Lu, et al. OCRBench v2: An improved benchmark for evaluating large multimodal models on visual text localization and reasoning. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track, pages 107930–107981, 2025. doi: 10.52202/085713-3253. URL https://arxiv.org/abs/2501.00321. arXiv:2501.00321.
  11. [11]Suhaas Garre, Chris Mutty, Sushant Mehta, and Edwin Chen. Charography: A benchmark for professional chart understanding. arXiv preprint arXiv:2608.10677, 2026a. URL https://arxiv.org/abs/2608.10677. Accepted at the 2nd Workshop on Benchmarking Evidence-Aligned Multimodal Reasoning (BEAM 2), ECCV 2026.
  12. [13]Alexander Golubev, Maria Trofimova, Sergei Poleshaev, Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Simon Karasik, Sergey Abramov, Andrei Andriushchenko, Filipp Fisin, Sergei Skvortsov, and Boris Yangel. Training long-context, multi-turn software engineering agents with reinforcement learning. arXiv preprint arXiv:2508.03501, 2025. URL https://arxiv.org/abs/2508.03501.
  13. [14]Anisha Gunjal, Anthony Wang, Elaine Lau, Yaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746, 2025. URL https://arxiv.org/abs/2507.17746.
  14. [15]Harbor Framework Team. Harbor: A framework for evaluating and optimizing agents and models in container environments, 2026. URL https://doi.org/10.5281/zenodo.20953922. Software, concept DOI. Includes Terminus 2, Harbor’s reference agent (https://www.harborframework.com/docs/agents/terminus-2). Version 0.22.0, August 22, 2026. Accessed November 9, 2026.
  15. [16]Yun He, Wenzhe Li, Hejia Zhang, Songlin Li, Karishma Mandyam, Sopan Khosla, Yuanhao Xiong, Nanshu Wang, Xiaoliang Peng, Beibin Li, et al. AdvancedIF: Rubric-based benchmarking and reinforcement learning for advancing LLM instruction following. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 18003–18022, San Diego, California, United States, 2026. Association for Computational Linguistics. doi: 10.18653/v1/2026.acl-long.820. URL https://aclanthology.org/2026.acl-long.820/. arXiv:2511.10507.DOI
  16. [17]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022. URL https://arxiv.org/abs/2106.09685. arXiv:2106.09685.
  17. [18]Mingxin Huang, Yongxin Shi, Dezhi Peng, Songxuan Lai, Zecheng Xie, and Lianwen Jin. OCR-Reasoning benchmark: Unveiling the true capabilities of MLLMs in complex text-rich image reasoning. In International Conference on Learning Representations (ICLR), 2026a. URL https://arxiv.org/abs/2505.17163. arXiv:2505.17163.
  18. [19]Wenqi Huang, Charley Lee, Leonard Tng, and Serena Ge. DeepSWE: Measuring frontier coding agents on original, long-horizon engineering tasks. arXiv preprint arXiv:2607.07946, 2026b. URL https://arxiv.org/abs/2607.07946. Datacurve. Leaderboard (v1.1, 113 tasks): https://deepswe.datacurve.ai/; code and data: https://github.com/datacurve-ai/deep-swe (commit 0b9fabbb). Accessed November 9, 2026.
  19. [20]Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2310.06770. arXiv:2310.06770.
  20. [21]Devvrit Khatri, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, David Brandfonbrener, and Rishabh Agarwal. The art of scaling reinforcement learning compute for LLMs. arXiv preprint arXiv:2510.13786, 2025. URL https://arxiv.org/abs/2510.13786.
  21. [22]Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tülu 3: Pushing frontiers on open language model post-training. In Conference on Language Modeling (COLM), 2025. URL https://arxiv.org/abs/2411.15124. arXiv:2411.15124.
  22. [24]Kilian Lieret and Carlos E. Jimenez. mini-SWE-agent: The minimal AI software engineering agent, 2026. URL https://github.com/SWE-agent/mini-swe-agent. Software, version 2.4.6 (July 23, 2026); MIT license. Accessed November 9, 2026.
  23. [25]Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding R1-Zero-like training: A critical perspective. In Conference on Language Modeling (COLM), 2025. URL https://arxiv.org/abs/2503.20783. arXiv:2503.20783. Introduces Dr. GRPO.
  24. [26]Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019. URL https://arxiv.org/abs/1711.05101. arXiv:1711.05101.
  25. [27]Michael Luo, Naman Jain, Jaskirat Singh, Sijun Tan, Ameen Patel, Qingyang Wu, Alpay Ariyak, Colin Cai, Tarun Venkat, Shang Zhu, et al. DeepSWE: Training a fully open-sourced, state-of-the-art coding agent by scaling RL, 2025. URL https://www.together.ai/blog/deepswe. Agentica and Together AI blog post, July 2, 2025. Accessed November 9, 2026.
  26. [28]Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmohammadi, et al. ChartQAPro: A more diverse and challenging benchmark for chart question answering. In Findings of the Association for Computational Linguistics: ACL 2025, pages 19123–19151, Vienna, Austria, 2025. Association for Computational Linguistics. doi: 10.18653/v1/2025.findings-acl.978. URL https://aclanthology.org/2025.findings-acl.978/. arXiv:2504.05506.DOI
  27. [29]Sushant Mehta, Liudas Panavas, Suhaas Garre, and Edwin Chen. ComplexConstraints and beyond: Expert rubrics for RLVR. arXiv preprint arXiv:2606.09118, 2026a. URL https://arxiv.org/abs/2606.09118. Accepted to the GEM workshop at ACL 2026.
  28. [30]Sushant Mehta, Logan Ritchie, and Edwin Chen. Cross-benchmark transfer from RL on agentic coding tasks. arXiv preprint arXiv:2610.00890, 2026b. URL https://arxiv.org/abs/2610.00890.
  29. [31]Sushant Mehta, Logan Ritchie, Liudas Panavas, and Edwin Chen. Cross-benchmark generalization in long-horizon agents. arXiv preprint arXiv:2608.00181, 2026c. URL https://arxiv.org/abs/2608.00181. Accepted at the COLM 2026 Workshop on Agent Behavior.
  30. [32]Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, et al. Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces. In International Conference on Learning Representations (ICLR), 2026. URL https://arxiv.org/abs/2601.11868. arXiv:2601.11868. Introduces Terminal-Bench 2.0.
  31. [33]Evan Miller. Adding error bars to evals: A statistical approach to language model evaluations. arXiv preprint arXiv:2411.00640, 2024. URL https://arxiv.org/abs/2411.00640.
  32. [34]MiniMax, Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changqing Yu, Chao Wang, et al. MiniMax-M1: Scaling test-time compute efficiently with lightning attention. arXiv preprint arXiv:2506.13585, 2025. URL https://arxiv.org/abs/2506.13585. Introduces the CISPO objective.
  33. [35]NVIDIA. NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16, 2026. URL https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16. Hugging Face model card. Released October 15, 2026; OpenMDW-1.1 license; card revision fb0ee0b, last modified October 22, 2026. Accessed November 9, 2026.
  34. [37]Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with SWE-Gym. arXiv preprint arXiv:2412.21139, 2024. URL https://arxiv.org/abs/2412.21139.
  35. [38]Liudas Panavas, Sebastian Minus, Bradley Monton, Derek Ray, Suhaas Garre, Sushant Mehta, and Edwin Chen. HANDBOOK.md: A benchmark for long-context agentic instruction following. arXiv preprint arXiv:2607.25398, 2026. URL https://arxiv.org/abs/2607.25398. Accepted to the Workshop on Agent Behavior (WAB) at COLM 2026.
  36. [39]Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, and Hanneke Huisingh. Generalizing verifiable instruction following. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track, pages 54870–54896, 2025. doi: 10.52202/085713-1645. URL https://arxiv.org/abs/2507.02833. arXiv:2507.02833. Introduces IFBench.
  37. [40]Logan Ritchie, Sushant Mehta, Liudas Panavas, and Edwin Chen. Post-training on office work improves software engineering: A behavioral account of cross-domain transfer. arXiv preprint arXiv:2608.01604, 2026. URL https://arxiv.org/abs/2608.01604.
  38. [41]John Schulman and Thinking Machines Lab. LoRA without regret. Thinking Machines Lab: Connectionism, 2025. doi: 10.64434/tml.20250929. URL https://thinkingmachines.ai/blog/lora/. September 29, 2025.DOI
  39. [42]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. URL https://arxiv.org/abs/2402.03300. Introduces Group Relative Policy Optimization (GRPO).
  40. [44]Liyan Tang, Grace Kim, Xinyu Zhao, Thom Lake, Wenxuan Ding, Fangcong Yin, Prasann Singhal, Manya Wadhwa, Zeyu Leo Liu, Zayne Sprague, et al. ChartMuseum: Testing visual reasoning capabilities of large vision language models. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), Datasets and Benchmarks Track, pages 153757–153805, 2025. doi: 10.52202/085713-4628. URL https://arxiv.org/abs/2505.13444. arXiv:2505.13444.
  41. [45]Terminal-Bench Team. Terminal-Bench 2.1, 2026. URL https://www.tbench.ai/news/terminal-bench-2-1. Release post, May 6, 2026: a revision of Terminal-Bench 2.0 that fixes 28 of its 89 tasks. Dataset: https://github.com/harbor-framework/terminal-bench-2-1 (commit 7131e43). Accessed November 9, 2026.
  42. [46]Thinking Machines Lab. Tinker, 2026a. URL https://thinkingmachines.ai/tinker/.

Paper details

vlnc:W_tvct84es
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents