WorksComputer Science
ViStep-RL: Stepwise Reinforcement Learning for Long-Horizon Visual Agents
Abstract
Long-horizon visual agents must do more than inspect images: they must turn observations into evidence that remains usable as a task unfolds. Existing active-perception methods teach agents where and how to look, while a final-answer reward gives little structure for deciding which observations satisfy dependent subgoals or should be retained for later use. We present ViStep-RL (Visual Stepwise Reinforcement Learning), which models execution as a dependency-linked graph of evidence-bearing subgoals. A verifier tracks which subgoals are supported by valid evidence, and changes in graph potential provide a net-progress learning signal; an active memory protocol preserves evidence across turns and links retrieval to downstream use. On ViInteract, ViStep-RL reaches 60.0% strict completion with Qwen2.5-VL-7B, compared with 10.0% for the base model and 46.5% after supervised fine-tuning. Controlled studies show that incremental task-state rewards outperform cumulative rewards by 3.2 points, while active evidence writing improves strict completion by 7.75 points over automatic writing. Under a common interaction framework, the task-trained ViStep-RL policy achieves 59.5% answer accuracy, 24.0 points above the strongest of six general-purpose reference models. Held-out episodes confirm the gain at both Qwen scales, and gains extend to three external visual reasoning benchmarks.
Introduction
Long-horizon visual tasks require agents to integrate information across a sequence of observations, tool calls, and decisions. An agent may inspect a frame, identify an object, compare it with an earlier image, and rely on that comparison several steps later. Beyond deciding where to look, the agent must convert observations into evidence that remains available to support dependent task steps. When a useful observation is forgotten, detached from its source, or used before its prerequisites are established, subsequent reasoning may fail to complete the task.
Recent reinforcement-learning methods make visual perception more active. ViGoRL grounds reasoning in image regions and learns when to zoom; MGP-O extends grounding to multi-turn high-resolution inspection; DeepEyes encourages image operations during reasoning; and ACTIVE-o3 trains active-perception policies [12, 29, 51, 54]. These methods establish that an agent can learn how to acquire visual information. Long-horizon execution additionally requires tracking when acquired information qualifies as task evidence, which subsequent decisions depend on it, and whether it remains accessible as the visual context changes. Treating observations as isolated perceptual outputs leaves this evolving evidence state implicit.
We make the evidence state explicit by representing each task as a dependency graph. Each node encodes a verifiable subgoal and its evidence requirements. A node is verified only when both its evidence requirements and prerequisites are satisfied. This representation allows the verifier to track established subgoals, unresolved dependencies, and evidence that must remain available for subsequent execution. It also distinguishes genuine task progress from repeated interaction that leaves the underlying state unchanged.
ViStep-RL (Visual Stepwise Reinforcement Learning) uses this evolving task state to guide agent training. After each transition, a verifier evaluates the evidence and checks whether subgoal dependencies are satisfied. Changes in normalized graph potential provide a signed progress signal for increases and decreases in task progress. An active memory protocol records provenance-linked observations, retrieves them after changes in image context, and connects retrieval to the dependent subgoal that uses the evidence. The task graph and expected evidence remain outside the policy context: during training, the verifier supplies the transition rewards used to compare actions at similar stages of the task. This design links visual acquisition, evidence persistence, and downstream task completion in one execution objective.
We evaluate ViStep-RL on ViInteract, a benchmark comprising complete visual-agent episodes spanning temporal interaction, multi-image reasoning, visual-tool use, OCR, pose, and mask/measurement tasks. Its primary evaluation set, Core200, contains 200 episodes, with the domain composition reported in the supplement. Beyond region localization and observation requests, ViInteract evaluates whether an agent completes the task through a sequence of visual actions, tool calls, and state updates. On Core200, ViStep-RL improves Strict completion over SFT by 13.5 and 5.5 percentage points for Qwen2.5-VL-7B and Qwen2.5-VL-3B, respectively. Under matched training and interaction budgets, ViStep-RL also improves Strict over standard terminal-answer GRPO by 11.5 points at 7B and 5.0 points at 3B. On the separate Process500 panel, incremental graph rewards improve Strict completion by 3.2 percentage points over cumulative graph rewards. In the writing–retrieval factorial, active evidence writing improves Strict completion by 7.75 percentage points over automatic writing, averaged across retrieval conditions. With retrieval held fixed, active writing also improves final-answer accuracy by 8.0 points with learned retrieval and 7.0 points with BM25. Together, these results support the complementary roles of progress-based training and evidence persistence in completing visual tasks. Evaluations of task-trained variants on MMLongBench-Doc, MuirBench, and MMSI-Bench further examine document, multi-image, and spatial reasoning. Our main contributions are summarized as follows:
We propose ViStep-RL, a verifier-guided framework for long-horizon visual execution. Its dependency-aware graph reward uses changes in task-state potential to credit progress without repeatedly rewarding unchanged states.
A task-coupled memory mechanism integrates evidence writing and retrieval with downstream verification, preserving provenance-linked observations across turns.
We introduce ViInteract, a benchmark for end-to-end visual-agent execution, and evaluate our approach across diverse tasks and two model scales.
Related Work
Visual reasoning, grounding, and benchmarks. Vision-language research has evolved from visual question answering to grounding, high-resolution perception, and multi-image reasoning. Early benchmarks evaluate visual and textual evidence [9, 13, 23, 24, 34], while recent suites extend to compositional perception and multimodal reasoning [7, 8, 14, 20, 35, 46, 48, 50]. Advances in LLaVA, Qwen2-VL, and InternVL strengthen these capabilities [5, 18, 37]. Recent interactive approaches, including ViGoRL, MGPO, DeepEyes, and ACTIVE-o3, enable active visual grounding and perception through image operations [12, 29, 51, 54]. Unlike these perception-centered settings, ViInteract evaluates end-to-end task completion across heterogeneous tools, dependencies, and persistent state.
Interactive agents and reinforcement learning. Tool-using agents integrate language reasoning with external actions, from ReAct and Toolformer to multimodal interactive systems [30, 40, 44, 45]. Benchmarks such as AgentBench, WebArena, OSWorld, and Android-in-the-Wild evaluate long-horizon interaction and dynamically evolving environment states [19, 28, 41, 53]. Reinforcement learning optimizes such trajectories through policy gradients and stabilized updates, including PPO and group-relative optimization methods [1, 31, 32, 39, 47]. Terminal correctness alone provides sparse and delayed feedback for complex multi-step tasks. Process supervision evaluates intermediate reasoning steps [17], while AgentPRM learns process rewards from Monte Carlo rollout targets [6]. Verifiable Process Rewards use task oracles to score intermediate actions [49]; Percep-tRL rewards spatial fidelity during visual inspection [10]. ViStep-RL tracks what happens to an observation after inspection: its source must remain identifiable and its evidence available when a later subgoal calls for it.
Memory and long-horizon evidence management. Long-horizon agents preserve evidence through retrieval and persistent memory. RAG and RETRO introduce external retrieval [4, 16], while MemGPT, MemoryBank, Reflexion, Voyager, and Generative Agents explore memory and experience reuse [26, 27, 33, 36, 52]. Self-RAG and HippoRAG further investigate selective retrieval and structured memory [2, 11]. Agent Workflow Memory induces reusable action routines [38], and A-Mem dynamically links and updates stored memory notes [42]. ViStep-RL focuses on whether visual evidence acquired within an episode remains supported and satisfies downstream dependencies. It incorporates memory writing and retrieval into an executable task graph, using factorial comparisons and memory-destruction interventions to assess their contribution to task progress.
Method
Visual Execution and Evidence Verification
A visual observation can be several actions old by the time the agent needs it. In a TextOCR episode, for example, the agent reads text on successive images and later identifies the image containing the requested word. Once the image leaves view, the word and its source must still be available together. Figure 1 shows how ViStep-RL connects these decisions.
Figure 1. ViStep-RL: from visual observations to verified task progress. (a) Source-grounded examples illustrate object counting, pose measurement, and chart reading. (b) Tool observations are stored with provenance and retrieved after visual context changes. (c) Schematic changes in strict verification: new evidence establishes progress, unchanged verification earns no additional graph credit, and invalidation retracts dependent memory verification. Auxiliary retrieval coverage is defined separately in the training potential.
The agent chooses what to inspect, store, and retrieve. After each action, a task program verifies the evidence against explicit task requirements. The policy learns by comparing actions taken at similar stages of the same task. Each ingredient is established: verifiable task graphs, potential-based shaping [25] and group-relative updates. ViStep-RL ties them to one object, the provenance-tracked evidence record, which the verifier checks, the potential rewards and the action weight credits.
Formally, an episode is a finite-horizon interaction , where is the environment state after transition . The policy samples from its public history, which includes the instruction, current visual input, legal actions, and returned observations, including failures. The verifier has access to the task graph and expected evidence.
Execution runtime and Harness. The runtime executes the policy’s actions within a fixed budget. It dispatches visual tools, advances the current image, handles memory operations, and records the final answer. Native evaluations use the trained agent’s action protocol. For comparisons across models, the shared Harness provides a common public interface for image progression, tool calls, returned observations, and answer submission.
Active evidence memory. In native Qwen execution, advancing to a new visual turn starts a fresh prompt. The previous image, its dialogue, and the text returned by OCR leave the prompt together. Explicit memory records persist in the environment, and a search or retrieval can return their contents to the policy. The native policy can preserve a tool observation by writing its payload, source image or semantic identity, and references to the tool events that produced it. A later search can then recover both the observation and its origin. In the OCR example, this association lets the agent trace a retrieved word back to the image it must identify.
We represent the required execution as a directed acyclic graph . Each node specifies a subgoal with a local evidence predicate; edges encode prerequisites. Nodes cover visual evidence, computation, memory operations, and the final answer. Let indicate whether node ’s local predicate holds in the current state and recorded execution history, including its provenance conditions. Let contain all ancestors of , excluding itself. Strict node verification is
For a root node, . A non-root subgoal receives credit only when its evidence is valid and all prerequisites are satisfied. Later writes cannot alter an empty search result, so assigning credit requires a new search. A search that returned the required ID can become verified once its prerequisites hold. The trace records what happened; the edges determine when it counts. With positive node weights , set to one by default, verified progress is
Each task program derives its graph, evidence predicates and answer from source annotations and deterministic rules. An LLM assists with instruction wording. Replaying a reference trajectory checks executability. For an OCR write, the verifier checks the text, the source image and the successful tool call cited by the record.
Strict completion requires every node, including the final answer, . The program compiler enforces two structural rules, detailed in the supplement. The answer node is the unique sink: every evidence, computation, memory and retrieval node reaches it, so the answer predicate cites records whose producing events trace back through the graph. Every edge is enforced by provenance. For , the predicate cites an event produced by node , so the order required for correctness is part of the predicate the product checks. An action therefore receives a node’s credit when it supplies the missing cited evidence or completes the last missing prerequisite. The edges decide when that credit enters ; § 4.3 measures the effect.
Net-Progress Reward
An agent may retain the same observation across several actions. We reward the transition that changes its contribution to the task [25]. Let indicate a protocol violation, failed state or tool check, or an incorrect or unsupported answer on action . The charge for an erroneous action equals the mean node share, , so a mistake costs as much as one average subgoal on that graph. The transition reward is
A valid answer earns credit through its own node. Summing over an episode gives
Holding a record unchanged earns no further progress. A cumulative reward pays for the same record at every later step: a node verified at step and kept verified contributes , whereas (4) pays once. With the action budget fixed, reaching a subgoal earlier leaves more actions for the nodes that remain.
Proposition 1 (Shaping consistency). (i) if and only if , and, because the answer node is the unique sink, if and only if . (ii) Losing a node’s verification and later regaining it earns zero net progress over that cycle, while every error inside it is still charged. (iii) , where sums the retrieval-node weights, and any trace with ends at most on the progress term.
The supplement proves these claims. Full training credit coincides with Strict completion, a re-verification cycle adds no progress, and auxiliary retrieval credit remains strictly below a fully verified trace on the progress term.
Retrieval coverage and verified evidence. A search can recover a record before the verifier has established its provenance. We use the returned IDs to give the search immediate feedback while the verifier tracks whether the recovered evidence supports a dependent subgoal. For a retrieval node , let be its required record IDs and the set returned through step , after resolving aliases. Its coverage is . Set for other nodes or an empty required set. We use the same node weights as in Eq. (2) to define the training potential
For binary , : an unverified retrieval node earns credit for each required ID it recovers, and node never contributes more than . Hence , with the gap confined to unverified retrieval nodes, and on every non-retrieval node. Coverage enters training only; all reported metrics use . § 4.3 varies the weight on this auxiliary term.
Graph-Guided Policy Optimization
For a given prompt, we retain groups whose rollout returns vary. We compare actions from the same prompt, execution phase and visual turn, binning pre-action progress to the nearest tenth. Actions in a bin can have different evidence histories; the bin supplies a stage-specific reward scale. Let be the centered, standardized reward in this bin, with at zero variance. Centering alone can give a failed action a positive score when its peers fare worse. We use the transition reward to set the update direction and the within-bin comparison to adjust its magnitude. Writing with , the signed action weight is
where and . The clipped term gives feedback even in a singleton bin. It saturates at , below the unit scale of . The relative term amplifies positive rewards above the bin mean and negative rewards below it. Since , multiplying Eq. (6) by gives . A failed action therefore receives no positive weight, even when every peer does worse.
Each update keeps the actions with the largest , up to two per rollout at 3B and four at 7B. All tokens of a selected action share that action’s weight in a clipped policy loss regularized toward the SFT reference. § 4.2 compares this action-level update with a trajectory-level update. The supplement gives thresholds and the token loss.
Experiments
Benchmark and Evaluation Protocol
ViInteract spans ten execution domains and 34 task-program types. It comprises 76,000 SFT episodes, a 15,000-episode RL source pool, and 4,000 benchmark candidates. The benchmark pool covers nine of the ten domains and 29 program types. Its primary evaluation set, Core200, contains 200 frozen episodes with an application-weighted composition. Process500 (500 episodes) measures execution behavior, and Holdout200 (200 episodes, selected without model predictions) tests fresh instances. Table 1 compares the evaluation contract with established visual and interactive suites; the supplement gives the taxonomy and panel composition.
| Dataset / suite | Tool interface | Benchmark instances | Suite task families | Visual input | Agent interaction | Progress scoring | Evidence-path verification |
| Mantis-Eval [14] | None | 217 | 4 | ✓ | – | – | – |
| MuirBench [35] | None | 2,600 | 12 | ✓ | – | – | – |
| VisualWebArena [15] | Browser tools | 910 | 3 | ✓ | ✓ | – | – |
| OSWorld [41] | GUI tools | 369 | 5 | ✓ | ✓ | – | – |
| AgentBoard [21] | Environment tools | 1,013 | 9 | – | ✓ | ✓ | – |
| ViInteract (ours) | Visual tools | 4,000 | 10 | ✓ | ✓ | ✓ | ✓ |
Table 1. Table 1. Scale, task coverage, and evaluation capabilities. Instances count benchmark candidates. Task families count top-level categories across the full suite, including all ten ViInteract execution domains. ViInteract combines visual interaction, intermediate progress scoring, and provenance- and prerequisite-checked evidence paths to evaluate complete task execution across multiple steps.
Each task pairs visual inputs with reference evidence, prerequisite relations and an answer; the agent sees only the current image and public observations. Extending AgentBoard [21], ViInteract tests whether an observation stays usable after its image leaves view.
Metrics. Answer scores the final response; Strict requires every evidence predicate. Graph measures the required-node verification rate, Tool the tool-success rate, Protocol the schema-valid action rate, and State the accepted transition rate. In Table 2, direct API runs use a separate matched panel and Harness runs use Core200. For native Qwen, Answer and Strict use Core200; Graph, Tool, Protocol and State use Process500. Paired intervals resample shared episode IDs.
| Model / Backbone | Answer | Strict | Graph | Tool | Protocol | State |
| (a) Direct API evaluation | ||||||
| GPT-4o | 12.0 | 0.5 | 25.9 | 49.2 | 67.4 | 67.4 |
| Claude Opus 4.6 | 18.0 | 11.5 | 37.6 | 54.6 | 95.9 | 95.6 |
| Qwen3-VL-Plus | 3.0 | 2.5 | 16.0 | 19.3 | 49.3 | 49.3 |
| Qwen3.8-Flash | 9.5 | 8.5 | 33.2 | 41.6 | 54.1 | 54.0 |
| Gemini 3.8 Flash | 20.5 | 2.5 | 29.0 | 39.8 | 92.2 | 92.2 |
| Kimi K2.5 | 13.5 | 7.0 | 37.4 | 55.3 | 96.3 | 96.3 |
| (b) API models with Harness | ||||||
| GPT-4o | 35.0 | 30.5 | 41.3 | 69.6 | 95.3 | 99.9 |
| Claude Opus 4.6 | 30.5 | 26.5 | 35.2 | 57.8 | 86.1 | 100.0 |
| Qwen3-VL-Plus | 31.5 | 18.5 | 29.7 | 64.1 | 85.9 | 99.9 |
| Qwen3.8-Flash | 35.5 | 19.5 | 33.5 | 66.6 | 98.0 | 99.7 |
| Gemini 3.8 Flash | 34.5 | 2.5 | 21.5 | 58.3 | 99.3 | 99.9 |
| Kimi K2.5 | 22.0 | 14.5 | 31.7 | 42.2 | 66.3 | 100.0 |
| (c) Qwen2.5-VL training-stage comparison | ||||||
| Qwen2.5-VL-3B Base | 0.0 | 0.0 | 11.5 | 40.6 | 68.9 | 88.8 |
| Qwen2.5-VL-3B SFT | 19.0 | 15.0 | 29.5 | 55.5 | 93.1 | 73.3 |
| Qwen2.5-VL-3B Standard GRPO | 20.0 | 15.5 | 32.7 | 58.3 | 92.4 | 75.0 |
| Qwen2.5-VL-3B ViStep-RL | 24.0 | 20.5 | 51.8 | 75.6 | 97.6 | 82.4 |
| Qwen2.5-VL-7B Base | 12.0 | 10.0 | 9.3 | 16.4 | 61.5 | 94.8 |
| Qwen2.5-VL-7B SFT | 55.0 | 46.5 | 42.0 | 57.9 | 62.4 | 61.3 |
| Qwen2.5-VL-7B Standard GRPO | 58.0 | 48.5 | 45.8 | 64.2 | 69.0 | 64.7 |
| Qwen2.5-VL-7B ViStep-RL | 67.5 | 60.0 | 61.6 | 86.7 | 91.2 | 91.0 |
Table 2. Main results on ViInteract. API models are evaluated directly in (a) and with the shared Harness in (b). Qwen training stages use native agent execution in (c), where Answer and Strict use Core200 and the remaining four metrics use Process500. Scores are percentages (↑). Bold marks the best value within each API group and each Qwen backbone.
Training and evaluation controls. We evaluate Qwen2.5-VL [3] at 3B and 7B. We adapt attention and MLP projections with LoRA and evaluate each policy under fixed decoding settings. Qwen2.5-VL-7B training consists of 1,000 updates over 32,000 rollouts and takes 36.94 GPU-hours on four GPUs. The release audit found no shared episode IDs, complete image paths, or source-sequence IDs between training and the three evaluation panels. The supplement gives implementation details.
Post-Training and General-Purpose Models
Task-graph training improves completion at both Qwen scales. On Core200, Qwen2.5-VL-7B completes 120 of 200 episodes under the Strict criterion after ViStep-RL, up from 93 after SFT. The gain is 13.5 percentage points (paired 95% bootstrap interval [8.0, 19.0]). The 3B model also improves, from 15.0% to 20.5% Strict (Table 2(c)).
Standard GRPO provides a matched reinforcement-learning control. Both methods start from the same SFT policy and use the same tools, action budget, rollouts and optimizer updates. Standard GRPO uses final-answer correctness as its reward and shares one group-relative advantage across each trajectory; it updates every action, ViStep-RL at most four per 7B rollout. ViStep-RL improves Strict from 15.5% to 20.5% at 3B and from 48.5% to 60.0% at 7B. The corresponding Answer gains are 4.0 and 9.5 points. On Process500, Graph improves by 19.1 and 15.8 points, and Tool by 17.3 and 22.5 points. Crossing the two design choices at 7B helps isolate their respective contributions. With action-level updates fixed, graph reward raises Strict from 53.5% to 60.0%. With graph reward fixed, action-level updates raise it from 54.0% to 60.0%. § 4.3 separates the components of the graph reward.
The 7B Core200 traces show a change in which actions succeed: from SFT to ViStep-RL, required writes completed rise from 46.24% to 54.67% and retrievals from 60.51% to 69.99%, while mean actions fall from 15.76 to 15.00.
General-purpose models under the shared Harness. With shared public prompts, tools and scorer on Core200, six API models reach 22.0–35.5% Answer; Qwen2.5-VL-7B scores 16.5% at Base and 59.5% after task training. The API models achieve 99.7–100% state-transition acceptance, yet only 2.5–30.5% of their episodes satisfy Strict. Strict thus separates valid execution from evidence-supported answers.
Reward Design: Incremental Task Progress
All policies in Table 3 share the same Qwen2.5-VL-3B initialization, training budget and action-level update. The first row tests Eq. (4): the cumulative variant instead rewards the current potential at every step. Rewarding its change gives 25.8% Strict on Process500 versus 22.6% for cumulative reward (paired 95% interval for the gain [1.0, 5.6] points, McNemar ). On Holdout200, the same comparison similarly gives 15.5% versus 12.0%.
| Training variant | Progress | Other nodes | Answer node | Process500 Strict ↑ | Process500 Graph ↑ | Holdout200 Strict ↑ | Holdout200 Graph ↑ |
| Cumulative reward | Cumulative | ✓ | ✓ | 22.6 | 49.4 | 12.0 | 33.6 |
| No progress reward | None | ✓ | ✓ | 23.8 | 54.1 | 12.5 | 33.6 |
| No prerequisite gating | Incremental | × | × | 25.2 | 51.7 | 14.5 | 35.0 |
| No answer prerequisite gate | Incremental | ✓ | × | 23.4 | 51.4 | 14.5 | 35.3 |
| ViStep-RL | Incremental | ✓ | ✓ | 25.8 | 51.8 | 15.5 | 36.3 |
Table 3. Ablation of progress rewards and verification constraints. Progress specifies the training reward. The two gate columns indicate where prerequisites apply during training. All variants share matched training budgets and are scored with the same full verifier on identical episodes within each panel. Strict and Graph are percentages; higher is better.
The remaining rows isolate the progress measure. Removing the progress term leaves the execution cost in Eq. (3) and drops Strict from 25.8% to 23.8% on Process500. Every change to the graph lowers Strict on Process500: removing all prerequisite gates gives 25.2%, a separately trained random-DAG control 24.6%, and removing only the answer node’s gate 23.4%. Both gate removals reach 14.5% Strict on Holdout200 against 15.5% for ViStep-RL, and the full graph gives the highest Holdout200 Graph score, 36.3%. The largest single effect is the switch from cumulative incremental reward, which Proposition 1(ii) predicts: the cumulative form pays again for every node it re-verifies.
The coverage sweep tests the effect of the auxiliary term in Eq. (5). Giving unverified retrieval nodes zero coverage credit yields 122/500 Strict successes on Process500 and 26/200 on Holdout200. With full coverage credit, the counts rise to 129 and 31. Proposition 1(iii) bounds this auxiliary credit below a fully verified trace.
Memory: Evidence Selection and Persistence
For the memory analysis, Table 4(a) fixes the same 3B weights and Core200 IDs as the native comparison, with all four conditions sharing a bounded store: eight records, 512 serialized characters per record, and four returns per retrieval. Choosing what to write improves Answer by 8.0 points under learned retrieval and by 7.0 under BM25.
| (a) Writing retrieval, /cell | ||||
| Write / retrieval | Answer | Strict | Graph | Tool |
| Active / Active | 24.5 | 17.0 | 49.0 | 69.6 |
| Active / BM25 | 22.0 | 14.5 | 48.5 | 70.8 |
| Automatic / Active | 16.5 | 8.5 | 43.2 | 60.4 |
| Automatic / BM25 | 15.0 | 7.5 | 43.6 | 62.0 |
| (b) Evidence interventions, Holdout200 | ||||
| Intervention | Answer | Strict | Graph | Tool |
| ViStep-RL | 21.0 | 15.5 | 36.33 | 66.33 |
| Disable memory writes | 0.0 | 0.0 | 21.45 | 34.32 |
| Corrupt stored memory | 3.5 | 0.0 | 21.65 | 60.47 |
| Remove required tools | 6.5 | 4.5 | 17.84 | 33.09 |
Table 4. Effects of memory and evidence interventions. (a) Fixed 3B policy and memory budget on Core200. (b) Holdout200; Answer scores final responses independently of evidence checks.
eraged over retrievers, active writing gains 7.75 points in Strict; averaged over writing modes, learned retrieval gains 1.75 points over BM25.
On Holdout200, the intact policy answers 42 correctly, compared with zero when writes are disabled and seven when stored records are corrupted (Table 4(b)). Strict rejects those seven because their evidence chains fail. Under corruption, 60.47% of tool calls still succeed: a successful call can return a record that no longer supports the answer.
Human evaluation. Three annotators rated the 7B traces of all 200 Core200 episodes for SFT, Standard GRPO and ViStep-RL, with model names hidden and order randomized. Before seeing the final answer, each judged whether a record matched its source image, whether it named that image and the producing tool event, and whether the retrieved records sufficed for a later decision. A trace counts when at least two annotators accept it. In Figure 4, ViStep-RL reaches 68.0% provenance validity and 55.0% downstream support, against 63.5% and 45.0% for SFT and 62.5% and 51.0% for Standard GRPO. Evidence validity stays within 80.5–85.5% across policies, which share one visual backbone. Mean helpfulness rises from 3.09 and 3.10 to 3.24. The supplement gives rating instructions and agreement.
Figure 4. Human evaluation of stored evidence on 200 paired Core200 episodes (7B, three annotators, majority vote). Error bars are 95% bootstrap intervals; the right column is the gain over SFT.
Action Budget and Model–Harness Interaction
Figure 2(a, Figure 2b) varies the action cap. Across all seven budgets, ViStep-RL exceeds SFT by 3.0–6.5 points in Answer and 3.0–7.5 in Strict. Strict peaks at 21.0% at budget 48.
Figure 2. Action budget and visual horizon. (a, b) Qwen2.5-VL-3B SFT and ViStep-RL on 200 episodes per budget. (c) Qwen2.5-VL-7B Strict completion on Core200, grouped by visual turns; episode counts are shown below group labels, and numbers above the SFT and ViStep-RL bars give completed episodes. T+A uses terminal reward with action updates; G+T uses graph reward with trajectory updates.
Visual horizon. Grouping 7B Core200 episodes by images shown, Figure 2(c) shows completions rising from 6 to 13 at 6–8 turns and from 1 to 4 at nine or more turns.
Table 5 compares Direct observation notes with tool-enabled Harness execution on identical Core200 IDs. Harness raises Answer by 1.5 points at Base and 11.0 after training. ViStep-RL uses 8.67× the input tokens and 3.16× the calls, with latency rising from 23.58 to 37.00 seconds per episode.
| Base | ViStep-RL | |||
| Direct | Harness | Direct | Harness | |
| Answer (%) | 16.5 | 18.0 | 18.0 | 29.0 |
| Parse fail (%) | 4.5 | 3.0 | 5.0 | 0.5 |
| Input tokens | 2,798.17 | 22,197.81 | 2,796.97 | 24,249.02 |
| Output tokens | 324.58 | 703.60 | 455.61 | 903.83 |
| Calls | 4.815 | 15.850 | 4.815 | 15.235 |
| Time (s) | 13.38 | 30.38 | 23.58 | 37.00 |
Table 5. Answer quality and cost on Core200. Both 3B policies receive the same public inputs. Direct supplies observation notes; Harness also provides a tool controller. Native execution is evaluated in Table 2(c). Costs are per episode.
Generalization to New Instances and Tasks
New visual instances. Holdout200 excludes Core200 and earlier panels and is selected without model predictions. SFT-to-ViStep-RL Strict rises from 8.0% to 15.5% at 3B and from 40.0% to 50.5% at 7B.
External visual reasoning. Figure 3 compares Qwen2.5-VL-3B ViStep-RL with its reference under matched inputs and prompts. MMLongBench-Doc [22] Accuracy rises from 15.00% to 19.10%, with a 27.05-point gain on unanswerable questions. MuirBench [35] rises from 35.62% to 40.15%, gaining 19.86 points in retrieval and 12.00 in geographic reasoning. MMSI-Bench [43] rises from 25.50% to 27.20%.
Figure 3. Category-wise performance on external benchmarks. Accuracy changes for Qwen2.5-VL-3B ViStep-RL relative to matched references on (a) MMLongBench-Doc, (b) MuirBench and (c) MMSI-Bench. Teal bars indicate improvements, while coral bars indicate declines. Cam., Obj., and Reg. denote camera, object, and region.
Conclusion
ViStep-RL trains visual agents to retain and recover evidence across visual turns through graph-based progress rewards and action-level updates. Under matched interaction resources, 7B Strict rises from 48.5% to 60.0% over terminal-answer GRPO. Selective writing improves Answer under a fixed memory budget, while human evaluation shows gains in provenance and downstream support.
LLM Usage Statement
OpenAI Codex assisted with manuscript drafting, language editing, literature checks, and preparation of analysis scripts. LLMs also assisted with refining the wording of dataset instructions; task graphs and answer labels were determined by programmatic rules.
Theoretical Analysis
This section states the structural assumptions on a task program, characterizes the verified set, and proves that the net-progress return and the signed action weight are consistent with Strict completion. Algorithm S1 summarizes one training iteration.
Notation and assumptions. Write and for the locally satisfied and the verified nodes. A set is ancestor-closed if implies . Let , and let . The ViInteract program compiler (Section B) builds every task program with the following properties:
(A1) node weights are positive;
(A2) no node is verified and no record is returned at , so ;
(A3) the answer node is the unique sink of , and ;
(A4) every edge is a provenance condition of : the evidence of node cites an event produced by node .
Under (A3), . Indeed, following outgoing edges from any node must terminate because is finite and acyclic, and it can only terminate at a node without outgoing edges, which is . Let denote the answer share.
The Verified Set
Lemma S1 (Closure). is the largest ancestor-closed subset of , and VERIFY returns it.
Proof. By Eq. (1), if and only if . Hence . If and , then , so and is ancestor-closed. If is ancestor-closed and , then , so .
Let be the output of VERIFY. Every node added to lies in and has its parents in ; by induction on the order of addition, . Suppose and choose

Figure S1.
in it that is earliest in a topological order. Every parent of lies in and precedes , so it lies in . Since , VERIFY would add before terminating, a contradiction. Hence .
Lemma S2 (Counterfactual credit). Suppose action changes only the predicate of node . If changes from 1 to 0, then and . If changes from 0 to 1, then with , and .
Proof. Loss. Here . The set lies in . It is ancestor-closed: if had an ancestor , then , so , a contradiction. Conversely, any ancestor-closed lies in by Lemma S1, and it contains no , since otherwise . By Lemma S1, .
Gain. For , the set does not contain , so the status of is unchanged. Every was unverified at because , and it is verified at exactly when .
Lemma S2 identifies the progress of an action with the weight of the subgoals whose verification would differ had the action not occurred. Evidence that a descendant gathered earlier is credited at the action that completes its last missing prerequisite, since that action is the one that makes the descendant verifiable.
Corollary S3 (Gating prices downstream evidence). Let be the ungated potential. When an action invalidates a verified node ,
with equality if and only if has no verified descendant.
Proof. The equality on the left is Lemma S2; the one on the right holds because only changes. Since and all weights are positive, the inequality holds, and it is strict exactly when .
Under the ungated potential, corrupting a record that supports verified descendants costs one node share and leaves the descendants credited although their source no longer holds. The gated potential charges the full supported subtree. This is the property that the memory-corruption and write-disable interventions in the main paper exercise.
Proof of Proposition 1
Part (i). If , every , so and . Conversely, with positive weights forces , hence . For the training potential, forces for every node. By (A3), , so . Because , , so . The converse follows from .
Part (ii). Summing Eq. (3) over gives
If the state returns to its earlier verified set and coverage, the progress term vanishes, and each error inside the cycle still contributes .
Part (iii). For binary , , which is zero on non-retrieval nodes and lies in on retrieval nodes. Summing with the node weights gives . If , then and , so the answer node contributes nothing and . □
Return Maximizers are Strict Completions
Theorem S4. Every trajectory satisfies , with equality if and only if and . Every trajectory with satisfies .
Proof. By Eq. (4) and (A2), . If and no error occurs, Proposition 1(i) gives and . Conversely, suppose while or some . Since and the error term is nonnegative, forces both and . Proposition 1(i) then gives , a contradiction. The second claim is Proposition 1(iii) together with . □
Corollary S5. For every policy , .
Proof. Let . By Theorem S4, . Rearranging gives the bound. □
The expected return therefore controls the Strict completion rate. A policy cannot approach the maximal return while leaving a fixed fraction of episodes incomplete.
Potential Differences are the Loop-Neutral Rewards
Definition S6. A progress reward pays at transition . It is loop-neutral if for every closed sequence in .
Loop neutrality states that holding, losing, and regaining the same evidence earns nothing. A reward that violates it pays a policy for repeating some cycle of verification states.
Theorem S7. is loop-neutral if and only if for some .
Proof. If , the sum over a closed sequence telescopes to . Conversely, let be loop-neutral and set . The loop gives . The loop gives . For any , the loop gives . □
Eq. (3) takes , the choice under which Theorem S4 holds without further calibration.
Corollary S8 (Cumulative reward). The cumulative reward is not loop-neutral. Moreover, if , there exist upper-frame trajectories with and with $d_T = 1 such that for every , while .
Proof. The one-step loop with earns . For the second claim, let verify all non-answer nodes by step 1 and hold them until without answering, so for all . Let progress at a constant rate, , and complete at . Then , which is positive under the stated condition on . The returns follow from Theorem S4. □
With unit weights, whenever . The cumulative objective then prefers holding partial evidence without answering over steady progress to completion, and the preference grows linearly with the horizon. The incremental return ranks the same pair correctly at every horizon.
Sign Consistency of the Action Weight
Proposition S9. For every action, . Proof. If , then and , so . If , then . If , then .
Centered advantages lack this property. In a bin with rewards and , the first action has although its reward is negative. At the gradient contribution of a selected action is . By Proposition S9, a step along this direction raises the probability of the action, to first order, exactly when . Selecting actions by preserves the sign, so every retained update moves the policy toward verified progress and away from errors.
Sensitivity to Verification Errors
The potential-difference objective permits a direct analysis of verification errors on a fixed recorded trajectory. Let be a perturbed training potential, where includes the effect of local evidence errors and their propagation through the dependency graph. Compute each transition difference from these consistently recorded potentials. The change in progress return is
Thus, if , the absolute progress-return error is at most , or when the initial potential is fixed exactly. By comparison, a cumulative state-potential return changes by , whose worst-case magnitude is . Differencing keeps a persistent potential error from accumulating with trajectory length.
Let denote the perturbed execution cost, with in Eq. (3). The resulting changes in local reward and episode return are
An error confined to an intermediate potential can change the weights of adjacent actions even when its contributions to the episode return cancel. The episode-sum bound applies to fixed traces. Action weights are computed from the perturbed transition rewards and their comparison bins, so their sensitivity also depends on reward signs and within-bin variation.
Benchmark Card and Training Recipe
ViInteract construction. The ViInteract release contains 76,000 SFT episodes across nine source components, ten execution domains, and 34 task-program types; the 15,000-episode RL source pool and the 4,000-candidate benchmark pool each cover eight source components, nine execution domains, and 29 task-program types. The additional SFT component is COCO spatial relation, with five relation-program types. Evaluation panels are derived from the benchmark candidate pool. In Core200, the Memory Control component contributes 16 temporal-memory episodes and 2 selective-identity episodes, which is why the component-to-domain expansion contains nine domains. Core200 is an application-weighted panel with fixed component/length quotas. It contains 200 frozen episodes, 781 distinct source-image IDs, with no source-image ID shared between episodes; its payload manifest has SHA256 prefix 4b71dd74. Holdout200 is selected after excluding Core200 and the earlier diagnostic panels, contains 787 unique source images, and has manifest SHA256 prefix 80343458. Holdout200 is the held-out release used for transfer results. The two releases use the same public interaction protocol: only the current image is visible, the previous image is evicted after NEXT_IMAGE, visual tools return typed observations, and memory records require an evidence reference to a successful tool event.
Task-program construction and validation. We build episodes from existing visual data using parameterized task-program templates. Source annotations supply reference observations; deterministic rules instantiate graph nodes, prerequisite edges, expected answers, and scripted interactions. LLM-assisted paraphrasing produces instruction wording from this task specification. A gold replay then executes the program under the public runtime contract to check that the constructed task can be completed. Every task program has the final-answer node as its unique sink. Each edge is also a provenance requirement of the child predicate: holds only if node ’s evidence cites an event produced by node . The graph edges therefore coincide with the evidence references checked by the verifier. Table S2 lists the execution domains, and Table S3 compares ViInteract with related visual-agent settings.
| Component / domain | Primary execution requirement | |
| COCO Detection / visual-tool | 50 | locate and verify objects |
| TextOCR / OCR-memory | 35 | extract, write, retrieve text |
| ChartQA / multi-chart | 30 | compare chart evidence |
| COCO Pose / pose-memory | 20 | measure pose relations |
| Memory Control / temporal memory | 16 | preserve state across turns |
| Memory Control / selective identity | 2 | retain identity evidence |
| AITW Temporal / temporal UI | 17 | execute ordered actions |
| SOP Identity / multi-view identity | 15 | match across views |
| LVIS Segmentation / mask-measure | 15 | segment and measure |
| Total | 200 | episodes |
Table S2. ViInteract benchmark card. Counts refer to the frozen Core200 composition.
| Setting | Multi-turn visual input | Active visual operation | Typed tools | Persistent evidence | Dependency-gated success | Primary learning target |
| ViGoRL [29] | ✓ | Grounding / zoom | – | – | – | Spatially grounded reasoning |
| MGPO [12] | ✓ | Multi-turn crop | – | – | – | High-resolution visual reasoning |
| DeepEyes [51] | ✓ | Image operations | – | – | – | Thinking with images |
| ACTIVE-o3 [54] | ✓ | Active perception | Modular sensing | – | – | Information acquisition |
| MuirBench [35] | Multi-image | – | – | – | – | Multi-image understanding |
| MMLongBench-Doc [22] | Long visual document | – | – | – | – | Long-context document understanding |
| ViInteract + ViStep-RL | Streaming images | Learned tool policy | ✓ | ✓ | ✓ | Evidence-state progress |
Table S3. Positioning against active visual reasoning settings. The table compares the execution contract and learning target. A checkmark denotes an explicit part of the reported setting; a dash denotes that the capability is outside the benchmark or method target.
Reconstructing the frozen panels. Table S1 and Table S4 specify the frozen panel composition. The panel builder is deterministic given the source pool, exclusion manifests, and seed. For every candidate, we first assign its source component and turn-length bucket –––. For the six short-capable components (ChartQA, COCO Detection, COCO Pose, LVIS Segmentation, SOP Identity, and TextOCR), one-, two-, and three-turn controls are derived from the first qualifying visual turns; the derived task program keeps only nodes verified by a gold replay and rewrites the final turn to the public answer schema. Existing four-or-more-turn episodes are retained without shortening. Candidates whose source episode or image path occurs in an exclusion manifest are removed before selection. Within each component/length cell, rows are sorted by SHA256( :episode_id) and greedily accepted until the predeclared quota is filled, rejecting repeated source IDs and repeated source images. The selected rows are sorted by the same rank, replayed through the official runtime, and frozen with a payload hash. The Core200 and Holdout200 builders use seed 20260907; Holdout200 applies the same procedure after excluding Core200 and all earlier diagnostic panels. No model prediction is read during selection. Table S5 reports the release size and structural statistics.
| Source component | Core200 1 | Core200 2 | Core200 3 | Core200 4–5 | Core200 6–8 | Core200 9–12 | Core200 13+ | Holdout200 1 | Holdout200 2 | Holdout200 3 | Holdout200 4–5 | Holdout200 6–8 | Holdout200 9–12 | Holdout200 13+ |
| COCO Detection | 5 | 15 | 15 | 15 | – | – | – | 5 | 14 | 14 | – | – | – | – |
| TextOCR | 2 | 8 | 12 | 6 | 7 | – | – | 2 | 8 | 12 | 6 | 3 | – | – |
| ChartQA | 2 | 10 | 8 | 10 | – | – | – | 2 | 10 | 8 | 3 | – | – | – |
| COCO Pose | 1 | 3 | 3 | 4 | 9 | – | – | 1 | 3 | 3 | 4 | – | – | – |
| LVIS Segmentation | – | 3 | 2 | 2 | 8 | – | – | – | 3 | 2 | 2 | 8 | – | – |
| SOP Identity | – | 3 | 2 | 3 | 7 | – | – | – | 3 | 2 | 3 | 7 | – | – |
| AITW Temporal | – | – | – | 3 | 3 | 5 | 6 | – | – | – | 40 | 3 | 5 | 6 |
| Memory Control | – | – | – | 3 | 3 | 6 | 6 | – | – | – | 3 | 3 | 6 | 6 |
| Total | 10 | 42 | 42 | 46 | 37 | 11 | 12 | 10 | 41 | 41 | 61 | 24 | 11 | 12 |
Table S1. Exact panel quotas used by the deterministic builder. A dash denotes an empty component/length cell. The component totals reproduce the Core200 and Holdout200 panels; the resulting manifests and source-pool hash identify the released instances.
| Source component | Core200 | Holdout200 |
| AITW temporal | 17 | 54 |
| ChartQA | 30 | 23 |
| COCO detection | 50 | 33 |
| COCO pose | 20 | 11 |
| LVIS segmentation | 15 | 15 |
| Memory control | 18 | 18 |
| SOP identity | 15 | 15 |
| TextOCR | 35 | 31 |
| Length 1 | 10 | 10 |
| Length 2 | 42 | 41 |
| Length 3 | 42 | 41 |
| Length 4–5 | 46 | 61 |
| Length 6–8 | 37 | 24 |
| Length 9–12 | 11 | 11 |
| Length 13+ | 12 | 12 |
Table S4. Panel composition by source component and episode length. Counts are frozen before the corresponding evaluation matrix.
| Release | Images/ep. | Turns/ep. | Nodes/ep. | Unique images | ID |
| Core200 | 1–16 | 1–16 | 3–33 | 781 | 4b71 |
| Holdout200 | 1–16 | 1–16 | 3–33 | 787 | 8034 |
| Mean (Core) | 4.03 | 4.82 | 10.80 | – | – |
| Mean (Holdout) | 4.05 | 4.66 | 10.12 | – | – |
Table S5. Frozen release statistics. Core200 and Holdout200 are disjoint evaluation releases; prefixes identify the payload manifests recorded in the release ledger.
Data isolation audit. The SFT snapshot contains 76,000 training episodes and the GRPO snapshot contains 15,000 training episodes. We compared each training snapshot with Core200, Holdout200, and the 500-item process panel using episode IDs, complete image paths, and source-sequence IDs. Every comparison has zero overlap; Core200 and Holdout200 are also disjoint on all three keys. The raw numeric image ID is a local identifier reused across source components and is therefore not used as a leakage key. The 500 episodes used for the reward comparisons in main-text Table 3 have payload hash prefix 00930e46. A separate check of these episodes, their 2,500 distinct image paths and 215 source sequence identifiers found zero overlap with either the SFT or RL training split on these keys. The composition tables give the component and length mix of each panel; Holdout200 contains 54 AITW temporal episodes and Core200 contains 17.
Episode semantics. An episode is a sequence of public turns with a current image, a task instruction, legal actions, and returned tool or memory observations. The global action budget is 32 for the reported final-answer protocol; generation is capped at 768 tokens and image pixels at 262,144. The hidden task program defines dependency-gated nodes for evidence acquisition, cross-image reasoning, memory writing, memory retrieval, and the final answer. Answer checks the final structured response. Strict additionally requires every dependency-gated node to be verified with the required provenance. The policy never receives the hidden node identifiers or expected evidence values.
Graph construction rules. The compiler builds each graph from the evidence that the answer schema cites. An edge is created exactly when the predicate of node cites an event produced by node , such as a memory write citing the OCR call that produced its payload; no other edges are added. The answer node is appended last and receives an edge from every node that has no other child, so it is the unique sink and every node is an ancestor of it. The answer node carries no retrieval coverage. Positive weights default to one. These rules give properties (A1)–(A4) of Section A for every released program.
Task-graph authoring and verification. Each released episode stores a task program whose nodes are derived from source annotations, typed tool outputs, deterministic computations, memory operations, and the final answer schema. A node records its type, expected value or source annotation, parent node identifiers, and an optional weight; the runtime assigns weight one when no weight is specified. The verifier accepts a visual node only when the corresponding typed observation contains the expected evidence. A memory node additionally requires a matching payload or claim, an image or semantic identity, and evidence references that resolve to successful evidence-producing calls in the same trajectory. A retrieval node is scored during training by , but is strict-verified only at complete coverage. The verifier recomputes the dependency fixed point after every transition, so deleting or corrupting a memory record retracts that node and every dependent node. The same fixed point defines both the Strict metric and the progress signal.
Top-level release schema and verifier fixed point. Table S6 and Table S7 distinguish public record fields from hidden verifier fields. Each frozen JSON object has an immutable public payload and a hidden evaluation payload. The public fields define what the policy can observe; the hidden fields define what the scorer can verify. After every action, the runtime (1) applies the public state transition, (2) records the typed observation and action provenance, (3) checks evidence and memory payloads, (4) activates nodes whose local predicates and parents are satisfied, and (5) repeats activation until the node set reaches a fixed point. It then emits the verified node set, retrieval coverage, and the before/after potentials. A state update that invalidates a parent automatically retracts all dependent nodes before the next credit difference is computed.
| Field | Stored content | Visibility |
id, episode_id | Stable episode and source identifiers | Public |
dataset_version, partition, split | Release and partition metadata | Public |
domain, release_component | Task family and source component | Public |
source.sequence.id, images | Source sequence and ordered image records | Public |
student.visible | Instruction, current image, legal actions, observations | Public |
tool.state | Typed tool outputs and action provenance | Public |
task.program | Nodes, dependencies, expected evidence, weights | Hidden |
hidden.eval | Strict predicates and answer checks | Hidden |
counterfactual.protocol | Intervention hooks for memory/tool studies | Hidden |
metadata, immutable.payload.hash | Construction metadata and payload identity | Hidden |
Table S6. Top-level ViInteract release schema. Public fields are visible to the policy; hidden fields are consumed only by the verifier and scorer.
| Field | Stored content | Runtime use |
node.id | Stable node identifier | Dependency references |
type | Evidence, memory, retrieval, computation, or answer | Selects verifier |
depends.on | Parent node identifiers | Fixed-point ordering |
expected | Source value, annotation, or deterministic predicate | Evidence match |
evidence.refs | Producing tool-call identifiers | Provenance check |
weight | Optional positive node weight | Potential normalization |
memory.refs | Required memory identifiers/fields | Write and retrieval checks |
Table S7. Task-program fields and verification rules. Hidden fields are consumed by the runtime and scorer; they are excluded from the policy observation.
Controlled evaluation interfaces. The memory factorial uses the frozen Core200 release and the same 3B trained weights as the native comparison. All four conditions run under a wrapper with at most eight stored records, 512 serialized characters per record and four records per retrieval. Active writing lets the policy select what to store, while automatic writing stores observations through the wrapper. Storage limits and retrieval limits are shared. Actual write counts and occupied storage depend on the policy. The Active/Active row is evaluated with these constraints.
The model–Harness factorial uses the same Core200 episode IDs in every condition. Its Direct arm exposes images sequentially and accumulates public observation notes for a common final-answer request. The Harness arm enables tool calls and manages image progression, context and memory. The native training-stage comparison instead asks the policy to act through the agent protocol, including legal tool calls and answer submission. Answer scores are compared within each interface.
The Standard GRPO baselines use the same SFT initialization, tools, action budget and training volume as ViStep-RL at each scale. The baseline rewards terminal-answer correctness and normalizes episode returns within each rollout group, sharing the resulting advantage across the trajectory. In the 7B terminal-reward control with action-level updates, only the final answer action receives the answer reward. All preceding actions receive zero reward and therefore zero signed reward weight. Answer and Strict are reported on Core200 and the remaining four metrics on Process500.
Action weights and policy loss. For each prompt we sample eight trajectories and admit the group when the population standard deviation of its returns exceeds . All groups in this comparison share the same prompt. Any other group contributes zero policy-loss weight. A comparison bin contains actions with the same phase, visual turn, and round(). For bin mean and population standard deviation of transition rewards, we use , with when . In Eq. (6), . Rewards with receive zero weight. A singleton bin or a bin with identical rewards has zero standardized term. Its nonzero transition rewards still contribute through . A zero transition reward always receives zero weight.
For a generated token with prefix in selected action , let , where sampled the action. Let for the frozen SFT reference. The per-token loss is
We average first over generated tokens within an action, then over selected actions within a rollout, and finally over rollouts. The loss uses immediate transition weights from main-text Eq. (6). The episode return is used for group admission. Selected actions are ranked by absolute weight, with seeded random ordering for ties. We select up to two actions per resumed 3B rollout and four per 7B rollout. When fewer actions are available, padding entries receive zero loss weight.
Training recipe. The primary Qwen runs use rank-16 LoRA with alpha 32, learning rate , seed 42, eight rollouts per prompt, temperature 0.9, and top-p 0.95. Training uses bfloat16 precision, a 768-token generation cap and at most 262,144 pixels per image. The 7B run caps prompts at 8,192 tokens and rollouts at 32 actions, selecting up to four actions per rollout. The resumed 3B run caps prompts at 11,520 tokens and rollouts at 16 actions, selecting up to two. For each transition, the environment records the public state, action, observation, memory provenance, and the before/after training potential. The primary Qwen2.5-VL-7B run completed 1,000 updates and 32,000 sampled rollouts on four GPUs, using the final saved adapter for evaluation. The primary native Qwen2.5-VL-3B policy is the step-700 check-
point; the matched Holdout200 write-disable and corruption runs use the same 3B policy as their intact control.
Credit computation. Table S8 describes how the recorded Qwen transitions enter the policy update in main-text Eq. (6) and Eq. (10) above.
| Step | Operation and emitted state |
| 1 | Sample a rollout group under the current policy. |
| 2 | Execute each action and record the public observation, typed tool result, memory provenance, and verifier state. |
| 3 | Recompute the dependency fixed point and evaluate and the execution cost. |
| 4 | Check variation in rollout returns, then compare transition rewards within phase, turn, and progress bins. |
| 5 | Select actions by absolute weight and apply the clipped token loss with the frozen SFT reference. |
Table S8. Qwen policy update. The verifier evaluates every environment transition; each selected action receives its own signed weight.
External evaluation protocol. On MMLongBench-Doc, MuirBench and MMSI-Bench, we compare Qwen2.5-VL-3B ViStep-RL with a reference model of the same backbone. Within each benchmark, both models receive identical inputs and use the same evaluation prompt.
Execution Analysis on Core200
Table S9 reports the interaction cost of Qwen2.5-VL-7B on the same Core200 episodes. ViStep-RL completes more episodes than SFT with fewer actions, fewer action tokens, and less time per episode.
| Stage | Actions | Action tokens | Time (s) |
| Base | 19.54 | 1,386.8 | 44.87 |
| SFT | 15.76 | 1,921.0 | 72.46 |
| ViStep-RL | 15.00 | 1,732.2 | 67.43 |
Table S9. Interaction efficiency on Core200. Qwen2.5-VL-7B; values are per episode.
The paired ViStep-RL minus SFT action-count difference is −0.765 actions per episode. Action tokens are estimated by re-tokenizing saved action text; time is the per-episode share of batch wall time.
The native 7B comparison uses 200 matched Core200 episodes. ViStep-RL gains 32 Strict successes over SFT and loses five, a net gain of 27 episodes (13.5 points). The 95% paired percentile bootstrap interval is [8.0, 19.0] points over 10,000 episode-ID resamples with seed 42, and the two-sided exact McNemar test gives .
Human Evaluation Protocol
Sample and raters. The study covers all 200 Core200 episodes and the Qwen2.5-VL-7B traces of SFT, Standard GRPO and ViStep-RL, giving 600 traces. Three annotators rate every trace independently. Model names are hidden, and the three traces of an episode appear in random order.
Rating instructions. Annotators first see the task instruction, the source images, the tool outputs and the memory write and retrieval log, with the final answer hidden. They answer three binary questions and one graded question.
Evidence validity: every stored record matches the content of its source image.
Provenance validity: every stored record names its source image and the tool event that produced it.
Downstream support: the records retrieved before the final decision suffice to determine the answer.
Helpfulness: how much the trace helps a reader check the answer, from 1 (not at all) to 5 (fully).
A binary criterion holds for a trace when at least two annotators accept it. A trace is rated helpful when at least two annotators give a score of 4 or higher. Intervals are percentile intervals from 10,000 episode-level bootstrap resamples with seed 0.
| Criterion | Fleiss’ | Unanimous (%) |
| Evidence validity | 0.744 | 88.2 |
| Provenance validity | 0.716 | 79.7 |
| Downstream support | 0.674 | 75.7 |
Table S10. Inter-annotator agreement over all 600 traces.
Agreement. All three binary criteria reach substantial agreement. For helpfulness, 89.8% of annotator pairs give scores within one point of each other.
Qualitative Trajectory Comparison
Figure S1 follows the same TextOCR episode across all three training stages. All stages successfully read and store “JAG,” “owed,” and “think” from images 1–3, respectively. The three stages therefore differ only in how they use the stored evidence. The final question is asked with no image visible. In each node strip, the first six blocks represent three OCR and three memory-write nodes; the last two represent task-level retrieval and final-answer verification. Teal denotes verified nodes and gray denotes unverified nodes.
Base searches for a record whose label is “think,” although its stored labels describe source regions, and receives no records in 11 consecutive searches. SFT searches using the first region’s label and retrieves only the “JAG” record, then answers Image 1. ViStep-RL searches by the shared claim exact_scene_text, obtains all three records ordered by image index, and answers Image 3. Final task verification is 6/8, 6/8, and 8/8 nodes for Base, SFT, and ViStep-RL, respectively, with 20, 15, and 14 total actions. Completing the episode requires retrieving all three records with a query that matches how they were stored.
Figure S1 (PDF p. 19). Same evidence, different task completion. Matched Qwen2.5-VL-7B trajectories on one TextOCR episode. All stages read and store the same evidence; after image eviction, only ViStep-RL retrieves all required records and answers correctly. Counts cover the full recorded trajectories.
General-Purpose Model Evaluation
Evaluation protocol. Harness runs use the 200 Core200 episode IDs and Direct runs use the matched 200-episode Direct panel of the main paper. All API runs use temperature 0, seed 42, one final submission, a 32-action global tool budget, at most four tools per image, and at most eight tools in the final phase. Generation is capped at 768 tokens and images at 262,144 pixels. Direct supplies rolling image-observation notes; Harness adds tool dispatch and retains the returned public tool observations. Every endpoint receives the same public instructions, and ViStep-RL runs inside the same Harness. Table S11 gives paired differences on the shared Harness.
| API model | Difference | 95% CI |
| GPT-4o | +24.5 | [17.5, 31.5] |
| Claude Opus 4.6 | +29.0 | [21.0, 37.0] |
| Qwen3-VL-Plus | +28.0 | [20.0, 36.0] |
| Qwen3.8-Flash | +24.0 | [16.5, 31.5] |
| Gemini 3.8 Flash | +25.0 | [17.0, 32.5] |
| Kimi K2.5 | +37.5 | [29.5, 45.5] |
Table S11. Paired advantage of the trained Qwen2.5-VL-7B with Harness over each API model with Harness. Differences are percentage points on the same 200 items.
Intervals use 10,000 paired episode-ID bootstrap resamples with seed 42 and lexicographically sorted IDs. Each resample keeps both models’ outcomes for an episode together. These intervals describe uncertainty across the 200 evaluated episodes.
References
- [1]Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce-style optimization for learning from human feedback in llms. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12248–12267, 2024. 2arxiv.org/abs/2402.14740
- [2]Akari Asai, Zeqiu Wu, Yizhong Wang, Avi Sil, and Hannaneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In International conference on learning representations, pages 9112–9141, 2024. 2arxiv.org/abs/2310.11511
- [3]Shuai Bai, Keqin Chen, Xue-Jing Liu, Jia-Lin Wang, Wen-Bin Ge, Si-Bo Song, Kai Dang, Peng Wang, Shi-Jie Wang, Jun Tang, Hu-Men Zhong, Yuanzhi Zhu, Ming-Hsuan Yang, Zhao-Hai Li, Jian-Qiang Wan, Peng-Fei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tian-Bao Xie, Zezen Cheng, Hang Zhang, Zhi-Bo Yang, Hai-Yang Xu, and Jun-Yang Lin. Qwen2.5-vl technical report. arXiv, abs/2502.13923, 2025. 6arxiv.org/abs/2502.13923
- [4]Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. In International conference on machine learning, pages 2206–2240. PMLR, 2022. 2arxiv.org/abs/2112.04426
- [5]Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024. 2
- [6]Sanjiban Choudhury. Process reward models for llm agents: Practical framework and directions. arXiv preprint arXiv:2502.10325, 2025. 2arxiv.org/abs/2502.10325
- [7]Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. Advances in Neural Information Processing Systems, 38, 2026. 2
- [8]Xingyu Fu, Yushi Hu, Bangzhen Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. In European Conference on Computer Vision, pages 148–166. Springer, 2024. 2arxiv.org/abs/2404.12390
- [9]Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904–6913, 2017. 2
- [10]Tongkun Guan, Zhibo Yang, Mingkun Yang, Haocheng Wang, Wei Shen, and Xiaokang Yang. Perceptrl: Making visual perception redrawable in mlms from answers to interactions. arXiv preprint arXiv:2610.37471, 2026. 2arxiv.org/abs/2610.37471
- [11]Bernal J Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. Hipporag: Neurobiologically inspired long-term memory for large language models. Advances in neural information processing systems, 37:59532–59569, 2024. 2arxiv.org/abs/2405.14831
- [12]Xinyu Huang, Yuhao Dong, Weiwei Tian, Bo Li, Rui Feng, and Ziwei Liu. High-resolution visual reasoning via multi-turn grounding-based reinforcement learning. arXiv preprint arXiv:2507.05920, 2025. 1, 2, 17arxiv.org/abs/2507.05920
- [13]Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6693–6702. IEEE, 2019. 2arxiv.org/abs/1902.09506
- [14]Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. arXiv preprint arXiv:2405.01483, 2024. 2, 4arxiv.org/abs/2405.01483
- [15]Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried. Visualwebarena: Evaluating multimodal agents on realistic visual web tasks. pages 881–905, 2024. 4
- [16]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474, 2020. 2arxiv.org/abs/2005.11401
- [17]Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations, pages 39578–39601, 2024. 2arxiv.org/abs/2305.20050
- [18]Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. 2
- [19]Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xue, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. In International Conference on Learning Representations, pages 52989–53046, 2024. 2arxiv.org/abs/2308.03688
- [20]Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hanneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations, pages 23439–23554, 2024. 2arxiv.org/abs/2310.02255
- [21]Chang Ma, Jiuming Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. Agentboard: An analytical evaluation board of multi-turn llm agents. Advances in neural information processing systems, 37:74325–74362, 2024. 4, 6arxiv.org/abs/2401.13178
- [22]Yubo Ma, Yuhang Zang, Liangyu Chen, Meiqi Chen, Yizhu Jiao, Xinze Li, Xinyuan Lu, Ziyu Liu, Yan Ma, Xiaoyi Dong, et al. MLmongbench-doc: Benchmarking long-context document understanding with visualizations. Advances in Neural Information Processing Systems, 37:95963–96010, 2024. 8, 17arxiv.org/abs/2407.01523
- [23]Ahmed Masry, Jia Qing Tan, Shafiq Joty, Enamul Hoque, et al. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, pages 2263–2279, 2022. 2
- [24]Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2199–2208. IEEE, 2021. 2arxiv.org/abs/2007.00398
- [25]Andrew Y Ng, Daishi Harada, and Stuart Russell. Policy invariance under reward transformations: Theory and application to reward shaping. In Icml, pages 278–287. Citeseer, 1999. 3, 4
- [26]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G Patil, Ion Stoica, and Joseph E Gonzalez. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560, 2023. 2arxiv.org/abs/2310.08560
- [27]Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023. 2arxiv.org/abs/2304.03442
- [28]Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. Androidinthe wild: A large-scale dataset for android device control. Advances in Neural Information Processing Systems, 36:59708–59728, 2023. 2
- [29]Gabriel Sarch, Snigdha Saha, Najib Khandelwal, Ayush Jain, Michael Tarr, Aviral Kumar, and Katerina Fragkiadaki. Grounded reinforcement learning for visual reasoning. Advances in Neural Information Processing Systems, 38:150977–151013, 2026. 1, 2, 17
- [30]Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in neural information processing systems, 36:68539–68551, 2023. 2arxiv.org/abs/2302.04761
- [31]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. 2arxiv.org/abs/1707.06347
- [32]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. 2arxiv.org/abs/2402.03300
- [33]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652, 2023. 2arxiv.org/abs/2303.11366
- [34]Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8309–8318. IEEE, 2019. 2arxiv.org/abs/1904.08920
- [35]Fei Wang, Xingyu Fu, James Y Huang, Zekun Li, Qin Liu, Xiaogeng Liu, Mingyu Derek Ma, Nan Xu, Wenxuan Zhou, Kai Zhang, et al. A comprehensive benchmark for robust multi-image understanding. In International Conference on Learning Representations, pages 62624–62650, 2025. 2, 4, 8, 17arxiv.org/abs/2406.09411
- [36]Guanzhi Bai, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023. 2
- [37]Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 2arxiv.org/abs/2409.12191
- [38]Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory. arXiv preprint arXiv:2409.07429, 2024. 2arxiv.org/abs/2409.07429
- [39]Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256, 1992. 2
- [40]Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing, and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023. 2arxiv.org/abs/2303.04671
- [41]Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh J Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, et al. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments. Advances in Neural Information Processing Systems, 37:52040–52094, 2024. 2, 4arxiv.org/abs/2404.07972
- [42]Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents. pages 17577–17604, 2026. 2
- [43]Sihan Yang, Kunse Xु, Yiman Xie, Sizhe Yang, Mo Li, Jingli Lin, Chenming Zhu, Xiaochen Chen, Haodong Duan, Xiangyu Yue, et al. Mmsi-bench: A benchmark for multi-image spatial intelligence. In International Conference on Learning Representations, pages 157051–157088, 2026. 8arxiv.org/abs/2505.23764
- [44]Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303.11381, 2023. 2arxiv.org/abs/2303.11381
- [45]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. 2arxiv.org/abs/2210.03629
- [46]Kaining Ying, Fanqing Meng, Jin Wang, Zhiqian Li, Han Lin, Yue Yang, Hao Zhang, Wenbo Zhang, Yuqi Lin, Shuo Liu, et al. Mmt-bench: A comprehensive multimodal benchmark for evaluating large vision-language models towards multitask agi. arXiv preprint arXiv:2404.16006, 2024. 2arxiv.org/abs/2404.16006
- [47]Qiying Yu, Zheng Chang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. Advances in Neural Information Processing Systems, 38:113222–113244, 2026. 2arxiv.org/abs/2503.14476
- [48]Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490, 2023. 2arxiv.org/abs/2308.02490
- [49]Huining Yuan, Zelai Xu, Huajie Wang, Xiangmin Yi, Jiaxuan Gao, Xiao-Ping Zhang, Yu Wang, Chao Yu, and Yi Wu. Veri-fiable process rewards for agentic reasoning. arXiv preprint arXiv:2605.10325, 2026. 2arxiv.org/abs/2605.10325
- [50]Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9556–9567, 2024. 2arxiv.org/abs/2311.16502
- [51]Ziwei Zheng, Minghao Yang, Jing Hong, Chenxiao Zhao, Guohai Xu, Le Yang, and Chao Shen. Deepeyes: Incentiviz-ing” thinking with images” via reinforcement learning. In International Conference on Learning Representations, pages 126775–126798, 2026. 1, 2, 17arxiv.org/abs/2505.14362
- [52]Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI conference on artificial intelligence, pages 19724–19731, 2024. 2arxiv.org/abs/2305.10250
- [53]Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations, pages 15585–15606, 2024. 2arxiv.org/abs/2307.13854
- [54]Muzhi Zhu, Hao Zhong, Canyu Zhao, Zongze Du, Mingyu Liu, Zheng Huang, Anzhou Li, Hao Chen, Cheng Zou, Jingdong Chen, et al. Active-o3: Empowering mllms with active perception via pure reinforcement learning. arXiv preprint arXiv:2505.21457, 2025. 1, 2, 17arxiv.org/abs/2505.21457




