WorksComputer Science
Recover, Don't Just Continue: Learning from Feedback for Physical Failure Understanding and Correction
Abstract
Multimodal large language models are regarded as the cognitive foundation of embodied intelligence, but planning from instructions alone cannot make them reliable. During execution, actions deviate from intended effects, constrained by the embodiment and environment. An embodied brain must understand why execution deviates, identify what can be reliably realized, and revise decisions using interaction experience. These correspond to causal failure localization, capability-constrained adaptation, and experience-driven revision. Current embodied foundation models largely lack these capabilities. To address this, we introduce Embodied-Corrector. We construct failure-recovery data by injecting perturbations at subtask boundaries of successful trajectories, replaying affected subtasks, and keeping failures that are observable from public evidence and recoverable from the current state. We align actions, consequences, failure causes, and recoveries into Judge-Cause-Sub-Fine supervision. Training uses evaluation-first SFT: the model learns Judge and Cause first, then Sub and Fine conditioned on accurate prefixes. Since deployment uses self-generated prefixes, evaluation errors propagate into recovery planning, and response-level rewards are too coarse to locate affected planning tokens. We therefore use OPD to internalize the planning distribution under accurate prefixes into that under self-generated ones: the model generates plans from self-generated prefixes, while a contrastive forward pass with accurate prefixes compares token probabilities to reinforce or suppress planning tokens. At deployment, this branch is removed. Experiments show stronger failure analysis and corrective planning: with a frozen VLA, composite task success improves by 6.1 percentage points; with fixed GPT-5.5 execution, manipulation and navigation tasks improve by 13.9 percentage points.
Project: https://pm1255.github.io/embodied-corrector/
Email: [email protected]
Introduction
Multimodal large language models are becoming an important foundation for embodied intelligence [14, 8, 1, 43]. By combining visual understanding with language reasoning, they can interpret task goals, reason about scene relations, and decompose complex instructions into executable steps [26, 20, 48 Song et al.,] 2023; Pan et al. [39]. Recent embodied foundation models such as RynnBrain (Dang et al. [11]), RoboBrain 2.0 (BAAI RoboBrain Team [7]), and Embodied-Reasoner (Zhang et al. [62]) have advanced spatio-temporal memory, physical-world reasoning, and embodied planning; VeBrain-1.5 (VeBrain Team [51]) further unifies multimodal perception, spatial reasoning, and robot control in a shared decision interface. However, a reliable embodied brain requires more than planning. During physical execution, actions deviate from intended effects, and what can be realized is jointly constrained by the embodiment, controller, and environment (Anonymous [4]). An embodied brain must therefore understand why execution deviated, recognize what the current executor can reliably realize, and adjust decisions using interaction experience. This raises the central question: how can a multimodal model evolve from planning what to do into an embodied brain that understands physical interaction, adapts to execution capability, and corrects its behavior after failure?
We argue that this requires three related but non-substitutable capabilities. First, physical causal understanding and failure localization (Ying et al. [61]; Liu et al. [30]; Anonymous [6]) requires determining not only that an execution failed, but also which action, physical relation, or unmet prerequisite caused it. Existing failure-memory and detection methods can flag unsuccessful outcomes (Ying et al. [61]), yet a model may observe an object not placed into a container while failing to distinguish incorrect grasp timing, inappropriate contact pose, or an earlier uncompleted step. Effective correction requires tracing execution back to the condition that first invalidated progress. Second, execution-capability identification and adaptation (Physical Intelligence Team (2026); Ghosh et al. [18]) requires translating intent into a form the current executor can reliably realize. Even a correct diagnosis is ineffective if the recovery exceeds the controller’s capability. The brain should adapt instruction content and granularity, e.g., decomposing a long request into state-conditioned subtasks for a frozen VLA Physical Intelligence Team (2026); Brohan et al. [9]; O’Neill et al. [37]. Third, in-context learning from interaction experience (Chen et al. [10]; Zhang et al. [65]; Sridhar et al. [50]) requires information from one execution to change the next decision. Prior work indicates that a robot’s own physical interaction experience can support in-context learning even when the policy model is frozen (Chen et al. [10]). Failure observations, previous actions, and successful corrections should become evidence for planning rather than accumulating in context while the model repeats the same behavior (Anonymous [3]).
We first examine whether current embodied brains possess these capabilities. On failed EB-ALFRED (Yang et al. [57]) executions, we supply frozen multimodal models with the same task goal, visual observations, executed actions, and execution feedback, and inspect predicted failure cause and proposed next-step correction. As shown in Figure 1(a), models such as RynnBrain (Dang et al. [11]), MiMo-Embodied (Xiaomi MiMo Team [55]), and InternVL3 (OpenGVLab [38]) frequently produce plausible descriptions while failing to identify the condition that actually blocks progress, and their plans often fail to repair it. This exposes weak physical causal understanding and failure localization. The missing capability is not merely recognizing failure, but transforming execution evidence into an explanation of why it failed and what must change next.
Figure 1. Failure diagnosis and feedback controls on EB-ALFRED. (a) Frozen multimodal models struggle to identify the physical cause of failed execution. (b) With the model and action interface fixed, cause-level feedback improves replanning more than repeating the failed outcome.
We next ask what form this missing information should take. Keeping the evaluated brain, visual input, action interface, and decoding fixed on EB-ALFRED, we vary only the feedback after failure. As shown in Figure 1(b), repeating the failed outcome provides little benefit (Shinn et al. [46]), whereas supplying the concrete failure cause consistently improves performance. For RoboBrain2.5-4B, MiMo-Embodied-7B, and InternVL3-8B, cause-level feedback changes task success from 0.22 to 0.28, from 0.20 to 0.27, and from 0.20 to 0.24. Useful feedback does not simply state failure; it identifies which expected condition was violated and why that prevents progress, revising the brain’s interpretation of the task state. Together, these diagnostics indicate that current models lack not only failure analysis, but also execution-capability adaptation and in-context learning from interaction: outcome repetition alone does not change subsequent decisions, and only explicit causal feedback enables correction.
Motivated by this, we introduce Embodied-Corrector, which couples the three capabilities through data, training, and inference. For physical causal understanding and failure localization, we build controlled failure–recovery data by branching successful trajectories at subtask boundaries and intervening on navigation, reachability, grasp pose, control timing, articulation state, task prerequisites, and target grounding. Each branch shares the preceding successful context while changing one controlled factor, and we retain only failures observable from public execution evidence and recoverable from the realized state. The resulting paired failure-recovery experience connects what changed, what consequence it caused, and what must be restored.
To obtain such data at scale, we build an automated pipeline that coordinates simulation environments, robot execution interfaces, multimodal models, and tools for trajectory generation, failure construction, cause annotation, and correction verification, with a Harness providing runtime support for interaction and data collection. Each sample contains four fields: Judge determines whether the current subtask is completed; Cause identifies the failure cause supported by observable evidence; Sub decides whether to advance to the next subtask or redo the current subtask; and Fine specifies how to execute Sub, including actions, target arguments, execution order, and verification steps. Learning from such data is challenging: Judge/Cause errors contaminate Sub/Fine; SFT trains Sub/Fine with accurate Judge/Cause prefixes, whereas deployment can only use self-generated Judge/Cause, creating a prefix distribution mismatch; response-level rewards are too coarse to locate the planning tokens affected by evaluation errors; and Fine must satisfy the realizability constraints of a frozen executor. We therefore adopt “evaluation-first SFT + OPD internalization.” In the first stage, we train Judge/Cause, and then supervise Sub/Fine generation with accurate Judge/Cause prefixes. In the second stage, we use OPD: during training, the model generates Sub/Fine from self-generated Judge/Cause prefixes, while a contrastive forward pass with accurate Judge/Cause prefixes compares token probabilities and reinforces planning words supported by accurate diagnoses or suppresses contradictory ones. Accurate diagnoses are used only for scoring and do not replace self-generated prefixes; at deployment, the contrastive branch is removed, and the model still autonomously generates Judge/Cause and Sub/Fine. Thus, a single inference outputs the complete Judge–Cause–Sub–Fine and produces state-conditioned recovery plans realizable by a frozen executor.
Experiments evaluate corrective reasoning and execution with fixed backends. First, on brain-level corrective reasoning, Embodied-Corrector substantially improves failure-cause analysis and produces recovery plans consistent with the identified physical cause, demonstrating stronger physical causal reasoning rather than outcome classification alone. Second, with the VLA frozen, execution-aware decomposition improves by 5.4 percentage points on Composite-Seen tasks and by 6.1 percentage points on Composite-Unseen tasks, showing that the brain can adapt task granularity to reduce the burden on a fixed executor. Third, with the GPT-5.5 code generator, robot functions, perception module, and execution budget all fixed, feeding Embodied-Corrector’s diagnosis and recovery back into the next planning context improves all ten execution conditions over the untrained corrective brain, with a macro-average gain of 13.9 percentage points. This shows that execution feedback is not merely stored in history: it changes the next workflow produced by the same planner. Our contributions are threefold:
We identify physical causal understanding, execution-capability adaptation, and in-context learning from interaction feedback as three complementary capabilities for corrective embodied reasoning, and diagnose their absence in current multimodal embodied brains.
We introduce Embodied-Corrector and a controlled failure-recovery data construction pipeline connecting actions, observable physical consequences, failure causes, and executable recoveries, together with evaluation-first SFT.
We evaluate correction from brain-level failure reasoning to fixed-backend execution, showing the learned brain can explain failures, adapt instructions to a frozen VLA, and provide corrective context that changes the subsequent thinking of a code generator.
Related Work
Embodied agent architectures. Language-model robot systems commonly separate semantic planning from physical execution [53, 17, 31]. SayCan grounds skill selection with learned affordances, Inner Monologue returns environment feedback to the planning context, and ReAct alternates reasoning with tool use [2, 21, 60]. Code as Policies turns language plans into executable programs, while CaP-X evaluates code agents through perception, geometry, and manipulation interfaces [27, 16]. Harness VLA routes plans to frozen VLA skills, and FineVLA and InstructVLA show that language guidance must retain execution-critical attributes [64, 19, 58, 15, 59]. These systems motivate our fixed execution backends, but our central question is upstream: how failure evidence should change the brain plan before the backend attempts the next recovery.
Embodied execution correction. Correction begins with feedback grounded in what the robot actually did [22]. Inner Monologue feeds success, scene, and human feedback back into high-level planning, while model-based runtime monitoring detects anomalous or risky execution states [21, 29]. At the controller level, VLA-Corrector compares predicted and observed visual dynamics, truncates stale action chunks, and triggers local replanning [40]. Language-agent methods such as Reflexion and Self-Refine revise a prior answer from textual feedback [47, 35], but their feedback need not identify a physically observable cause or verify that a repair changes the environment. ASPIRE comes closest to that closed loop: its execution engine exposes fine-grained multimodal traces for failure diagnosis, repair synthesis, and validation [32]. Embodied-Reasoner, RynnBrain, and RxBrain additionally provide multimodal task reasoning and execution evaluation [63, 12, 25]. Our focus is the missing task-level connection among these capabilities: after each completed subtask, identify the observable cause, revise the remaining dependency plan from the current state, and verify that the same fixed executor realizes the correction.
Memory for embodied agents. Voyager stores reusable skills; BrainMem separates working, episodic, and semantic memory; RoboMemory retrieves procedural experience; and Harness VLA retains execution traces [52, 34, 23, 64]. These systems show that memory supports long-horizon state tracking, similar-task planning, and skill reuse, but retrieval alone does not show that a repeated physical error was prevented [28]. Our architecture therefore separates volatile working state from environment-validated correction episodes and only consolidates a semantic rule after multiple compatible successes. This write policy treats memory as evidence-backed context for correction rather than as authority over the current observation.
Failure Data and Corrective Brain Training
Recent code-based embodied systems improve robot-function invocation, skill composition, and routing to fixed execution backends, while execution-aware frameworks provide trajectory feedback for diagnosis and recovery [16, 13, 40]. However, they provide limited supervision for task-level failure understanding: models are rarely trained to judge whether a subtask succeeded, identify the broken observable prerequisite, and decide whether to advance or recover. We therefore construct controlled failure trajectories from successful demonstrations and convert them into supervision for outcome judgment, cause diagnosis, recovery-subtask selection, and fine-grained planning. Because evaluation and planning are difficult to learn jointly, we use an evaluation-first curriculum and finally apply on-policy distillation (OPD) to internalize verified evaluation into the planning-token distribution.
Dataset and Benchmark Construction

Figure 2. Controlled failure-data construction. Successful trajectories are branched through controlled perturbations and replayed to produce aligned supervision for outcome judgment, cause diagnosis, recovery-subtask selection, and fine-grained planning.
Controlled failure trajectories. We begin with successful RoboCasa365 and OmniBehavior trajectories [36, 24]. Subtask boundaries are identified from task-graph transitions and task-relevant state changes, such as reaching a target, grasping an object, or opening a container. At each boundary, we retain the task goal , current subtask, executed workflow , pre-execution observation , and corresponding successful segment. We then clone the simulator state before the subtask, inject one known perturbation, and execute the same subtask again. Each branch changes only one factor, making its failure source identifiable from the intervention. Perturbations cover navigation, reachability, grasp pose, control timing, articulation, task prerequisites, and target grounding. The replay produces a post-execution observation and execution trace , including function arguments, returns, exceptions, and key visual frames.
Correction-target construction. Judge and Cause are generated from the known perturbation and intervention point. Judge records whether the perturbed execution completes the current subtask. Cause combines the perturbation with its realized consequence and describes the failure using publicly observable evidence. For example, a changed grasp height that leaves the object unattached is written as “the gripper closed above the target and the object was not grasped,” rather than exposing a hidden simulator parameter. For a failed example, the current subtask remains incomplete and is used as Sub. For a successful example, Sub is the next subtask in the original trajectory. Thus, Sub measures whether the model can decide to advance or repeat the interrupted step. Fine specifies how to execute Sub. We replay a corrective execution from the realized failed state to obtain an aligned corrected segment. A fixed closed-source model summarizes the failed and corrected traces into an ordered fine-grained plan while preserving the target object, motion direction, end-effector state, action order, and verification steps. Fine is therefore a language description of the corrected trajectory rather than a template selected from the perturbation type.
Filtering and benchmark construction. We discard ineffective perturbations, unsynchronized media, inconsistent object identities, invalid traces, and failures that cannot be identified from observations. Source trajectories are split before perturbation generation. The corrective training set contains aligned Judge, Cause, Sub, and Fine targets from training trajectories. EmbodiedDiag-Bench is generated only from held-out trajectories and therefore shares neither source trajectories nor successful prefixes with training. We report each field separately, while the joint score measures whether the model can transform execution evidence into a corrected task state and recovery decision.
Evaluation-First Brain Training and OPD Internalization

Figure 3. Overview of Embodied-Corrector training. Training proceeds from execution evaluation to recovery planning, followed by NEPO optimization that reduces the mismatch between verified evaluation prefixes used during training and model-generated prefixes used at deployment.
Embodied-Corrector is a unified multimodal VQA model that autoregressively generates Judge, Cause, Sub, and Fine. An incorrect evaluation prefix can distort the recovery plan, while a response-level reward is too coarse to identify the responsible planning token. We therefore first train evaluation, then recovery planning, and finally use OPD to internalize verified evaluation. Deployment still uses one inference to generate the complete response.
Stage 1: Evaluation-first supervised training. We first train Judge and Cause, requiring the model to determine whether the subtask was completed and explain its observable failure cause. We then train Sub and Fine conditioned on verified Judge and Cause. This curriculum teaches the dependency from execution outcome to cause diagnosis and recovery planning without requiring difficult joint prediction from the beginning.
Stage 2: OPD-based evaluation internalization. Teacher-forced training generates Sub and Fine conditioned on accurate Judge/Cause, whereas deployment uses the model’s own generated Judge/Cause; this prefix distribution mismatch allows evaluation errors to propagate into subsequent planning tokens. We therefore use OPD to internalize the planning distribution under accurate Judge/Cause prefixes into the planning distribution under the model’s own prefixes. For the -th training sample, let denote the input, including the task goal, executed workflow, pre- and post-execution observations, and execution trace; let denote the accurate Judge/Cause, where is the subtask-completion label and is the failure-cause label obtained from controlled perturbation and replay; let denote the model’s self-generated Judge/Cause; let denote the planning token sequence for Sub/Fine, where is the sequence length, is a planning-token position, is a candidate token in the vocabulary, denotes the first generated planning tokens, and is the old-policy snapshot before the update, used to compute both prefix distributions under the same policy for comparability. Using the same old policy, we compute the probability of token at the -th planning position under the two prefixes:
where is the distribution under the model’s self-generated Judge/Cause prefix and is the distribution under the accurate Judge/Cause prefix; their token-level signed advantage is
When , the accurate Judge/Cause increases support for token , so this token should be reinforced; when , the accurate Judge/Cause decreases support for token , so this token should be suppressed. Note that is used only to compute and score planning tokens, and does not replace the model’s self-generated prefix ; at deployment, the accurate branch is removed entirely and no labels are exposed.
Noise-aware asymmetric OPD. Under prefix mismatch, at high-entropy positions is flatter, making more sensitive to prefix noise; negative signals may conflict with the task-level RL reward and destabilize training [45, 67, 33]. We therefore introduce NEPO: for the -th planning position, we first compute the entropy of the self-generated distribution ,
and then define the gate value
where is the sigmoid function, is the entropy threshold, and is the gate temperature. The gate decreases smoothly with entropy, attenuating negative signals more strongly at uncertain positions. Evaluating at the sampled token , the final token-level advantage is
where retains positive signals and retains negative signals; that is, positive advantages are always retained, while negative advantages are attenuated according to entropy. The complete response still receives the task-level VQA/RL reward , and each planning token additionally receives , where controls the relative weight of the OPD term; the two are jointly updated under a GRPO-style objective, and only Embodied-Corrector is updated. At deployment, the accurate Judge/Cause branch is removed, and a single inference generates the complete Judge–Cause–Sub–Fine response.
Corrective Harness
Figure 4 shows how the trained brain enters the execution loop. A fixed closed-source planner combines the goal, current observation, execution history, and retrieved memory to generate workflow code over analytic robot functions or a frozen VLA. After execution, Embodied-Corrector reads the goal, executed workflow, boundary observations, and public trace, then returns Judge, Cause, Sub, and Fine. The planner uses this record to revise the next workflow from the current state while retaining completed progress. Retrieved memory is provided to the planner; corrections enter episodic memory only after environment verification. Appendix § 8 gives the formal interface, memory policy, and detailed workflow.

Figure 4. Architecture and closed-loop workflow of the Embodied-Corrector Harness. Separating execution evaluation, recovery planning, fixed code/VLA realization, and environment-validated memory turns failure evidence into executable correction while preserving prior progress.
Experiments
We first evaluate execution understanding and corrective planning on EmbodiedDiag-Bench, followed by transfer to EB-ALFRED. Harness-level experiments then test two mutually exclusive simulated execution settings: code workflows with the VLA function disabled, and VLA-only workflows with all analytic robot functions disabled. We next analyze post-training dynamics and corrective judgments as subtask rewards, and conclude with physical validation on the Piper platform.
Brain-Level Evaluation
Corrective reasoning on EmbodiedDiag-Bench. We first remove the code and action executors and directly test whether the brain can produce a coherent correction from a completed physical execution. EmbodiedDiag-Bench contains held-out navigation and manipulation failures and evaluates four fields. Judge determines whether the current subtask was completed, Cause identifies the failed entity, relation, and outcome, Sub decides whether to advance or complete the current subtask again, and Fine specifies the required actions, grounded arguments, ordering, and intermediate checks.
Table 1 shows that general VLMs can occasionally distinguish success from failure, but their substantially lower Cause and Fine scores indicate that outcome recognition alone does not yield a usable correction. SFT substantially improves consistency among the four fields, while NEPO provides the most balanced performance across navigation and manipulation. It reaches Judge accuracies of 79.6% and 81.2%, Cause accuracies of 58.2% and 54.6%, and at least 63.7% on all four recovery-planning scores. The model therefore learns a relation between a failure cause and the way the interrupted subtask should be executed.
| Model | Navigation | Operation | ||||||
| Sub | Fine | Judge | Cause | Sub | Fine | Judge | Cause | |
| GPT-5.5 | 72.7 | 39.4 | 52.8 | 39.8 | 65.4 | 41.1 | 57.7 | 43.2 |
| RoboBrain2.5-4B | 26.1 | 21.2 | 31.5 | 12.2 | 29.7 | 24.1 | 33.5 | 12.3 |
| RynnBrain-8B | 23.4 | 19.4 | 23.6 | 13.6 | 31.6 | 25.8 | 31.9 | 18.7 |
| InternVL3-8B | 21.8 | 17.7 | 48.3 | 26.2 | 32.1 | 26.8 | 40.1 | 17.9 |
| MiMo-Embodied-7B | 31.6 | 25.3 | 30.1 | 16.9 | 35.6 | 27.6 | 22.2 | 19.3 |
| Qwen2.5-VL-7B | 23.2 | 19.3 | 52.5 | 23.8 | 28.8 | 26.0 | 33.7 | 13.5 |
| Qwen3-VL-32B | 48.7 | 39.8 | 49.5 | 29.2 | 42.9 | 34.3 | 54.6 | 33.2 |
| RynnBrain-SFT | 68.6 | 66.7 | 75.9 | 53.4 | 62.5 | 61.6 | 80.7 | 51.4 |
| RynnBrain-OPD | 70.0 | 68.0 | 79.0 | 57.0 | 65.4 | 61.0 | 80.0 | 54.0 |
| RynnBrain-GRPO | 73.0 | 69.2 | 74.2 | 51.5 | 63.6 | 62.1 | 76.3 | 49.6 |
| RynnBrain-NEPO | 72.6 | 71.6 | 79.6 | 58.2 | 64.9 | 63.7 | 81.2 | 54.6 |
| - w/o Ent.-Gate | 69.9 | 68.5 | 73.1 | 50.8 | 63.1 | 61.4 | 75.0 | 48.7 |
Table 1. Brain-level corrective reasoning on EmbodiedDiag-Bench (%). Judge and Cause measure whether the brain understands the realized execution, while Sub and Fine measure whether that understanding produces the correct continuation.
Transfer to an external planning benchmark. Because EmbodiedDiag-Bench follows the same output structure as the training data, we additionally evaluate frozen checkpoints on EB-ALFRED (Yang et al., [56]). No explicit failure prompt, code executor, VLA, or corrective Harness is used. All models receive the same instruction, observation, action space, history, and step limit. This experiment tests whether failure-oriented training learns reusable task dependencies rather than only the Judge–Cause–Sub–Fine response format.
As shown in Table 2, Embodied-Corrector improves over the base brain on all six subsets. Relative to SFT, the additional gains of NEPO are most pronounced on Complex, Spatial, and Long tasks, which require stronger prerequisite tracking and action ordering. Controlled failure data therefore improves ordinary embodied planning by teaching the model how task progress depends on previously established conditions.
| Model | Base | Com. | Cplx. | Vis. | Spat. | Long |
| GPT-4o | 64 | 54 | 68 | 46 | 52 | 54 |
| Claude-3.7-Sonnet | 68 | 68 | 70 | 68 | 62 | 70 |
| InternVL3-8B | 20 | 14 | 14 | 12 | 0 | 2 |
| Ovis2-16B | 26 | 16 | 24 | 12 | 16 | 4 |
| Qwen3-VL-32B | 62 | 48 | 46 | 42 | 38 | 46 |
| Gemma-3-12B-IT | 32 | 26 | 38 | 22 | 20 | 12 |
| RoboBrain2.5-4B | 22 | 16 | 34 | 6 | 16 | 24 |
| MiMo-Embodied-7B | 20 | 16 | 24 | 18 | 20 | 18 |
| RynnBrain-8B | 30 | 28 | 44 | 26 | 24 | 28 |
| RynnBrain-SFT | 72 | 48 | 62 | 54 | 56 | 52 |
| RynnBrain-OPD | 74 | 50 | 64 | 52 | 58 | 54 |
| RynnBrain-GRPO | 76 | 54 | 72 | 56 | 60 | 58 |
| RynnBrain-NEPO | 78 | 56 | 74 | 60 | 64 | 62 |
| - w/o Ent.-Gate | 74 | 52 | 68 | 54 | 58 | 56 |
Table 2. Planning transfer to EB-ALFRED (%). Frozen checkpoints receive the same inputs and step limit, without explicit failure prompts or a corrective Harness. Gains on Complex, Spatial, and Long tasks indicate reusable prerequisite tracking and action ordering. Com., Cplx., Vis., and Spat. denote Common, Complex, Visual, and Spatial.
The smaller, less structured gains on appearance-dependent cases show that correction training can reason over visual evidence but cannot replace information absent from the observation.
Harness-Level Evaluation
Brain-level VQA accuracy does not establish that a correction remains useful after workflow generation and physical execution. We therefore evaluate the complete Harness under two mutually exclusive function configurations. The first disables the VLA function and tests correction through analytic robot functions and generated code. The second disables all other robot functions and tests coordination with a frozen VLA. The planning model, memory, execution interface, and budget remain fixed within each comparison.
Code execution without the VLA. We remove from the function library, so the GPT code generator can invoke only analytic functions for object detection, navigation, pose estimation, arm motion, gripper control, and articulation. Each function returns its arguments, result or exception, and available pre- and post-execution observations, combining an ASPIRE-style execution trace with a CaP-X-style functional robot interface [32, 16].
We fix the code model GPT-5.5, function documentation, perception module, tasks, seeds, and execution budget, and replace only the corrective brain. Embodied-Corrector outperforms the untrained brain under all ten conditions. Relative to GPT self-correction, it performs better in Libero-Pro. Because neither code generation nor the robot functions change, the result shows that a Judge–Cause–Sub–Fine response enables the same workflow model to revise the relevant function calls instead of repeating the original failed program.
Frozen-VLA execution. In the complementary setting, we evaluate HarnessVLA augmented with our Embodied-Corrector, while the execution library contains only fVLA. Analytic functions for object detection, navigation, pose control, or gripper control are disabled. Embodied-Corrector produces a state-conditioned subtask, fine-grained action text, and execution-error diagnosis and correction; GPT converts these outputs into HarnessVLA code that invokes a frozen or through vla_execute. The None condition uses the same pipeline without the Corrector. VLA parameters, observations, action interface, and task split are unchanged, so the comparison isolates whether high-level correction and decomposition reduce the execution burden on an existing action model [64, 36].
Figure 5. Frozen-VLA success on RoboCasa365 (%). Corrective planning improves composite tasks, while atomic success decreases.
The effect depends strongly on task structure. Atomic-Seen tasks already provide short instructions close to the frozen VLA’s training distribution. Reformulating them can introduce an unnecessary language interface error, and atomic success consequently decreases for both policies. Composite-Seen and Composite-Unseen tasks instead require the VLA to map a long compositional instruction directly to actions. Embodied-Corrector converts the instruction into short, state-conditioned subtasks and substantially improves both groups. For , Composite-Seen and Composite-Unseen success increases from 7.1% and 1.2% to 12.5% and 7.3%, respectively, producing an overall gain. The result shows that the brain does not replace low-level visuomotor control; it reduces the temporal and compositional complexity that a fixed VLA must solve at once. The atomic degradation identifies brain–VLA language alignment as a remaining system bottleneck.
Post-Training Dynamics
We next examine whether NEPO changes optimization behavior rather than only final benchmark accuracy. GRPO [45], GSPO [66], OPD, and NEPO start from the same evaluation-first SFT checkpoint and use the same multimodal VQA data, number of samples, response budget, seed schedule, and update count. Figure 6 reports policy loss, answer-level VQA reward, and sampled-token entropy.
Figure 6. VQA post-training dynamics from a shared SFT checkpoint. Curves report policy loss, answer-level reward, and sampled-token entropy under matched training settings. NEPO achieves the highest reward while avoiding the low-entropy regime of vanilla OPD.
Policy loss is non-monotonic for all methods and does not by itself explain their final separation. Vanilla OPD provides dense token-level guidance, but low-quality student prefixes create distribution mismatch between the student and privileged branches. The resulting noisy negative probability gaps gradually move OPD toward a lower-entropy regime. NEPO preserves positive privileged guidance while gating negative signals at uncertain positions. It consequently obtains the highest reward while avoiding the entropy contraction of vanilla OPD. The joint reward–entropy pattern shows that asymmetric gating improves the reliability of OPD supervision.
Corrective Brain as a Subtask Reward
We additionally freeze either the base brain or Embodied-Corrector and use its binary subtask-completion judgment as an auxiliary reward for VLA PPO (Schulman et al., [44]). All conditions share the same policy initialization, terminal environment reward, tasks, interaction budget, and optimization setup. This experiment tests whether the learned judgment can shorten delayed credit assignment rather than merely improve VQA classification.
The trained corrective brain primarily benefits Long tasks, whose intermediate dependencies are difficult to localize from terminal reward alone. On the shorter Spatial, Object, and Goal suites, subtask judgment often duplicates information already contained in the terminal reward and provides limited benefit. Inaccurate judgments from the base brain can instead direct policy updates away from eventual task success. A brain reward is therefore useful when it reliably exposes delayed task structure, neutral when it repeats the environment signal, and harmful when the two disagree.
| VLA Brain | Spa. | Obj. | Goal | Long | Avg. |
| None | 66 | 69 | 65 | 42 | 60.5 |
| RynnBrain-8B | 62 | 66 | 62 | 38 | 57.0 |
| RynnBrain-NEPO | 65 | 70 | 64 | 49 | 62.0 |
| None | 86 | 78 | 79 | 56 | 74.8 |
| RynnBrain-8B | 82 | 75 | 75 | 52 | 71.0 |
| RynnBrain-NEPO | 85 | 79 | 77 | 63 | 76.0 |
Table 4. LIBERO success with terminal reward and optional frozen-brain subtask reward (%). Corrective judgments provide intermediate feedback and chiefly improve Long tasks with delayed dependencies. Spa. and Obj. denote Spatial and Object.
Real-Robot Validation
We evaluate a Piper manipulator with an end-effector gripper and an RGB-D camera on five tabletop tasks: placing a Rubik’s cube inside a blue plate; stacking the cube on a blue block; sorting an orange-colored fruit into an orange plate and a blue block into a blue plate, with other objects as distractors; moving the cube to the left of the blue block; and placing both designated fruits inside an open basket.
Each frozen backend is evaluated with and without Embodied-Corrector over 20 episodes per task, yielding 100 episodes per method and 400 overall. Correction uses observations and public execution traces to revise the next VLA instruction from the current physical state, preserving completed progress without resetting the scene. Detailed platform settings, success criteria, correction budgets, and representative execution frames remain in Appendix § 12.
Figure 7 shows improvements on every task for both backends. Macro-average success rises from 45% to 62% for and from 51% to 70% for LingBot-VLA. LingBot-VLA color sorting gains 25 percentage points (45% to 70%); spatial rearrangement remains the hardest task, at 45% and 55% with correction.
Figure 7. Piper task success (%; 20 episodes per task and method). Gray and purple markers show each frozen VLA without and with Embodied-Corrector. Macro averages weight the five tasks equally.
The real-robot gains primarily stem from the correction mechanism rather than from strengthening the underlying VLA: it shifts failure handling from outcome-level retries to boundary-level cause localization and state-conditioned recovery. After each action chunk, the Corrector uses boundary observations and public execution traces to determine whether the current subtask is complete and to identify the observable failure cause; without resetting the scene, it then either advances or re-executes only the interrupted subtask, while rewriting the remaining goal into short, state-conditioned instructions that the frozen executor can reliably realize. This both prevents local execution deviations from propagating along task dependencies and reduces the compositional burden that a long instruction places on the frozen VLA at once.
Figure 8. Initial (top) and result (bottom) observations from the five Piper tasks, ordered as in the text. Result frames show the scene after the gripper releases the manipulated objects. Detailed execution sequences and success criteria are provided in Appendix § 12.
Conclusion and Limitations
We presented Embodied-Corrector, viewing correction not as an auxiliary response to planning failure but as a necessary process for keeping subsequent plans grounded in the physical state. By aligning expected postconditions, observed outcomes, observable failure causes, and recovery decisions, it revises the task state before continuing, converting local deviations into specific recoveries rather than letting them propagate through task dependencies. Real-robot evaluation currently covers five tabletop tasks; broader task and embodiment generalization remain to be evaluated. Observation noise and state uncertainty still complicate outcome judgment and recovery verification; future work will broaden real-world evaluation through uncertainty-aware postcondition assessment and safety-constrained recovery.
Implementation Details
Data Composition
Training uses two sources: our controlled correction data and the Embodied-Reasoner corpus. The controlled data start from 300 RoboCasa365 tasks, containing 6,210 episodes and 2,078,151 raw rounds, and 50 OmniBehavior tasks, containing 328 episodes and 358,608 raw rounds ([36, 24]). We inject controlled perturbations at subtask boundaries and construct Judge, Cause, Sub, and Fine supervision from the resulting executions. After filtering for effective perturbations, complete observations, observable failures, and duplicate trajectories, the corpus contains 1,583,014 valid rounds from 148,415 rollouts. Source trajectories are split before perturbation branches are generated, preventing examples with the same successful prefix from appearing in both the training set and EmbodiedDiag-Bench.
Evaluation-first SFT mixes the controlled correction data with general embodied-planning data from Embodied-Reasoner. The controlled data teach execution judgment, failure-cause analysis, and corrective planning, whereas Embodied-Reasoner preserves general task decomposition, object-relation reasoning, and action-dependency knowledge ([63]).
The second stage also operates on fixed multimodal VQA examples and contains no robot interaction or environment rollout. Its data consist of 28,871 correction responses sampled from the student distribution and 3,003 Embodied-Reasoner planning examples, producing 31,874 records in total. All post-training methods share the same data, initialization, prompts, and response budget; they differ only in their reward objective and in whether privileged evaluation and entropy gating are used.
Unified Harness Evaluation Protocol
Common workflow. All execution experiments use the same “plan–execute functions–correct–revise workflow” loop. A fixed closed-source GPT-5.5 model performs both high-level task planning and workflow generation. It receives the task goal, current observation, within-episode execution history, previous correction, robot-function documentation, and retrieved memory, and produces the next workflow.
Multimodal memory is provided only to GPT-5.5 and is not an input to Embodied-Corrector. Working memory maintains the current observation, completed steps, latest correction, and remaining task. Episodic memory stores environment-validated correction cases, while semantic memory stores cause–repair rules consolidated from repeated successful cases. To keep evaluation trials independent, formal experiments retain only within-episode working memory and disable cross-episode episodic retrieval, memory writing, and semantic-rule updates.
After a robot-function call or VLA action chunk returns, the Harness records pre- and post-execution images, function arguments, return values or exceptions, and observable changes in robot and object state. Following the execution-trace-based correction mechanism of ASPIRE, this multimodal evidence is passed to Embodied-Corrector ([32]). The corrector receives neither retrieved memory nor hidden simulator state. In a single inference, it produces Judge, Cause, Sub, and Fine: Judge determines whether the current subtask was completed, Cause identifies a failure supported by observable evidence, Sub specifies the next subtask, and Fine describes how that subtask should be executed.
The structured correction is then returned to GPT-5.5. After success, GPT-5.5 preserves completed progress and continues the remaining task. After failure, it revises the subsequent workflow according to Cause, Sub, and Fine. GPT-5.5 receives only the structured correction record, not the private reasoning of Embodied-Corrector. OmniBehavior, LIBERO-PRO, and RoboCasa365 execution experiments use simulation and benchmark-defined success predicates. Piper hardware uses the independent evaluator and protocol in Appendix § 12.
Code Harness with the VLA function disabled. In the first configuration, is removed from the function library. GPT-5.5 can compose only analytic robot functions, such as detect_object, navigate_to, estimate_pose, move_to_pose, control_gripper, and open_joint. These functions share a common argument, return, and exception interface and expose multimodal execution traces for correction. This configuration combines a CaP-X-style functional robot interface with ASPIRE-style execution feedback [16, 32].
OmniBehavior separately evaluates navigation and manipulation success for the radio and soda-can tasks, giving four metrics: Radio-Navigation, Radio-Operation, Soda-Navigation, and Soda-Operation. Each task–metric condition is evaluated from 25 distinct random initializations. Each reported cell is therefore the mean binary success over 25 complete trials.
The first three rows of Table 3 (OpenVLA, , and ) are results reported by CaP-X under its original evaluation protocols and were not rerun in our Harness; they are therefore excluded from the 25- and 100-trial settings described above [16].
| Model | OmniBehavior | LIBERO-PRO | ||||||||
| Rad.-Nav | Rad.-Op | Soda-Nav | Soda-Op | Obj.-P | Obj.-T | Goal-P | Goal-T | Spa.-P | Spa.-T | |
| OpenVLA | 4 | 0 | 2 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 8 | 2 | 6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| 20 | 8 | 18 | 6 | 17 | 1 | 38 | 0 | 20 | 1 | |
| GPT 5.5 | 76 | 44 | 80 | 60 | 61 | 58 | 72 | 77 | 58 | 66 |
| RynnBrain | 72 | 36 | 72 | 56 | 48 | 46 | 64 | 62 | 45 | 49 |
| RynnBrain-NEPO | 76 | 40 | 80 | 64 | 69 | 67 | 82 | 85 | 56 | 70 |
Table 3. Code-Harness evaluation on OmniBehavior and LIBERO-PRO (%). The VLA function is disabled in the code workflow. The GPT-5.5 code model, analytic robot functions, perception, and execution budget stay fixed when comparing corrective brains.
For the remaining three rows, all compared conditions use the same GPT-5.5 model, robot-function documentation, perception module, task set, initial-state distribution, and execution budget. Only the correction component is changed among GPT-5.5 self-correction, RynnBrain-8B, and Embodied-Corrector. Cross-task memory writing and skill-library updates are disabled, so any change in success measures whether the correction helps the same planner revise subsequent function calls.
HarnessVLA execution with a frozen VLA as the only available robot function. All conditions use HarnessVLA for robot execution. In this configuration, the function library exposed to the GPT code generator contains only vla_execute, backed by a frozen or . Analytic functions for object detection, navigation, pose control, gripper control, and articulation are disabled.
At fixed action-chunk boundaries, Embodied-Corrector receives the overall task, the previous stage instruction, and the pre- and post-execution observations. In brain-assisted conditions, it generates a complete Judge–Cause–Sub–Fine record in one inference. This record contains subtask decomposition, fine-grained textual action planning, and execution-error diagnosis and correction. These structured outputs are then passed to GPT, which generates executable HarnessVLA code. Under the frozen-VLA-only configuration, the generated code can invoke only vla_execute; in configurations with a larger function library, the same code-generation mechanism can also invoke analytic tools or VLA functions.
The generated code is executed by HarnessVLA. The frozen VLA receives image observations and robot state through the unchanged HarnessVLA interface, together with the language instruction specified by the generated execution call. VLA parameters, observation preprocessing, action space, and task horizon remain unchanged across all conditions.
Evaluation is performed through the same HarnessVLA execution pipeline. It covers 18 Atomic-Seen, 16 Composite-Seen, and 16 Composite-Unseen RoboCasa365 tasks [36]. Each brain-assisted condition runs 50 independent trials per task, yielding 2,500 episodes for each brain–VLA pair: 900 Atomic-Seen, 800 Composite-Seen, and 800 Composite-Unseen episodes.
Detailed protocol for the perturbations
Shared intervention protocol. Each example starts from a retained successful trajectory at the state immediately before a subtask boundary. We clone that state and hold the task instruction, successful prefix, intended object and target, and all non-selected execution factors fixed. We then intervene on exactly one of the seven factors below and replay only the affected subtask. An intervention is retained only if the expected postcondition is violated, the consequence is observable in the synchronized public observation or execution record, and the branch remains recoverable. Ineffective interventions, branches with multiple changed factors, and failures that can be identified only from hidden simulator state are discarded. Recovery always begins from the realized failed state rather than from a reset. Because scene geometry, joint ranges, and skill duration differ across tasks, perturbation validity is defined by the resulting predicate change rather than by one global displacement, angle, or timestep range. This construction makes the factor name a causal intervention label, not a free-form description assigned after a naturally occurring failure.
Navigation. This intervention changes the executed navigation goal, the desired terminal heading, or both, while leaving the semantic destination and manipulation request unchanged. It is applied at a navigation boundary before the affected navigation skill is replayed. The resulting stop can lie outside the interaction region or face the workspace from an incorrect approach direction, so the target-region or pose postcondition fails even though locomotion itself may finish normally. A matched correction estimates a new reachable goal and heading from the current observation, navigates there, verifies that the interaction region is in range, and then resumes the blocked manipulation. This factor tests whether the brain can distinguish successful motion from task-relevant arrival and repair a geometric navigation error instead of repeating the same command.
Reachability. Reachability keeps the intended destination fixed but makes the original waypoint or terminal stop pose unusable. The intervention either blocks that pose or invalidates it under the planner’s collision and kinematic feasibility checks. Unlike Navigation, which perturbs where or how the agent is asked to arrive, Reachability perturbs whether the nominal route or stop configuration can be executed. The observed consequence is a no-path result, an unreachable target, or a stop from which the next end-effector goal cannot be reached. Recovery resamples a collision-free intermediate waypoint or stop pose, verifies planning feasibility, and returns to the original subtask. The diagnostic requirement is to propose an alternative feasible route or base configuration rather than misclassifying the failure as object absence or a grasp error.
Grasp pose. This intervention changes the end-effector approach direction, contact height, or grasp orientation for the specified object while preserving object identity and the requested downstream task. The perturbed grasp can miss the object, contact an unstable region, or initially contact and then slip. Offline construction uses pose, contact, and attachment signals to confirm the label, while the deployed brain must infer the failure from public images and the execution record. The correction re-localizes the object, recomputes a reachable pre-grasp and grasp pose, approaches without collision, closes the gripper, and verifies attachment before lifting or transporting. This factor evaluates spatial diagnosis at the object–gripper interface and whether the recovery restores a stable grasp rather than merely issuing another close command.
Timing. Timing changes when the gripper open or close command occurs relative to approach, contact, lift, transport, or release. The geometric target and grasp pose remain fixed, isolating temporal coordination from spatial accuracy. An early close can prevent the gripper from enclosing the object; a late close can begin after contact has been lost; an early open can release during transport; and a late open can leave placement incomplete. The common observable outcomes are an unsecured object or a drop. Recovery first restores the relevant pre-contact or pre-release relation, then issues the gripper command at the correct phase and verifies attachment or release before continuing. This factor tests whether the brain attributes failure to action order and synchronization rather than unnecessarily changing the object, goal, or grasp geometry.
Container. The Container intervention changes the articulation state of a receptacle such as a drawer, cabinet, or door by leaving it only partially open or partially closed at the subtask boundary. The container identity and requested content operation remain unchanged, but the opening, clearance, or required joint-state predicate is absent. The next reach, placement, retrieval, or closure can therefore be infeasible even if its own command is well formed. A matched correction re-localizes the articulated part, completes the required open or close motion past the task-defined postcondition, verifies the resulting space or state, and resumes the blocked content operation. This factor tests whether the brain recognizes articulation as a physical precondition rather than repeatedly repairing the downstream grasp or placement.
Prerequisite. This intervention removes one dependency from the executed task prefix while keeping the blocked subtask request unchanged. Typical dependencies include reaching the workspace before manipulation, opening a container before insertion or retrieval, holding an object before transport, or clearing a destination before placement. The requested subtask may be syntactically valid but is infeasible in the realized state because its first required predicate is false. Recovery identifies the earliest unmet dependency, executes and verifies it, and then returns to the original blocked subtask without discarding the successful prefix. This factor directly measures causal task-graph reasoning: the correct response must repair an earlier condition, not paraphrase or locally retry the final command.
Target. Target intervention changes the grounded same-class instance or the executable reference while preserving the task instruction and skill type. For example, a command intended for one bowl can be bound to another visible bowl, so a technically successful grasp or placement still changes the wrong entity. The failure is accepted only when instance identity and the task relation disagree and the mismatch is observable in the retained evidence. Recovery resolves the instruction against the current scene, re-grounds the specified instance and destination, undoes or safely terminates the wrong-object branch when necessary, and replans the blocked operation. This factor tests reference resolution and instance-level grounding, separating semantic target errors from failures of navigation, reachability, or control.
Corrective Harness
Problem formulation. Given a long-horizon task goal , at execution round , a fixed closed-source GPT model receives the current observation , execution history , retrieved memory , the previous correction record , and the robot-function documentation . It outputs an executable workflow . After the workflow is executed, the Harness returns the new observation and execution trace . Embodied-Corrector then produces a structured correction record from the resulting execution evidence:
Here, denotes workflow code composed of robot-function calls, and describes callable functions for object detection, navigation, pose control, gripper control, articulation, and a frozen VLA. The outputs , , and denote the subtask completion judgment, the failure cause supported by execution evidence, and the recovery proposal from the current state, respectively. Retrieved memory is provided only to the GPT model, whereas Embodied-Corrector reasons only from the task, the executed workflow, the pre- and post-execution observations, and the returned trace. The GPT model then conditions on in the next round to continue or revise the workflow while preserving valid prior progress.
Workflow. As illustrated in Figure 4, the closed-source planning model first retrieves task-relevant context from the multimodal memory shown on the left. Working memory maintains the current task state, the latest execution outcome, and the unfinished subtask. Episodic memory stores environment-validated successful corrections, such as a case in which a skipped grasp must be inserted before transport. Semantic memory consolidates reusable task rules, such as requiring object attachment before transport and verifying that attachment before proceeding. These memories are provided only to the closed-source planning and workflow-code model, not to Embodied-Corrector. When retrieved experience conflicts with the current observation, the current physical evidence takes precedence.
The closed-source planning model combines the task goal, current state, and retrieved memory to directly generate a workflow over robot functions. Following the function-based robot interface of CaP-X, the function set may include detect_object, navigate_to, approach, close_gripper, lift, open_joint, and verify_attachment ([16]). Following Harness VLA, a frozen VLA is exposed through the same function library as vla_execute(instruction, observation, robot_state), which realizes local contact-rich skills such as grasping, placing, and articulation ([64]). The VLA is therefore not a separate execution system outside the workflow. It is one robot function, alongside navigation, pose control, gripper control, and other analytic functions.
After a robot function or VLA function is executed, the fixed execution Harness returns the pre- and post-execution observations, function arguments, return value or exception, and observable changes in robot and object state. Following the execution-trace-based correction mechanism of ASPIRE, this multimodal evidence is passed to Embodied-Corrector ([32]). The corrector neither retrieves memory nor directly generates executable workflow code. Instead, it determines whether the requested subtask was actually completed, identifies a failure cause supported by the observations, and proposes the task condition or recovery steps that must be restored. In the LIBERO-PRO orange-juice example in the figure, the failed program attempts placement with an empty gripper while the carton remains on the floor. The corrector therefore marks placement as incomplete, identifies the omitted pickup prerequisite, and proposes locating and grasping the carton, verifying attachment, and placing it in the basket. The lower strip shows frames from the failed rollout followed by frames from the regenerated rollout, rather than one uninterrupted trajectory.
The correction record is then returned to the same closed-source planning model. After success, the planner preserves completed progress and continues the remaining workflow. After failure, it combines the recovery proposal with retrieved memory to regenerate the required function calls—for example, target localization, approach, gripper closure, lifting, and attachment verification before resuming placement. The new physical outcome is evaluated again, forming a loop of “memory retrieval–workflow generation–function execution–execution correction–workflow revision.” Working memory is updated after every round. A correction enters episodic memory only after the environment confirms that it restored task progress, and repeated compatible successes may later be consolidated into semantic rules. During formal evaluation, the closed-source planner, robot functions, frozen VLA, perception interface, and execution budget are held fixed, so the experiment measures whether Embodied-Corrector can restore task progress by changing the subsequent workflow.
Public Robot Tool Interfaces for Closed-Loop Correction
Overview
At the beginning of each subtask, the code-generation model receives a complete list of tool interfaces, including function signatures, parameter meanings, return values, coordinate frame conventions, and major failure conditions. The model is not allowed to send undefined robot control commands directly; instead, it generates Python programs composed solely of these public tools. This design decouples high-level reasoning from low-level execution: the “brain” is responsible for determining whether a subtask is completed, analyzing failure reasons, and generating fine-grained recovery plans; the code model translates recovery plans into calls for perception, geometric computation, grasp planning, and motion control.
The tool interfaces also serve as the adaptation boundary between simulation and real hardware. The simulation and real-robot backends may use different camera drivers, segmentation services, inverse kinematics solvers, and arm controllers, but they must preserve identical tool names, parameter semantics, coordinate frames, and return structures. Currently, the LIBERO/ASPIRE backend provides 24 public functions, among which point_prompt_molmo is registered only when the Molmo service is available. The code-generation model may define auxiliary functions local to the task, but it cannot bypass these interfaces to access private simulator states.
Environment success determination is not part of the tool interfaces. The generated program can only read public observations and robot states; it cannot access rewards, task-completion flags, ground-truth object poses, or internal contact states of the simulator. After the program terminates, an independent environment evaluator computes success or failure based on task predicates and returns the result to the Harness. Therefore, the model’s own completion declarations do not directly affect the experimental results.
Public Tool Interface Summary
The following public tools are available to the code-generation model. The interface covers visual perception, 3D geometry, grasp planning, and robot control. All state-changing operations must be executed through the motion-control tools, while environment rewards and task-completion predicates remain hidden from the generated program.
Observation
get_observation()– Returns a multimodal observation dictionary containing RGB, depth, intrinsics, and camera-to-world transforms from the main and wrist cameras, along with robot joint angles, end-effector pose, and gripper state. This is the sole entry point for obtaining the current environment state.
Vision
segment_sam3_text_prompt(rgb, text_prompt)– Returns a list of mask candidates (each with binary mask, bounding box, and confidence) for the text-described instance; returns an empty list if no target is detected.segment_sam3_point_prompt(rgb, point_coords)– Segments the instance at the given pixel coordinate, useful when text-based segmentation is unstable or disambiguation is needed.point_prompt_molmo(image, text_prompt)– Returns the pixel coordinates of a text-described target using a vision-language model; registered only when the Molmo service is available.
Grasp Planning
plan_grasp(depth, intrinsics, segmentation) – Generates grasp candidates (transform matrix and confidence) from a depth image and target mask using Contact-GraspNet. Returned poses are in the camera frame and must be transformed to the world frame; reports failure if no candidate.
plan_grasp_from_point_clouds(pc_full, pc_segment) – Generates grasp candidates from the full scene point cloud and target object point cloud, suitable for multi-view fusion.
select_top_down_grasp(grasps, scores, cam_to_world, vertical_threshold=0.8) – Transforms candidates to the world frame, filters for near-vertical grasps, and returns the highest-scoring valid candidate.
Geometry
depth_to_point_cloud(depth, intrinsics) – Back-projects the entire depth map into a point cloud in the camera frame using the pinhole model.
mask_to_world_points(mask, depth, intrinsics, extrinsics) – Back-projects valid depth pixels within the mask to 3D and transforms them to the world frame.
pixel_to_world_point(u, v, z, intrinsics, extrinsics) – Converts a single pixel and its depth to a 3D world point, useful for turning vision model pointing into robot targets.
get_oriented_bounding_box_from_3d_points(points) – Computes the oriented bounding box (center, extent, rotation matrix) of the target point cloud.
interpolate_segment(p1, p2, step=0.03) – Generates evenly spaced waypoints between two endpoints, used for staged approach, lifting, transport, or pushing.
normalize_vector(v) – Normalises a direction vector; near-zero vectors are left unchanged to avoid numerical issues.
Point Cloud Processing
subsample_point_cloud(pc, max_points=10000) – Downsamples the point cloud to reduce time and memory overhead.
filter_noise(points, colors=None) – Removes outliers using density-based clustering; preserves colour correspondence if provided.
Transformations
transform_points(points, transform_matrix) – Applies a homogeneous transform to a point set or organised point cloud.
rotation_matrix_to_quaternion(R) – Converts a rotation matrix to a unit wxyz quaternion.
decompose_transform(T) – Decomposes a homogeneous grasp transform into position and wxyz quaternion for control interfaces.
Motion Planning and Control
solve_ik(position, quaternion_wxyz) – Solves inverse kinematics for the Franka arm (7-DOF). Falls back to predefined poses if the target is unreachable; reports error if all fail.
move_to_joints(joints) – Blocking move to the specified joint configuration; returns after motion completes.
goto_pose(position, quaternion_wxyz, z_approach=0.0) – Wraps IK and joint motion. When z_approach is set, moves to a point above the target then descends.
State Recovery
goto_home_joint_position() – Returns the arm to a predefined home configuration. Should not be called while carrying an object.
Gripper Control
open_gripper() – Fully opens the gripper.
close_gripper()– Fully closes the gripper. Closing alone does not imply successful grasping; subsequent observations must verify stable lifting.
Typical Tool Calling Workflow
For a pick-and-place subtask, the generated program first calls get_observation to acquire images and camera parameters, then uses text or point prompts to segment the target object. The target mask is converted to a world-frame point cloud via mask_to_world_points, which is used either to estimate the object centre or to generate grasp candidates. Subsequently, the program calls select_top_down_grasp, decompose_transform, and solve_ik to obtain an executable pose, then executes the grasp through open_gripper, goto_pose, close_gripper, and a lifting motion. After placement, the program calls get_observation again to acquire evidence of the outcome. Only when the external environment evaluator confirms that the task predicate holds does the Harness mark the subtask as successful.
Tool Call Logging and Error Correction
Every tool call is written into a structured trajectory that includes at least the call order, function name, parameter summary, execution time, result summary, and corresponding visual keyframes. The correction model receives the task instruction, the failed program, the public observations, and this execution trajectory, and outputs four structured fields: whether the subtask succeeded, the observable failure reason, the next subtask, and a fine-grained recovery plan. The code model then generates a revised program based on this record. In this way, the correction results reported in the paper are grounded in verifiable environmental evidence, rather than in the model’s subjective judgment of its own program success.
Correction Cases
The following examples show how our Embodied-Corrector resolves representative execution failures in LIBERO-PRO and RoboCasa365.
LIBERO-PRO: correction guides program regeneration
L1, Prerequisite: pick up the orange juice before placement. Task. Pick the orange juice and place it in the basket. Failed execution. The generated program starts from placement even though the orange-juice carton has not been grasped. The empty gripper moves to the basket and opens, so neither the object state nor the task predicate changes. Correction. Embodied-Corrector marks the current subtask as unsuccessful, identifies the omitted pickup as the first causal error, selects “localize and grasp the orange juice” as the next subtask, and specifies grasp, attachment verification, lift, and basket placement in the fine-grained plan. The code generator uses this record to restore the missing dependency instead of refining the already invalid placement motion. Correct execution. The regenerated program localizes and grasps the carton, verifies that it moves with the gripper, transports it over the basket opening, and releases it inside. The paired rollout figure contrasts the empty placement attempt with the corrected placement and the environment-verified success state.
L2, Target: resolve the requested bowl before control. Task. Pick the black bowl that is not between the plate and the ramekin and place it on the plate. Failed execution. The generated program admits a dark non-bowl region into its candidate set and plans around that false referent. Although the arm executes plausible motions, acting on the wrong instance cannot satisfy the requested spatial relation. Correction. Embodied-Corrector attributes the failure to target selection, retains the placement objective, and asks the code generator to ground the plate, ramekin, and true bowl instances before applying “not between.” The fine-grained plan explicitly rejects non-bowl masks and the bowl lying on the plate–ramekin relation. Correct execution. The corrected program selects the other black bowl, grasps and lifts it, clears the nearby ramekin, and releases it over the plate center. The final spatial relation is accepted by the environment.

Figure 9. Prerequisite correction in LIBERO-PRO case L1. Twelve raw observations contrast an empty-gripper placement attempt with recovery of the missing pickup step. The corrected program grasps the orange-juice carton, verifies attachment, and places it in the basket.
RoboCasa365: correction revises the instruction to a frozen VLA
In RoboCasa365, control returns to Embodied-Corrector after each VLA action chunk, while the frozen continues to generate low-level actions. Unlike the paired LIBERO-PRO rollouts, each RoboCasa365 example is one continuous rollout. Each figure shows twelve chronological observations spanning unsuccessful grasps, corrective instructions, and the terminal successful state. These are raw observations from the same rollout; complete videos provide the temporal evidence.
R1, High and offset grasp: align the avocado. Task. Pick the avocado from the sink and place it on the plate located on the counter. Failed execution. The rollout reaches over the sink but repeatedly closes without enclosing the avocado. The object remains in the basin, exposing coupled errors in descent and lateral alignment. Correction. Embodied-Corrector uses the unsuccessful status, boundary observations, and the history of empty grasp attempts to change the next subtask to side approach and output a directional alignment instruction. As the view changes, the corrector requests small left or right adjustments and only then restores the close-and-lift action. This is an observation-conditioned correction while the VLA remains frozen. Correct execution. The corrected instruction brings the gripper into the basin, restores lateral alignment, and leads to the avocado leaving the sink and satisfying the environment’s object–plate predicate.
R2, Offset and late grasp: reposition before contacting the onion. Task. Pick the onion from the sink and place it on the plate located on the counter. Failed execution. The first pickup instruction contains directional motion, but the contact timing is wrong: the gripper closes too late, knocks the onion, and leaves it in the sink. Repeating closure without changing the approach does not recover the object. Correction. The corrector receives the directional plan, boundary observations, and the earlier action history showing the unsuccessful grasp. It keeps the pickup subtask but makes recovery executable by coordinating forward, lateral, and downward repositioning before closure. Later updates continue to adapt the approach as the view changes, rather than repeating an unchanged close command. Correct execution. The corrected approach encloses the onion, carries it out of the sink, and satisfies the requested object–plate relation. RoboCasa365 verifies successful termination from its public task predicate.

Figure 10. Target correction in LIBERO-PRO case L2. Twelve raw observations contrast false-target execution with re-grounding of the requested black bowl. The corrected program excludes non-bowl regions, applies the spatial constraint, and verifies placement on the plate.
Across these cases, successful correction depends on replacing the failed execution premise: restore a prerequisite, re-ground the target, or rebuild a stable grasp by changing approach height, lateral alignment, and contact timing. Embodied-Corrector supplies this structured change, while the downstream code generator or frozen VLA executes it and the environment verifies the final result.

Figure 11. High and offset grasp correction in RoboCasa365 case R1. Twelve raw observations from one continuous rollout show failed avocado grasps followed by directional re-alignment. With the VLA frozen, corrected instructions restore pickup and successful placement on the plate.

Figure 12. Offset and late-contact correction in RoboCasa365 case R2. Twelve raw observations from one continuous rollout show the onion left in the sink after late closure. Corrected instructions coordinate re-approach and closure before successful plate placement with the frozen VLA.
Additional Evaluation Protocols
EmbodiedDiag-Bench
EmbodiedDiag-Bench evaluates whether a model can understand failures from execution evidence and generate a recovery plan. It directly evaluates four structured fields and does not require physical robot control.
Each input contains the task goal, current subtask, execution history, pre- and post-execution observations, action parameters, function returns, and exception information. The model generates Judge, Cause, Sub, and Fine in that order. Judge is scored with binary accuracy. Cause identifies the failed entity, physical relation, and observable outcome; correctness requires the predicted components to agree with the reference. Sub identifies the recovery direction and target subtask, including continuation, repetition, recovery of an earlier prerequisite, or termination. Fine is matched by executable postconditions, necessary actions, target parameters, action order, and result verification rather than exact string match.
We score the four fields independently; joint correctness requires all fields to be correct. Open-ended Fine outputs that cannot be judged automatically are evaluated by two independent annotators using a shared rubric. The test set is generated only from held-out source trajectories; the train/test split is completed before perturbation generation, so successful execution prefixes are not shared with training.
EmbodiedBench/EB-ALFRED
EmbodiedBench/EB-ALFRED evaluates transfer to ordinary embodied planning. It provides no explicit failure prompt and does not use an online correction loop, a corrective Harness, a VLA backend, or an additional code executor.
All models receive the same task instruction, visual observations, action space, execution history, maximum execution steps, decoding settings, maximum response length, and random seeds. Inference starts from frozen checkpoints without test-time parameter updates or human feedback.
We report success rates on Base, Common, Complex, Visual, Spatial, and Long. The benchmark environment predicate determines episode success. We first average within each task and then macro-average across tasks; Table 2 reports the resulting success rates. This experiment measures ordinary planning and tests whether controlled failure data improves prerequisite tracking, action ordering, and task dependencies.
Piper Real-Robot Experiments
We evaluate Embodied-Corrector on five Piper tabletop tasks using two frozen VLA backends, ([41]) and LingBot-VLA ([54]), each with and without correction.
Real-Robot Platform
The platform comprises a Piper manipulator, an end-effector gripper, and an RGB-D camera. The frozen VLA generates an action chunk from the instruction and current visual observation. For each executed chunk, the system records pre- and post-execution RGB-D observations, robot state, the active instruction, execution outcome, exceptions, and execution time.
Embodied-Corrector receives the goal, current subtask, execution history, boundary observations, and public execution trace. Its four output fields are Judge, which assesses subtask completion; Cause, which describes observable outcomes and failure evidence; Sub, which specifies the next subtask; and Fine, which provides recovery actions and verification steps. The correction record becomes the next VLA instruction, preserving completed progress.
The corrector cannot access hidden environment states, task rewards, private contact states, or human-provided success labels. VLA parameters remain fixed throughout evaluation.
Tasks and Success Criteria
The tasks cover target selection, stacking, color sorting, spatial rearrangement, and sequential container placement. Figure 13 shows the actual objects and executions.
Figure 13. Representative Piper executions for the five real-robot tasks. Each row shows four chronological frames from one video, read from left to right. Action labels describe visible execution stages; timestamps are relative to the start of the corresponding video. The color-sorting task targets the orange-colored fruit and blue block, and the basket task requires both designated fruits. Frames illustrate physical execution; aggregate success rates are reported in Figure 7.
Place Rubik’s cube on the plate. The robot selects the Rubik’s cube among the tabletop objects and places it inside the blue plate. Success requires the released cube to rest within the plate, rather than in the gripper or outside the plate.
Stack Rubik’s cube on the blue block. The robot places the Rubik’s cube on the blue block. Success requires the stack to remain stable after release throughout the evaluation observation period.
Sort the two target objects by color. The robot places the orange-colored fruit in the orange plate and the blue block in the blue plate; other objects are distractors. Both targets must be released inside their matching plates, without omissions or color mismatches.
Move Rubik’s cube to the left of the blue block. Let and denote the tabletop centers of the Rubik’s cube and blue block, with increasing from left to right in the evaluation frame. Success requires
Both distance bounds are strict.
Put both designated fruits in the open basket. The robot places the orange-colored fruit and red fruit into the open basket. Both must lie fully inside after release; an object held by the gripper, on the rim, or outside does not satisfy the criterion.
Evaluation Protocol
Each method runs independent episodes per task: 100 per method and 400 overall. Each episode starts from a predefined physical arrangement and an initial VLA action chunk.
With Embodied-Corrector, the system reacquires RGB-D observations and robot state after each chunk and assesses subtask completion. Completed subtasks advance while preserving progress; incomplete subtasks trigger recovery instructions. Recovery continues from the actual robot and object configuration, without resetting the scene within an episode.
Each episode allows at most correction attempts, counted as recovery instructions for incomplete subtasks. Judgments that advance completed subtasks do not consume this allowance. Success requires meeting the task criterion within the allowance; exhaustion without completion is failure. Collision risk, workspace violations, risk of object damage, or a manual safety stop immediately terminate the episode as unsuccessful.
Metrics and Results
An independent task evaluator determines final success using Sec. G.2; model-generated “success” text does not determine outcomes. Task success rates use 20 episodes each, and the macro average weights the five tasks equally.
The protocol additionally records first-pass success (completion without correction), correction attempts, total execution time, VLA action chunks, corrector invocations, correction latency, and safety terminations. Invocations include assessments of completed subtasks and therefore differ from correction attempts. Figure 7 reports final success rates.
Embodied-Corrector improves all five tasks for both backends. Macro success rises from 45% to 62% for and from 51% to 70% for LingBot-VLA: gains of 17 and 19 percentage points. The largest task-specific gain is LingBot-VLA color sorting, from 45% to 70%. Spatial rearrangement remains the hardest task for both augmented systems (45% and 55%), indicating that precise placement remains challenging.
References
- [1]Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, and 1 others. 2022a. Do as i can, not as i say: Grounding language in robotic affordances. In Conference on Robot Learning (CoRL).
- [2]Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, and 1 others. 2022b. Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691.arxiv.org/abs/2204.01691
- [3]Anonymous. 2026a. Embodied failure immunity: Turning robot failures into transferable antigens. Zenodo preprint.
- [4]Anonymous. 2026b. From world action models to embodied brains: A roadmap for open-world physical intelligence. Preprint, arXiv:2607.11689.
- [6]Anonymous. 2026d. Unleashing vla potentials in autonomous driving via explicit learning from failures. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
- [7]BAAI RoboBrain Team. 2026. Robobrain 2.0: See better. think harder. do smarter.
- [8]Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, and 1 others. 2023a. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL).arxiv.org/abs/2307.15818
- [9]Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, and 1 others. 2023b. Rt-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS).arxiv.org/abs/2212.06817
- [10]Fu Chen, Xin Ding, Bingjia Huang, Xiangyu Li, Mingju Wang, Jiawei He, Kun Li, Wei Sun, Yunxin Liu, Hao Wu, and Ting Cao. 2026. Zeva: In-context causal learning for generalizable embodied manipulation. In arXiv preprint.arxiv.org/abs/2608.30880
- [11]Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, Minghao Zhu, Xiao Lin, Yang Bai, Qian Jiang, Yaxi Zhao, Minghua Zeng, Junlong Gao, Yuming Jiang, Jun Cen, and 7 others. 2026a. Rynnbrain: Open embodied foundation models. Preprint, arXiv:2602.14979.arxiv.org/abs/2602.14979
- [12]Ronghao Dang, Jiayan Guo, Bohan Hou, Sicong Leng, Kehan Li, Xin Li, Jiangpin Liu, Yunxuan Mao, Zhikai Wang, Yuqian Yuan, and 1 others. 2026b. RynnBrain: Open embodied foundation models. arXiv preprint arXiv:2602.14979.arxiv.org/abs/2602.14979
- [13]Xin Ding, Liang Mi, Mingzhe Huang, Zixuan Wang, Chao Zhang, Zixu Hao, Fu Chen, Xiangyu Li, Yikai Zheng, Yaoyu Guo, Weijun Wang, Kun Li, Hao Wu, Yunxin Liu, and Ting Cao. 2026. Zetta: An efficient closed-loop embodied harness for self-evolving physical intelligence. arXiv preprint arXiv:2608.16590.arxiv.org/abs/2608.16590
- [14]Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Thompson, Quan Vuong, Tianhe Yu, and 1 others. 2023. Palm-e: An embodied multimodal language model. In International Conference on Machine Learning (ICML).arxiv.org/abs/2303.03378
- [16]Letian Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, Guanzhi Wang, Dantong Niu, Fei-Fei Li, Guanya Shi, Jiajun Wu, Shankar Sastry, Yuke Zhu, Ken Goldberg, and Linxi “Jim” Fan. 2026. CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. arXiv preprint arXiv:2603.22435.arxiv.org/abs/2603.22435
- [17]Liane Galanti, Dhruv Shah, and Tri Dao. 2026. Addressing the orchestration gap in generalist robots via physical agency. arXiv preprint arXiv:2607.21725.arxiv.org/abs/2607.21725
- [18]Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, and 1 others. 2024. Octo: An open-source generalist robot policy. In Robotics: Science and Systems (RSS).arxiv.org/abs/2405.12213
- [19]Xintong Hu, Xuhong Huang, Jinyu Zhang, Yutong Yao, Yuchong Sun, Qiuye Wang, Mingsheng Li, Sicheng Xie, Yitao Liu, Junho Chen, and 1 others. 2026. FineVLA: Fine-grained instruction alignment for steerable vision-language-action policies. arXiv preprint arXiv:2605.27284.arxiv.org/abs/2605.27284
- [20]Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Thompson, Igor Mordatch, Yevgen Chebotar, and 1 others. 2022a. Inner monologue: Embodied reasoning through planning with language models. In Conference on Robot Learning (CoRL).arxiv.org/abs/2207.05608
- [21]Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, and 1 others. 2022b. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608.arxiv.org/abs/2207.05608
- [22]Zitai Huang, Taiyi Su, Jian Zhu, Jianjun Zhang, Chong Ma, Tianbin Liu, Weiyi Lu, Yi Xu, and Hanli Wang. 2026. MaskHarness-WAM: Instance-grounded harnessing for long-horizon robot manipulation. arXiv preprint arXiv:2609.19974.arxiv.org/abs/2609.19974
- [23]Mingcong Lei, Honghao Cai, Yuyuan Yang, Yimou Wu, Jinke Ren, Zezhou Cui, Liangchen Tan, Junkun Hong, Gehan Hu, Shuangyu Zhu, and 1 others. 2025. RoboMemory: A brain-inspired multi-memory agentic framework for interactive environmental learning in physical embodied systems. arXiv preprint arXiv:2508.01415.arxiv.org/abs/2508.01415
- [24]Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín-Martín, Chen Wang, Gabrial Levine, Wensi Ai, Benjamin Martinez, and 1 others. 2024. BEHAVIOR-1K: A human-centered, embodied AI benchmark with 1,000 everyday activities and realistic simulation. arXiv preprint arXiv:2403.09227.arxiv.org/abs/2403.09227
- [25]Haotian Liang, Mingkang Chen, Yufei Huang, Yuchun Guo, Xiaomeng Zhu, Xiangli Shi, Kaixuan Wang, Yunxuan Mao, Weijie Zhou, Ling Chen, and 1 others. 2026a. RexBrain: Embodied cognition foundation model with joint language-visual reasoning and imagination. arXiv preprint arXiv:2607.14187.arxiv.org/abs/2607.14187
- [26]Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. 2023a. Code as policies: Language model programs for embodied control. In IEEE International Conference on Robotics and Automation (ICRA).arxiv.org/abs/2209.07753
- [27]Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, and 1 others. 2023b. Code as policies: Language model programs for embodied control. In IEEE International Conference on Robotics and Automation.arxiv.org/abs/2209.07753
- [28]Lizhou Liang, Xinyu Zhong, Miao Pan, Xiaohe Zhou, Xuanyu Liu, Qinfeng Li, Peng Li, Jintao Chen, Xuhong Zhang, and Wenqi Zhang. 2026b. Embodiedmemory-bench: Benchmarking embodied memory for long-horizon embodied tasks. arXiv preprint arXiv:2609.28236.arxiv.org/abs/2609.28236
- [29]Huihan Liu, Shivin Dass, Roberto Martín-Martín, and Yuke Zhu. 2023. Model-based runtime monitoring with interactive imitation learning. arXiv preprint arXiv:2310.17552.arxiv.org/abs/2310.17552
- [30]Xuqi Liu, Minghe Gao, Juncheng Li, and Siliang Tang. 2026a. Neurvla: Unleashing failure-handling capability of vision-language-action models via neural-symbolic reasoning. In International Conference on Machine Learning (ICML).
- [31]Yang Liu, Weixing Chen, Xinshuai Song, Tao Pu, Siwen Mo, Yongjie Bai, Zihao Chen, Qianran Sun, Liruo Zhong, Ying Shen, and Liang Lin. 2026b. PhyAgentOS: A self-evolving operating system for embodied agents with decoupled cognitive planning and physical execution. arXiv preprint arXiv:2607.16636.arxiv.org/abs/2607.16636
- [32]Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, Ken Goldberg, Ang Chen, Mosharaf Chowdhury, Yuke Zhu, Linxi “Jim” Fan, and Guanzhi Wang. 2026a. ASPIRE: Agentic /skills discovery for robotics. arXiv preprint arXiv:2607.00272.arxiv.org/abs/2607.00272
- [33]Zhengxi Lu, Zhiyuan Yao, Zhuowen Han, Zi-Han Wang, Jinyang Wu, Qi Gu, Xunliang Cai, Weiming Lu, Jun Xiao, Yueting Zhuang, and Yongliang Shen. 2026b. Self-distilled agentic reinforcement learning. arXiv preprint arXiv:2605.15155.arxiv.org/abs/2605.15155
- [34]Xiaoyu Ma, Lianyu Hu, Wenbing Tang, Zixuan Hu, Zeqin Liao, Zhizhen Wu, and Yang Liu. 2026. BrainMem: Brain-inspired evolving memory for embodied agent task planning. arXiv preprint arXiv:2604.16331.arxiv.org/abs/2604.16331
- [35]Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, and 1 others. 2023. Self-Refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, volume 36.arxiv.org/abs/2303.17651
- [36]Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddikuri, and Yuke Zhu. 2026. RoboCasa365: A large-scale simulation framework for training and benchmarking generalist robots. arXiv preprint arXiv:2603.04356.arxiv.org/abs/2603.04356
- [37]Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddikuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Aaron Pooley, Agrim Gupta, Ajay Mandlekar, and 1 others. 2024. Open ex-embodiment: Robotic learning datasets and rt-x models. In IEEE International Conference on Robotics and Automation (ICRA).arxiv.org/abs/2310.08864
- [38]OpenGVLab. 2025. InternVL3: Advancing the frontier of native multimodal pre-training.
- [39]Miao Pan, Wangjie Gan, Jintao Chen, Wenqi Zhang, Bing Sun, Jianwei Yin, and Xuhong Zhang. 2026a. Ground what you see: Hallucination-resistant mllms via caption feedback, diversity-aware sampling, and conflict regularization. arXiv preprint arXiv:2601.06224.arxiv.org/abs/2601.06224
- [40]Yi Pan, Miao Pan, Qi Lu, Jiaming Huang, Man Zhang, Siteng Huang, Xin Li, Jie Zhang, Yongliang Shen, Xuhong Zhang, and Wenqi Zhang. 2026b. VLA-Corrector: Lightweight detect-and-correct inference for adaptive action horizon. arXiv preprint arXiv:2607.01804.arxiv.org/abs/2607.01804
- [41]Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, and 1 others. 2025. π0.5: A vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054.arxiv.org/abs/2504.16054
- [43]Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, and 1 others. 2022. A generalist agent. Transactions on Machine Learning Research (TMLR).arxiv.org/abs/2205.06175
- [44]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.arxiv.org/abs/1707.06347
- [45]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.arxiv.org/abs/2402.03300
- [46]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023a. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS).arxiv.org/abs/2303.11366
- [47]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023b. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems.arxiv.org/abs/2303.11366
- [48]Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2023. Promptgpt: Generating situated robot task plans using large language models. In IEEE International Conference on Robotics and Automation (ICRA).arxiv.org/abs/2209.11302
- [50]Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lee. 2025. RicI: Adding in-context adaptability to pre-trained vision-language-action models. In Conference on Robot Learning (CoRL), volume 305 of Proceedings of Machine Learning Research, pages 5022–5038.DOI
- [51]VeBrain Team. 2026. Visual embodied brain-1.5: Enhanced perception, spatial reasoning and robot control in spaces. In IEEE International Conference on Robotics and Automation (ICRA). IEEE Xplore.
- [52]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291.arxiv.org/abs/2305.16291
- [53]Qi Wang, Tianyi Wang, Chengyang Li, Shikun Ban, Yurun Chen, Yizhong Ge, Jason Qin, Chengtai Li, and Wentao Zhu. 2026. Towards the harness of embodied agents. arXiv preprint arXiv:2608.11246.arxiv.org/abs/2608.11246
- [54]Wei Wu, Fan Lu, Yunnan Wang, Shuai Yang, Shi Liu, Fangjing Wang, Qian Zhu, He Sun, Yong Wang, Shuailei Ma, and 1 others. 2026. A pragmatic VLA foundation model. arXiv preprint arXiv:2601.18692.arxiv.org/abs/2601.18692
- [55]Xiaomi MiMo Team. 2026. MiMo-Embodied: X-embodied foundation model technical report.
- [56]Rui Yang, Hanyang Chen, Junyu Zhang, Mark Zhao, Cheng Qian, Kangrui Wang, Qineng Wang, Teja Venkat Koripella, Marzieh Movahedi, Jiaming Li, and 1 others. 2025a. EmbodiedBench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. arXiv preprint arXiv:2502.09560.arxiv.org/abs/2502.09560
- [57]Rui Yang, Hanyang Chen, Junyu Zhang, Yuchen Zhao, Yifan Zhang, Yixin Chen, and 1 others. 2025b. Embodied-bench: Comprehensive benchmarking multi-modal large language models for vision-driven embodied agents. In International Conference on Machine Learning (ICML).
- [58]Shuai Yang, Hao Li, Bin Wang, Yilun Chen, Yang Tian, Tai Wang, Hanqing Wang, Feng Zhao, Yiyi Liao, and Jiangmiao Pang. 2026. Vision-language-action instruction tuning: From understanding to manipulation. In International Conference on Learning Representations.arxiv.org/abs/2507.17520
- [59]Kaixiang Yao, Xu Wang, Miao Pan, Hu Xiyue, Weishi Wang, Daniel Dahlmeier, Jintao Chen, Yongliang Shen, Xuhong Zhang, and Wenqi Zhang. 2026. Spatial-interactor: Learning spatial reasoning through interaction with the observable physical world. arXiv preprint arXiv:2609.29308.
- [60]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, and 1 others. 2023. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations.arxiv.org/abs/2210.03629
- [61]Chenduo Ying and 1 others. 2026. Robofailing: Timely failure detection and causal reasoning in robotic manipulation. In Annual Meeting of the Association for Computational Linguistics (ACL).
- [62]Wenqi Zhang, Mengna Wang, Gangao Liu, Huixin Xu, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Jiajun Liu, Weiming Lu, Peng Li, and Yueting Zhuang. 2026a. Embodied-reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks. In International Conference on Learning Representations (ICLR).
- [63]Wenqi Zhang, Mengna Wang, Gangao Liu, Xu Huixin, Yiwei Jiang, Yongliang Shen, and 1 others. 2025. Embodied-Reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks. arXiv preprint arXiv:2503.21696.arxiv.org/abs/2503.21696
- [64]Yixian Zhang, Huanming Zhang, Feng Gao, Xiao Li, Zihao Liu, ChunyAng Zhu, Jiaxing Qiu, Yuchen Yan, Jiyuan Liu, Wenhao Tang, and 1 others. 2026b. Harness VLA: Steering frozen VLAs into reliable manipulation primitives via memory-guided agents. arXiv preprint arXiv:2607.08448.arxiv.org/abs/2607.08448
- [65]Yue Zhang, Rui Wang, Jiehong Lin, Zhongrui Wang, and Xiaojian Qi. 2026c. Retrieval-vla: Training-free in-context adaptation for vision-language-action models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, pages 1358–1367.DOI
- [66]Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. Group sequence policy optimization. arXiv preprint arXiv:2507.18071.arxiv.org/abs/2507.18071
- [67]Zhide Zhong, Haodong Yan, Junfeng Li, Junjie He, Tianran Zhang, and Haoang Li. 2026. VLA-OPD: Bridging offline SFT and online RL for vision-language-action models via on-policy distillation. arXiv preprint arXiv:2603.26666.arxiv.org/abs/2603.26666





