betaFeedback

WorksComputer Science

Learning the Task or Learning the Verifier? Rethinking Criteria Exposure in Embodied Self-Improvement

Abstract

Self-improving embodied agents are increasingly being developed and compared on robot simulation benchmarks. Several recent works expose the benchmark's success criteria to the learning agent, a practice we call Criteria Exposure (CE). These criteria, originally written to verify policies trained by imitation learning, often demand more than the corresponding language instructions, so a score from such a verifier may not faithfully reflect whether the instructed task was completed, especially for robot-control programs written by an agent. As embodied coding agents report ever higher benchmark scores, a natural question arises: when a coding agent is exposed to the benchmark's criteria, does it generate code that better performs the instructed task, or code that simply overfits to the verification criteria? We conduct controlled experiments with a self-improving agent with and without CE, and find that with CE the agent quickly produces code with a higher success rate under the given criteria, but this advantage disappears when the criteria are changed slightly without departing from the original instructions. More importantly, the skills the agent develops under CE transfer poorly across tasks. We further propose instruction-faithful criteria and show that they better reflect the performance of agent-generated programs.

Introduction

Agents that write code to control robots with multimodal language models complete many manipulation tasks zero-shot [12, 15, 19, 29], but preserve nothing from one episode to the next, which makes self-improvement across episodes a natural next step [28, 31, 36]. Such agents are developed and compared on embodied simulation benchmarks [9, 20, 22] originally built to evaluate end-to-end policies trained by imitating demonstrations, such as vision-language-action (VLA) models [4, 6, 17, 25] and world action models (WAMs) [26]. The actual success criteria of these benchmarks often ask for more than the instruction; in the press_by_number task of RoboDqo, for instance, the criteria require the blue button to be pressed exactly twice, between two groups of red presses, which the instruction never states (Figure 1a). For policies trained by imitation, this gap is harmless, since their demonstrations already contain every detail the criteria verify: order, counts, end states, and the return of both arms. An agent that receives only the instruction, however, lacks this information. Existing self-improving methods therefore expose the criteria to the agent to different degrees, from rule descriptions [10] and published scoring tables [13, 18] to numeric thresholds [14], formal goal predicates [32], and the source code of the verifier [3]. A robot in the real world, by contrast, receives only its instruction and, at most, a success judged by a human or a VLM. Our research question is: when a self-improving embodied agent is exposed to the success criteria, does the code it iteratively produce better perform the instructed task, or simply overfit to the verification criteria?

Figure 1. Learning the task or the verifier? (a): in press_by_number, the criteria CorigC^{\mathrm{orig}} verify a button protocol that the instruction does not state (highlighted). Without Criteria Exposure (w/o CE) the learner sees only whether each episode succeeded; with it (w/ CE) it also reads the criteria code, score and stages. Right of each setting: the test successes of its library on 20 layouts, judged by CorigC^{\mathrm{orig}} and by the instruction-faithful criteria CfaithC^{\mathrm{faith}}. (b): libraries learned under CorigC^{\mathrm{orig}} on the 16 criteria-limited tasks, 320 test episodes each, with every episode judged by both criteria (§ 4.2.1). The lead of w/ CE under CorigC^{\mathrm{orig}}, 51.6% against 27.8%, is gone under CfaithC^{\mathrm{faith}} (45.0% against 47.2%).

Studies of benchmark validity and of specification gaming show that criteria can demand more than the instruction states and that agents exploit a verifier they can inspect [16, 37, 40], but we are not aware of work that measures how much of a reported gain in embodied self-improvement comes from reading the criteria, or whether it survives when the criteria change. We therefore compare two settings of one fixed self-improvement loop, in which a frozen coding agent acts and a learner agent turns its episodes into a library of skills and memory: with Criteria Exposure (w/ CE), the learner reads the criteria; without it (w/o CE), the learner sees only whether each episode succeeded. We ask three questions: on which tasks exposure raises the score; whether its lead survives a change of criteria, both of the criteria that judge the episodes and of the criteria the learner reads; and whether what it adds carries over to new tasks and to two further benchmarks, LIBERO-PRO and RoboCasa365. To change the criteria, we rewrite them for the RoboDojo tasks whose criteria ask for more than the instruction, so that they verify only what the instruction asks. On these tasks every episode is judged by both versions, and in a separate run the learner reads the rewritten criteria instead of the original ones.

Our experiments reveal three findings. First, under the original criteria, exposure helps only on tasks whose criteria ask for more than the instruction; when criteria and instruction agree, learning from success alone does just as well. Second, the score of exposure follows the criteria it read: when the same episodes are judged by instruction-faithful criteria, its advantage disappears, as the exposed libraries lose successes while the success-only libraries gain them (Figure 1b); conversely, a library learned from the rewritten criteria scores far lower when judged by the original ones. Third, the lead of exposure does not reappear on new RoboDojo tasks, and across benchmarks neither the exposed nor the success-only library gives a clear gain over the empty library; only experience gathered within the target benchmark does.

Taken together, these results suggest a practical principle for evaluating embodied self-improvement: the learner should receive only the feedback a robot would receive in the real world, and the benchmark’s criteria should verify what the instruction says. A gain that holds when the criteria are changed but still agree with the instruction reflects what the agent has learned about the task. Seen this way, whether the learner reads the criteria is not an implementation detail of a self-improving agent but a choice about measurement, and it determines what a reported gain means.

Related Work

Code-writing embodied agents and their benchmarks. Language models plan and act for robots by selecting skills [1], writing policy code [19], or composing value maps [15]; recent agents close the loop with code replanning and thin tool layers [12, 13, 29], and frontier models now control robots zero-shot. These agents are evaluated on benchmarks built for vision-language-action policies trained on demonstrations [4, 6, 17, 25]: RoboDojo [9], LIBERO and LIBERO-PRO [20, 38], RoboCasa365 [22], and RoboTwin 2.0 [8]. These works score agents with criteria written for demonstration-trained policies; whether the criteria verify only what the instruction asks is rarely examined on its own, and we measure how this gap enters the score of a self-improving agent.

Self-improving agents and the feedback their learners see. In digital domains, programs are evolved against automatic evaluators [23, 34], agents reflect on textual feedback [28, 36], skills are verified in a game world [31], and code is revised against tests [33]. Embodied systems differ in how much of the success criteria their learners or agents see. Some give only success or keep the grader isolated [21, 27]; others add rule descriptions with coarse feedback [10], published scoring tables and partial numeric thresholds [13, 14, 18], formal goal predicates [32], or the source code of the verifier [3]. In RATs, the verifier reads goal predicates and ground truth while the policy writer’s side is filtered [35]. In these systems the amount of criteria information is mostly fixed by design rather than varied as an experimental factor. The closest comparison is an ablation of EmbodiedEvo between task descriptions alone and descriptions with the wiki, but both arms are judged by the original criteria [18]. We treat exposure as the variable and test what it gains under a second set of criteria.

Specification gaming and benchmark validity. Optimizing a proxy diverges from the intended objective [2, 30], with measurable effects of reward misspecification [24]. Language agents exploit tests and evaluators they can inspect [5, 37], and agentic benchmarks need checks of their own validity [16, 40]. Where rule-based criteria are missing, success is judged by VLMs [11] or by autonomous real-world evaluation [39]. Building on these studies, we further show that on embodied benchmarks the act of reading the criteria turns the gap between criteria and instruction into score, and we measure this share with counterfactual criteria that score the same instructions.

Preliminaries and Study Design

This section describes the controlled study we use to analyze what a self-improving embodied agent learns from the success criteria. The study compares two settings of one loop, w/ CE and w/o CE, and answers three questions: where exposure raises the score under the original criteria, whether the advantage holds when the criteria change, and whether the library carries over to new tasks and benchmarks. We describe in turn the loop and its two settings, criteria-limited tasks, instruction-faithful criteria, and the evaluation protocol.

Self-Improvement Loop and Criteria Exposure

A task τ=(Iτ,Dτ,Cτ)\tau= (I_{\tau}, D_{\tau}, C_{\tau}) has an instruction IτI_{\tau}, a distribution of layouts DτD_{\tau}, and success criteria CτC_{\tau} that judge each trajectory; some benchmarks also report a score and stages. The success the instruction asks for, SτS_{\tau}, is defined by IτI_{\tau} and cannot be computed directly. A frozen actor π\pi, GPT-6 Astra with high reasoning effort, acts through the primitives of PyRUA-Lean [29]. In each round a learner UU, Claude Code with Opus 5.5 and high reasoning effort, makes one revision of the library L\mathcal{L}, a set of callable skills and memory entries, from the evidence Eν(B)E_{\nu}(B) of the last batch, and the revision is rolled back if the batch clearly regresses. Each task has its own library, learned over five rounds on five development layouts. Algorithm 1 and Figure 2 summarize the loop for one task.

Figure 2. Figure 2 Overview of the self-improvement loop. Left, learning: the actor (GPT-6 Astra, high reasoning effort) writes and executes Python cells that call the robot’s primitives on five development layouts, and sees the result of each cell. Between batches the learner (Claude Code with Opus 5.5, high reasoning effort) turns the evidence into one revision of the library Lk\mathcal{L}_k, whose skills are callable code that reports what it did and whose memory entries say when to apply what, with the evidence for it; a revision is usually tried first in verification rollouts, and the loop rolls back to the best revision so far when a batch falls clearly below it. Whether the learner also sees the success criteria is the one switch between the two settings, w/ CE and w/o CE (the switch under the learner). Right, evaluation: the final library LK\mathcal{L}_K is frozen and read-only, and the same actor runs alone on layouts the loop never saw. actor never reads them. The actor, the primitives, perception and benchmark scoring are frozen and only the library changes; the two settings share the loop and differ in what the learner reads, so differences between them reflect exposure rather than the loop itself.

The loop runs in two settings. Without Criteria Exposure (w/o CE), the learner sees only whether each episode succeeded; the criteria code, the score and the stages are withheld. A robot in the real world has no criteria code and receives success from a human or a VLM, so this is the setting in simulation closest to the real world. With Criteria Exposure (w/ CE), the learner also reads the criteria code of the task and the score and stages of each episode, as systems that give the learner the source of the verifier do; the

Algorithm 1 Self-improvement loop

Require: actor π\pi; dev layouts DdevD_{\mathrm{dev}}; rounds KK; setting v∈{w/ CE,w/o CE}v \in\{\text{w/ CE}, \text{w/o CE}\}; margin θ=2\theta= 2

  1. L0←(S0,M0)=(∅,∅)\mathcal{L}_0 \leftarrow(S_0, M_0) = (\varnothing, \varnothing) — skills and memory

  2. L⋆←L0\mathcal{L}^{\star} \leftarrow\mathcal{L}_0; H←[ ]H \leftarrow[\ ]; B0←Run⁡(π,L0,Ddev)B_0 \leftarrow\operatorname{Run}(\pi, \mathcal{L}_0, D_{\mathrm{dev}}) — zero-shot batch

  3. for k=1,…,Kk = 1, \ldots, K do

  4. E←Evidence⁡(Bk−1,v)E \leftarrow\operatorname{Evidence}(B_{k-1}, v) — trajectories; criteria, score and stages only under w/ CE

  5. (ΔS,ΔM,h)←Improve⁡(E,H,Lk−1)(\Delta S, \Delta M, h) \leftarrow\operatorname{Improve}(E, H, \mathcal{L}_{k-1}) — edit skills, memory or both; verify by rollouts

  6. Lk←Freeze⁡(Sk−1⊕ΔS,Mk−1⊕ΔM)\mathcal{L}_k \leftarrow\operatorname{Freeze}(S_{k-1} \oplus\Delta S, M_{k-1} \oplus\Delta M) — structural checks only, not success

  7. Bk←Run⁡(π,Lk,Ddev)B_k \leftarrow\operatorname{Run}(\pi, \mathcal{L}_k, D_{\mathrm{dev}})

  8. if BkB_k has ≥θ\ge\theta fewer successes than the batch of L⋆\mathcal{L}^{\star} then

  9. Lk←L⋆\mathcal{L}_k \leftarrow\mathcal{L}^{\star} — rollback

  10. else if BkB_k is the best batch so far then

  11. L⋆←Lk\mathcal{L}^{\star} \leftarrow\mathcal{L}_k

  12. append (ΔS,ΔM,Bk,h)(\Delta S, \Delta M, B_k, h) to HH — change, result, hypothesis

  13. return LK\mathcal{L}_K — the active library after round KK

Criteria-limited Tasks

We call a task criteria-limited if CτC_{\tau} disagrees with SτS_{\tau} on some trajectories, either by rejecting a trajectory SτS_{\tau} accepts or by accepting one SτS_{\tau} rejects (criteria looser than the instruction); all other tasks are faithful. Such tasks appear in every benchmark we examined, RoboDojo, LIBERO [20] (whose instructions and goals LIBERO-PRO keeps), RoboCasa365 and RoboTwin 2.0 [8], and their disagreements fall into a few kinds. The criteria may ask for more than the instruction states, such as an order, a count or an end state: in RoboDojo, press_by_number requires a button protocol its instruction never gives. They may be stricter than the instruction needs, such as an unstated tolerance for an object resting on another in LIBERO. They may contradict the instruction, as in a LIBERO task that asks to stack a bowl on the front bowl while its criteria require the back one. Or they may verify less than the instruction asks: in RoboTwin 2.0, shake_bottle_horizontally succeeds once the bottle is lifted.

We rewrite criteria and re-judge episodes on RoboDojo only. Its criteria report a score and stages besides success, so an exposed learner reads the most complete information there, and we compared the criteria of all of its tasks with their instructions; rewriting the criteria of other benchmarks and repeating these experiments on them is left to future work. We distinguish seven types: hidden rules, hard thresholds, timing and irreversible events, criteria defects, extra end states, loose criteria, and contradictions. In RoboDojo, after a manual review of every task, the 20 criteria-limited tasks have 26 variants, and hidden rules and hard thresholds are the most common types. Criteria-limited is defined with respect to the instruction the agent receives at reset; the RoboDojo website publishes part of these rules in its task descriptions and scoring tables.

Instruction-faithful Criteria

For each criteria-limited task we rewrite only the criteria code, never the instruction, and leave the original repository untouched: hidden rules are removed or relaxed, defects fixed, hard thresholds relaxed to what still clearly satisfies the instruction, loose criteria tightened where the simulator state can detect the difference, and the return of both arms kept. The two criteria judged the same episodes independently, and every episode whose verdict flipped was checked by hand without finding a misjudgment by the new criteria. Each evaluation episode is then judged by both criteria, giving SRorig\mathrm{SR}_{\mathrm{orig}} and SRfaith\mathrm{SR}_{\mathrm{faith}}. Since both criteria score the same instruction on the same episode, any difference between the two success rates comes from what the criteria verify, not from behavior. Cfaith\mathcal{C}_{\mathrm{faith}} is ours and serves as a proxy for Sτ\mathcal{S}_{\tau}; we discuss this in section 5.

Experiment Settings and Evaluation Protocol

To isolate the effect of exposure, the two settings of each comparison use the same actor, primitives, round budget and development layouts. The 25 learned tasks comprise 16 criteria-limited and 9 faithful tasks, grouped in advance by whether their criteria were rewritten; Appendix § 7.§ 4 lists the tasks. Development uses layouts 0–4 of seed 1 and testing layouts 0–19 of seed 0. We report the success rate (SR) and compare settings layout by layout. All experiments use the same actor and learner. To reuse libraries, we distill the libraries of ten source tasks into one shared library and apply it to disjoint target tasks, in RoboDojo and on two further benchmarks with aligned primitives.

We use three simulated benchmarks with different roles. RoboDojo is the main benchmark: its rule-based criteria report a score and stages, and all three questions are answered on it. LIBERO-PRO is a single-arm tabletop benchmark where zero-shot success is already near the ceiling; it tests whether transfer still has room at high scores. RoboCasa365 has household kitchen scenes and the lowest zero-shot scores; it tests cross-benchmark experience and also hosts in-benchmark learning.

Experiments and Analysis

We test three hypotheses:

  • H1. Exposure raises the score only on tasks whose criteria ask for more than the instruction (§ 4.1).

  • H2. If exposure teaches the verifier rather than the task, its lead shrinks or reverses when the same episodes are judged by instruction-faithful criteria (§ 4.2.1), and each exposed library scores best under whichever criteria its learner read (§ 4.2.2).

  • H3. What exposure adds does not carry over to new tasks or benchmarks (§ 4.3).

Exposure under the Original Criteria

We first judge the benchmark’s way, by the original criteria, which are also the criteria the exposed learner reads.

Success-only versus Exposed Self-Improvement

Setup. We learn one library per task and setting on the 25 learned tasks (16 criteria-limited, 9 faithful), over five rounds on five development layouts, and test each library on layouts 0–19 of seed 0 against the empty library. The two groups were fixed before any result was seen.

Results. Table 1 and Figure 3 show three patterns. First, learning from success alone doubles success, from 24.6% to 47.8%, and reading the criteria adds another 14.2 points (62.0%). Second, this extra gain comes from the criteria-limited tasks: on faithful tasks the two settings are level (83.3% and 80.6%), while on criteria-limited tasks exposure leads, 51.6% against 27.8%. Third, among criteria-limited tasks the lead depends on whether an unstated rule shows in failed episodes: in build_tower the leftover blocks reveal it and both settings reach 20/20, whereas the button protocol of press_by_number, blue pressed twice and between the two red groups, never shows, and the success-only learner never finds it.

Figure 3. Development success by round, on the 16 criteria-limited and the 9 faithful tasks. Dashed: w/ CE; solid: w/o CE; round 0 is the empty library. On criteria-limited tasks both settings start at 6% and part in round 1 (40% against 28%); after that the success-only learner barely moves and ends at 31%, against 55% with exposure. On faithful tasks the success-only learner is ahead through round 4 and both end above 90% (91% and 95%); its lead, at most 16 points, is close to the 14-point gap of round 0 (66% against 52%), where both run the same empty library.

TasksEmpty library L∅\mathcal{L}_{\varnothing}w/o CEw/ CE
All (25)24.647.8 (+23.2)62.0 (+37.4)
Criteria-limited (16)8.427.8 (+19.4)51.6 (+43.2)
Faithful (9)53.383.3 (+30.0)80.6 (+27.3)

Table 1. Test success under the original criteria (SR, %, layouts 0–19 of seed 0). Parentheses: difference from the empty library in points. The criteria-limited tasks count the episodes of § 4.2.1, judged by CorigC^{\mathrm{orig}}.

Synthesis

  • Exposure pays off only where the criteria ask for more than the instruction. Richer feedback is often assumed to make self-improvement better. Our results show a condition: when the criteria verify what the instruction says, success alone tells the learner what the task requires, and what remains, a reliable procedure, is learned equally well in both settings. Exposure helps only with rules that the instruction omits and that failures do not reveal.

  • The advantage of exposure measures the benchmark as much as the agent. Because it comes from what the instruction leaves out, the gain of exposure grows with the gap between a benchmark’s instructions and its criteria. Some of what the criteria reveal is legitimate task knowledge, but it belongs in the instruction: benchmarks that state in the instruction what their criteria verify (counts, order, end states, the reset of the arms) and release versioned criteria with a comparison against the instructions would let self-improvement be compared on the task alone.

Changing the Criteria: Task or Verifier?

§ 4.1 judged with the criteria the exposed learner read. Changing them separates two explanations. If exposure teaches the task, its lead should survive under CfaithC^{\mathrm{faith}}; if it teaches the verifier, the lead should shrink or reverse, and each library should score best under whichever criteria its learner read. We change the criteria used for judging and the criteria that are read.

Re-judging with Instruction-faithful Criteria

Setup. We run the libraries of § 4.1.1 on the 16 learned criteria-limited tasks, all of which have rewritten criteria, 20 layouts each, and judge every episode by both criteria.

Results. Figure 4 shows the lead of exposure disappear. Under CorigC^{\mathrm{orig}} the exposed library leads, 51.6% against 27.8%: paired by task and layout, it alone succeeds on 93 layouts and the success-only library alone on 17. Judged by CfaithC^{\mathrm{faith}}, the same episodes give 45.0% for the exposed library and 47.2% for the success-only one, a difference within noise (47 against 54 layouts); the gap shrinks by 25.9 points (95% bootstrap interval over tasks 4.7–54.4). The success-only library loses almost nothing: CfaithC^{\mathrm{faith}} accepts all but 2 of its 89 successes under CorigC^{\mathrm{orig}}, and it gains where the original criteria demand an unstated protocol, as in press_by_number, swap_blocks and make_kong. The exposed library loses where it had learned that protocol, and on tasks whose criteria changes mostly fix defects or relax thresholds the two criteria give nearly the same scores.

Figure 4. The same episodes judged by the original and the instruction-faithful criteria. Libraries learned under CorigC^{\mathrm{orig}} on the 16 criteria-limited tasks, 320 episodes per library. Light bars: judged by CorigC^{\mathrm{orig}}; solid bars: the same episodes judged by CfaithC^{\mathrm{faith}}; arrows give the change in points.

Swapping the Criteria that Are Read

Setup. We then let the learner read CfaithC^{\mathrm{faith}} instead (ten tasks, five rounds) and compare the resulting library LCEfaith\mathcal{L}^{\mathrm{faith}}_{\mathrm{CE}} with LCEorig\mathcal{L}^{\mathrm{orig}}_{\mathrm{CE}} on the same ten tasks, each judged by the criteria its learner read and by the other.

Results. Figure 5 shows that each exposed library does well only under the criteria it read: LCEorig\mathcal{L}^{\mathrm{orig}}_{\mathrm{CE}} falls from 57.0% to 45.0% under CfaithC^{\mathrm{faith}}, and LCEfaith\mathcal{L}^{\mathrm{faith}}_{\mathrm{CE}} from 65.0% to 13.5% under CorigC^{\mathrm{orig}}. The fall is steeper for LCEfaith\mathcal{L}^{\mathrm{faith}}_{\mathrm{CE}} because the two directions differ: CfaithC^{\mathrm{faith}} mostly removes or relaxes the hidden rules and thresholds of CorigC^{\mathrm{orig}}, so moving LCEorig\mathcal{L}^{\mathrm{orig}}_{\mathrm{CE}} to CfaithC^{\mathrm{faith}} loosens the verification, whereas moving LCEfaith\mathcal{L}^{\mathrm{faith}}_{\mathrm{CE}} to CorigC^{\mathrm{orig}} adds back every rule the library never saw. The score follows the criteria that were read even where they are looser: having read CfaithC^{\mathrm{faith}}, the library no longer sets the bottle upright in pour_by_language, which CorigC^{\mathrm{orig}} requires, and solves 19 of its 20 test layouts under CfaithC^{\mathrm{faith}} but none under CorigC^{\mathrm{orig}}. This does not tell which criteria are right, which § 4.2.1 addresses, but it shows that the score of exposure does not carry between two criteria that score the same instruction.

Figure 5. Each exposed library judged by the criteria it read and by the other. Both on the ten tasks of LCEfaith\mathcal{L}^{\mathrm{faith}}_{\mathrm{CE}}, seed-0 test, 200 episodes per library. LCEorig\mathcal{L}^{\mathrm{orig}}_{\mathrm{CE}}: their episodes in Figure 4. LCEfaith\mathcal{L}^{\mathrm{faith}}_{\mathrm{CE}}: its final library on the same layouts. Light bars: judged by the criteria the learner read; solid bars: the same episodes judged by the other criteria; arrows give the change in points.

Synthesis and Implications

  • Exposure makes self-improvement fit the criteria rather than learn the instructed task. A higher score from reading the criteria is commonly taken as faster learning of the task. A learner that reads the criteria, however, revises the library until it passes the verifier it reads, so the library learns the task only as far as the criteria agree with the instruction. Under instruction-faithful criteria the exposed library loses successes that the success-only library gains, and each exposed library scores best only under the criteria it read: on press_by_number it solves all 20 layouts under CorigC^{\mathrm{orig}} and none under CfaithC^{\mathrm{faith}}, and the library that read CfaithC^{\mathrm{faith}} solves 65.0% of its test episodes under CfaithC^{\mathrm{faith}} but 13.5% under CorigC^{\mathrm{orig}}.

Self-improvement should therefore be measured without exposure by default, with the success rate under instruction-faithful criteria reported alongside: a robot in the real world has no criteria code, so the part of a gain that survives a change of criteria is the part to expect there.

  • Exposure changes what the learner writes down. A learner that reads the criteria can explain a failure by the condition it violated, so its experience is phrased in the terms of one verifier; a learner that sees only success has to find a physical cause, and its rules are closer to regularities of the robot. About a third of the exposed memory entries cite the criteria code, and none of the success-only ones do.

Reusing the Library: What Carries Over

Finally, we carry the libraries to new tasks in RoboDojo and to two further benchmarks.

Across Tasks within RoboDojo

Setup. We distill the libraries of ten learned tasks into one library per setting and apply it to ten target tasks that none of the source tasks covers, paired with the empty library on the same layouts.

Results. Both settings help on the target tasks (first row of Table 2): the empty library solves 65 of 200 episodes, the success-only library 80 and the exposed one 71. The 14.2-point lead of exposure on its own tasks does not reappear.

TargetLibrary sourceL∅\mathcal{L}_{\varnothing}w/o CEw/ CE
RoboDojo, 10 target tasks × 20 layoutsRoboDojo65/20080/200 (+15)71/200 (+6)
LIBERO-PRO, 6 tasks × 10 seedsRoboDojo45/6048/60 (+3)46/60 (+1)
RoboCasa365, 10 target tasks × 10 seedsRoboDojo22/10024/100 (+2)24/100 (+2)
RoboCasa365, 10 target tasks × 10 seedsRoboCasa36522/10033/100 (+11)29/100 (+7)

Table 2. Table 2 Reusing libraries on new tasks and benchmarks (successes, each benchmark’s own criteria). The RoboDojo libraries are distilled from ten learned tasks and carried over to LIBERO-PRO and RoboCasa365; the last row uses libraries distilled from other RoboCasa365 tasks. Every row is paired with the empty library on the same layouts; parentheses give the change in successes against it.

Across Benchmarks

Setup. We apply the same libraries, translated to each target’s prompt and primitives, on LIBERO-PRO and RoboCasa365, whose primitives we aligned with RoboDojo’s. On LIBERO-PRO the empty library already solves 34 of the 40 tasks on all five seeds, so we use the 6 others, with ten seeds each. On RoboCasa365 we also distill one library per setting from other RoboCasa365 tasks, as an in-benchmark reference.

Results. The libraries from RoboDojo add little on either benchmark (Table 2): 48 and 46 of 60 episodes against 45 on LIBERO-PRO, and 24 and 24 of 100 against 22 on RoboCasa365. The libraries distilled within RoboCasa365 do better, 33 (w/o CE) and 29 (w/ CE) of 100.

Synthesis and Implications

  • What exposure adds is specific to the task it was learned on. One might expect the setting that scores higher on its own tasks to transfer better. It does not: the lead of exposure disappears on target tasks. Exposed experience carries the conditions and precision needed to pass one task’s criteria, which become noise or overhead elsewhere, whereas rules about the gripper hold across tasks.

  • Experience transfers where the bottleneck is shared. Within a benchmark the robot, the scenes and perception stay the same, and learned rules carry over. Across benchmarks the general part of the experience is already covered by the target’s prompt, while what the target lacks has no counterpart in the source, so the transferred skills go unused; experience gathered on the target benchmark, even on other tasks, addresses its own bottleneck and helps.

Limitations and Future Work

The instruction-faithful criteria are ours, and the feedback is simulated. CfaithC^{\mathrm{faith}} serves as our proxy for the success the instruction asks for, SτS_{\tau}. Neither setting’s libraries saw the rewritten criteria when they were written; rules and types were fixed in advance; a second seed was judged independently and every flipped episode checked. Still, only RoboDojo’s criteria were rewritten, and success-only feedback in simulation is cleaner than the success a human or a VLM reports for a robot in the real world, which has no criteria code. Next steps are an independent review, released criteria code, rewritten criteria for a second benchmark, and VLM success detection in place of the simulator’s success [11].

Exposure is studied at two levels, as current systems implement it. Most existing systems sit at intermediate levels (rule descriptions, published scoring tables, partial thresholds), which we did not study separately. A next step is to expose these levels one at a time and measure how the gain under CorigC^{\mathrm{orig}}, and its loss under CfaithC^{\mathrm{faith}}, grow with what the learner reads.

Samples are small and the mechanisms are inferred. Each task has 20 test layouts, so quantitative claims use pooled results and single tasks are described only qualitatively; and distillation variance was not estimated. The accounts of what exposure writes into the library, of what carries over and of how the actor uses transferred libraries rest on entry text, skill calls and action statistics, not on ablations. Next steps are repeated distillations and ablations that remove or swap individual entries, such as the re-measuring entries of the distilled w/ CE library or the memory entries of a translated library.

Conclusion

In this paper, we studied what a self-improving embodied agent learns when it reads the success criteria, by comparing one self-improvement loop with and without Criteria Exposure, under the benchmark’s own and instruction-faithful criteria, and on new tasks and benchmarks. Our results show that self-improvement from success alone is real and survives a change of criteria, whereas the score that Criteria Exposure adds comes mainly from satisfying the criteria that were read: it appears only where the criteria and the instruction disagree, disappears under instruction-faithful criteria, and across benchmarks neither the exposed nor the success-only library gives a clear gain, while only in-benchmark experience helps. These findings provide a more careful reading of reported self-improvement gains on embodied benchmarks and practical guidance for measuring them: use the feedback a robot receives in the real world, and judge by criteria that verify what the instruction says. A robot in the real world never sees its verifier; the agent we measure should not either.

Implementation Details

Experimental protocol

Benchmark. RoboDojo’s simulation track uses a dual-arm ARX-X5 with RGB observations only; its 42 tasks span five capability dimensions. The 20 tasks on which the zero-shot actor solves fewer than half of the episodes of the official evaluation seed, with five stronger ones, make the 25 compared tasks (Table 4). Every run uses one version of RoboDojo (Table 3); task source, criteria, leaderboard script and evaluation layouts are identical file by file across the checkouts we compare.

Benchmark
CodeRoboDojo commit cb2a37d58dbb (2026-10-08), XPolicyLab 53c5da5a01bc
AssetsHF dataset RoboDojo-Benchmark/RoboDojo, revision 35efbc7dedfd (2026-09-28)
SimulatorIsaac Sim 5.1.0, Isaac Lab 0.54.3; suite ARX-X5, RGB only
Actor
ModelGPT-6 Astra, reasoning effort high, at most 8,192 output tokens per request
InterfacePyRUA-Lean cells, RGB perception, no VLA, ARX-X5 suite
Per episodeat most 100 cells, 200 MLLM requests, 20 nudges, 90 minutes of policy time; the task’s own native-action limit
Learner
AgentClaude Code, Claude Opus 5.5, effort high
Per sessionup to 2 attempts of 120 minutes; at most 8 rollouts, 3 per dev layout
Librarytask-local skills and memory, at most 40,000 characters per task
Campaign
Splitsdev: seed 1, layouts 0–4; test: seed 0, layouts 0–19
Rounds6 including round 0 (five rounds of improvement)
Rollbackto the best complete earlier batch, if a complete batch has ≥ 2 fewer successes

Table 3. Settings (the same in every run).

RoboDojo (25)align_blocks, build_tower, deposit_coin, fill_egg_holder, fill_pen_holder, hang_mugs_random, insert_key, insert_tubes, make_kong, make_toast_random, match_and_pick_from_conveyor, organize_table, pack_objects_into_box, pick_from_conveyor_by_image, play_Xylophone, play_stacking_toy, plug_in_charger, pour_by_language, pour_liquid_into_cup, press_by_number, stack_bowls_random, store_laptop_and_headphones, store_tools_in_toolbox, swap_blocks, sweep_blocks_random
LIBERO-PRO (6)libero_10_swap tasks 5, 6, 9; libero_goal_swap tasks 1, 5, 9
RoboCasa365 (10)OpenCabinet, PickPlaceDrawerToCounter, TurnOffStove, HeatKebabSandwich, LoadDishwasher, MakeIceLemonade, SeparateFreezerRack, SteamInMicrowave, StirVegetables, WaffleReheat
RoboCasa365 sources (10)CloseBlenderLid, CloseToasterOvenDoor, CoffeeSetupMug, OpenStandMixerHead, TurnOnSinkFaucet, ArrangeBreadBasket, GatherTableware, PackIdenticalLunches, SearingMeat, StoreLeftoversInBowl

Table 4. Tasks.

Conditions. The actor’s model, primitives and prompt are the same in every condition. The learner with Criteria Exposure reads the criteria code, each episode’s score and its passed stages; the one without sees full trajectories and success only. All conditions are compared over five rounds of improvement (rounds 0–5). The groups (Appendix § 8) rest on reading the code, the learners’ diagnoses and the failed rollouts.

Layouts. Development reruns layouts 0–4 of official evaluation seed 1 every round; the test layouts are layouts 0–19 of seed 0 (500 episodes per condition), which no learner sees.

Statistics. Development batches are noisy: rerunning one library on one layout changes the verdict in 4% of cases, and the two round-0 batches of the same empty library differ in at least 13% of episodes. We therefore draw conclusions from the test layouts and pooled curves, and read single-task development curves only qualitatively.

Implementation and settings

Actor. GPT-6 Astra writes one code cell per MLLM call against PyRUA-Lean’s frozen robo API [29]: RGB perception (SAM 3 [7] and multi-view measurement) and motion primitives, with no VLA policy, depth or simulator state. The library is installed as a package library/: cells import its skills, and its live memory entries are appended to the prompt.

Library. A skill is a function of robo that imports only standard modules, numpy and other library modules, does no I/O and returns {"ok": ..., "why": ..., ...}. A memory entry is ## {task}-<stage>-<nnn> followed by stage, symptom, applies, do, status and evidence. A task’s library holds at most 40,000 characters, all of which the actor reads.

Learner. Claude Code reads the session, the task’s dev runs, the API source and, with Criteria Exposure, the task’s criteria; it writes only the candidate library, its verification rollouts and its notes. Without Criteria Exposure, criteria, scores and stages are removed from the evidence.

Acceptance and rollback. A candidate may change only the task’s own directory, must keep every memory entry identifier, must cite a judged rollout or dev batch for each entry newly marked dev_verified, and must pass the static skill rules and the size limit; none of this looks at success, and no candidate was refused. The active revision rolls back to the best earlier complete batch when a complete batch has two or more fewer successes; ties go to the batch that passed more stages, and a batch with an episode lost to infrastructure neither triggers a rollback nor becomes the best.

Evaluation details

The 25 learned tasks are grouped by whether their criteria were rewritten. Layouts that the benchmark marks unstable are replaced by the next layout in the official order. Under dual judging an episode ends early only when both criteria have reached a verdict, so neither verdict is truncated.

Tasks

Comparing Criteria with Instructions in RoboDojo

Criteria mechanism. RoboDojo criteria are sequences of stages: check stages in order, final_check at termination, and trigger_check, evaluated only at the step its condition becomes true; query requires exact counts and fails the episode for good once a count is exceeded; conditions nest with alternating AND and OR. Of the 42 tasks, 33 require both arms to return within 15 cm and 20° of home, while only 5 instructions ask for a reset.

Method. For each task we compare gen_instruction, run_reward, get_score and the configuration, grade the severity as high, medium or low, and confirm by hand; the review is mainly a static reading of the code.

Types. The type names are ours.

  • Hidden rule: the criteria verify conditions the instruction does not state (order, counts, relative position), e.g. the button protocol of press_by_number or the tile to draw in make_kong.

  • Hard threshold: the tolerance is stricter than clearly satisfying the instruction requires, e.g. 97% of the liquid in pour_liquid_into_cup.

  • Timing, event triggers and irreversibility: verification at one step only, or permanent failure once an event happens, e.g. pouring judged only when the bottle returns upright; a third press in press_by_number.

  • Criteria defect: the criteria code or an asset annotation is wrong, e.g. mug 14 in hang_mugs_random, the bug in solve_equation.

  • Extra end state: an end state the instruction does not mention, e.g. in make_toast the two remaining slices must stay on the bread shelf.

  • Loose criteria: trajectories that skip the action the instruction asks for are accepted, e.g. sweep_blocks accepts blocks picked up without the broom.

  • Contradiction: the criteria require the opposite of the instruction, e.g. in classify_objects_by_language the categories are named after asset folders, so a chocolate bunny counts as a wooden toy.

Counts. 23 tasks are consistent and 19 inconsistent (hidden rule 9, hidden rule and hard threshold 5, hard threshold 3, contradiction 1, defect 1); with solve_equation, whose criteria have a bug, 20 tasks and 26 variants are criteria-limited. By type, at medium or high severity: hidden rule 13, hard threshold 13, timing and events 10, defect 9, extra end state 5, loose criteria 4, contradiction 3. Other typical cases are make_kong (a tile must be drawn) and build_tower (only one configuration is accepted).

Published rules. The task descriptions and scoring tables on the RoboDojo website publish part of these rules, for example the button order of press_by_number; the recipes of EmbodiedEvo and HexaShell come from there [13, 18]. Table 5 gives the tasks by type. A task is criteria-limited only if the manual review confirms that its criteria and its instruction disagree; the table also lists the findings for tasks that the review kept as faithful.

TaskHidden ruleHard thresholdTiming/irrev.Criteria defectExtra end stateLoose criteriaContradictionArms home
align_blocks❍❍✓
arrange_largest_numberr^{r}✓
build_tower●❍❍✓
classify_objects✓
classify_objects_by_language❍✓
cover_blocks❍❍✓
deposit_coin❍✓
fasten_screws❍❍❍✓
fill_egg_holder✓
fill_pen_holder❍✓
fold_clothesr^{r}❍❍✓
general_pickup
hang_mugsr^{r}●✓
imitate_sorting_sequence❍✓
insert_key❍
insert_tubes✓
make_kong●❍❍
make_toastr^{r}❍❍✓
match_and_pick_from_conveyor❍
organize_table❍❍✓
pack_objects_into_boxr^{r}❍❍✓
pick_from_conveyor_by_image
play_stacking_toy❍❍✓
play_tic_tac_toe❍❍❍
play_Xylophone❍
plug_in_charger✓
pour_balls_into_vase❍❍✓
pour_by_language❍❍❍
pour_liquid_into_cupr^{r}❍❍❍
press_by_number❍●●●●✓
push_Tr^{r}❍✓
put_bottles_into_dustbin✓
solve_equation❍✓
sort_nesting_dolls_by_sizer^{r}❍✓
stack_blocksr^{r}✓
stack_blocks_by_language✓
stack_bowlsr^{r}✓
store_laptop_and_headphonesr^{r}❍✓
store_tools_in_toolbox❍✓
swap_blocks❍●✓
swap_T❍●✓
sweep_blocksr^{r}❍❍✓
Total13 (2)13 (1)10 (1)9 (3)5 (0)4 (0)3 (1)33

Table 5. Tasks by type. ●: high severity; ❍: medium; blank: low or none. The last column marks the 33 tasks whose criteria require both arms to return home (not counted as a type). Total: tasks at medium or high severity (high in parentheses). rr: reviewed together with its _random variant.

Instruction-faithful Criteria

Rules. (1) Change only run_reward and get_score, never the instruction. (2) Keep the return of both arms. (3) Remove or relax hidden rules. (4) Fix defects. (5) Tighten loose criteria where the simulator state detects the difference reliably. (6) Relax hard thresholds to what still clearly satisfies the instruction, listing old and new values. (7) Correct asset annotations in memory only. (8) Leave the original repository untouched. New criteria primitives check distance and orientation on the same feature, containment in an elliptic cylinder, and connected towers.

Validation. Offline unit tests; shadow judging and reruns on seed 0; on seed 2 both criteria judge the same episodes independently (an episode ends only when both have a verdict), and every flipped episode was checked by hand.

Decisions of our own. “Tighten” in fasten_screws means a nut sinking at least 35 mm. We also decided what counts as a tower in build_tower and which edges of the laptop may face down in store_laptop_and_headphones. Table 6 lists every change.

TaskTypeCorigC^{\mathrm{orig}}CfaithC^{\mathrm{faith}}
build_towerHone accepted 6-level configuration: given boards on given pillar pairs, pillars grouped {1,5}/{2,6}\{1,5\}/\{2,6\}, all 8 parts tilted <5∘< 5^\circ, top two levels centred within 2.3 cm, 20∘20^\circ between boardsone connected stack of all 8 parts, each on the table or on another part, longest support chain ≥4\ge4 levels, all parts at rest (<2< 2 mm per step)
D“A on B” compares heights onlybottom within 1.5 cm of the top below and overlapping in xyxy
–score tiers of the fixed configuration≥3\ge3 parts in 2 levels: 10; ≥5\ge5 in 3 levels: 30; full: 100
fasten_screwsTnut sunk ≥9\ge9 mm on its bolt (“tighten”)≥35\ge35 mm (seated ≈38\approx38 mm)
Rnut coaxial with bolt within 1 mmwithin 4 mm
fill_egg_holderD“in holder” by the oriented box, which reaches 5.5 cm behind the bodyalso within 5.5 cm of the holder centre in its own frame
fill_pen_holderHevery pen tip down (4 checks); xyxy of the pen’s check end within 3.5 cm of the axisno tip direction; pen centre within 3.5 cm of the axis (depth 3.5 cm kept)
Tholder never verified (“hold . . . then put it back down”)holder lifted >2> 2 cm at some step and back within 1.5 cm of its start height at the end
hang_mugsDorientation compared with hook 0 only, distance with any hook (hooks 2 and 3 never pass)distance <4.5< 4.5 cm, zz axis <60∘< 60^\circ and xx axis ±60∘\pm60^\circ on the same hook (values kept)
Dmug/00014 handle frame 0 annotated as identity rotationorientation of the other mugs’ handles
insert_keyHkey turned clockwise by 3030–90∘90^\circturned >30∘> 30^\circ in either direction
Thandover not verifiedboth hands have held the key >2> 2 cm off the table (gripper reading 0.01–0.6) before insertion; new score tier
make_kongHdraw the replacement tile, stand it at (0.319, −0.15) within 1.5 cm and 7∘7^\circ, grippers open; tile untouched before the kongremoved
Rother hand tiles standing within 7∘7^\circwithin 20∘20^\circ
Tstanding pair mahjong4_0/4_1 not verifiedmust stay standing (upstream fix 21b8d23)
make_toastDeach slot accepts any slice (one slice can fill both)two different slices, one per slot
D_random: slot corners of toaster/00004 mixed up, one corner of toaster/00002 slot 2 wrongcorrected from the USD geometry (3 mm margin)
Hexactly 2 slices left on the rack; all 4 slices upright within 30∘30^\circremoved
Torder of bread and lever not verifiedstage 1: both slices in, lever <85%< 85\%; stage 2: lever >85%> 85\%, arms back
organize_tableRfigurine on stand: stand centre within {4.5,2.5,4.5,2.0,3.2}\{4.5, 2.5, 4.5, 2.0, 3.2\} cm of the base (models 5, 6, 7, 8, 11)within the base’s contact radius, {7.15,4.27,6.78,2.41,4.83}\{7.15, 4.27, 6.78, 2.41, 4.83\} cm
Hmouse local yy within 45∘45^\circ of world +x+x (signed)mouse upright within 15∘15^\circ, heading free
Hkeyboard local +y+y within 45∘45^\circ of world +y+y+y+y or −y-y
Talarm clock: height onlyalso over the drawer footprint
Dat-res checkes in get_score void (state shared with run_reward)dropped from get_score
pack_objects_into_boxHobject long axis within 45∘45^\circ of the box’s long side; box heading within 40∘40^\circremoved (front within 60∘60^\circ of −x-x, box upright 15∘15^\circ kept)
play_tic_tac_toeDpiece within 1.5 cm of the cell centre, while the opponent’s moves count at 1.8 cm1.8 cm (run_reward and get_score)
pour_balls_into_vaseD“in vase”: xyxy hull of the oriented box and above the bottom (a tilted 27×24×28 cm box for vase/00002)inside the elliptic cylinder inscribed in the body, between bottom and rim
TaskTypeCorigC^{\mathrm{orig}}CfaithC^{\mathrm{faith}}
Hcup upright within 15∘15^\circ after pouringremoved
pour_by_languageDorder verified only when either arm reaches its originverified at every step
Heach bowl judged only at the step its bottle is back within 30∘30^\circ of uprightjudged at every step, bottle may stay tilted
Rbowl ≥95%\ge95\%, bottle ≤10%\le10\%bowl ≥90%\ge90\%, bottle ≤15%\le15\%
Treset of the arms (“Then reset the robot arm”) not verifiedall three bowls poured with both arms at the origin
pour_liquid_into_cupHjudged only at the step the bottle is back within 30∘30^\circ of uprightjudged at every step
Rcup ≥97%\ge97\% (bottle ≤15%\le15\%)cup ≥90%\ge90\% (bottle ≤15%\le15\%)
press_by_numberDpress counted when the cap goes from >0.95> 0.95 to <0.5< 0.5 of its travel, released at >0.9> 0.9 (a released cap rests near 0.75)from >0.65> 0.65 to <0.5< 0.5, released at >0.65> 0.65
Hleft red ×n0\times n_0, blue, middle red ×n1\times n_1, blue; blue exactly twice (contradicts “then press the blue button to confirm”)each red button exactly its card’s number, in any order, then blue once; any extra press or an early blue fails
solve_equationDposition, xx and yy orientation each satisfied by any candidate pieceall three by the same piece (as upstream 29910f3; 1.8 cm and 45∘45^\circ kept)
store_laptop_and_headphonesHheadset: whole orientation within 30∘30^\circ of the annotationheadset zz axis up within 30∘30^\circ, heading free (headband within 5 cm kept)
Hlaptop vertical: hinge up within 10∘10^\circany edge down (±x\pm x or ±y\pm y up within 10∘10^\circ)
Hin the stand: the front-edge midpoint of the baseany of the four edge midpoints
store_tools_in_toolboxRtool orientation within 10∘10^\circ of its slotwithin 15∘15^\circ; also in get_score
swap_blocksDbutton counting as in press_by_number (0.95/0.9)0.65
Hblock lifted 3 cm before each move (3 stages)removed
Tblocks may leave the swapped positions after the last pressstill swapped at the end
sweep_blocksTbroom never verified (“Pick up the broom”)lifted 5 cm above its start at some step
Rdustpan left of the broom by >0.1> 0.1 mby >0> 0
Rdustpan tilted <5∘< 5^\circ<20∘< 20^\circ
Hdustpan not lifted >1> 1 cmremoved
Tblocks over the pan in xyxy (a block under the pan passes)each block inside the pan in the dustpan’s frame

Table 6. Changes from CorigC^{\mathrm{orig}} to CfaithC^{\mathrm{faith}}. Type: D, defect fixed; H, hidden rule removed; R, hard threshold relaxed; T, loose criterion tightened. A _random variant receives the same change unless marked. Conditions not listed, including the return of both arms, are unchanged. Asset annotations are corrected in memory only. Liquid thresholds are shares of the poured liquid in the target and left in the bottle.

Online Dual-judged Results

Table 7 gives the per-task counts of § 4.2.1.

Task\multicolumn{4}{c}{w/ CE}\multicolumn{4}{c}{w/o CE}
n11n_{11}n10n_{10}n01n_{01}n00n_{00}n11n_{11}n10n_{10}n01n_{01}n00n_{00}
build_tower2000020000
fill_egg_holder4001650015
fill_pen_holder1100921116
hang_mugs_random7011210217
insert_key0002000020
make_kong2000000200
make_toast_random7201111018
organize_table8021012026
pack_objects_into_box1500510028
pour_by_language1900119001
pour_liquid_into_cup1503210028
press_by_number0200000191
store_laptop_and_headphones0002000218
store_tools_in_toolbox0002000020
swap_blocks0002000146
sweep_blocks_random1250370013
Total (320 each)13827614987264167
Successes, CorigC^{\mathrm{orig}} / CfaithC^{\mathrm{faith}}\multicolumn{4}{c}{165 / 144}\multicolumn{4}{c}{89 / 151}

Table 7. Table 7 The same episodes judged by both criteria, per task (seed 0, 20 layouts per task and library). n11n_{11}: success under CorigC^{\mathrm{orig}} and CfaithC^{\mathrm{faith}}; n10n_{10}: only under CorigC^{\mathrm{orig}}; n01n_{01}: only under CfaithC^{\mathrm{faith}}; n00n_{00}: under neither. Libraries of section 4.1.1.

Reading the Instruction-faithful Criteria

Configuration. Ten of the 16 rewritten criteria-limited tasks, two per RoboDojo dimension, learned with Criteria Exposure to CfaithC^{\mathrm{faith}} for five rounds: round 0 runs the empty library, and rounds 1–5 make one revision each. Each round runs the five development layouts 0–4 of seed 1, 300 episodes in all; a revision with at least two fewer successes than the best earlier round is rolled back, and no test runs during learning. Each episode is judged by CfaithC^{\mathrm{faith}}, which drives the loop, and by CorigC^{\mathrm{orig}} alongside, which is only logged. The actor is GPT-6 Astra with high reasoning effort; the learner is Claude Code with Claude Opus 5.5 and high reasoning effort, with 120 minutes and at most eight replays per session. The learner could read the development episodes of the parent revision (code cells, model turns, primitive calls, frames, prompt, verdicts and scores), the findings of earlier rounds, the PyRUA-Lean API sources, and the CfaithC^{\mathrm{faith}} criteria code of its own task. It was not given the CorigC^{\mathrm{orig}} criteria code, other tasks’ criteria or runs, the assets, or the sources and notes of the rewrite. The seed-0 test takes the library active after round 5 (for fill_pen_holder that of round 1, since rounds 2–5 were rolled back) and runs layouts 0–19 of seed 0 with the same actor, each episode judged by CfaithC^{\mathrm{faith}} and by CorigC^{\mathrm{orig}}.

TaskDim.CfaithC^{\mathrm{faith}}CorigC^{\mathrm{orig}}
sweep_blocks_randomGen.20/2020/20
hang_mugs_randomGen.4/202/20
insert_keyPrec.0/200/20
build_towerPrec.20/200/20
make_kongLong20/200/20
fill_pen_holderLong7/205/20
swap_blocksMem.20/200/20
press_by_numberMem.20/200/20
store_tools_in_toolboxOpen0/200/20
pour_by_languageOpen19/200/20
Total130/20027/200

Table 8. The library learned by reading CfaithC^{\mathrm{faith}}, judged by both criteria. Seed-0 test of the final library on 20 layouts per task, the same episodes judged by CfaithC^{\mathrm{faith}}, which the learner read, and by CorigC^{\mathrm{orig}}. Dim.: RoboDojo dimension (Generalization, Precision, Long-horizon, Memory, Open).

What Exposure Writes into the Library

Setup. We classify the memory entries of the ten source libraries of the RoboDojo transfer (section G), for both settings, by whether they cite the criteria code, and compare entries that the two settings wrote from the same source failures.

Results. Two patterns emerge. First, an exposed library writes its experience in the language of the criteria: 19 of the 55 exposed entries cite the criteria code (47% of the characters), and none of the 53 success-only entries does. Figure 6 shows three exposed entries: the grip-and-pull of the button cap in swap_blocks and the ban on sweeping in sweep_blocks_random, which cite the criteria code, and the hidden protocol of press_by_number, which restates the order of the checks in words.

Excerpts of three exposed memory entries

Figure 6. Exposed entries are written in the language of the criteria. Excerpts of three memory entries of the exposed source libraries, verbatim (“. . . ” marks elisions). The first fixes an order of presses that the instruction does not state, with the verification order of the criteria as evidence; the second turns a verified condition (“above 90%”) into a grip-and-pull of the button cap; the third tells the actor not to use the broom the instruction names (“pick up the broom, hand it over to the right hand”), because the verifier never looks at it. All three are marked dev_verified; the on-10 distillation dropped all three.

Such entries fail under CfaithC^{\mathrm{faith}} and have to be removed for transfer: distilling on-10 dropped 9 entries on “what the benchmark checks”. Second, a learner that cannot see the criteria has to find a physical cause. From the same source failures, in which the gripper closes on nothing, the success-only learner traced the cause to a half-open gripper whose fingers KEEP moves close again, and wrote “pass OPEN on every move of an empty gripper”; the exposed learner recorded the symptom and kept a formula that in effect half-opens the gripper (Figure 7). Not all exposed entries concern the verifier, though: some carry information the instruction should state such as the pen orientation in fill_pen_holder.

Two library entries and four bar charts comparing the empty, off-10 and on-10 libraries

Figure 7. Two library entries with the same number, from the same source failures. Top: the source entry each learner wrote on make_toast_random, where closes held nothing, and the grasp entry distilled from it. The success-only learner traces the empty closes to KEEP moves and off-10 generalises the cause; the exposed learner records the symptom and on-10 keeps a formula that opens the gripper only part way. Verbatim; “. . . ” marks elisions. Bottom: what the actor does with them on the ten target tasks, for the empty library, off-10 and on-10. Closes after a half-open gripper and KEEP moves grasp nothing in 28% of cases, closes after OPEN moves in 3%.

press_by_number, press_by_number-manipulate-001 (learner w/ CE)
do: Press in exactly this order: LEFT red xN_left (left arm) -> BLUE once (right arm) ->
MIDDLE red xN_middle (right arm) -> BLUE once more -> ... Blue is pressed exactly TWICE, never more. ...
evidence: task checks in order: repeat(button0 below 0.5 / above 0.9) x num0; button2 below 0.5; ...
swap_blocks, swap_blocks-verify-001 (learner w/ CE)
do: the benchmark judges after every action, in order: lift block 1 -> block 1 on the empty mat ->
cap pressed below 50% of its travel -> cap seen back ABOVE 90% -> ... a push alone ... does not
bring the cap back above 90%: the fingers must grip the cap and pull it up.
sweep_blocks_random, sweep_blocks_random-manipulate-001 (learner w/ CE)
do: do NOT pick up, hand over or sweep with the broom. The benchmark only checks, in one action:
every object’s centre inside the dustpan’s bin, dustpan not lifted and upright, broom’s x >
dustpan’s x + 0.10 m, both arms home. Carry the objects in by hand: ...

Library Transfer

To reuse libraries across tasks, we distill the libraries of ten source tasks into one shared library and apply it to disjoint target tasks; entries of exposed libraries that are tied to the criteria are removed during distillation. Across benchmarks, the libraries are applied on LIBERO-PRO and RoboCasa365 with primitives aligned to those of RoboDojo (Appendix § 16); within RoboCasa365 we also run both settings of the loop and distill one library from each, as a reference for in-benchmark experience. In RoboDojo, the success-only and exposed libraries off-10 and on-10 take ten learned tasks as sources and are applied to ten target tasks that no source covers (Table 2); off-rc10 and on-rc10 are distilled within RoboCasa365 from both settings of the loop. The case study is in Appendix § 14.

Brief. The distiller receives, for each source task, the library as the actor read it in the last evaluated revision (retired entries removed), the learner’s last diagnosis and the development successes per round, and the robot API source; it does not see the target tasks and is told not to look for them. It writes memory entries of at most 12,000 characters in the source format (stage, symptom, applies, do, evidence) and at most two skill modules of 15,000 characters. Every entry must state a general rule, name no task, object, dimension or goal configuration in do, and cite evidence from at least two source tasks, or from one task where it decided between failing and succeeding rounds, flagged as a single-task rule; a separate notes file, not given to the actor, records the source of every entry and what was left out. The two briefs differ in one passage on the learners: “they never saw the benchmark’s judging code” (w/o CE) against “A rule that only describes how one task is judged (its thresholds, its check order) does not transfer; keep what is about perception and manipulation” (w/ CE).

Process. Each library is distilled in one session by Claude Code with Claude Opus 5.5 (effort high). Before evaluation, every library is checked for the character limits, the entry fields, source task names, and that each library function it names exists and parses; skills are exercised against a stub robot with the real API signatures, not in simulation.

Results. Table 9 gives the ten target tasks. Off-10 leads on-10 by 9 episodes, and most of the difference sits in insert_tubes and solve_equation (Appendix § 14).

TaskBudgetSuccesses L∅L_{\varnothing}Successes off-10Successes on-10OPEN (%) L∅L_{\varnothing}OPEN (%) off-10OPEN (%) on-10Re-measure actions L∅L_{\varnothing}Re-measure actions off-10Re-measure actions on-10Episodes off/on
fasten_screws19001112123889{sup}†0{sup}†173230{sup}†227{sup}†9/5
fold_clothes_random500610955993551920/20
hang_mugs_random80021039100{sup}†6{sup}†114148{sup}†147{sup}†2/4
insert_key3000002897333577620/20
insert_tubes5001211638955405711620/19
play_tic_tac_toe11001216163890019509720/20
pour_balls_into_vase600811123975144567418/18
solve_equation3001419152797233355620/20
store_laptop_and_headphones8000012282{sup}†1{sup}†6368{sup}†145{sup}†9/5
store_tools_in_toolbox9000002995{sup}†63{sup}†140152{sup}†141{sup}†9/1
Total65/20080/20071/20035913–––147/132

Table 9. The ten target tasks with the empty library, off-10 and on-10. Successes under CorigC^{\mathrm{orig}} on layouts 0–19 of seed 0. OPEN: share of move_pose calls of an empty gripper that pass gripper=OPEN among those passing OPEN or KEEP. Re-measure: actions per episode spent on moves that are directly followed by a wrist-camera perception call. OPEN and re-measure use only the first run of each layout; episodes run again are excluded, so off-10 and on-10 have the number in the last column and the empty library 20 per task. Values marked † rest on fewer than ten episodes.

Case Study: What Carries Over within RoboDojo

Setup. We follow the ten-source distillation of § 4.3.1, whose success-only and exposed libraries we call off-10 and on-10, on the ten target tasks. For each episode we count the gripper arguments passed to primitives, the moves made to re-measure with a wrist camera, and the use of the action budget.

Results. General rules carry over and single-task precision does not. Both libraries hold a grasp entry with the same number, drawn from the same source failures (Figure 7). The success-only learner traced the failures to their cause, fingers that KEEP moves close again, and wrote a general rule; the exposed learner kept a formula that half-opens the gripper. The actor follows each grasp entry, passing OPEN on 35%, 91% and 3% of empty-gripper moves with the empty library, off-10 and on-10, and on solve_equation, which needs a thin disc picked up, off-10 solves 19/20 layouts against 14 and 15. On-10 instead asks to re-measure to 1 mm before most actions; on insert_tubes it spends 116 of 500 actions per episode on re-measuring (40 empty, 57 off-10) and solves 6/20 against 12 and 11.

On every task, on-10 makes the most re-measure moves of the three libraries; in actions it spends the most on seven of the ten tasks, and off-10 slightly more on fasten_screws, hang_mugs_random and store_tools_in_toolbox, where off-10 and on-10 have few such episodes. Of the 14 failed on-10 episodes of insert_tubes, 9 used at least 95% of the budget, and its successful episodes needed 442 actions on average against 397 (empty) and 392 (off-10).

Two layouts show the difference step by step. On layout 4 of solve_equation, a thin disc has to be placed on a pad within 300 actions: off-10 passes OPEN on every move of an empty gripper and holds the disc at its first close (success after 125 actions), while on-10 descends half open with KEEP, all five of its closes grasp nothing, and it ends after 292 actions with the pad empty. Layout 19 of insert_tubes is in Figure 8.

Four-column, three-row sequence of robot-arm images showing action 120, action 250, action 330, and last action for the empty library, off-10, and on-10

Figure 8. insert_tubes, layout 19: re-measuring uses up the budget. Three tubes go into separate holes of the rack within 500 actions. Head camera after the given action, rows the empty library, off-10 and on-10; the last column is each episode’s last action (352, 388, 499). At action 330 the empty library inserts its third tube while on-10’s first tube lies on the table; at its last action on-10 has all three tubes in the rack but has not withdrawn the arm.

LibraryResultRecord
L∅\mathcal{L}_{\varnothing}success, 352No re-measure move: inserts from the head-camera estimates, releasing the tubes at actions 123, 213 and 327; judged a success during the return home.
off-10success, 388One re-measure move (25 actions); releases at actions 115, 249 and 373, 3 blocked descents; judged a success as the arm withdraws.
on-10fail, 4999 re-measure moves (126 actions) and wrist triangulation (triangulate, as asked by place-008). The first tube falls out after its release at action 100; the other two are seated at actions 215 and 307; the fallen tube is grasped again at action 370 and seated at action 478. The budget runs out at action 499, before the arms are home.

Table 8 (PDF p. 21). Library Result Record

Why Cross-benchmark Libraries Help Little

Setup. The libraries off-10 and on-10 of Appendix § 14 are translated to the prompt and primitives of LIBERO-PRO and RoboCasa365; off-rc10 and on-rc10 are distilled from RoboCasa365 tasks disjoint from its ten target tasks. At the start of an episode the library is placed in the actor’s working directory: a skill runs only if the actor’s code imports it, while the memory entries are appended to the prompt. We count the episodes whose code imports or calls a skill.

Results. Three reasons explain why neither setting’s library gives a clear gain across benchmarks.

First, the translated skills are almost never called. Even within RoboDojo only 5% (off-10) and 20.5% (on-10) of the episodes call a skill. None of the 200 RoboCasa365 episodes with off-10 or on-10 calls one, and on LIBERO-PRO only 5 and 12 of 60 episodes do. The libraries distilled within RoboCasa365 are called in 158 of 200 episodes, mostly their perception functions for two-view localization and joint margins. Across benchmarks, the memory entries appended to the prompt are the only part of a translated library the actor uses.

Second, what the translated libraries hold is not what the targets need. They cover grasping from above, placing and withdrawing at a table; translation removed the content about two arms, hand-overs and the table geometry, and found no source experience for moving the base. Most of the ten RoboCasa365 target tasks open doors, turn knobs, pull drawers or reach into racks, and need the base to move. The libraries distilled within RoboCasa365 cover exactly these: moving the base in front of a door, swinging a hinged part about its axis, approaching under an overhang, and recovering when the arm is near its joint limits. The perception experience does not carry either: the millimetre-level precision learned in RoboDojo does not hold on RoboCasa365, where the measured height of an object top is off by 2–8 cm.

Third, LIBERO-PRO leaves little room. With the aligned primitives, the empty library already solves 188 of 200 episodes on the four swap suites, and only six of the 40 tasks are not solved on every seed. On four of these six every library solves 8–10 of 10 seeds; the differences come from two tasks, putting a book into the back compartment of a caddy (empty, off-10, on-10: 6 / 8 / 5 of 10) and putting a mug into the microwave and closing it (2 / 3 / 3). The same library solves different numbers on the two halves of the seeds (empty 21 / 24, off-10 22 / 26, on-10 25 / 21, of 30 each), a spread larger than the gain of any library.

Interpretation. Transfer helps where source and target share the bottleneck. Within RoboDojo the robot, the table and perception stay the same, and off-10 gains on target tasks. Across benchmarks, the general part of the experience (grasping, placing, finishing) is what the target’s prompt and the actor already handle, and what the target lacks, moving the base and operating articulated parts, has no counterpart in the source. A library distilled on the target benchmark, even from other tasks, covers that bottleneck, is called in most episodes, and helps. Two caveats apply: a skill that is never called does not mean the library has no effect, since its memory entries are always in the prompt; and we evaluated one way of translating a library, not variants such as skills without memory.

Cross-benchmark Primitive Alignment

With the aligned primitives, the empty library solves 41, 47, 50 and 50 of 50 episodes on the four LIBERO-PRO swap suites (10, goal, object, spatial), against 36, 45, 34 and 39 for the published PyRUA-Lean [29]; on RoboCasa365 it solves 55/90 atomic and 54/160 composite episodes, against 59/90 and 32/160 published. Table 10 summarises the comparison.

BenchmarkSuiteAlignedPublished
LIBERO-PROlibero_10_swap41/5036/50
LIBERO-PROlibero_goal_swap47/5045/50
LIBERO-PROlibero_object_swap50/5044/50
LIBERO-PROlibero_spatial_swap50/5039/50
RoboCasa365atomic55/9059/90
RoboCasa365composite54/16032/160

Table 10. Table 10 Primitive alignment. Successes of the empty library with the primitives aligned to RoboDojo, against the published no-VLA PyRUA-Lean code agent on the same tasks and seeds [29].

On LIBERO-PRO, 34 of the 40 tasks are solved on all five seeds with the empty library; the other six are the tasks of § 4.3.2.

References

  1. [1]Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, et al. Do as I can, not as I say: Grounding language in robotic affordances. In Conference on Robot Learning (CoRL), 2022.arxiv.org/abs/2204.01691
  2. [2]Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. Concrete problems in AI safety. arXiv preprint arXiv:1606.06565, 2016.arxiv.org/abs/1606.06565
  3. [3]Jingwen Bai, Guochao Bai, Canbo Zhang, Shuangqi Li, Minchao Hu, Weichun Huang, and Shuang Liu. Compile Once, Run Cheap: Frozen, Portable Skills for Embodied Agents, and What Recursive Self-Improvement Adds. arXiv preprint arXiv:2610.25114, 2026.arxiv.org/abs/2610.25114
  4. [4]Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, et al. π0: A vision-language-action flow model for general robot control. In Robotics: Science and Systems (RSS), 2025.DOI
  5. [5]Alexander Bondarenko, Denis Volk, Dmitrii Volkov, and Jeffrey Ladish. Demonstrating specification gaming in reasoning models. arXiv preprint arXiv:2502.13295, 2025.arxiv.org/abs/2502.13295
  6. [6]Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), 2023.arxiv.org/abs/2307.15818
  7. [7]Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, et al. SAM 3: Segment anything with concepts. arXiv preprint arXiv:2511.16719, 2025.arxiv.org/abs/2511.16719
  8. [8]Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, et al. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. arXiv preprint arXiv:2506.18088, 2025.arxiv.org/abs/2506.18088
  9. [9]Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su, Haoran Lu, Weijie Wan, Baijun Chen, Songling Liu, Haowen Yan, et al. Robodojo: A unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. arXiv preprint arXiv:2607.04434, 2026.arxiv.org/abs/2607.04434
  10. [10]Yitong Chen, Zezheng Huai, Sixian Li, Yubang Wang, Haozhe Zhang, Yifei Zhang, Hechang Chen, Jingjing Gong, Yu-Gang Jiang, and Xipeng Qiu. ETA: A New Agentic Paradigm for Embodied Tasks. arXiv preprint arXiv:2608.03924, 2026.arxiv.org/abs/2608.03924
  11. [11]Yuqing Du, Ksenia Konyushkova, Misha Denil, Akhil Raju, Jessica Landon, Felix Hill, Nando de Freitas, and Serkan Cabi. Vision-Language Models as Success Detectors. arXiv preprint arXiv:2303.07280, 2023.arxiv.org/abs/2303.07280
  12. [12]Letian Fu, Justin Yu, Karim El-Refai, Ethan Kou, Haoru Xue, Huang Huang, Wenli Xiao, et al. CaP-X: A framework for benchmarking and improving coding agents for robot manipulation. In International Conference on Machine Learning (ICML), 2026.arxiv.org/abs/2603.22435
  13. [13]Hongcheng Gao, Alex Zhou, Jay Zhu, Zelin Zheng, Shijia Ge, Xiping Li, Di Wu, Yazhe Wang, Jianshu Zeng, Xuan Shangguan, Yongli Wang, Yexing Wan, Lingyu He, Zhiqi Jia, Sihang Wu, and Xiao He. HexaShell: A Thin Harness for Exploring the Limits of VLM Agents as Robot Coding Agents. arXiv preprint arXiv:2610.23479, 2026.
  14. [14]Runhan Huang, Xiangcheng Zhang, Haoyu Chen, Yitong Li, Yen-Jen Wang, Bohan Lyu, Haotian Ye, Chuye Hong, Yixuan Wang, Chloe Su, Haoyu Xiong, Yangzhen Wu, Enze Xie, Travis Manderson, Stefano Ermon, Pieter Abbeel, Antonio Torralba, Dawn Song, Weirui Ye, and Yilun Du. Scaling online exploration for robot agents. arXiv preprint arXiv:2610.22496, 2026. URL https://arxiv.org/abs/2610.22496.
  15. [15]Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei. VoxPoser: Composable 3D value maps for robotic manipulation with language models. In Conference on Robot Learning (CoRL), 2023.arxiv.org/abs/2307.05973
  16. [16]Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, Nitya Nadgir, and Arvind Narayanan. AI agents that matter. arXiv preprint arXiv:2407.01502, 2024.arxiv.org/abs/2407.01502
  17. [17]Moo Jin Kim, Karl Pertsch, Siddharth Sidhdharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024.arxiv.org/abs/2406.09246
  18. [18]Wenyi Li, Yizhe Chi, Youjie Zheng, Chunyu Liu, Xiaoqiu Wang, Mingju Gao, Zheng Jiang, Deyao Hong, Calvin Xiao, Kaisen Yang, and Qinhuai Na. EmbodiedEvo: Self-evolving coding agent for physical intelligence. arXiv preprint arXiv:2610.18867, 2026. URL https://arxiv.org/abs/2610.18867.
  19. [19]Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In IEEE International Conference on Robotics and Automation (ICRA), 2023.arxiv.org/abs/2209.07753
  20. [20]Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2023.arxiv.org/abs/2306.03310
  21. [21]Runyu Lu, Yubo Wu, Ethan Kou, Letian Fu, Wenli Xiao, Ajay Mandlekar, Yinzhen Xu, Guanya Shi, et al. ASPIRE: Agentic / skills discovery for robotics. arXiv preprint arXiv:2607.00272, 2026.arxiv.org/abs/2607.00272
  22. [22]Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddikuri, and Yuke Zhu. RoboCasa365: A large-scale simulation framework for training and benchmarking generalist robots. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2603.04356
  23. [23]Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131, 2025. URL https://arxiv.org/abs/2506.13131.
  24. [24]Alexander Pan, Kush Bhatia, and Jacob Steinhardt. The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models. arXiv preprint arXiv:2201.03544, 2022.arxiv.org/abs/2201.03544
  25. [25]Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, et al. π₀.₅: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025.arxiv.org/abs/2504.16054
  26. [26]Qiuhong Shen, Shihua Zhang, Yue Liao, Qi Li, Zhenxiong Tan, Shizun Wang, Shuicheng Yan, and Xinchao Wang. World Action Models: A Survey. arXiv preprint arXiv:2606.20781, 2026.arxiv.org/abs/2606.20781
  27. [27]Zeyu Shen, Haoxiang You, Yilang Liu, Zhicheng Zheng, Lihan Zha, Kashy Yamasaki, Mingtong Zhang, Suning Huang, Jiankai Sun, Qianzhong Chen, Lucy He, Kaiyuan Liu, Haoran Chang, Katerina Fragiadaki, Dhruv Shah, Mac Schwager, Peter Henderson, Ian Abraham, and Canwen Xu. EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics. arXiv preprint arXiv:2609.27308, 2026.arxiv.org/abs/2609.27308
  28. [28]Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. arXiv preprint arXiv:2303.11366, 2023.arxiv.org/abs/2303.11366
  29. [29]Ruiyang Si, Jianxin Bi, Shunyu Yang, Rui Ni, Wenbo Huang, Qiang Wang, Shulong Jiang, Yintianrun Zhang, Feixiang Ruan, Kewei Zhang, Duoqin Wang, Xiuyu Li, Haiwen Feng, Zhen Dong, and Daquan Zhou. Fewer tokens, better action: GPT-6 Astra robot agents with 14% higher success rate but 65% fewer tokens. arXiv preprint arXiv:2610.01939, 2026.arxiv.org/abs/2610.01939
  30. [30]Joar Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward hacking. arXiv preprint arXiv:2209.13085, 2022.arxiv.org/abs/2209.13085
  31. [31]Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023.arxiv.org/abs/2305.16291
  32. [32]Xin Wang, Wenhao Wu, Menghao Zhang, Zhi Wang, Kun Shao, Jian Luan, Yang Li, Qing Li, Shangding Gu, Huichi Zhou, Shuqing Shi, Fei Ni, Shuo Lu, Weichen Meng, Kang Li, Jin Wu, Kang Zhao, Shangmin Guo, Gen Li, Yongqiang Tang, Zhizhong Zhang, Yuan Xie, and Heng Qu. HarnessPAI: An Evolving Harness for Physical AI. arXiv preprint arXiv:2609.29166, 2026.arxiv.org/abs/2609.29166
  33. [33]John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2405.15793
  34. [34]Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel machine: Open-ended evolution of self-improving agents. arXiv preprint arXiv:2505.22954, 2025.arxiv.org/abs/2505.22954
  35. [35]Junyi Zhang, Jiaxin Ge, Hanjun Yoo, Letian Fu, Zihan Yang, Yaowei Liu, Raj Saravanan, Shaofeng Yin, Justin Yu, Dantong Niu, Zirui Wang, Roei Herzig, Ken Goldberg, Yutong Bai, David M. Chan, Ion Stoica, Angjoo Kanazawa, Jiahui Lei, Haiwen Feng, and Trevor Darrell. Playful Agentic Robot Learning. arXiv preprint arXiv:2606.19419, 2026.arxiv.org/abs/2606.19419
  36. [36]Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners. In AAAI Conference on Artificial Intelligence (AAAI), 2024.arxiv.org/abs/2308.10144
  37. [37]Ziqian Zhong, Aditi Raghunathan, and Nicholas Carlini. ImpossibleBench: Measuring LLMs’ Propensity of Exploiting Test Cases. arXiv preprint arXiv:2510.20270, 2025.arxiv.org/abs/2510.20270
  38. [38]Xueyang Zhou, Yangming Xu, Guiyao Tie, Yongchao Chen, Guowen Zhang, Duanfeng Chu, Pan Zhou, and Lichao Sun. LIBERO-PRO: Towards robust and fair evaluation of vision-language-action models beyond memorization. arXiv preprint arXiv:2510.03827, 2025.arxiv.org/abs/2510.03827
  39. [39]Zhiyuan Zhou, Pranav Atreya, You Liang Tan, Karl Pertsch, and Sergey Levine. AutoEval: Autonomous Evaluation of Generalist Robot Manipulation Policies in the Real World. arXiv preprint arXiv:2503.24278, 2025.arxiv.org/abs/2503.24278
  40. [40]Yuxuan Zhu, Tengjun Jin, Yada Pruksachatkun, Andy Zhang, Shu Liu, Sasha Cui, Sayash Kapoor, Shayne Longpre, Kevin Meng, Rebecca Weiss, Fazl Barez, Rahul Gupta, Jwala Dhamala, Jacob Merizian, Mario Giulianelli, Harry Coppock, Cozmin Ududec, Jasjeet Sekhon, Jacob Steinhardt, Antony Kellermann, Sarah Schwettmann, Matei Zaharia, Ion Stoica, Percy Liang, and Daniel Kang. Establishing Best Practices for Building Rigorous Agentic Benchmarks. arXiv preprint arXiv:2507.02825, 2025.arxiv.org/abs/2507.02825

Paper details

vlnc:W_8nry2uv9
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 12

downloads · 1

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents