WorksComputer Science
Robotic Reward Models in the Foundation-Model Era
Abstract
Robotic reward models (RRMs) use foundation models to replace per-task reward engineering with reusable, task-conditioned evaluation. Yet under one name, their outputs answer different questions, from whether a task succeeded to which execution is better. This survey organizes 114 methods into five semantic families by the question their native outputs answer: Outcome, Absolute Progress, Relative, Temporal Value, and Hybrid. Within this taxonomy, we review how model design determines what an evaluator can judge, how raw experience becomes supervision, and how existing evaluation practice tests the result. The review exposes a gap: agreement with demonstration order and downstream policy gains do not show whether a reward tracks task fulfillment. We therefore introduce RRM-Bench-1K, 1,117 real-robot videos of closed-loop executions with task-state and camera-evidence labels, and four contrast suites that separate the credited condition from its usual cues. A unified study of 27 evaluators shows that the strongest final scores rank whole executions well enough for selection and filtering. Dense credit, however, follows the progression of an execution more closely than its task state, and no evaluator signals when the decisive view is missing. How much fidelity a reward model needs therefore depends on the decision it drives. We outline directions from real-failure supervision to metrics that separate task state from chronology, and open-source RRM-Hub, a toolbox in which 14 existing RRMs can be readily applied and new evaluators tested on the same capabilities.
Introduction
Foundation models let robot policies address more tasks and environments, as vision–language–action (VLA) models such as [1] and [2] illustrate. Every task a policy attempts must also be judged: improving the policy, selecting its data, and monitoring its execution all turn on whether the robot’s behavior satisfies the task. Conventional reward design supplies this judgment task by task, with manually specified success criteria and shaping terms [3] (Figure 1(a)), an effort that grows with every task a general policy adds. Robotic reward models (RRMs) replace it with reusable, task-conditioned evaluators that use foundation models as representations, supervisors, or judges to assess robot behavior against a task specification (Figure 1(b)).

Figure 1. From per-task reward engineering to task-conditioned reward modeling. (a) Each task receives its own hand-designed reward, a distance to the target plus a task-specific success condition, such as the bowl resting on the plate with the gripper open. (b) One shared evaluator maps visual evidence and instruction to a reward for every task.
Early RRMs derived rewards from pretrained foundation models. ZSRM [4], RoboCLIP [5], and VLM-RM [6] score the compatibility between observations and a task description or demonstration, and use it as a reward or threshold it into a binary goal reward. RL-VLM-F [7] distills pairwise judgments of a vision–language model (VLM) into a task reward. Recent evaluators such as ReWiND [8], Robo-Dopamine [9], RoboReward [10], and Robometer [11] are trained on multi-task robot data with supervision from successful and failed executions, partial completion, temporal relations, and trajectory preferences. They now serve several uses: stage and progress estimation (SARM [12]), detection of planning and execution errors (Guardian [13]), value estimation for reinforcement learning (ViVa [14]), and candidate-action scoring and selection (Rover [15]).
Under the common name reward, these evaluators return success probabilities, completion percentages, preference utilities, expected returns, and structured diagnoses. Each answers a different question: a success probability says whether the task is done, a preference says which execution is better, and an expected return says what the policy can still achieve. These differences matter once a prediction becomes a control signal: ranking candidates, shaping a policy, and deciding when to stop place different demands on the same score. We therefore organize RRMs by the question their native output answers, where a native output is the quantity a model returns before any downstream transformation. The questions define five semantic families: Outcome, Absolute Progress, Relative, Temporal Value, and Hybrid (§ 2.2; Figure 2).

Figure 2. Taxonomy of the 114 methods in Table S1 by semantic family and design route (§ 3.2–§ 4.3), with the number of methods per family in parentheses. * evaluates generated behavior; ‡ marks a protocol or harness.
These output differences call for different tests of evaluator quality, and the common tests answer narrower questions than they are read to answer. Agreement with frame order on successful demonstrations [16] asks whether later frames score higher, a condition that any score rising with time also meets; it does not establish whether the score tracks task fulfillment when progress stalls or is reversed [17], [18]. Downstream policy performance reflects the evaluator, learning algorithm, and control system together [18]. Neither test separates task fulfillment from the cues that usually accompany it: motion, a completed manipulation, elapsed time, and a view that shows the outcome. On a real robot, Tian et al. Tian et al. [19] find that learned reward models over-reward unsafe or only superficially task-like behavior that human evaluators reject, and CriticHack [20] shows that optimizing a policy against a frozen learned reward can raise both task success and wrong-object failures.
Current benchmarks rarely contain the executions in which these cues part from the task. Four such cases arise in closed-loop executions (Figure 8): (A) motion without its effect, (B) a wrong destination or another broken constraint, (C) an achieved condition that is later lost, and (D) a missing decisive view [9]. A test set sampled from natural executions holds few instances of the first three and leaves the fourth unlabeled (§ 6.1), and no existing RRM benchmark draws all its failures from natural executions or presents one execution under both sufficient and reduced views (Table 1). RRM-Bench-1K therefore builds a contrast suite for each case from real executions: (A) and (B) probe proxy validity, (C) temporal validity, and (D) evidence validity. Each suite fixes an ordering or an abstention between paired judgments rather than a target value, so every family is tested through its native output.
| Benchmark | Measurement target | Scale | Natural failures | Regression labels | Graded endpoints | Paired evidence | Scope variants |
| ManiRewardBench [53] | Stage progress; success detection | 653 episodes; 130 tasks | ◐ | – | ◐ | – | – |
| RBM-EVAL [11] | Progress alignment; ranking | 571 OOD videos | ◐ | – | ● | – | – |
| PRIMO Bench [118] | Progress under task and scene shift | 18k samples; humanoid set | – | – | – | – | – |
| RoboRewardBench [10] | Episode-end progress grade (1–5) | 2,831 test items | ◐ | – | ● | – | – |
| RoboRMBench [159] | Paraphrase invariance of rewards | 2,390 trajectories | ◐ | – | ● | – | – |
| Progress-Bench [52] | Progress from a demo; answerability | >3k instances | – | – | – | ◐ | – |
| OpenGVL [160] | Temporal order of shuffled frames | 200 trajectories; 4 datasets | – | – | – | – | – |
| RoboPulse [126] | Pairwise progress discrimination | 1,800 pairs; 816 tasks | – | ◐ | – | – | – |
| Suboptimal-Value-Bench [18] | Value under hesitation and retry | 800 trajectories; 15 tasks | ◐ | ● | – | – | – |
| RoboPulse++ [161] | Interval-level progress direction | 2,244 intervals | ◐ | ● | – | – | – |
| FailBench [162] | Binary success detection | 2,197 attempts; 14 sources | ◐ | – | ◐ | – | – |
| RoboValue [163] | Grounding, progress, recovery, and subtask credit | 2,792 test trajectories; 15 sim + 20 real tasks | – | ● | ◐ | – | – |
| RRM-Bench-1K (ours) | Fulfillment, retraction, and evidence | 1,117 videos | ● | ● | ● | ● | ● |
Table 1. RRM-Bench-1K and existing benchmarks for robotic reward models. Scale follows each source’s unit. ●: yes; ◐: partial; –: no.
Figure 8. The four contrast suites of RRM-Bench-1K on real robot executions. Each panel names the critical evidence, the expected behavior, and the gold relation; the count after each suite name is its number of videos. (A) and (B) test proxy validity, (C) temporal validity, and (D) evidence validity.
Relying on a reward model takes three answers: what its output credits, whether its evidence suffices, and which decisions it can drive. This article is organized around them. Five families answer the first, four suites test the first two, and the third depends on the decision: the fidelity it needs grows with the optimization pressure it puts on the reward.
Adjacent surveys cover foundation models in robotics [21], robotic video world models [22], VLA models [23], reward models in deep reinforcement learning [24], and reward and process reward models for large language models [25], [26]. Concurrent surveys organize progress reward models for robot learning [27] by input–output interface, signal construction, and data and benchmarks, and verifiers for robot policies [28] by who supplies the judgment. Unlike these surveys, this article spans all five families, organizes them by the question their native outputs answer, and evaluates them empirically on the proposed RRM-Bench-1K.
This article makes three contributions:
A survey and semantics-first taxonomy of foundation-era RRMs. We organize 114 methods by semantic family and design route, explaining which quantity each route estimates and which behavior differences change its output. Recording each method’s output, query evidence, learning mechanism, and training data (Table S1) shows that one semantics arises from several mechanisms and that the evidence interface bounds what a result can establish.
RRM-Bench-1K and a unified empirical study. We annotate 1,117 real-robot videos of closed-loop executions on five embodiments, from up to three cameras, with task-state labels for the conditions that hold at each moment and camera-evidence labels for whether a view suffices to judge the task. Four contrast suites separate motion from its effect, a satisfied constraint from a broken one, an achieved condition from one later lost, and a decisive view from its absence. A unified protocol evaluates 27 evaluators on these suites and measures endpoint judgment, dense credit, and evidence use separately instead of merging them into one score (Table 3). The gaps it exposes motivate six research directions (§ 8).
RRM-Hub: an open-source toolbox. Common adapters for 14 existing RRMs preserve each model’s native outputs and supported inputs, so they can be applied readily and new evaluators tested under the same protocols. Stored raw outputs and per-capability reports make comparisons reproducible.
| Model | VOC | Falling F1 | V | P/X | |
| Specialized evaluators, 2B–4B | |||||
| ProcVLM-2B [56]† | 0.382 | 0.175 | 0.201 | 0.469 | 0.407 |
| VLAC-2B [109]† | 0.370 | 0.093 | 0.171 | 0.357 | 0.331 |
| Robo-Dopamine-GRM-3B [9] | 0.324 | 0.113 | 0.202 | 0.399 | 0.356 |
| Robo-Dopamine-GRM-2.0-4B-Preview [9] | 0.598 | 0.311 | 0.242 | 0.514 | 0.452 |
| Robometer-4B [11]† | 0.832 | 0.473 | 0.214 | 0.582 | 0.527 |
| RoboReward-4B [10]† | 0.807 | 0.314 | 0.072 | 0.254 | 0.189 |
| RynnValue-4B [94]† | 0.459 | 0.419 | 0.235 | 0.572 | 0.450 |
| Specialized evaluators, 7B–8B | |||||
| PRIMO-R1-7B [118]† | 0.828 | 0.285 | 0.154 | 0.431 | 0.426 |
| LRM-8B [110] (Progress) | 0.198 | 0.157 | 0.136 | 0.158 | 0.215 |
| Robo-Dopamine-GRM-8B [9] | 0.389 | 0.263 | 0.211 | 0.442 | 0.388 |
| Robo-Dopamine-GRM-2.0-8B-Preview [9] | 0.605 | 0.280 | 0.226 | 0.504 | 0.447 |
| RynnValue-8B [94]† | 0.522 | 0.390 | 0.236 | 0.570 | 0.441 |
| SOLE-R1-8B [55] | 0.683 | 0.196 | 0.225 | 0.494 | 0.498 |
| TOPReward [53] (Qwen3-VL-8B)† | 0.379 | 0.081 | 0.182 | 0.434 | 0.427 |
| VLAC-8B [109]† | 0.414 | 0.120 | 0.169 | 0.366 | 0.358 |
| Specialized evaluators, 27B–30B | |||||
| Flora-Rewarder-27B [116] | 0.915 | 0.572 | 0.252 | 0.551 | 0.511 |
| VLAC-Cut-30B-A3B [119] | 0.695 | 0.261 | 0.272 | 0.588 | 0.561 |
| Specialized evaluators, proprietary backbone | |||||
| GVL [16] (Gemini 3.8 Flash) | 0.788 | 0.271 | 0.240 | 0.553 | 0.506 |
| General-purpose VLMs, Sequence style | |||||
| GPT-6.1 Sol [190] | 0.898 | 0.695 | 0.366 | 0.559 | 0.491 |
| GPT-6 Luna [191] | 0.805 | 0.444 | 0.252 | 0.466 | 0.337 |
| GPT-5.6 Terra [192] | 0.878 | 0.632 | 0.217 | 0.389 | 0.294 |
| Claude Opus 5.5 [193] | 0.929 | 0.646 | 0.354 | 0.618 | 0.557 |
| Claude Sonnet 5.5 [194] | 0.889 | 0.559 | 0.184 | 0.493 | 0.474 |
| Claude Haiku 5.5 [195] | 0.807 | 0.332 | 0.122 | 0.393 | 0.369 |
| Gemini 3.8 Flash [196] | 0.891 | 0.601 | 0.305 | 0.560 | 0.531 |
| Decision model, Sequence style | |||||
| Jev 1.14 [188] | 0.905 | 0.496 | 0.256 | 0.534 | 0.525 |
Table 3. Conventional and direction metrics of the single-view evaluators. V and P/X give Macro-F1 under sufficient and reduced camera evidence on the same transitions. Bold marks the best result in each panel. †Curve interpolated to 1 FPS.
| Method / component | Native output | Query evidence | Learning mechanism | Training / data source |
| OUTCOME (12 methods) | ||||
| ZSRM (binary variant) [4] | Binary goal condition | Image + Task | CLIP adaptation + spatial labels | Exploration images + descriptions |
| SuccessVQA (Flamingo detector) [29] | Binary success answer | Video + Task | Flamingo VQA fine-tuning | Human success annotations |
| RoboCLIP [5] | Episode compatibility | Traj. + Ref./Task | Frozen video–text alignment | No RRM training |
| RoboFuME (reward classifier) [30] | Binary completion | Image + Task | MiniGPT-4 fine-tuning | Demo endpoints + unsuccessful images |
| Sentinel (VLM monitor)‡ [33] | Failure forecast + rationale | Video + Task (+ Ref./Goal) | Prompted chain-of-thought video QA | Frozen VLMs; no training |
| SAFE [34] | Running failure score | Latent + Hist. | MLP/LSTM probe + functional conformal prediction | Successful/failed VLA rollouts |
| I-FailSense [31] | Binary success/failure | Video + Task | Failure classification | Demos + synthetic data |
| RoboReward [10] | End-episode progress (1–5) | Video + Task | Progress supervision | Human-scored rollouts + augmented demos |
| World-VLA-Loop* [32] | Success score | Video + Init. + Action + Future | Reward regression | Demos + binary outcomes |
| VLAConf [35] | Calibrated success confidence | Latent + Hist. | Support estimation + calibration | Success demos + outcome labels |
| OpenVLA warning probe [36] | Failure probability | Latent | Failure classification | Rollout latents + outcomes |
| FAVOR (Anchor Verifier) [37] | Deviation-event probability | Image + Ref. + Act. hist. | DINOv2 + local labels + episode-level multiple-instance learning | Clean/perturbed rollouts + outcomes |
| ABSOLUTE PROGRESS (29 methods) | ||||
| VIP [38] | Goal distance | Image + Goal | Implicit value | Human video |
| LIV [39] | Value | Image + Task + Goal | Language–image value | Human video |
| VLM-RM [6] | Compatibility | Image + Task | Frozen image–text similarity | No RRM training |
| Adapt2Reward [40] | Compatibility | Video + Task | Failure-prompt alignment | Human + robot-failure video |
| Video-Language Critic [44] | Clip value | Video + Task | Contrastive temporal rank | Auto time labels |
| GVL [16] | Completion % | Video + Task | Prompted frame order | Frozen VLM |
| PROGRESSOR [45] | Progress distribution | Image + Init. + Goal | Progress regression + RL | Demonstrations |
| ReWiND [8] | Progress | Video + Task | Progress + rewind regression | Demonstrations + Open X-Embodiment |
| VITA [46] | Normalized progress | Image + Task + Hist. | CLIP adaptation + regression | Demonstration time labels |
| SPARTA-L (reward) [41] | Object-state progress | Video + Task | VLM region labels + tracking | No RRM training |
| World-Env (ReachVLA)* [60] | Completion probability | Video + Task + Future | Success classification | Simulator frames |
| ProgressLM [52] | Progress / unanswerable | Image + Ref./Task | VLM SFT + GRPO | ProgressLM-45K |
| TOPReward [53] | Progress (log-prob) | Video + Task | Truth-token probability | Frozen video VLM |
| Rewarding DINO [42] | Reward | Multi-view + Task | Reward ranking | Analytic labels + frozen DINOv3 |
| R²VLM [54] | Completion % + rationale | Video + Task + Hist. | Qwen2.5-VL SFT + multi-turn RL | ALFRED + Ego4D process labels |
| ProgressVLA [61] | Progress | Image + Init. + Task + Future | Temporal regression | Robot video |
| SOLE-R1 [55] | Progress + rationale | Video + Task + Init. | SFT + RLVR | Synthetic process reasoning |
| ProcVLM [56] | Progress + proc. rationale | Video + Task | Procedure-grounded regression | Demos + synthetic labels |
| Feat2Go [57] | Binned structural progress | Image + Task | Qwen3-VL progress learning | Subgoal/feature-derived labels |
| NavWAM [62] | Progress value | Image + Goal + Proprio. | Joint progress diffusion | Simulated + real navigation |
| RARM [47] | Confidence-gated progress | Video + Reference | Temporal contrast | Unlabeled video |
| Method / component | Native output | Query evidence | Learning mechanism | Training / data source |
| ReTVL [48] | Value | Video + Task | Temporal + retry supervision | Outcomes + keypoints |
| WVM [18] | Progress sequence | Video + Task | Progress-value learning | Robot + human video |
| WARP-RM [49] | Progress (+ velocity) | Video | Warp-consistent progress | Successful robot video |
| LLM-as-a-Verifier (VLM verifier)‡ [58] | Progress (score-token expectation) | Video (pair) + Task | Prompted score-token expectation + repetition | Frozen VLM (Qwen 3.6); no training |
| DenseReward [50] | Progress | Video + Task | Per-step regression | Synthetic failures + outcomes |
| AnyviewMeter [51] | Task progress | Video + Task + Geometry (+ Multi-view) | Robometer LoRA + ray conditioning | Labeled multiview trajectories |
| HiRE (potential) [43] | Success/failure support potential | Image + Ref. | Frozen DINOv2/SigLIP kernel score | Success frames + failed endpoints |
| ARS‡ [59] | Per-frame progress | Traj. + Task (+ Ref.) | VLM event proposal + verification | Frozen VLM; optional outcome labels |
| RELATIVE (23 methods) | ||||
| RL-VLM-F [7] | Scalar task utility | Image | Bradley–Terry preference | VLM pair labels |
| DecisionNCE-T [63] | Transition-direction score | Pair + Task | CLIP implicit-preference learning | EPIC-KITCHENS video + language |
| Rank2Reward (ranking head) [68] | Temporal-order utility | Image | Bradley–Terry on R3M features | Ordered passive-video frames |
| VLP (preference scorer) [69] | Trajectory utility | Video + Task | CLIP cross-modal preference learning | Auto-generated video orderings |
| VARP (reward head) [70] | Preference utility | State | Preference learning + policy reg. | VLM trajectory-sketch labels |
| RoboMonkey [77] | Rank | Image + Task + Action | Preference ranking | Synthetic action pairs |
| TimeRewarder [64] | Signed temporal distance | Pair | Timestamp self-supervision | Robot + human video |
| RoVer [15] | Rank + direction | Image + Task + Action | Ranking + direction | Synthetic pairs |
| EVE (selection verifiers)‡ [78] | Candidate selection | Image + Task + Action | Prompted specialist VLM selection | Frozen VLMs; no verifier fitting |
| Robo-Dopamine (incl. 2.0) [9], [65] | Relative progress | Pair + Multi-view + Task + Ref. (+ Hist. in 2.0) | Hop-label regression + signed-Hop replay (2.0) | Human keyframes + auto pairs + synthetic negatives (2.0) |
| CoVer [79] | Alignment score | Image + Task + Act. hist. + Action | InfoNCE alignment | Demos + rephrases |
| MVR [71] | State relevance utility | Privileged simulator state | Bradley–Terry order + embed. reg. | Frozen ViCLIP |
| GIFT (transferred reward) [72] | Intent-conditioned utility | Traj. + Criterion | GPT-4o alignment + MaxEnt IRL | Preferred/non-preferred demos |
| TapSampling [80] | Progress delta | Image + Task + Action | Temporal delta | Ordered + reversed pairs |
| Small Stuff (pair critic) [66] | Pairwise progress judgment | Pair + Task | VLM pairwise fine-tuning | Successful/failed robot rollouts |
| FPL [73] | Criterion-cond. utility | Trajectory + Criterion | Preference utility | Human freeform pairs |
| VLM-AR3L (reward head) [74] | Utility / pair preference | Image / Image pair | Bradley–Terry + Siamese learning | VLM preference labels |
| Dream2Reward [67] | Signed transition alignment | Pair + Task + Hist. | Frozen DINOv3/Qwen3 + flow field | Positive demonstrations |
| Listwise VLM reward [75] | Preference utility | Image | Plackett–Luce reward learning | VLM-ranked robot behavior |
| RAPID (reward head) [76] | Preference utility | Image | Bradley–Terry + adaptive queries | VLM-labeled rollout pairs |
| PreferenceFlow (reward) [81] | Action preference utility | Latent + Action | Preference + gradient regularization | Intervention pairs + expert demos |
| TeV (energy head) [82] | Action energy | Video + Task + Act. hist. + Action | Temporal contrastive ranking | Demos + constructed negatives |
| DiVeR (discriminator) [83] | Candidate utility | Latent + Action | Criticality-weighted classifier | Successful/failed VLA rollouts |
| TEMPORAL VALUE (26 methods) | ||||
| V-PTR (robot Q) [95] | Task Q-value | Image + Task + Action | Video TD pretraining + offline RL | Ego4D + multi-task robot data |
| CLIP-Motion (visual reward) [84] | Motion stage / stage reward | Pair + Task | CLIP motion classification | Rollouts + state-derived motion labels |
| VICtoR [85] | Stage-aware motion progress | Image + Task | CLIP stage/progress learning | Action-free videos + instructions |
| REDS [86] | Subtask-conditioned reward | Video + Task | EPIC matching + CLIP alignment | Segmented expert/suboptimal demos |
| TeViR [87] | Reached-reference stage | Image + Ref. + Hist. | CLIP-text-conditioned video matching | Task videos + generated references |
| Method / component | Native output | Query evidence | Learning mechanism | Training / data source |
| Hume [96] | Q-value | Image + Task + Action | Cal-QL (conservative TD) | Robot demos, sparse reward |
| T2-VLM‡ [88] | Tracked subgoal reward | Image + Task + Hist. | VLM subgoals + Bayesian tracking | Frozen Qwen2-VL + SAM2 |
| Self-Improving EFM (steps-to-go) [93] | Steps-to-go distribution | Pair / Multi-view + Task (+ Proprio.) | PaLI steps-to-go prediction + BC | Imitation data with hindsight step counts |
| SARM [12] | Stage + progress | Video + Proprioceptive state | Stage/progress supervision | Process labels |
| RECAP (value function) [97] | Return distribution (steps to success) | Multi-view + Task + Proprio. | VLM-initialized distributional MC returns | Demos + labeled rollouts + interventions |
| PALM [89] | Progress | Image + Task + Proprio. | Action–progress diffusion | Process-labeled demonstrations |
| Cosmos Policy [98] | Return value | Multi-view + Task | Joint diffusion + MC returns | Demos + task rollouts |
| MARVL [90] | Staged similarity | Image + Task + Goal | View-invariant CLIP tuning | Auto simulator frames |
| RISE* [99] | Progress/TD value | Multi-view + Task + Future | Progress + TD value | Demos + success/failure rollouts |
| ViVa [14] | Task value | Multi-view + Proprio. | Monte Carlo returns + dynamics | Task rollouts |
| Frozen-VLA value probes [100] | Discounted outcome | Latent | Ridge regression on VLA features | Success/failure rollouts |
| SARM2 [91] | Stage + value | Video + Multi-view + Task + Proprio. | Stage/value mixture | Primitive labels + rollouts |
| MV-WAM [101] | Return value | Multi-view + Task + Action | MC return + joint dynamics | Robot demonstrations |
| DGAC [102] | Value / candidate rank | Image + Action + Future | Value + future ranking | Human video + robot rollouts |
| STDR [92] | Stage + within-stage progress | Video | Stage/progress + validity gates | VLM-segmented demos + negatives |
| Robo-ValueRL (value estimator) [103] | Normalized time-to-go value | Multi-view + Task + Hist. | Distributional remaining-time learning + failure penalty | Robot demos + online rollouts |
| WCM [104] | History value | Video + Task | MC return + future latent | Demos + rollouts |
| RynnValue [94] | Temporal distance (absolute + signed) | Video + Task | Timestamp-distance learning | Embodied video |
| SeeQ (critic) [105] | Subtask Q-value | Multi-view + Task + Action | Subtask supervision + TD | RoboCOIN + target-task data |
| eVTA (reward token) [106] | Expected future success | Video + Task | Terminal-outcome TD learning | Mixed-quality policy rollouts |
| MetaWAM [107] | Return value | Multi-view + Task + Proprio. + Action | Joint diffusion + meta-frame value | Cosmos Policy RoboCasa data |
| HYBRID (24 methods) | ||||
| AHA [117] | Sub-task verdict + failure explanation | Video + Multi-view + Task | LLaVA-1.5 instruction tuning | FailGen synthetic failures + VQA data |
| GRAPE (ranking reward) [108] | Stage cost + self-likelihood + success | Traj. + Task + Latent | VLM stages + GPT-4o cost functions | No RRM training |
| VLAC [109] | Relative progress + done | Pair + Task | Temporal-delta + done | Robot + human video |
| Guardian [13] | Verdict + reasoning trace | Multi-view + Task | Failure-CoT tuning | Synthetic failures |
| ReWorld* [120] | Hierarchical four aspects | Video + Task | Preference learning | GPT-4o-scored robot videos |
| Robometer [11] | Progress + success + pref. | Video (pair) + Task | Multi-head temporal learning | Time + outcomes + auto prefs. |
| PRIMO-R1 [118] | Progress + failure + CoT | Video + Task | Process SFT + RL | Process labels |
| Large Reward Models [110] | Multi-part reward | Image (pair) + Task | Three LoRA interfaces (DPO + SFT) | Robot, human, and simulated demos (time labels) |
| PRM-as-a-Judge‡ [126] | Outcome–process–diagnosis profile | Video + Task | Zero-shot PRM aggregation | No training |
| VLLR [111] | Subgoal progress (critic init.) + self-certainty + outcome | Pair + Task + Graph + Latent | Prompted progress for critic warm-up + certainty | No RRM training (zero-shot LLM/VLM) |
| ARM [112] | Advantage + completion | Video + Multi-view + Task | Dual-head classification | Human process labels |
| RoboAlign-R1* [121] | Six scores | Video + Task | Teacher SFT + distillation | Rule + human rubric labels |
| Reward as an Agent* [122] | 4-aspect scores + rationale | Video + Task | Prompted judging | Human audit |
| RoboGaze*‡ [123] | Localized failure events + severity | Video + Task + Init. | Grounded routing + specialist VLMs + critic | Frozen VLMs; no training |
| CoRe [113] | Formal + residual reward | Image + Sim. + Action | Formal + preference residual | VLM labels |
| ACE-Brain-0.5 [17] | Progress + preference | Video + Task | Process + preference learning | Labeled video |
| Method / component | Native output | Query evidence | Learning mechanism | Training / data source |
| VLAC-Cut [119] | Signed progress + diagnosis + cut points | Video + Task (+ Ref.) | Process-annotation instruction tuning | Re-annotated robot video with failures and recoveries |
| Thea‡ [127] | Status + evidence + cause | Image + Criterion | Prompted post-hook | No training |
| TrustRoboReward [114] | Scores / comparisons + rationale | Video (pair) + Task | Point/pair consistency | VLM + human correction |
| FIERCE [115] | Progress + future failure risk | Video + Task + Proprio. + Action | Joint dynamics + reward learning | Robot data + outcome-labeled rollouts |
| EVO-WAM (verification)∗‡ [124] | Completion + action consistency | Multi-view + Video + Task + Init. + Proprio. + Action | VLM judging + inverse dynamics | Recorded video–action pairs |
| VeriFine-Judge [128] | Rubric scores + diagnostics | Video + State + Task + Action + Rationale | Human-calibrated VLM distillation | Teacher scores + human corrections |
| Flora-Rewarder [116] | Episode grade (1–5) + step action quality + rubric scores | Video + Multi-view + Task | Full VLM fine-tuning with three prompt types | Human, recorded-success, and VLM-generated labels |
| RoboJudge-9B∗ [125] | Physical adherence + instruction alignment + rationale | Video + Task | Qwen3.5-9B SFT + GRPO | Human-verified scores + model-written rationales |
Table S1. Comparison of 114 robotic reward methods and evaluative protocols, one evaluative component per row, grouped by semantic family in the order of Section 2.2 and ordered by first release within each family. * evaluates generated behavior; ‡ marks a protocol or harness. Query abbreviations and component notes follow the table.
Section 2 defines RRMs, their five families, and the scope; Sections 3 and 4 review model designs, data foundations, and evaluation practice; Sections 5 and 6 present RRM-Bench-1K, RRM-Hub, and the unified study; and Section 7 discusses open challenges.
Definitions and Scope
Reward Design without Foundation Models
Before foundation models, each robot task or domain received its own reward (Figure 1(a)): specified by hand, computed by geometric task oracles, recovered through inverse reinforcement learning [129], [130], [131], [132], [133], or learned from human preferences [134], success examples [135], [136], or self-supervised video [137]. This line continues in evaluators without foundation models: expert-video prediction (VIPER [138], Diffusion Reward [139]), goal distance over object graphs (GORDON [140]), progress from video (ProgAgent [141]) or tactile maps (TaRL [142]), trajectory prefixes, complete rollouts, paired executions, or success–failure discrimination (Success Visitation Matching [143]), preference-aligned features (RAPL [144]), and, beside a pretrained policy, value-based action ranking (VGPS [145]) or failure verification (VLA-Corrector [146]). Eureka [147], Text2Reward [148], and Language to Rewards [149] prompt a large language model to write per-task reward code, which Hybrid evaluators such as CoRe [113] and GRAPE [108] combine with other judgments.
Task-Conditioned Evaluation and Five Semantic Families
A robotic reward model is a task-conditioned evaluator that draws on CLIP [150] or a later foundation model for representation, supervision, or the judgment itself. It assesses robot behavior against a task specification , given as language, a goal image, a reference demonstration, or a structured goal, supplied with each query or fixed in the evaluator. It reads execution evidence admitted by an interface : single observations, temporal windows,
trajectory prefixes, complete rollouts, paired executions, or synchronized camera views, optionally with actions, proprioception, or simulator state. For an evaluative query , such as terminal success, progress, pairwise preference, or value, the evaluator is a map
whose native output space holds a label, probability, score, ordering, distribution, or structured judgment.
Outputs of the same form can carry different meanings: a scalar may encode visual compatibility, task progress, a preference utility, or an expected return. We therefore assign each evaluator to the family whose question its native output answers; what it is conditioned on, the specification and the evidence that admits, is recorded for every entry but does not decide the family (§ 4.4). Outcome, Absolute Progress, Relative, and Temporal Value each answer one question, and Hybrid composes them (Figure 2); within each family, Section 3 groups methods by design route, that is, by how they produce the output.
Outcome (is the task done, or will it fail?) judges whether a declared event has occurred or will occur within a horizon: terminal success, end-of-episode grades, and failure forecasts such as SAFE’s running failure score [34] (Figure 3). Absolute Progress (how far along is it?) covers pointwise estimates of advancement on a whole-task scale, anchored by a goal, reference, learned scale, or task procedure, such as the progress of ProcVLM [56] (Figure 4). Relative (which is better?) covers quantities defined by ordering under a fixed task or criterion, from a direct comparator’s judgment of a pair or candidate set, such as the signed hop between two states that the Robo-Dopamine GRM [9] returns (Figure 5), to a pointwise utility defined by comparisons. Temporal Value (which stage, and how well is it going?) covers targets measured against a reference the execution itself sets: the stage it has reached, the time it still needs, such as the seconds to completion predicted by RynnValue [94] (Figure 6), or the return expected under the policy that continues it, as in or ; the target, not temporal input, defines the family, so a whole-task progress estimate reading the full video history is Absolute Progress. Hybrid (what happened, and on what evidence?) covers evaluators that compose judgments from several families or as the fields of a structured verification judgment, as Robometer [11] does with progress, success, and preference from one backbone (Figure 7).
Figure 3. SAFE [34], a representative Outcome model. (a) A light head on a frozen VLA maps its features to a running failure score . (b) rises as the second shoe stalls and crosses the conformal threshold , which raises the alarm before the execution fails.
Figure 4. ProcVLM [56], a representative Absolute Progress model. (a) Given observations and instruction , the language-model head first generates the remaining actions, and the value head then reads this generated procedure to output progress . (b) rises as fewer actions remain and reaches when both cans are in the box.
Figure 5. The Robo-Dopamine GRM [9], a representative Relative model. (a) Given start and goal states, the GRM compares a before and an after state and returns a signed hop . (b) Hops are scaled to the distance that remains or to the progress made, so chained hops keep progress in ; removing the green block gives .

Figure 6. RynnValue [94], a representative Temporal Value model. (a) Value () and displacement () query tokens interleaved with the frames return the seconds to completion and the signed displacement . (b) Each placement advances the execution by s, and falls from 12 s to 0 s.

Figure 7. Robometer [11], a representative Hybrid model. (a) Three heads on one VLM backbone return per-frame progress , success , and preference over two trajectories. (b) rises with each placed block, turns 1 only at the end, and is preferred to a ending in the wrong order.
Table S1 lists the family, output, query evidence, learning mechanism, and training data of all 114 methods.
Scope and Corpus
A method enters the corpus when it meets four criteria: (i) its evaluative use is validated on robot behavior, robot observations, or a behavior representation; (ii) its output answers a task-conditioned question about success, progress, preference, value, behavior quality, or verification and is used or evaluated as a decision signal; (iii) CLIP or a later foundation model contributes to the evaluator through representations, adaptation, supervision, distillation (including a standalone model trained on foundation-model labels), or direct judgment; and (iv) the work designs or evaluates the evaluator rather than only applying one. A policy, video generator, or world model qualifies only through a component that judges task fulfillment and can be queried or ablated on its own; its action likelihood, reachability, or predicted observations describe what is probable or feasible, not whether the task is fulfilled.
Model Designs and Evaluative Semantics
Architecture and semantics are separate axes, so we first trace the evidence-to-decision path that all evaluators share and then review the five families along the design routes of Figure 2. Figure 3–Figure 7 show one representative method per family on simulated RoboTwin 2.0 executions [151], with its framework in (a) and its native output, with illustrative values, in (b); ProcVLM [56], Robo-Dopamine [9], Rynn Value [94], and Robometer [11] are also evaluated in Section 6.
From Evidence to Decisions
An evaluator encodes the task specification and the evidence its interface admits, such as a camera set, an embodiment descriptor, a candidate action, or a predicted future, and a query-specific head returns the native output. A downstream rule then consumes the prediction: SAFE [34] (Figure 3) raises an alarm when its failure score first exceeds a threshold calibrated on successful rollouts, shaping uses a progress estimate as the potential of a shaped reward [152], and selection ranks candidates by a preference utility. Each rule places its own demand on the evaluator: selection needs only the order of the candidates, whereas shaping exposes the reward to a policy that optimizes against it (Section 7.6).
This pipeline separates semantics (the estimated quantity), form (its encoding), and role (its use), and the family follows semantics alone, whatever a method calls its output. For methods with several outputs, we set aside policy outputs and deterministic transformations and assign the family the remaining outputs share. A rationale, confidence estimate, or abstention gate attached to one judgment keeps that judgment’s family, as in ProcVLM (Figure 4), whereas independently meaningful judgments from different families make a model Hybrid, as in Robometer (Figure 7).
Outcome Models
Outcome models judge whether a declared event, such as success or failure, has occurred or will occur within a horizon. Episode-level judgments need the decisive state in view, which suits acceptance and filtering; predictive monitors such as SAFE (Figure 3) forecast the event from signals that precede it, which suits stopping and intervention.
Episode-Level Outcome and Ordinal Judgments
These judges read a foundation representation out into a verdict or a grade and differ in what the readout learns to separate: positive and negative goal descriptions in CLIP [150] space (ZSRM [4]), whether a video shows the task accomplished (SuccessVQA [29], fine-tuned from Flamingo [153]), or demonstrations and unsuccessful or synthetic data, judged from an image or a
whole trajectory (RoboFuME [30], I-FailSense [31]); proprietary embodied-reasoning models such as Gemini Robotics ER 2 [154] list success detection among their specialties. Grades and similarities extend the verdict: a 1–5 end-of-episode grade (RoboReward [10]), similarity to a demonstration (RoboCLIP [5]), or a thresholded reward regressed from generated-video latents (World-VLA-Loop [32]). A verdict can reject only the failed endings that its negative examples cover, so rejecting a wrong object or destination requires such negatives or grounding in the instruction.
Predictive and Policy-Internal Failure Monitors
A forecast’s main design choice is what the monitor reads. Policy-internal monitors read the acting policy’s features, which signal failure before it shows in the scene but must be refit per policy: a failure score learned from outcome-labeled rollouts (SAFE [34]), a probe of frozen OpenVLA [155] activations that flags steps within 15 steps of failure [36], or calibrated feature support (VLAConf [35]). External monitors read the execution and apply across policies, asking a frontier VLM whether the robot will finish in time (Sentinel [33]) or detecting task-breaking deviations from anchors a world action model generates (FAVOR [37]).
Evaluation implications: An Outcome claim needs an operational event, a declared horizon, and evidence showing the event; grades call for ordinal metrics, probabilities for calibration, and online monitors for alarm timing against the decisive event (Figure 3(b)). Endpoint and dense judgment are separate capabilities: RoboReward filters completed executions well yet detects regressions worst (§ 7.2).
Absolute Progress Models
Absolute Progress models answer how far an execution has advanced on an anchored scale; ProcVLM (Figure 4), for instance, anchors progress to the remaining actions. Routes differ in where the anchor comes from: a goal or demonstration representation, demonstration time, task semantics read by a VLM, or the system that produces the behavior. The anchor decides how a score treats a step away from the goal configuration while the task advances, such as a retraction to regrasp: a representation distance can read it as a regression, a time-supervised estimate can credit it whether or not it serves the task, and a procedure-based estimate can credit it if the generated procedure contains it.
Distances to Goal or Demonstration Representations
The most direct anchor is a goal or demonstration in a representation space, as in the goal-conditioned value representation of VIP [38], the language–visual goal alignment of LIV [39], and the zero-shot image–text compatibility of VLM-RM [6]. How the alignment is learned decides which deviations the score registers: learned failure prompts (Adapt2Reward [40]), a position code only for the first frame so that time cannot be read as progress (ReWiND [8]), a light head on frozen DINOv3 [156] and text features that ranks states like analytic rewards (Rewarding DINO [42]), and anchors grounded in tracked object masks (SPARTA [41]) or in successful and failed states (HiRE [43]), which respond to changes that a global similarity averages away.
Temporal Supervision and Progress Proxies
Temporal order within demonstrations offers a second scale, and with it the assumption that achievement advances with time, which holds on successful demonstrations. GVL [16] counters the assumption by asking a frozen VLM for the completion of shuffled frames; others learn the scale from demonstrations with online refinement (PROGRESSOR [45]), a ranking objective (Video-Language Critic [44]), reference alignment (RARM [47]), distributional outputs (WARP-RM [49]), or test-time updates (VITA [46]), and AnyviewMeter [51] adapts Robometer to multiview input. Labels that fall teach the score to register setbacks: chunks of normalized progress whose frames are kept, repeated, or reversed (World Value Models, WVM [18]), preferences around annotated retries (ReTVL [48]), or simulated trajectories with failures and recoveries (DenseReward [50]).
Prompted, Token-Level, and Reasoning-Based Progress
VLM-based estimators, prompted or fine-tuned, reach the anchor through task semantics and differ in how the score is read out: the probability of a task-truth token (TOPReward [53]), the expected score token over repeated queries, which also serves as a shaping reward (LLM-as-a-Verifier [58]), or a score that follows reasoning, as a process rationale (SOLE-R1 [55]), a generated procedure (ProcVLM), or recurrent steps over long tasks (R²VLM [54]). Feat2Go [57] learns binned progress from feature similarity to subgoal states, ARS [59] assembles progress from events that VLM agents verify over a rollout, and ProgressHM [52] can return an explicit unanswerable response, which supports evidence-sufficiency tests. A generated procedure can name the object, destination, and order that a constraint check needs, but a wrong decomposition yields a smooth score for the wrong procedure.
Policy-Coupled and World-Model-Coupled Progress
Progress can also be estimated inside the system that proposes or predicts behavior, which shows the score what the system intends next but ties it to that system: World-Env (RehearseVLA) [60] scores imagined video for reward and early termination, NavWAM [62] predicts goal progress with future observations, and ProgressVLA [61] guides action generation by the progress of predicted frames.
Evaluation implications: An Absolute Progress claim rests on an anchor, a declared scale, and a measured response to the task state. Time-supervised and goal-anchored scores part ways at motion without its effect and at a lost achievement (Figure 8(A), (C)), so tests must decouple the task state from elapsed time; where the two diverge, most evaluated models follow elapsed time (Section 6.5).
Relative Models
Relative models judge which of two executions, states, or actions better satisfies the task. A direct comparator judges the items it is given, often two states of one execution, so its sign registers a local change, a regression included: the General Reward Model (GRM) of Robo-Dopamine (Figure 5) returns a signed hop between a before and an after state, and progress is chained from the hops. A utility learned from preferences scores one item at a time, commonly through the Bradley–Terry model [157], in which the probability that one item is preferred to another is a logistic function of their utility difference, so its values carry meaning only as an ordering of items like those compared in training.
Direct Pairwise and Transition Comparators
For transitions, the pair supplies the reference: a signed temporal distance between frames (TimeRewarder [64]), hops between multiview before/after states (Robo-Dopamine), refined with rollout history, synthetic negatives, and a curriculum over hop sizes (Robo-Dopamine 2.0 [65]), or comparison with the instruction (DecisionNCE-T [63]), a predicted successful transition (Dream2Reward [67]), or pairs of observed and predicted candidates (a pair critic [66]).
Pointwise Utilities Learned from Preferences
The criterion that labels the training pairs defines the utility: VLM preferences on observation pairs (RL-VLM-F [7]), human comparisons under a natural-language criterion (Freeform Preference Learning, FPL [73]), multi-view orderings from a video-language model (MVR [71]), or instruction-conditioned video scores (VLP [69]). Rank2Reward [68] learns temporal order with a Bradley–Terry head and inherits the time assumption of § 4, and GIFT [72] transfers a maximum-entropy inverse-reinforcement-learning reward [131] to new scenes, with GPT-4o [158] inferring intent. Others make comparisons more informative: diverse queries that stop once the reward stabilizes (RAPID [76]), trajectory sketches (VARP [70]), ranked lists (a listwise VLM reward [75]), or joint pointwise and comparative pathways (VLM-AR3L [74]).
Candidate-Action Comparators
When the alternatives are actions, ordering becomes a control primitive for test-time selection. Scorers learn from synthetic preferences (RoboMonkey [77]), predict a refinement direction (RoVer [15]), align instructions with actions (CoVer [79]), or verify each candidate’s progress (TapSampling [80]), and some sit beside a frozen policy as an energy verifier (TeV [82]), a success-visitation discriminator (DiVeR [83]), preference-guided flow matching (PreferenceFlow [81]), or zero-shot VLM verifiers (EVE [78]). Ranking gains depend on the proposal pool as well as the scorer, so best-of- gains can come from sampling alone, and a mismatched verifier can hurt: in the CoVer study, a RoboMonkey verifier cut simulated success of a [1] policy to under a fifth on red-teaming instructions.
Evaluation implications: Relative models are judged by their choices: pairwise accuracy, ties, ranking metrics, cycles among direct comparisons, and robustness to paraphrases and the candidate pool. Signed comparators must also give a lost achievement a negative step (Figure 5(b)); Robo-Dopamine-GRM-3B does so when a shelf falls (Figure 9) yet recalls under half the falls (Table 2).
| Model | Rising Precision | Rising Recall | Rising F1 | Falling Precision | Falling Recall | Falling F1 | Both classes Macro-F1 | Both classes Accuracy |
| Specialized evaluators, 2B–4B | ||||||||
| ProcVLM-2B [56]† | 0.924 | 0.540 | 0.682 | 0.128 | 0.465 | 0.201 | 0.441 | 0.533 |
| VLAC-2B [109]† | 0.922 | 0.374 | 0.532 | 0.118 | 0.312 | 0.171 | 0.352 | 0.368 |
| Robo-Dopamine-GRM-3B [9] | 0.908 | 0.412 | 0.566 | 0.135 | 0.401 | 0.202 | 0.384 | 0.411 |
| Multiview‡ | 0.929 | 0.593 | 0.724 | 0.161 | 0.482 | 0.242 | 0.483 | 0.581 |
| Robo-Dopamine-GRM-2.0-4B-Preview [9] | 0.927 | 0.594 | 0.724 | 0.158 | 0.518 | 0.242 | 0.483 | 0.586 |
| Multiview‡ | 0.940 | 0.730 | 0.822 | 0.219 | 0.567 | 0.316 | 0.569 | 0.713 |
| Robo-ValueRL [103]‡ | 0.909 | 0.514 | 0.656 | 0.105 | 0.526 | 0.175 | 0.416 | 0.515 |
| Robometer-4B [11]† | 0.916 | 0.871 | 0.893 | 0.181 | 0.263 | 0.214 | 0.554 | 0.812 |
| RoboReward-4B [10]† | 0.937 | 0.263 | 0.410 | 0.209 | 0.044 | 0.072 | 0.241 | 0.241 |
| RynnValue-4B [94]† | 0.929 | 0.677 | 0.783 | 0.153 | 0.507 | 0.235 | 0.509 | 0.660 |
| Specialized evaluators, 7B–8B | ||||||||
| PRIMO-R1-7B [118]† | 0.914 | 0.571 | 0.703 | 0.135 | 0.181 | 0.154 | 0.429 | 0.531 |
| LRM-8B [110] (Contrastive)§ | 0.921 | 0.171 | 0.288 | 0.071 | 0.754 | 0.130 | 0.209 | 0.213 |
| LRM-8B [110] (Progress) | 0.924 | 0.155 | 0.265 | 0.127 | 0.148 | 0.136 | 0.201 | 0.154 |
| Robo-Dopamine-GRM-8B [9] | 0.921 | 0.456 | 0.610 | 0.141 | 0.422 | 0.211 | 0.411 | 0.453 |
| Multiview‡ | 0.933 | 0.604 | 0.733 | 0.176 | 0.489 | 0.259 | 0.496 | 0.592 |
| Robo-Dopamine-GRM-2.0-8B-Preview [9] | 0.931 | 0.594 | 0.725 | 0.148 | 0.478 | 0.226 | 0.476 | 0.582 |
| Multiview‡ | 0.938 | 0.727 | 0.819 | 0.212 | 0.552 | 0.307 | 0.563 | 0.709 |
| RynnValue-8B [94]† | 0.926 | 0.684 | 0.787 | 0.160 | 0.445 | 0.236 | 0.511 | 0.661 |
| SOLE-R1-8B [55] | 0.925 | 0.685 | 0.787 | 0.146 | 0.492 | 0.225 | 0.506 | 0.666 |
| Multiview‡ | 0.934 | 0.745 | 0.829 | 0.181 | 0.515 | 0.267 | 0.548 | 0.723 |
| TOPReward [53] (Qwen3-VL-8B [189])† | 0.913 | 0.558 | 0.693 | 0.111 | 0.503 | 0.182 | 0.437 | 0.552 |
| VLAC-8B [109]† | 0.917 | 0.423 | 0.579 | 0.120 | 0.285 | 0.169 | 0.374 | 0.410 |
| Specialized evaluators, 27B–30B | ||||||||
| Flora-Rewarder-27B [116] | 0.930 | 0.748 | 0.829 | 0.172 | 0.472 | 0.252 | 0.541 | 0.721 |
| Multiview‡ | 0.924 | 0.735 | 0.819 | 0.154 | 0.443 | 0.229 | 0.524 | 0.706 |
| VLAC-Cut-30B-A3B [119] | 0.922 | 0.916 | 0.919 | 0.276 | 0.268 | 0.272 | 0.596 | 0.853 |
| Specialized evaluators, proprietary backbone | ||||||||
| GVL [16] (Gemini 3.8 Flash) | 0.928 | 0.730 | 0.817 | 0.167 | 0.422 | 0.240 | 0.528 | 0.700 |
| General-purpose VLMs, Pair style | ||||||||
| GPT-6.1 Sol [190] | 0.929 | 0.920 | 0.925 | 0.326 | 0.355 | 0.340 | 0.632 | 0.865 |
| GPT-6 Luna [191] | 0.917 | 0.640 | 0.754 | 0.123 | 0.466 | 0.195 | 0.474 | 0.623 |
| GPT-5.6 Terra [192] | 0.914 | 0.908 | 0.911 | 0.200 | 0.212 | 0.206 | 0.558 | 0.840 |
| Claude Opus 5.5 [193] | 0.925 | 0.900 | 0.912 | 0.264 | 0.331 | 0.294 | 0.603 | 0.844 |
| Claude Sonnet 5.5 [194] | 0.914 | 0.886 | 0.900 | 0.183 | 0.236 | 0.206 | 0.553 | 0.822 |
| Claude Haiku 5.5 [195] | 0.923 | 0.702 | 0.798 | 0.145 | 0.464 | 0.221 | 0.509 | 0.679 |
| Gemini 3.8 Flash [196] | 0.931 | 0.896 | 0.913 | 0.287 | 0.384 | 0.329 | 0.621 | 0.846 |
| General-purpose VLMs, Sequence style | ||||||||
| GPT-6.1 Sol [190] | 0.969 | 0.598 | 0.740 | 0.736 | 0.244 | 0.366 | 0.553 | 0.563 |
| GPT-6 Luna [191] | 0.954 | 0.419 | 0.582 | 0.442 | 0.176 | 0.252 | 0.417 | 0.395 |
| GPT-5.6 Terra [192] | 0.953 | 0.394 | 0.557 | 0.679 | 0.129 | 0.217 | 0.387 | 0.368 |
| Claude Opus 5.5 [193] | 0.948 | 0.849 | 0.895 | 0.697 | 0.237 | 0.354 | 0.624 | 0.789 |
| Claude Sonnet 5.5 [194] | 0.932 | 0.769 | 0.843 | 0.529 | 0.112 | 0.184 | 0.514 | 0.705 |
| Claude Haiku 5.5 [195] | 0.935 | 0.542 | 0.686 | 0.322 | 0.075 | 0.122 | 0.404 | 0.496 |
| Gemini 3.8 Flash [196] | 0.960 | 0.720 | 0.823 | 0.688 | 0.196 | 0.305 | 0.564 | 0.669 |
| Decision model | ||||||||
| Jev 1.14 [188] (Pair) | 0.916 | 0.743 | 0.820 | 0.138 | 0.350 | 0.198 | 0.509 | 0.704 |
| Jev 1.14 [188] (Sequence) | 0.929 | 0.751 | 0.831 | 0.183 | 0.427 | 0.256 | 0.543 | 0.719 |
Table 2. Progress-direction classification on RRM-Bench-1K. Bold marks the best single-view result in each panel. †Curve interpolated to 1 FPS. ‡Multiview input (Section 6.4); all other rows take single-view input. §Scored on adjacent source frames.
Figure 9. Reward profiles of nine evaluators on one execution of put the books back. The task-state reference (GT, gray) drops to zero when the robot knocks the shelf over at 19.6 s; red bands mark its reversals, and is each model’s Pearson correlation with it.
Temporal Value Models
Temporal Value models estimate progress within a stage, the time left, or the return still expected; RynnValue (Figure 6), for instance, counts the seconds to completion. A return-based value , the discounted task reward expected from the current prefix, also depends on the policy that continues the execution, and on the next action as well. These targets share temporal input but answer different questions, so a stage error corrupts within-stage progress, a speed change the time to go, and a policy change .
Stage-Decomposed Values
Stage decomposition replaces one global reference with a sequence of local ones, and the designs differ in how they recognize the stage: with CLIP (VIcToR [85], CLIP-Motion [84]) jointly with within-stage progress (SARM [12]), through primitive stages shared across tasks (SARM2 [91]), from VLM-segmented demonstrations with grasp verification (STDR [92]), or jointly with a policy’s actions (PALM [89]). Subtask structure can also condition the reward (REDS [86]), MARVL [90] or be tracked over time as VLM-proposed subgoals (T2-VLM [88]) or the farthest-reached frame of a generated reference video (TeViR [87]). Local references keep an early achievement from saturating a global score and expose regressions within a stage, which makes stage assignment the critical step.
Temporal Distance Values
Temporal distance can be counted in seconds or in control steps, so a slow but correct execution has more time to go than a fast one in the same task state. RynnValue learns its targets from timestamps, so speed, pauses, and the completion cutoff all enter them, and suppresses temporal shortcuts with random temporal sampling, order shuffling, and value-isolation attention. Self-Improving Embodied Foundation Models [93] count steps instead, differencing the expected steps to an instructed goal into a shaped reward and thresholding them into a success detector.
Policy-Indexed and Value-in-the-Loop Models
When the target is expected return, the policy becomes part of its meaning. V-PTR [95] pretrains a value function on human video for robot Q-learning, and RECAP [97], the training recipe of , fits a VLM-initialized value function to Monte Carlo returns and thresholds its advantages into an indicator that conditions the policy, so its value is that of the data-collecting policy; Robo-ValueRL [103] conditions such an estimator on visual history. Critics can also co-evolve with the policy or read its representations, as a reward token refitted by temporal-difference (TD) bootstrapping (eVTA [106]), a Q-value for candidate chunks (SeeQ [105]), or a linear probe of discounted outcomes (frozen-VLA value probes [100]). Others act inside the control loop: they repurpose a video generator as a value model (ViVa [14]), rank sampled or corrective actions (Hume [96], DGAC [102]), or attach return or progress values to the futures a world model
generates (Cosmos Policy [98], RISE [99], WCM [104], MV-WAM [101], MetaWAM [107]).
Evaluation implications: A Temporal Value result is read against its declared target and the policy or data distribution behind it. The target fixes the expected response to reversal, stagnation, and retry: a lost achievement can leave expected return high if recovery is likely, so RRM-Bench-1K tests only the sign of a policy-indexed value’s response to a fall (§ 6.4).
Hybrid Models
Hybrid models answer several evaluative questions at once, such as progress and failure risk, or what failed and where; Robometer (Figure 7), for instance, returns progress, success, and a preference from three heads on one backbone. Hybrid designs combine judgments through heads on a shared backbone, composite rewards, or critics and judge ensembles that return a verdict with its cause and evidence, and more outputs add information only when each passes its own family’s tests and their disagreements are resolved.
Composite Rewards and Multi-Interface Models
One strategy combines separate judgments across families: a VLM-preference residual on a code-based formal reward (CoRe [113]), a critic initialized with VLM-estimated sub-goal progress (VLLR [111]), or rollouts ranked by likelihood, success, and a foundation-model keypoint cost for preference optimization (GRAPE [108]). Others keep each judgment queryable, pairing progress with preference (ACE-Brain-0.5 [17]), a segment-level advantage with a completion probability (ARM [112]), a progress delta with a done signal (VLAC [109]), an episode grade with step-level action quality (Flora-ReWarder [116]), or observed progress with a causal failure-risk head (FIERCE [115]). The outputs must also agree: the temporal-contrast, progress, and completion interfaces of Large Reward Models (LRM) [110] can disagree, and TrustRoboReward [114] repairs contradictory training labels through preference-ordered isotonic editing.
Structured Process Critics and Failure Diagnosis
A diagnosis ties the verdict to its cause. AHA [117] and Guardian [13], tuned on failures perturbed from or synthesized in simulation, judge from multiview images whether a subtask or a plan succeeded and why, Guardian with a reasoning trace; PRIMO-R1 [118] learns by supervised and then reinforcement fine-tuning to reason about progress in video and transfers zero-shot to failure detection; VLAC-Cut [119] pairs signed progress with a diagnosis of each change and cuts a rollout into segments to keep or discard.
Judges for Generated Robot Behavior
Judges of generated behavior separate realism from task completion, since a realistic video can still fail the task. ReWorld [120] scores generated videos for physical realism, task completion, embodiment plausibility, and visual quality, RoboAlign-R1 [121] distills such a multidimensional teacher, and RoboJudge-9B [125] rates physical adherence and instruction alignment separately. Agentic judges plan, vote, and reflect (Reward as an Agent [122]), route subtask segments to specialist VLMs returning time-localized failure events (RoboGaze [123]), or check inverse-dynamics consistency of generated video and actions (EVO-WAM [124]).
Protocol Judges and Runtime Verification
Protocol and action judges also return structured judgments. PRM-as-a-Judge [126] aggregates existing process reward models zero-shot into an Outcome–Process–Diagnosis profile, and VeriFine-Judge [128] distills a frontier-VLM teacher into a judge of instruction consistency and safety of proposed actions. At runtime, judgments decide whether to continue: Thea’s Evaluation as Exit Codes [127] checks the observation against an explicit post-condition and returns process, success, or failure with evidence.
Evaluation implications: Hybrid models call for field-wise evaluation: completion, failure type, localization, confidence, and correction utility each need their own test, explanations are checked against the actual failure, and composite rewards need each component measured directly.
Cross-Cutting Design Dimensions
Semantics and mechanism vary independently: Absolute Progress comes from embedding distances in VIP, demonstration time in VITA, token probabilities in TOPReward, and generated procedures in ProcVLM, and one backbone yields different evaluators under different queries (Section 6.5). Four further dimensions fix what a result can establish. The evidence interface bounds what is identifiable, from goal compatibility in a single frame and the direction of change in a before–after pair to an online alarm on a causal prefix and an outcome on a complete rollout. Transfer is shown by tests of the native evaluator, with any fitting, test-time adaptation, or hindsight access stated. Provenance determines which errors travel with a model: frozen representations carry pretraining biases, adaptation its label shortcuts, and distillation the teacher’s errors. Coupling determines what can be tested in isolation, since policy-internal and world-model-coupled heads change with the system they serve. Table S1 records each method’s evidence, provenance, and native output, and RRM-Bench-1K uses them to assign evaluators to tests (§ 6.4); reporting the raw prediction with each derived signal keeps a frame index from passing as progress.
Data Foundations and Evaluation Practice
Most evaluators learn from data, so how raw experience becomes targets decides what their judgment means. We follow this path to supervision and use, then ask whether evaluations resolve the ambiguities training leaves.
Sources of Raw Experience
Robot demonstrations and failures: Large teleoperation corpora such as Open X-Embodiment [164], DROID [165], and BridgeData [166] provide broad task coverage and are widely used to train and evaluate reward models [56] [10, 16], but they consist mostly of successful demonstrations, so the boundary between acceptable, suboptimal, and failed behavior is thinly covered. Recent collections add failures: RoboMIND [167] releases about 5,000 real teleoperation failures annotated with their causes, and RBM-1M [11] includes failed and suboptimal executions among more than one million trajectories from 21 platforms.
Human and internet video: Passive video extends semantic and temporal coverage beyond the available robot trials: EPIC-KITCHENS [168] serves LIV [39] and PROGRESSOR [45], EgoDex [169] serves LRM [110] and WVM [18], action-free video serves V-PTR [95] and VICToR [85], VLAC [109] mixes human and robot video, and EgoRecovery [170] records human recovery behavior. Whether the corresponding robot action is feasible depends on embodiment, viewpoint, and contact dynamics.
Simulation and autonomous rollouts: Simulation makes failure coverage controllable: RoboFAC [171], FailSafe [172], Guardian [13], SOLE-R1 [55], and DenseReward [50] generate failures with privileged geometric or phase labels but inherit the simulator’s appearance, dynamics, and failure taxonomy. Policy rollouts yield errors closer to the deployment distribution, with coverage set by the policy that produced them, as in the rollouts that refine Cosmos Policy [98], World-VLA-Loop [32], RISE [99], and ViVa [14].
Human and model annotation: Human annotation is sparse but captures qualities that proxies omit: double-blind policy comparisons and critiques in RoboArena [173], stage labels in SARM [12], synchronized keyframes in Robo-Dopamine [9], progressive, regressive, or stagnant transitions in ARM [112], retry keypoints in ReTVL [48], and preferences along annotator-named axes in FPL [73]. VLMs scale up pairwise labels, rationales, and subtask annotations, from RL-VLM-F [7] to the ProcCorpus of ProcVLM [56], trading annotation cost against coverage and teacher error.
From Trajectories to Supervision
Label construction decides which differences the evaluator is taught to preserve. Time and order labels teach stage and sequence, outcome labels finished versus failed endings, preferences a relative ordering, and event or condition labels local change and constraints. Any other distinction can only transfer from pretraining or other data.
Temporal and outcome-derived targets: Normalized time, frame order, temporal distance, and discounted terminal outcomes are inexpensive labels that a demonstration supplies with little further annotation [14], [46], [64], [94], [109]. They equate achievement with elapsed time, which holds on successful demonstrations, so motion without its effect and an achievement that is later lost, the cases of suites (A) and (C), receive no label of their own, and a return target further depends on the reward definition and the policy.
Synthetic and natural negatives: Because successful demonstrations dominate, negatives are often synthesized from them, and each transformation teaches its own distinction. Rewound videos, as in ReWiND [8], teach that backward playback scores lower, although a real regression such as a dropped object unfolds forward in time; clipped episodes, as in RoboReward [10], teach that a truncated execution earns partial credit; and counterfactual instructions, in RoboReward and CoVer [79], teach task-level grounding, whereas a wrong destination keeps the instruction and changes one relation in the scene, the case of suite (B). Perturbations in AHA [117] and FailSafe [172] and world-model failures in Dream2Fix [174] come closer to physical errors but follow the perturbation model, so an evaluator can learn its artifacts rather than the failure. Natural failures, such as the RoboArena [173] rollouts that RoboReward includes, supply the subtle errors of execution quality and safety that synthetic negatives underrepresent [19], and few label sets state whether decisive evidence is visible (suite (D)).
Preferences and intent: Preferences replace a numeric target with a criterion: human judgments, VLM labels, analytic re-wards, distances to expert actions, and trajectory order each give better its own meaning [7], [42], [73], [77]. VARP [70], PreferenceFlow [81], and GIFT [72] draw preferences from trajectory sketches, human interventions, and inferred intent.
Stage and language constructions: For long tasks, SARM [12], SARM2 [91], REDS [86], VicTor [85], and STDR [92] attach local progress to inferred or annotated stages; a local label inherits its segmentation errors, so stage assignment and within-stage accuracy need separate measurement. Question–answer, reasoning–score, and rubric formats fit labels to VLM training [52], [118], [121].
Training Objectives, Use, and Provenance
Different targets share learning tools: regression, distributional prediction, temporal-difference learning, contrastive alignment, ranking, and token prediction [7], [18], [38], [39], [118], with Bradley–Terry [157] and Plackett–Luce [175], [176] losses for pairwise and listwise preferences [75]. One loss serves several targets, so target and interface define the native output. The consuming decision sets demand on calibration, latency, and error cost, and a new decision rule can reuse an existing evaluator: Reward-DAgger [177] uses a frozen Robometer [11] to decide when to request human intervention, and EVOLVE-VLA [178] accumulates the deltas of VLAC [109] against milestone frames into a progress value that rewards and ends test-time rollouts.
Scale does not remove assumptions built into labels: more successful demonstrations can leave a subtle failure boundary unchanged, and synthetic or VLM-generated labels can share shortcuts with their source models. High temporal correlation can then reflect a time-labeling rule, and agreement with a related VLM an inherited bias; the suites of § 6.1 separate them from task-state sensitivity.
Existing Evaluation Practice
Protocols in Current Use
A metric is informative when its target matches the claim: temporal correlation measures ordering in time, pairwise accuracy a relative choice, outcome accuracy the recognition of an event, and scalar error agreement with a numerical target [118] [10, 11, 16]. High value-order correlation (VOC) on successful videos therefore supports a claim about ordering, whereas recognizing effects or reversals requires tests that contain them, and downstream policy success reflects the evaluator with initialization, exploration, optimization, and the candidate pool.
Benchmarks and System-Level Resources
Dedicated benchmarks broaden these protocols (Table 1): beyond its columns, Progress-Bench [52] tests answerability, RoboPulse++ [161] compares pair and sequence inference, RoboValue [163] branches seeded failures by recovery outcome, and CriticHack [20] stresses a frozen learned reward through policy optimization. Adjacent resources evaluate failure explanation and correction (Robo-Fail [179], AHA [117], RoboFAC [171], ViFailback [180]), process and streaming task understanding (RoboProcess-Bench [181], RoboChrono [182]), and generated robot video (RobotWorldBench [121], RoboGazeBench [123], RoboJudge-Bench [125]). System-level resources supply realistic behavior from which evaluator tests can be built: RoboArena [173] aggregates double-blind pairwise preferences into policy rankings on one embodiment, the DROID Franka setup; RoboChallenge [183] provides large-scale real-robot evaluation on single- and dual-arm robots and the rollouts of RRM-Bench-1K; WorldEval [184] and dWorldEval [185] evaluate policies through generated futures; and a deployment study on the SO-101 arm [186] characterizes naturally occurring failures.
Remaining Gaps
Table 1 compares the test sets on five properties: natural failures arise during the recorded executions, regression labels mark where progress is lost, graded endpoints distinguish zero, partial, and full fulfillment, paired evidence presents one execution under views labeled sufficient and reduced, and scope variants narrow the task within an execution. Partial marks denote failures that are partly constructed or of undocumented origin, regressions constructed by reversing frames, binary endpoints, and evidence varied by editing or mismatching inputs. The other test sets each fully provide at most one of these properties, and none fully provides paired evidence or scope variants, so coverage stays thin where convenient cues disagree with the task.
RRM-Bench-1K: A Capability Benchmark
Relying on a reward model requires that its score rise for the right reason and rest on evidence that shows it. When a robot asked to wipe a stain sweeps a rag across the table and stops while the stain remains (Figure 8(A)), a score that tracks motion, elapsed time, or resemblance to successful videos still rises. RRM-Bench-1K is built around such cases and combines the five properties of Table 1. Natural failures: its 1,117 videos record learned policies on a real robot platform, each execution kept whole with its failures, partial completions, retries, and recoveries. Regression labels: task-state labels record which task conditions hold at each moment and withdraw credit when an achieved condition is lost. Graded endpoints: the final task state grades every video as zero, partial, or full fulfillment. Paired evidence: up to three cameras record each execution, and each view is labeled by whether it shows enough to judge the task. Scope variants: some videos narrow the task scope within an execution. Four suites use these properties.
Four Decisive Cases and Their Contrast Suites
Each suite separates the credited condition from a cue that usually accompanies it in natural executions (Figure 8). A suite pairs an execution that earns credit with a counterpart that shares the cue but not the credit, a hard negative in (A)–(C) and a reduced view in (D). It names the critical evidence that separates them and fixes a gold relation between their judgments, an ordering or an abstention rather than a target value, so each family is tested through its native output (§ 6.4). Success and failure controls anchor each suite. Cases (A) and (B) test proxy quality, whether credit follows the specified object, destination, order, and effect; (C) tests temporal validity, whether credit lasts as long as the credited condition holds; and (D) tests evidence validity, whether the observations support the evaluator's judgment.
(A) Action–Effect separates task-like motion from its effect, which arrive together when an execution succeeds. It includes executions whose motion resembles success while the required effect is verifiably absent, such as wiping with an empty gripper or pressing a switch that does not flip. The terminal effect is the critical evidence: an execution that achieves it must outscore a motion-only attempt, and a score that ranks the two alike follows motion or elapsed time.
(B) Semantic/Ordered Constraint separates a successful manipulation from a correct one. It includes wrong objects or destinations, omitted instances, and violated prerequisites, in which the grasp, transport, and release can all succeed. The critical evidence is the object and its destination read against the instruction, so only an evaluator that checks them ranks the constraint-valid execution higher.
(C) Persistence/Reversal separates an achieved condition from one that still holds. It pairs conditions that persist with ones achieved and later lost; conditions that were never achieved form a separate case. Credit must fall at the reversal: an evaluator that keeps it accumulates history, and one that withdraws it tracks the current task state.
(D) Visibility separates what happened from what a camera shows. It shows executions under reduced camera evidence, most beside a sufficient view of the same execution, so the task state stays fixed while the evidence changes. Without the decisive state in view, the expected behavior is to abstain or widen uncertainty; an evaluator without such an output is measured by how far its judgment degrades.
Natural executions rarely produce cases (A)–(C) on their own (Supplementary § 12): the great majority of task-state falls end a failed attempt or follow a drop or release failure, and lost achievements, wrong destinations, and actions without effect make up the rest. Case (D), by contrast, is common: over a third of the single views give partial or insufficient evidence (§ 6.3). On successful demonstrations each pair of factors moves together, and a test set sampled at random would contain few instances of (A)–(C) and leave (D) unlabeled. The suites concentrate these cases while keeping every video a real execution, and each isolates a capability that dense shaping and monitoring need. No evaluator in § 7 handles all four.
Data from Real Closed-Loop Executions
RRM-Bench-1K is built on closed-loop executions recorded on RoboChallenge Table30 [183] and Table30 V2 [187], a real-robot platform that runs learned policies on standardized tasks and scores each execution with a rubric. Because the policies act in closed loop, their failures arise naturally and are recorded under one protocol. The benchmark covers 52 tasks on five embodiments: three single-arm robots (ARX-5, UR5, and Franka) and two dual-arm robots (Cobot Magic ALOHA and D00S-W1). Up to three platform cameras record each execution; the camera views of whole executions and of their scope variants (§ 6.3) form the 1,117 videos, and a video recorded alongside other cameras can also be scored together with them as one multiview input.
Task-State and Camera-Evidence Labels
We annotate each execution with task units, events, task-state labels, and camera-evidence labels, and derive the scope variants and contrast suites from these annotations. Annotation proceeds in three steps. First, to fix what earns credit, we write for each task the task units of its instruction: object and destination constraints, repeated instances, and, where declared, prerequisites, each with the rule by which it keeps its credit. A placement that must persist loses its credit when undone, whereas a required action stays credited after the robot moves on. Second, to locate when the task state changes, annotators review each execution frame by frame and mark the events: completions, failed attempts, losses, and reversals, with retries and re-achievements where they occur. Each event is time-stamped and linked to a unit, and its type decides whether credit is added, withdrawn, or unchanged. Third, to obtain a dense reference without judging every frame, a rule-based compiler turns the units and events into a score over time for each declared task scope, either the full task or a scope variant that covers one object, one unit, or a window of units under a narrower instruction, applying that scope’s credit, prerequisite, and reversal rules. One set of events thus serves every scope of an execution (Supplementary § 12.1). The resulting label records the task state rather than the motion performed. An action-history reference, which accumulates credit for completed actions, is kept for comparison; the two references end differently in over a third of the executions, and the task-state reference falls at least once in over half.
Whether a condition holds and whether a camera shows it are labeled separately. The state label is read from the views showing the decisive conditions, and each camera view is labeled as giving sufficient (V), partial (P), or insufficient (X) evidence to judge the task. An execution that no view can establish stays unresolved, excluded from state metrics. Most videos are labeled V and about a third P; X is rare.
Evaluation Protocol and Metrics
Eligibility and settings. Each evaluator enters the tests its native query and output support, Relative models through comparisons of observation pairs. The direction test fixes only the sign of a change, so a time-to-go or policy-indexed value (§ 4.2) also enters it, and a terminal grade enters on successive prefixes. Specialized evaluators run through their documented interfaces with native outputs; departures are given in Supplementary § 12.1 and Table S2.
| Evaluator | Family | Input of one query | Native output | 1 FPS curve |
| ProcVLM-2B† | Absolute Progress | All frames (at most 512), eight-observation window | Progress, 0–100% | Interpolated |
| VLAC-2B†, VLAC-8B† | Hybrid | 64 uniform frames, judged in adjacent pairs | Cumulative signed progress | Interpolated |
| Robo-Dopamine-GRM-3B, -8B, GRM-2.0-4B-Preview, -8B-Preview | Relative | Every tenth frame against the first frame and a blank goal image; the view fills all three camera slots | Relative progress, three modes averaged | Read at whole seconds |
| Robo-ValueRL‡ | Temporal Value | Three views of every tenth frame, with one earlier frame of the base camera | Remaining steps ; value | Smoothed as in the official script |
| Robometer-4B† | Hybrid | Eight uniform frames in one pass | Progress in per frame | Interpolated |
| RoboReward-4B† | Outcome | 16 prefixes, each scored as an episode | Grade ; | Interpolated |
| RynnValue-4B†, -8B† | Temporal Value | Eight uniform frames in one pass | Negative remaining time in seconds | Interpolated |
| PRIMO-R1-7B† | Hybrid | 16 prefixes of 32 frames each | Progress, 0–100% | Interpolated |
| LRM-8B (Contrastive)§ | Hybrid | Two adjacent source frames | Signed change in | None |
| LRM-8B (Progress) | Hybrid | One source frame | Progress in | Read at whole seconds |
| SOLE-R1-8B | Absolute Progress | Frames at 1 FPS | Progress, 0–100%, with a rationale | Native |
| TOPReward (Qwen3-VL-8B)† | Absolute Progress | 64 uniform frames, 15 prefixes, in Robometer’s implementation | Log-probability of the answer token, min–max normalized | Interpolated |
| Flora-Rewarder-27B | Hybrid | Each 1 FPS prefix, scored as an episode | Grade ; | Native |
| VLAC-Cut-30B-A3B | Hybrid | Frames at 2 FPS in one pass | Keypoints of time and progress in % | Read at whole seconds |
| GVL (Gemini 3.8 Flash) | Absolute Progress | All 1 FPS frames in shuffled order, with the first frame as a 0% anchor | Completion, 0–100%, per frame | Native |
Table S2. Specialized evaluators as run: family in Table S1, input of one query, native output, and source of the 1 FPS curve. Marks as in Table 2.
Task fulfillment and false credit. Endpoint judgments are compared within each task, whose executions share instruction but differ in final completion. The study asks whether the highest final score selects the most complete execution and how often a failed execution reaches the median final score of fully completed ones. On failures of suites (A) and (B), this share is false credit.
Temporal response. Following RoboPulse++ [161], direction classification labels each 1 FPS transition as rising or falling and reports per-class precision, recall, and F1 with Macro-F1 and accuracy (Table 2). Every non-flat transition of the task-state reference is scored, and an unchanged prediction is a miss for both classes. Event-level tests compare a model’s recall of falls following a specific cause with its own rate of predicting a fall anywhere; a model that predicts falls at random recalls every cause at that rate.
Conventional metrics. VOC correlates predicted values with elapsed time, averaged over successful executions, so it measures temporal ordering where time and task state advance together. Kendall’s is the rank agreement of final predictions with endpoint fulfillment within each task, averaged over tasks, so it measures endpoint ranking (Table 3). Both use native outputs, with VOC in the Pearson form of Robometer [11] and as in Robometer and RynnValue [94].
Evidence and scope. The paired-evidence test scores each P/X video and its V counterpart on the same transitions against one task-state label, so only the camera changes (Table 3). Scope analysis compares each scope variant with the full task of its execution through unit mappings, because a narrower scope can be complete while the full task is not.
Controls. Normalized elapsed time, ignoring video, and a predictor that always answers rising are shortcut controls; the action-history reference (§ 6.3) is a label control.
RRM-Hub Toolbox
RRM-Hub is an open-source toolbox for testing a reward model on the questions of this benchmark. A model enters through an adapter that declares the queries, inputs, and native outputs it supports. The toolbox runs each eligible test, keeps the raw outputs, and reports each capability separately, so a new evaluator is compared with the published results suite by suite. Existing RRMs run on new videos through the same interface. Adapters cover the 14 specialized methods of § 7.1 with their checkpoints and interfaces, and the general-purpose VLMs and Jev 1.14 [188] are queried with fixed prompts (Supplementary § 12.1).
Unified Empirical Study
The study asks what each evaluator’s output credits, at the endpoint and under broken constraints (suites (A) and (B)) and as the task state changes (C), and whether its evidence suffices (D), then turns to the query protocol, task scope, and metrics that explain where these capabilities part. Table 2 and Table 3 and Figure 9 report the results, and Supplementary § 12 gives settings and per-model values.
Evaluated Models
Table 2 lists the 27 evaluators with their sources. Its 19 specialized evaluators come from 14 methods whose native outputs span all five families; Supplementary § 12.1 gives their interfaces and settings. Robo-ValueRL takes only
multiview input, and six others are scored with it too (Section 6.4). Seven proprietary general-purpose VLMs are queried in two styles with a fixed prompt: Pair style presents two observations from one video, and Sequence style the video’s observations in temporal order. The decision model Jev 1.14 [188] returns probabilities over fixed answers and is queried in both styles.
Proxy Validity: Endpoint Selection and False Credit
Endpoint judgments order executions more reliably than dense judgments track change. Within each task whose executions differ in final completion, the highest final score picks the most complete execution in three quarters of cases or more for Sequence-queried GPT-6.1 Sol, Jev 1.14, and Flora-Rewarder, against fewer than half at random (Supplementary § 12.2). Final scores also filter well: the top tenth of the executions, as ranked by Robometer, RoboReward, or GPT-6.1 Sol, holds about twice the share of fully completed executions that a random selection holds. RoboReward, with the lowest falling F1 among specialized evaluators, filters this well: endpoint and dense judgment are separate capabilities.
The failed executions of the Action–Effect and Semantic/Ordered Constraint suites (Figure 8(A)–(B)) divide the evaluators. Under Flora-Rewarder, GPT-6.1 Sol, and Robometer, few reach the median final score of their task’s completed executions; under VLAC-Cut, TOPReward, and VLAC-8B, half or more do. TOPReward also selects the most complete execution less often than chance.
Broken constraints are the hardest case for dense judgments. On the Semantic/Ordered Constraint suite, most evaluators in Table 3 score below the always-rising predictor on Macro-F1, more than on any other suite (Supplementary § 12.2). General-purpose VLMs recall falls after a drop or a release failure well above their overall rate of predicting a fall, but barely exceed that rate for falls after a placement at a wrong destination. Specialized evaluators respond to wrong placements more often, Robometer and VLAC-2B most clearly, yet each recalls at most half of these falls. A drop looks like a setback, whereas a wrong placement looks like progress unless its destination is checked against the instruction, suggesting dense credit follows the appearance of progress more closely than the instruction’s constraints.
Findings.
The strongest final scores select and filter whole executions well, even from an evaluator that responds to few falls: endpoint judgment and dense credit are separate capabilities.
Credit on the failed executions of suites (A) and (B) divides the evaluators, and broken constraints are the hardest case: a wrong placement looks like progress unless its destination is checked against the instruction.
Temporal Validity: Does Credit Track Task State?
No evaluator tracks task-state falls reliably, and the responses take three forms (Table 2). RoboReward and LRM-8B (Progress) leave their output unchanged on about three quarters of the transitions; RoboReward’s five grades are too coarse to register most changes. Most specialized evaluators fluctuate: they predict falls more often than the task state falls, and the median one predicts a fall on a quarter of the transitions where the task state rises. Sequence-queried VLMs rarely predict a fall and miss most falls.
Losses of an achieved condition, the cases of suite (C) (Figure 8(c)), are rare among the falls but draw a clearer response, and Persistence/Reversal is the suite on which the fewest evaluators fall below the always-rising predictor. All VLM rows and most single-view specialized evaluators recall lost achievements above their overall rate of predict-
ing a fall, and Robometer and Robo-Dopamine-GRM-2.0-8B-Preview exceed that rate the most, recalling more than half of them. VLAC-Cut, which leads the single-view specialized evaluators in falling F1, recalls few. In Figure 9, five of the nine evaluators withdraw credit when the shelf falls, whereas PRIMO-R1 and VLAC-Cut end higher than before.
Credit can also follow the action: where the action-history reference advances but the task state does not, the three Pair-queried VLMs best in Macro-F1 predict more rises than where both are flat (Supplementary § 12.3).
Findings.
No evaluator tracks task-state falls reliably: outputs stay unchanged, fluctuate with spurious falls, or rarely fall.
Lost achievements draw clearer responses than other falls, yet the single-view specialized evaluator that detects falls best recalls few, so direction accuracy and sensitivity to lost achievements are separate capabilities.
Evidence Validity: Does the View Matter?
Hiding the decisive evidence (Figure 8(D)) lowers Macro-F1 for nearly every evaluator in Table 3, and no evaluated interface can report insufficient evidence. Views with no task evidence at all reveal whether an evaluator knows it cannot judge. Most Pair-queried evaluators still report progress on them, close to the always-rising predictor, whereas Sequence-queried VLMs hold their output nearly constant, which a direction test against the hidden task state counts as error. An unseen state is neither success nor failure and needs an output and a score of its own (§ 8.4).
Extra cameras help evaluators with separate camera slots. With the video’s camera in the primary slot and its execution’s other cameras in the rest (‡in Table 2), all four Robo-Dopamine checkpoints and SOLE-R1 gain. With three synchronized cameras, a reduced primary view plus the other two beats the sufficient view alone for all five, so the others recover what the primary view hides. Flora-Rewarder, given views tiled in one frame, does not gain. Judgment also depends on the primary camera: swapping it changes the predicted direction more often than rerunning the input (Supplementary § 12.4).
Findings.
Hiding the decisive view lowers Macro-F1 for nearly every evaluator, and where no view shows the task, most Pair-queried evaluators still report progress: no evaluated interface can report insufficient evidence.
Evaluators with separate camera slots recover what a reduced view hides, yet their judgment changes with the primary camera.
evaluators with the lowest falling F1, and Robometer and PRIMO-R1 follow suit. Across the single-view specialized evaluators, VOC and falling F1 are only weakly related, far less than across the VLMs. measures endpoint ranking and also orders the specialized evaluators differently from falling F1, so neither metric replaces a test of credit withdrawal. Outputs themselves lean toward elapsed time: on executions whose task state is non-monotone or ends below completion, nearly two thirds of evaluators correlate more with elapsed time than with task state on most of them. Because nine in ten transitions rise, the always-rising predictor beats half of the single-view specialized evaluators on Macro-F1, so the falling class carries the discrimination.
Findings.
Switching from Pair to Sequence queries trades falling recall for falling precision, so the query is part of the evaluator.
VOC rewards chronology: a control that reads only elapsed time exceeds every evaluator on it, so credit withdrawal needs a test of its own.
Summary of Findings
Three findings separate capabilities that a single leaderboard merges. Endpoint judgment and dense credit: RoboReward filters among the best and responds to the fewest falls, whereas VLAC-Cut leads the single-view specialized evaluators in falling F1 yet gives success-level credit to half the failed executions of suites (A) and (B), so which evaluator to use depends on the decision it serves. Execution progression and task fulfillment: wrong placements, which look like progress unless checked against the instruction, make broken constraints the hardest case for dense judgments, and most evaluators follow elapsed time more than task state where the two diverge; the procedure and constraint checks of § 4 target this gap. Evidence and its use: no evaluator signals that the decisive view is missing, extra cameras help where the interface fuses them, and the query style moves results by up to 0.17 Macro-F1.
Challenges and Research Directions
Scale alone does not close the gaps of § 7.5: within a model series, the larger checkpoint scores much like the smaller one (Table 2). The gaps trace instead to what evaluators learn from, how they represent the task, how they are queried, what they observe, and how they are measured and used, and the directions below follow this order.
Learning from Real Failures
Detecting drops and recognizing wrong destinations are different abilities: VLMs register the first far more often than the second (§ 7.2). Most task-state falls end a failed attempt or follow a drop (§ 6.1), whereas reward models learn mostly from successful demonstrations and synthetic negatives (§ 5.2). RoboMIND [167] annotates real teleoperation failures by cause, and VLAC-Cut [119], trained with signed progress on executions with failure, regression, and recovery, leads single-view specialized evaluators in falling F1 but recalls few lost achievements (Section 6.3).
Closed-loop platforms and deployment studies already log such failures at scale [173, 183, 186]. The open question is what supervision each failure mechanism needs: signed targets letting credit fall, labels by cause (failed attempt, drop, wrong destination, lost achievement), and constraint labels for errors motion does not reveal. A gain would show as per-cause fall recall above the evaluator’s fall rate, without more false falls where the task advances.
Task Structure Beyond the Instruction
An instruction names the goal but rarely the phases that reach it or the constraints that make it correct. Evaluators that register a drop still miss a placement at the wrong destination (Figure 8(B), § 7.2), and goal-distance anchors can read necessary detours as regressions (§ 4). Structure has entered reward models as stage-decomposed rewards and values [85, 91, 92] [12], as stage reasoning before scoring [54, 55, 56], and as post-conditions Thea [127] checks at runtime. Evaluators that state the current stage before scoring still predict a fall on about a third of transitions where the task advances (Supplementary § 12.3). The question is how to combine judgments of physical state change with task-constraint checks. A structured specification of a task’s units, constraints, and order can condition progress within phases such as approach, transport, and release, while post-condition checks verify what motion does not reveal, such as the destination or required effect (Figure 8(A)–(B)). Lower false credit on suites (A) and (B) at unchanged endpoint ranking would measure the gain.
Query Design
Pair queries recall more falls with more false alarms, Sequence queries fewer with fewer, and the same backbone scores differently under each protocol (Section 6.5). Existing work varies the query: GVL [16] asks for the completion of shuffled frames, TOPReward [53] reads token probabilities, and RoboRMBench [159] tests consistency under paraphrased instructions. The prompt, query style, frame selection, and output format therefore belong to an evaluator’s specification (§ 4.4) and to every reported result. The research question is how local event comparison and long-horizon task state can combine, as in protocols that compare adjacent observations within whole-video context, or prompts that spell out a task’s structure and test whether the evaluator uses it. Continuous, calibrated outputs avoid coarse ordinal levels that miss most changes. A protocol that combines the falling recall of Pair queries with the falling precision of Sequence queries would be an advance.
Evidence Sufficiency and Abstention
Reduced views lower Macro-F1 for nearly every evaluator, and where no view shows the task, most Pair-queried evaluators still report progress (Section 6.4). An evaluator must recognize when it cannot judge, since a judgment without evidence misleads filtering, shaping, and monitoring. Three capabilities supply this. Abstention reports insufficient evidence as a state separate from a low score; the closest existing interface, the unanswerable response of ProgressLM [52], flags a mismatch with the reference demonstration rather than a missing view. Calibration lets confidence fall with the evidence: WARR- RM [49] predicts a distribution over progress and RARM [47] gates uncertain matches, but neither has been tested against what the camera shows. Active evidence acquisition selects, fuses, or requests the decisive view: AnyviewMeter [51] adapts Robometer [11] to multiview input via camera-ray conditioning, Robo-Dopamine [9] reads synchronized views, and extra cameras help where the interface fuses them. Training with variable camera sets and missing views builds these capabilities, and per-view evidence labels, as in RRM-Bench-1K, make them testable. An advance would show as abstention without a decisive view, confidence following the evidence label, and multiview gains above a vote over single views from whichever camera fills the primary slot.
Measuring What Matters
VOC rewards chronology, and the always-rising predictor outscores half of the single-view specialized evaluators on Macro-F1, because elapsed time and task state mostly advance together (Section 6.5). Learned evaluators with a high VOC can keep their credit when the task state collapses (Figure 9). Progress metrics therefore need the normalized-time and always-rising references beside them, the falling class in its own column, and credit withdrawal tested per event, with kept as the measure of endpoint ranking. Test sets need executions in which time and task state diverge, such as failures, partial completions, and lost achievements, labeled from what physically happened. Views without task evidence need their own scoring: there an abstention or an unchanged output is correct, and a score against the hidden task state penalizes the evaluator that recognizes it cannot judge. RRM-Hub reports these capabilities separately, so a new evaluator can show which of them it improves.
From Evaluators to Decisions
Task-state fidelity and decision utility are different properties, because each decision rule (§ 3.1) consumes a different capability. Selection among candidate actions or executions and filtering of data need only the order of whole executions, which the strongest evaluators provide, including some that respond to few falls (§ 7.2). Monitoring needs failures detected as they occur, and recovery needs to know whether a setback can still be repaired, which a failure forecast or answers and current-state credit does not. Completion, time to go, and the chance of future success respond differently to a setback (§ 4.2) and serve acceptance, scheduling, and intervention, respectively. Shaping makes the strongest demand, since the policy optimizes against the reward and seeks executions where motion, time, or appearance part from the task state, the cases of suites (A)–(C); optimization against a frozen reward can raise both task success and wrong-object failures [20]. The fidelity a decision needs grows with the optimization pressure it puts on the reward. Shaping and monitoring need credit that falls with the task state, including at placements that violate the instruction, and every rule must recognize missing evidence; no evaluated model meets these demands yet (§ 7.2–6.4). Joint models keeping these outputs separate and consistent can serve several rules.
Evaluators already act inside the policy loop, as a causal failure-risk head in FIERCE [115], a candidate verifier in DiVeR [83], and an intervention gate in Reward-DAgger [177], where a strong candidate pool can mask a weak verifier (§ 4.1). The offline selection and filtering tests of § 7.2 lead to their closed-loop counterparts: candidate selection, rejection of known-bad behavior, and controlled policy updates under deployment conditions, including control deadlines, with each evaluator chosen for its role and tested with its rule. Full policy optimization comes last, because it places the most pressure on the reward.
Conclusion
In the foundation-model era, reward names several questions rather than one quantity. This survey organizes robotic reward models into five semantic families (Outcome, Absolute Progress, Relative, Temporal Value, and Hybrid) by the question their native outputs answer and relates each design route to the quantity it estimates, its supervision, and its evidence. Relying on such an evaluator takes three answers: what its output credits, whether its evidence suffices, and which decisions it can drive. RRM-Bench-1K tests the first two on whole real-robot executions, with contrast suites that separate the credited condition from its usual cues, task-state labels that withdraw lost credit, and camera views labeled by the evidence they show. The strongest final scores rank executions well enough for selection and filtering, but dense credit follows the progression of an execution more closely than its task state, VOC hides this gap because it rewards chronology, and no evaluator signals when the decisive view is missing. The query protocol and the evidence interface change these results and so belong to the evaluator.
The third answer depends on the decision: the fidelity a rule needs grows with the optimization pressure it places on the reward, so dense shaping and monitoring await evaluators that withdraw credit when an achieved condition is lost, verify constraints that motion does not reveal, and abstain without the decisive view. The path runs through real failures in training data, structured task specifications and queries, evidence-aware interfaces, and metrics that separate task state from chronology, and RRM-Hub lets new evaluators be tested on each of these capabilities. A policy learns what its reward credits, so general-purpose robots will be only as reliable as their evaluators.
Biographies
Guoqiang Liang
Guoqiang Liang is currently a Ph.D. student at Show Lab, National University of Singapore (NUS). Liang’s research interests include visual generation and embodied AI.
Yiqi Lin
Yiqi Lin is currently a Ph.D. student at Show Lab, NUS. Lin’s research interests include multimodal learning, video generation and editing, vision–language models, and robot learning.
Shangchen Zhou
Shangchen Zhou received the Ph.D. degree in computer science from Nanyang Technological University (NTU), and then joined MM-Lab@NTU as a Research Assistant Professor. Zhou’s research interests include image and video enhancement, generation, and editing.
Wenzheng Zeng
Wenzheng Zeng is currently a Ph.D. student at Show Lab, NUS. Zeng’s research interests include video understanding, multimodal foundation models, and their applications to robotics.
Yanzhe Chen
Yanzhe Chen is currently a Ph.D. student at Show Lab, NUS. Chen’s research interests include agents, robotics, and multimedia.
Zechen Bai
Zechen Bai is currently a Ph.D. candidate at Show Lab, NUS. Bai’s research interests include multimodal foundation models and agents.
Mike Zheng Shou
Mike Zheng Shou received the Ph.D. degree in electrical engineering from Columbia University, and was a Research Scientist at Facebook AI, before joining NUS. Where Shou is currently an Associate Professor and leads Show Lab. Shou is a Fellow of the National Research Foundation (NRF) Singapore, serves as an Associate Editor of IEEE TPAMI, and has served as an Area Chair for CVPR, ECCV, ICCV, and ACM MM. Shou’s research interests include computer vision and multimodal learning.
Method Inventory
Query evidence. Image: one observation; Video: a temporal window; Traj.: a complete trajectory (visual or structured state); Pair: two observations or clips queried jointly; Multi-view: synchronized camera views; Proprio.: proprioceptive state; Latent: policy-internal features or distributions; Sim.: privileged simulator state; State: structured state features; Future: a model-predicted observation. Task: instruction, task ID, or evaluative question; Goal: goal image/configuration; Init.: initial observation; Ref.: reference observations, demonstration, or support set; Criterion: preference axis/post-condition; Graph: structured scene context; Action: candidate action; Act. hist.: preceding actions; Hist.: retained execution or reasoning context; Geometry: camera calibration/rays; Rationale: the candidate policy’s proposed reasoning. A slash denotes alternatives; Pair is mandatory and (pair) is optional. Task identity may be fixed in the fitted evaluator rather than supplied with each query. Teacher inputs used only during training are omitted.
Component boundaries. Robo-Dopamine 2.0 is consolidated with the original method. DecisionNCE-T and Rank2Rew’s ranking head have relative semantics; the potential variant and density regularizer are separate constructions. AnyviewMeter is a geometry-conditioned Robometer adaptation. FAVOR and TeViR use generated references to assess actual execution. Small Stuff records the observed-pair interface. HiRE uses fixed historical support buffers; ARS requires complete rollouts. LLM-as-a-Verifier records the robotics instantiation of a cross-domain protocol. VeriFine judges proposals and reasoning rather than executed outcomes. VLLR’s VLM progress only initializes the critic; self-certainty and the sparse success reward drive policy optimization.
Evaluation Details and Additional Results
This section records the details behind § 6 and § 7. Section S2.1 gives the data sources, the annotation procedure, the input, scoring, and reporting rules, and the settings fixed for each evaluated model, with the protocol of each specialized evaluator (Table S2), the prompts of the general-purpose VLMs, and the run conditions that differ between requests. Sections S2.2–S2.5 give the per-model values behind the findings of § 7.2–6.5.
Evaluation Settings
Data sources. The RoboChallenge index lists 44,461 finished rollouts on tbl:30 and tbl:30 V2, more than 120,000 camera videos. A candidate pool of 1,500 rollouts, 750 per version and 4,228 videos, is drawn across zero, partial, and full platform scores. Annotators screen this pool, with a small set of added candidates, in the workbench of Figure S1 and annotate the executions they keep, whose camera views and scope variants form the 1,117 videos.

Figure S1. Annotators verify model-proposed events against the video before an execution enters the benchmark. In the annotation workbench, models mark candidate events on the video timeline (S: start, E: completion, R: failure or reversal). The annotator checks each claim in time order with Agree or Disagree, may assign a diagnostic category, marks actions the models missed, and then keeps, excludes, or flags the execution. Shown: an execution of wipe the table at 7.0 s.
Task-state labels. Dense targets combine the human-reviewed events of § 6.3 with rule-based state construction. Annotators verify model-proposed events against the video and mark the events the models miss (Figure S1). Each event keeps its provenance (reviewed mark, approximate anchor, or derived transition) and a timing-quality flag. The transient credit that the compiler grants during an attempt is withdrawn if the attempt fails; the provenance flag keeps such a rule-derived peak from being mistaken for a physical reversal, which requires evidence that the condition was first attained and then lost. Figure S2 shows the label of the execution of Figure 9 with its units and events. The normalized task-state score is an operational score defined by the task rubric. Because the state label comes from the views that show the decisive conditions, a placement that a wrist camera misses keeps its label. The platform’s rubric score is a separate target, reported apart from the task-state label.

Figure S2. The task-state label withdraws credit when an achieved condition is lost, and every change traces to a time-stamped event linked to a task unit. Synchronized views of the put-the-books-back execution of Figure 9 at 19.6 s, as the robot sweeps the shelved book and the shelf off the table, above the compiled label (blue) and the action-history reference (gray, dashed); below, the four task units and the events behind the curve. Three failed attempts rise and fall back, and the sweep revokes both completed units, so the label ends at 0 and the reference at 0.25.
Inputs. Sampling, image resolution, prompts, decoding, and score transformations are fixed for each model. Each model receives its documented evidence, including reference demonstrations or goal images where required.
Scoring. Unlike RoboPulse++ [161], which labels annotated intervals, smooths continuous outputs, and drops steps near interval boundaries, we score every non-flat transition of the task-state reference without smoothing or boundary filtering. Because an unchanged prediction counts as a miss for both classes, it lowers recall and accuracy but not precision. VOC and , defined in § 6.4, are computed on native outputs with the implementation of Robometer [11].
Specialized evaluators. The open-checkpoint roster comprises ProcVLM [56], VLAC [109] at 2B and 8B, Robo-Dopamine [9] (GRM-3B, GRM-8B, and the GRM-2.0 4B and 8B previews), Robo-ValueRL [103], Robometer [11], RoboReward [10], RynnValue [94] at 4B and 8B, PRIMO-R1 [118], Large Reward Models (LRM) [110], SOLE-R1 [55], TOPReward [53] applied to Qwen3-VL-8B-Instruct [189], VLAC-Cut [119], and Flora-Rewarder [116]. GVL [16] runs on the proprietary Gemini 3.8 Flash [196]. Table S2 gives the family, the input of one query, the native output, and the source of the 1 FPS curve of each. For models with several inference formulations, we evaluate RoboReward, PRIMO-R1, and Flora-Rewarder on the episode endpoint and on successive prefixes, and LRM through its completion, contrastive, and progress interfaces. The open models receive single-view videos. Robo-ValueRL requires several camera views at once and is scored only with multiview input; Robo-Dopamine, SOLE-R1, and Flora-Rewarder are scored with both (§ 12.4).
Flora-Rewarder serves terminal, dense, and rubric prompts from one set of weights. We use its terminal head, whose 1–5 episode grade gives the progress value under the score-token probabilities, and score every 1 FPS causal prefix as a complete episode with the upstream frame selection, time metadata, and prompt. Its dense head grades action quality rather than progress and its rubric head requires per-task rubrics, so neither is evaluated. Robo-Dopamine scores the change between two observations, each filling three camera slots, against a start and a goal image. A single view fills all three slots, with the video’s first frame as the start and the official blank goal image, and the forward, incremental, and backward estimates are averaged as in its paper. The GRM-2.0 preview cards document this use; the GRM 1.0 cards show three views with a goal image; § 12.4 reports all four checkpoints with the other cameras in the remaining slots.
For direction tests, each model’s native curve is differenced at 1 FPS; curves saved only at sparse time points are interpolated and marked as such, and specialized evaluators have no Pair-style result. LRM’s contrastive interface compares the current frame with an earlier one; we query it on adjacent source frames (about 30 FPS) and mark these results accordingly. Terminal grades and LRM’s completion interface answer an outcome question and are not scored for direction.
General-purpose VLMs. The VLMs share one system prompt and one user template per query style, written after the method description of RoboPulse++ [161] and reproduced below. A request holds the rendered template and then, for each observation, a label and its image at native resolution; no timestamp, frame number, or camera name is sent. In Sequence style, one request holds all 1 FPS observations of a video in temporal order, a median of 15 and at most 110, and the model returns a completion value from 0 to 100 for each. The predicted direction of a transition is the sign of the difference between consecutive values, and equal values are an unchanged prediction. In Pair style, the two observations of each transition form their own request, labeled BEFORE and AFTER and sent without earlier context, and the model answers +1 or −1. Answers are parsed strictly, apart from format wrappers such as a Markdown fence, and a request without a valid answer leaves its transitions without a prediction. With three views (§ 12.4), each observation lists its three images under the same templates. GVL keeps its own prompt: it receives the 1 FPS frames in shuffled order with the first frame as a zero anchor, and its scores are returned to temporal order.
System prompt, both styles.
You evaluate progress in robot manipulation tasks from visual observations. Use the given task instruction to identify the required physical goal and compare task-relevant states. Judge visible evidence rather than assuming that later observations must show more progress. Progress may increase, decrease, or remain unchanged, and the execution may end without success. Treat the task instruction as the goal to evaluate. Follow the requested output format exactly.
User prompt, Pair style.
Task instruction: <task_instruction> $task_instruction </task_instruction>
You will receive two temporally adjacent observations, labeled BEFORE and AFTER. Images under the same observation label are simultaneous views, not additional time steps. Compare the physical task progress in AFTER with BEFORE. Consider the task-relevant objects, their relations to the goal, and any achieved state that is lost. Ignore appearance changes that do not affect the task. Do not use elapsed time alone as evidence of progress. Make a forced binary direction judgment: +1: AFTER is closer to satisfying the task than BEFORE. −1: AFTER is further from satisfying the task than BEFORE. If the visual evidence is ambiguous, choose the direction best supported by the observations. This binary baseline has no unchanged or abstention output. Return exactly one of the following, with no explanation: <score>+1</score> <score>-1</score>
User prompt, Sequence style.
Task instruction: <task_instruction> $task_instruction </task_instruction>
You will receive observations in chronological order, indexed from 0. Images under the same observation index are simultaneous views, not additional time steps. Estimate absolute physical task completion for every observation on a consistent scale from 0 to 100. Zero means no achieved task progress; 100 means that all requirements of the task are satisfied. Use intermediate values for partial completion. Base the scores on task-relevant physical states and the available temporal context. An achieved state that is undone can reduce the score. Unchanged states can receive equal scores. Do not force monotonic increases, do not force the first observation to 0, and do not force the last observation to 100. A clip can begin during an ongoing task and can end in failure. Elapsed time and robot motion alone do not establish completion. Return a JSON object with exactly one key, "progress". Its value must be an array of exactly finite numbers in [0, 100], in the supplied observation order. Return no Markdown or explanation.
GPT-6.1 Sol [190] runs at low reasoning effort and GPT-6 Luna [191] and GPT-5.6 Terra [192] at medium. Claude Opus 5.5 [193], Claude Sonnet 5.5 [194], and Claude Haiku 5.5 [195] run at medium effort and original image scale, and Gemini 3.8 Flash runs at its medium thinking level. Sequence request takes a median of 8.8k–17.5k input tokens, depending on the VLM.
Decision model. Jev 1.14 [188], from TypeSafe AI, returns probabilities over fixed answer options instead of text. It is queried in both styles with the VLM task templates, and both styles use the same frames, letterboxed to pixels at JPEG quality 70. In Sequence style, each video is sent once with all its 1 FPS observations and one five-level completion question per observation (0, 25, 50, 75, or 100% complete), and the expected level under the returned probabilities is its progress value. In Pair style, it sees the two observations of each transition and chooses between rise and fall; the more probable option is its prediction.
RRM-Hub records. Each adapter declares its source paper, the official code it calls at a pinned commit, its weights, the camera views it requires, its default frame sampling, and the type and range of its native output, which is kept without rescaling. The benchmark labels give each video’s execution, task scope, camera view, evidence label, and suites, so each result can be traced to the videos it covers. The licenses of the wrapped code and weights are listed separately.
Reporting. Table 2 reports progress-direction classification for every evaluated model over 17,381 non-flat 1 FPS transitions, 1,702 of them falling; transitions without a prediction, at most 1,323 per 1 FPS row, are counted separately and excluded from its direction metrics. Within a model series, the larger checkpoint changes Macro-F1 by −0.007 to +0.027 (Table 2). Table 3 lists the 26 evaluators with a single-view curve; Robo-ValueRL, LRM-8B (Contrastive), and the Pair-style queries yield none.
Reward profiles. Each panel of Figure 9 and of Figure S3–Figure S6 shows the highest-VOC model (Table 3) of the group in its subtitle at native timestamps, after a fixed unit conversion. Two exceptions apply: the general-purpose VLM group shows its highest- and lowest-VOC models, Claude Opus 5.5 and GPT-6 Luna, and the 4B panel shows Robo-Dopamine-GRM-2.0-4B-Preview (third of four by VOC), the only 4B model with dense outputs.
Figure S3. Most evaluators credit the watering motion rather than its effect. In this execution from the Action–Effect suite (A), the watering motion ends at 11.6 s without watering the plant, and the can later tips over, so the task-state reference (GT, gray) falls to zero. Claude Opus 5.5 and five others keep or raise their credit after the failed watering; only GPT-6 Luna, SOLE-R1, and Robo-Dopamine-GRM-2.0 lower it. Layout as in Figure 9.
Figure S6. Without the decisive view, most evaluators withhold the credit they give the same execution when its outcome is visible. In this successful execution from the Visibility suite (D), the robot opens a drawer, places a bottle opener inside, and closes it; the wrist camera (P) faces the drawer front and does not show the placement, whereas the front camera (V) shows every step. Claude Opus 5.5 ends at 1.00 on the front view but 0.04 on the wrist view, and PRIMO-R1 at 1.00 but 0.33; Robo-Dopamine-GRM-2.0 instead ends higher on the wrist view. Layout as in Figure 9, with each model’s output on both views.
Proxy Validity
The first analysis covers the Action–Effect and Semantic/Ordered Constraint suites and their controls, with Action–Effect executions scored on views that show the effect.
Endpoint judgments order executions more reliably than dense judgments track change. In each of the 28 tasks whose executions differ in final completion, we select the execution with the highest final score; ties are averaged, and VLMs are queried in Sequence style. The selected execution is the most complete one in 85% of cases for GPT-6.1 Sol, 82% for GPT-5.6 Terra, and 75% for Jev 1.14 and Flora-Rewarder, against 44% for a random choice, and TOPReward (38%) falls below chance. The 33 zero-completion executions of 11 tasks show how much endpoint credit a failed execution receives. The share of them that reaches the median final score of fully completed executions of the same task is 15% for GPT-6.1 Sol and Claude Sonnet 5.5, 18% for Flora-Rewarder, and 21% for Robometer, but 58% for TOPReward, ProcVLM, and Robo-Dopamine-GRM-3B and 76% for LRM-8B (Progress), whose output rarely changes. Measured the same way on the 34 failed executions of the Action–Effect and Semantic/Ordered Constraint suites, false credit is 3% for Flora-Rewarder, 6% for GPT-6.1 Sol, 9% for GPT-5.6 Terra and Robometer, and 12% for Claude Opus 5.5 and Jev 1.14, but 50% for VLAC-Cut, 56% for TOPReward, Robo-Dopamine-GRM-3B, and LRM-8B (Progress), and 72% for VLAC-8B.
Final scores also work as a filter. When the executions are ranked by final score and the top 10% kept, Robometer keeps only fully completed executions, RoboReward 93%, Flora-Rewarder and RynnValue-8B 90%, and GPT-6.1 Sol 87%, whereas a random selection holds 47%. RoboReward reaches this precision with the lowest falling F1 among specialized evaluators (0.072). The GRM-2.0 previews keep 50%, and ProcVLM (45%) and VLAC-8B (40%) do no better than random.
Among the suites, semantic constraints are the hardest for dense judgments (Table S3). On the transitions of the Semantic/Ordered Constraint suite, 19 of the 26 evaluators score below an always-rising predictor, more than on any other suite; the Sequence-queried VLMs reach a falling F1 of at most 0.11 there, and the best falling F1 is VLAC-Cut’s 0.229. Even on the clean-success controls, where nearly every predicted fall is an error, 21 of the 26 score below the always-rising predictor.
| Model | (A) Action–Effect M | (A) Action–Effect F | (B) Semantic/Ordered M | (B) Semantic/Ordered F | (C) Persistence/Reversal M | (C) Persistence/Reversal F | (D) Visibility M | (D) Visibility F | Control Clean success M | Control Clean success F | Control Obvious failure M | Control Obvious failure F |
| Specialized evaluators, 2B–4B | ||||||||||||
| ProcVLM-2B{sup}†` | 0.459 | 0.247 | 0.407 | 0.145 | 0.469 | 0.257 | 0.411 | 0.168 | 0.369 | 0.014 | 0.572 | 0.487 |
| VLAC-2B{sup}†` | 0.241 | 0.146 | 0.269 | 0.136 | 0.395 | 0.212 | 0.322 | 0.162 | 0.352 | 0.025 | 0.269 | 0.337 |
| Robo-Dopamine-GRM-3B | 0.372 | 0.203 | 0.374 | 0.145 | 0.423 | 0.263 | 0.351 | 0.170 | 0.266 | 0.019 | 0.342 | 0.344 |
| Robo-Dopamine-GRM-2.0-4B-Preview | 0.476 | 0.235 | 0.440 | 0.169 | 0.518 | 0.312 | 0.452 | 0.212 | 0.398 | 0.033 | 0.544 | 0.467 |
| Robometer-4B{sup}†` | 0.519 | 0.144 | 0.515 | 0.147 | 0.572 | 0.262 | 0.529 | 0.205 | 0.507 | 0.067 | 0.556 | 0.306 |
| RoboReward-4B{sup}†` | 0.283 | 0.107 | 0.205 | 0.038 | 0.244 | 0.082 | 0.195 | 0.052 | 0.232 | 0.000 | 0.196 | 0.000 |
| RynnValue-4B{sup}†` | 0.494 | 0.264 | 0.460 | 0.161 | 0.542 | 0.299 | 0.456 | 0.187 | 0.434 | 0.027 | 0.567 | 0.429 |
| Specialized evaluators, 7B–8B | ||||||||||||
| PRIMO-R1-7B{sup}†` | 0.431 | 0.154 | 0.428 | 0.144 | 0.428 | 0.171 | 0.424 | 0.157 | 0.395 | 0.045 | 0.383 | 0.174 |
| LRM-8B (Progress) | 0.237 | 0.153 | 0.176 | 0.081 | 0.206 | 0.175 | 0.213 | 0.138 | 0.151 | 0.014 | 0.319 | 0.235 |
| Robo-Dopamine-GRM-8B | 0.402 | 0.212 | 0.389 | 0.152 | 0.445 | 0.275 | 0.386 | 0.187 | 0.307 | 0.019 | 0.378 | 0.396 |
| Robo-Dopamine-GRM-2.0-8B-Preview | 0.481 | 0.246 | 0.436 | 0.158 | 0.504 | 0.289 | 0.448 | 0.201 | 0.400 | 0.029 | 0.532 | 0.460 |
| RynnValue-8B{sup}†` | 0.526 | 0.265 | 0.445 | 0.162 | 0.546 | 0.296 | 0.449 | 0.188 | 0.436 | 0.023 | 0.542 | 0.417 |
| SOLE-R1-8B | 0.532 | 0.257 | 0.480 | 0.170 | 0.523 | 0.278 | 0.498 | 0.208 | 0.421 | 0.028 | 0.594 | 0.483 |
| TOPReward (Qwen3-VL-8B){sup}†` | 0.445 | 0.206 | 0.406 | 0.122 | 0.461 | 0.237 | 0.427 | 0.169 | 0.367 | 0.021 | 0.477 | 0.391 |
| VLAC-8B{sup}†` | 0.293 | 0.163 | 0.248 | 0.113 | 0.426 | 0.212 | 0.348 | 0.153 | 0.384 | 0.034 | 0.381 | 0.440 |
| Specialized evaluators, 27B–30B | ||||||||||||
| Flora-Rewarder-27B | 0.555 | 0.274 | 0.501 | 0.185 | 0.568 | 0.317 | 0.511 | 0.208 | 0.459 | 0.037 | 0.605 | 0.478 |
| VLAC-Cut-30B-A3B | 0.568 | 0.232 | 0.568 | 0.229 | 0.623 | 0.335 | 0.561 | 0.205 | 0.506 | 0.045 | 0.414 | 0.034 |
| Specialized evaluators, proprietary backbone | ||||||||||||
| GVL (Gemini 3.8 Flash) | 0.532 | 0.253 | 0.486 | 0.187 | 0.556 | 0.295 | 0.506 | 0.227 | 0.449 | 0.013 | 0.602 | 0.476 |
| General-purpose VLMs, Sequence style | ||||||||||||
| GPT-6.1 Sol | 0.561 | 0.352 | 0.412 | 0.114 | 0.589 | 0.438 | 0.494 | 0.291 | 0.556 | 0.308 | 0.410 | 0.105 |
| GPT-6 Luna | 0.425 | 0.198 | 0.276 | 0.073 | 0.446 | 0.308 | 0.343 | 0.182 | 0.413 | 0.147 | 0.400 | 0.140 |
| GPT-5.6 Terra | 0.421 | 0.140 | 0.254 | 0.007 | 0.402 | 0.271 | 0.302 | 0.136 | 0.362 | 0.087 | 0.346 | 0.073 |
| Claude Opus 5.5 | 0.617 | 0.333 | 0.492 | 0.112 | 0.655 | 0.419 | 0.563 | 0.243 | 0.651 | 0.357 | 0.472 | 0.172 |
| Claude Sonnet 5.5 | 0.530 | 0.204 | 0.416 | 0.042 | 0.526 | 0.221 | 0.479 | 0.137 | 0.469 | 0.000 | 0.432 | 0.107 |
| Claude Haiku 5.5 | 0.441 | 0.103 | 0.354 | 0.034 | 0.393 | 0.151 | 0.372 | 0.084 | 0.408 | 0.030 | 0.462 | 0.140 |
| Gemini 3.8 Flash | 0.503 | 0.194 | 0.401 | 0.031 | 0.610 | 0.386 | 0.533 | 0.268 | 0.556 | 0.222 | 0.407 | 0.037 |
| Decision model, Sequence style | ||||||||||||
| Jew 1.14 | 0.551 | 0.253 | 0.496 | 0.159 | 0.566 | 0.318 | 0.525 | 0.232 | 0.457 | 0.041 | 0.564 | 0.412 |
Table S3. Direction results by contrast suite and control group: Macro-F1 (M) and falling F1 (F). Gray: Macro-F1 below the always-rising predictor on the same transitions. Bold: best in each panel. {sup}†`Curve interpolated to 1 FPS.
General-purpose VLMs keep crediting placements at a wrong destination. On the 54 falls that follow a placement at a wrong destination (§ 12.3), no VLM row exceeds its overall rate of predicting a fall by more than 0.04: Pair-queried VLMs recall 4–35% of these falls and Sequence-queried VLMs 0–6%. On the 704 falls that follow a drop or a release failure, every VLM row exceeds it by 0.10–0.36. Specialized evaluators respond to wrong placements more often: Robometer and VLAC-2B exceed their fall rate by 0.19, and ProcVLM, SOLE-R1, VLAC-Cut, and Flora-Rewarder by 0.10–0.15, although none recalls more than half of these falls.
Temporal Validity
The second analysis separates direction recognition from the response to specific events. Rule-derived attempt peaks form a separate slice from verified physical reversals, and an uncompleted final step counts as an omission, not as the loss of a previously achieved state.
The falling class consists mostly of short local events. Of the 1,702 falling transitions of Table 2, 863 (50.7%) end a failed attempt and 704 (41.4%) follow a drop or a release failure. Another 54 (3.2%) follow a placement at a wrong destination, and 15 follow an action that completed without its effect. Only 66 (3.9%) follow the loss of a previously achieved condition, in seven executions.
Evaluators respond to these events in three ways. RoboReward and LRM-8B (Progress) leave their output unchanged on 73% of the transitions. Most single-view specialized evaluators fluctuate instead: 15 of the 18 predict a fall on 13.9–44.3% of the transitions, although only 9.8% fall, and 17 of the 18 have a falling precision of 0.111–0.209, 1.1–2.1 times the base rate. On the transitions where the task state rises, the median specialized evaluator predicts a fall 25% of the time. Apart from the near-constant RoboReward, this rate ranges from 7.6% for VLAC-Cut to 44% for TOPReward, with 34% for ProcVLM and 31% for SOLE-R1; Pair-queried GPT-6.1 Sol predicts a fall on 8.0% of these transitions. Sequence-queried VLMs predict a fall on only 1.9–3.9% of the transitions, leave the score unchanged on 16–61% of them, and recall 7.5–24.4% of the falls. In Sequence queries, Jev 1.14 scores by an expected level that varies almost continuously, so it leaves the score unchanged on 4.3% of the transitions; like most specialized evaluators, it predicts a fall on 22.8% of them, with a falling precision of 0.183.
Switching from Pair to Sequence queries raises falling precision and lowers falling recall for all seven VLMs. The query protocol alone changes the result: Gemini 3.8 Flash reaches a Macro-F1 of 0.528 under the GVL protocol, 0.564 with Sequence queries, and 0.621 with Pair queries, and the Pair and Sequence results of one VLM differ by up to 0.171 in either direction. Jev 1.14 depends on the query as well: its Pair and Sequence answers agree on only 65% of the transitions, and unlike six of the seven VLMs it scores higher with Sequence queries (Macro-F1 0.543 against 0.509), which raise both its falling precision and its falling recall.
Figure 9 shows these responses on one execution chosen from reversal-rich executions. GPT-6 Luna follows the shape of the reference at a third of its height, and ProcVLM, SOLE-R1, and both Robo-Dopamine models fluctuate while the task state rises. When the shelf falls, five of the nine models drop to 0.05 or below, but PRIMO-R1 and VLAC-Cut end higher than before and Flora-Rewarder stays near 0.5, although all three have a higher VOC than Robo-Dopamine-GRM-3B (0.324), which drops.
Figure S3–Figure S5 show one more execution from each of suites (A)–(C) in the same layout.
Figure S5. Through repeated failures, only some evaluators follow the task state. In this execution from the Persistence/Reversal suite (C), three attempts fail before two flowers stay in the vase, so the task-state reference ends at 0.5. Both Robo-Dopamine models withdraw their credit at each failure and end near the reference, whereas VLAC-Cut and PRIMO-R1 rise through every failure. Claude Opus 5.5 rises during each carry and falls back when the carry ends, whether or not the flower stays, and so ends near zero.
The task-state and action-history references end differently in 73 executions, and the task-state reference falls at least once in 122. Where the two references disagree, the evaluators split. On the 922 transitions where the action history advances but the task state does not, Pair-queried GPT-6.1 Sol, Gemini 3.8 Flash, and Claude Opus 5.5 predict a rise 0.09–0.10 more often than where both references are flat, so they follow the action history. Flora-Rewarder, VLAC-Cut, and SOLE-R1 show no such difference (−0.11 to +0.01). Instead, they predict rises at high rates throughout, including on 52%, 72%, and 51% of the transitions where the task state falls; Robometer does so on 74%.
Responses to specific losses separate the evaluators further, again measured against each row’s overall rate of predicting a fall. Sequence-queried VLMs recall 12–39% of the 704 falls that follow a drop or a release failure, although they predict a fall on only 1.9–3.9% of transitions. On the 66 falls that follow the loss of an achieved condition, all 14 VLM rows and 15 of the 18 single-view specialized evaluators recall more than their fall rate. Robometer recalls 55% of them, 0.40 above its rate, and GRM-2.0-8B-Previev 68% (+0.35). RoboReward recalls none, and VLAC-Cut, which leads the single-view specialized evaluators in Macro-F1 and falling F1, recalls 15%, 0.06 above its rate.
Evidence Validity
The third analysis pairs sufficient and reduced views of the same execution and task scope (Table 3). Of the 1,117 videos, 715 are labeled V, 374 P, and 28 X; the paired-evidence set holds the 347 P/X videos with a V counterpart, all but six of them P, and 22 of the X videos, which show no task evidence, form a separate stress condition.
Hiding the decisive evidence lowers Macro-F1 for 24 of the 26 single-view evaluators, by at least 0.04 for 15 of them. The largest drops are 0.129 for RynnValue-8B and GPT-6 Luna, 0.122 for RynnValue-4B, and 0.095 for GPT-5.6 Terra; Flora-Rewarder drops by 0.040. SOLE-R1 and LRM-8B (Progress) score slightly higher on reduced views, the latter below 0.22 on both. The VLMs and Jev 1.14 must answer every query, and specialized evaluators return a score at every query, so no evaluated model can signal that the evidence is insufficient, and answerability has no eligible model. Figure S6 shows one successful execution under both views: with the placement outside the wrist camera’s view, most models end far below the credit they give on the front view.
The 22 views without task evidence (194 transitions, 12 of them falling) show how evaluators behave when the task is not visible at all. In Pair queries, six of the seven VLMs still predict a rise on 55–99% of these transitions and Jev 1.14 on 96%; Flora-Rewarder, VLAC-Cut, and SOLE-R1 do so on 60–66%. On these transitions the six VLMs reach a Macro-F1 of 0.41–0.53 and Jev 1.14 0.473, against 0.484 for a predictor that always answers rising. In Sequence queries, the VLMs return nearly constant curves, with at most 7.2% rises, which the direction test scores as errors because the task progresses outside the view. Under the same query, Jev 1.14 predicts a rise on 59% of these transitions, against 89% on the V videos recorded alongside them, for a Macro-F1 of 0.415.
Adding views has the opposite effect for evaluators that can combine them. On the 205 clips recorded by three cameras with verified synchronization, from 111 executions (a clip is an execution or a scope variant with all its camera views), the VLMs also receive the three views in one query, scored on the same transitions and against the same reference as the three videos. We compare the four VLMs whose single-view and multiview queries share an access route. In Pair queries, GPT-6.1 Sol gains 0.092 Macro-F1 over each clip’s main view (from 0.635 to 0.727), which is a wrist camera in 129 of the 205 clips. It also exceeds a majority vote over the three single views by 0.048, so it combines the views. Its falling F1 rises the most, from 0.348 to 0.514. Sol gains 0.086 on clips whose three views are all labeled V, and the gain persists without the 69 clips of arrange_flowers. GPT-5.6 Terra gains 0.042 and Gemini 3.8 Flash 0.022 in Pair queries. In Sequence queries, Sol gains 0.045, but for all four VLMs a majority vote over single views exceeds the joint input. Jev 1.14 does not gain in Sequence queries: its joint input scores 0.581, against 0.584 for the main view and 0.593 for the vote. Seven clips exceed its input limit as three separate images and are sent as one synchronized three-view mosaic; the result holds without them.
Specialized evaluators that accept several cameras are scored on every video recorded alongside other cameras († in Table 2): the video’s camera fills the primary slot and the other cameras of its execution fill the remaining slots, so the multiview and single-view results of a video share transitions and reference. Of these 1,116 videos, 804 come from clips with three cameras, 189 of them with wrist cameras aligned by timestamp, and 312 from clips with one fixed and one wrist camera, where Robo-Dopamine and Robo-ValueRL repeat the primary view in the third slot. SOLE-R1 receives one external and one wrist camera, and Flora-Rewarder the views tiled into one frame as its documentation specifies. On the three-camera videos, the four Robo-Dopamine checkpoints gain 0.072–0.101 Macro-F1 over the same single view and SOLE-R1 gains 0.048, all with 95% bootstrap intervals over executions above zero, and their falling F1 rises by 0.045–0.092. The Robo-Dopamine gains hold for V, P, and X views, for fixed and wrist cameras, and in every suite. With a P or X view in the primary slot of a clip with verified synchronization, all five exceed the single-view result of a V view of the same clip, so the other cameras supply the evidence that the primary view lacks. Flora-Rewarder does not gain (, with an interval that includes zero); over all videos it loses 0.034 on V views, which already show the decisive condition, holds on P views, and gains on the 28 X views. Robo-ValueRL reaches a Macro-F1 of 0.416, against 0.474 for the always-rising predictor on the same transitions. Because single-view use of Robo-Dopamine repeats one image in slots built for different cameras, its gains also measure the cost of that use.
The multiview judgment depends on which camera fills the primary slot. On 18 clips that were also rerun with identical input, swapping the primary camera lowers the share of transitions with the same predicted direction below that of the rerun for all seven evaluators, by 0.10 to 0.49. A multiview result is therefore an average over choices of the primary camera, not a property of the camera set.
Task Scope and Metric Sensitivity
Scope variants test whether the evaluator respects the declared scope of completion. In 322 pairs from 132 scope variants of 39 executions, the scope variant ends complete while the full task is not, with a median full-task reference of 0.57 at that moment. Scored under its own instruction, the scope variant receives more credit than the full task at the same moment in 68–90% of pairs for the Sequence-queried VLMs, but in only 29–38% for the four Robo-Dopamine models, 39% for VLAC-Cut, 45% for SOLE-R1, and 17% for LRM-8B (Progress), whose two scores tie in 47% of pairs. For these evaluators, completing the narrower instruction usually earns no more than the credit of the unfinished task.
We then compare two conventional metrics, VOC and Kendall’s , with falling F1 from the direction test (Table 3).
The normalized-time control never observes the video, yet its VOC of 1.000 exceeds that of every evaluator, while its and falling F1 are zero: it assigns every execution the same final value and never predicts a decrease. VOC therefore rewards chronology, not task state. Learned evaluators show the same dissociation. RoboReward has the fourth-highest VOC among specialized evaluators (0.807) and the lowest falling F1 (0.072), and Robometer and PRIMO-R1 follow the same pattern, with VOCs of 0.832 and 0.828 but falling F1 of only 0.214 and 0.154. Flora-Rewarder leads the specialized evaluators on VOC (0.915) and (0.572) but ranks second on falling F1 (0.252). Jev 1.14 has the third-highest VOC of all evaluators (0.905) but ranks fifth on falling F1 (0.256) and seventh on (0.496). In 52 of the 153 pairs of specialized evaluators, the one with the higher VOC has the lower falling F1. VOC and falling F1 agree more closely across the seven general-purpose VLMs (Spearman correlation 0.71, against 0.38 across the 18 specialized evaluators). On the executions alone, scored on one camera view each, the model rankings are stable (Spearman correlation 0.95 for VOC and 0.83 for against the full set), and the normalized-time control again attains a VOC of 1.000.
The direction test has its own trivial control. Because 90.2% of the transitions rise, a predictor that always answers “rising” reaches a Macro-F1 of 0.474, above 9 of the 18 single-view specialized evaluators in Table 2, and the rising precision of every single-view evaluator lies between 0.908 and 0.969; the falling class carries the discrimination. Endpoint ordering and regression detection also rank the evaluators differently. Among the specialized evaluators, and falling F1 have a Spearman correlation of 0.45 on all videos and 0.32 on the executions; Flora-Rewarder, Robometer, and RynnValue-4B lead on , whereas VLAC-Cut, Flora-Rewarder, and Robo-Dopamine-GRM-2.0-4B-Preview lead on falling F1.
Per-execution correlations show the same pull of elapsed time. On the 131 executions whose task-state reference is non-monotone or ends below completion, the output of 17 of the 27 evaluators correlates more with elapsed time than with the task state on most of them. The share is 75% for Robometer and 69–70% for RoboReward, PRIMO-R1, Flora-Rewarder, and Claude Haiku 5.5, all with VOCs above 0.80, against 39% for GPT-4.1 Sol and 37–39% for the GRM-2.0 previews; the median difference between the two correlations stays within 0.05 for every evaluator.
Figure S4. Both general-purpose VLMs credit a placement in the wrong basket as progress. In this execution from the Semantic/Ordered Constraint suite (B), seen from the wrist camera, the robot releases the product into the wrong basket, so the task-state reference returns to zero. Claude Opus 5.5 and GPT-6 Luna keep rising, Opus to full progress. Several specialized evaluators dip at the release but rise again, and only VLAC-Cut ends near zero.
References
- [20]Luo, J., Xu, X., Wang, S., Zhou, Y., & Zhang, Z. (2026). CriticHack: Evaluating visual rewards under robot policy optimization. https://doi.org/arXiv:2601.02527
- [21]R. Firoozi et al., “Foundation models in robotics: Applications, challenges, and the future,” Int. J. Robot. Res., vol. 44, no. 5, pp. 701–739, 2025.arxiv.org/abs/2312.07843
- [22]Z. Mei et al., “Robotic video world models: A survey of applications, research challenges, future directions,” arXiv:2601.07823, 2026.arxiv.org/abs/2601.07823
- [23]Y. Ma, Z. Song, J. Zhuang, J. Hao, and I. King, “A survey on vision-language-action models for embodied AI,” IEEE Trans. Neural Netw. Learn. Syst., vol. 37, no. 7, pp. 3031–3051, 2026.DOI
- [24]R. Yu et al., “Reward models in deep reinforcement learning: A survey,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2025.arxiv.org/abs/2506.15421
- [25]J. Zhong et al., “A comprehensive survey of reward models: Taxonomy, applications, challenges, and future,” arXiv:2504.12328, 2025.arxiv.org/abs/2504.12328
- [26]Zheng, C. et . al . (2026). A comprehensive survey of process reward models: Data generation, model construction, and usage. Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL).
- [27]J. Zhang et al., “Progress reward modeling for robotic learning: A comprehensive survey,” arXiv:2607.21655, 2026.arxiv.org/abs/2607.21655
- [28]Y. Wan et al., “No Free Checker: A survey of verifiers for robot policies,” arXiv:2609.09250, 2026.arxiv.org/abs/2609.09250
- [29]Y. Du et al., “Vision-language models as success detectors,” in Proc. Conf. Lifelong Learn. Agents (CoLLAs), 2023.arxiv.org/abs/2303.07280
- [30]J. Yang, M. Sobol Mark, B. Vu, A. Sharma, J. Bohg, and C. Finn, “Robo fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.DOI
- [31]C. Grislain, H. Rahimi, O. Sigaud, and M. Chetouani, “I-FailSense: Towards general robotic failure detection with vision-language models,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.arxiv.org/abs/2509.16072
- [32]Liu, X., Bai, Z., Cai, H. C., Ma, Y., & Zhou, M. Z. (2026). World-VLA-Loop: Closed-loop learning of video world model and VLA policy. https://doi.org/arXiv:2602.06508
- [33]Q. Agi et al., “Unpacking failure modes of generative policies: Runtime monitoring of consistency and progress,” in Proc. Conf. Robot Learn. (CoRL), 2024.arxiv.org/abs/2410.04640
- [34]Q. Gu et al., “SAFE: Multitask failure detection for vision-language-action models,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2025.arxiv.org/abs/2506.09937
- [35]D. Huang et al., “VLAConf: Calibrated task-success confidence for vision-language-action models,” arXiv:2605.29605, 2026.arXiv
- [36]D. T. Mahato and R. Ren, “Early warning signals for OpenVLA failure under visual distribution shift,” arXiv:2606.29699, 2026.arXiv
- [37]Z. Qin, Z. Tan, and X. Wang, “Future anchored verification and online recovery for world action models,” arXiv:2610.06280, 2026.arXiv
- [38]Y. J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V. Kumar, and A. Zhang, “VIP: Towards universal visual reward and representation via value-implicit pre-training,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023.arxiv.org/abs/2210.00030
- [39]Y. J. Ma, V. Kumar, A. Zhang, O. Bastani, and D. Jayaraman, “LIV: Language-image representations and rewards for robotic control,” in Proc. Int. Conf. Mach. Learn. (ICML), 2023.arxiv.org/abs/2306.00958
- [40]Y. Yang et al., “Adapt2Reward: Adapting video-language models to generalizable robotic rewards via failure prompts,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2024.arxiv.org/abs/2407.14872
- [41]P. Mandikal, J. Hu, S. Dass, S. Majumder, R. Martín-Martín, and K. Grauman, “Mash, spread, slice! Learning to manipulate object states via visual spatial progress,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.arxiv.org/abs/2509.24129
- [42]P. Krack, T. Jülg, W. Burgard, and F. Walter, “Rewarding DINO: Predicting dense rewards with vision foundation models,” arXiv:2603.16978, 2026.arXiv
- [43]H. Niu, Z. Han, Y. Ji, Z. Li, and K. Sreenath, “HiRE: Hindsight reward editing for policy finetuning,” arXiv:2609.27068, 2026.arXiv
- [44]M. Alakuijala et al., “Video-Language Critic: Transferable reward functions for language-conditioned robotics,” Trans. Mach. Learn. Res., 2025.arxiv.org/abs/2405.19988
- [45]T. W. Ayalew, X. Zhang, K. Y. Wu, T. Jiang, M. Maire, and M. R. Walter, “PROGRESSOR: A perceptually guided reward estimator with self-supervised online refinement,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025.arxiv.org/abs/2411.17764
- [46]C. Ziakas and A. Russo, “VITA: Zero-shot value functions via test-time adaptation of vision-language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2506.10085
- [47]P. Yang et al., “RARM: Confidence-gated progress reward modeling for RL in manipulation,” arXiv:2606.22027, 2026.arXiv
- [48]X. Qin et al., “Beyond monotonic progress: Retry-supervised value learning for robot imitation,” arXiv:2606.24633, 2026.arXiv
- [49]J. Yu et al., “WARP-RM: A warp-augmented relative progress reward model for data curation,” arXiv:2606.28320, 2026.arXiv
- [50]Y. Fang et al., “DenseReward: Dense reward learning via failure synthesis for robotic manipulation,” arXiv:2607.13033, …arXiv
- [51]Tu, Y., Tan, R., Yan, J., Hu, J., & Lv, C. (2026). AnyViewMeter: Adapting robotic reward models with camera geometry and multi-view attention. 2609.20106
- [52]J. Zhang et al., “ProgressLM: Towards progress reasoning in vision-language models,” in Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL), 2026.arxiv.org/abs/2601.15224
- [53]S. Chen et al., “TOPReward: Token probabilities as hidden zero-shot rewards for robotics,” arXiv:2602.19313, 2026.arXiv
- [54]Y. Zhang et al., “Recurrent reasoning with vision-language models for estimating long-horizon embodied task progress,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026.arxiv.org/abs/2603.17312
- [55]P. Schroeder, T. Weng, K. Schmeckpeper, E. Rosen, S. Hart, and O. Biza, “SOLE-R1: Video-language reasoning as the sole reward for on-robot reinforcement learning,” arXiv:2603.28730, 2026.arXiv
- [56]Y. Feng et al., “ProcVLM: Learning procedure-grounded progress rewards for robotic manipulation,” arXiv:2605.08774, 2026.arXiv
- [57]J. Shu, Z. Lin, B. Wei, and Y. Wang, “Feat2Go: Visual feature-grounded value estimation for embodied reinforcement learning,” arXiv:2605.30795, 2026.arXiv
- [58]J. Kwok et al., “LLM-as-a-Verifier: A general-purpose verification framework,” arXiv:2607.05391, 2026.arXiv
- [59]S. Hu et al., “ARS: Augmented reward system for robot learning,” arXiv:2609.34484, 2026.arXiv
- [60]Xiao, J. et . al . (2026). RehearseVLA: Simulated post-training for VLAs with physically-consistent world model. Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR).
- [61]Yang, Q., Li, J., Yang, Y., & Mu, M. (2026). ProgressVLA: Progress-guided diffusion policy for vision-language robotic manipulation. 2603.27670
- [62]D. Azuma et al., “NavWAM: A navigation world action model for goal-conditioned visual navigation,” arXiv:2606.13494, 2026.arXiv
- [63]J. Li et al., “DecisionNCE: Embodied multimodal representations via implicit preference learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2024.arxiv.org/abs/2402.18137
- [64]Y. Liu, C. Wen, Y. Hu, D. Jayaraman, and Y. Gao, “TimeRewarder: Learning dense reward from passive videos via frame-wise temporal distance,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.arxiv.org/abs/2509.26627
- [65]Y. Xu et al., “Robo-Dopamine 2.0: History-conditioned and OOD-aware process reward modeling for robotic manipulation,” arXiv:2608.15680, 2026.arXiv
- [66]S. Sudhakar, L. Liang, S. Rammohan, P. Tokmakov, R. Zemel, and C. Vondrick, “Robot critics that sweat the small stuff,” arXiv:2606.21572, 2026.arXiv
- [67]H. Zhang, Z. Zeng, B. Wang, L. Li, L. Lin, and L. Cheng, “Dream2Reward: Transition-alignment reward models from positive demonstrations for robotic manipulation,” arXiv:2608.18787, 2026.arXiv
- [68]D. Yang, D. Tjia, J. Berg, D. Damen, P. Agrawal, and A. Gupta, “Rank2Reward: Learning shaped reward functions from passive video,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.arxiv.org/abs/2404.14735
- [69]R. Liu, C. Bai, J. Lyu, S. Sun, Y. Du, and X. Li, “VLP: Vision-language preference learning for embodied manipulation,” arXiv:2502.11918, 2025.arXiv
- [70]A. Singh et al., “VARP: Reinforcement learning from vision-language model feedback with agent regularized preferences,” arXiv:2503.13817, 2025.arXiv
- [71]L. Luo, G. Zhang, H. Xu, Y. Yang, C. Fang, and Q. Li, “MVR: Multi-view video reward shaping for reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2603.01694
- [72]F. Amin, N. Dennler, and A. Bobu, “GIFT: Generalizing intent for flexible test-time rewards,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.arxiv.org/abs/2603.22574
- [73]M. Torne, A. Mahajan, A. Bhat, and C. Finn, “Freeform preference learning for robotic manipulation,” arXiv:2606.32027, 2026.arXiv
- [74]K.-C. Chen, W. Chen, W.-F. Sun, and M.-C. Hu, “VLM-AR3L: Vision-language models for absolute and relative rewards in reinforcement learning,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2026.arxiv.org/abs/2607.00483
- [75]S. Katkuri, M. Kawada, and J. Wachs, “Beyond pairwise feedback: Listwise vision-language supervision for preference-based reward learning,” arXiv:2608.25350, 2026.arXiv
- [76]Jouyali, L., Demeester, E., & Tsogikas, N. (2026). Scaling vision-language reward learning for robot manipulation in parallel simulation. 2609.21767
- [77]J. Kwok et al., “RoboMonkey: Scaling test-time sampling and verification for vision-language-action models,” in Proc. Conf. Robot Learn. (CoRL), 2025.arxiv.org/abs/2506.17811
- [78]Y. Ali et al., “EVE: A generator-verifier system for generative policies,” arXiv:2512.21430, 2025.arXiv
- [79]J. Kwok et al., “Scaling verification can be more effective than scaling policy learning for vision-language-action alignment,” arXiv:2602.12281, 2026.arXiv
- [80]S. Zhao, S. Zhang, S. Yang, W. Zhao, S. Wang, and X. Ji, “TapSampling: Inference-time sampling with a task-progress-understanding verifier for robotic manipulation,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.arxiv.org/abs/2605.25547
- [81]Yang, T., Shi, D., Li, R., & Wang, D. (2026). PreferenceFlow: Test-time guidance of flow-matching robot policies from human interventions. 2609.36872
- [82]H. Wang, W. Wu, Y. Yan, and B. Zhou, “Looking back to move forward: Temporal verification for generative robot policies,” arXiv:2609.39038, 2026.arXiv
- [83]S. Park et al., “DiVeR: Decision-critical verifier learning for VLA test-time scaling,” arXiv:2610.04933, 2026.arXiv
- [84]X. Dang and S. Edelkamp, “CLIP-Motion: Learning reward functions for robotic actions using consecutive observations,” arXiv:2311.03485, 2023.arXiv
- [85]K.-H. Hung, P.-C. Lo, J.-F. Yeh, H.-Y. Hsu, Y.-T. Chen, and W. H. Hsu, “VICToR: Learning hierarchical vision-instruction correlation rewards for long-horizon manipulation,” arXiv:2405.16545, 2024.arXiv
- [86]C. Kim et al., “Subtask-aware visual reward learning from segmented demonstrations,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.arxiv.org/abs/2502.20630
- [87]Y. Chen, H. Li, Z. Jiang, H. Wen, and D. Zhao, “TeViR: Text-to-video reward with diffusion models for efficient reinforcement learning,” arXiv:2505.19769, 2025.arXiv
- [88]Y. Zhao et al., “Training-free generation of temporally consistent rewards from VLMs,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025.arxiv.org/abs/2507.04789
- [89]Y. Liu et al., “PALM: Progress-aware policy learning via affordance reasoning for long-horizon robotic manipulation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026.
- [90]X. Zhou et al., “MARVL: Multi-stage guidance for robotic manipulation via vision-language models,” arXiv:2602.15872, 2026.arxiv.org/abs/2602.15872
- [91]Q. Chen et al., “SARM2: Multi-task stage aware reward modeling for self improving robotic manipulation,” arXiv:2606.10305, 2026.arxiv.org/abs/2606.10305
- [92]Y. Yang et al., “Stage-transition dense reward modeling for reinforcement learning,” arXiv:2606.31377, 2026.arxiv.org/abs/2606.31377
- [93]S. K. S. Ghasemipour, A. Wahid, J. Tompson, P. Sanketi, and I. Mordatch, “Self-improving embodied foundation models,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2025.
- [94]D. Huang et al., “RynnValue: Scaling robotic value foundation models with temporal distance,” arXiv:2608.09853, 2026.arxiv.org/abs/2608.09853
- [95]C. Bhateja et al., “Robotic offline RL from internet videos via value-function learning,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.
- [96]H. Song et al., “Hume: Introducing system-2 thinking in visual-language-action model,” arXiv:2505.21432, 2025.arxiv.org/abs/2505.21432
- [97]Physical Intelligence et al., “π₀.₆: A VLA that learns from experience,” arXiv:2511.14759, 2025.arxiv.org/abs/2511.14759
- [98]M. J. Kim et al., “Cosmos Policy: Fine-tuning video models for visuomotor control and planning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.
- [99]J. Yang et al., “Self-improving robot policy with compositional world model,” in Proc. Robot.: Sci. Syst. (RSS), 2026.
- [100]J. Zhang et al., “What frozen VLAs already know about success: A probing study of value-like structure in foundation robot policies,” arXiv:2605.28527, 2026.arxiv.org/abs/2605.28527
- [101]J. Chen et al., “MV-WAM: Manifold-aware world action model with value augmentation,” arXiv:2606.21088, 2026.arxiv.org/abs/2606.21088
- [102]H. Chen et al., “Robot self-improvement via human-video dynamics models,” arXiv:2606.21406, 2026.arxiv.org/abs/2606.21406
- [103]W. Xia et al., “Robo-ValueRL: Reliable value estimation for offline-to-online reinforcement learning,” arXiv:2607.09866, 2026.arxiv.org/abs/2607.09866
- [104]S. Fei, X. Yu, S. Wang, X. Zhao, J. Gong, and X. Qiu, “WCM: A world critic model for vision-language-action reinforcement learning,” arXiv:2607.29613, 2026.arxiv.org/abs/2607.29613
- [105]S. Singh, Z. Hu, M. Sobol Mark, J. Yu, Z. Erickson, and A. Kumar, “SeeQ: Training generalist value functions for long-horizon robotic manipulation,” arXiv:2609.22085, 2026.arxiv.org/abs/2609.22085
- [106]D. Wu et al., “Demonstration-free success-probability reward learning for generalist robot policies,” arXiv:2609.33653, 2026.arxiv.org/abs/2609.33653
- [107]G. Liang, Y. Lin, Y. Chen, K. Y. Ma, and M. Z. Shou, “From video frames to robot meta-frames: A unified latent interface for world action models,” arXiv:2611.14365, 2026.arxiv.org/abs/2611.14365
- [108]Z. Zhang et al., “GRAPE: Generalizing robot policy via preference alignment,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.
- [109]S. Zhai et al., “A vision-language-action-critic model for robotic real-world reinforcement learning,” arXiv:2509.15937, 2025.arxiv.org/abs/2509.15937
- [110]Y. Wu et al., “Large Reward Models: Generalizable online robot reward generation with vision-language models,” arXiv:2603.16065, 2026.arxiv.org/abs/2603.16065
- [111]S. Yong et al., “Generalizable dense reward for long-horizon robotic tasks,” arXiv:2604.00055, 2026.arxiv.org/abs/2604.00055
- [112]Y. Mao et al., “ARM: Advantage reward modeling for long-horizon manipulation,” arXiv:2604.03037, 2026.arxiv.org/abs/2604.03037
- [113]H. Ni, T. Lu, and Y. Cai, “CoRe: Combined rewards with vision-language model feedback for preference-aligned reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.
- [114]Y. Wang et al., “TrustboReward: Preference-ordered isotonic score editing for multi-paradigm robot reward models,” arXiv:2608.08491, 2026.arxiv.org/abs/2608.08491
- [115]R. Tan, Y. Tu, Y. Yan, L. Yu, X. Tian, and C. Lv, “FIERCE: From generalist robot policies to fast specialists via progress-failure feedback,” arXiv:2609.18651, 2026.arxiv.org/abs/2609.18651
- [116]S. Leng et al., “FLORA: An open model suite for scaling robotics data engines,” arXiv:2610.31779, 2026.arxiv.org/abs/2610.31779
- [117]J. Duan et al., “AHA: A vision-language-model for detecting and reasoning over failures in robotic manipulation,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.
- [118]Y. Liu et al., “From passive observer to active critic: Reinforcement learning elicits process reasoning for robotic manipulation,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2026.
- [119]S. Zhai et al., “HELP: Human-efficient large-scale robot post-training with rollout segmentation,” arXiv:2607.09776, 2026.arxiv.org/abs/2607.09776
- [120]B. Peng et al., “ReWorld: Multi-dimensional reward modeling for embodied world models,” arXiv:2601.12428, 2026.arxiv.org/abs/2601.12428
- [121]H. Wu et al., “RoboAlign-R1: Distilled multimodal reward alignment for robot video world models,” arXiv:2605.03821, 2026.arxiv.org/abs/2605.03821
- [122]P. Li, Z. Lin, Q. Wu, Y. Lv, F. Wang, and S. You, “Reward as an agent for embodied world models,” arXiv:2606.19990, 2026.arxiv.org/abs/2606.19990
- [123]M.-L. Nguyen et al., “RoboGaze: Evaluating robot world models via structured vision-language analysis,” arXiv:2608.26385, 2026.arxiv.org/abs/2608.26385
- [124]S. Zhou et al., “EVO-WAM: Evolving world action models through video-action verification,” arXiv:2609.38057, 2026.arxiv.org/abs/2609.38057
- [125]S. Ma et al., “RoboJudge: Multimodal language models as judges for embodied video generation,” arXiv:2610.27782, 2026.arxiv.org/abs/2610.27782
- [126]Y. Ji et al., “PRM-as-a-Judge: A dense evaluation paradigm for fine-grained robotic auditing,” arXiv:2603.21669, 2026.arxiv.org/abs/2603.21669
- [127]Q. Wang et al., “Towards the harness of embodied agents,” arXiv:2608.11246, 2026.arxiv.org/abs/2608.11246
- [128]Z. Zhou et al., “VeriFine: Scaling verification for self-improvement in embodied reasoning,” arXiv:2610.08761, 2026.arxiv.org/abs/2610.08761
- [129]A. Y. Ng and S. Russell, “Algorithms for inverse reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2000.
- [130]P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2004.
- [131]B. D. Ziebart, A. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning,” in Proc. AAAI Conf. Artif. Intell., 2008.
- [132]J. Ho and S. Ermon, “Generative adversarial imitation learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2016.
- [133]F. K. Luo, and S. Levine, “Learning robust rewards with adversarial inverse reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2018.
- [134]P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017.
- [135]J. Fu, A. Singh, D. Ghosh, L. Yang, and S. Levine, “Variational inverse control with events: A general framework for data-driven reward definition,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2018.
- [136]A. Singh, L. Yang, C. Finn, and S. Levine, “End-to-end robotic reinforcement learning without reward engineering,” in Proc. Robot.: Sci. Syst. (RSS), 2019.
- [137]P. Sermanet et al., “Time-contrastive networks: Self-supervised learning from video,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2018.
- [138]A. Escontrela et al., “Video prediction models as rewards for reinforcement learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2023.
- [139]T. Huang, G. Jiang, Y. Ze, and H. Xu, “Diffusion Reward: Learning rewards via conditional video diffusion,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2024.
- [140]A. Protopapa, D. Buoso, F. Pistilli, G. Chavaltzaki, and G. Averta, “GORDON: Graph-based object-centric rewards for decomposition of long-horizon manipulation,” arXiv:2608.03753, 2026.arxiv.org/abs/2608.03753
- [141]J. Tan, G. Adineera, and J. Kim, “ProgAgent: A continual RL agent with progress-aware rewards,” arXiv:2603.07784, 2026.arxiv.org/abs/2603.07784
- [142]P.-Y. Wu, D.-J. Chang, S.-Y. Hsiao, H.-M. Chen, Y.-C. Su, and T.-W. Ke, “TaRL: Learning general and physical rewards from tactile demonstrations,” arXiv:2609.36785, 2026.arxiv.org/abs/2609.36785
- [143]R. Tsao, A. Wagenmaker, and S. Levine, “Learning process rewards via success visitation matching for efficient RL,” arXiv:2606.23640, 2026.arxiv.org/abs/2606.23640
- [144]R. Tian, Y. Wu, C. Xu, M. Tomizuka, J. Malik, and A. Bajcsy, “Maximizing alignment with minimal feedback: Efficiently learning rewards for visuomotor robot policy alignment,” arXiv:2412.04835, 2024.arxiv.org/abs/2412.04835
- [145]M. Nakamoto, O. Mees, A. Kumar, and S. Levine, “Steering your generalists: Improving robotic foundation models via value guidance,” in Proc. Conf. Robot Learn. (CoRL), 2024.
- [146]C. Song, B. Qian, Y. Feng, and Z. Song, “VLA-Corrector: Stage-aware observable state understanding for prompt-based closed-loop recovery of vision-language-action policies,” arXiv:2609.06508, 2026.arxiv.org/abs/2609.06508
- [147]Y. J. Ma et al., “Eureka: Human-level reward design via coding large language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
- [148]T. Xie et al., “Text2Reward: Reward shaping with language models for reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.arxiv.org/abs/2309.11489
- [149]W. Yu et al., “Language to rewards for robotic skill synthesis,” in Proc. Conf. Robot Learn. (CoRL), 2023.arxiv.org/abs/2306.08647
- [150]A. Radford et al., “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. Mach. Learn. (ICML), 2021.arxiv.org/abs/2103.00020
- [151]T. Chen et al., “RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.arxiv.org/abs/2506.18088
- [152]Ng, A. Y., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping. Proc. Int. Conf. Mach. Learn. (ICML).
- [153]J.-B. Alayrac et al., “Flamingo: A visual language model for few-shot learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2022.arxiv.org/abs/2204.14198
- [154]DeepMind, G. (2026). Gemini Robotics ER 2 model card. <a href="https://doi.org/https://deepmind.google/models/model-cards/gemini-robotics-er-2/" rel="noopener noreferrer" target="_blank">https://doi.org/https://deepmind.google/models/model-cards/gemini-robotics-er-2/</a>
- [155]M. J. Kim et al., “OpenVLA: An open-source vision-language-action model,” in Proc. Conf. Robot Learn. (CoRL), 2024.arxiv.org/abs/2406.09246
- [156]Siméoni, O., & al., . et . (2025). DINOv3. https://doi.org/2508.10104
- [157]R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs: I. The method of paired comparisons,” Biometrika, vol. 39, no. 3/4, pp. 324–345, 1952.DOI
- [158]OpenAI. (2024). GPT-4o system card. https://doi.org/2410.21276
- [159]Jeung, W., & al., . et . (2026). Same trajectory, contradictory rewards (RoboRMBench): Paraphrase fragility in vision language reward models. https://doi.org/2609.05401
- [160]Budzianowski, P., & al., . et . (2025). OpenGVL – Benchmarking visual temporal progress for data curation. https://doi.org/2509.17321
- [161]Liu, Y., & al., . et . (2026). PRM-as-a-Judge 1.5: A toolkit for robot process assessment. https://doi.org/2608.14284
- [162]Z. Navasardyan, T. Danielyan, and H. Davtyan, “FailBench: How reliable are VLMs at judging robot task success?” arXiv:2609.03611, 2026.arxiv.org/abs/2609.03611
- [163]Liu, S., & al., . et . (2026). RoboValue: A fine-grained sim-and-real benchmark for unified evaluation of robotic value models. https://doi.org/2610.26663
- [164]Open X-Embodiment Collaboration et al., “Open X-Embodiment: Robotic learning datasets and RT-X models,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.arxiv.org/abs/2310.08864
- [165]A. Khazatsky et al., “DROID: A large-scale in-the-wild robot manipulation dataset,” in Proc. Robot.: Sci. Syst. (RSS), 2024.arxiv.org/abs/2403.12945
- [166]H. Walke et al., “BridgeData V2: A dataset for robot learning at scale,” in Proc. Conf. Robot Learn. (CoRL), 2023.arxiv.org/abs/2308.12952
- [167]K. Wu et al., “RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation,” in Proc. Robot.: Sci. Syst. (RSS), 2025.arxiv.org/abs/2412.13877
- [168]Damen, D., & al., . et . (2018). Scaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100. Int. J. Comput. Vis., 130(1), 33–55.
- [169]R. Hoque, P. Huang, D. J. Yoon, M. Sivapurapu, and J. Zhang, “EgoDex: Learning dexterous manipulation from large-scale egocentric video,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2505.11709
- [170]Ge, Z., & al., . et . (2026). EgoRecovery: Acquiring failure recovery ability through human recovery demonstration. https://doi.org/2607.19745
- [171]Ye, Z., & al., . et . (2025). RoboFAC: A comprehensive framework for robotic failure analysis and correction. https://doi.org/2505.12224
- [172]Lin, Z., & al., . et . (2025). FailSafe: Reasoning and recovery from failures in vision-language-action models. https://doi.org/2510.01642
- [173]P. Atreya et al., “RoboArena: Distributed real-world evaluation of generalist robot policies,” in Proc. Conf. Robot Learn. (CoRL), 2025.arxiv.org/abs/2506.18123
- [174]Li, D., & al., . et . (2026). Learning actionable manipulation recovery via counterfactual failure synthesis. https://doi.org/2603.13528
- [175]R. L. Plackett, “The analysis of permutations,” J. Roy. Statist. Soc. Ser. C, Appl. Statist., vol. 24, no. 2, pp. 193–202, 1975.DOI
- [176]Luce, R. D. (1959). Individual Choice Behavior: A Theoretical Analysis.
- [177]Li, R., Korkmaz, K., & Bıyık, E. (2026). Reward-DAgger: Robot-gated interactive imitation learning with general-purpose progress-based reward models. https://doi.org/2610.04054
- [178]Z. Bai, C. Gao, and M. Z. Shou, “EVOLVE-VLA: Test-time training from environment feedback for vision-language-action models,” arXiv:2512.14666, 2025.arxiv.org/abs/2512.14666
- [179]Z. Liu, A. Bahety, and S. Song, “REFLECT: Summarizing robot experiences for failure explanation and correction,” in Proc. Conf. Robot Learn. (CoRL), 2023.arxiv.org/abs/2306.15724
- [180]X. Zeng et al., “Diagnose, correct, and learn from manipulation failures via visual symbols,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026.arxiv.org/abs/2512.02787
- [181]Xia, D., & al., . et . (2026). Robo2Process: Benchmarking and learning process-aware understanding in vision-language robotic manipulation. https://doi.org/2606.13040
- [182]Wu, Y., & al., . et . (2026). RoboChrono: A real robot benchmark for streaming task understanding. https://doi.org/2609.36605
- [183]Yakefu, A., & al., . et . (2025). RoboChallenge: Large-scale real-robot evaluation of embodied policies. https://doi.org/2510.17950
- [184]Li, Y., Zhu, Y., Wen, C., Shen, X., & Xu, Y. (2022). WorldEval: World model as real-world robot policies evaluator. https://doi.org/2505.19017
- [185]Li, Y., & al., . et . (2026). Scaling robot policy evaluation via discrete diffusion world model. Proc. Int. Conf. Mach. Learn. (ICML).
- [186]Y. Yu and X. Qiu, “Benchmarking vision-language-action models on SO-101: Failure and recovery analysis,” arXiv:2606.08881, 2026.arxiv.org/abs/2606.08881
- [187]Team, R. D. (2026). Table30 V2: Evaluating generalized models by real robots at scale. In Dexmal, Tech. Rep.. https://robochallenge.ai/robachallenge_techreport_V2.pdfrobochallenge.ai/robachallenge_techreport_V2.pdf
- [188]AI, T. (2026). “Jev: For the builders AND dreamers”. <a href="https://doi.org/https://typesafe.ai/blog/jev-1-14" rel="noopener noreferrer" target="_blank">https://doi.org/https://typesafe.ai/blog/jev-1-14</a>
- [189]Team, Q. (2025). Qwen3-VL technical report. https://doi.org/2511.21631
- [190]OpenAI. (2026). Addendum to GPT-6 Astra system card: GPT-6.1 Sol. <a href="https://doi.org/https://deploymentsafety.openai.com/gpt-6-1-sol" rel="noopener noreferrer" target="_blank">https://doi.org/https://deploymentsafety.openai.com/gpt-6-1-sol</a>
- [191]OpenAI. (2026). GPT-6 Astra system card. <a href="https://doi.org/https://deploymentsafety.openai.com/gpt-6-astra" rel="noopener noreferrer" target="_blank">https://doi.org/https://deploymentsafety.openai.com/gpt-6-astra</a>
- [192]OpenAI. (2026). GPT-5.6 system card. <a href="https://doi.org/https://deploymentsafety.openai.com/gpt-5-6" rel="noopener noreferrer" target="_blank">https://doi.org/https://deploymentsafety.openai.com/gpt-5-6</a>
- [193]Anthropic. (2026). System card: Claude Opus 5.5. <a href="https://doi.org/https://www.anthropic.com/claude-opus-5-5-system-card" rel="noopener noreferrer" target="_blank">https://doi.org/https://www.anthropic.com/claude-opus-5-5-system-card</a>
- [194]Anthropic. (2026). System card: Claude Sonnet 5.5. <a href="https://doi.org/https://www.anthropic.com/claude-sonnet-5-5-system-card" rel="noopener noreferrer" target="_blank">https://doi.org/https://www.anthropic.com/claude-sonnet-5-5-system-card</a>
- [195]Anthropic. (2026). System card: Claude Haiku 5.5. <a href="https://doi.org/https://www.anthropic.com/document/claude-haiku-5-5-system-card" rel="noopener noreferrer" target="_blank">https://doi.org/https://www.anthropic.com/document/claude-haiku-5-5-system-card</a>
- [196]DeepMind, G. (2026). Gemini 3.8 Flash model card. <a href="https://doi.org/https://deepmind.google/models/model-cards/gemini-3-8-flash/" rel="noopener noreferrer" target="_blank">https://doi.org/https://deepmind.google/models/model-cards/gemini-3-8-flash/</a>








