betaFeedback

WorksComputer Science

Robotic Reward Models in the Foundation-Model Era

Abstract

Robotic reward models (RRMs) use foundation models to replace per-task reward engineering with reusable, task-conditioned evaluation. Yet under one name, their outputs answer different questions, from whether a task succeeded to which execution is better. This survey organizes 114 methods into five semantic families by the question their native outputs answer: Outcome, Absolute Progress, Relative, Temporal Value, and Hybrid. Within this taxonomy, we review how model design determines what an evaluator can judge, how raw experience becomes supervision, and how existing evaluation practice tests the result. The review exposes a gap: agreement with demonstration order and downstream policy gains do not show whether a reward tracks task fulfillment. We therefore introduce RRM-Bench-1K, 1,117 real-robot videos of closed-loop executions with task-state and camera-evidence labels, and four contrast suites that separate the credited condition from its usual cues. A unified study of 27 evaluators shows that the strongest final scores rank whole executions well enough for selection and filtering. Dense credit, however, follows the progression of an execution more closely than its task state, and no evaluator signals when the decisive view is missing. How much fidelity a reward model needs therefore depends on the decision it drives. We outline directions from real-failure supervision to metrics that separate task state from chronology, and open-source RRM-Hub, a toolbox in which 14 existing RRMs can be readily applied and new evaluators tested on the same capabilities.

Introduction

Foundation models let robot policies address more tasks and environments, as vision–language–action (VLA) models such as π0\pi_0 [1] and π0.5\pi_{0.5} [2] illustrate. Every task a policy attempts must also be judged: improving the policy, selecting its data, and monitoring its execution all turn on whether the robot’s behavior satisfies the task. Conventional reward design supplies this judgment task by task, with manually specified success criteria and shaping terms [3] (Figure 1(a)), an effort that grows with every task a general policy adds. Robotic reward models (RRMs) replace it with reusable, task-conditioned evaluators that use foundation models as representations, supervisors, or judges to assess robot behavior against a task specification (Figure 1(b)).

From per-task reward engineering to task-conditioned reward modeling

Figure 1. From per-task reward engineering to task-conditioned reward modeling. (a) Each task receives its own hand-designed reward, a distance dd to the target plus a task-specific success condition, such as the bowl resting on the plate with the gripper open. (b) One shared evaluator F\mathcal{F} maps visual evidence eie_i and instruction gig_i to a reward r^i\hat{r}_i for every task.

Early RRMs derived rewards from pretrained foundation models. ZSRM [4], RoboCLIP [5], and VLM-RM [6] score the compatibility between observations and a task description or demonstration, and use it as a reward or threshold it into a binary goal reward. RL-VLM-F [7] distills pairwise judgments of a vision–language model (VLM) into a task reward. Recent evaluators such as ReWiND [8], Robo-Dopamine [9], RoboReward [10], and Robometer [11] are trained on multi-task robot data with supervision from successful and failed executions, partial completion, temporal relations, and trajectory preferences. They now serve several uses: stage and progress estimation (SARM [12]), detection of planning and execution errors (Guardian [13]), value estimation for reinforcement learning (ViVa [14]), and candidate-action scoring and selection (Rover [15]).

Under the common name reward, these evaluators return success probabilities, completion percentages, preference utilities, expected returns, and structured diagnoses. Each answers a different question: a success probability says whether the task is done, a preference says which execution is better, and an expected return says what the policy can still achieve. These differences matter once a prediction becomes a control signal: ranking candidates, shaping a policy, and deciding when to stop place different demands on the same score. We therefore organize RRMs by the question their native output answers, where a native output is the quantity a model returns before any downstream transformation. The questions define five semantic families: Outcome, Absolute Progress, Relative, Temporal Value, and Hybrid (§ 2.2; Figure 2).

Taxonomy of robotic reward models by semantic family and design route

Figure 2. Taxonomy of the 114 methods in Table S1 by semantic family and design route (§ 3.2–§ 4.3), with the number of methods per family in parentheses. * evaluates generated behavior; ‡ marks a protocol or harness.

These output differences call for different tests of evaluator quality, and the common tests answer narrower questions than they are read to answer. Agreement with frame order on successful demonstrations [16] asks whether later frames score higher, a condition that any score rising with time also meets; it does not establish whether the score tracks task fulfillment when progress stalls or is reversed [17], [18]. Downstream policy performance reflects the evaluator, learning algorithm, and control system together [18]. Neither test separates task fulfillment from the cues that usually accompany it: motion, a completed manipulation, elapsed time, and a view that shows the outcome. On a real robot, Tian et al. Tian et al. [19] find that learned reward models over-reward unsafe or only superficially task-like behavior that human evaluators reject, and CriticHack [20] shows that optimizing a policy against a frozen learned reward can raise both task success and wrong-object failures.

Current benchmarks rarely contain the executions in which these cues part from the task. Four such cases arise in closed-loop executions (Figure 8): (A) motion without its effect, (B) a wrong destination or another broken constraint, (C) an achieved condition that is later lost, and (D) a missing decisive view [9]. A test set sampled from natural executions holds few instances of the first three and leaves the fourth unlabeled (§ 6.1), and no existing RRM benchmark draws all its failures from natural executions or presents one execution under both sufficient and reduced views (Table 1). RRM-Bench-1K therefore builds a contrast suite for each case from real executions: (A) and (B) probe proxy validity, (C) temporal validity, and (D) evidence validity. Each suite fixes an ordering or an abstention between paired judgments rather than a target value, so every family is tested through its native output.

BenchmarkMeasurement targetScaleNatural failuresRegression labelsGraded endpointsPaired evidenceScope variants
ManiRewardBench [53]Stage progress; success detection653 episodes; 130 tasks◐–◐––
RBM-EVAL [11]Progress alignment; ranking571 OOD videos◐–●––
PRIMO Bench [118]Progress under task and scene shift18k samples; humanoid set–––––
RoboRewardBench [10]Episode-end progress grade (1–5)2,831 test items◐–●––
RoboRMBench [159]Paraphrase invariance of rewards2,390 trajectories◐–●––
Progress-Bench [52]Progress from a demo; answerability>3k instances–––◐–
OpenGVL [160]Temporal order of shuffled frames200 trajectories; 4 datasets–––––
RoboPulse [126]Pairwise progress discrimination1,800 pairs; 816 tasks–◐–––
Suboptimal-Value-Bench [18]Value under hesitation and retry800 trajectories; 15 tasks◐●–––
RoboPulse++ [161]Interval-level progress direction2,244 intervals◐●–––
FailBench [162]Binary success detection2,197 attempts; 14 sources◐–◐––
RoboValue [163]Grounding, progress, recovery, and subtask credit2,792 test trajectories; 15 sim + 20 real tasks–●◐––
RRM-Bench-1K (ours)Fulfillment, retraction, and evidence1,117 videos●●●●●

Table 1. RRM-Bench-1K and existing benchmarks for robotic reward models. Scale follows each source’s unit. ●: yes; ◐: partial; –: no.

Figure 8. The four contrast suites of RRM-Bench-1K on real robot executions. Each panel names the critical evidence, the expected behavior, and the gold relation; the count after each suite name is its number of videos. (A) and (B) test proxy validity, (C) temporal validity, and (D) evidence validity.

Relying on a reward model takes three answers: what its output credits, whether its evidence suffices, and which decisions it can drive. This article is organized around them. Five families answer the first, four suites test the first two, and the third depends on the decision: the fidelity it needs grows with the optimization pressure it puts on the reward.

Adjacent surveys cover foundation models in robotics [21], robotic video world models [22], VLA models [23], reward models in deep reinforcement learning [24], and reward and process reward models for large language models [25], [26]. Concurrent surveys organize progress reward models for robot learning [27] by input–output interface, signal construction, and data and benchmarks, and verifiers for robot policies [28] by who supplies the judgment. Unlike these surveys, this article spans all five families, organizes them by the question their native outputs answer, and evaluates them empirically on the proposed RRM-Bench-1K.

This article makes three contributions:

  • A survey and semantics-first taxonomy of foundation-era RRMs. We organize 114 methods by semantic family and design route, explaining which quantity each route estimates and which behavior differences change its output. Recording each method’s output, query evidence, learning mechanism, and training data (Table S1) shows that one semantics arises from several mechanisms and that the evidence interface bounds what a result can establish.

  • RRM-Bench-1K and a unified empirical study. We annotate 1,117 real-robot videos of closed-loop executions on five embodiments, from up to three cameras, with task-state labels for the conditions that hold at each moment and camera-evidence labels for whether a view suffices to judge the task. Four contrast suites separate motion from its effect, a satisfied constraint from a broken one, an achieved condition from one later lost, and a decisive view from its absence. A unified protocol evaluates 27 evaluators on these suites and measures endpoint judgment, dense credit, and evidence use separately instead of merging them into one score (Table 3). The gaps it exposes motivate six research directions (§ 8).

  • RRM-Hub: an open-source toolbox. Common adapters for 14 existing RRMs preserve each model’s native outputs and supported inputs, so they can be applied readily and new evaluators tested under the same protocols. Stored raw outputs and per-capability reports make comparisons reproducible.

ModelVOCτa\tau_aFalling F1VP/X
Specialized evaluators, 2B–4B
ProcVLM-2B [56]†0.3820.1750.2010.4690.407
VLAC-2B [109]†0.3700.0930.1710.3570.331
Robo-Dopamine-GRM-3B [9]0.3240.1130.2020.3990.356
Robo-Dopamine-GRM-2.0-4B-Preview [9]0.5980.3110.2420.5140.452
Robometer-4B [11]†0.8320.4730.2140.5820.527
RoboReward-4B [10]†0.8070.3140.0720.2540.189
RynnValue-4B [94]†0.4590.4190.2350.5720.450
Specialized evaluators, 7B–8B
PRIMO-R1-7B [118]†0.8280.2850.1540.4310.426
LRM-8B [110] (Progress)0.1980.1570.1360.1580.215
Robo-Dopamine-GRM-8B [9]0.3890.2630.2110.4420.388
Robo-Dopamine-GRM-2.0-8B-Preview [9]0.6050.2800.2260.5040.447
RynnValue-8B [94]†0.5220.3900.2360.5700.441
SOLE-R1-8B [55]0.6830.1960.2250.4940.498
TOPReward [53] (Qwen3-VL-8B)†0.3790.0810.1820.4340.427
VLAC-8B [109]†0.4140.1200.1690.3660.358
Specialized evaluators, 27B–30B
Flora-Rewarder-27B [116]0.9150.5720.2520.5510.511
VLAC-Cut-30B-A3B [119]0.6950.2610.2720.5880.561
Specialized evaluators, proprietary backbone
GVL [16] (Gemini 3.8 Flash)0.7880.2710.2400.5530.506
General-purpose VLMs, Sequence style
GPT-6.1 Sol [190]0.8980.6950.3660.5590.491
GPT-6 Luna [191]0.8050.4440.2520.4660.337
GPT-5.6 Terra [192]0.8780.6320.2170.3890.294
Claude Opus 5.5 [193]0.9290.6460.3540.6180.557
Claude Sonnet 5.5 [194]0.8890.5590.1840.4930.474
Claude Haiku 5.5 [195]0.8070.3320.1220.3930.369
Gemini 3.8 Flash [196]0.8910.6010.3050.5600.531
Decision model, Sequence style
Jev 1.14 [188]0.9050.4960.2560.5340.525

Table 3. Conventional and direction metrics of the single-view evaluators. V and P/X give Macro-F1 under sufficient and reduced camera evidence on the same transitions. Bold marks the best result in each panel. †Curve interpolated to 1 FPS.

Method / componentNative outputQuery evidenceLearning mechanismTraining / data source
OUTCOME (12 methods)
ZSRM (binary variant) [4]Binary goal conditionImage + TaskCLIP adaptation + spatial labelsExploration images + descriptions
SuccessVQA (Flamingo detector) [29]Binary success answerVideo + TaskFlamingo VQA fine-tuningHuman success annotations
RoboCLIP [5]Episode compatibilityTraj. + Ref./TaskFrozen video–text alignmentNo RRM training
RoboFuME (reward classifier) [30]Binary completionImage + TaskMiniGPT-4 fine-tuningDemo endpoints + unsuccessful images
Sentinel (VLM monitor)‡ [33]Failure forecast + rationaleVideo + Task (+ Ref./Goal)Prompted chain-of-thought video QAFrozen VLMs; no training
SAFE [34]Running failure scoreLatent + Hist.MLP/LSTM probe + functional conformal predictionSuccessful/failed VLA rollouts
I-FailSense [31]Binary success/failureVideo + TaskFailure classificationDemos + synthetic data
RoboReward [10]End-episode progress (1–5)Video + TaskProgress supervisionHuman-scored rollouts + augmented demos
World-VLA-Loop* [32]Success scoreVideo + Init. + Action + FutureReward regressionDemos + binary outcomes
VLAConf [35]Calibrated success confidenceLatent + Hist.Support estimation + calibrationSuccess demos + outcome labels
OpenVLA warning probe [36]Failure probabilityLatentFailure classificationRollout latents + outcomes
FAVOR (Anchor Verifier) [37]Deviation-event probabilityImage + Ref. + Act. hist.DINOv2 + local labels + episode-level multiple-instance learningClean/perturbed rollouts + outcomes
ABSOLUTE PROGRESS (29 methods)
VIP [38]Goal distanceImage + GoalImplicit valueHuman video
LIV [39]ValueImage + Task + GoalLanguage–image valueHuman video
VLM-RM [6]CompatibilityImage + TaskFrozen image–text similarityNo RRM training
Adapt2Reward [40]CompatibilityVideo + TaskFailure-prompt alignmentHuman + robot-failure video
Video-Language Critic [44]Clip valueVideo + TaskContrastive temporal rankAuto time labels
GVL [16]Completion %Video + TaskPrompted frame orderFrozen VLM
PROGRESSOR [45]Progress distributionImage + Init. + GoalProgress regression + RLDemonstrations
ReWiND [8]ProgressVideo + TaskProgress + rewind regressionDemonstrations + Open X-Embodiment
VITA [46]Normalized progressImage + Task + Hist.CLIP adaptation + regressionDemonstration time labels
SPARTA-L (reward) [41]Object-state progressVideo + TaskVLM region labels + trackingNo RRM training
World-Env (ReachVLA)* [60]Completion probabilityVideo + Task + FutureSuccess classificationSimulator frames
ProgressLM [52]Progress / unanswerableImage + Ref./TaskVLM SFT + GRPOProgressLM-45K
TOPReward [53]Progress (log-prob)Video + TaskTruth-token probabilityFrozen video VLM
Rewarding DINO [42]RewardMulti-view + TaskReward rankingAnalytic labels + frozen DINOv3
R²VLM [54]Completion % + rationaleVideo + Task + Hist.Qwen2.5-VL SFT + multi-turn RLALFRED + Ego4D process labels
ProgressVLA [61]ProgressImage + Init. + Task + FutureTemporal regressionRobot video
SOLE-R1 [55]Progress + rationaleVideo + Task + Init.SFT + RLVRSynthetic process reasoning
ProcVLM [56]Progress + proc. rationaleVideo + TaskProcedure-grounded regressionDemos + synthetic labels
Feat2Go [57]Binned structural progressImage + TaskQwen3-VL progress learningSubgoal/feature-derived labels
NavWAM [62]Progress valueImage + Goal + Proprio.Joint progress diffusionSimulated + real navigation
RARM [47]Confidence-gated progressVideo + ReferenceTemporal contrastUnlabeled video
Method / componentNative outputQuery evidenceLearning mechanismTraining / data source
ReTVL [48]ValueVideo + TaskTemporal + retry supervisionOutcomes + keypoints
WVM [18]Progress sequenceVideo + TaskProgress-value learningRobot + human video
WARP-RM [49]Progress (+ velocity)VideoWarp-consistent progressSuccessful robot video
LLM-as-a-Verifier (VLM verifier)‡ [58]Progress (score-token expectation)Video (pair) + TaskPrompted score-token expectation + repetitionFrozen VLM (Qwen 3.6); no training
DenseReward [50]ProgressVideo + TaskPer-step regressionSynthetic failures + outcomes
AnyviewMeter [51]Task progressVideo + Task + Geometry (+ Multi-view)Robometer LoRA + ray conditioningLabeled multiview trajectories
HiRE (potential) [43]Success/failure support potentialImage + Ref.Frozen DINOv2/SigLIP kernel scoreSuccess frames + failed endpoints
ARS‡ [59]Per-frame progressTraj. + Task (+ Ref.)VLM event proposal + verificationFrozen VLM; optional outcome labels
RELATIVE (23 methods)
RL-VLM-F [7]Scalar task utilityImageBradley–Terry preferenceVLM pair labels
DecisionNCE-T [63]Transition-direction scorePair + TaskCLIP implicit-preference learningEPIC-KITCHENS video + language
Rank2Reward (ranking head) [68]Temporal-order utilityImageBradley–Terry on R3M featuresOrdered passive-video frames
VLP (preference scorer) [69]Trajectory utilityVideo + TaskCLIP cross-modal preference learningAuto-generated video orderings
VARP (reward head) [70]Preference utilityStatePreference learning + policy reg.VLM trajectory-sketch labels
RoboMonkey [77]RankImage + Task + ActionPreference rankingSynthetic action pairs
TimeRewarder [64]Signed temporal distancePairTimestamp self-supervisionRobot + human video
RoVer [15]Rank + directionImage + Task + ActionRanking + directionSynthetic pairs
EVE (selection verifiers)‡ [78]Candidate selectionImage + Task + ActionPrompted specialist VLM selectionFrozen VLMs; no verifier fitting
Robo-Dopamine (incl. 2.0) [9], [65]Relative progressPair + Multi-view + Task + Ref. (+ Hist. in 2.0)Hop-label regression + signed-Hop replay (2.0)Human keyframes + auto pairs + synthetic negatives (2.0)
CoVer [79]Alignment scoreImage + Task + Act. hist. + ActionInfoNCE alignmentDemos + rephrases
MVR [71]State relevance utilityPrivileged simulator stateBradley–Terry order + embed. reg.Frozen ViCLIP
GIFT (transferred reward) [72]Intent-conditioned utilityTraj. + CriterionGPT-4o alignment + MaxEnt IRLPreferred/non-preferred demos
TapSampling [80]Progress deltaImage + Task + ActionTemporal deltaOrdered + reversed pairs
Small Stuff (pair critic) [66]Pairwise progress judgmentPair + TaskVLM pairwise fine-tuningSuccessful/failed robot rollouts
FPL [73]Criterion-cond. utilityTrajectory + CriterionPreference utilityHuman freeform pairs
VLM-AR3L (reward head) [74]Utility / pair preferenceImage / Image pairBradley–Terry + Siamese learningVLM preference labels
Dream2Reward [67]Signed transition alignmentPair + Task + Hist.Frozen DINOv3/Qwen3 + flow fieldPositive demonstrations
Listwise VLM reward [75]Preference utilityImagePlackett–Luce reward learningVLM-ranked robot behavior
RAPID (reward head) [76]Preference utilityImageBradley–Terry + adaptive queriesVLM-labeled rollout pairs
PreferenceFlow (reward) [81]Action preference utilityLatent + ActionPreference + gradient regularizationIntervention pairs + expert demos
TeV (energy head) [82]Action energyVideo + Task + Act. hist. + ActionTemporal contrastive rankingDemos + constructed negatives
DiVeR (discriminator) [83]Candidate utilityLatent + ActionCriticality-weighted classifierSuccessful/failed VLA rollouts
TEMPORAL VALUE (26 methods)
V-PTR (robot Q) [95]Task Q-valueImage + Task + ActionVideo TD pretraining + offline RLEgo4D + multi-task robot data
CLIP-Motion (visual reward) [84]Motion stage / stage rewardPair + TaskCLIP motion classificationRollouts + state-derived motion labels
VICtoR [85]Stage-aware motion progressImage + TaskCLIP stage/progress learningAction-free videos + instructions
REDS [86]Subtask-conditioned rewardVideo + TaskEPIC matching + CLIP alignmentSegmented expert/suboptimal demos
TeViR [87]Reached-reference stageImage + Ref. + Hist.CLIP-text-conditioned video matchingTask videos + generated references
Method / componentNative outputQuery evidenceLearning mechanismTraining / data source
Hume [96]Q-valueImage + Task + ActionCal-QL (conservative TD)Robot demos, sparse reward
T2-VLM‡ [88]Tracked subgoal rewardImage + Task + Hist.VLM subgoals + Bayesian trackingFrozen Qwen2-VL + SAM2
Self-Improving EFM (steps-to-go) [93]Steps-to-go distributionPair / Multi-view + Task (+ Proprio.)PaLI steps-to-go prediction + BCImitation data with hindsight step counts
SARM [12]Stage + progressVideo + Proprioceptive stateStage/progress supervisionProcess labels
RECAP (value function) [97]Return distribution (steps to success)Multi-view + Task + Proprio.VLM-initialized distributional MC returnsDemos + labeled rollouts + interventions
PALM [89]ProgressImage + Task + Proprio.Action–progress diffusionProcess-labeled demonstrations
Cosmos Policy [98]Return valueMulti-view + TaskJoint diffusion + MC returnsDemos + task rollouts
MARVL [90]Staged similarityImage + Task + GoalView-invariant CLIP tuningAuto simulator frames
RISE* [99]Progress/TD valueMulti-view + Task + FutureProgress + TD valueDemos + success/failure rollouts
ViVa [14]Task valueMulti-view + Proprio.Monte Carlo returns + dynamicsTask rollouts
Frozen-VLA value probes [100]Discounted outcomeLatentRidge regression on VLA featuresSuccess/failure rollouts
SARM2 [91]Stage + valueVideo + Multi-view + Task + Proprio.Stage/value mixturePrimitive labels + rollouts
MV-WAM [101]Return valueMulti-view + Task + ActionMC return + joint dynamicsRobot demonstrations
DGAC [102]Value / candidate rankImage + Action + FutureValue + future rankingHuman video + robot rollouts
STDR [92]Stage + within-stage progressVideoStage/progress + validity gatesVLM-segmented demos + negatives
Robo-ValueRL (value estimator) [103]Normalized time-to-go valueMulti-view + Task + Hist.Distributional remaining-time learning + failure penaltyRobot demos + online rollouts
WCM [104]History valueVideo + TaskMC return + future latentDemos + rollouts
RynnValue [94]Temporal distance (absolute + signed)Video + TaskTimestamp-distance learningEmbodied video
SeeQ (critic) [105]Subtask Q-valueMulti-view + Task + ActionSubtask supervision + TDRoboCOIN + target-task data
eVTA (reward token) [106]Expected future successVideo + TaskTerminal-outcome TD learningMixed-quality policy rollouts
MetaWAM [107]Return valueMulti-view + Task + Proprio. + ActionJoint diffusion + meta-frame valueCosmos Policy RoboCasa data
HYBRID (24 methods)
AHA [117]Sub-task verdict + failure explanationVideo + Multi-view + TaskLLaVA-1.5 instruction tuningFailGen synthetic failures + VQA data
GRAPE (ranking reward) [108]Stage cost + self-likelihood + successTraj. + Task + LatentVLM stages + GPT-4o cost functionsNo RRM training
VLAC [109]Relative progress + donePair + TaskTemporal-delta + doneRobot + human video
Guardian [13]Verdict + reasoning traceMulti-view + TaskFailure-CoT tuningSynthetic failures
ReWorld* [120]Hierarchical four aspectsVideo + TaskPreference learningGPT-4o-scored robot videos
Robometer [11]Progress + success + pref.Video (pair) + TaskMulti-head temporal learningTime + outcomes + auto prefs.
PRIMO-R1 [118]Progress + failure + CoTVideo + TaskProcess SFT + RLProcess labels
Large Reward Models [110]Multi-part rewardImage (pair) + TaskThree LoRA interfaces (DPO + SFT)Robot, human, and simulated demos (time labels)
PRM-as-a-Judge‡ [126]Outcome–process–diagnosis profileVideo + TaskZero-shot PRM aggregationNo training
VLLR [111]Subgoal progress (critic init.) + self-certainty + outcomePair + Task + Graph + LatentPrompted progress for critic warm-up + certaintyNo RRM training (zero-shot LLM/VLM)
ARM [112]Advantage + completionVideo + Multi-view + TaskDual-head classificationHuman process labels
RoboAlign-R1* [121]Six scoresVideo + TaskTeacher SFT + distillationRule + human rubric labels
Reward as an Agent* [122]4-aspect scores + rationaleVideo + TaskPrompted judgingHuman audit
RoboGaze*‡ [123]Localized failure events + severityVideo + Task + Init.Grounded routing + specialist VLMs + criticFrozen VLMs; no training
CoRe [113]Formal + residual rewardImage + Sim. + ActionFormal + preference residualVLM labels
ACE-Brain-0.5 [17]Progress + preferenceVideo + TaskProcess + preference learningLabeled video
Method / componentNative outputQuery evidenceLearning mechanismTraining / data source
VLAC-Cut [119]Signed progress + diagnosis + cut pointsVideo + Task (+ Ref.)Process-annotation instruction tuningRe-annotated robot video with failures and recoveries
Thea‡ [127]Status + evidence + causeImage + CriterionPrompted post-hookNo training
TrustRoboReward [114]Scores / comparisons + rationaleVideo (pair) + TaskPoint/pair consistencyVLM + human correction
FIERCE [115]Progress + future failure riskVideo + Task + Proprio. + ActionJoint dynamics + reward learningRobot data + outcome-labeled rollouts
EVO-WAM (verification)∗‡ [124]Completion + action consistencyMulti-view + Video + Task + Init. + Proprio. + ActionVLM judging + inverse dynamicsRecorded video–action pairs
VeriFine-Judge [128]Rubric scores + diagnosticsVideo + State + Task + Action + RationaleHuman-calibrated VLM distillationTeacher scores + human corrections
Flora-Rewarder [116]Episode grade (1–5) + step action quality + rubric scoresVideo + Multi-view + TaskFull VLM fine-tuning with three prompt typesHuman, recorded-success, and VLM-generated labels
RoboJudge-9B∗ [125]Physical adherence + instruction alignment + rationaleVideo + TaskQwen3.5-9B SFT + GRPOHuman-verified scores + model-written rationales

Table S1. Comparison of 114 robotic reward methods and evaluative protocols, one evaluative component per row, grouped by semantic family in the order of Section 2.2 and ordered by first release within each family. * evaluates generated behavior; ‡ marks a protocol or harness. Query abbreviations and component notes follow the table.

Section 2 defines RRMs, their five families, and the scope; Sections 3 and 4 review model designs, data foundations, and evaluation practice; Sections 5 and 6 present RRM-Bench-1K, RRM-Hub, and the unified study; and Section 7 discusses open challenges.

Definitions and Scope

Reward Design without Foundation Models

Before foundation models, each robot task or domain received its own reward (Figure 1(a)): specified by hand, computed by geometric task oracles, recovered through inverse reinforcement learning [129], [130], [131], [132], [133], or learned from human preferences [134], success examples [135], [136], or self-supervised video [137]. This line continues in evaluators without foundation models: expert-video prediction (VIPER [138], Diffusion Reward [139]), goal distance over object graphs (GORDON [140]), progress from video (ProgAgent [141]) or tactile maps (TaRL [142]), trajectory prefixes, complete rollouts, paired executions, or success–failure discrimination (Success Visitation Matching [143]), preference-aligned features (RAPL [144]), and, beside a pretrained policy, value-based action ranking (VGPS [145]) or failure verification (VLA-Corrector [146]). Eureka [147], Text2Reward [148], and Language to Rewards [149] prompt a large language model to write per-task reward code, which Hybrid evaluators such as CoRe [113] and GRAPE [108] combine with other judgments.

Task-Conditioned Evaluation and Five Semantic Families

A robotic reward model is a task-conditioned evaluator that draws on CLIP [150] or a later foundation model for representation, supervision, or the judgment itself. It assesses robot behavior against a task specification g∈Gg \in\mathcal{G}, given as language, a goal image, a reference demonstration, or a structured goal, supplied with each query or fixed in the evaluator. It reads execution evidence e∈Ee \in\mathcal{E} admitted by an interface ι\iota: single observations, temporal windows,

trajectory prefixes, complete rollouts, paired executions, or synchronized camera views, optionally with actions, proprioception, or simulator state. For an evaluative query qq, such as terminal success, progress, pairwise preference, or value, the evaluator is a map

y^(q)=Fθ(q)(g,e),y^(q)∈Yq,(1)\hat{y}^{(q)} = F_{\theta}^{(q)}(g,e), \qquad\hat{y}^{(q)} \in\mathcal{Y}_q, \tag*{(1)}

whose native output space Yq\mathcal{Y}_q holds a label, probability, score, ordering, distribution, or structured judgment.

Outputs of the same form can carry different meanings: a scalar may encode visual compatibility, task progress, a preference utility, or an expected return. We therefore assign each evaluator to the family whose question its native output answers; what it is conditioned on, the specification gg and the evidence that ee admits, is recorded for every entry but does not decide the family (§ 4.4). Outcome, Absolute Progress, Relative, and Temporal Value each answer one question, and Hybrid composes them (Figure 2); within each family, Section 3 groups methods by design route, that is, by how they produce the output.

Outcome (is the task done, or will it fail?) judges whether a declared event has occurred or will occur within a horizon: terminal success, end-of-episode grades, and failure forecasts such as SAFE’s running failure score [34] (Figure 3). Absolute Progress (how far along is it?) covers pointwise estimates of advancement on a whole-task scale, anchored by a goal, reference, learned scale, or task procedure, such as the progress pt∈[0,1]p_t \in[0, 1] of ProcVLM [56] (Figure 4). Relative (which is better?) covers quantities defined by ordering under a fixed task or criterion, from a direct comparator’s judgment of a pair or candidate set, such as the signed hop between two states that the Robo-Dopamine GRM [9] returns (Figure 5), to a pointwise utility defined by comparisons. Temporal Value (which stage, and how well is it going?) covers targets measured against a reference the execution itself sets: the stage it has reached, the time it still needs, such as the seconds to completion predicted by RynnValue [94] (Figure 6), or the return expected under the policy that continues it, as in VπV^\pi or QπQ^\pi; the target, not temporal input, defines the family, so a whole-task progress estimate reading the full video history is Absolute Progress. Hybrid (what happened, and on what evidence?) covers evaluators that compose judgments from several families or as the fields of a structured verification judgment, as Robometer [11] does with progress, success, and preference from one backbone (Figure 7).

Figure 3. SAFE [34], a representative Outcome model. (a) A light head on a frozen VLA maps its features ete_t to a running failure score sts_t. (b) sts_t rises as the second shoe stalls and crosses the conformal threshold ηt\eta_t, which raises the alarm before the execution fails.

Figure 4. ProcVLM [56], a representative Absolute Progress model. (a) Given observations o1:to_{1:t} and instruction ℓ\ell, the language-model head first generates the remaining actions, and the value head then reads this generated procedure to output progress pt∈[0,1]p_t \in[0,1]. (b) ptp_t rises as fewer actions remain and reaches p=1p=1 when both cans are in the box.

Figure 5. The Robo-Dopamine GRM [9], a representative Relative model. (a) Given start and goal states, the GRM compares a before and an after state and returns a signed hop H∈[−1,1]H \in[-1,1]. (b) Hops are scaled to the distance that remains or to the progress made, so chained hops keep progress in [0,1][0,1]; removing the green block gives −0.50-0.50.

RynnValue: value queries in the sequence and output of seconds to completion and displacement

Figure 6. RynnValue [94], a representative Temporal Value model. (a) Value (VV) and displacement (ΔV\Delta V) query tokens interleaved with the frames return the seconds to completion dtd_t and the signed displacement Δ\Delta. (b) Each placement advances the execution by Δ=+6\Delta=+6 s, and dtd_t falls from 12 s to 0 s.

Robometer: three heads on one VLM backbone and outputs for progress, success, and preference

Figure 7. Robometer [11], a representative Hybrid model. (a) Three heads on one VLM backbone return per-frame progress ptp_t, success ctc_t, and preference P(τ1≻τ2)P(\tau_1 \succ\tau_2) over two trajectories. (b) ptp_t rises with each placed block, ctc_t turns 1 only at the end, and τ1\tau_1 is preferred to a τ2\tau_2 ending in the wrong order.

Table S1 lists the family, output, query evidence, learning mechanism, and training data of all 114 methods.

Scope and Corpus

A method enters the corpus when it meets four criteria: (i) its evaluative use is validated on robot behavior, robot observations, or a behavior representation; (ii) its output answers a task-conditioned question about success, progress, preference, value, behavior quality, or verification and is used or evaluated as a decision signal; (iii) CLIP or a later foundation model contributes to the evaluator through representations, adaptation, supervision, distillation (including a standalone model trained on foundation-model labels), or direct judgment; and (iv) the work designs or evaluates the evaluator rather than only applying one. A policy, video generator, or world model qualifies only through a component that judges task fulfillment and can be queried or ablated on its own; its action likelihood, reachability, or predicted observations describe what is probable or feasible, not whether the task is fulfilled.

Model Designs and Evaluative Semantics

Architecture and semantics are separate axes, so we first trace the evidence-to-decision path that all evaluators share and then review the five families along the design routes of Figure 2. Figure 3–Figure 7 show one representative method per family on simulated RoboTwin 2.0 executions [151], with its framework in (a) and its native output, with illustrative values, in (b); ProcVLM [56], Robo-Dopamine [9], Rynn Value [94], and Robometer [11] are also evaluated in Section 6.

From Evidence to Decisions

An evaluator encodes the task specification and the evidence its interface admits, such as a camera set, an embodiment descriptor, a candidate action, or a predicted future, and a query-specific head returns the native output. A downstream rule then consumes the prediction: SAFE [34] (Figure 3) raises an alarm when its failure score sts_t first exceeds a threshold ηt\eta_t calibrated on successful rollouts, shaping uses a progress estimate ptp_t as the potential of a shaped reward [152], and selection ranks candidates by a preference utility. Each rule places its own demand on the evaluator: selection needs only the order of the candidates, whereas shaping exposes the reward to a policy that optimizes against it (Section 7.6).

This pipeline separates semantics (the estimated quantity), form (its encoding), and role (its use), and the family follows semantics alone, whatever a method calls its output. For methods with several outputs, we set aside policy outputs and deterministic transformations and assign the family the remaining outputs share. A rationale, confidence estimate, or abstention gate attached to one judgment keeps that judgment’s family, as in ProcVLM (Figure 4), whereas independently meaningful judgments from different families make a model Hybrid, as in Robometer (Figure 7).

Outcome Models

Outcome models judge whether a declared event, such as success or failure, has occurred or will occur within a horizon. Episode-level judgments need the decisive state in view, which suits acceptance and filtering; predictive monitors such as SAFE (Figure 3) forecast the event from signals that precede it, which suits stopping and intervention.

Episode-Level Outcome and Ordinal Judgments

These judges read a foundation representation out into a verdict or a grade and differ in what the readout learns to separate: positive and negative goal descriptions in CLIP [150] space (ZSRM [4]), whether a video shows the task accomplished (SuccessVQA [29], fine-tuned from Flamingo [153]), or demonstrations and unsuccessful or synthetic data, judged from an image or a

whole trajectory (RoboFuME [30], I-FailSense [31]); proprietary embodied-reasoning models such as Gemini Robotics ER 2 [154] list success detection among their specialties. Grades and similarities extend the verdict: a 1–5 end-of-episode grade (RoboReward [10]), similarity to a demonstration (RoboCLIP [5]), or a thresholded reward regressed from generated-video latents (World-VLA-Loop [32]). A verdict can reject only the failed endings that its negative examples cover, so rejecting a wrong object or destination requires such negatives or grounding in the instruction.

Predictive and Policy-Internal Failure Monitors

A forecast’s main design choice is what the monitor reads. Policy-internal monitors read the acting policy’s features, which signal failure before it shows in the scene but must be refit per policy: a failure score learned from outcome-labeled rollouts (SAFE [34]), a probe of frozen OpenVLA [155] activations that flags steps within 15 steps of failure [36], or calibrated feature support (VLAConf [35]). External monitors read the execution and apply across policies, asking a frontier VLM whether the robot will finish in time (Sentinel [33]) or detecting task-breaking deviations from anchors a world action model generates (FAVOR [37]).

Evaluation implications: An Outcome claim needs an operational event, a declared horizon, and evidence showing the event; grades call for ordinal metrics, probabilities for calibration, and online monitors for alarm timing against the decisive event (Figure 3(b)). Endpoint and dense judgment are separate capabilities: RoboReward filters completed executions well yet detects regressions worst (§ 7.2).

Absolute Progress Models

Absolute Progress models answer how far an execution has advanced on an anchored scale; ProcVLM (Figure 4), for instance, anchors progress to the remaining actions. Routes differ in where the anchor comes from: a goal or demonstration representation, demonstration time, task semantics read by a VLM, or the system that produces the behavior. The anchor decides how a score treats a step away from the goal configuration while the task advances, such as a retraction to regrasp: a representation distance can read it as a regression, a time-supervised estimate can credit it whether or not it serves the task, and a procedure-based estimate can credit it if the generated procedure contains it.

Distances to Goal or Demonstration Representations

The most direct anchor is a goal or demonstration in a representation space, as in the goal-conditioned value representation of VIP [38], the language–visual goal alignment of LIV [39], and the zero-shot image–text compatibility of VLM-RM [6]. How the alignment is learned decides which deviations the score registers: learned failure prompts (Adapt2Reward [40]), a position code only for the first frame so that time cannot be read as progress (ReWiND [8]), a light head on frozen DINOv3 [156] and text features that ranks states like analytic rewards (Rewarding DINO [42]), and anchors grounded in tracked object masks (SPARTA [41]) or in successful and failed states (HiRE [43]), which respond to changes that a global similarity averages away.

Temporal Supervision and Progress Proxies

Temporal order within demonstrations offers a second scale, and with it the assumption that achievement advances with time, which holds on successful demonstrations. GVL [16] counters the assumption by asking a frozen VLM for the completion of shuffled frames; others learn the scale from demonstrations with online refinement (PROGRESSOR [45]), a ranking objective (Video-Language Critic [44]), reference alignment (RARM [47]), distributional outputs (WARP-RM [49]), or test-time updates (VITA [46]), and AnyviewMeter [51] adapts Robometer to multiview input. Labels that fall teach the score to register setbacks: chunks of normalized progress t/Tt/T whose frames are kept, repeated, or reversed (World Value Models, WVM [18]), preferences around annotated retries (ReTVL [48]), or simulated trajectories with failures and recoveries (DenseReward [50]).

Prompted, Token-Level, and Reasoning-Based Progress

VLM-based estimators, prompted or fine-tuned, reach the anchor through task semantics and differ in how the score is read out: the probability of a task-truth token (TOPReward [53]), the expected score token over repeated queries, which also serves as a shaping reward (LLM-as-a-Verifier [58]), or a score that follows reasoning, as a process rationale (SOLE-R1 [55]), a generated procedure (ProcVLM), or recurrent steps over long tasks (R²VLM [54]). Feat2Go [57] learns binned progress from feature similarity to subgoal states, ARS [59] assembles progress from events that VLM agents verify over a rollout, and ProgressHM [52] can return an explicit unanswerable response, which supports evidence-sufficiency tests. A generated procedure can name the object, destination, and order that a constraint check needs, but a wrong decomposition yields a smooth score for the wrong procedure.

Policy-Coupled and World-Model-Coupled Progress

Progress can also be estimated inside the system that proposes or predicts behavior, which shows the score what the system intends next but ties it to that system: World-Env (RehearseVLA) [60] scores imagined video for reward and early termination, NavWAM [62] predicts goal progress with future observations, and ProgressVLA [61] guides action generation by the progress of predicted frames.

Evaluation implications: An Absolute Progress claim rests on an anchor, a declared scale, and a measured response to the task state. Time-supervised and goal-anchored scores part ways at motion without its effect and at a lost achievement (Figure 8(A), (C)), so tests must decouple the task state from elapsed time; where the two diverge, most evaluated models follow elapsed time (Section 6.5).

Relative Models

Relative models judge which of two executions, states, or actions better satisfies the task. A direct comparator judges the items it is given, often two states of one execution, so its sign registers a local change, a regression included: the General Reward Model (GRM) of Robo-Dopamine (Figure 5) returns a signed hop between a before and an after state, and progress is chained from the hops. A utility learned from preferences scores one item at a time, commonly through the Bradley–Terry model [157], in which the probability that one item is preferred to another is a logistic function of their utility difference, so its values carry meaning only as an ordering of items like those compared in training.

Direct Pairwise and Transition Comparators

For transitions, the pair supplies the reference: a signed temporal distance between frames (TimeRewarder [64]), hops between multiview before/after states (Robo-Dopamine), refined with rollout history, synthetic negatives, and a curriculum over hop sizes (Robo-Dopamine 2.0 [65]), or comparison with the instruction (DecisionNCE-T [63]), a predicted successful transition (Dream2Reward [67]), or pairs of observed and predicted candidates (a pair critic [66]).

Pointwise Utilities Learned from Preferences

The criterion that labels the training pairs defines the utility: VLM preferences on observation pairs (RL-VLM-F [7]), human comparisons under a natural-language criterion (Freeform Preference Learning, FPL [73]), multi-view orderings from a video-language model (MVR [71]), or instruction-conditioned video scores (VLP [69]). Rank2Reward [68] learns temporal order with a Bradley–Terry head and inherits the time assumption of § 4, and GIFT [72] transfers a maximum-entropy inverse-reinforcement-learning reward [131] to new scenes, with GPT-4o [158] inferring intent. Others make comparisons more informative: diverse queries that stop once the reward stabilizes (RAPID [76]), trajectory sketches (VARP [70]), ranked lists (a listwise VLM reward [75]), or joint pointwise and comparative pathways (VLM-AR3L [74]).

Candidate-Action Comparators

When the alternatives are actions, ordering becomes a control primitive for test-time selection. Scorers learn from synthetic preferences (RoboMonkey [77]), predict a refinement direction (RoVer [15]), align instructions with actions (CoVer [79]), or verify each candidate’s progress (TapSampling [80]), and some sit beside a frozen policy as an energy verifier (TeV [82]), a success-visitation discriminator (DiVeR [83]), preference-guided flow matching (PreferenceFlow [81]), or zero-shot VLM verifiers (EVE [78]). Ranking gains depend on the proposal pool as well as the scorer, so best-of-NN gains can come from sampling alone, and a mismatched verifier can hurt: in the CoVer study, a RoboMonkey verifier cut simulated success of a π0\pi_0 [1] policy to under a fifth on red-teaming instructions.

Evaluation implications: Relative models are judged by their choices: pairwise accuracy, ties, ranking metrics, cycles among direct comparisons, and robustness to paraphrases and the candidate pool. Signed comparators must also give a lost achievement a negative step (Figure 5(b)); Robo-Dopamine-GRM-3B does so when a shelf falls (Figure 9) yet recalls under half the falls (Table 2).

ModelRising PrecisionRising RecallRising F1Falling PrecisionFalling RecallFalling F1Both classes Macro-F1Both classes Accuracy
Specialized evaluators, 2B–4B
ProcVLM-2B [56]†0.9240.5400.6820.1280.4650.2010.4410.533
VLAC-2B [109]†0.9220.3740.5320.1180.3120.1710.3520.368
Robo-Dopamine-GRM-3B [9]0.9080.4120.5660.1350.4010.2020.3840.411
Multiview‡0.9290.5930.7240.1610.4820.2420.4830.581
Robo-Dopamine-GRM-2.0-4B-Preview [9]0.9270.5940.7240.1580.5180.2420.4830.586
Multiview‡0.9400.7300.8220.2190.5670.3160.5690.713
Robo-ValueRL [103]‡0.9090.5140.6560.1050.5260.1750.4160.515
Robometer-4B [11]†0.9160.8710.8930.1810.2630.2140.5540.812
RoboReward-4B [10]†0.9370.2630.4100.2090.0440.0720.2410.241
RynnValue-4B [94]†0.9290.6770.7830.1530.5070.2350.5090.660
Specialized evaluators, 7B–8B
PRIMO-R1-7B [118]†0.9140.5710.7030.1350.1810.1540.4290.531
LRM-8B [110] (Contrastive)§0.9210.1710.2880.0710.7540.1300.2090.213
LRM-8B [110] (Progress)0.9240.1550.2650.1270.1480.1360.2010.154
Robo-Dopamine-GRM-8B [9]0.9210.4560.6100.1410.4220.2110.4110.453
Multiview‡0.9330.6040.7330.1760.4890.2590.4960.592
Robo-Dopamine-GRM-2.0-8B-Preview [9]0.9310.5940.7250.1480.4780.2260.4760.582
Multiview‡0.9380.7270.8190.2120.5520.3070.5630.709
RynnValue-8B [94]†0.9260.6840.7870.1600.4450.2360.5110.661
SOLE-R1-8B [55]0.9250.6850.7870.1460.4920.2250.5060.666
Multiview‡0.9340.7450.8290.1810.5150.2670.5480.723
TOPReward [53] (Qwen3-VL-8B [189])†0.9130.5580.6930.1110.5030.1820.4370.552
VLAC-8B [109]†0.9170.4230.5790.1200.2850.1690.3740.410
Specialized evaluators, 27B–30B
Flora-Rewarder-27B [116]0.9300.7480.8290.1720.4720.2520.5410.721
Multiview‡0.9240.7350.8190.1540.4430.2290.5240.706
VLAC-Cut-30B-A3B [119]0.9220.9160.9190.2760.2680.2720.5960.853
Specialized evaluators, proprietary backbone
GVL [16] (Gemini 3.8 Flash)0.9280.7300.8170.1670.4220.2400.5280.700
General-purpose VLMs, Pair style
GPT-6.1 Sol [190]0.9290.9200.9250.3260.3550.3400.6320.865
GPT-6 Luna [191]0.9170.6400.7540.1230.4660.1950.4740.623
GPT-5.6 Terra [192]0.9140.9080.9110.2000.2120.2060.5580.840
Claude Opus 5.5 [193]0.9250.9000.9120.2640.3310.2940.6030.844
Claude Sonnet 5.5 [194]0.9140.8860.9000.1830.2360.2060.5530.822
Claude Haiku 5.5 [195]0.9230.7020.7980.1450.4640.2210.5090.679
Gemini 3.8 Flash [196]0.9310.8960.9130.2870.3840.3290.6210.846
General-purpose VLMs, Sequence style
GPT-6.1 Sol [190]0.9690.5980.7400.7360.2440.3660.5530.563
GPT-6 Luna [191]0.9540.4190.5820.4420.1760.2520.4170.395
GPT-5.6 Terra [192]0.9530.3940.5570.6790.1290.2170.3870.368
Claude Opus 5.5 [193]0.9480.8490.8950.6970.2370.3540.6240.789
Claude Sonnet 5.5 [194]0.9320.7690.8430.5290.1120.1840.5140.705
Claude Haiku 5.5 [195]0.9350.5420.6860.3220.0750.1220.4040.496
Gemini 3.8 Flash [196]0.9600.7200.8230.6880.1960.3050.5640.669
Decision model
Jev 1.14 [188] (Pair)0.9160.7430.8200.1380.3500.1980.5090.704
Jev 1.14 [188] (Sequence)0.9290.7510.8310.1830.4270.2560.5430.719

Table 2. Progress-direction classification on RRM-Bench-1K. Bold marks the best single-view result in each panel. †Curve interpolated to 1 FPS. ‡Multiview input (Section 6.4); all other rows take single-view input. §Scored on adjacent source frames.

Figure 9. Reward profiles of nine evaluators on one execution of put the books back. The task-state reference (GT, gray) drops to zero when the robot knocks the shelf over at 19.6 s; red bands mark its reversals, and rr is each model’s Pearson correlation with it.

Temporal Value Models

Temporal Value models estimate progress within a stage, the time left, or the return still expected; RynnValue (Figure 6), for instance, counts the seconds to completion. A return-based value VπV^\pi, the discounted task reward expected from the current prefix, also depends on the policy π\pi that continues the execution, and QπQ^\pi on the next action as well. These targets share temporal input but answer different questions, so a stage error corrupts within-stage progress, a speed change the time to go, and a policy change VπV^\pi.

Stage-Decomposed Values

Stage decomposition replaces one global reference with a sequence of local ones, and the designs differ in how they recognize the stage: with CLIP (VIcToR [85], CLIP-Motion [84]) jointly with within-stage progress (SARM [12]), through primitive stages shared across tasks (SARM2 [91]), from VLM-segmented demonstrations with grasp verification (STDR [92]), or jointly with a policy’s actions (PALM [89]). Subtask structure can also condition the reward (REDS [86]), MARVL [90] or be tracked over time as VLM-proposed subgoals (T2-VLM [88]) or the farthest-reached frame of a generated reference video (TeViR [87]). Local references keep an early achievement from saturating a global score and expose regressions within a stage, which makes stage assignment the critical step.

Temporal Distance Values

Temporal distance can be counted in seconds or in control steps, so a slow but correct execution has more time to go than a fast one in the same task state. RynnValue learns its targets from timestamps, so speed, pauses, and the completion cutoff all enter them, and suppresses temporal shortcuts with random temporal sampling, order shuffling, and value-isolation attention. Self-Improving Embodied Foundation Models [93] count steps instead, differencing the expected steps to an instructed goal into a shaped reward and thresholding them into a success detector.

Policy-Indexed and Value-in-the-Loop Models

When the target is expected return, the policy becomes part of its meaning. V-PTR [95] pretrains a value function on human video for robot Q-learning, and RECAP [97], the training recipe of π0.6∗\pi^{*}_{0.6}, fits a VLM-initialized value function to Monte Carlo returns and thresholds its advantages into an indicator that conditions the policy, so its value is that of the data-collecting policy; Robo-ValueRL [103] conditions such an estimator on visual history. Critics can also co-evolve with the policy or read its representations, as a reward token refitted by temporal-difference (TD) bootstrapping (eVTA [106]), a Q-value for candidate chunks (SeeQ [105]), or a linear probe of discounted outcomes (frozen-VLA value probes [100]). Others act inside the control loop: they repurpose a video generator as a value model (ViVa [14]), rank sampled or corrective actions (Hume [96], DGAC [102]), or attach return or progress values to the futures a world model

generates (Cosmos Policy [98], RISE [99], WCM [104], MV-WAM [101], MetaWAM [107]).

Evaluation implications: A Temporal Value result is read against its declared target and the policy or data distribution behind it. The target fixes the expected response to reversal, stagnation, and retry: a lost achievement can leave expected return high if recovery is likely, so RRM-Bench-1K tests only the sign of a policy-indexed value’s response to a fall (§ 6.4).

Hybrid Models

Hybrid models answer several evaluative questions at once, such as progress and failure risk, or what failed and where; Robometer (Figure 7), for instance, returns progress, success, and a preference from three heads on one backbone. Hybrid designs combine judgments through heads on a shared backbone, composite rewards, or critics and judge ensembles that return a verdict with its cause and evidence, and more outputs add information only when each passes its own family’s tests and their disagreements are resolved.

Composite Rewards and Multi-Interface Models

One strategy combines separate judgments across families: a VLM-preference residual on a code-based formal reward (CoRe [113]), a critic initialized with VLM-estimated sub-goal progress (VLLR [111]), or rollouts ranked by likelihood, success, and a foundation-model keypoint cost for preference optimization (GRAPE [108]). Others keep each judgment queryable, pairing progress with preference (ACE-Brain-0.5 [17]), a segment-level advantage with a completion probability (ARM [112]), a progress delta with a done signal (VLAC [109]), an episode grade with step-level action quality (Flora-ReWarder [116]), or observed progress with a causal failure-risk head (FIERCE [115]). The outputs must also agree: the temporal-contrast, progress, and completion interfaces of Large Reward Models (LRM) [110] can disagree, and TrustRoboReward [114] repairs contradictory training labels through preference-ordered isotonic editing.

Structured Process Critics and Failure Diagnosis

A diagnosis ties the verdict to its cause. AHA [117] and Guardian [13], tuned on failures perturbed from or synthesized in simulation, judge from multiview images whether a subtask or a plan succeeded and why, Guardian with a reasoning trace; PRIMO-R1 [118] learns by supervised and then reinforcement fine-tuning to reason about progress in video and transfers zero-shot to failure detection; VLAC-Cut [119] pairs signed progress with a diagnosis of each change and cuts a rollout into segments to keep or discard.

Judges for Generated Robot Behavior

Judges of generated behavior separate realism from task completion, since a realistic video can still fail the task. ReWorld [120] scores generated videos for physical realism, task completion, embodiment plausibility, and visual quality, RoboAlign-R1 [121] distills such a multidimensional teacher, and RoboJudge-9B [125] rates physical adherence and instruction alignment separately. Agentic judges plan, vote, and reflect (Reward as an Agent [122]), route subtask segments to specialist VLMs returning time-localized failure events (RoboGaze [123]), or check inverse-dynamics consistency of generated video and actions (EVO-WAM [124]).

Protocol Judges and Runtime Verification

Protocol and action judges also return structured judgments. PRM-as-a-Judge [126] aggregates existing process reward models zero-shot into an Outcome–Process–Diagnosis profile, and VeriFine-Judge [128] distills a frontier-VLM teacher into a judge of instruction consistency and safety of proposed actions. At runtime, judgments decide whether to continue: Thea’s Evaluation as Exit Codes [127] checks the observation against an explicit post-condition and returns process, success, or failure with evidence.

Evaluation implications: Hybrid models call for field-wise evaluation: completion, failure type, localization, confidence, and correction utility each need their own test, explanations are checked against the actual failure, and composite rewards need each component measured directly.

Cross-Cutting Design Dimensions

Semantics and mechanism vary independently: Absolute Progress comes from embedding distances in VIP, demonstration time in VITA, token probabilities in TOPReward, and generated procedures in ProcVLM, and one backbone yields different evaluators under different queries (Section 6.5). Four further dimensions fix what a result can establish. The evidence interface bounds what is identifiable, from goal compatibility in a single frame and the direction of change in a before–after pair to an online alarm on a causal prefix and an outcome on a complete rollout. Transfer is shown by tests of the native evaluator, with any fitting, test-time adaptation, or hindsight access stated. Provenance determines which errors travel with a model: frozen representations carry pretraining biases, adaptation its label shortcuts, and distillation the teacher’s errors. Coupling determines what can be tested in isolation, since policy-internal and world-model-coupled heads change with the system they serve. Table S1 records each method’s evidence, provenance, and native output, and RRM-Bench-1K uses them to assign evaluators to tests (§ 6.4); reporting the raw prediction with each derived signal keeps a frame index from passing as progress.

Data Foundations and Evaluation Practice

Most evaluators learn from data, so how raw experience becomes targets decides what their judgment means. We follow this path to supervision and use, then ask whether evaluations resolve the ambiguities training leaves.

Sources of Raw Experience

Robot demonstrations and failures: Large teleoperation corpora such as Open X-Embodiment [164], DROID [165], and BridgeData [166] provide broad task coverage and are widely used to train and evaluate reward models [56] [10, 16], but they consist mostly of successful demonstrations, so the boundary between acceptable, suboptimal, and failed behavior is thinly covered. Recent collections add failures: RoboMIND [167] releases about 5,000 real teleoperation failures annotated with their causes, and RBM-1M [11] includes failed and suboptimal executions among more than one million trajectories from 21 platforms.

Human and internet video: Passive video extends semantic and temporal coverage beyond the available robot trials: EPIC-KITCHENS [168] serves LIV [39] and PROGRESSOR [45], EgoDex [169] serves LRM [110] and WVM [18], action-free video serves V-PTR [95] and VICToR [85], VLAC [109] mixes human and robot video, and EgoRecovery [170] records human recovery behavior. Whether the corresponding robot action is feasible depends on embodiment, viewpoint, and contact dynamics.

Simulation and autonomous rollouts: Simulation makes failure coverage controllable: RoboFAC [171], FailSafe [172], Guardian [13], SOLE-R1 [55], and DenseReward [50] generate failures with privileged geometric or phase labels but inherit the simulator’s appearance, dynamics, and failure taxonomy. Policy rollouts yield errors closer to the deployment distribution, with coverage set by the policy that produced them, as in the rollouts that refine Cosmos Policy [98], World-VLA-Loop [32], RISE [99], and ViVa [14].

Human and model annotation: Human annotation is sparse but captures qualities that proxies omit: double-blind policy comparisons and critiques in RoboArena [173], stage labels in SARM [12], synchronized keyframes in Robo-Dopamine [9], progressive, regressive, or stagnant transitions in ARM [112], retry keypoints in ReTVL [48], and preferences along annotator-named axes in FPL [73]. VLMs scale up pairwise labels, rationales, and subtask annotations, from RL-VLM-F [7] to the ProcCorpus of ProcVLM [56], trading annotation cost against coverage and teacher error.

From Trajectories to Supervision

Label construction decides which differences the evaluator is taught to preserve. Time and order labels teach stage and sequence, outcome labels finished versus failed endings, preferences a relative ordering, and event or condition labels local change and constraints. Any other distinction can only transfer from pretraining or other data.

Temporal and outcome-derived targets: Normalized time, frame order, temporal distance, and discounted terminal outcomes are inexpensive labels that a demonstration supplies with little further annotation [14], [46], [64], [94], [109]. They equate achievement with elapsed time, which holds on successful demonstrations, so motion without its effect and an achievement that is later lost, the cases of suites (A) and (C), receive no label of their own, and a return target further depends on the reward definition and the policy.

Synthetic and natural negatives: Because successful demonstrations dominate, negatives are often synthesized from them, and each transformation teaches its own distinction. Rewound videos, as in ReWiND [8], teach that backward playback scores lower, although a real regression such as a dropped object unfolds forward in time; clipped episodes, as in RoboReward [10], teach that a truncated execution earns partial credit; and counterfactual instructions, in RoboReward and CoVer [79], teach task-level grounding, whereas a wrong destination keeps the instruction and changes one relation in the scene, the case of suite (B). Perturbations in AHA [117] and FailSafe [172] and world-model failures in Dream2Fix [174] come closer to physical errors but follow the perturbation model, so an evaluator can learn its artifacts rather than the failure. Natural failures, such as the RoboArena [173] rollouts that RoboReward includes, supply the subtle errors of execution quality and safety that synthetic negatives underrepresent [19], and few label sets state whether decisive evidence is visible (suite (D)).

Preferences and intent: Preferences replace a numeric target with a criterion: human judgments, VLM labels, analytic re-wards, distances to expert actions, and trajectory order each give better its own meaning [7], [42], [73], [77]. VARP [70], PreferenceFlow [81], and GIFT [72] draw preferences from trajectory sketches, human interventions, and inferred intent.

Stage and language constructions: For long tasks, SARM [12], SARM2 [91], REDS [86], VicTor [85], and STDR [92] attach local progress to inferred or annotated stages; a local label inherits its segmentation errors, so stage assignment and within-stage accuracy need separate measurement. Question–answer, reasoning–score, and rubric formats fit labels to VLM training [52], [118], [121].

Training Objectives, Use, and Provenance

Different targets share learning tools: regression, distributional prediction, temporal-difference learning, contrastive alignment, ranking, and token prediction [7], [18], [38], [39], [118], with Bradley–Terry [157] and Plackett–Luce [175], [176] losses for pairwise and listwise preferences [75]. One loss serves several targets, so target and interface define the native output. The consuming decision sets demand on calibration, latency, and error cost, and a new decision rule can reuse an existing evaluator: Reward-DAgger [177] uses a frozen Robometer [11] to decide when to request human intervention, and EVOLVE-VLA [178] accumulates the deltas of VLAC [109] against milestone frames into a progress value that rewards and ends test-time rollouts.

Scale does not remove assumptions built into labels: more successful demonstrations can leave a subtle failure boundary unchanged, and synthetic or VLM-generated labels can share shortcuts with their source models. High temporal correlation can then reflect a time-labeling rule, and agreement with a related VLM an inherited bias; the suites of § 6.1 separate them from task-state sensitivity.

Existing Evaluation Practice

Protocols in Current Use

A metric is informative when its target matches the claim: temporal correlation measures ordering in time, pairwise accuracy a relative choice, outcome accuracy the recognition of an event, and scalar error agreement with a numerical target [118] [10, 11, 16]. High value-order correlation (VOC) on successful videos therefore supports a claim about ordering, whereas recognizing effects or reversals requires tests that contain them, and downstream policy success reflects the evaluator with initialization, exploration, optimization, and the candidate pool.

Benchmarks and System-Level Resources

Dedicated benchmarks broaden these protocols (Table 1): beyond its columns, Progress-Bench [52] tests answerability, RoboPulse++ [161] compares pair and sequence inference, RoboValue [163] branches seeded failures by recovery outcome, and CriticHack [20] stresses a frozen learned reward through policy optimization. Adjacent resources evaluate failure explanation and correction (Robo-Fail [179], AHA [117], RoboFAC [171], ViFailback [180]), process and streaming task understanding (RoboProcess-Bench [181], RoboChrono [182]), and generated robot video (RobotWorldBench [121], RoboGazeBench [123], RoboJudge-Bench [125]). System-level resources supply realistic behavior from which evaluator tests can be built: RoboArena [173] aggregates double-blind pairwise preferences into policy rankings on one embodiment, the DROID Franka setup; RoboChallenge [183] provides large-scale real-robot evaluation on single- and dual-arm robots and the rollouts of RRM-Bench-1K; WorldEval [184] and dWorldEval [185] evaluate policies through generated futures; and a deployment study on the SO-101 arm [186] characterizes naturally occurring failures.

Remaining Gaps

Table 1 compares the test sets on five properties: natural failures arise during the recorded executions, regression labels mark where progress is lost, graded endpoints distinguish zero, partial, and full fulfillment, paired evidence presents one execution under views labeled sufficient and reduced, and scope variants narrow the task within an execution. Partial marks denote failures that are partly constructed or of undocumented origin, regressions constructed by reversing frames, binary endpoints, and evidence varied by editing or mismatching inputs. The other test sets each fully provide at most one of these properties, and none fully provides paired evidence or scope variants, so coverage stays thin where convenient cues disagree with the task.

RRM-Bench-1K: A Capability Benchmark

Relying on a reward model requires that its score rise for the right reason and rest on evidence that shows it. When a robot asked to wipe a stain sweeps a rag across the table and stops while the stain remains (Figure 8(A)), a score that tracks motion, elapsed time, or resemblance to successful videos still rises. RRM-Bench-1K is built around such cases and combines the five properties of Table 1. Natural failures: its 1,117 videos record learned policies on a real robot platform, each execution kept whole with its failures, partial completions, retries, and recoveries. Regression labels: task-state labels record which task conditions hold at each moment and withdraw credit when an achieved condition is lost. Graded endpoints: the final task state grades every video as zero, partial, or full fulfillment. Paired evidence: up to three cameras record each execution, and each view is labeled by whether it shows enough to judge the task. Scope variants: some videos narrow the task scope within an execution. Four suites use these properties.

Four Decisive Cases and Their Contrast Suites

Each suite separates the credited condition from a cue that usually accompanies it in natural executions (Figure 8). A suite pairs an execution that earns credit with a counterpart that shares the cue but not the credit, a hard negative in (A)–(C) and a reduced view in (D). It names the critical evidence that separates them and fixes a gold relation between their judgments, an ordering or an abstention rather than a target value, so each family is tested through its native output (§ 6.4). Success and failure controls anchor each suite. Cases (A) and (B) test proxy quality, whether credit follows the specified object, destination, order, and effect; (C) tests temporal validity, whether credit lasts as long as the credited condition holds; and (D) tests evidence validity, whether the observations support the evaluator's judgment.

(A) Action–Effect separates task-like motion from its effect, which arrive together when an execution succeeds. It includes executions whose motion resembles success while the required effect is verifiably absent, such as wiping with an empty gripper or pressing a switch that does not flip. The terminal effect is the critical evidence: an execution that achieves it must outscore a motion-only attempt, and a score that ranks the two alike follows motion or elapsed time.

(B) Semantic/Ordered Constraint separates a successful manipulation from a correct one. It includes wrong objects or destinations, omitted instances, and violated prerequisites, in which the grasp, transport, and release can all succeed. The critical evidence is the object and its destination read against the instruction, so only an evaluator that checks them ranks the constraint-valid execution higher.

(C) Persistence/Reversal separates an achieved condition from one that still holds. It pairs conditions that persist with ones achieved and later lost; conditions that were never achieved form a separate case. Credit must fall at the reversal: an evaluator that keeps it accumulates history, and one that withdraws it tracks the current task state.

(D) Visibility separates what happened from what a camera shows. It shows executions under reduced camera evidence, most beside a sufficient view of the same execution, so the task state stays fixed while the evidence changes. Without the decisive state in view, the expected behavior is to abstain or widen uncertainty; an evaluator without such an output is measured by how far its judgment degrades.

Natural executions rarely produce cases (A)–(C) on their own (Supplementary § 12): the great majority of task-state falls end a failed attempt or follow a drop or release failure, and lost achievements, wrong destinations, and actions without effect make up the rest. Case (D), by contrast, is common: over a third of the single views give partial or insufficient evidence (§ 6.3). On successful demonstrations each pair of factors moves together, and a test set sampled at random would contain few instances of (A)–(C) and leave (D) unlabeled. The suites concentrate these cases while keeping every video a real execution, and each isolates a capability that dense shaping and monitoring need. No evaluator in § 7 handles all four.

Data from Real Closed-Loop Executions

RRM-Bench-1K is built on closed-loop executions recorded on RoboChallenge Table30 [183] and Table30 V2 [187], a real-robot platform that runs learned policies on standardized tasks and scores each execution with a rubric. Because the policies act in closed loop, their failures arise naturally and are recorded under one protocol. The benchmark covers 52 tasks on five embodiments: three single-arm robots (ARX-5, UR5, and Franka) and two dual-arm robots (Cobot Magic ALOHA and D00S-W1). Up to three platform cameras record each execution; the camera views of whole executions and of their scope variants (§ 6.3) form the 1,117 videos, and a video recorded alongside other cameras can also be scored together with them as one multiview input.

Task-State and Camera-Evidence Labels

We annotate each execution with task units, events, task-state labels, and camera-evidence labels, and derive the scope variants and contrast suites from these annotations. Annotation proceeds in three steps. First, to fix what earns credit, we write for each task the task units of its instruction: object and destination constraints, repeated instances, and, where declared, prerequisites, each with the rule by which it keeps its credit. A placement that must persist loses its credit when undone, whereas a required action stays credited after the robot moves on. Second, to locate when the task state changes, annotators review each execution frame by frame and mark the events: completions, failed attempts, losses, and reversals, with retries and re-achievements where they occur. Each event is time-stamped and linked to a unit, and its type decides whether credit is added, withdrawn, or unchanged. Third, to obtain a dense reference without judging every frame, a rule-based compiler turns the units and events into a score over time for each declared task scope, either the full task or a scope variant that covers one object, one unit, or a window of units under a narrower instruction, applying that scope’s credit, prerequisite, and reversal rules. One set of events thus serves every scope of an execution (Supplementary § 12.1). The resulting label records the task state rather than the motion performed. An action-history reference, which accumulates credit for completed actions, is kept for comparison; the two references end differently in over a third of the executions, and the task-state reference falls at least once in over half.

Whether a condition holds and whether a camera shows it are labeled separately. The state label is read from the views showing the decisive conditions, and each camera view is labeled as giving sufficient (V), partial (P), or insufficient (X) evidence to judge the task. An execution that no view can establish stays unresolved, excluded from state metrics. Most videos are labeled V and about a third P; X is rare.

Evaluation Protocol and Metrics

Eligibility and settings. Each evaluator enters the tests its native query and output support, Relative models through comparisons of observation pairs. The direction test fixes only the sign of a change, so a time-to-go or policy-indexed value (§ 4.2) also enters it, and a terminal grade enters on successive prefixes. Specialized evaluators run through their documented interfaces with native outputs; departures are given in Supplementary § 12.1 and Table S2.

EvaluatorFamilyInput of one queryNative output1 FPS curve
ProcVLM-2B†Absolute ProgressAll frames (at most 512), eight-observation windowProgress, 0–100%Interpolated
VLAC-2B†, VLAC-8B†Hybrid64 uniform frames, judged in adjacent pairsCumulative signed progressInterpolated
Robo-Dopamine-GRM-3B, -8B, GRM-2.0-4B-Preview, -8B-PreviewRelativeEvery tenth frame against the first frame and a blank goal image; the view fills all three camera slotsRelative progress, three modes averagedRead at whole seconds
Robo-ValueRL‡Temporal ValueThree views of every tenth frame, with one earlier frame of the base cameraRemaining steps nn; value 1−n/50001 - n/5000Smoothed as in the official script
Robometer-4B†HybridEight uniform frames in one passProgress in [0,1][0, 1] per frameInterpolated
RoboReward-4B†Outcome16 prefixes, each scored as an episodeGrade s∈{1,…,5}s \in\{1, \ldots, 5\}; (s−1)/4(s - 1)/4Interpolated
RynnValue-4B†, -8B†Temporal ValueEight uniform frames in one passNegative remaining time in secondsInterpolated
PRIMO-R1-7B†Hybrid16 prefixes of 32 frames eachProgress, 0–100%Interpolated
LRM-8B (Contrastive)§HybridTwo adjacent source framesSigned change in {−1,0,+1}\{-1, 0, +1\}None
LRM-8B (Progress)HybridOne source frameProgress in [0,1][0, 1]Read at whole seconds
SOLE-R1-8BAbsolute ProgressFrames at 1 FPSProgress, 0–100%, with a rationaleNative
TOPReward (Qwen3-VL-8B)†Absolute Progress64 uniform frames, 15 prefixes, in Robometer’s implementationLog-probability of the answer token, min–max normalizedInterpolated
Flora-Rewarder-27BHybridEach 1 FPS prefix, scored as an episodeGrade s∈{1,…,5}s \in\{1, \ldots, 5\}; (E[s]−1)/4(\mathbb{E}[s] - 1)/4Native
VLAC-Cut-30B-A3BHybridFrames at 2 FPS in one passKeypoints of time and progress in %Read at whole seconds
GVL (Gemini 3.8 Flash)Absolute ProgressAll 1 FPS frames in shuffled order, with the first frame as a 0% anchorCompletion, 0–100%, per frameNative

Table S2. Specialized evaluators as run: family in Table S1, input of one query, native output, and source of the 1 FPS curve. Marks as in Table 2.

Task fulfillment and false credit. Endpoint judgments are compared within each task, whose executions share instruction but differ in final completion. The study asks whether the highest final score selects the most complete execution and how often a failed execution reaches the median final score of fully completed ones. On failures of suites (A) and (B), this share is false credit.

Temporal response. Following RoboPulse++ [161], direction classification labels each 1 FPS transition as rising or falling and reports per-class precision, recall, and F1 with Macro-F1 and accuracy (Table 2). Every non-flat transition of the task-state reference is scored, and an unchanged prediction is a miss for both classes. Event-level tests compare a model’s recall of falls following a specific cause with its own rate of predicting a fall anywhere; a model that predicts falls at random recalls every cause at that rate.

Conventional metrics. VOC correlates predicted values with elapsed time, averaged over successful executions, so it measures temporal ordering where time and task state advance together. Kendall’s τa\tau_a is the rank agreement of final predictions with endpoint fulfillment within each task, averaged over tasks, so it measures endpoint ranking (Table 3). Both use native outputs, with VOC in the Pearson form of Robometer [11] and τa\tau_a as in Robometer and RynnValue [94].

Evidence and scope. The paired-evidence test scores each P/X video and its V counterpart on the same transitions against one task-state label, so only the camera changes (Table 3). Scope analysis compares each scope variant with the full task of its execution through unit mappings, because a narrower scope can be complete while the full task is not.

Controls. Normalized elapsed time, ignoring video, and a predictor that always answers rising are shortcut controls; the action-history reference (§ 6.3) is a label control.

RRM-Hub Toolbox

RRM-Hub is an open-source toolbox for testing a reward model on the questions of this benchmark. A model enters through an adapter that declares the queries, inputs, and native outputs it supports. The toolbox runs each eligible test, keeps the raw outputs, and reports each capability separately, so a new evaluator is compared with the published results suite by suite. Existing RRMs run on new videos through the same interface. Adapters cover the 14 specialized methods of § 7.1 with their checkpoints and interfaces, and the general-purpose VLMs and Jev 1.14 [188] are queried with fixed prompts (Supplementary § 12.1).

Unified Empirical Study

The study asks what each evaluator’s output credits, at the endpoint and under broken constraints (suites (A) and (B)) and as the task state changes (C), and whether its evidence suffices (D), then turns to the query protocol, task scope, and metrics that explain where these capabilities part. Table 2 and Table 3 and Figure 9 report the results, and Supplementary § 12 gives settings and per-model values.

Evaluated Models

Table 2 lists the 27 evaluators with their sources. Its 19 specialized evaluators come from 14 methods whose native outputs span all five families; Supplementary § 12.1 gives their interfaces and settings. Robo-ValueRL takes only

multiview input, and six others are scored with it too (Section 6.4). Seven proprietary general-purpose VLMs are queried in two styles with a fixed prompt: Pair style presents two observations from one video, and Sequence style the video’s observations in temporal order. The decision model Jev 1.14 [188] returns probabilities over fixed answers and is queried in both styles.

Proxy Validity: Endpoint Selection and False Credit

Endpoint judgments order executions more reliably than dense judgments track change. Within each task whose executions differ in final completion, the highest final score picks the most complete execution in three quarters of cases or more for Sequence-queried GPT-6.1 Sol, Jev 1.14, and Flora-Rewarder, against fewer than half at random (Supplementary § 12.2). Final scores also filter well: the top tenth of the executions, as ranked by Robometer, RoboReward, or GPT-6.1 Sol, holds about twice the share of fully completed executions that a random selection holds. RoboReward, with the lowest falling F1 among specialized evaluators, filters this well: endpoint and dense judgment are separate capabilities.

The failed executions of the Action–Effect and Semantic/Ordered Constraint suites (Figure 8(A)–(B)) divide the evaluators. Under Flora-Rewarder, GPT-6.1 Sol, and Robometer, few reach the median final score of their task’s completed executions; under VLAC-Cut, TOPReward, and VLAC-8B, half or more do. TOPReward also selects the most complete execution less often than chance.

Broken constraints are the hardest case for dense judgments. On the Semantic/Ordered Constraint suite, most evaluators in Table 3 score below the always-rising predictor on Macro-F1, more than on any other suite (Supplementary § 12.2). General-purpose VLMs recall falls after a drop or a release failure well above their overall rate of predicting a fall, but barely exceed that rate for falls after a placement at a wrong destination. Specialized evaluators respond to wrong placements more often, Robometer and VLAC-2B most clearly, yet each recalls at most half of these falls. A drop looks like a setback, whereas a wrong placement looks like progress unless its destination is checked against the instruction, suggesting dense credit follows the appearance of progress more closely than the instruction’s constraints.

Findings.

  • The strongest final scores select and filter whole executions well, even from an evaluator that responds to few falls: endpoint judgment and dense credit are separate capabilities.

  • Credit on the failed executions of suites (A) and (B) divides the evaluators, and broken constraints are the hardest case: a wrong placement looks like progress unless its destination is checked against the instruction.

Temporal Validity: Does Credit Track Task State?

No evaluator tracks task-state falls reliably, and the responses take three forms (Table 2). RoboReward and LRM-8B (Progress) leave their output unchanged on about three quarters of the transitions; RoboReward’s five grades are too coarse to register most changes. Most specialized evaluators fluctuate: they predict falls more often than the task state falls, and the median one predicts a fall on a quarter of the transitions where the task state rises. Sequence-queried VLMs rarely predict a fall and miss most falls.

Losses of an achieved condition, the cases of suite (C) (Figure 8(c)), are rare among the falls but draw a clearer response, and Persistence/Reversal is the suite on which the fewest evaluators fall below the always-rising predictor. All VLM rows and most single-view specialized evaluators recall lost achievements above their overall rate of predict-

ing a fall, and Robometer and Robo-Dopamine-GRM-2.0-8B-Preview exceed that rate the most, recalling more than half of them. VLAC-Cut, which leads the single-view specialized evaluators in falling F1, recalls few. In Figure 9, five of the nine evaluators withdraw credit when the shelf falls, whereas PRIMO-R1 and VLAC-Cut end higher than before.

Credit can also follow the action: where the action-history reference advances but the task state does not, the three Pair-queried VLMs best in Macro-F1 predict more rises than where both are flat (Supplementary § 12.3).

Findings.

  • No evaluator tracks task-state falls reliably: outputs stay unchanged, fluctuate with spurious falls, or rarely fall.

  • Lost achievements draw clearer responses than other falls, yet the single-view specialized evaluator that detects falls best recalls few, so direction accuracy and sensitivity to lost achievements are separate capabilities.

Evidence Validity: Does the View Matter?

Hiding the decisive evidence (Figure 8(D)) lowers Macro-F1 for nearly every evaluator in Table 3, and no evaluated interface can report insufficient evidence. Views with no task evidence at all reveal whether an evaluator knows it cannot judge. Most Pair-queried evaluators still report progress on them, close to the always-rising predictor, whereas Sequence-queried VLMs hold their output nearly constant, which a direction test against the hidden task state counts as error. An unseen state is neither success nor failure and needs an output and a score of its own (§ 8.4).

Extra cameras help evaluators with separate camera slots. With the video’s camera in the primary slot and its execution’s other cameras in the rest (‡in Table 2), all four Robo-Dopamine checkpoints and SOLE-R1 gain. With three synchronized cameras, a reduced primary view plus the other two beats the sufficient view alone for all five, so the others recover what the primary view hides. Flora-Rewarder, given views tiled in one frame, does not gain. Judgment also depends on the primary camera: swapping it changes the predicted direction more often than rerunning the input (Supplementary § 12.4).

Findings.

  • Hiding the decisive view lowers Macro-F1 for nearly every evaluator, and where no view shows the task, most Pair-queried evaluators still report progress: no evaluated interface can report insufficient evidence.

  • Evaluators with separate camera slots recover what a reduced view hides, yet their judgment changes with the primary camera.

evaluators with the lowest falling F1, and Robometer and PRIMO-R1 follow suit. Across the single-view specialized evaluators, VOC and falling F1 are only weakly related, far less than across the VLMs. τa\tau_a measures endpoint ranking and also orders the specialized evaluators differently from falling F1, so neither metric replaces a test of credit withdrawal. Outputs themselves lean toward elapsed time: on executions whose task state is non-monotone or ends below completion, nearly two thirds of evaluators correlate more with elapsed time than with task state on most of them. Because nine in ten transitions rise, the always-rising predictor beats half of the single-view specialized evaluators on Macro-F1, so the falling class carries the discrimination.

Findings.

  • Switching from Pair to Sequence queries trades falling recall for falling precision, so the query is part of the evaluator.

  • VOC rewards chronology: a control that reads only elapsed time exceeds every evaluator on it, so credit withdrawal needs a test of its own.

Summary of Findings

Three findings separate capabilities that a single leaderboard merges. Endpoint judgment and dense credit: RoboReward filters among the best and responds to the fewest falls, whereas VLAC-Cut leads the single-view specialized evaluators in falling F1 yet gives success-level credit to half the failed executions of suites (A) and (B), so which evaluator to use depends on the decision it serves. Execution progression and task fulfillment: wrong placements, which look like progress unless checked against the instruction, make broken constraints the hardest case for dense judgments, and most evaluators follow elapsed time more than task state where the two diverge; the procedure and constraint checks of § 4 target this gap. Evidence and its use: no evaluator signals that the decisive view is missing, extra cameras help where the interface fuses them, and the query style moves results by up to 0.17 Macro-F1.

Challenges and Research Directions

Scale alone does not close the gaps of § 7.5: within a model series, the larger checkpoint scores much like the smaller one (Table 2). The gaps trace instead to what evaluators learn from, how they represent the task, how they are queried, what they observe, and how they are measured and used, and the directions below follow this order.

Learning from Real Failures

Detecting drops and recognizing wrong destinations are different abilities: VLMs register the first far more often than the second (§ 7.2). Most task-state falls end a failed attempt or follow a drop (§ 6.1), whereas reward models learn mostly from successful demonstrations and synthetic negatives (§ 5.2). RoboMIND [167] annotates real teleoperation failures by cause, and VLAC-Cut [119], trained with signed progress on executions with failure, regression, and recovery, leads single-view specialized evaluators in falling F1 but recalls few lost achievements (Section 6.3).

Closed-loop platforms and deployment studies already log such failures at scale [173, 183, 186]. The open question is what supervision each failure mechanism needs: signed targets letting credit fall, labels by cause (failed attempt, drop, wrong destination, lost achievement), and constraint labels for errors motion does not reveal. A gain would show as per-cause fall recall above the evaluator’s fall rate, without more false falls where the task advances.

Task Structure Beyond the Instruction

An instruction names the goal but rarely the phases that reach it or the constraints that make it correct. Evaluators that register a drop still miss a placement at the wrong destination (Figure 8(B), § 7.2), and goal-distance anchors can read necessary detours as regressions (§ 4). Structure has entered reward models as stage-decomposed rewards and values [85, 91, 92] [12], as stage reasoning before scoring [54, 55, 56], and as post-conditions Thea [127] checks at runtime. Evaluators that state the current stage before scoring still predict a fall on about a third of transitions where the task advances (Supplementary § 12.3). The question is how to combine judgments of physical state change with task-constraint checks. A structured specification of a task’s units, constraints, and order can condition progress within phases such as approach, transport, and release, while post-condition checks verify what motion does not reveal, such as the destination or required effect (Figure 8(A)–(B)). Lower false credit on suites (A) and (B) at unchanged endpoint ranking would measure the gain.

Query Design

Pair queries recall more falls with more false alarms, Sequence queries fewer with fewer, and the same backbone scores differently under each protocol (Section 6.5). Existing work varies the query: GVL [16] asks for the completion of shuffled frames, TOPReward [53] reads token probabilities, and RoboRMBench [159] tests consistency under paraphrased instructions. The prompt, query style, frame selection, and output format therefore belong to an evaluator’s specification (§ 4.4) and to every reported result. The research question is how local event comparison and long-horizon task state can combine, as in protocols that compare adjacent observations within whole-video context, or prompts that spell out a task’s structure and test whether the evaluator uses it. Continuous, calibrated outputs avoid coarse ordinal levels that miss most changes. A protocol that combines the falling recall of Pair queries with the falling precision of Sequence queries would be an advance.

Evidence Sufficiency and Abstention

Reduced views lower Macro-F1 for nearly every evaluator, and where no view shows the task, most Pair-queried evaluators still report progress (Section 6.4). An evaluator must recognize when it cannot judge, since a judgment without evidence misleads filtering, shaping, and monitoring. Three capabilities supply this. Abstention reports insufficient evidence as a state separate from a low score; the closest existing interface, the unanswerable response of ProgressLM [52], flags a mismatch with the reference demonstration rather than a missing view. Calibration lets confidence fall with the evidence: WARR- RM [49] predicts a distribution over progress and RARM [47] gates uncertain matches, but neither has been tested against what the camera shows. Active evidence acquisition selects, fuses, or requests the decisive view: AnyviewMeter [51] adapts Robometer [11] to multiview input via camera-ray conditioning, Robo-Dopamine [9] reads synchronized views, and extra cameras help where the interface fuses them. Training with variable camera sets and missing views builds these capabilities, and per-view evidence labels, as in RRM-Bench-1K, make them testable. An advance would show as abstention without a decisive view, confidence following the evidence label, and multiview gains above a vote over single views from whichever camera fills the primary slot.

Measuring What Matters

VOC rewards chronology, and the always-rising predictor outscores half of the single-view specialized evaluators on Macro-F1, because elapsed time and task state mostly advance together (Section 6.5). Learned evaluators with a high VOC can keep their credit when the task state collapses (Figure 9). Progress metrics therefore need the normalized-time and always-rising references beside them, the falling class in its own column, and credit withdrawal tested per event, with τa\tau_a kept as the measure of endpoint ranking. Test sets need executions in which time and task state diverge, such as failures, partial completions, and lost achievements, labeled from what physically happened. Views without task evidence need their own scoring: there an abstention or an unchanged output is correct, and a score against the hidden task state penalizes the evaluator that recognizes it cannot judge. RRM-Hub reports these capabilities separately, so a new evaluator can show which of them it improves.

From Evaluators to Decisions

Task-state fidelity and decision utility are different properties, because each decision rule (§ 3.1) consumes a different capability. Selection among candidate actions or executions and filtering of data need only the order of whole executions, which the strongest evaluators provide, including some that respond to few falls (§ 7.2). Monitoring needs failures detected as they occur, and recovery needs to know whether a setback can still be repaired, which a failure forecast or VπV^{\pi} answers and current-state credit does not. Completion, time to go, and the chance of future success respond differently to a setback (§ 4.2) and serve acceptance, scheduling, and intervention, respectively. Shaping makes the strongest demand, since the policy optimizes against the reward and seeks executions where motion, time, or appearance part from the task state, the cases of suites (A)–(C); optimization against a frozen reward can raise both task success and wrong-object failures [20]. The fidelity a decision needs grows with the optimization pressure it puts on the reward. Shaping and monitoring need credit that falls with the task state, including at placements that violate the instruction, and every rule must recognize missing evidence; no evaluated model meets these demands yet (§ 7.2–6.4). Joint models keeping these outputs separate and consistent can serve several rules.

Evaluators already act inside the policy loop, as a causal failure-risk head in FIERCE [115], a candidate verifier in DiVeR [83], and an intervention gate in Reward-DAgger [177], where a strong candidate pool can mask a weak verifier (§ 4.1). The offline selection and filtering tests of § 7.2 lead to their closed-loop counterparts: candidate selection, rejection of known-bad behavior, and controlled policy updates under deployment conditions, including control deadlines, with each evaluator chosen for its role and tested with its rule. Full policy optimization comes last, because it places the most pressure on the reward.

Conclusion

In the foundation-model era, reward names several questions rather than one quantity. This survey organizes robotic reward models into five semantic families (Outcome, Absolute Progress, Relative, Temporal Value, and Hybrid) by the question their native outputs answer and relates each design route to the quantity it estimates, its supervision, and its evidence. Relying on such an evaluator takes three answers: what its output credits, whether its evidence suffices, and which decisions it can drive. RRM-Bench-1K tests the first two on whole real-robot executions, with contrast suites that separate the credited condition from its usual cues, task-state labels that withdraw lost credit, and camera views labeled by the evidence they show. The strongest final scores rank executions well enough for selection and filtering, but dense credit follows the progression of an execution more closely than its task state, VOC hides this gap because it rewards chronology, and no evaluator signals when the decisive view is missing. The query protocol and the evidence interface change these results and so belong to the evaluator.

The third answer depends on the decision: the fidelity a rule needs grows with the optimization pressure it places on the reward, so dense shaping and monitoring await evaluators that withdraw credit when an achieved condition is lost, verify constraints that motion does not reveal, and abstain without the decisive view. The path runs through real failures in training data, structured task specifications and queries, evidence-aware interfaces, and metrics that separate task state from chronology, and RRM-Hub lets new evaluators be tested on each of these capabilities. A policy learns what its reward credits, so general-purpose robots will be only as reliable as their evaluators.

Biographies

Guoqiang Liang

Guoqiang Liang is currently a Ph.D. student at Show Lab, National University of Singapore (NUS). Liang’s research interests include visual generation and embodied AI.

Yiqi Lin

Yiqi Lin is currently a Ph.D. student at Show Lab, NUS. Lin’s research interests include multimodal learning, video generation and editing, vision–language models, and robot learning.

Shangchen Zhou

Shangchen Zhou received the Ph.D. degree in computer science from Nanyang Technological University (NTU), and then joined MM-Lab@NTU as a Research Assistant Professor. Zhou’s research interests include image and video enhancement, generation, and editing.

Wenzheng Zeng

Wenzheng Zeng is currently a Ph.D. student at Show Lab, NUS. Zeng’s research interests include video understanding, multimodal foundation models, and their applications to robotics.

Yanzhe Chen

Yanzhe Chen is currently a Ph.D. student at Show Lab, NUS. Chen’s research interests include agents, robotics, and multimedia.

Zechen Bai

Zechen Bai is currently a Ph.D. candidate at Show Lab, NUS. Bai’s research interests include multimodal foundation models and agents.

Mike Zheng Shou

Mike Zheng Shou received the Ph.D. degree in electrical engineering from Columbia University, and was a Research Scientist at Facebook AI, before joining NUS. Where Shou is currently an Associate Professor and leads Show Lab. Shou is a Fellow of the National Research Foundation (NRF) Singapore, serves as an Associate Editor of IEEE TPAMI, and has served as an Area Chair for CVPR, ECCV, ICCV, and ACM MM. Shou’s research interests include computer vision and multimodal learning.

Method Inventory

Query evidence. Image: one observation; Video: a temporal window; Traj.: a complete trajectory (visual or structured state); Pair: two observations or clips queried jointly; Multi-view: synchronized camera views; Proprio.: proprioceptive state; Latent: policy-internal features or distributions; Sim.: privileged simulator state; State: structured state features; Future: a model-predicted observation. Task: instruction, task ID, or evaluative question; Goal: goal image/configuration; Init.: initial observation; Ref.: reference observations, demonstration, or support set; Criterion: preference axis/post-condition; Graph: structured scene context; Action: candidate action; Act. hist.: preceding actions; Hist.: retained execution or reasoning context; Geometry: camera calibration/rays; Rationale: the candidate policy’s proposed reasoning. A slash denotes alternatives; Pair is mandatory and (pair) is optional. Task identity may be fixed in the fitted evaluator rather than supplied with each query. Teacher inputs used only during training are omitted.

Component boundaries. Robo-Dopamine 2.0 is consolidated with the original method. DecisionNCE-T and Rank2Rew’s ranking head have relative semantics; the potential variant and density regularizer are separate constructions. AnyviewMeter is a geometry-conditioned Robometer adaptation. FAVOR and TeViR use generated references to assess actual execution. Small Stuff records the observed-pair interface. HiRE uses fixed historical support buffers; ARS requires complete rollouts. LLM-as-a-Verifier records the robotics instantiation of a cross-domain protocol. VeriFine judges proposals and reasoning rather than executed outcomes. VLLR’s VLM progress only initializes the critic; self-certainty and the sparse success reward drive policy optimization.

Evaluation Details and Additional Results

This section records the details behind § 6 and § 7. Section S2.1 gives the data sources, the annotation procedure, the input, scoring, and reporting rules, and the settings fixed for each evaluated model, with the protocol of each specialized evaluator (Table S2), the prompts of the general-purpose VLMs, and the run conditions that differ between requests. Sections S2.2–S2.5 give the per-model values behind the findings of § 7.2–6.5.

Evaluation Settings

Data sources. The RoboChallenge index lists 44,461 finished rollouts on tbl:30 and tbl:30 V2, more than 120,000 camera videos. A candidate pool of 1,500 rollouts, 750 per version and 4,228 videos, is drawn across zero, partial, and full platform scores. Annotators screen this pool, with a small set of added candidates, in the workbench of Figure S1 and annotate the executions they keep, whose camera views and scope variants form the 1,117 videos.

Annotation workbench showing model-proposed events and annotator verification

Figure S1. Annotators verify model-proposed events against the video before an execution enters the benchmark. In the annotation workbench, models mark candidate events on the video timeline (S: start, E: completion, R: failure or reversal). The annotator checks each claim in time order with Agree or Disagree, may assign a diagnostic category, marks actions the models missed, and then keeps, excludes, or flags the execution. Shown: an execution of wipe the table at 7.0 s.

Task-state labels. Dense targets combine the human-reviewed events of § 6.3 with rule-based state construction. Annotators verify model-proposed events against the video and mark the events the models miss (Figure S1). Each event keeps its provenance (reviewed mark, approximate anchor, or derived transition) and a timing-quality flag. The transient credit that the compiler grants during an attempt is withdrawn if the attempt fails; the provenance flag keeps such a rule-derived peak from being mistaken for a physical reversal, which requires evidence that the condition was first attained and then lost. Figure S2 shows the label of the execution of Figure 9 with its units and events. The normalized task-state score is an operational score defined by the task rubric. Because the state label comes from the views that show the decisive conditions, a placement that a wrist camera misses keeps its label. The platform’s rubric score is a separate target, reported apart from the task-state label.

Synchronized views, clip timeline, task-state GT breakdown, and event timeline

Figure S2. The task-state label withdraws credit when an achieved condition is lost, and every change traces to a time-stamped event linked to a task unit. Synchronized views of the put-the-books-back execution of Figure 9 at 19.6 s, as the robot sweeps the shelved book and the shelf off the table, above the compiled label (blue) and the action-history reference (gray, dashed); below, the four task units and the events behind the curve. Three failed attempts rise and fall back, and the sweep revokes both completed units, so the label ends at 0 and the reference at 0.25.

Inputs. Sampling, image resolution, prompts, decoding, and score transformations are fixed for each model. Each model receives its documented evidence, including reference demonstrations or goal images where required.

Scoring. Unlike RoboPulse++ [161], which labels annotated intervals, smooths continuous outputs, and drops steps near interval boundaries, we score every non-flat transition of the task-state reference without smoothing or boundary filtering. Because an unchanged prediction counts as a miss for both classes, it lowers recall and accuracy but not precision. VOC and τa\tau_a, defined in § 6.4, are computed on native outputs with the implementation of Robometer [11].

Specialized evaluators. The open-checkpoint roster comprises ProcVLM [56], VLAC [109] at 2B and 8B, Robo-Dopamine [9] (GRM-3B, GRM-8B, and the GRM-2.0 4B and 8B previews), Robo-ValueRL [103], Robometer [11], RoboReward [10], RynnValue [94] at 4B and 8B, PRIMO-R1 [118], Large Reward Models (LRM) [110], SOLE-R1 [55], TOPReward [53] applied to Qwen3-VL-8B-Instruct [189], VLAC-Cut [119], and Flora-Rewarder [116]. GVL [16] runs on the proprietary Gemini 3.8 Flash [196]. Table S2 gives the family, the input of one query, the native output, and the source of the 1 FPS curve of each. For models with several inference formulations, we evaluate RoboReward, PRIMO-R1, and Flora-Rewarder on the episode endpoint and on successive prefixes, and LRM through its completion, contrastive, and progress interfaces. The open models receive single-view videos. Robo-ValueRL requires several camera views at once and is scored only with multiview input; Robo-Dopamine, SOLE-R1, and Flora-Rewarder are scored with both (§ 12.4).

Flora-Rewarder serves terminal, dense, and rubric prompts from one set of weights. We use its terminal head, whose 1–5 episode grade ss gives the progress value (E[s]−1)/4(\mathbb{E}[s] - 1)/4 under the score-token probabilities, and score every 1 FPS causal prefix as a complete episode with the upstream frame selection, time metadata, and prompt. Its dense head grades action quality rather than progress and its rubric head requires per-task rubrics, so neither is evaluated. Robo-Dopamine scores the change between two observations, each filling three camera slots, against a start and a goal image. A single view fills all three slots, with the video’s first frame as the start and the official blank goal image, and the forward, incremental, and backward estimates are averaged as in its paper. The GRM-2.0 preview cards document this use; the GRM 1.0 cards show three views with a goal image; § 12.4 reports all four checkpoints with the other cameras in the remaining slots.

For direction tests, each model’s native curve is differenced at 1 FPS; curves saved only at sparse time points are interpolated and marked as such, and specialized evaluators have no Pair-style result. LRM’s contrastive interface compares the current frame with an earlier one; we query it on adjacent source frames (about 30 FPS) and mark these results accordingly. Terminal grades and LRM’s completion interface answer an outcome question and are not scored for direction.

General-purpose VLMs. The VLMs share one system prompt and one user template per query style, written after the method description of RoboPulse++ [161] and reproduced below. A request holds the rendered template and then, for each observation, a label and its image at native resolution; no timestamp, frame number, or camera name is sent. In Sequence style, one request holds all 1 FPS observations of a video in temporal order, a median of 15 and at most 110, and the model returns a completion value from 0 to 100 for each. The predicted direction of a transition is the sign of the difference between consecutive values, and equal values are an unchanged prediction. In Pair style, the two observations of each transition form their own request, labeled BEFORE and AFTER and sent without earlier context, and the model answers +1 or −1. Answers are parsed strictly, apart from format wrappers such as a Markdown fence, and a request without a valid answer leaves its transitions without a prediction. With three views (§ 12.4), each observation lists its three images under the same templates. GVL keeps its own prompt: it receives the 1 FPS frames in shuffled order with the first frame as a zero anchor, and its scores are returned to temporal order.

System prompt, both styles.

You evaluate progress in robot manipulation tasks from visual observations. Use the given task instruction to identify the required physical goal and compare task-relevant states. Judge visible evidence rather than assuming that later observations must show more progress. Progress may increase, decrease, or remain unchanged, and the execution may end without success. Treat the task instruction as the goal to evaluate. Follow the requested output format exactly.

User prompt, Pair style.

Task instruction: <task_instruction> $task_instruction </task_instruction>

You will receive two temporally adjacent observations, labeled BEFORE and AFTER. Images under the same observation label are simultaneous views, not additional time steps. Compare the physical task progress in AFTER with BEFORE. Consider the task-relevant objects, their relations to the goal, and any achieved state that is lost. Ignore appearance changes that do not affect the task. Do not use elapsed time alone as evidence of progress. Make a forced binary direction judgment: +1: AFTER is closer to satisfying the task than BEFORE. −1: AFTER is further from satisfying the task than BEFORE. If the visual evidence is ambiguous, choose the direction best supported by the observations. This binary baseline has no unchanged or abstention output. Return exactly one of the following, with no explanation: <score>+1</score> <score>-1</score>

User prompt, Sequence style.

Task instruction: <task_instruction> $task_instruction </task_instruction>

You will receive nobservationsn_{\mathrm{observations}} observations in chronological order, indexed from 0. Images under the same observation index are simultaneous views, not additional time steps. Estimate absolute physical task completion for every observation on a consistent scale from 0 to 100. Zero means no achieved task progress; 100 means that all requirements of the task are satisfied. Use intermediate values for partial completion. Base the scores on task-relevant physical states and the available temporal context. An achieved state that is undone can reduce the score. Unchanged states can receive equal scores. Do not force monotonic increases, do not force the first observation to 0, and do not force the last observation to 100. A clip can begin during an ongoing task and can end in failure. Elapsed time and robot motion alone do not establish completion. Return a JSON object with exactly one key, "progress". Its value must be an array of exactly nobservationsn_{\mathrm{observations}} finite numbers in [0, 100], in the supplied observation order. Return no Markdown or explanation.

GPT-6.1 Sol [190] runs at low reasoning effort and GPT-6 Luna [191] and GPT-5.6 Terra [192] at medium. Claude Opus 5.5 [193], Claude Sonnet 5.5 [194], and Claude Haiku 5.5 [195] run at medium effort and original image scale, and Gemini 3.8 Flash runs at its medium thinking level. Sequence request takes a median of 8.8k–17.5k input tokens, depending on the VLM.

Decision model. Jev 1.14 [188], from TypeSafe AI, returns probabilities over fixed answer options instead of text. It is queried in both styles with the VLM task templates, and both styles use the same frames, letterboxed to 280×280280 \times280 pixels at JPEG quality 70. In Sequence style, each video is sent once with all its 1 FPS observations and one five-level completion question per observation (0, 25, 50, 75, or 100% complete), and the expected level under the returned probabilities is its progress value. In Pair style, it sees the two observations of each transition and chooses between rise and fall; the more probable option is its prediction.

RRM-Hub records. Each adapter declares its source paper, the official code it calls at a pinned commit, its weights, the camera views it requires, its default frame sampling, and the type and range of its native output, which is kept without rescaling. The benchmark labels give each video’s execution, task scope, camera view, evidence label, and suites, so each result can be traced to the videos it covers. The licenses of the wrapped code and weights are listed separately.

Reporting. Table 2 reports progress-direction classification for every evaluated model over 17,381 non-flat 1 FPS transitions, 1,702 of them falling; transitions without a prediction, at most 1,323 per 1 FPS row, are counted separately and excluded from its direction metrics. Within a model series, the larger checkpoint changes Macro-F1 by −0.007 to +0.027 (Table 2). Table 3 lists the 26 evaluators with a single-view curve; Robo-ValueRL, LRM-8B (Contrastive), and the Pair-style queries yield none.

Reward profiles. Each panel of Figure 9 and of Figure S3–Figure S6 shows the highest-VOC model (Table 3) of the group in its subtitle at native timestamps, after a fixed unit conversion. Two exceptions apply: the general-purpose VLM group shows its highest- and lowest-VOC models, Claude Opus 5.5 and GPT-6 Luna, and the 4B panel shows Robo-Dopamine-GRM-2.0-4B-Preview (third of four by VOC), the only 4B model with dense outputs.

Figure S3. Most evaluators credit the watering motion rather than its effect. In this execution from the Action–Effect suite (A), the watering motion ends at 11.6 s without watering the plant, and the can later tips over, so the task-state reference (GT, gray) falls to zero. Claude Opus 5.5 and five others keep or raise their credit after the failed watering; only GPT-6 Luna, SOLE-R1, and Robo-Dopamine-GRM-2.0 lower it. Layout as in Figure 9.

Figure S6. Without the decisive view, most evaluators withhold the credit they give the same execution when its outcome is visible. In this successful execution from the Visibility suite (D), the robot opens a drawer, places a bottle opener inside, and closes it; the wrist camera (P) faces the drawer front and does not show the placement, whereas the front camera (V) shows every step. Claude Opus 5.5 ends at 1.00 on the front view but 0.04 on the wrist view, and PRIMO-R1 at 1.00 but 0.33; Robo-Dopamine-GRM-2.0 instead ends higher on the wrist view. Layout as in Figure 9, with each model’s output on both views.

Proxy Validity

The first analysis covers the Action–Effect and Semantic/Ordered Constraint suites and their controls, with Action–Effect executions scored on views that show the effect.

Endpoint judgments order executions more reliably than dense judgments track change. In each of the 28 tasks whose executions differ in final completion, we select the execution with the highest final score; ties are averaged, and VLMs are queried in Sequence style. The selected execution is the most complete one in 85% of cases for GPT-6.1 Sol, 82% for GPT-5.6 Terra, and 75% for Jev 1.14 and Flora-Rewarder, against 44% for a random choice, and TOPReward (38%) falls below chance. The 33 zero-completion executions of 11 tasks show how much endpoint credit a failed execution receives. The share of them that reaches the median final score of fully completed executions of the same task is 15% for GPT-6.1 Sol and Claude Sonnet 5.5, 18% for Flora-Rewarder, and 21% for Robometer, but 58% for TOPReward, ProcVLM, and Robo-Dopamine-GRM-3B and 76% for LRM-8B (Progress), whose output rarely changes. Measured the same way on the 34 failed executions of the Action–Effect and Semantic/Ordered Constraint suites, false credit is 3% for Flora-Rewarder, 6% for GPT-6.1 Sol, 9% for GPT-5.6 Terra and Robometer, and 12% for Claude Opus 5.5 and Jev 1.14, but 50% for VLAC-Cut, 56% for TOPReward, Robo-Dopamine-GRM-3B, and LRM-8B (Progress), and 72% for VLAC-8B.

Final scores also work as a filter. When the executions are ranked by final score and the top 10% kept, Robometer keeps only fully completed executions, RoboReward 93%, Flora-Rewarder and RynnValue-8B 90%, and GPT-6.1 Sol 87%, whereas a random selection holds 47%. RoboReward reaches this precision with the lowest falling F1 among specialized evaluators (0.072). The GRM-2.0 previews keep 50%, and ProcVLM (45%) and VLAC-8B (40%) do no better than random.

Among the suites, semantic constraints are the hardest for dense judgments (Table S3). On the transitions of the Semantic/Ordered Constraint suite, 19 of the 26 evaluators score below an always-rising predictor, more than on any other suite; the Sequence-queried VLMs reach a falling F1 of at most 0.11 there, and the best falling F1 is VLAC-Cut’s 0.229. Even on the clean-success controls, where nearly every predicted fall is an error, 21 of the 26 score below the always-rising predictor.

Model(A) Action–Effect M(A) Action–Effect F(B) Semantic/Ordered M(B) Semantic/Ordered F(C) Persistence/Reversal M(C) Persistence/Reversal F(D) Visibility M(D) Visibility FControl Clean success MControl Clean success FControl Obvious failure MControl Obvious failure F
Specialized evaluators, 2B–4B
ProcVLM-2B{sup}†`0.4590.2470.4070.1450.4690.2570.4110.1680.3690.0140.5720.487
VLAC-2B{sup}†`0.2410.1460.2690.1360.3950.2120.3220.1620.3520.0250.2690.337
Robo-Dopamine-GRM-3B0.3720.2030.3740.1450.4230.2630.3510.1700.2660.0190.3420.344
Robo-Dopamine-GRM-2.0-4B-Preview0.4760.2350.4400.1690.5180.3120.4520.2120.3980.0330.5440.467
Robometer-4B{sup}†`0.5190.1440.5150.1470.5720.2620.5290.2050.5070.0670.5560.306
RoboReward-4B{sup}†`0.2830.1070.2050.0380.2440.0820.1950.0520.2320.0000.1960.000
RynnValue-4B{sup}†`0.4940.2640.4600.1610.5420.2990.4560.1870.4340.0270.5670.429
Specialized evaluators, 7B–8B
PRIMO-R1-7B{sup}†`0.4310.1540.4280.1440.4280.1710.4240.1570.3950.0450.3830.174
LRM-8B (Progress)0.2370.1530.1760.0810.2060.1750.2130.1380.1510.0140.3190.235
Robo-Dopamine-GRM-8B0.4020.2120.3890.1520.4450.2750.3860.1870.3070.0190.3780.396
Robo-Dopamine-GRM-2.0-8B-Preview0.4810.2460.4360.1580.5040.2890.4480.2010.4000.0290.5320.460
RynnValue-8B{sup}†`0.5260.2650.4450.1620.5460.2960.4490.1880.4360.0230.5420.417
SOLE-R1-8B0.5320.2570.4800.1700.5230.2780.4980.2080.4210.0280.5940.483
TOPReward (Qwen3-VL-8B){sup}†`0.4450.2060.4060.1220.4610.2370.4270.1690.3670.0210.4770.391
VLAC-8B{sup}†`0.2930.1630.2480.1130.4260.2120.3480.1530.3840.0340.3810.440
Specialized evaluators, 27B–30B
Flora-Rewarder-27B0.5550.2740.5010.1850.5680.3170.5110.2080.4590.0370.6050.478
VLAC-Cut-30B-A3B0.5680.2320.5680.2290.6230.3350.5610.2050.5060.0450.4140.034
Specialized evaluators, proprietary backbone
GVL (Gemini 3.8 Flash)0.5320.2530.4860.1870.5560.2950.5060.2270.4490.0130.6020.476
General-purpose VLMs, Sequence style
GPT-6.1 Sol0.5610.3520.4120.1140.5890.4380.4940.2910.5560.3080.4100.105
GPT-6 Luna0.4250.1980.2760.0730.4460.3080.3430.1820.4130.1470.4000.140
GPT-5.6 Terra0.4210.1400.2540.0070.4020.2710.3020.1360.3620.0870.3460.073
Claude Opus 5.50.6170.3330.4920.1120.6550.4190.5630.2430.6510.3570.4720.172
Claude Sonnet 5.50.5300.2040.4160.0420.5260.2210.4790.1370.4690.0000.4320.107
Claude Haiku 5.50.4410.1030.3540.0340.3930.1510.3720.0840.4080.0300.4620.140
Gemini 3.8 Flash0.5030.1940.4010.0310.6100.3860.5330.2680.5560.2220.4070.037
Decision model, Sequence style
Jew 1.140.5510.2530.4960.1590.5660.3180.5250.2320.4570.0410.5640.412

Table S3. Direction results by contrast suite and control group: Macro-F1 (M) and falling F1 (F). Gray: Macro-F1 below the always-rising predictor on the same transitions. Bold: best in each panel. {sup}†`Curve interpolated to 1 FPS.

General-purpose VLMs keep crediting placements at a wrong destination. On the 54 falls that follow a placement at a wrong destination (§ 12.3), no VLM row exceeds its overall rate of predicting a fall by more than 0.04: Pair-queried VLMs recall 4–35% of these falls and Sequence-queried VLMs 0–6%. On the 704 falls that follow a drop or a release failure, every VLM row exceeds it by 0.10–0.36. Specialized evaluators respond to wrong placements more often: Robometer and VLAC-2B exceed their fall rate by 0.19, and ProcVLM, SOLE-R1, VLAC-Cut, and Flora-Rewarder by 0.10–0.15, although none recalls more than half of these falls.

Temporal Validity

The second analysis separates direction recognition from the response to specific events. Rule-derived attempt peaks form a separate slice from verified physical reversals, and an uncompleted final step counts as an omission, not as the loss of a previously achieved state.

The falling class consists mostly of short local events. Of the 1,702 falling transitions of Table 2, 863 (50.7%) end a failed attempt and 704 (41.4%) follow a drop or a release failure. Another 54 (3.2%) follow a placement at a wrong destination, and 15 follow an action that completed without its effect. Only 66 (3.9%) follow the loss of a previously achieved condition, in seven executions.

Evaluators respond to these events in three ways. RoboReward and LRM-8B (Progress) leave their output unchanged on 73% of the transitions. Most single-view specialized evaluators fluctuate instead: 15 of the 18 predict a fall on 13.9–44.3% of the transitions, although only 9.8% fall, and 17 of the 18 have a falling precision of 0.111–0.209, 1.1–2.1 times the base rate. On the transitions where the task state rises, the median specialized evaluator predicts a fall 25% of the time. Apart from the near-constant RoboReward, this rate ranges from 7.6% for VLAC-Cut to 44% for TOPReward, with 34% for ProcVLM and 31% for SOLE-R1; Pair-queried GPT-6.1 Sol predicts a fall on 8.0% of these transitions. Sequence-queried VLMs predict a fall on only 1.9–3.9% of the transitions, leave the score unchanged on 16–61% of them, and recall 7.5–24.4% of the falls. In Sequence queries, Jev 1.14 scores by an expected level that varies almost continuously, so it leaves the score unchanged on 4.3% of the transitions; like most specialized evaluators, it predicts a fall on 22.8% of them, with a falling precision of 0.183.

Switching from Pair to Sequence queries raises falling precision and lowers falling recall for all seven VLMs. The query protocol alone changes the result: Gemini 3.8 Flash reaches a Macro-F1 of 0.528 under the GVL protocol, 0.564 with Sequence queries, and 0.621 with Pair queries, and the Pair and Sequence results of one VLM differ by up to 0.171 in either direction. Jev 1.14 depends on the query as well: its Pair and Sequence answers agree on only 65% of the transitions, and unlike six of the seven VLMs it scores higher with Sequence queries (Macro-F1 0.543 against 0.509), which raise both its falling precision and its falling recall.

Figure 9 shows these responses on one execution chosen from reversal-rich executions. GPT-6 Luna follows the shape of the reference at a third of its height, and ProcVLM, SOLE-R1, and both Robo-Dopamine models fluctuate while the task state rises. When the shelf falls, five of the nine models drop to 0.05 or below, but PRIMO-R1 and VLAC-Cut end higher than before and Flora-Rewarder stays near 0.5, although all three have a higher VOC than Robo-Dopamine-GRM-3B (0.324), which drops.

Figure S3–Figure S5 show one more execution from each of suites (A)–(C) in the same layout.

Figure S5. Through repeated failures, only some evaluators follow the task state. In this execution from the Persistence/Reversal suite (C), three attempts fail before two flowers stay in the vase, so the task-state reference ends at 0.5. Both Robo-Dopamine models withdraw their credit at each failure and end near the reference, whereas VLAC-Cut and PRIMO-R1 rise through every failure. Claude Opus 5.5 rises during each carry and falls back when the carry ends, whether or not the flower stays, and so ends near zero.

The task-state and action-history references end differently in 73 executions, and the task-state reference falls at least once in 122. Where the two references disagree, the evaluators split. On the 922 transitions where the action history advances but the task state does not, Pair-queried GPT-6.1 Sol, Gemini 3.8 Flash, and Claude Opus 5.5 predict a rise 0.09–0.10 more often than where both references are flat, so they follow the action history. Flora-Rewarder, VLAC-Cut, and SOLE-R1 show no such difference (−0.11 to +0.01). Instead, they predict rises at high rates throughout, including on 52%, 72%, and 51% of the transitions where the task state falls; Robometer does so on 74%.

Responses to specific losses separate the evaluators further, again measured against each row’s overall rate of predicting a fall. Sequence-queried VLMs recall 12–39% of the 704 falls that follow a drop or a release failure, although they predict a fall on only 1.9–3.9% of transitions. On the 66 falls that follow the loss of an achieved condition, all 14 VLM rows and 15 of the 18 single-view specialized evaluators recall more than their fall rate. Robometer recalls 55% of them, 0.40 above its rate, and GRM-2.0-8B-Previev 68% (+0.35). RoboReward recalls none, and VLAC-Cut, which leads the single-view specialized evaluators in Macro-F1 and falling F1, recalls 15%, 0.06 above its rate.

Evidence Validity

The third analysis pairs sufficient and reduced views of the same execution and task scope (Table 3). Of the 1,117 videos, 715 are labeled V, 374 P, and 28 X; the paired-evidence set holds the 347 P/X videos with a V counterpart, all but six of them P, and 22 of the X videos, which show no task evidence, form a separate stress condition.

Hiding the decisive evidence lowers Macro-F1 for 24 of the 26 single-view evaluators, by at least 0.04 for 15 of them. The largest drops are 0.129 for RynnValue-8B and GPT-6 Luna, 0.122 for RynnValue-4B, and 0.095 for GPT-5.6 Terra; Flora-Rewarder drops by 0.040. SOLE-R1 and LRM-8B (Progress) score slightly higher on reduced views, the latter below 0.22 on both. The VLMs and Jev 1.14 must answer every query, and specialized evaluators return a score at every query, so no evaluated model can signal that the evidence is insufficient, and answerability has no eligible model. Figure S6 shows one successful execution under both views: with the placement outside the wrist camera’s view, most models end far below the credit they give on the front view.

The 22 views without task evidence (194 transitions, 12 of them falling) show how evaluators behave when the task is not visible at all. In Pair queries, six of the seven VLMs still predict a rise on 55–99% of these transitions and Jev 1.14 on 96%; Flora-Rewarder, VLAC-Cut, and SOLE-R1 do so on 60–66%. On these transitions the six VLMs reach a Macro-F1 of 0.41–0.53 and Jev 1.14 0.473, against 0.484 for a predictor that always answers rising. In Sequence queries, the VLMs return nearly constant curves, with at most 7.2% rises, which the direction test scores as errors because the task progresses outside the view. Under the same query, Jev 1.14 predicts a rise on 59% of these transitions, against 89% on the V videos recorded alongside them, for a Macro-F1 of 0.415.

Adding views has the opposite effect for evaluators that can combine them. On the 205 clips recorded by three cameras with verified synchronization, from 111 executions (a clip is an execution or a scope variant with all its camera views), the VLMs also receive the three views in one query, scored on the same transitions and against the same reference as the three videos. We compare the four VLMs whose single-view and multiview queries share an access route. In Pair queries, GPT-6.1 Sol gains 0.092 Macro-F1 over each clip’s main view (from 0.635 to 0.727), which is a wrist camera in 129 of the 205 clips. It also exceeds a majority vote over the three single views by 0.048, so it combines the views. Its falling F1 rises the most, from 0.348 to 0.514. Sol gains 0.086 on clips whose three views are all labeled V, and the gain persists without the 69 clips of arrange_flowers. GPT-5.6 Terra gains 0.042 and Gemini 3.8 Flash 0.022 in Pair queries. In Sequence queries, Sol gains 0.045, but for all four VLMs a majority vote over single views exceeds the joint input. Jev 1.14 does not gain in Sequence queries: its joint input scores 0.581, against 0.584 for the main view and 0.593 for the vote. Seven clips exceed its input limit as three separate images and are sent as one synchronized three-view mosaic; the result holds without them.

Specialized evaluators that accept several cameras are scored on every video recorded alongside other cameras († in Table 2): the video’s camera fills the primary slot and the other cameras of its execution fill the remaining slots, so the multiview and single-view results of a video share transitions and reference. Of these 1,116 videos, 804 come from clips with three cameras, 189 of them with wrist cameras aligned by timestamp, and 312 from clips with one fixed and one wrist camera, where Robo-Dopamine and Robo-ValueRL repeat the primary view in the third slot. SOLE-R1 receives one external and one wrist camera, and Flora-Rewarder the views tiled into one frame as its documentation specifies. On the three-camera videos, the four Robo-Dopamine checkpoints gain 0.072–0.101 Macro-F1 over the same single view and SOLE-R1 gains 0.048, all with 95% bootstrap intervals over executions above zero, and their falling F1 rises by 0.045–0.092. The Robo-Dopamine gains hold for V, P, and X views, for fixed and wrist cameras, and in every suite. With a P or X view in the primary slot of a clip with verified synchronization, all five exceed the single-view result of a V view of the same clip, so the other cameras supply the evidence that the primary view lacks. Flora-Rewarder does not gain (−0.010-0.010, with an interval that includes zero); over all videos it loses 0.034 on V views, which already show the decisive condition, holds on P views, and gains on the 28 X views. Robo-ValueRL reaches a Macro-F1 of 0.416, against 0.474 for the always-rising predictor on the same transitions. Because single-view use of Robo-Dopamine repeats one image in slots built for different cameras, its gains also measure the cost of that use.

The multiview judgment depends on which camera fills the primary slot. On 18 clips that were also rerun with identical input, swapping the primary camera lowers the share of transitions with the same predicted direction below that of the rerun for all seven evaluators, by 0.10 to 0.49. A multiview result is therefore an average over choices of the primary camera, not a property of the camera set.

Task Scope and Metric Sensitivity

Scope variants test whether the evaluator respects the declared scope of completion. In 322 pairs from 132 scope variants of 39 executions, the scope variant ends complete while the full task is not, with a median full-task reference of 0.57 at that moment. Scored under its own instruction, the scope variant receives more credit than the full task at the same moment in 68–90% of pairs for the Sequence-queried VLMs, but in only 29–38% for the four Robo-Dopamine models, 39% for VLAC-Cut, 45% for SOLE-R1, and 17% for LRM-8B (Progress), whose two scores tie in 47% of pairs. For these evaluators, completing the narrower instruction usually earns no more than the credit of the unfinished task.

We then compare two conventional metrics, VOC and Kendall’s τa\tau_a, with falling F1 from the direction test (Table 3).

The normalized-time control never observes the video, yet its VOC of 1.000 exceeds that of every evaluator, while its τa\tau_a and falling F1 are zero: it assigns every execution the same final value and never predicts a decrease. VOC therefore rewards chronology, not task state. Learned evaluators show the same dissociation. RoboReward has the fourth-highest VOC among specialized evaluators (0.807) and the lowest falling F1 (0.072), and Robometer and PRIMO-R1 follow the same pattern, with VOCs of 0.832 and 0.828 but falling F1 of only 0.214 and 0.154. Flora-Rewarder leads the specialized evaluators on VOC (0.915) and τa\tau_a (0.572) but ranks second on falling F1 (0.252). Jev 1.14 has the third-highest VOC of all evaluators (0.905) but ranks fifth on falling F1 (0.256) and seventh on τa\tau_a (0.496). In 52 of the 153 pairs of specialized evaluators, the one with the higher VOC has the lower falling F1. VOC and falling F1 agree more closely across the seven general-purpose VLMs (Spearman correlation 0.71, against 0.38 across the 18 specialized evaluators). On the executions alone, scored on one camera view each, the model rankings are stable (Spearman correlation 0.95 for VOC and 0.83 for τa\tau_a against the full set), and the normalized-time control again attains a VOC of 1.000.

The direction test has its own trivial control. Because 90.2% of the transitions rise, a predictor that always answers “rising” reaches a Macro-F1 of 0.474, above 9 of the 18 single-view specialized evaluators in Table 2, and the rising precision of every single-view evaluator lies between 0.908 and 0.969; the falling class carries the discrimination. Endpoint ordering and regression detection also rank the evaluators differently. Among the specialized evaluators, τa\tau_a and falling F1 have a Spearman correlation of 0.45 on all videos and 0.32 on the executions; Flora-Rewarder, Robometer, and RynnValue-4B lead on τa\tau_a, whereas VLAC-Cut, Flora-Rewarder, and Robo-Dopamine-GRM-2.0-4B-Preview lead on falling F1.

Per-execution correlations show the same pull of elapsed time. On the 131 executions whose task-state reference is non-monotone or ends below completion, the output of 17 of the 27 evaluators correlates more with elapsed time than with the task state on most of them. The share is 75% for Robometer and 69–70% for RoboReward, PRIMO-R1, Flora-Rewarder, and Claude Haiku 5.5, all with VOCs above 0.80, against 39% for GPT-4.1 Sol and 37–39% for the GRM-2.0 previews; the median difference between the two correlations stays within 0.05 for every evaluator.

Figure S4. Both general-purpose VLMs credit a placement in the wrong basket as progress. In this execution from the Semantic/Ordered Constraint suite (B), seen from the wrist camera, the robot releases the product into the wrong basket, so the task-state reference returns to zero. Claude Opus 5.5 and GPT-6 Luna keep rising, Opus to full progress. Several specialized evaluators dip at the release but rise again, and only VLAC-Cut ends near zero.

References

  1. [20]Luo, J., Xu, X., Wang, S., Zhou, Y., & Zhang, Z. (2026). CriticHack: Evaluating visual rewards under robot policy optimization. https://doi.org/arXiv:2601.02527
  2. [21]R. Firoozi et al., “Foundation models in robotics: Applications, challenges, and the future,” Int. J. Robot. Res., vol. 44, no. 5, pp. 701–739, 2025.arxiv.org/abs/2312.07843
  3. [22]Z. Mei et al., “Robotic video world models: A survey of applications, research challenges, future directions,” arXiv:2601.07823, 2026.arxiv.org/abs/2601.07823
  4. [23]Y. Ma, Z. Song, J. Zhuang, J. Hao, and I. King, “A survey on vision-language-action models for embodied AI,” IEEE Trans. Neural Netw. Learn. Syst., vol. 37, no. 7, pp. 3031–3051, 2026.DOI
  5. [24]R. Yu et al., “Reward models in deep reinforcement learning: A survey,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2025.arxiv.org/abs/2506.15421
  6. [25]J. Zhong et al., “A comprehensive survey of reward models: Taxonomy, applications, challenges, and future,” arXiv:2504.12328, 2025.arxiv.org/abs/2504.12328
  7. [26]Zheng, C. et . al . (2026). A comprehensive survey of process reward models: Data generation, model construction, and usage. Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL).
  8. [27]J. Zhang et al., “Progress reward modeling for robotic learning: A comprehensive survey,” arXiv:2607.21655, 2026.arxiv.org/abs/2607.21655
  9. [28]Y. Wan et al., “No Free Checker: A survey of verifiers for robot policies,” arXiv:2609.09250, 2026.arxiv.org/abs/2609.09250
  10. [29]Y. Du et al., “Vision-language models as success detectors,” in Proc. Conf. Lifelong Learn. Agents (CoLLAs), 2023.arxiv.org/abs/2303.07280
  11. [30]J. Yang, M. Sobol Mark, B. Vu, A. Sharma, J. Bohg, and C. Finn, “Robo fine-tuning made easy: Pre-training rewards and policies for autonomous real-world reinforcement learning,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.DOI
  12. [31]C. Grislain, H. Rahimi, O. Sigaud, and M. Chetouani, “I-FailSense: Towards general robotic failure detection with vision-language models,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.arxiv.org/abs/2509.16072
  13. [32]Liu, X., Bai, Z., Cai, H. C., Ma, Y., & Zhou, M. Z. (2026). World-VLA-Loop: Closed-loop learning of video world model and VLA policy. https://doi.org/arXiv:2602.06508
  14. [33]Q. Agi et al., “Unpacking failure modes of generative policies: Runtime monitoring of consistency and progress,” in Proc. Conf. Robot Learn. (CoRL), 2024.arxiv.org/abs/2410.04640
  15. [34]Q. Gu et al., “SAFE: Multitask failure detection for vision-language-action models,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2025.arxiv.org/abs/2506.09937
  16. [35]D. Huang et al., “VLAConf: Calibrated task-success confidence for vision-language-action models,” arXiv:2605.29605, 2026.arXiv
  17. [36]D. T. Mahato and R. Ren, “Early warning signals for OpenVLA failure under visual distribution shift,” arXiv:2606.29699, 2026.arXiv
  18. [37]Z. Qin, Z. Tan, and X. Wang, “Future anchored verification and online recovery for world action models,” arXiv:2610.06280, 2026.arXiv
  19. [38]Y. J. Ma, S. Sodhani, D. Jayaraman, O. Bastani, V. Kumar, and A. Zhang, “VIP: Towards universal visual reward and representation via value-implicit pre-training,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2023.arxiv.org/abs/2210.00030
  20. [39]Y. J. Ma, V. Kumar, A. Zhang, O. Bastani, and D. Jayaraman, “LIV: Language-image representations and rewards for robotic control,” in Proc. Int. Conf. Mach. Learn. (ICML), 2023.arxiv.org/abs/2306.00958
  21. [40]Y. Yang et al., “Adapt2Reward: Adapting video-language models to generalizable robotic rewards via failure prompts,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2024.arxiv.org/abs/2407.14872
  22. [41]P. Mandikal, J. Hu, S. Dass, S. Majumder, R. Martín-Martín, and K. Grauman, “Mash, spread, slice! Learning to manipulate object states via visual spatial progress,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.arxiv.org/abs/2509.24129
  23. [42]P. Krack, T. Jülg, W. Burgard, and F. Walter, “Rewarding DINO: Predicting dense rewards with vision foundation models,” arXiv:2603.16978, 2026.arXiv
  24. [43]H. Niu, Z. Han, Y. Ji, Z. Li, and K. Sreenath, “HiRE: Hindsight reward editing for policy finetuning,” arXiv:2609.27068, 2026.arXiv
  25. [44]M. Alakuijala et al., “Video-Language Critic: Transferable reward functions for language-conditioned robotics,” Trans. Mach. Learn. Res., 2025.arxiv.org/abs/2405.19988
  26. [45]T. W. Ayalew, X. Zhang, K. Y. Wu, T. Jiang, M. Maire, and M. R. Walter, “PROGRESSOR: A perceptually guided reward estimator with self-supervised online refinement,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025.arxiv.org/abs/2411.17764
  27. [46]C. Ziakas and A. Russo, “VITA: Zero-shot value functions via test-time adaptation of vision-language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2506.10085
  28. [47]P. Yang et al., “RARM: Confidence-gated progress reward modeling for RL in manipulation,” arXiv:2606.22027, 2026.arXiv
  29. [48]X. Qin et al., “Beyond monotonic progress: Retry-supervised value learning for robot imitation,” arXiv:2606.24633, 2026.arXiv
  30. [49]J. Yu et al., “WARP-RM: A warp-augmented relative progress reward model for data curation,” arXiv:2606.28320, 2026.arXiv
  31. [50]Y. Fang et al., “DenseReward: Dense reward learning via failure synthesis for robotic manipulation,” arXiv:2607.13033, …arXiv
  32. [51]Tu, Y., Tan, R., Yan, J., Hu, J., & Lv, C. (2026). AnyViewMeter: Adapting robotic reward models with camera geometry and multi-view attention. 2609.20106
  33. [52]J. Zhang et al., “ProgressLM: Towards progress reasoning in vision-language models,” in Proc. Annu. Meeting Assoc. Comput. Linguistics (ACL), 2026.arxiv.org/abs/2601.15224
  34. [53]S. Chen et al., “TOPReward: Token probabilities as hidden zero-shot rewards for robotics,” arXiv:2602.19313, 2026.arXiv
  35. [54]Y. Zhang et al., “Recurrent reasoning with vision-language models for estimating long-horizon embodied task progress,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026.arxiv.org/abs/2603.17312
  36. [55]P. Schroeder, T. Weng, K. Schmeckpeper, E. Rosen, S. Hart, and O. Biza, “SOLE-R1: Video-language reasoning as the sole reward for on-robot reinforcement learning,” arXiv:2603.28730, 2026.arXiv
  37. [56]Y. Feng et al., “ProcVLM: Learning procedure-grounded progress rewards for robotic manipulation,” arXiv:2605.08774, 2026.arXiv
  38. [57]J. Shu, Z. Lin, B. Wei, and Y. Wang, “Feat2Go: Visual feature-grounded value estimation for embodied reinforcement learning,” arXiv:2605.30795, 2026.arXiv
  39. [58]J. Kwok et al., “LLM-as-a-Verifier: A general-purpose verification framework,” arXiv:2607.05391, 2026.arXiv
  40. [59]S. Hu et al., “ARS: Augmented reward system for robot learning,” arXiv:2609.34484, 2026.arXiv
  41. [60]Xiao, J. et . al . (2026). RehearseVLA: Simulated post-training for VLAs with physically-consistent world model. Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR).
  42. [61]Yang, Q., Li, J., Yang, Y., & Mu, M. (2026). ProgressVLA: Progress-guided diffusion policy for vision-language robotic manipulation. 2603.27670
  43. [62]D. Azuma et al., “NavWAM: A navigation world action model for goal-conditioned visual navigation,” arXiv:2606.13494, 2026.arXiv
  44. [63]J. Li et al., “DecisionNCE: Embodied multimodal representations via implicit preference learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2024.arxiv.org/abs/2402.18137
  45. [64]Y. Liu, C. Wen, Y. Hu, D. Jayaraman, and Y. Gao, “TimeRewarder: Learning dense reward from passive videos via frame-wise temporal distance,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.arxiv.org/abs/2509.26627
  46. [65]Y. Xu et al., “Robo-Dopamine 2.0: History-conditioned and OOD-aware process reward modeling for robotic manipulation,” arXiv:2608.15680, 2026.arXiv
  47. [66]S. Sudhakar, L. Liang, S. Rammohan, P. Tokmakov, R. Zemel, and C. Vondrick, “Robot critics that sweat the small stuff,” arXiv:2606.21572, 2026.arXiv
  48. [67]H. Zhang, Z. Zeng, B. Wang, L. Li, L. Lin, and L. Cheng, “Dream2Reward: Transition-alignment reward models from positive demonstrations for robotic manipulation,” arXiv:2608.18787, 2026.arXiv
  49. [68]D. Yang, D. Tjia, J. Berg, D. Damen, P. Agrawal, and A. Gupta, “Rank2Reward: Learning shaped reward functions from passive video,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.arxiv.org/abs/2404.14735
  50. [69]R. Liu, C. Bai, J. Lyu, S. Sun, Y. Du, and X. Li, “VLP: Vision-language preference learning for embodied manipulation,” arXiv:2502.11918, 2025.arXiv
  51. [70]A. Singh et al., “VARP: Reinforcement learning from vision-language model feedback with agent regularized preferences,” arXiv:2503.13817, 2025.arXiv
  52. [71]L. Luo, G. Zhang, H. Xu, Y. Yang, C. Fang, and Q. Li, “MVR: Multi-view video reward shaping for reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2603.01694
  53. [72]F. Amin, N. Dennler, and A. Bobu, “GIFT: Generalizing intent for flexible test-time rewards,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.arxiv.org/abs/2603.22574
  54. [73]M. Torne, A. Mahajan, A. Bhat, and C. Finn, “Freeform preference learning for robotic manipulation,” arXiv:2606.32027, 2026.arXiv
  55. [74]K.-C. Chen, W. Chen, W.-F. Sun, and M.-C. Hu, “VLM-AR3L: Vision-language models for absolute and relative rewards in reinforcement learning,” in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2026.arxiv.org/abs/2607.00483
  56. [75]S. Katkuri, M. Kawada, and J. Wachs, “Beyond pairwise feedback: Listwise vision-language supervision for preference-based reward learning,” arXiv:2608.25350, 2026.arXiv
  57. [76]Jouyali, L., Demeester, E., & Tsogikas, N. (2026). Scaling vision-language reward learning for robot manipulation in parallel simulation. 2609.21767
  58. [77]J. Kwok et al., “RoboMonkey: Scaling test-time sampling and verification for vision-language-action models,” in Proc. Conf. Robot Learn. (CoRL), 2025.arxiv.org/abs/2506.17811
  59. [78]Y. Ali et al., “EVE: A generator-verifier system for generative policies,” arXiv:2512.21430, 2025.arXiv
  60. [79]J. Kwok et al., “Scaling verification can be more effective than scaling policy learning for vision-language-action alignment,” arXiv:2602.12281, 2026.arXiv
  61. [80]S. Zhao, S. Zhang, S. Yang, W. Zhao, S. Wang, and X. Ji, “TapSampling: Inference-time sampling with a task-progress-understanding verifier for robotic manipulation,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.arxiv.org/abs/2605.25547
  62. [81]Yang, T., Shi, D., Li, R., & Wang, D. (2026). PreferenceFlow: Test-time guidance of flow-matching robot policies from human interventions. 2609.36872
  63. [82]H. Wang, W. Wu, Y. Yan, and B. Zhou, “Looking back to move forward: Temporal verification for generative robot policies,” arXiv:2609.39038, 2026.arXiv
  64. [83]S. Park et al., “DiVeR: Decision-critical verifier learning for VLA test-time scaling,” arXiv:2610.04933, 2026.arXiv
  65. [84]X. Dang and S. Edelkamp, “CLIP-Motion: Learning reward functions for robotic actions using consecutive observations,” arXiv:2311.03485, 2023.arXiv
  66. [85]K.-H. Hung, P.-C. Lo, J.-F. Yeh, H.-Y. Hsu, Y.-T. Chen, and W. H. Hsu, “VICToR: Learning hierarchical vision-instruction correlation rewards for long-horizon manipulation,” arXiv:2405.16545, 2024.arXiv
  67. [86]C. Kim et al., “Subtask-aware visual reward learning from segmented demonstrations,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.arxiv.org/abs/2502.20630
  68. [87]Y. Chen, H. Li, Z. Jiang, H. Wen, and D. Zhao, “TeViR: Text-to-video reward with diffusion models for efficient reinforcement learning,” arXiv:2505.19769, 2025.arXiv
  69. [88]Y. Zhao et al., “Training-free generation of temporally consistent rewards from VLMs,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2025.arxiv.org/abs/2507.04789
  70. [89]Y. Liu et al., “PALM: Progress-aware policy learning via affordance reasoning for long-horizon robotic manipulation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026.
  71. [90]X. Zhou et al., “MARVL: Multi-stage guidance for robotic manipulation via vision-language models,” arXiv:2602.15872, 2026.arxiv.org/abs/2602.15872
  72. [91]Q. Chen et al., “SARM2: Multi-task stage aware reward modeling for self improving robotic manipulation,” arXiv:2606.10305, 2026.arxiv.org/abs/2606.10305
  73. [92]Y. Yang et al., “Stage-transition dense reward modeling for reinforcement learning,” arXiv:2606.31377, 2026.arxiv.org/abs/2606.31377
  74. [93]S. K. S. Ghasemipour, A. Wahid, J. Tompson, P. Sanketi, and I. Mordatch, “Self-improving embodied foundation models,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2025.
  75. [94]D. Huang et al., “RynnValue: Scaling robotic value foundation models with temporal distance,” arXiv:2608.09853, 2026.arxiv.org/abs/2608.09853
  76. [95]C. Bhateja et al., “Robotic offline RL from internet videos via value-function learning,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.
  77. [96]H. Song et al., “Hume: Introducing system-2 thinking in visual-language-action model,” arXiv:2505.21432, 2025.arxiv.org/abs/2505.21432
  78. [97]Physical Intelligence et al., “π₀.₆: A VLA that learns from experience,” arXiv:2511.14759, 2025.arxiv.org/abs/2511.14759
  79. [98]M. J. Kim et al., “Cosmos Policy: Fine-tuning video models for visuomotor control and planning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.
  80. [99]J. Yang et al., “Self-improving robot policy with compositional world model,” in Proc. Robot.: Sci. Syst. (RSS), 2026.
  81. [100]J. Zhang et al., “What frozen VLAs already know about success: A probing study of value-like structure in foundation robot policies,” arXiv:2605.28527, 2026.arxiv.org/abs/2605.28527
  82. [101]J. Chen et al., “MV-WAM: Manifold-aware world action model with value augmentation,” arXiv:2606.21088, 2026.arxiv.org/abs/2606.21088
  83. [102]H. Chen et al., “Robot self-improvement via human-video dynamics models,” arXiv:2606.21406, 2026.arxiv.org/abs/2606.21406
  84. [103]W. Xia et al., “Robo-ValueRL: Reliable value estimation for offline-to-online reinforcement learning,” arXiv:2607.09866, 2026.arxiv.org/abs/2607.09866
  85. [104]S. Fei, X. Yu, S. Wang, X. Zhao, J. Gong, and X. Qiu, “WCM: A world critic model for vision-language-action reinforcement learning,” arXiv:2607.29613, 2026.arxiv.org/abs/2607.29613
  86. [105]S. Singh, Z. Hu, M. Sobol Mark, J. Yu, Z. Erickson, and A. Kumar, “SeeQ: Training generalist value functions for long-horizon robotic manipulation,” arXiv:2609.22085, 2026.arxiv.org/abs/2609.22085
  87. [106]D. Wu et al., “Demonstration-free success-probability reward learning for generalist robot policies,” arXiv:2609.33653, 2026.arxiv.org/abs/2609.33653
  88. [107]G. Liang, Y. Lin, Y. Chen, K. Y. Ma, and M. Z. Shou, “From video frames to robot meta-frames: A unified latent interface for world action models,” arXiv:2611.14365, 2026.arxiv.org/abs/2611.14365
  89. [108]Z. Zhang et al., “GRAPE: Generalizing robot policy via preference alignment,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2026.
  90. [109]S. Zhai et al., “A vision-language-action-critic model for robotic real-world reinforcement learning,” arXiv:2509.15937, 2025.arxiv.org/abs/2509.15937
  91. [110]Y. Wu et al., “Large Reward Models: Generalizable online robot reward generation with vision-language models,” arXiv:2603.16065, 2026.arxiv.org/abs/2603.16065
  92. [111]S. Yong et al., “Generalizable dense reward for long-horizon robotic tasks,” arXiv:2604.00055, 2026.arxiv.org/abs/2604.00055
  93. [112]Y. Mao et al., “ARM: Advantage reward modeling for long-horizon manipulation,” arXiv:2604.03037, 2026.arxiv.org/abs/2604.03037
  94. [113]H. Ni, T. Lu, and Y. Cai, “CoRe: Combined rewards with vision-language model feedback for preference-aligned reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.
  95. [114]Y. Wang et al., “TrustboReward: Preference-ordered isotonic score editing for multi-paradigm robot reward models,” arXiv:2608.08491, 2026.arxiv.org/abs/2608.08491
  96. [115]R. Tan, Y. Tu, Y. Yan, L. Yu, X. Tian, and C. Lv, “FIERCE: From generalist robot policies to fast specialists via progress-failure feedback,” arXiv:2609.18651, 2026.arxiv.org/abs/2609.18651
  97. [116]S. Leng et al., “FLORA: An open model suite for scaling robotics data engines,” arXiv:2610.31779, 2026.arxiv.org/abs/2610.31779
  98. [117]J. Duan et al., “AHA: A vision-language-model for detecting and reasoning over failures in robotic manipulation,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025.
  99. [118]Y. Liu et al., “From passive observer to active critic: Reinforcement learning elicits process reasoning for robotic manipulation,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2026.
  100. [119]S. Zhai et al., “HELP: Human-efficient large-scale robot post-training with rollout segmentation,” arXiv:2607.09776, 2026.arxiv.org/abs/2607.09776
  101. [120]B. Peng et al., “ReWorld: Multi-dimensional reward modeling for embodied world models,” arXiv:2601.12428, 2026.arxiv.org/abs/2601.12428
  102. [121]H. Wu et al., “RoboAlign-R1: Distilled multimodal reward alignment for robot video world models,” arXiv:2605.03821, 2026.arxiv.org/abs/2605.03821
  103. [122]P. Li, Z. Lin, Q. Wu, Y. Lv, F. Wang, and S. You, “Reward as an agent for embodied world models,” arXiv:2606.19990, 2026.arxiv.org/abs/2606.19990
  104. [123]M.-L. Nguyen et al., “RoboGaze: Evaluating robot world models via structured vision-language analysis,” arXiv:2608.26385, 2026.arxiv.org/abs/2608.26385
  105. [124]S. Zhou et al., “EVO-WAM: Evolving world action models through video-action verification,” arXiv:2609.38057, 2026.arxiv.org/abs/2609.38057
  106. [125]S. Ma et al., “RoboJudge: Multimodal language models as judges for embodied video generation,” arXiv:2610.27782, 2026.arxiv.org/abs/2610.27782
  107. [126]Y. Ji et al., “PRM-as-a-Judge: A dense evaluation paradigm for fine-grained robotic auditing,” arXiv:2603.21669, 2026.arxiv.org/abs/2603.21669
  108. [127]Q. Wang et al., “Towards the harness of embodied agents,” arXiv:2608.11246, 2026.arxiv.org/abs/2608.11246
  109. [128]Z. Zhou et al., “VeriFine: Scaling verification for self-improvement in embodied reasoning,” arXiv:2610.08761, 2026.arxiv.org/abs/2610.08761
  110. [129]A. Y. Ng and S. Russell, “Algorithms for inverse reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2000.
  111. [130]P. Abbeel and A. Y. Ng, “Apprenticeship learning via inverse reinforcement learning,” in Proc. Int. Conf. Mach. Learn. (ICML), 2004.
  112. [131]B. D. Ziebart, A. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning,” in Proc. AAAI Conf. Artif. Intell., 2008.
  113. [132]J. Ho and S. Ermon, “Generative adversarial imitation learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2016.
  114. [133]F. K. Luo, and S. Levine, “Learning robust rewards with adversarial inverse reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2018.
  115. [134]P. F. Christiano, J. Leike, T. B. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017.
  116. [135]J. Fu, A. Singh, D. Ghosh, L. Yang, and S. Levine, “Variational inverse control with events: A general framework for data-driven reward definition,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2018.
  117. [136]A. Singh, L. Yang, C. Finn, and S. Levine, “End-to-end robotic reinforcement learning without reward engineering,” in Proc. Robot.: Sci. Syst. (RSS), 2019.
  118. [137]P. Sermanet et al., “Time-contrastive networks: Self-supervised learning from video,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2018.
  119. [138]A. Escontrela et al., “Video prediction models as rewards for reinforcement learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2023.
  120. [139]T. Huang, G. Jiang, Y. Ze, and H. Xu, “Diffusion Reward: Learning rewards via conditional video diffusion,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2024.
  121. [140]A. Protopapa, D. Buoso, F. Pistilli, G. Chavaltzaki, and G. Averta, “GORDON: Graph-based object-centric rewards for decomposition of long-horizon manipulation,” arXiv:2608.03753, 2026.arxiv.org/abs/2608.03753
  122. [141]J. Tan, G. Adineera, and J. Kim, “ProgAgent: A continual RL agent with progress-aware rewards,” arXiv:2603.07784, 2026.arxiv.org/abs/2603.07784
  123. [142]P.-Y. Wu, D.-J. Chang, S.-Y. Hsiao, H.-M. Chen, Y.-C. Su, and T.-W. Ke, “TaRL: Learning general and physical rewards from tactile demonstrations,” arXiv:2609.36785, 2026.arxiv.org/abs/2609.36785
  124. [143]R. Tsao, A. Wagenmaker, and S. Levine, “Learning process rewards via success visitation matching for efficient RL,” arXiv:2606.23640, 2026.arxiv.org/abs/2606.23640
  125. [144]R. Tian, Y. Wu, C. Xu, M. Tomizuka, J. Malik, and A. Bajcsy, “Maximizing alignment with minimal feedback: Efficiently learning rewards for visuomotor robot policy alignment,” arXiv:2412.04835, 2024.arxiv.org/abs/2412.04835
  126. [145]M. Nakamoto, O. Mees, A. Kumar, and S. Levine, “Steering your generalists: Improving robotic foundation models via value guidance,” in Proc. Conf. Robot Learn. (CoRL), 2024.
  127. [146]C. Song, B. Qian, Y. Feng, and Z. Song, “VLA-Corrector: Stage-aware observable state understanding for prompt-based closed-loop recovery of vision-language-action policies,” arXiv:2609.06508, 2026.arxiv.org/abs/2609.06508
  128. [147]Y. J. Ma et al., “Eureka: Human-level reward design via coding large language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.
  129. [148]T. Xie et al., “Text2Reward: Reward shaping with language models for reinforcement learning,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2024.arxiv.org/abs/2309.11489
  130. [149]W. Yu et al., “Language to rewards for robotic skill synthesis,” in Proc. Conf. Robot Learn. (CoRL), 2023.arxiv.org/abs/2306.08647
  131. [150]A. Radford et al., “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. Mach. Learn. (ICML), 2021.arxiv.org/abs/2103.00020
  132. [151]T. Chen et al., “RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026.arxiv.org/abs/2506.18088
  133. [152]Ng, A. Y., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping. Proc. Int. Conf. Mach. Learn. (ICML).
  134. [153]J.-B. Alayrac et al., “Flamingo: A visual language model for few-shot learning,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2022.arxiv.org/abs/2204.14198
  135. [154]DeepMind, G. (2026). Gemini Robotics ER 2 model card. <a href="https://doi.org/https://deepmind.google/models/model-cards/gemini-robotics-er-2/" rel="noopener noreferrer" target="_blank">https://doi.org/https://deepmind.google/models/model-cards/gemini-robotics-er-2/</a>
  136. [155]M. J. Kim et al., “OpenVLA: An open-source vision-language-action model,” in Proc. Conf. Robot Learn. (CoRL), 2024.arxiv.org/abs/2406.09246
  137. [156]Siméoni, O., & al., . et . (2025). DINOv3. https://doi.org/2508.10104
  138. [157]R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs: I. The method of paired comparisons,” Biometrika, vol. 39, no. 3/4, pp. 324–345, 1952.DOI
  139. [158]OpenAI. (2024). GPT-4o system card. https://doi.org/2410.21276
  140. [159]Jeung, W., & al., . et . (2026). Same trajectory, contradictory rewards (RoboRMBench): Paraphrase fragility in vision language reward models. https://doi.org/2609.05401
  141. [160]Budzianowski, P., & al., . et . (2025). OpenGVL – Benchmarking visual temporal progress for data curation. https://doi.org/2509.17321
  142. [161]Liu, Y., & al., . et . (2026). PRM-as-a-Judge 1.5: A toolkit for robot process assessment. https://doi.org/2608.14284
  143. [162]Z. Navasardyan, T. Danielyan, and H. Davtyan, “FailBench: How reliable are VLMs at judging robot task success?” arXiv:2609.03611, 2026.arxiv.org/abs/2609.03611
  144. [163]Liu, S., & al., . et . (2026). RoboValue: A fine-grained sim-and-real benchmark for unified evaluation of robotic value models. https://doi.org/2610.26663
  145. [164]Open X-Embodiment Collaboration et al., “Open X-Embodiment: Robotic learning datasets and RT-X models,” in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), 2024.arxiv.org/abs/2310.08864
  146. [165]A. Khazatsky et al., “DROID: A large-scale in-the-wild robot manipulation dataset,” in Proc. Robot.: Sci. Syst. (RSS), 2024.arxiv.org/abs/2403.12945
  147. [166]H. Walke et al., “BridgeData V2: A dataset for robot learning at scale,” in Proc. Conf. Robot Learn. (CoRL), 2023.arxiv.org/abs/2308.12952
  148. [167]K. Wu et al., “RoboMIND: Benchmark on multi-embodiment intelligence normative data for robot manipulation,” in Proc. Robot.: Sci. Syst. (RSS), 2025.arxiv.org/abs/2412.13877
  149. [168]Damen, D., & al., . et . (2018). Scaling egocentric vision: Collection, pipeline and challenges for EPIC-KITCHENS-100. Int. J. Comput. Vis., 130(1), 33–55.
  150. [169]R. Hoque, P. Huang, D. J. Yoon, M. Sivapurapu, and J. Zhang, “EgoDex: Learning dexterous manipulation from large-scale egocentric video,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2026.arxiv.org/abs/2505.11709
  151. [170]Ge, Z., & al., . et . (2026). EgoRecovery: Acquiring failure recovery ability through human recovery demonstration. https://doi.org/2607.19745
  152. [171]Ye, Z., & al., . et . (2025). RoboFAC: A comprehensive framework for robotic failure analysis and correction. https://doi.org/2505.12224
  153. [172]Lin, Z., & al., . et . (2025). FailSafe: Reasoning and recovery from failures in vision-language-action models. https://doi.org/2510.01642
  154. [173]P. Atreya et al., “RoboArena: Distributed real-world evaluation of generalist robot policies,” in Proc. Conf. Robot Learn. (CoRL), 2025.arxiv.org/abs/2506.18123
  155. [174]Li, D., & al., . et . (2026). Learning actionable manipulation recovery via counterfactual failure synthesis. https://doi.org/2603.13528
  156. [175]R. L. Plackett, “The analysis of permutations,” J. Roy. Statist. Soc. Ser. C, Appl. Statist., vol. 24, no. 2, pp. 193–202, 1975.DOI
  157. [176]Luce, R. D. (1959). Individual Choice Behavior: A Theoretical Analysis.
  158. [177]Li, R., Korkmaz, K., & Bıyık, E. (2026). Reward-DAgger: Robot-gated interactive imitation learning with general-purpose progress-based reward models. https://doi.org/2610.04054
  159. [178]Z. Bai, C. Gao, and M. Z. Shou, “EVOLVE-VLA: Test-time training from environment feedback for vision-language-action models,” arXiv:2512.14666, 2025.arxiv.org/abs/2512.14666
  160. [179]Z. Liu, A. Bahety, and S. Song, “REFLECT: Summarizing robot experiences for failure explanation and correction,” in Proc. Conf. Robot Learn. (CoRL), 2023.arxiv.org/abs/2306.15724
  161. [180]X. Zeng et al., “Diagnose, correct, and learn from manipulation failures via visual symbols,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026.arxiv.org/abs/2512.02787
  162. [181]Xia, D., & al., . et . (2026). Robo2Process: Benchmarking and learning process-aware understanding in vision-language robotic manipulation. https://doi.org/2606.13040
  163. [182]Wu, Y., & al., . et . (2026). RoboChrono: A real robot benchmark for streaming task understanding. https://doi.org/2609.36605
  164. [183]Yakefu, A., & al., . et . (2025). RoboChallenge: Large-scale real-robot evaluation of embodied policies. https://doi.org/2510.17950
  165. [184]Li, Y., Zhu, Y., Wen, C., Shen, X., & Xu, Y. (2022). WorldEval: World model as real-world robot policies evaluator. https://doi.org/2505.19017
  166. [185]Li, Y., & al., . et . (2026). Scaling robot policy evaluation via discrete diffusion world model. Proc. Int. Conf. Mach. Learn. (ICML).
  167. [186]Y. Yu and X. Qiu, “Benchmarking vision-language-action models on SO-101: Failure and recovery analysis,” arXiv:2606.08881, 2026.arxiv.org/abs/2606.08881
  168. [187]Team, R. D. (2026). Table30 V2: Evaluating generalized models by real robots at scale. In Dexmal, Tech. Rep.. https://robochallenge.ai/robachallenge_techreport_V2.pdfrobochallenge.ai/robachallenge_techreport_V2.pdf
  169. [188]AI, T. (2026). “Jev: For the builders AND dreamers”. <a href="https://doi.org/https://typesafe.ai/blog/jev-1-14" rel="noopener noreferrer" target="_blank">https://doi.org/https://typesafe.ai/blog/jev-1-14</a>
  170. [189]Team, Q. (2025). Qwen3-VL technical report. https://doi.org/2511.21631
  171. [190]OpenAI. (2026). Addendum to GPT-6 Astra system card: GPT-6.1 Sol. <a href="https://doi.org/https://deploymentsafety.openai.com/gpt-6-1-sol" rel="noopener noreferrer" target="_blank">https://doi.org/https://deploymentsafety.openai.com/gpt-6-1-sol</a>
  172. [191]OpenAI. (2026). GPT-6 Astra system card. <a href="https://doi.org/https://deploymentsafety.openai.com/gpt-6-astra" rel="noopener noreferrer" target="_blank">https://doi.org/https://deploymentsafety.openai.com/gpt-6-astra</a>
  173. [192]OpenAI. (2026). GPT-5.6 system card. <a href="https://doi.org/https://deploymentsafety.openai.com/gpt-5-6" rel="noopener noreferrer" target="_blank">https://doi.org/https://deploymentsafety.openai.com/gpt-5-6</a>
  174. [193]Anthropic. (2026). System card: Claude Opus 5.5. <a href="https://doi.org/https://www.anthropic.com/claude-opus-5-5-system-card" rel="noopener noreferrer" target="_blank">https://doi.org/https://www.anthropic.com/claude-opus-5-5-system-card</a>
  175. [194]Anthropic. (2026). System card: Claude Sonnet 5.5. <a href="https://doi.org/https://www.anthropic.com/claude-sonnet-5-5-system-card" rel="noopener noreferrer" target="_blank">https://doi.org/https://www.anthropic.com/claude-sonnet-5-5-system-card</a>
  176. [195]Anthropic. (2026). System card: Claude Haiku 5.5. <a href="https://doi.org/https://www.anthropic.com/document/claude-haiku-5-5-system-card" rel="noopener noreferrer" target="_blank">https://doi.org/https://www.anthropic.com/document/claude-haiku-5-5-system-card</a>
  177. [196]DeepMind, G. (2026). Gemini 3.8 Flash model card. <a href="https://doi.org/https://deepmind.google/models/model-cards/gemini-3-8-flash/" rel="noopener noreferrer" target="_blank">https://doi.org/https://deepmind.google/models/model-cards/gemini-3-8-flash/</a>

Paper details

vlnc:W_jydk3u2p
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents