betaFeedback

WorksComputer Science

RageVLA: Augmentation-Trained Perception and Action for State-of-the-Art Robustness on LIBERO

Abstract

Vision-language-action (VLA) models look saturated on LIBERO, yet most of them fall apart on LIBERO-PRO as soon as an object moves or the instruction asks for something new. We argue that what is missing is not scale but decoupling. In these policies, what to do is tied to where things were during training, to the scene around them and to the wording of the instruction. We cut these ties in two places. In the data, LIBERO demonstrations are re-arranged by general rules: every object and fixture is moved and rotated, tasks are carried into other environments and onto other surfaces, and every instruction is paraphrased. In the policy, a perception engine on frozen DINOv3 features, trained on this augmented data, hands the action model the scene in decoupled form: a list of objects with their positions, and a compact code of what the gripper is looking at, next to image and instruction embeddings from TinyCLIP. A flow-matching diffusion transformer, trained from scratch, acts on this view. We call this combination of Robust Augmentation and Grounded Encoding RageVLA. With 0.21 billion parameters, RageVLA sets a new state of the art on LIBERO-PRO among VLA policies, scoring 0.72 against 0.53 for the best published model, and a larger variant reaches 0.73. Two further tests on LIBERO-Object show that the policy acts on what it perceives. When the object positions it is given are shuffled while the camera images stay correct, its success falls from 0.965 to 0.215. When the target object is displaced by 28 cm, it still succeeds in 0.865 of the episodes, where π0.5π_{0.5}, run through the same harness, succeeds in 0.105.

Introduction

LIBERO [1] has become the default benchmark for vision-language-action (VLA) policies, and its four evaluation suites are now saturated: recent models report average success well above 90% [2, 3, 4]. LIBERO-PRO [5] asks what these numbers measure. It keeps LIBERO’s scenes and perturbs one factor at a time: object appearance (Obj), initial positions (Pos), instruction wording (Sem), the task itself (Task) and the table (Env). Strong LIBERO models collapse under two of these. On the LIBERO-PRO leaderboard, OpenVLA [6], π0\pi_0 [7], MolmoAct [8] and NORA [9] score zero on almost every Pos and Task cell, and the best model, π0.5\pi_{0.5} [10], averages 0.11 over them. The models have learned to replay trajectories.

The benchmark’s headline number hides this. LIBERO-PRO summarizes a model by its mean over all cells, but Obj and Sem leave both the layout and the task unchanged, so a model that replays its training trajectories still scores highly on them.

OpenVLA, for example, has a total of 0.52 while solving none of the position-perturbed or task-changed tests. Figure 1 separates the two readings.

Figure 1. LIBERO-PRO’s official total (grey) against the mean over the position and task perturbations only (colored). Prior VLAs keep high totals while scoring close to zero once objects move or the task changes.

This paper takes a simple position: these policies fail because what they do is coupled to things it should not depend on. Each LIBERO task is demonstrated 50 times in almost the same layout, so the identity of an object is tied to where it stands, the motion is tied to the scene around it, and the instruction is tied to one wording and one initial layout. A policy trained on such data has no reason to separate them, whatever its size. Robustness under perturbation requires decoupling these factors, and it has to happen in two places at once.

The first is the data. We re-arrange LIBERO’s demonstrations by general rules. Re-placing every object and fixture at a random position and orientation decouples what an object is from where it is. Carrying tasks into other environments and onto other surfaces decouples the manipulation from its background. Paraphrasing decouples the task from its wording. The second is the policy’s input. Variation in the data helps only if the policy can read it, so a perception engine hands the action model the same factors in separated form: a list of objects with their classes and global positions, a wrist code of the surfaces and depths in front of the gripper, and one image and one instruction embedding. With accurate object positions as an input, the policy does not need to recall a trajectory; it can act on the scene in front of it. The two parts are designed for each other: without varied data the position input carries no information that the policy needs, and without the position input the variation has to be recovered from pixels. We call the resulting system RageVLA, for Robust Augmentation with Grounded Encoding.

Our contributions are as follows.

  • A general augmentation pipeline for LIBERO-style demonstrations. Every object and fixture is re-placed at a random position and orientation, tasks are transplanted into LIBERO’s other environments and onto its other surfaces, and every instruction is paraphrased (Sec. III). The rules are general: no layout, object pair or instruction is taken from LIBERO-PRO.

  • RageVLA, an object-centric flow-matching policy. Small perception adapters on a frozen DINOv3 backbone [11] produce an object list and a wrist-view code, TinyCLIP [12] provides one image and one text embedding,

and a diffusion transformer trained from scratch produces actions (Sec. IV).

  • State-of-the-art LIBERO-PRO results among VLA policies on the full protocol, with 0.21 billion parameters in total (§ 5).

  • Direct evidence that the policy acts on perceived positions. Changing the object positions given to the policy at inference changes its behaviour even in LIBERO’s original layouts, and under LIBERO-PRO’s graded object displacement the policy degrades far more slowly than π0.5\pi_{0.5} evaluated in the same harness (§ 5-D).

VLAs on LIBERO: OpenVLA [6] and OpenVLA-OFT [2] fine-tune a 7B vision-language model. π0\pi_0 and π0.5\pi_{0.5} [7, 10] attach a flow-matching action expert to a 3B vision-language model. MolmoAct [8] and NORA [9] decode discrete action tokens autoregressively, and X-VLA [4] pairs a Florence encoder with a flow-matching transformer. Smaller policies such as SmolVLA [13] and Octo [14] trade accuracy for size. All of these compute actions from the image tokens of one vision-language backbone. RageVLA factorizes the input instead: explicit object positions and a wrist code come from small perception adapters, and the vision-language input comes from an aligned image-text encoder, TinyCLIP [12], a distilled CLIP [15], as one image embedding and one instruction embedding.

Robustness benchmarks: LIBERO-PRO [5] and LIBERO-Plus [16] perturb LIBERO’s objects, layouts, language, cameras and scenes, and show that LIBERO success does not transfer. We evaluate on LIBERO-PRO with its released files and its own perturbation code, including its graded object-displacement suites, which no leaderboard entry reports.

Data generation for imitation learning: MimicGen [17] re-targets object-centric segments of human demonstrations

to new object poses and validates them in simulation. Our augmentation follows the same principle on LIBERO and extends it to full-scene re-layouts, fixtures, other environments and other surfaces.

Policies and conditioning: Diffusion Policy [18] and flow-matching action heads [7] model action chunks generatively. We use a diffusion transformer [19] trained with flow matching [20, 21] and the parameter-efficient conditioning of PiXArt-α\alpha [22]. Our object head follows CenterNet [23] on DINOv3 features [11].

LLM agents as policies: Code-as-policies [24] and, recently, ASPIRE [25] use a large language model to write robot programs on top of perception and motion-planning interfaces. ASPIRE reports high success on LIBERO-PRO’s position and task perturbations. It is a different paradigm, in which a frontier language model writes and debugs a program for each task, and we discuss it separately from learned VLA policies (§ 5-F).

Decoupling in the Data: General Augmentation

All data starts from LIBERO’s human demonstrations, 50 for each of its 130 tasks. Generation stores simulator states and actions only, and images are rendered afterwards. Like MimicGen [17], we generate new demonstrations by transforming segments of the human ones into new scenes. Each demonstration is cut into object-centric phases (approach, grasp, transport, place, fixture interaction), and each phase is assigned the entity it acts on. For a phase acting on entity oo, every end-effector pose Te,tET^{E}_{e,t} of the source demonstration is first expressed in the frame of oo at its source pose ToWT^{W}_{o}, and then mapped back to the world through the entity’s new pose T~oW\tilde{T}^{W}_{o}:

T~e,tW=T~oW(ToW)−1Te,tW.(1)\tilde{T}^{W}_{e,t} = \tilde{T}^{W}_{o}\left(T^{W}_{o}\right)^{-1}T^{W}_{e,t}. \tag*{(1)}

The relative motion between gripper and object is therefore preserved wherever the object is placed and however it is rotated. Consecutive phases are joined by interpolated transit motions, and the gripper commands are kept from the source. Every generated trajectory is re-simulated and kept only if the task’s own success predicate holds at the end. Seeds depend only on the task, the demonstration, the generator and the round, so the data can be regenerated exactly.

Random re-layout: In every generated trajectory the scene is laid out anew. In most attempts the layout is uniform: every fixture is placed anywhere on the table with a yaw anywhere in ±180∘\pm180^\circ, carrying its contents, and every free object is placed anywhere within the robot’s reach with a yaw drawn according to its symmetry. The others use smaller random displacements of the object to be grasped, the destination and the distractors. Tasks whose instruction is relational (“the bowl next to the plate”) receive only the smaller displacements, so that the instruction remains true. Candidate layouts are filtered for workspace bounds, clearance, stability and reachability before re-simulation. No positions are exchanged between objects, and no layout is taken from any file other than LIBERO’s own scene definitions.

Environment transplants: A task is carried into every other LIBERO environment that can host it, and the same random re-layout is applied on top. A transplant is valid when the task involves at most one fixture and at most three entities in total, and every object or fixture named in the instruction exists exactly once in the target environment, so that the instruction stays unambiguous. Objects are never inserted into a scene. Goal regions that are named after an entity, such as the front of the stove, move with that entity. The transplanted trajectory keeps the instruction of the task it carries.

Surface transplants: Every task is also moved onto each of LIBERO’s other table surfaces, again with a random re-layout on top.

Language: A language model wrote 20 paraphrases for each instruction. During training the original instruction or one of its paraphrases is drawn at random.

Quota and split: Each task receives a fixed quota of successful trajectories per round, split between its own environment, other environments and other surfaces (Appendix A). Data is generated in rounds with different seeds; all but one are used for training, and the last is held out as a test set and is used only to measure perception and action error (Table I).

SplitTrajectoriesSamples
Training132,6102.79M
Held-out test1,75136.6k

Table I. Data. All generated trajectories passed the task’s success check in simulation. Samples are taken every 8 control steps.

What the data decouples: Each rule removes one correlation that LIBERO’s demonstrations contain. Random re-layout decouples the identity of an object from its place: almost no training trajectory keeps an object or a fixture where LIBERO placed it, so the policy cannot succeed by recalling a path and has to read where the objects are from its inputs. Environment and surface transplants, each with a new layout on top, decouple the manipulation from its background: the room and the table change independently of where the objects stand, so neither can serve as a landmark for a stored motion. Paraphrases decouple the task from its wording. The policy thus learns to grasp an object at any point of the workspace, in any orientation, and in any environment or on any surface within LIBERO (Figure 2).

Three rows of original layouts, perturbed test layouts, and completed tasks

Figure 2. RageVLA s1 on the held-out test data. Each row is one task: the original LIBERO layout, the perturbed test layout with fixtures and objects moved and rotated, and the scene after the policy has completed the task from the perturbed layout.

General rules, not benchmark cases: None of these rules is derived from LIBERO-PRO’s perturbations. LIBERO-PRO’s position test exchanges the regions of fixed pairs of objects; we never exchange positions and never read its files, and a swapped layout is simply one of the layouts that uniform placement can produce. Its environment test moves tasks onto one other table (Appendix C); our rule carries every task into every environment and onto every surface that can host it. We randomize neither object textures nor colors, so appearance is the one factor we leave coupled, and § 5 shows where that costs.

Decoupling in the Policy: RageVLA

The data separates where objects are from what the task is; the policy’s input has to keep them separate. RageVLA therefore does not ask the action model to recover the layout from image tokens. Perception is a separate, supervised stage that outputs the layout explicitly, and the action model is

conditioned on that output. Figure 3 gives an overview. The policy sees LIBERO’s standard agentview and wrist cameras, the robot’s proprioceptive state and the instruction, and it outputs a chunk of 20 future actions. The system has two parts. The perception engine (174M parameters) consists of DINOv3, shared by both cameras and counted once (86M), the two adapters (4M) and TinyCLIP (84M); only the adapters are trained. The action engine is the context encoder and diffusion transformer, 35M parameters in RageVLA s1 and 100M in RageVLA n1, trained from scratch. The complete inference stack therefore has 0.21 and 0.27 billion parameters. Appendix A gives layer-level details, losses and hyperparameters.

Figure 3. RageVLA, with the parameter count of every unit. Grey units are pre-trained and frozen, teal units are the trained perception adapters and their outputs, and the action model is shown in coral. Two small adapters on DINOv3 features turn the agentview into an object list and the wrist view into a compact code. TinyCLIP turns the agentview and the instruction into one embedding each. A context encoder fuses these inputs, and a flow-matching diffusion transformer attends to them to denoise an action chunk. The dashed decoder is used only to train the wrist encoder.

Why DINOv3

Both views are encoded by a frozen DINOv3 ViT-B/16 [11] at the cameras’ native resolution. DINOv3 is trained with Gram anchoring, which keeps its dense patch features well localized over long self-supervised training. Its patch features therefore carry local geometry (edges, surface orientation, relative depth and part boundaries) as well as object identity, which is what a light head needs in order to locate objects in 3-D and to read depth. We combine features from several depths of the network: earlier layers are sharper in position, the last is stronger in identity. The backbone is never fine-tuned, so the adapters cannot overfit it to LIBERO’s textures.

Agentview object head

A small convolutional head with skip connections, in the style of CenterNet [23], predicts for every cell of the feature grid a class heatmap together with the object’s sub-cell offset, yaw and height. Heatmap peaks become detections. Each peak defines a camera ray, and intersecting the ray with the horizontal plane at the predicted height gives the object’s 3-D position. The policy receives the result as a list of objects, each with its class, global position, yaw and confidence. Every label comes from the simulator.

Wrist encoder

The wrist image is encoded by the same frozen DINOv3. A small network compresses its features into a few tokens. To make this code useful for grasping, a decoder that is used only during training must reconstruct two dense maps from it: the class of what is visible in each cell, including surfaces, the gripper fingers and any held object, and its depth. The code must therefore keep what is under and around the gripper and how far away it is.

Perception accuracy

Table II reports both adapters on held-out frames, including frames from the held-out test set, whose layouts were never

ValidationHeld-out test
Objects found99.3%99.5%
Object position error, median0.26 cm0.27 cm
Wrist cell class accuracy99.8%
Wrist object cell accuracy99.4%
Wrist depth error, median0.31 cm

Table II. Perception accuracy on held-out frames (validation split and held-out test set).

seen in training. The object head finds more than 99% of objects with a median position error below 3 mm, so the policy receives global object positions that are accurate within a small tolerance wherever the objects are placed. The wrist code keeps the class of almost every cell under the gripper and its depth.

Vision-language conditioning with TinyCLIP

Beyond the two perception adapters, all visual-semantic and linguistic information that the policy receives comes from TinyCLIP [12], a distilled CLIP [15]. Its image tower embeds the agentview into one vector and its text tower embeds the instruction into another. Because the text tower is frozen, all instructions and paraphrases are embedded once and stored; an unseen instruction is embedded at inference.

The contrast with VLM-based VLAs is one of scale. Their actions are computed from hundreds of high-dimensional tokens per image, produced by a backbone with billions of parameters. RageVLA receives two 512-dimensional vectors from an 84M-parameter encoder. The larger models bring broad pre-trained knowledge that RageVLA lacks. We do not claim that the small encoder is better. Our claim is narrower: on LIBERO-PRO, pre-trained knowledge did not prevent memorization, while a small aligned embedding combined with explicit perception and varied data did not prevent generalization.

The image embedding is partly redundant with the object list and the wrist code, and we keep it deliberately. During training each input group (objects, wrist, image, text) is dropped independently with a small probability, which discourages reliance on any single channel. Sec. V-D measures how the trained policy uses each of them.

Flow-matching diffusion transformer

Each input becomes one or more tokens: one per detected object, eight for the wrist code, and one each for the image embedding, the text embedding and the robot state. A small transformer encoder fuses them into a context. The denoiser is a diffusion transformer [19] over the action tokens of the chunk. Each block applies self-attention, cross-attention to the context and a feed-forward layer, and is modulated by the flow time through the shared conditioning scheme of PixArt-α [22], which saves parameters that we spend on depth. We train two sizes, RageVLA s1 with 35M parameters and RageVLA n1 with 100M (Appendix A).

The model is trained from scratch with the flow-matching objective [20], [21]: it learns the velocity that carries noise to the action chunk, integrates it in a few steps at inference, executes the first half of the chunk and re-plans. On the held-out test set the action error equals the validation error (Appendix A), so the policy is not fitted to the trajectories it was trained on.

Experiments

Protocol

We follow LIBERO-PRO’s single-perturbation evaluation, in which one perturbation is active at a time and every cell of the table is 10 tasks with 50 initial states each, 12,000 episodes in total. For Obj, Pos, Sem and Task we use the released task and initial-state files. No Env files are released, so we generate them with LIBERO-PRO’s own perturbation code. The Pos column is the benchmark’s swap test, as on the leaderboard, and the policy always receives the instruction stored in the evaluated task file. Appendix C explains why both points matter. Episode limits and success checks are the official ones, and colliding initial states are not filtered. Baseline numbers in the table are copied unchanged from the LIBERO-PRO leaderboard [26]; the only baseline we ran ourselves is π0.5\pi_{0.5} on LIBERO-Object (Sec. V-D). We report one training seed. Both RageVLA s1 and RageVLA n1 were evaluated on all 12,000 episodes with zero errors; charts show RageVLA s1.

Main results

Table III and Figure 4 give the full table. RageVLA s1 has the highest total, 0.72 against 0.53 for π0.5\pi_{0.5}. Its advantage is concentrated where prior models fail. It is ahead of every prior model in every Task cell, on Pos in three of four suites (0.53, 0.51 and 0.94 on Spatial, LIBERO-10 and Obj, against at most 0.20 for any prior model), and on Env in two suites. On Obj and Sem, where most models already score highly, it is comparable but not better. The leaderboard’s Task column depends on which instruction the evaluated policy was given (Appendix C). Against an independent re-evaluation of π0.5\pi_{0.5} that uses the same instruction source as we do, the Task mean is 0.52 against 0.26 rather than against 0.01. Its weakest cell is Obj on LIBERO-10, which is consistent with the absence of appearance augmentation: appearance is the factor that our data leaves coupled. RageVLA n1 reaches a total of 0.73 with the same profile: it is stronger on Spatial Pos (0.62) and on Spatial and LIBERO-10 Env, and weaker on Object Obj. With an action engine three times larger, it is only 0.01 ahead of s1, which suggests that the inputs and the data, not the size of the action model, limit the scores.

ModelLIBERO-Goal ObjPosSemTaskEnvLIBERO-Spatial ObjPosSemTaskEnvLIBERO-10 ObjPosSemTaskEnvLIBERO-Object ObjPosSemTaskEnvTotal
OpenVLA0.960.000.980.000.980.970.000.970.000.890.810.000.960.000.850.980.000.980.000.000.52
π0\pi_00.940.000.930.000.390.950.000.970.000.600.790.000.820.000.270.940.000.900.000.290.44
π0.5\pi_{0.5}0.970.380.970.000.460.970.200.970.010.460.920.080.930.010.460.980.170.960.010.730.53
MolmoAct0.680.000.850.00–0.900.000.880.00–0.540.000.740.06–0.920.060.960.00–0.41
NORA0.580.000.880.00–0.920.000.910.00–0.460.000.740.00–0.860.000.920.00–0.40
X-VLA0.680.010.980.09–0.970.000.960.00–0.620.000.950.10–0.890.020.980.08–0.46
RageVLA s1 (ours)0.720.360.950.200.840.920.530.900.510.860.450.510.880.420.860.780.940.990.960.870.72
RageVLA n1 (ours)0.790.380.950.170.850.940.620.840.540.940.480.430.910.380.880.680.960.990.960.870.73

Table III. LIBERO-PRO success rates with one perturbation active at a time. Prior VLAs are taken from the LIBERO-PRO leaderboard, and a dash means not reported. The best value per column is in bold.

Figure 4. LIBERO-PRO success per suite and perturbation. Rows above the line are published VLAs from the leaderboard, and a dash means not reported.

Reading the table without memorization

Table IV splits LIBERO-PRO’s columns. On Obj and Sem every compared model scores highly, and RageVLA s1 is in the same range. On Pos and Task the six prior VLAs average between 0.00 and 0.11, and RageVLA s1 averages 0.55 with a far smaller model. Including Env, which only three prior models report, RageVLA s1 reaches 0.65 against at most 0.25. RageVLA n1 has 0.56 and 0.66.

ModelParamsTotalObj+SemPos+TaskPos+Task+Env
OpenVLA7.50B0.520.950.000.23
π0\pi_03.30B0.440.900.000.13
π0.5\pi_{0.5}3.30B0.530.960.110.25
MolmoAct8.10B0.410.810.01–
NORA3.80B0.400.780.00–
X-VLA0.90B0.460.880.04–
RageVLA s1 (ours)0.21B0.720.830.550.65
RageVLA n1 (ours)0.27B0.730.820.560.66

Table IV. Splitting the LIBERO-PRO total. Obj and Sem leave layout and task unchanged, so models that replay training trajectories still score highly on them. The columns that move objects or change the task separate the models.

Does the policy act on what it perceives?

The LIBERO-PRO table shows that RageVLA succeeds under perturbation. It does not show how: a policy could still replay trajectories that happen to cover the test layouts. We test this directly on LIBERO-Object with RageVLA s1. Every number below is from 10 tasks with 20 initial states each, 200 episodes, under the protocol of the main table (95% binomial half-width at most 0.07).

Changing one input at inference: We alter one input of the action model and leave everything else untouched, including the camera images (Fig. 5). Removing the object list or the wrist code applies the mask that input dropout applies during training. Shuffling gives every detected object the position and yaw of the next detected object and keeps its class, so the policy receives a complete but wrong layout.

Shuffling is the decisive case. Success falls from 0.965 to 0.215 in LIBERO’s original layouts and from 0.965 to 0.03 in the swapped ones. The original layouts are where a replayed trajectory would work best, and the images there are unchanged, yet the policy follows the positions it is given and fails. Its actions are computed from the object list, not recalled.

Removing an input shows the redundancy that the design intends. In the original layouts the policy loses nothing without the object list (0.965) and little without the wrist code (0.94). In the swapped layouts it falls to 0.69 and 0.78: both inputs carry the robustness to moved objects, and neither is a single point of failure, because the image embedding and the remaining input still locate the target in most episodes.

Graded displacement: LIBERO-PRO also ships ten LIBERO-Object suites in which the object to be picked is translated by 7 to 35 cm along one table axis (Appendix C).

Figure 5. RageVLA s1 on LIBERO-Object with one input changed at inference, 200 episodes per bar. The camera images are never changed. Shuffling gives each detected object the position of another one.

They measure robustness as a function of distance and are not part of the leaderboard. We run RageVLA s1 on them and, as a reference, the official π0.5\pi_{0.5} LIBERO checkpoint of openpi [3], through the same harness, episodes and success checks, with observations built as in openpi’s own LIBERO evaluation. In this harness π0.5\pi_{0.5} solves all 200 unperturbed episodes and 0.195 of the swap episodes, close to its leaderboard value of 0.17, so the harness reproduces its published behaviour.

Figure 6 shows the result. π0.5\pi_{0.5} loses success steadily with distance: 0.60 and 0.61 at 14 cm along xx and yy, 0.11 and 0.07 at 28 cm, and none at 35 cm. RageVLA s1 stays at 0.865 or above up to 28 cm along xx, and at 0.835 or above up to 21 cm along yy.

Figure 6. Success on LIBERO-Object when the object to be picked is displaced (LIBERO-PRO’s graded suites), 200 episodes per point, both models in the same harness. Solid: all episodes. Dashed: without the episodes that no policy can solve, because the object starts out of reach, the basket is moved off the table or the file cannot be loaded.

At the largest displacements part of the drop belongs to the benchmark files and not to the policy. At 35 cm along xx the object starts beyond the reach of the arm in five of the ten tasks. Along yy the files move the basket off the table in one task at 28 cm and in four at 35 cm, and one file at 35 cm cannot be loaded. These episodes cannot be solved by any policy, and we count them as failures in the solid curves. On the remaining episodes RageVLA s1 succeeds in 0.83 at 35 cm along xx, and in 0.55 and 0.52 at 28 and 35 cm along yy, while π0.5\pi_{0.5} remains at 0.00 at 35 cm (Table VII). Large displacements along yy are the weak case of RageVLA s1.

AxisShiftRageVLA s1 AllRageVLA s1 Feasibleπ0.5\pi_{0.5} Allπ0.5\pi_{0.5} FeasibleInfeasible
xx7 cm0.9800.9800.9100.9100
xx14 cm0.8800.8800.5950.5950
xx21 cm0.9050.9050.3950.3950
xx28 cm0.8650.8650.1050.1050
xx35 cm0.4450.8300.0000.000100
yy7 cm0.9950.9950.9200.9200
yy14 cm0.9400.9400.6100.6100
yy21 cm0.8350.8350.1850.1850
yy28 cm0.4950.5500.0700.07820
yy35 cm0.2600.5200.0000.000100
Unperturbed (Std)0.9651.000
Swap (Pos)0.9650.195

Table VII. LIBERO-OBJECT UNDER LIBERO-PRO’S GRADED OBJECT SHIFT, 200 EPISODES PER LEVEL. “FEASIBLE” EXCLUDES THE EPISODES THAT NO POLICY CAN SOLVE; THEIR NUMBER IS IN THE LAST COLUMN.

What this shows: The comparison is between two complete systems. π0.5\pi_{0.5} was trained on LIBERO’s original demonstrations and RageVLA s1 on our augmented data, so the gap is the effect of decoupled data and decoupled inputs together. What the two tests establish is the mechanism: the behaviour of RageVLA s1 is determined by the object positions it perceives, in the original layouts as well as in perturbed ones, and it holds as the displacement grows.

Standard LIBERO

On the unperturbed tasks of the same runs both RageVLA s1 and n1 succeed 0.96 of the time (Appendix, Table VI), slightly below the best reported LIBERO numbers. We read this as consistent with the training objective: the policy never sees a demonstration replayed in its original layout often enough to memorize it, since almost every training trajectory is laid out anew. The few points it gives up on standard LIBERO are the same property that lets it keep most of its success when objects move, the environment changes or the task changes.

GoalSpatialLIBERO-10ObjectMean
s10.960.960.950.980.96
n10.970.950.920.980.96

Table VI. RageVLA on the unperturbed tasks, from the same runs.

Comparison with an LLM coding agent

ASPIRE [25] reports high success on LIBERO-PRO’s Pos and Task perturbations (Figure 7), and close to full success with its strongest language model. The two approaches answer different questions. ASPIRE trains no network weights. For each task, a frontier language model writes a program against perception, geometry and motion-planning interfaces, debugs it on a set of practice layouts, and distills validated repairs into a reusable skill library. This synthesis has a per-task cost in agent time and API usage. In the authors’ model comparison on the ten LIBERO-PRO Goal tasks under the position perturbation, run without their evolutionary search, the two most successful models need roughly an hour of agent time and $12 to $18 per task, whereas RageVLA produces an action chunk in milliseconds. RageVLA is a learned closed-loop policy that runs a small network at every control step and needs no per-task program synthesis before deployment.

Figure 7. Pos and Task on the three suites reported by ASPIRE [25]. ASPIRE synthesizes a program per task with a frontier language model, and the other two are learned policies.

The protocols also differ, since ASPIRE evaluates on its own seeded layouts and uses depth and segmentation inputs, so the comparison is indicative rather than controlled. We therefore compare against VLA policies and report ASPIRE as a reference for what program synthesis with a frontier model can reach.

Limitations

RageVLA is trained on 130 simulated tasks, and all results are in simulation. It does not understand goals outside its training distribution, and we do not claim generality beyond LIBERO. The object head detects LIBERO’s object and fixture classes only.

The perception adapters are supervised with simulator labels and use the known camera calibration. On a real robot both would need a new source. No color or texture augmentation was used, which shows in the Obj column on LIBERO-10.

The augmented data and the perception interface were designed together and are evaluated together; we do not measure the contribution of either one alone. The tests of § 5-D change inputs at inference, cover one suite with 200 episodes per number, and compare a π0.5\pi_{0.5} checkpoint trained on the original demonstrations. Displacements of 28 cm and more along the yy axis still roughly halve the success of RageVLA s1.

On the evaluation side, baseline numbers in the main table come from the LIBERO-PRO leaderboard rather than from our own runs, and our Env column uses files generated with LIBERO-PRO’s released code (Appendix C). Results are from one training seed.

Conclusion

On LIBERO-PRO, what separates policies is not their parameter count but whether what they do is decoupled from where things were during training. RageVLA decouples twice: its training data varies layout, environment, surface and wording by general rules, and its inputs present object positions, wrist geometry and the instruction as separate signals. A 0.21-billion-parameter policy built this way, with an action engine of 35M parameters trained from scratch, achieves the best LIBERO-PRO result among VLAs and is the only one with substantial success under both position and task perturbations. Its actions follow the object positions it perceives, and it keeps most of its success when the target is displaced by up to 28 cm along one axis, where π0.5\pi_{0.5} has lost almost all of it. The next steps are appearance augmentation, training data with far more than 130 tasks, and a real robot.

Acknowledgment

This work uses DINO Materials (DINOv3 [11]) provided by Meta under the DINOv3 License.

Appendix A: Implementation details

Augmentation: About 70% of attempts use the uniform layout and 30% the smaller displacements. Per round each task receives about 400 successful trajectories: 240 in its own environment, 100 in other environments and 60 on other surfaces; a task without a valid transplant puts that quota into its own environment. Fixture yaw is uniform in ±180∘\pm180^\circ, object yaw follows each object’s rotational symmetry. Each generator attempt is re-simulated and at most three attempts per requested success are made. Trajectories with no sample at stride 8 are dropped. Six tasks have paraphrases that change the meaning, which we kept.

Agentview object head: The image is rendered at 2562256^2 and encoded by DINOv3 ViT-B/16 into a 16×1616 \times16 grid; features of layers 4, 8 and 12 enter a convolutional head with skip connections (1.7M parameters) whose 1×11 \times1 outputs are a heatmap over the object and fixture classes plus the gripper, a 2-D offset, the yaw as (sin⁡kψ,cos⁡kψ)(\sin k\psi,\cos k\psi) with kk set by the object’s symmetry, and the height. The loss is

Lagent=Lfocal+Loff+Lyaw+5Lz,(2)\mathcal{L}_{\mathrm{agent}}=\mathcal{L}_{\mathrm{focal}}+\mathcal{L}_{\mathrm{off}}+\mathcal{L}_{\mathrm{yaw}}+5\mathcal{L}_z, \tag*{(2)}

with the CenterNet focal loss [23], [27] and L1 losses at positive cells. Both adapters are trained on 432,592 frames, six phased frames per training trajectory, with batch 256.

Wrist encoder: The native 2562256^2 wrist image is encoded by the same DINOv3. The 768-dimensional features are projected to 256 dimensions and compressed into eight 32-dimensional tokens (2.6M parameters). The training-only decoder (2.1M parameters) predicts per cell a class and a log-depth and refines them into a 64×6464 \times64 log-depth map. The loss is

Lwrist=CEw(c^,c)+2∥log⁡d^16−log⁡d16∥1+2∥log⁡d^64−log⁡d64∥1,(3)\mathcal{L}_{\mathrm{wrist}}=\mathrm{CE}_w(\hat{c},c)+2\left\lVert\log\hat{d}_{16}-\log d_{16}\right\rVert_1 +2\left\lVert\log\hat{d}_{64}-\log d_{64}\right\rVert_1, \tag*{(3)}

where the class weights are the square root of inverse class frequency, clipped to [0.2,8][0.2,8].

TinyCLIP: We use TinyCLIP-ViT-40M/32 with a 39.4M image tower and a 44.3M text tower. For comparison, the hidden states that VLM-based policies condition on have 2,048 to 4,096 dimensions per token.

Flow matching: With a noise sample x0x_0, a target action chunk aa and a flow time τ∈(0,1)\tau\in(0,1), the model sees xτ=(1−τ)x0+τax_\tau=(1-\tau)x_0+\tau a and predicts the velocity a−x0a-x_0 under a squared error. At inference it integrates the velocity in ten Euler steps.

Action model: Table V lists the hyperparameters. Objects are embedded as a class embedding plus an MLP of Fourier features of the position, concatenated with yaw and score. Absent objects and dropped inputs are masked in the context encoder. The proprioceptive state is the end-effector position relative to the base, its axis-angle orientation and the two finger positions. On the held-out test set, RageVLA s1 has a sampled-chunk position error of 0.112 against 0.110 on validation.

RageVLA s1RageVLA n1
Width, heads384, 6512, 8
Denoiser blocks1220
Parameters35M100M
Peak learning rate6⋅10−46 \cdot10^{-4}3⋅10−43 \cdot10^{-4}
Warm-up steps5002,000
Epochs2318
Action chunk20 actions of 7 dimensions20 actions of 7 dimensions
Time conditioningadaLN-single [22]adaLN-single [22]
Flow timeτ∼sigmoid⁡(N(0,1))\tau\sim\operatorname{sigmoid}(\mathcal{N}(0,1))τ∼sigmoid⁡(N(0,1))\tau\sim\operatorname{sigmoid}(\mathcal{N}(0,1))
OptimizerAdamW, cosine decayAdamW, cosine decay
EMA0.9990.999
Input dropoutper input groupper input group
Sampling10 Euler steps, 10 of 20 actions executed10 Euler steps, 10 of 20 actions executed

Table V. Action model and training hyperparameters.

Appendix B: Additional results

Figure 8 shows the mean of every model per perturbation type, Figure 9 plots success on Pos and Task against model size, and Figure 10 gives the results of both models per suite. The policy samples fresh noise at every re-plan, so each cell is an estimate from 500 stochastic episodes with a 95% binomial half-width of at most 0.045.

Figure 8. Success per perturbation type, averaged over the four suites. Models that do not report Env are omitted there.

Figure 9. Success on Pos and Task against total parameter count, including vision encoders.

Figure 10. RageVLA per suite and column, including the unperturbed tasks (Std), for both model sizes.

Graded shift results: Table VII gives the numbers behind Figure 6. π0.5\pi_{0.5} is the official pi05_libero checkpoint [3]; its observations (images padded and resized to 224 pixels, end-effector position, axis-angle orientation and gripper state) and its re-planning interval of five steps follow openpi’s LIBERO evaluation. Both models use the first 20 released initial states of every task, the instruction stored in the task file, and the LIBERO’s step limit and success check for the suite. An episode is counted as infeasible, independently of the policy, if the object to be picked starts more than 0.855 m from the robot base, if the file places the basket off the table, or if the environment cannot be built from the file; the last case is one task at 35 cm along yy, whose placement region is empty.

On the swap suite RageVLA s1 scores 0.965 here and 0.94 in Table III; the checkpoint is the same, and the difference is 20 against 50 initial states per task.

Notes on the LIBERO-PRO protocol

Two details of the benchmark’s released code affect how its columns should be read. We state what we evaluated and what we could establish about the published entries.

Two position tests: LIBERO-PRO contains two different position perturbations, and only one of them is in the leaderboard. (i) Swap: the suites libero_{goal, spatial, 10, object}_swap, selected with use_swap, exchange the initial region of a task object with that of one object from a fixed candidate list (ood_spatial_relation.yaml). BDDL and initial-state files are released for all 40 tasks and the perturbation has no magnitude parameter. The reference evaluation flow maps the Pos setting to these suites; this is the Pos column of Table III, for prior models and for RageVLA s1. (ii) Graded shift: for LIBERO-Object only, the repository ships ten further suites (libero_object_temp_x0.1–x0.5 and y0.1–y0.5) in which the initial region of the object to be picked is translated along one table axis. In the released files one 0.1 step is 7 cm, so the five levels are 7, 14, 21, 28 and 35 cm (Table VII); objects that would overlap the new region are moved off the table, and at the two largest yy levels this includes the basket in one and four of the ten tasks. These suites underlie the displacement-sensitivity curve of [5] (Figure 4 of its v2, in undefined “units”: OpenVLA and π0\pi_0 fail beyond 0.2, π0.5\pi_{0.5} beyond 0.4). No levels exist for the other three suites and no leaderboard entry reports them. A later dataset release adds an initial position-and-yaw case for all 40 tasks as part of a seven-case robustness extension; it is not in the leaderboard either. All Pos numbers in this paper refer to (i); the suites of (ii) are evaluated separately in Sec. V-D.

Swap (leaderboard Pos)Graded shift
Suitesall four (40 tasks)LIBERO-Object (10 tasks)
What moves2–4 objects exchange regionspicked object only
Magnitudefixed by the object pair7, 14, 21, 28, 35 cm; xx or yy
Reported inleaderboard, Table IIIFigure 4 of [5]
Used hereTable IIISec. V-D

Table VIII. THE TWO POSITION TESTS IN LIBERO-PRO’S RELEASE.

Environment files: LIBERO-PRO’s Env perturbation (use_environment, ood_environment.yaml) rewrites the scene of a task so that it takes place on the living-room table; five LIBERO-10 tasks are already there and stay unchanged. The release contains no task or initial-state files for this perturbation. We produced them with the release’s own perturbation code, with only this flag set, and drew 50 initial states per task with the release’s initial-state procedure, without filtering.

Which instruction the policy receives: The Sem and Task perturbations rewrite the (:language) field of the BDDL (Task also rewrites the goal). The benchmark’s task loader derives the instruction from the BDDL file name, which the perturbations leave unchanged, and the usual runners pass that string to the policy [28, 29]. With such a runner the policy is given the original instruction in both columns: Sem then equals the unperturbed suite, and Task checks a new goal while stating the old one. The maintainers recommend reading the instruction from the environment instead [28], and that is what we do, so our Sem column tests the paraphrases and our Task column states the new goal.

We cannot determine which instruction source produced each leaderboard entry. An independent re-evaluation of the public π0.5\pi_{0.5} LIBERO checkpoint [29] comes close to the leaderboard total with file-name instructions (0.54 against 0.53) and reports that, with BDDL instructions, Sem is unchanged (0.96) while Task rises from 0.04 to 0.26 (Table IX). Against that figure RageVLA s1’s Task advantage is 0.52 to 0.26 rather than 0.52 to 0.01; it is slightly lower on Goal and Spatial. The same source reports 0.27 for π0.5\pi_{0.5} on Pos, averaged over suites (leaderboard: 0.21; RageVLA s1: 0.59). These figures come from an issue thread, not from a peer-reviewed source, and we did not re-run them; we include them because they bear directly on how the Task column is read.

GoalSpatialLIBERO-10ObjectMean
π0.5\pi_{0.5}, leaderboard0.000.010.010.010.01
π0.5\pi_{0.5}, BDDL instruction [29]0.220.530.180.100.26
RageVLA s1, BDDL instruction0.200.510.420.960.52

Table IX. Task perturbation by instruction source. π0.5\pi_{0.5} rows: leaderboard [26] and an independent re-evaluation with the BDDL instruction [29] (100 episodes per task).

References

References

  1. [1]B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone, “LIBERO: Benchmarking knowledge transfer for lifelong robot learning,” in Advances in Neural Information Processing Systems (Datasets and Benchmarks), 2023.arxiv.org/abs/2306.03310
  2. [2]M. J. Kim, C. Finn, and P. Liang, “Fine-tuning vision-language-action models: Optimizing speed and success,” arXiv preprint arXiv:2502.19645, 2025.arxiv.org/abs/2502.19645
  3. [3]Physical Intelligence, “openpi: LIBERO results for π₀.₅,” https://github.com/Physical-Intelligence/openpi/blob/main/examples/libero/README.md, 2025.
  4. [4]J. Zheng et al., “X-VLA: Soft-prompted transformer as scalable cross-embodiment vision-language-action model,” arXiv preprint arXiv:2510.10274, 2025.arxiv.org/abs/2510.10274
  5. [5]X. Zhou, Y. Xu, G. Tie, Y. Chen, G. Zhang, D. Chu, P. Zhou, and L. Sun, “LIBERO-PRO: Towards robust and fair evaluation of vision-language-action models beyond memorization,” arXiv preprint arXiv:2510.03827, 2025.arxiv.org/abs/2510.03827
  6. [6]M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi et al., “OpenVLA: An open-source vision-language-action model,” arXiv preprint arXiv:2406.09246, 2024.arxiv.org/abs/2406.09246
  7. [7]K. Black, N. Brown, D. Driess et al., “π₀: A vision-language-action flow model for general robot control,” arXiv preprint arXiv:2410.24164, 2024.arxiv.org/abs/2410.24164
  8. [8]J. Lee et al., “MolmoAct: Action reasoning models that can reason in space,” arXiv preprint arXiv:2508.07917, 2025.arxiv.org/abs/2508.07917
  9. [9]C.-Y. Hung et al., “NORA: A small open-sourced generalist vision language action model for embodied tasks,” arXiv preprint arXiv:2504.19854, 2025.arxiv.org/abs/2504.19854
  10. [10]Physical Intelligence, K. Black et al., “π₀.₅: a vision-language-action model with open-world generalization,” arXiv preprint arXiv:2504.16054, 2025.arxiv.org/abs/2504.16054
  11. [11]O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab et al., “DINOv3,” arXiv preprint arXiv:2508.10104, 2025.arxiv.org/abs/2508.10104
  12. [12]K. Wu, H. Peng, Z. Zhou et al., “TinyCLIP: CLIP distillation via affinity mimicking and weight inheritance,” in IEEE/CVF International Conference on Computer Vision, 2023.arxiv.org/abs/2309.12314
  13. [13]M. Shukor et al., “SmolVLA: A vision-language-action model for affordable and efficient robotics,” arXiv preprint arXiv:2506.01844, 2025.arxiv.org/abs/2506.01844
  14. [14]D. Ghosh, H. R. Walke, K. Pertsch et al., “Octo: An open-source generalist robot policy,” in Robotics: Science and Systems, 2024.arxiv.org/abs/2405.12213
  15. [15]A. Radford, J. W. Kim, C. Hallacy et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning, 2021.arxiv.org/abs/2103.00020
  16. [16]S. Fei et al., “LIBERO-Plus: In-depth robustness analysis of vision-language-action models,” arXiv preprint arXiv:2510.13626, 2025.arxiv.org/abs/2510.13626
  17. [17]A. Mandlekar, S. Nasiriany, B. Wen et al., “MimicGen: A data generation system for scalable robot learning using human demonstrations,” in Conference on Robot Learning, 2023.arxiv.org/abs/2310.17596
  18. [18]C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Robotics: Science and Systems, 2023.
  19. [19]W. Peebles and S. Xie, “Scalable diffusion models with transformers,” in IEEE/CVF International Conference on Computer Vision, 2023.arxiv.org/abs/2212.09748
  20. [20]Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” in International Conference on Learning Representations, 2023.arxiv.org/abs/2210.02747
  21. [21]X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning to generate and transfer data with rectified flow,” in International Conference on Learning Representations, 2023.arxiv.org/abs/2209.03003
  22. [22]J. Chen, J. Yu, C. Ge et al., “PixArt-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis,” in International Conference on Learning Representations, 2024.arxiv.org/abs/2310.00426
  23. [23]X. Zhou, D. Wang, and P. Krähenbühl, “Objects as points,” arXiv preprint arXiv:1904.07850, 2019.arxiv.org/abs/1904.07850
  24. [24]J. Liang, W. Huang, F. Xia et al., “Code as policies: Language model programs for embodied control,” in IEEE International Conference on Robotics and Automation, 2023.arxiv.org/abs/2209.07753
  25. [25]R. Lu, Y. Wu, E. Kou, H. Lin, L. Fu, W. Xiao, A. Mandlekar, Y. Xu, G. Shi, K. Goldberg, A. Chen, M. Chowdhury, J. Kautz, Y. Zhu, L. Fan, and G. Wang, “ASPIRE: Agentic /Skills discovery for robotics,” arXiv preprint arXiv:2607.00272, 2026, v2, Oct. 2026.arxiv.org/abs/2607.00272
  26. [26]X. Zhou, Y. Xu, G. Tie, Y. Chen, G. Zhang, D. Chu, P. Zhou, and L. Sun, “LIBERO-PRO model leaderboard,” https://github.com/Zxy-MLlab/LIBERO-PRO, 2025, accessed Nov. 2026.
  27. [27]T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in IEEE International Conference on Computer Vision, 2017.
  28. [28]LIBERO-PRO community, “Is there a bug related to the ‘TASK’ perturbation? (LIBERO-PRO issue #14, with maintainer reply),” https://github.com/Zxy-MLlab/LIBERO-PRO/issues/14, 2025, accessed Nov. 2026.
  29. [29]———, “Semantic (_lan) perturbation never reaches the policy (LIBERO-PRO issue #27),” https://github.com/Zxy-MLlab/LIBERO-PRO/issues/27, 2026, accessed Nov. 2026.

Paper details

vlnc:W_mbtgcceb
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents