betaFeedback

WorksComputer Science

PhyDEX: A Benchmark for Physical Reasoning in Vision-Language-Action Models via Dynamic Extrinsic Dexterity

Abstract

Vision-Language-Action (VLA) models have advanced rapidly in language-conditioned robotic manipulation, yet existing benchmarks mainly evaluate gripper–object contact and largely ignore how objects interact with the environment through gravity, friction, inertia, and mass distribution. As a result, it remains unclear whether current policies truly understand physics or merely interpolate from their training data. We address this gap through the lens of dynamic extrinsic dexterity and introduce PhyDEX, a throwing benchmark in which a robot must infer an object's mass, center of mass, and friction to send it to a destination beyond its kinematic workspace. PhyDEX contains 35K trajectories (143 hours) across nine tasks and 100 scenes, covering uniform, irregular, and dynamic-mass objects with static, multiple, and moving destinations. All data are generated by a scalable, fully automatic pipeline that combines grasp generation, motion planning, and closed-loop throw optimization under randomized physical properties. We evaluate three representative VLAs, GR00T N1.7, DynamicVLA, and ForceVLA, and find that all achieve success rates below 3%. Our analysis reveals two key bottlenecks: limited robustness of pretrained models, which causes most failures before the throw, and physical blindness, as providing ground-truth physical properties roughly doubles success. Experiments on a humanoid robot and in the real world show the same failure patterns. We hope PhyDEX will drive the development of physically grounded robot policies.

Project Page

Overview of PhyDEX

Figure 1. Overview of PhyDEX. Left: a robot arm throws an object into a target destination, which requires reasoning about the object’s mass, center of mass, and friction. Right: PhyDEX contains 35K trajectories across 9 tasks and 100 scenes, covering uniform, irregular, and dynamic objects (top). Current VLA policies all reach overall success rates below 3% (bottom left), while injecting ground-truth physical properties substantially improves performance when throwing uniform objects to a static single destination (bottom right).

Introduction

Physical understanding has widely recognized as a cornerstone of language-conditioned robotic manipulation systems [4, 15, 41, 56]. A broad spectrum of methods have emerged, ranging from hybrid frameworks [28, 53] to unified systems [19, 41, 63], from vision-language architectures [2, 20, 61] to multi-sensory models [38, 57], and from static methods [2, 15, 20, 21, 22, 43, 44, 61] to dynamic paradigm [17, 27, 31, 49, 62].

Despite this great progress, recent works predominantly focus on the intrinsic gripper–object contact, whether in standard manipulation tasks such as Pick-and-Place, Re-orientation, and Opening & Closing Articulated Objects [7, 30, 60] [37], or in contact-rich tasks such as Insertion, Fragile Grasping, and Wiping [11, 12, 16, 18, 26, 32, 38, 45, 57]. However, they largely overlook how the manipulators interacts with its surrounding environment (e.g., gravity, friction, inertia, aerodynamics, mass distribution, etc.), namely the newtonian mechanics underlying everyday physical interactions, which substantially limits their ability to reflect genuine physical intelligence.

To fill this gap, we recast language-conditioned robotic manipulation through the lens of dynamic extrinsic dexterity [8], focusing on throwing-based object manipulation. In this setting, the manipulator must estimate the object’s physical properties, such as its mass, center of mass (CoM), and friction, and plan the forces and contacts needed to send the object to a target location outside its own kinematic workspace.

We introduce PhyDEX, a high-fidelity synthetic manipulation dataset comprising 35k trajectories across 9 throwing tasks and 100 scenes. It covers rigid, irregular, and dynamic-mass objects, as well as diverse destination types, with all physical properties randomized. PhyDEX is generated by a fully decoupled, autonomous simulation pipeline that separates asset specification, skill policies, task composition, and rendering. Each task is built by retrieving embeddings, scenes, and objects from an asset library and composing scripted skill policies into complete behaviors. This compositional design enables automatic generation of object-level annotations, physical randomization, and collision-aware motion plans, supporting systematic VLA validation through fine-tuning and closed-loop evaluation.

To evaluate the physical understanding of current policies, we select three representative policies: GR00T N1.7 [3] as a general-purpose policy, DynamicVLA [54] as a high-frequency policy, and ForceVLA [57] as a multi-sensory VLA. Our experiments show that current policies are not yet reliable for tasks requiring physical understanding, with average success rates below 10%. We further identify generalization and the probing of object physical properties as the main bottlenecks of current systems. To confirm that this trend holds beyond a single setup, we additionally conduct evaluations across multiple platforms, including a humanoid robot and real-world experiments.

In summary, our contributions are threefold:

  • We propose PhyDEX, a diverse and comprehensive dataset of nine throwing tasks for studying physical understanding in robot manipulation.

  • We introduce a scalable construction pipeline that generates diverse, end-to-end data for throwing tasks.

  • We conduct extensive evaluations across multiple embodiments, including robot arms and humanoids, in both simulation and real-world experiments. Our results show that data scarcity and physical blindness are the main bottlenecks keeping current policies from performing reliably on physical understanding tasks.

Related Works

Language-conditioned Robot Manipulation

Recently, language-conditioned robot manipulation systems have attracted much attention and achieved great progress, owing to their generalizability, scalability, and potential for deployment in the physical world. Related methods can be roughly categorized into two branches: autoregressive-based approaches [5, 13, 19, 24, 25, 40, 50] and diffusion-based approaches [2, 15, 20, 23, 41, 51, 56]. While these methods have significantly advanced overall performance, their physical knowledge is interpolated from the training data distribution and its reliability remains unverified.

Language-conditioned Dynamic Manipulation

Our work is closely related to dynamic manipulation, which demands policies that are fast, reactive, and precise. Modeling such dynamics is challenging, as it relies on physical properties of both the objects and the manipulator that are difficult to estimate [35]. In the language-conditioned setting, most recent work tracks a target object under static and predictable conditions [27, 62], adopts asynchronous inference to hide latency [17, 31, 49], or extends control to more precise 6-DoF manipulation [29, 54]. While effective, these methods are evaluated primarily on kinematic accuracy. They optimize for trajectory matching and leave the understanding of physical properties implicit, so a failure cannot be attributed to kinematic error or to a misestimated physical property. In this work, we design a beyond-reach manipulation benchmark in which the object must be moved to a target location outside the kinematic range of the manipulator, making the understanding of physical properties a decisive factor for success.

Physical Property Understanding

To throw an object accurately to a target, a robot must first estimate the object’s physical properties. Studies in cognitive science [36, 59] suggest that humans infer such properties not only from visual appearance but also from material experience. Existing approaches to physical property estimation fall into two main lines of work: those that encode an object’s properties implicitly [1, 42, 52, 59] and those that predict them explicitly [33, 46, 48, 58]. While effective, these methods mostly target object-centric settings in general domains, and robotic scenarios remain underexplored. A few works in robot learning [33, 55] have tackled this problem, but they are limited to simple pick-and-place or contact-rich manipulation tasks. These tasks are not demanding enough to properly evaluate a model’s physical understanding. To fill this gap, we explicitly randomize object physical properties, such as mass, CoM, and friction, to evaluate a model’s ability to accurately estimate object properties and reason about physics.

PhyDEX

In this section, we present the task taxonomy, detailed dataset statistics, and the data creation pipeline, which are illustrated in Figure 2 and Figure 3, respectively.

Task taxonomy and statistics of PhyDEX

Figure 2. Task taxonomy and statistics of PhyDEX. Left: distribution of episodes across three object types (inner ring) and three destination types (outer ring). Right: example scenes for each of the nine object–destination combinations. boxes mark the target object and boxes mark the destination.

Task Taxonomy & Statistics

PhyDEX comprises 9 tasks, 35,357 episodes, and 143.2 hours of data recorded at 10 Hz. It is designed to evaluate how well policies estimate the physical properties of objects from different perspectives.

Tasks. In PhyDEX, policies perform a throwing task at three levels of object difficulty: (1) Uniform Object (31.8%): objects with uniformly distributed mass and a centered center of mass (CoM); (2) Irregular Object (39.2%): objects with non-uniform mass distribution and a CoM offset toward one end; and (3) Dynamic Object (29.0%): objects whose mass and CoM change during the interaction. In addition, to test generalization and timing, we use three destination settings: static single destination (31.76%), static multiple destinations (29.77%), and moving single destination (38.47%).

Assets. We provide 107 curated object assets, sourced from RoboCasa [37], RoboTwin 2.0 [7], and Objaverse-XL [9]: 20 destination objects, 46 uniform objects, 37 irregular objects, and 4 dynamic objects. For environments, PhyDEX includes 100 RoboCasa kitchen scenes spanning diverse layouts and styles.

Benchmark Construction Pipeline

In this section, we describe the data synthesis pipeline of PhyDEX. We first construct the environment by specifying the task, target object, destination, and scene. We then apply domain randomization to diversify these environments. Finally, we generate approaching and throwing trajectories and define the procedures for storing the resulting data.

Environment Construction

As illustrated in Figure 3, given a task description template, we first retrieve the task-relevant assets from the asset library: the target object, the destination object, and the scene. We use a UR5e arm equipped with a Robotiq 2F-85 gripper as the robot embodiment, represented as a stable USD asset. The scene assets are room-scale environments with diverse layouts and styles, each annotated with fine-grained manipulation areas to support natural, context-aware task construction. Our object library spans three mass categories (uniform, irregular, and dynamic) and covers a range of physical properties, sizes, and layouts. Objects are sampled equally across categories to keep the benchmark balanced. We also place distractor objects around the target object to encourage policy generalization. Destination objects likewise vary in motion direction, size, and layout. All functional annotations are managed through a unified interface, which makes asset configuration straightforward while keeping physical fidelity high.

Overview of the PhyDEX data generation pipeline

Figure 3. Overview of the PhyDEX data generation pipeline. (a) Environment construction: given a task template, we retrieve a target object, a destination, and a scene from the asset library to create a new task. (b) Domain randomization: we diversify each task visually (camera views, lighting, scene layout), in its trajectories (grasp pose, object pose, robot state), and in its physical properties (mass, CoM, friction). (c) Trajectory generation: GraspGen-X proposes grasp poses, cuRobo plans a collision-free approach, and an SLSQP-based optimizer computes the throwing trajectory q(t)q(t) from the initial joint state (q0,q˙0)(q_0, \dot{q}_0), the object properties (m,CoM,μ)(m, \mathrm{CoM}, \mu), and the target p∗p^*, refining it in a closed loop until the throw succeeds. (d) Data storage: each successful episode is saved in LeRobot format with multi-view RGB, force/torque signals, and actions.

Domain Randomization

To enrich the diversity of PhyDEX, we apply three types of randomization.

Visual diversity. We perturb the poses of the two side cameras and the wrist camera by up to ±5∘\pm5^\circ in rotation and ±5\pm5 cm in translation. We also build a library of 100 environment maps, each with randomized light temperature and intensity, to simulate diverse natural illumination conditions.

Trajectory diversity. We introduce additional randomness in grasp pose selection output from GraspGen-X [14]. For instance, our autonomous grasp generation pipeline produces millions of grasp candidates. We retain the top 64 high-confidence grasps that are geometrically diverse, requiring each selected grasp to differ from all previously selected ones by more than 1 cm in position or 20∘20^\circ in orientation, and randomly sample the final grasp pose from this set. We further randomize the object position by ±10\pm10 cm along the xx- and yy-axes, the object yaw by ±180∘\pm180^\circ, and the initial robot joint positions by ±0.09\pm0.09 rad.

Physical diversity. To evaluate whether policies can infer physical properties, we randomize each object’s physical parameters independently of its appearance, so visually identical objects may behave differently. Specifically, we scale the object mass by a factor in [0.5,2.0][0.5, 2.0], offset the center of mass by up to ±15\pm15 mm along each axis, and scale the friction coefficient by a factor in [0.8,1.2][0.8, 1.2].

Trajectory Generation

To produce plausible and successful trajectories, we introduce a closed-loop optimization procedure that executes each candidate trajectory end-to-end and iteratively refines it based on the resulting feedback. The procedure consists of three main steps.

Grasp generation. As described above, we pre-compute 64 grasp candidates for each object. Given the object’s placement pose in the scene, we first discard grasps that would collide with surrounding geometry, such as the supporting surface or nearby objects. The remaining feasible grasps are then ranked by confidence score and geometric diversity, and one is sampled from the top-ranked candidates and passed to the motion planning stage.

Motion planning. We then provide cuRobo [47] with the scene point cloud extracted from the simulator and the target grasp pose, and it generates a collision-free trajectory that approaches the object, grasps it and lifts it.

Throw optimization. After lifting, a throw planner takes the measured joint state, grasp offset, and the object’s mass, CoM, and inertia. It assumes a rigid grasp until release and drag-free ballistic flight afterward. It jointly optimizes the release joint configuration qrq_\mathrm{r}, the release joint velocities q˙r\dot{q}_\mathrm{r}, the throw duration TT, and the flight time τ\tau. Each joint follows a quintic polynomial set by the measured start state, the release state (qr,q˙r)(q_\mathrm{r},\dot{q}_\mathrm{r}), and zero acceleration at both ends, so these variables define the whole motion. Forward kinematics and the Jacobian give the object’s release position and velocity:

pr=f(qr),vr=J(qr)q˙r,(1)\mathbf{p}_\mathrm{r}=f(\mathbf{q}_\mathrm{r}),\qquad\mathbf{v}_\mathrm{r}=J(\mathbf{q}_\mathrm{r})\dot{\mathbf{q}}_\mathrm{r}, \tag*{(1)}

which must satisfy the ballistic interception constraint

pr+τvr+12gτ2=pdst(T+τ)+(δx,δy,0)T,(2)\mathbf{p}_\mathrm{r}+\tau\mathbf{v}_\mathrm{r}+\frac{1}{2}\mathbf{g}\tau^2=\mathbf{p}_\mathrm{dst}(T+\tau)+(\delta_x,\delta_y,0)^\mathsf{T}, \tag*{(2)}

where pdst(T+τ)\mathbf{p}_\mathrm{dst}(T+\tau) is the destination opening at arrival time, so moving targets are handled, and δ\delta is a horizontal aim offset that starts at zero. Starting from an IK-based initial guess, SLSQP minimizes joint velocities, accelerations, and duration. Along the trajectory it enforces joint, velocity, torque, and table-clearance limits, plus a descending arrival into the opening. A plan is accepted only if it satisfies Eq. (2) within 3 mm and all constraints at every physics step.

The plan runs in simulation, with the gripper opening timed to the control clock. Effects the planner does not fully model cause the executed release to differ from the plan: tracking error under the payload’s mass and CoM, friction-dependent slip in the grasp, and release delay. After a miss, we predict the horizontal miss ϵk\epsilon_k from the measured release state and update the aim offset as δk+1=δk−0.5ϵk\delta_{k+1}=\delta_k-0.5\epsilon_k, then re-plan. If no feasible throw remains, a new grasp is sampled. An episode is kept only when the simulator reports success.

Data Storage

Each successful trajectory is stored in the standard LeRobot format [6]. A record contains the language instruction, multi-view RGB images with their camera parameters, the robot’s proprioceptive states, and the action labels. It also holds a 2 s window of wrist force/torque (F/T) readings, which supports training multi-sensory VLA models such as ForceVLA [57]. Other modalities, such as depth maps, grounding annotations and bounding boxes, can be added to the recording with a simple configuration option. For each trajectory we also save an HDF5 file holding the scene metadata and a replayable copy of the trajectory, so the data can be reused and extended later.

Experiments

Experimental Setup

We evaluate three representative policies, each chosen to test a different hypothesis. First, we evaluate GR00T N1.7 [3], a general-purpose VLA, to examine whether a model that claims broad physical intelligence can actually perform this task. Second, we evaluate DynamicVLA [54] to test whether a highly reactive policy performs well on this dynamic manipulation problem. Finally, we include ForceVLA [57] to assess whether simply adding a force sensor to the policy is enough to give the model physical intelligence. Following standard robot manipulation benchmarks [7, 30] [37], we report success rate (SRSR) as our evaluation metric. To ensure reproducible and reliable results, we fine-tune a separate model for each task and evaluate each policy over 50 rollouts.

Implementation Details

We build our data generation and rendering pipeline on NVIDIA Isaac Sim [39], a GPU-accelerated simulator that provides photorealistic rendering and accurate physics. Data generation runs on 8 NVIDIA RTX A6000 GPUs with 128 parallel environments. The physics simulation runs at 120 Hz, and data are recorded at 10 Hz. Each RGB camera captures images at a resolution of 352×256352 \times256. The robot state consists of the 6 joint angles and the gripper open/close state. Wrist force/torque readings are stored over a 2 s window. We fine-tune every policy on 16 NVIDIA A100 (40GB) GPUs using the default hyperparameters released by its authors.

Overall Benchmark Results

As shown in Table 1, all three policies perform poorly on our benchmark, with overall success rates below 10%. Most successes occur on objects with uniform mass and static destinations, and no policy completes a single task with a dynamic destination. When objects have more challenging physical properties, such as non-uniform or dynamic mass, the policies rarely complete a full trajectory. The few successes by DynamicVLA and ForceVLA are likely due to chance rather than real competence. We attribute these failures to two main factors. (1) Limited robustness of pretrained models: Current policies struggle to generalize to our highly diverse data, and only about 21.24% of trajectories include a successful grasp of the object. This is most pronounced for DynamicVLA, which builds on the lightweight SmolLM2 [34] backbone to reach 25 Hz inference. However, this backbone is too weak for out-of-

DestinationGR00T N1.7 [2]DynamicVLA [54]ForceVLA [57]
UniformSingle6%0%10%
Multiple2%0%10%
Dynamic0%0%0%
IrregularSingle0%0%0%
Multiple0%2%0%
Dynamic0%0%0%
DynamicSingle0%0%0%
Multiple0%0%2%
Dynamic0%0%0%
Overall Average0.9%0.2%2.4%

Table 1. Performance comparison of three baseline VLA models on PhyDEX under full diversity (α=1\alpha= 1).

Humanoid scenes

Figure 3 (PDF p. 6). Top: humanoid scenes.

distribution tasks, and DynamicVLA grasps the object in only 3.57% of its trials. The other policies achieve higher grasping rates but still struggle under our domain randomization, which we discuss further in the following sections. (2) Physical blindness in policy design: Although an average of 21.24% of trajectories reach the throwing stage, the overall success rate is only about 1.2%, meaning that just 5.6% of throws succeed. This result shows that current models have not yet achieved a reliable understanding of physics, since these physical properties cannot be directly observed through images, language instructions, or additional sensors. We therefore believe that future policies require explicit mechanisms for probing and inferring such physical properties.

Further Analysis

Effect of Benchmark Diversity. As shown in Table 1, most policies achieve near-zero success rates on PhyDEX. We attribute this mainly to the high diversity of our benchmark. Existing benchmarks either apply limited domain randomization, as in LIBERO [30], or evaluate on a small subset of their data. For example, RoboSPA [10] provides 527K trajectories but evaluates on only 250 trajectories per task. Such settings can produce non-zero success rates that hide how poorly pretrained policies generalize. To test this explanation, we introduce a diversity score α∈[0,1]\alpha\in[0,1] that controls the strength of domain randomization. Each randomized parameter is normally sampled from a range [a,b][a,b], and we scale the width of this range by α\alpha around its nominal value. Thus α=1\alpha= 1 recovers the full PhyDEX setting and α=0\alpha= 0 disables randomization entirely. As shown in Table 2, success rates rise steadily as diversity decreases. ForceVLA improves the most, from 3.3% at α=1\alpha= 1 to 11.3% at α=0.5\alpha= 0.5 and 35.3% at α=0\alpha= 0, a more than tenfold gain. GR00T N1.7 follows the same trend, rising from 2% to 4.7% and then 9.3%, while DynamicVLA stays below 4% in every setting, which again points to the limited capacity of its lightweight backbone. These results confirm that the high diversity of PhyDEX is a major reason for the nearzero scores, and that current pretrained policies are far less robust than their results on less diverse benchmarks suggest. However, even without any randomization, no policy exceeds 44% on any object type, and performance remains lowest on dynamic objects, where GR00T N1.7 still fails completely. This shows that reducing diversity alone does not solve the task, and that inferring physical properties remains a core challenge.

α\alphaObjectGR00T N1.7 [2]DynamicVLA [54]ForceVLA [57]
1Uniform6%0%10%
1Irregular0%0%0%
1Dynamic0%0%0%
1Average2%0%3.3%
0.5Uniform6%2%16%
0.5Irregular6%4%10%
0.5Dynamic2%0%8%
0.5Average4.7%2%11.3%
0Uniform18%6%44%
0Irregular10%4%40%
0Dynamic0%2%22%
0Average9.3%4%35.3%

Table 2. Effect of benchmark diversity. Success rates on static single-destination tasks under different diversity scores α\alpha.

Effect of Physical Properties. As discussed above, we hypothesize that current policies need a better way to infer the physical properties of objects in order to complete these tasks. To test this hypothesis, we extract the ground-truth physical properties of each object and give them to the policy by concatenating them with the robot proprioception embedding. As shown in Figure 4, adding this physical prior clearly improves performance, raising SR\mathit{SR} by 12.5% for GR00T N1.7 and 24% for ForceVLA. DynamicVLA’s performance stays unchanged, which we attribute to its limited pretraining. These results support our hypothesis. We believe this finding gives the community evidence to motivate new designs, such as combining VLAs with reinforcement learning, injecting VLM priors, or adding richer sensor information.

Bar chart showing success rates for DynamicVLA, GR00T N1.7, and ForceVLA with and without physics ground truth

Figure 4. Effect of ground-truth physical properties. Success rates with and without access to the object’s ground-truth physical properties.

Cross-Embodiment Evaluation. To check whether these failures are specific to our UR5e arm, we build a small cross-embodiment dataset on a Unitree humanoid using the same generation pipeline. It contains 2,400 episodes with static destinations: 640 for uniform objects, 1,280 for irregular objects, and 480 for dynamic objects. We evaluate each policy over 50 rollouts. As shown in Table 3, DynamicVLA fails on nearly every task. Among the vision-only policies, GR00T N1.7 performs best, but its average success rate is only 6%. ForceVLA, which also uses force/torque input, reaches 8.7%, and thanks to the simpler scene setup it also succeeds on some dynamic-object trials (6%). Overall, the failure patterns on the humanoid closely match those on PhyDEX, which suggests these failures are shared across embodiments and are not caused by a single robot platform.

ObjectGR00T N1.7 [2]DynamicVLA [54]ForceVLA [57]
Uniform10%2%14%
Irregular8%0%6%
Dynamic0%0%6%
Average6%0.7%8.7%

Table 3. Cross-embodiment evaluation on a humanoid robot. Top: humanoid scenes. Bottom: success rates by object type.

Real-World Evaluation. To test whether our findings carry over to the physical world, we build a real-world setup that mirrors our simulation tasks (Figure 5). It consists of a UR10e arm with a Robotiq 2F-85 gripper and a wrist-mounted F/T sensor. A base camera (RealSense D455) and a wrist camera (RealSense D405) provide visual observations. Note that this arm differs from the UR5e used in simulation. For the target objects, we design three 3D-printed cubes that look identical but have different physical properties (Figure 6): a standard cube with uniform mass, a cube with a metal insert near one corner that shifts its center of mass, and a hollow lightweight cube that we pair with low-friction tape on the gripper fingers. Because the cubes look identical, a policy cannot tell them apart from visual cues and must instead infer their physical properties through interaction. In each trial, the robot must throw the cube into a basket placed outside its workspace. We collect demonstrations following the same principles as the PhyDEX generation pipeline (see Appendix B), with 300 demonstrations per object, fine-tune each policy on this data, and evaluate it over 50 rollouts per object. As shown in Table 4, DynamicVLA fails on every trial, and GR00T N1.7 reaches an average success rate of only 10.7%. Its performance drops as the physics deviates from the standard cube: 16% on the uniform cube, 12% on the shifted-CoM cube, and only 4% on the lightweight low-friction cube. In contrast, ForceVLA reaches 51.3% on average. We attribute this gap to its force/torque input, which directly measures the object’s weight and inertia once it is grasped. This finding agrees with our ground-truth physics experiment: when a policy has access to physical information, its performance improves substantially. Even so, ForceVLA also degrades on the lightweight low-friction cube (58% → 40%), where reduced friction causes the object to slip during the throw and a lighter mass makes its trajectory more sensitive to the release velocity. Overall, these results confirm that physical blindness is a key limitation of vision-only policies in the real world, and that physical sensing is a promising direction for addressing it.

Real-world experimental setup with a UR10e arm, Robotiq 2F-85 gripper, base camera, wrist camera, and F/T sensor

Figure 5. Real-world experimental setup. A UR10e arm with a Robotiq 2F-85 gripper and a wrist-mounted F/T sensor, observed by a base camera (RealSense D455) and a wrist camera (RealSense D405).

Three 3D-printed cubes with identical appearance and different physical properties

Figure 6. Real-world test objects. Three 3D-printed cubes with the same appearance but different physical properties. Left: a standard cube with uniform mass. Middle: a cube with a metal insert near one corner, which shifts its center of mass. Right: a hollow, lightweight cube, used with low-friction tape on the gripper fingers to reduce friction.

ObjectGR00T N1.7 [2]DynamicVLA [54]ForceVLA [57]
Cube 116%0%58%
Cube 212%0%56%
Cube 34%0%40%
Average10.7%0%51.3%

Table 4. Real-worl experiment. Top: real-world setup Bottom: success rates by destination type.

Failure Analysis. To understand where current policies break down, we sort every failed rollout into one of four failure modes, ordered by manipulation stage: ineffective reaching, where the arm never makes meaningful progress toward the target object; approaching error, where the arm reaches the object but arrives in a pose that does not allow a grasp; grasping error, where the gripper fails to secure the object or drops it before release; and throwing error, where the object is grasped and released but misses the destination. As shown in Figure 7, about 89% of all failures occur before the throw begins, and grasping error is the most common mode (38.5%), followed by ineffective reaching (28.8%) and approaching error (22.0%). These pre-throw failures are consistent with the limited robustness discussed in § 4.3: under our domain randomization of object poses, camera views, and scene layouts, policies often cannot even complete the grasp. The breakdown also differs across policies. DynamicVLA fails mainly in the earliest stages, with ineffective reaching and approaching errors together accounting for 64.2% of its failures, which matches its weak SmolLM2 backbone. Its throwing error is only 3.4% simply because it rarely gets far enough to throw. GR00T N1.7 and ForceVLA reach the object more reliably but still fail mostly at grasping (44.8% and 38.4%). Throwing errors make up a smaller share of failures (10.6% overall), but this reflects how few rollouts reach the throwing stage, not throwing competence: once a throw is attempted, it rarely succeeds. Together, these results suggest that improving physical understanding alone is not enough, and future policies must also become more robust in the reaching and grasping stages that come before it.

Failure analysis pie chart and stacked bar chart

Figure 7. Failure analysis. Breakdown of failure modes across all evaluated policies.

Conclusion & Discussion

In this work, we introduced PhyDEX, a benchmark that evaluates physical reasoning in VLA models through dynamic extrinsic dexterity. Unlike existing benchmarks that focus on gripper–object contact, PhyDEX requires policies to infer an object’s mass, center of mass, and friction and to throw it to a destination beyond the robot’s kinematic workspace. Built on a scalable and fully automatic pipeline, it provides 35K trajectories across nine task types and 100 scenes, covering uniform, irregular, and dynamic-mass objects with static, multiple, and moving destinations. Our evaluation of three representative VLAs, GR00T N1.7, DynamicVLA, and ForceVLA, shows that current policies are far from reliable on this problem, with overall success rates below 3%. We identify two main bottlenecks: limited robustness of pretrained models, which causes most failures to occur before the throw begins, and physical blindness, since policies cannot infer properties that are not directly observable. Giving policies ground-truth physical properties roughly doubles their success rate, and experiments on a humanoid robot show the same failure patterns, which suggests these limitations are not tied to a single embodiment. We hope PhyDEX will serve as a testbed for physically grounded robot learning and motivate new designs, such as explicit physical property estimation, VLA training with reinforcement learning, and richer multi-sensory inputs.

Limitations. Our study has several limitations. First, PhyDEX focuses on a single dynamic skill, throwing, and relies on simplified physical assumptions, randomizing only mass, center of mass, and friction. Even so, current policies already fail at this level of difficulty, which shows a clear lack of physical intelligence. Extending the benchmark to more skills, such as pushing, sliding, and pouring, and to richer physics, such as elasticity, deformable objects, and aerodynamic effects, would make it more comprehensive. Second, we evaluate only three representative VLA policies. Including a wider range of models would make the conclusions stronger. Third, we discuss alternative approaches only briefly. Methods such as in-context learning, which can adapt from a few interactions, and reinforcement learning, which can discover physical properties through trial and error, may be better at probing physical properties, and we leave a systematic study of them to future work.

Broader Impact. PhyDEX aims to advance robots that understand and act on the physical properties of the objects they handle. This ability is important for safe, reliable deployment in homes, warehouses, and factories, where objects vary widely in mass, shape, and friction. By exposing the physical blindness of current VLA policies, our benchmark can help the community build models that fail less often and less unpredictably in real-world settings. Because PhyDEX is fully synthetic and generated automatically, it also lowers the cost of studying dynamic manipulation, which is otherwise expensive and risky to collect on real hardware. At the same time, throwing is a high-speed, high-energy behavior, and a policy that misjudges an object’s properties could release it on an unintended trajectory and harm people or damage property nearby. Policies trained or validated on PhyDEX should therefore not be deployed without additional safety measures, such as restricted workspaces, speed and force limits, and human oversight. Moreover, strong performance in simulation does not guarantee safe behavior in the real world, so results on PhyDEX should be complemented by careful physical testing before deployment.

LLM Usage Statement

Claude Opus was used to assist with language editing. The authors are responsible for verifying all mathematical derivations, experimental results, references, and scientific claims in the final manuscript.

References

  1. [1]Maria Bauza, Ferran Alet, Yen-Chen Lin, Tomás Lozano-Pérez, Leslie P. Kaelbling, Phillip Isola, and Alberto Rodriguez. Omnipush: accurate, diverse, real-world dataset of pushing dynamics with rgb-d video. In 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4265–4272. IEEE, 2019. 3
  2. [2]Johan Bjorck, Fernando Castañeda, Nikita Cherniaev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025. 2, 5, 6, 8arxiv.org/abs/2503.14734
  3. [3]Johan Bjorck, Fernando Castañeda, Nikita Cherniaev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots. In ArXiv, 2025. 2, 5arxiv.org/abs/2503.14734
  4. [4]Rayan Bousetouane. Physical ai agents: Integrating cognitive intelligence with real-world action. arXiv preprint arXiv:2501.08944, 2025. 2arxiv.org/abs/2501.08944
  5. [5]Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Davis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alexander Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Deeksha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael S Ryoo, Grecia Salazar, Pannag R Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Vanhoucke, Steve Vega, Quan H Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. RT-1: Robotics Transformer for Real-World Control at Scale. In Proceedings of Robotics: Science and Systems, Daegu, Republic of Korea, 2023. 2arxiv.org/abs/2212.06817
  6. [6]Remi Cadene, Simon Alberti, Francesco Capuano, Michel Aractingi, Adil Zouitine, Pepijn Kooijmans, Jade Choghari, Martino Russi, Caroline Pascal, Steven Pamel, et al. LeRobot: An open-source library for end-to-end robot learning. In International Conference on Learning Representations, pages 122398–122417, 2026. 5arxiv.org/abs/2602.22818
  7. [7]Tianxiang Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan-Ang Gao, Kaixuan Wang, Zhixuan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, and Yao Mu. RoboTwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation. In Proceedings of the 43rd International Conference on Machine Learning, pages 13673–13699. PMLR, 2026. 2, 3, 6arxiv.org/abs/2506.18088
  8. [8]Nikhil Chavan Dafle, Alberto Rodriguez, Robert Paolini, Bowei Tang, Siddhartha S Srinivasa, Michael Erdmann, Matthew T Mason, Ivan Lundberg, Harald Staab, and Thomas Fuhlbrigge. Extrinsic dexterity: In-hand manipulation with external forces. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 1578–1585. IEEE, 2014. 2
  9. [9]Matt Deitke, Ruoshu Liu, Matthew Wallingford, Huong Ngo, Oscar Michel, Aditya Kusupati, Alan Fan, Christian Laforte, Vikram Voleti, Samir Yitzhak Gadre, et al. Objaverse-xl: A universe of 10m+ 3d objects. Advances in Neural Information Processing Systems, 36:35799–35813, 2023. 3arxiv.org/abs/2307.05663
  10. [10]Zhenxuan Fan, Bo Zhang, Yutong Lin, Yuqian Yuan, Juekai Lin, Liang Liang, Zhuoyi Huang, Wenqiao Zhang, Juncheng Li, Siliang Tang, et al. Robospa: Can vla models go beyond simple scenes and short-horizon tasks? In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 23457–23492, 2026. 6arxiv.org/abs/2609.05324
  11. [11]Hongjie Fang, Shirun Tang, Mingyu Mei, Haoxiang Qin, Zihao He, Jingjing Chen, Ying Feng, Chenxi Wang, Wanxi Liu, Zaixing He, et al. Force policy: Learning hybrid force-position control policy under interaction frame for contact-rich manipulation. Robotics Science and Systems, 2026. 2arxiv.org/abs/2602.22088
  12. [12]Hao-Shu Fang, Hongjie Fang, Zhenyu Tang, Jirong Liu, Chenxi Wang, Junbo Wang, Haoyi Zhu, and Cewu Lu. Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pages 653–660. IEEE, 2024. 2arxiv.org/abs/2307.00595
  13. [13]Ankit Goyal, Hugo Hadfield, Xuning Yang, Valts Blukis, and Fabio Ramos. Vla-0: Building state-of-the-art vlas with zero modification. arXiv preprint arXiv:2510.13054, 2025. 2arxiv.org/abs/2510.13054
  14. [14]Beijing Han, Yu-Wei Chao, Erwin Coumans, Clemens Eppner, Jia Deng, Stan Birchfield, and Adithyavairavan Murali. GraspGen: Cross-embodiment 6-dof diffusion-based grasping. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20878–20889, 2026. 4arxiv.org/abs/2606.00998
  15. [15]Jisang Kang, Seonghu Jeon, Jaewoo Jung, René Zurbrügg, Honggyu An, Tiffany Portela, Marco Hutter, Marc Pollefeys, Seungryong Kim, and Sunghwan Hong. Geometric action model for robot policy learning. arXiv preprint arXiv:2606.17046, 2026. 2arxiv.org/abs/2606.17046
  16. [16]Jialei Huang, Shuo Wang, Fanqi Lin, Yihang Hu, Chuan Wen, and Yang Gao. Tactile-vla: unlocking vision-language-action model’s physical knowledge for tactile generalization. arXiv preprint arXiv:2507.09160, 2025. 2arxiv.org/abs/2507.09160
  17. [17]Xiangyu Huang, Zhenlin Hua, Han Zhou, Shounak Sural, and Ragunathan Rajkumar. Rt-vla: Real-time vision-language-action models via knowledge distillation. arXiv preprint arXiv:2606.14010, 2026. 2arxiv.org/abs/2606.14010
  18. [18]Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, Beomjun Kim, Byungjun Yoon, Changsun Jang, Daewon Choi, Dongsu Han, et al. Rldx-1 technical report. arXiv preprint arXiv:2605.03269, 2026. 2arxiv.org/abs/2605.03269
  19. [19]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan P Foster, Pannag R Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open-source vision-language-action model. In Proceedings of The 8th Conference on Robot Learning, pages 2679–2713. PMLR, 2025. 2arxiv.org/abs/2406.09246
  20. [20]Moo Jin Kim, Yihui Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning. International Conference on Learning Representations(ICLR), 2026. 2arxiv.org/abs/2601.16163
  21. [21]Fuhao Li, Wenxuan Song, Han Zhao, Jingbo Wang, Pengxiang Ding, Donglin Wang, Long ZENG, and Haoang Li. Spatial forcing: Implicit spatial representation alignment for vision-language-action model. In International Conference on Learning Representations, pages 132324–132345, 2026.arxiv.org/abs/2510.12276
  22. [22]Peiyan Li, Yixiang Chen, Hongtao Wu, Xiao Ma, Xiangnan Wu, Yan Huang, Liang Wang, Tao Kong, and Tieniu Tan. Bridgevla: Input-output alignment for efficient 3d manipulation learning with vision-language models. Advances in Neural Information Processing Systems, 38:63635–63673, 2026. 2arxiv.org/abs/2506.07961
  23. [23]Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation. arXiv preprint arXiv:2411.19650, 2024. 2arxiv.org/abs/2411.19650
  24. [24]Shuang Li, Yihui Gao, Dorsa Sadigh, and Shuran Song. Unified video action model. arXiv preprint arXiv:2503.00200, 2025. 2arxiv.org/abs/2503.00200
  25. [25]Xiaoqi Li, Mingxu Zhang, Yiran Geng, Haoran Geng, Yuxing Long, Yan Shen, Renrui Zhang, Jiaming Liu, and Hao Dong. Manipllm: Embodied multimodal large language model for object-centric robotic manipulation. In Proceeded-arxiv.org/abs/2312.16217
  26. [26]Yang Li, Zhaxizhuoma Zhaxizhuoma, Hongru Jiang, Junjie Xia, Hongquan Zhang, Jinda Du, Yunsong Zhou, Jia Zeng, Ce Hao, Jiejì Ren, et al. Forcevla2: Unleashing hybrid force-position control with force awareness for contact-rich manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8911–8920, 2026. 2
  27. [27]Zhuoling Li, Jinrong Yang, Yong Zhao, Liangliang Ren, Xiaoyang Wu, Zhenhua Xu, and Hengshuang Zhao. Sim-to-real dynamic object manipulation on conveyor systems via optimization path shaping. arXiv preprint arXiv:2508.14042, 2025. 2arxiv.org/abs/2508.14042
  28. [28]Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng. Code as policies: Language model programs for embodied control. In 2023 IEEE International conference on robotics and automation (ICRA), pages 9493–9500. IEEE, 2023. 2arxiv.org/abs/2209.07753
  29. [29]Haoran Liao, Pengyue Wang, Shuoyu Chen, Kehan Cheng, Xuhang Chen, Yuhao Lin, Mu Lin, Zhizhao Liang, Xiaoyi Fan, Chengyi Xing, et al. Dynamicmanip: Enabling dynamic manipulation from a single static demonstration. arXiv preprint arXiv:2608.01452, 2026. 2arxiv.org/abs/2608.01452
  30. [30]Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36:44776–44791, 2023. 2, 6arxiv.org/abs/2306.03310
  31. [31]Songming Liu, Bangguo Li, Kai Ma, Lingxuan Wu, Hengkai Tan, Xiao Ouyang, Hang Su, and Jun Zhu. RDT2: Exploring the scaling limit of UMI data towards zero-shot cross-embodiment generalization. In Proceedings of the 43rd International Conference on Machine Learning, pages 77202–77225. PMLR, 2026. 2arxiv.org/abs/2602.03310
  32. [32]Wenhai Liu, Junbo Wang, Yiming Wang, Weiming Wang, and Cewu Lu. Forcemimic: Force-centric imitation learning with force-motion capture system for contact-rich manipulation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 1105–1112. IEEE, 2025. 2arxiv.org/abs/2410.07554
  33. [33]Yiwen Liu, Yujun Zhu, Kui Jia, Zhao Liao, Yangwei You, and Shuaijun Wang. Vitacphys: Physical property-aware grasping from human visual-tactile demonstrations. arXiv preprint arXiv:2608.21355, 2026. 3arxiv.org/abs/2608.21355
  34. [34]Anton Lozhkov, Elie Bakouch, Gabriel Martin Blazquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Agustín Piqueres Lajarín, Hynek Kydlíček, Vaibhav Srivastav, Joshua Lochner, et al. Smollm2: When smol goes big—data-centric training of a fully open small language model. In Second Conference on Language Modeling, 2025. 6DOI
  35. [35]Matthew T Mason and Kevin M Lynch. Dynamic manipulation. In Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’93), pages 152–159. IEEE, 1993. 2
  36. [36]Stephen McAdams. Recognition of sound sources and events. Thinking in sound: The cognitive psychology of human audition, 146:146–198, 1993. 3 [37] Soroush Nasiriany, Abhiram Maddakuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. RoboCasa: Large-Scale Simulation of Household Tasks for Generalist Robots. In Proceedings of Robotics: Science and Systems, Delft, Netherlands, 2024. 2, 3, 6
  37. [38]Dantong Niu, Zhuoyang Liu, Zekai Wang, Boning Shao, Zhao-Heng Yin, Anirudh Pai, Yuvan Sharma, Stefano Sar-avalle, Ruijie Zheng, Jing Wang, et al. T-rex: Tactile-reactive dexterous manipulation. arXiv preprint arXiv:2606.17055, 2026. 2arxiv.org/abs/2606.17055
  38. [39]NVIDIA. Isaac Sim. https://github.com/isaac-sim/IsaacSim, 2025. Accessed: 2026-11-15. 6
  39. [40]Karl Pertsch, Kyle Stachowicz, Brian Icher, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models. arXiv preprint arXiv:2501.09747, 2025. 2arxiv.org/abs/2501.09747
  40. [41]Physical Intelligence, Bo Ai, Ali Amin, Raichelle Aniceto, Ashwin Balakrishna, Greg Balke, Kevin Black, George Bokinsky, Shihiao Cao, Thomas Charbonnier, et al. π0.7: A steerable generalist robotic foundation model with emergent capabilities. arXiv preprint arXiv:2604.15483, 2026. 2arxiv.org/abs/2604.15483
  41. [42]Matthew Purri and Kristin Dana. Teaching cameras to feel: Estimating tactile physical properties of surfaces from images. In European Conference on Computer Vision, pages 1–20. Springer, 2020. 3arxiv.org/abs/2004.14487
  42. [43]Jingjing Qian, Boyuan Han, Chen Shi, Lei Xiao, Long Yang, Shaoshuai Shi, and Li Jiang. Geopredict: Leveraging predictive kinematics and 3d gaussian geometry for precise vla manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13529–13539, 2026. 2arxiv.org/abs/2512.16811
  43. [44]Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Jiayuan Gu, Zhigang Wang, Yan Ding, Bin Zhao, Dong Wang, and Xuelong Li. SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Models. In Proceedings of Robotics: Science and Systems, 2025. 2DOI
  44. [45]Shilin Shan, Chuhao Zhou, Ruize Wang, Xinyan Chen, Xiangyu Chen, Xinyu Zhou, Boyu Ma, Iris Yuxuan Hu, Jingliang Li, Celeste Yuxuan Hu, et al. Learning physical interaction: A survey of tactile-and force-aware robot learning. arXiv preprint arXiv:2608.07558, 2026. 2arxiv.org/abs/2608.07558
  45. [46]Trevor Standley, Ozan Sener, Dawn Chen, and Silvio Savarese. image2mass: Estimating the mass of an object from its image. In Conference on Robot Learning, pages 324–333. PMLR, 2017. 3
  46. [47]Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis, Alexander Millane, Helen Oleynikova, Ankur Handa, Fabio Ramos, et al. Curobo: Parallelized collision-free robot motion generation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 8112–8119. IEEE, 2023. 5DOI
  47. [48]Subramanian Sundaram, Petr Kellnhofer, Yunzhu Li, Jun-Yan Zhu, Antonio Torralba, and Wojciech Matusik. Learning the signatures of the human grasp using a scalable tactile glove. Nature, 569(7758):698–702, 2019. 3DOI
  48. [49]Jiaming Tang, Yufei Sun, Yilong Zhao, Shang Yang, Yujun Lin, Zhuoyang Zhang, James Hou, Yao Lu, Zhijian Liu, and Song Han. Vlash: Real-time vla via future-state-aware asynchronous inference. arXiv preprint arXiv:2512.01031, 2025. 2arxiv.org/abs/2512.01031
  49. [50]Junjie Wen, Yichen Zhu, Jinming Li, Minjie Zhu, Zhibin Tang, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, et al. Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation. IEEE Robotics and Automation Letters, 2025. 2arxiv.org/abs/2409.12514
  50. [51]Junjie Wen, Yichen Zhu, Minjie Zhu, Zhibin Tang, Jin-ming Li, Zhongyi Zhou, Xiaoyu Liu, Chaomin Shen, Yaxin Peng, and Feifei Feng. Diffusionvla: Scaling robot foundation models via unified diffusion and autoregression. In Forty-second International Conference on Machine Learning, 2025. 2
  51. [52]Jiajun Wu, Joseph J Lim, Hongyi Zhang, Joshua B Tenenbaum, and William T Freeman. Physics 101: Learning physical object properties from unlabeled videos. In BMVC, 2016. 3DOI
  52. [53]Wenli Xiao, Jia Xie, Tonghe Zhang, Haotian Lin, Letian Fu, Haoru Xue, Jalen Lu, Yi Yang, Cunxi Dai, Zi Wang, et al. Empire: Agentic robot policy self-improvement in the real world. Conference on Robot Learning, 2026. 2arxiv.org/abs/2606.19980
  53. [54]Haozhe Xie, Beichen Wen, Jiarui Zhang, Zhaoxi Chen, Fangzhou Hong, Haiwen Diao, and Ziwei Liu. Dynamicvla: A vision-language-action model for dynamic object manipulation. arXiv preprint arXiv:2601.22153, 2026. 2, 5, 6, 8arxiv.org/abs/2601.22153
  54. [55]William Xie, Maria Valentini, Jensen Lavering, and Nikolaus Correll. Deligrasp: Inferring object properties with llms for adaptive grasp policies. In Proceedings of The 8th Conference on Robot Learning, pages 1290–1309. PMLR, 2025. 3arxiv.org/abs/2403.07832
  55. [56]Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Shihyun Yu, George Kurian, Suneel Indurpur, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922, 2026. 2arxiv.org/abs/2602.15922
  56. [57]Jiawen Yu, Hairuo Liu, Qiaojun Yu, Jieji Ren, Ce Hao, Haitong Ding, Guangyu Huang, Guofan Huang, Yan Song, Panpan Cai, et al. Forcevla: Enhancing vla models with a force-aware moe for contact-rich manipulation. Advances in Neural Information Processing Systems, 38:93409–93439, 2026. 2, 5, 6, 8arxiv.org/abs/2505.22159
  57. [58]Samson Yu, Lin Kelvin, Anxing Xiao, Jiafei Duan, and Harold Soh. Octopi: Object Property Reasoning with Large Tactile-Language Models. In Proceedings of Robotics: Science and Systems, 2024. 3arxiv.org/abs/2405.02794
  58. [59]Wenzhen Yuan, Shaoxiang Wang, Siyuan Dong, and Edward Adelson. Connecting look and feel: Associating the visual and tactile properties of physical materials. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4494–4502. IEEE, 2017. 3arxiv.org/abs/1704.03822
  59. [60]Shiduo Zhang, Zhe Xu, Peiju Liu, Xiaopeng Yu, Yuan Li, Qinghui Gao, Zhaoye Fei, Zhangyue Yin, Zuxuan Wu, Yu-Gang Jiang, et al. Vlabench: A large-scale benchmark for language-conditioned robotics manipulation with long-horizon reasoning tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11142–11152, 2025. 2arxiv.org/abs/2412.18194
  60. [61]Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, Xin-jiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, He Wang, Zhizheng Zhang, et al. Dreamvla: a vision-language-action model dreamed with comprehensive world knowl-edge. Advances in Neural Information Processing Systems, 38:24195–24228, 2026. 2arxiv.org/abs/2507.04447
  61. [62]Yifan Zhang, Ruiping Wang, and Xilin Chen. Dynamic behavior cloning with temporal feature prediction: Enhanc-ing robotic arm manipulation in moving object tasks. IEEE Robotics and Automation Letters, 10(6):5209–5216, 2025. 2
  62. [63]Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023. 2arxiv.org/abs/2307.15818

Paper details

vlnc:W_9f82ru4c
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude Opus (multiple versions)

View statement
Looking for related papers…
Contents