DexPort: A Hand-Held Skill Interface for Dexterous Hand Data Collection

Image not extracted

Introduction

Multi-fingered hands allow robots to use different grasps, manipulate tools, and reorient objects in hand. Learning these behaviors from demonstrations requires data that capture detailed finger motion and contact-rich interactions. Teleoperation records executable robot actions, but it requires a complete robot setup. Collecting in-hand manipulation demonstrations through teleoperation is particularly challenging because the operator must coordinate many finger joints [1], [2], [3], [4], [5]. On the other hand, human hand demonstrations are easier to collect, but their motions must be estimated and retargeted to the robot, and they do not directly provide robot actions [6], [7], [8]. We therefore need a method that scales beyond a fixed robot setup while preserving executable hand actions.

To address this issue for parallel-jaw grippers, the Universal Manipulation Interface (UMI) [9] uses a portable, hand-held interface to collect demonstrations naturally and at scale. However, extending this paradigm to multi-fingered hands has not been trivial. Early efforts, including Dex-UMI [10] and DEXOP [11], can collect dexterous and contact-rich behaviors using wearable hand exoskeletons. However, designing a high-DoF exoskeleton involves a difficult trade-off between dexterity and wearability: supporting more independent finger motions increases both mechanical complexity and the operator’s control burden. In addition, these systems record exoskeleton configurations rather than the commands and resulting states of the deployed robot hand. RealDexUMI [12] preserves this information by collecting executable commands with the physical robot hand, but demonstrates the approach on a hand with only six active DoFs. It remains unclear how to preserve these benefits for a hand with many independently controlled joints without making the interface cumbersome.

Inspired by these works, we address three requirements for a portable interface for dexterous hands. Table I compares existing systems against these requirements. First, the interface should use the same end effector as the deployed robot (Same Emb.), so that hand motions need no retargeting and no embodiment gap is introduced. Second, it should record the commands sent to the hand (Action Data) rather than relying only on actions inferred from future measured states. This distinction matters during contact, when the commanded target and measured joint state may differ. Third, it should expose enough controllable degrees of freedom (DoF) to support tasks that require multi-fingered dexterity; otherwise, a simpler gripper would be sufficient. Reusing the deployment hand satisfies the first two requirements, but creating an intuitive and comfortable interface for high-DoF control is challenging, especially when the operator must control each joint directly.

System	DoF	Weight	Same Emb.	Action Data
UMI [9]	1	780 g	✓	✗
DexViTac [13]	6	<600 g	✗	✗
DEXOP [11]	12/9/7	>1000 g	✓	✗
DexEXO [14]	6	>1000 g	✓	✗
DEX-Mouse [15]	6	>1000 g	✓	✗
Exo-ViHa [16]	6	1100 g	✗	✓
DexUMI [10]	12/6	850 g	✓	✗
RealDexUMI [12]	6	680 g	✓	✓
DEXPORT	20	955 g	✓	✓

In this paper, we introduce DEXPORT, a portable data-collection interface built around these requirements. Our key design principle is to collect demonstrations with the same hand hardware used at deployment while controlling it through a small set of skills. Because the collection hand shares the deployment hand’s hardware and control interface, each demonstration records native commanded joint targets and resulting states. The operator moves the device to specify the robot’s wrist motion and selects or adjusts skills to control the hand. Parameterized skills produce power, pinch, and two-finger grasps, while a learned closed-loop skill generates the finger gaiting needed for in-hand rotation. This skill-level interface leaves task decisions to the operator while the robot hand produces the coordinated finger motion.

We evaluate DEXPORT on four tasks that require different forms of dexterity: selecting grasps for objects of different shapes, opening a drawer while holding an object in the same hand, rotating a tool in hand, and coordinating two hands to scan a barcode. We collect demonstrations both in the lab and in unstructured environments, without a robot arm or external tracking infrastructure. We then use behavior cloning to train visuomotor policies directly on the recorded wrist motions and hand commands, without skill labels. Our policies complete all four tasks and outperform a state-as-action baseline trained without commanded hand actions on each task. Across the three tasks scored by success rate, final-task success averages 64% versus 11%. Co-training with in-the-wild demonstrations further improves pick-and-place in every condition, including unseen objects and scenes with distractors. DEXPORT also collects demonstrations at roughly 1.4× the rate of glove teleoperation.

To summarize, our contributions are as follows:

• A portable and ergonomic data-collection interface that uses the same hand hardware and control space as the deployed robot. It avoids a hand embodiment gap and records native commanded actions and measured states.

• A skill-level control scheme that makes a high-DoF hand practical to operate during portable data collection. A compact set of parameterized and learned skills replaces joint-by-joint control while preserving the ability to perform dexterous tasks.

• An evaluation across four real-world tasks, including data collection in unstructured environments and visuomotor policies trained directly on the recorded wrist motions and hand commands, without skill labels.

Related Work

Hand-Held Grippers for Robot Learning. Hand-held interfaces offer a middle ground between robot teleoperation and unconstrained human video: they can be carried into diverse environments while retaining an end-effector-centered observation and action space. Early systems used instrumented reacher-grabbers or other portable tools and recovered their motion using structure from motion, visual odometry, or smartphone odometry [17, 18, 19]. UMI [9] makes this approach more accurate and broadly useful through a hand-held parallel-jaw gripper, a wide field-of-view camera, visual-inertial pose tracking, and a policy interface that accounts for deployment latency. Matching the collection and deployment grippers further reduces the embodiment gap. UMI demonstrates that portable collection can support precise and long-horizon behaviors, but its gripper has only one actuated degree of freedom. Extending these benefits to a multi-fingered hand requires a practical way to control its higher-dimensional action space.

Learning Dexterous Manipulation. Dexterous manipulation skills are commonly learned from simulation, human demonstrations, or robot teleoperation. Simulation can generate robot actions at scale and has enabled complex in-hand behaviors, but transferring policies that rely on contact dynamics and tactile observations to real hardware remains difficult [20, 21, 22]. Human demonstrations collected from videos, hand tracking, or motion-capture systems provide natural and diverse behaviors [23, 24] [3, 6, 7, 8]. However, hand motion must be estimated and retargeted, and the resulting data do not directly contain robot actions. Teleoperation instead records executable actions by mapping human-hand motion to a robot hand [25] [1, 2, 3, 4, 5]. Immersive visual feedback and haptic devices can improve control [26, 27, 28], but these systems generally require a complete robot setup and ask the operator to coordinate many finger joints. Lower-dimensional interfaces reduce this burden. HATO maps a small set of controller inputs to coordinated finger motions [29], while DexScrew uses a simulation-trained rotation policy as a skill during teleoperation [21]. These results motivate skill-level control, but neither system addresses portable collection with a high-DoF hand.

Portable Data Collection for Dexterous Hands. Recent systems enable dexterous data collection without a fixed robot setup. DexUMI, DEXOP, and DexEXO use wearable exoskeletons or passive hand mechanisms to constrain and measure the operator’s motion [10], [11], [14]. These systems efficiently collect demonstrations of contact-rich behaviors and reduce the human-to-robot kinematic gap. However, these passive collection mechanisms record hand configurations rather than both the commands and resulting states of the actuated hand used at deployment. Related systems improve kinematic alignment or tactile sensing through mechanically isomorphic and modular exoskeleton designs [30], [16]. Another line of work uses robot hands that operators carry or wear, reducing the contact-geometry gap and allowing robot-aligned data collection [31], [15]. RealDexUMI [12] follows the UMI principle of using the same end effector for collection and deployment. It also records robot commands, rather than inferring actions from measured states, but demonstrates this approach on a hand with six active DoFs. In contrast, DEXPORT targets a 20-DoF hand, for which joint-by-joint operation would be cumbersome. We use a compact set of skills to make the hand practical to operate while still recording the raw hand commands used to train the policy.

Method

In Sec. III-A, we introduce the hardware design of our portable data collection interface. Next, in Sec. III-B we describe our library of skills, designed to be intuitive and useful for learning dexterous policies. Finally, Sec. III-C explains how this data can be used for imitation learning.

Hardware Design

Transferable Hand-Camera Module. Our portable data-collection interface comprises a transferable hand-camera module and a collection handle built around a Meta Quest 2 controller, as shown in Figure 2(a). The module combines a five-fingered, 20-DoF Wuji robot hand with a wide-angle fisheye camera through a rigid 3D-printed flange. During collection, it attaches to the collection handle; during deployment, the same physical module mounts to the robot wrist. Reusing the module preserves the hand embodiment and camera-to-hand extrinsics across the two settings, so recorded hand targets transfer to the robot without retargeting. This satisfies the Same Emb. requirement. The mechanical design can be adapted to other hand-camera pairs by replacing the flange. We design flanges for both the Wuji 1 and Wuji 2.

Image not extracted

Recorded Signals. During data collection, the Quest 2 controller provides the wrist’s 6-DoF trajectory, from which we construct end-effector motion targets. The fisheye camera captures the hand-object interaction and surrounding task context, while the hand controller records both the commanded 20-DoF joint targets and the resulting joint states. Each demonstration therefore carries the actions actually sent to the hand, rather than actions inferred from measured states, satisfying the Action Data requirement. The rigid connections keep the controller-to-hand and camera-to-hand transforms fixed, so the tracked motion and visual observations remain expressed relative to the same hand frame.

Ergonomic Design. The complete hand-held assembly weighs 0.955 kg, of which the non-hand hardware contributes 0.220 kg. To support this load while leaving the thumb and index finger free to operate the controls, we extend the Quest 2 battery cover into a collection handle that connects the controller to the transferable module. A strap across the back of the operator’s hand carries part of the load, reducing the sustained grip force.

Our initial design placed the controller behind the robot hand, offsetting the operator’s grip from the assembly’s center of mass and creating a long moment arm, as shown in Figure 2(b). The final design positions the controller above the hand so that the grip lies near the center of mass. This shortens the moment arm and reduces the wrist torque that the operator must counteract.

Dexterous Control via Parameterized Skills

Control via skills is an effective way to operate high-dimensional robotic systems. For example, a humanoid can be commanded through a desired velocity rather than by specifying the motion of every joint. We apply the same principle to dexterous hands by separating the operator’s intent from the finger motions required to realize it. To this end, we design a compact set of skills that we find effective across a wide range of dexterous tasks. The skill interface is parameterized instead of discrete, allowing more wholistic control. For instance, the user controls the strength of the grasp by using the joystick to incrementally close or open the grasp, instead of snapping to a single fixed grasp.

Together, the skills keep all 20 joints of the hand available during collection without asking the operator to command each one, satisfying the DoF requirement that separates a dexterous interface from a simpler gripper. We implement this interface using a Meta Quest 2 controller, which provides two buttons, two continuous triggers, and a two-axis joystick. As illustrated in Figure 2(c) and detailed below, these inputs allow the operator to select and modulate the skills.

• Thumb Control. The two joystick axes control thumb flexion and extension along one axis and abduction and adduction along the other, following HATO [29]. This allows the operator to position the thumb independently for different grasp geometries.

• Power Grasp. One joystick axis controls the coordinated closure of the four non-thumb fingers. Moving the joystick incrementally loosens or tightens the grasp, while releasing it holds the current target. Unlike continuous trigger control [29], this persistent command does not require the operator to maintain pressure throughout a grasp. It is primarily used to wrap the hand around larger objects.

• Pinch Grasp. The same incremental command controls only the index finger, which closes against the independently positioned thumb. This skill supports small objects and grasps that require precise fingertip contact.

• Two-Finger Grasp. The index and middle fingers open or close together while the remaining fingers retain their targets. This skill supports multi-stage behaviors such as multi-object grasping and two-finger grasp-and-pull motions, without requiring the operator to coordinate the two fingers individually.

• In-Hand Rotation. A button invokes a learned closed-loop policy that generates the repeated finger gaiting required to rotate a grasped tool in hand. The policy controls the fingers while the operator continues to control the global motion of the hand.

Learning the In-Hand Rotation Skill. We learn the in-hand rotation skill in simulation. This relieves the operator of difficult low-level, in-hand manipulation, especially without finger-level haptic feedback. Similar to [21], we train a closed-loop screwing policy in simulation with PPO [32] in Isaac Lab [33].

We model the task as rotating a cylindrical handle on a revolute joint. In the real-world, the operator can make wrist adjustments, but we simplify the simulation environment so hand base is fixed, with the policy commanding fingers only. The policy learns delta actions over 12 actuated joints, with the ring and pinky finger fixed. The policy observes a three-step history, including measured joint positions, commanded targets, and the previous action. We learn a privileged critic which has access to the handle’s rotation angle.

We follow rewards and domain randomization similar to [21], which includes a handle rotation, fingertip proximity, bounded action smoothness, initial grasp proximity, joint torque, and mechanical work rewards. We randomize object mass, center of mass, friction, restitution, finger PD gains, initial grasp, and random disturbance forces. We train for $1.18 \times10^{8}$ environment steps across 4096 environments. During collection, the actor, a three-layer MLP, runs at 20 Hz on the hand’s joint encoders while the operator retains control of the wrist.

Policy Learning

Raw-Action Supervision. The skills are used only to assist the operator during data collection. The learned policy directly predicts wrist poses and 20-DoF hand joint targets. At each time step, we record the raw 20-DoF target produced by the active skill and sent to the hand controller, together with the resulting measured joint state. The policy receives no skill identity, skill parameter, or task-phase label. Instead, it is trained to predict the commanded joint targets directly. Thus, the skills structure how demonstrations are collected without constraining the representation learned by the policy. Switching the action label from the commanded target to the measured joint state is the single change that defines our state-as-action baseline in Sec. IV; every other part of the recipe below is shared.

Observation and Action Spaces. We train a diffusion policy [34] with a convolutional U-Net denoiser and a fine-tuned DINOv2 visual encoder. Images are augmented with a random crop and resize, color jitter, random grayscale, and Gaussian blur. The vision encoder is optimized with a reduced learning rate, one-tenth that of the remaining network. The policy observes the two most recent time steps, each consisting of a $224 \times224$ RGB image, the end-effector position represented relative to the current chunk start [9], and the measured 20-DoF hand state.

The policy predicts a 16-step action chunk containing end-effector poses relative to the start of the chunk and absolute 20-DoF hand targets. Each action step is 29-dimensional: 9 for the wrist (3 translation and a 6D rotation) and 20 for the hand. For bimanual tasks, we concatenate the observations and actions of the two hands without otherwise changing the architecture. We train 50 DDIM steps and sample with 10 steps. Following HATO [29], policy inference runs asynchronously to maintain smooth control.

Deployment Calibration. The collected end-effector actions are expressed in the Quest controller frame, whereas deployment actions must be expressed in the robot end-effector frame. We estimate the fixed transform between these frames. from paired controller and end-effector poses collected with the device mounted on the robot. Concretely, we bolt the module to the robot flange and record $N$ paired samples of the flange pose $R_i$, and the controller pose $Q_i$, from the headset. Each sample satisfies $Q_i = X R_i M$, where $X$ is the unknown robot base to headset transform and $M$ is the controller to flange transform, which is the mount transform we want. Any two samples eliminate $X$,

so $N$ poses give up to $N(N - 1)/2$ pairwise constraints on exactly the conjugation that deployment applies, $A^{\mathrm{ee}} = M A^{\mathrm{quest}} M^{-1}$. We fit the full 6-DoF $M$ by least squares on (1), independently for each of the two arms. We apply this transform to the predicted end-effector motions at deployment; the 20-DoF hand targets require no conversion.

Experiments

We evaluate DEXPORT through three research questions:

• Q1: Do the hand commands recorded by DEXPORT enable policies that succeed on diverse dexterous tasks?

• Q2: Does co-training with demonstrations collected in unstructured environments improve robustness to unseen objects and cluttered scenes?

• Q3: How does the collection throughput of DEXPORT compare with teleoperation and human-hand demos?

Experimental Setup

Robot Platform. We deploy policies on Tianji Marvin, a bimanual robot with two 7-DoF arms. The hand–camera modules used for collection (Sec. III-A) mount directly to its wrists, so the hand embodiment and camera-to-hand extrinsics are identical across collection and deployment. Hand targets therefore need no retargeting, and the only correction at deployment is the fixed controller-to-end-effector transform (Sec. III-C).

We showcase the modularity of our system by using different hands: the Wuji 1 hand for Pick-and-Place and Screwing, and the Wuji 2 hand for Drawer Stowing and Bimanual Barcode Scanning.

Tasks. We evaluate four tasks that exercise complementary forms of dexterity. Figure 4 shows a successful rollout of each, annotated with the phases we use to score partial success and with the grasp types that emerge:

Image not extracted

• Vegetable Pick-and-Place. The robot sequentially picks and places two vegetables into a plate. Variations in object size and shape require the policy to select between pinch and power grasps without explicit grasp labels.

• Screwing. The robot maintains a stable grasp while rotating a screwdriver in hand, testing sustained contact and repeated finger gaiting.

• Drawer Stowing. Without releasing a grasped object, the robot opens a drawer with the same hand, places the object inside, and closes the drawer. This task couples object retention with articulated-object manipulation.

• Bimanual Barcode Scanning. One hand grasps and presents an object while the other positions and operates a barcode scanner. Success requires coordinated bimanual positioning of the object and scanner.

Observations. Policies observe the fisheye view from the hand–camera module, shown for every task in Figure 3. For drawer stowing we add a third-person camera, because the grasped object occludes much of the scene while the drawer is pulled open. The operator is occasionally visible in the recorded frames, and we apply no post-processing.

Image not extracted

Datasets. We collect 1,605 laboratory demonstrations across the four tasks: 700 for vegetable pick-and-place, 200 for screwdriver manipulation, 380 for drawer stowing, and 325 for bimanual barcode scanning. We additionally collect 1,000 pick-and-place demonstrations in unstructured apartment and office environments. The co-trained policies use both the laboratory and in-the-wild pick-and-place datasets.

Training Variants and Baselines. To answer Q1, we compare policies trained on the commanded hand targets recorded by DEXPORT against a state-as-action baseline that instead uses measured hand states as action labels. To answer Q2, we compare the laboratory-only pick-and-place policy with two co-trained variants. To answer Q3, we compare the number of trajectories collected per 30 minutes using DEXPORT, teleoperation, and human-hand demonstrations.

Evaluation Protocol. We evaluate each policy for 15 episodes per task or evaluation condition. For vegetable pick-and-place, we test seen objects, unseen objects, and scenes with distractors; partial success denotes placing one of the two objects, while full success denotes placing both. For drawer stowing and bimanual scanning, we report completion of the task phases visualized in Figure 4. For screwdriver manipulation, we report the average hold duration in seconds and rotations per minute.

Policy Learning Across Dexterous Tasks (Q1)

Figure 5 reports imitation learning results across the four tasks. Policies trained on DEXPORT demonstrations acquire all of the target behaviors: they select between pinch and power grasps without grasp labels, hold an object while

Image not extracted

pulling open a drawer with the same hand, gait the fingers to rotate a screwdriver in hand, and coordinate two arms to present an object to a scanner.

The state-as-action baseline, which uses measured joint states as action labels, is worse on every task. For instance, we find that the state-as-action baseline is unable to sufficiently grasp the object. For the scanning task, grasping the package is nearly impossible for the baseline, whereas with robot commands, we exert the necessary force to hold the item tightly. For the drawer task, the laundry often falls during the drawer opening segment. We visualize this in Figure 7. Because a stage cannot succeed once an earlier stage has failed, the deficit compounds rather than appearing as a constant offset: on bimanual scanning the baseline grasps the first object in 80% of episodes, holds both in 13%, and never completes a scan. The screwdriver task exposes the same weakness through a different metric. While the baseline holds the tool more than twice as long as our policy (31.7 s vs. 15.6 s), it rotates it at a third of the rate and is much less effective. These results highlight the importance of Action Data, since access to robot commands for the deployment hand is crucial for a higher success rate. This is a major limitation for data collection interfaces based on exoskeletons or hardware without action data.

Image not extracted

	DEXPORT	Skill-Teleop	Glove-Teleop	Human hand
Traj. / 30 min	122	68	86	275
Portable	✓	✗	✗	✓
Same hand emb.	✓	✓	✓	✗
Action data	✓	✓	✓	✗

Co-Training with In-the-Wild Data (Q2)

We collect a large in-the-wild pick-and-place dataset across several sites, including an apartment room and a range of office tables. DEXPORT requires neither a robot arm nor external tracking infrastructure, thus enabling this portable data collection. We use the same training objects, with the difference being diversity in scene, background, and lighting rather than in object identity, as Figure 3 illustrates.

Figure 6 shows that full success improves in every condition and grows with the amount of in-the-wild data. With 1,000 in-the-wild demonstrations, full success rises from 40% to 87% under distractors and from 20% to 73% on unseen objects, so the benefit transfers to new objects. It also improves on seen objects, from 47% to 73%. App. A reports a lower data regime.

Image not extracted

Data Collection Throughput (Q3)

We measure throughput as the number of trajectories collected during 30 minutes of continuous operation on the unimanual pick-and-place task, comparing DEXPORT against

bare-hand human demonstration and teleoperation. For Glove Teleoperation, the operator uses the Wuji Glove for hand tracking and the VR controller for the end-effector. For Skill Teleoperation, we use the VR controller for end-effector control and the same hand skills as DEXPORT. The operator is expert-level for each style of data collection.

Table II shows that DEXPORT collects roughly 1.4× as many trajectories as Glove Teleoperation and 1.8× as many trajectories as Skill Teleoperation. While Skill Teleoperation is a simpler interface, reducing the operator’s cognitive burden, Glove Teleoperation still remains faster because the hand tracking is more responsive and flexible. Generally, both are much slower than DEXPORT. When teleoperating, the operator is out of the scene, removing the direct visual and haptic cues. This makes approaching an object and performing precise tasks, such as pinch grasping a small object, much slower and more difficult. Human hand data collection remains about 2.3× faster than DEXPORT, but we regard it as an upper bound on speed rather than a competing source of training data, as human hand data provides neither the deployed hand’s embodiment nor its commanded actions.

Conclusions

We presented DEXPORT, a portable interface for collecting demonstrations with high-DoF dexterous hands. The system uses the same hand hardware for collection and deployment, preserving its contact geometry while recording its commanded targets and resulting joint states. A compact set of skills makes the hand practical to operate by separating task-level intent from detailed finger coordination. These skills assist only the demonstrator; the learned policy directly predicts the raw hand commands without skill or task-phase labels. In our experiments, DEXPORT collects 122 trajectories per 30 minutes, roughly 1.4× the rate of teleoperation. Policies trained on the resulting demonstrations succeed on all four tasks and consistently outperform a state-as-action baseline, which never completes a bimanual barcode scan. With 700 laboratory demonstrations, co-training with in-the-wild data further improves pick-and-place success in every evaluation condition. Together, these design choices provide a practical and scalable solution for collecting portable, executable demonstrations for dexterous robot learning.

Limitations. Despite its ergonomic design, the weight of the device limits uninterrupted use and rotation-heavy motions. As the weight of the hand is the primary bottleneck, we expect that future hardware developments for lighter and smaller hands can address this. Our skills also cover only part of the space of dexterous behaviors, and future work may explore extending them with additional skills learned in simulation, such as other in-hand manipulation skills, learned grasps, and more.

Acknowledgment

The authors would like to thank Keyshawn King and Volodymyr Behma for data collection and support, and Andrew Wright for initial hardware designs. We also thank Tingfan Wu, Anas Houssaini, Jensen Gao, and the FAR team for feedback and support. rotation angle, and angular velocity, the applied joint torques, and the previous action. The policy outputs a 20-D action $a\in[-1,1]^{20}$, and each joint target moves by 0.04 $a$ rad per step within the joint limits, which is a scaling factor inspired by [21]. The ring and pinky fingers are fixed, and the policy controls the remaining 12 joints.

Term	Weight	Definition
Handle rotation	6.0	$\operatorname{clip}(\dot{\theta},-4,4)$
Fingertip proximity	2.0	$\operatorname{clip}(1-\bar{d}/0.035,0,1)$
Action smoothness	1.0	$\exp(-(a_t-a_{t-1})^2/0.5^2)$
Pose deviation	−6.0	$\lVert q-q_0\rVert^2$
Over-speed	−0.3	$\max(\dot{\theta}-10,0)$
Joint torque	−0.1	$\lVert\tau\rVert^2$
Mechanical work	−0.01	$(\sum_j|\tau_j||\dot{q}_j|)^2$

Real-World Execution. During collection, the actor runs at 20 Hz on joint positions measured by the hand’s encoders and holds its targets between policy steps, while the operator controls the wrist through the collection handle.

Policy Learning

Architecture. Table VI lists the hyperparameters of the diffusion policy in Sec. III-C. The end-effector pose is represented relative to the current end-effector frame, with a 3-D translation and a 6-D rotation. Each predicted wrist action is a relative pose with respect to the start of the chunk, and each hand action is an absolute 20-D target. Bimanual policies concatenate the observations and actions of both hands and additionally observe the relative pose between them. All inputs and targets are normalized using their 2nd and 98th percentiles.

Hyperparameter	Value
Vision encoder	DINOv2 ViT-S/14 [37], fine-tuned
Denoiser	1D conditional U-Net [34], FiLM [38]
U-Net channels per level
Pick-and-place in-the-wild	(512, 1024, 2048)
Pick-and-place baseline	(256, 512, 1024)
Screwdriver	(512, 1024, 2048)
Drawer stowing	(512, 1024, 2048)
Bimanual scanning	(512, 1024, 2048)
Noise scheduler	DDIM [39], squared cosine
Prediction target	Noise ($\epsilon$)
Train / inference steps	50 / 10
Optimizer	AdamW, $\beta=(0.9,0.999)$
Weight decay	0.01
Learning rate	$1\times10^{-4}$ (vision encoder $1\times10^{-5}$)
LR schedule	Cosine, 500 warmup steps
Batch size	64
Gradient steps	$5\times10^5$
EMA decay	$\min(1-(1+t)^{-3/4},0.9999)$
Random crop	212 px, resized to 224
Color jitter	(0.3, 0.4, 0.5, 0.08)
Random grayscale	$p=0.1$
Gaussian blur	$k=5$, $\sigma\in[0.1,1.5]$, $p=0.2$

Deployment

Mount Calibration. To estimate the controller-to-flange transform $M$ in (1), we bolt the module to each robot flange and jog the arm through about 30 poses that tilt the flange axis by at least 30°, pausing 1.5 s at each pose before recording the paired flange and controller poses. We initialize $M$ with a closed-form hand–eye solution and refine it by nonlinear least squares.

Control. The policy runs at 30 Hz, matching the collection rate. Following HATO [29], inference runs asynchronously: each new 16-step chunk is aligned to the control tick at which its observation was taken, the first 8 steps are kept, steps whose time has passed are dropped, and the hand and wrist commands are sent at each tick average all chunks that cover it. We sample each chunk with 10 DDIM steps. Predicted wrist motions are converted to flange targets with the mount transform and solved for the 7-DoF arm with Levenberg–Marquardt inverse kinematics in PyRoki [40].