Embodied at Sea: Imitation Learning for Multi-Task Underwater Manipulation

Introduction

Marine conservationists and archaeologists routinely perform tasks such as environmental monitoring and specimen retrieval, constrained by poor visibility, dynamic currents, and the physiological limits of human divers. At deeper depths, monitoring requires repeated dives that are slow, costly, and risky due to lengthy decompression procedures [1]. Automating these operations, e.g. via robots, could accelerate scientific research and reduce diver risk.

Recent advances in Behavior Cloning (BC) [2] have enabled robust single-task robot policies from relatively little data. While prior works explore the performance of these approaches in aquatic robotics [3, 4], deployment in real marine archaeology and conservation missions remains largely unexplored.

BC-based policies, particularly Vision-Language-Action (VLA) models, have shown remarkable generalization to novel tasks and objects, making them an appealing candidate for underwater manipulation. However, they are trained almost exclusively on terrestrial datasets featuring stable lighting and fixed-base manipulators, while underwater vehicles operating in marine environments experience unconstrained 6-DOF motion, drag, turbidity, visual distortion, occlusion, and distractors (see Figure 1). Whether these policies retain their generalization capabilities under such conditions and across diverse archaeological sites remains an open question.

By enabling increased levels of autonomy for underwater robotic manipulation tasks, there will be scientific value not just within the field of robotics, but as well in the fields of marine archaeology, oceanography, geology, biology, and environmental conservation. This leads to a motivating question: how can the recent advances in terrestrial behavior cloning be leveraged for underwater environments?

In addressing this driving question, the work presented below yields the following contributions:

1. To our knowledge, the world’s first instance of successfully implementing a multi-task VLA underwater on real hardware, where a VLA is defined as a single Vision-Language-Action model that can execute multiple tasks and be commanded to execute a specific task via language based inputs.

2. A real-world case study of BC manipulation models deployed on an underwater Remotely Operated Vehicle (ROV) at a Mediterranean shipwreck for archaeology and conservation tasks.

3. A thorough investigation into how different proprioception configurations and underwater sensor modalities in a model’s observation and action spaces affect generalist model behavior underwater, including a new action space that leverages an underwater robot depth sensor and feedback control for stable motion in the vertical direction.

4. Introducing the BLUEMBODIED KIT as an open source repository that enables users to collect underwater teleoperation data, train single (ACT) and multi-task models (DynamicVLA, SmolVLA), and run inference on Blue Robotics’ BlueROV2 while staying compatible with BlueOS.

Related Work

Autonomy in Underwater Robot Systems

Underwater robot systems (specifically, ROVs (Remotely Operated Vehicles) and AUVs (Autonomous Underwater Vehicles) [5]) have been used for decades, largely for exploration and scientific sampling. However, most AUVs lack manipulation abilities, while many ROVs are equipped with end-effectors. ROV end-effectors are typically remotely operated by humans [6], while autonomous underwater manipulation has traditionally relied on classical control techniques, including PID, sliding-mode, and model predictive control [7]. The machine-learned approaches to developing robot manipulation control policies have largely been utilizing reinforcement learning (RL), e.g. [8], [9], but these works lack real-world ocean deployments.

Single-Task BC in Terrestrial Contexts

BC for single task policies can be described as training a model to output the appropriate signals that can accomplish a single task. The two most notable single task policy models are Action Chunking Transformers (ACT) [2] and Diffusion Policy [10] (DP). When introduced in [2] the ACT models demonstrated the ability to accomplish highly dexterous tasks with only 100 demonstrations. These models have been augmented in several ways since then, including the addition of mobility [11]. While ACT models have not demonstrated much generalization, DP approaches have often yielded better flexibility but at the cost of needing significantly more demonstrations to achieve high success rates on a task. An advantage of single task policies over most generalist policies is the small size, (e.g. millions of model parameters as opposed to billions of model parameters), which can lead to faster training times and lower inference latency at model deployment.

Generalist Policies in Terrestrial Contexts

BC for generalist policies can be described as training a single model to be able to accomplish many tasks. Vision Language Action models (VLAs), a broad class of generalist models that take images and text-based language commands as inputs and output robot actions, have become increasingly popular for their robustness. Some of the first VLAs included RT-1 [12], RT-2 [13], and RT-X [14], as well as OpenVLA [15]. Later, $\pi_0$ [16] from Physical Intelligence demonstrated a multi-expert system that leverages the world generalizability of both Vision Language Models (VLMs) and the action fidelity of diffusion-based policies. A key characteristic of the $\pi_0$ architecture is that it is built upon a large pre-trained VLM backbone (i.e. SigLIP + Gemma), providing cross-attention to another action expert that leverages flow matching to generate robot actions, or controls to be sent to a low-level joint control system. This dual architecture has been mimicked by other research teams such as in [17] from NVIDIA. Although VLAs have shown strong performance in terrestrial settings, there is no standing literature describing how they perform in a marine environment, and no standing literature successfully implementing VLAs on an ROV.

Data Collection Strategies

The most popular data source for training models continues to be teleoperation, in which a robot manipulator is teleoperated by a human via a leader-follower control system [18], or VR system [19]. These systems generate high-quality data because the data consists of actual robot trajectories in the identical format the models are being trained to generate. Other approaches, like capturing trajectories from videos of humans completing tasks, are typically noisy and require a difficult re-mapping of the human hand’s trajectory to a robot’s end-effector trajectory. However, much progress is being made in this area to increase the efficiency of data collection [20]. To note, the data sets generated with large scale human video data have not been collected for use in the underwater domain. When underwater data is collected, it is from human teleoperated demonstrations (e.g., [3], [4]).

Adapting BC to Underwater Contexts

BC models for robotic manipulation have enabled increasingly generalizable robotic behaviors on terrestrial tasks. Action Chunking Transformers (ACT) [2] learns complex manipulation from low-cost demonstrations, Diffusion Policy [10], SmolVLA [21], and $\pi_0$ [16] extends this idea via large-scale pretraining, and DynamicVLA [22] improves inference efficiency for dynamic tasks by overlapping inference and execution across action chunks.

Other works adapt BC beyond their training distribution: AirVLA [23] modifies $\pi_0$ [16] actions via a physics-aware, embodiment-specific module, UMI-Underwater [4] adapts land-trained diffusion policies underwater with a depth-based affordance predictor and land-data collected from a hand-held device meant to replicate a robot’s end-effector [24], and AquaBot [3] shows learning-based policies can improve through underwater data collection. These works demonstrate the utility of BC underwater; however, they all lack underwater demonstrations with full VLA capabilities, such as the ability to command the robot with text based instructions to execute one of many underwater tasks that the single model is capable of. Recent work in [25], [26], and [27] has attempted to leverage simulation data in the pursuit of underwater BC, given that ocean data is not accessible for many researchers. While synthetic data from simulation also provides another scalable source of underwater training data [25], many tasks resulting from simulation data suffer from the sim2real gap [28], and typically fails to capture the complexity of the underwater world. Results from [26], [27] are notably not evaluated in the real world. Results from [3] are derived from human teleoperated data and tested in a pool, but are limited to a single task due to the diffusion policy [10] backbone, and cannot perceive color due to their reliance on depth instead of features. Given the standing literature, fine-tuning multi-task models using underwater data and executing these models underwater to perform complex manipulation tasks remains a critical challenge.

Methodology

This work presents some of the first applications of BC to underwater environments. Documented below are the different data sets collected and robot control policy model architectures trained with such data, all motivated to provide autonomous control of underwater robots for underwater tasks in underwater environments.

Data Collection

Robot demonstrations were collected across 3 tasks in an in-lab pool, as well as 2 tasks in an ocean setting.

1. Pool Deployment: To investigate how BC-based policies perform underwater in a controlled environment, three task families were explored (see Figure 2): grab, approaching an object and grasping it with the end-effector to simulate sampling a specimen, moor, approaching a stationary post or rope and latching on with the end-effector to simulate a docking procedure, and remove, approaching a ring on a post and unhooking it to simulate a rope line release procedure.

In total, 1616 human-teleoperated episodes were collected on these tasks (see Figure 3), which are split between calm, turbulence free episodes (established as the ”control”) and turbulent episodes created from a human volunteer manually agitating the water to create a sloshing, ”wave” like motion.

2. Marine Deployment: To examine the performance of BC-based policies in real ocean conditions, two task families were explored (see Figure 4): grab, approaching an object and grasping it with the end-effector, and moor, approaching a stationary post or rope and latching on with the end-effector to simulate a docking procedure.

In total, 748 human-teleoperated episodes were collected across three sites, progressing from a preliminary controlled test pool set (79 episodes, which is excluded from the Pool Study) to the Coralita X127 shipwreck (335 episodes total) and finally to Manoel Island Marina, Malta (334 episodes total). To improve dataset robustness, our collection pipeline includes recovery trajectories obtained by resampling directly from failure states. To make our policies more robust for field deployment, both data collection and testing rollouts occurred in the daytime and nighttime (see Figure 4).

Model Descriptions

Unlike fixed-base manipulators, ROVs are continuously jostled by waves and currents across all six degrees of freedom, shifting the camera viewpoint and the vehicle’s pose relative to the target between consecutive actions. As a result, a stationary object to be manipulated in a world frame

appears to move in the robot frame. In especially turbulent waters, free-floating manipulation objects tend to sway or move as well, adding another dimension of uncertainty to a policy’s predicted trajectory. This turns an otherwise static task into a dynamic one that penalizes any policy too slow to react. As such, three different model architectures were explored, each of which has a promise of faster inference time, a characteristic necessary for dynamic environments. As well, new observation and action spaces are needed for underwater robots. Notably, the use of these observations and

#	Observation Space	Obs. Dim.	Grab the rod succ.	Grab the rod part.	Grab the rod %	Grab the rod % w/ part.	Moor on Post %	Remove Ring %	Overall %	Overall % w/ part.	Latency (ms)
#1	none (images only)†	0	8	12	26.7	66.7	60.0	53.3	46.7	60.0	101.7 ± 1.5
#2	attitude, (control dataset)	3	4	8	13.3	40.0	60.0	93.3	55.6	64.4	101.0 ± 0.6
#3	attitude	3	10	19	33.3	96.7	86.7	96.7	72.2	93.3	95.2 ± 5.0
#4	attitude + $\Delta$ tokens (1.2 s)	3	8	14	26.7	73.3	63.3	46.7	45.6	61.1	102.4 ± 1.2
#5	attitude + $\Delta$ tokens (2.0 s)	3	6	8	20.0	46.7	50.0	93.3	54.4	63.3	100.0 ± 0.5
#6	attitude + $\Delta$ tokens (4.0 s)	3	11	16	36.7	90.0	60.0	66.7	54.4	72.2	102.8 ± 1.9
#7	attitude + $\Delta$ tokens (6.0 s)	3	5	18	16.7	76.7	40.0	43.3	33.3	53.3	128.7 ± 2.8
#8	attitude + raw $\Delta$	6	9	15	30.0	80.0	36.7	76.7	47.8	64.4	115.4 ± 11.6
#9	attitude + IMU	13	10	12	33.3	73.3	73.3	66.7	57.8	71.1	111.9 ± 9.9
#10	attitude + IMU + $\Delta$ tokens	13	12	9	40.0	70.0	40.0	73.3	51.1	61.1	116.4 ± 12.0
#11	attitude + IMU + raw $\Delta$	16	9	13	30.0	73.3	30.0	76.7	45.6	60.0	113.5 ± 11.5
#12	attitude + IMU + thruster PWM	19	11	9	36.7	66.7	46.7	66.7	50.0	60.0	98.7 ± 1.6
#13	attitude + IMU + PWM + $\Delta$ tokens	19	9	16	30.0	83.3	50.0	93.3	57.8	75.6	102.6 ± 0.7
#14	attitude + IMU + PWM + raw $\Delta$	22	10	15	33.3	83.3	50.0	73.3	52.2	68.9	113.9 ± 11.8

actions are novel within the VLA research space.

1) Model Architectures: The three model architectures explored for this work include ACT [2], SmolVLA [21], and DynamicVLA [22]. Chronologically, ocean deployments occurred before the pool deployments, and all models trained for the ocean were trained on site to reflect realistic field situations. Specifically, one ACT and one SmolVLA policy per task family (from the data set collected) were trained and evaluated on-hardware in the ocean. ACT and SmolVLA were chosen for their ease of use, with ACT being faster to train on site versus SmolVLA.

For in-lab pool experiments, where a more thorough exploration and ablation studies could be performed, the model architecture DynamicVLA [22] was chosen due to its low computational overhead and reactivity to target object motion and environmental disturbances. Figure 6 illustrates the DynamicVLA architecture, but with a different observation space and action space.

2) Observation Spaces: While most aspects of the model architectures do not differ from those published in [2, 21, 22], the observation spaces and even video data content differ greatly.

For the two model architectures trained for ocean deployment, a six-dimensional observational space was used that includes: roll, pitch, yaw, depth, depth set-point, and the depth controller’s output, plus RGB video (see Figure 1).

For the DynamicVLA models trained for in-lab pool deployments, a greater observation space was considered. At each control step during an episode of data, the ROV reports a 19-channel telemetry row over MAVLink at 50Hz, from which the maximal observation space is $o_t = [\phi, \theta, \psi, a_x, a_y, a_z, \omega_x, \omega_y, \omega_z, v_z, \mathrm{vib}_x, \mathrm{vib}_y, \mathrm{vib}_z, u_1, u_2, u_3, u_4, u_5, u_6]^T$, where:

1. The ATTITUDE block consists of roll ($\phi$), pitch ($\theta$), and yaw ($\psi$) (rad), forming a 3-dimensional observation space.

2. The IMU block adds ten channels: body-frame linear accelerations $a_x$, $a_y$, and $a_z$ ($\mathrm{m}/\mathrm{s}^2$) from raw IMU readings; EKF-fused angular rates $\omega_x$ (rollspeed), $\omega_y$ (pitchspeed), and $\omega_z$ (yawspeed) (rad/s) from ATTITUDE; vertical velocity $v_z$ (climb) (m/s) from VFR HUD; and the three vibration measurements $\mathrm{vib}_x$, $\mathrm{vib}_y$, and $\mathrm{vib}_z$ ($\mathrm{m}/\mathrm{s}^2$) from VIBRATION. Combined with the ATTITUDE block, this forms a 13-dimensional observation space.

3. The THRUSTER PWM block adds six commanded thruster outputs $u_1$, $u_2$, $u_3$, $u_4$, $u_5$, $u_6$ from the raw server output ($\mu\mathrm{s}$), with a neutral value of 1500 $\mu\mathrm{s}$. Combined with the ATTITUDE and IMU blocks, this forms a 19-dimensional observation space.

From these observations different combinations are selected to evaluate their effect on policy performance.

To investigate how turbulence affects VLA performance, a 3-dim disturbance channel, raw $\Delta_{x,y,z}$, is derived alongside the 19-dim observation space. At each timestep, the ROV’s expected acceleration was estimated from its orientation and thruster commands. The difference between this expected acceleration and the measured acceleration was treated as the environmental disturbance:

• $a(t) = [a_x, a_y, a_z]$: measured linear acceleration.

• $G_{\mathrm{orient}}(t)$: acceleration caused by gravity given the ROV’s orientation.

• $u(t) = \operatorname{PWM}(t) - 1500$: centered thruster commands.

• $\Phi$: mapping from thruster commands to acceleration.

• $\Delta(t)$: estimated environmental disturbance.

However, a single frame of $\Delta$ carries no turbulence history. Because ocean disturbances are low-frequency and periodic, turbulence only becomes legible over a window of frames. Rather than feed this window in as raw floats, the window is then tokenized with the FAST tokenizer [29]. This gives the policy a compressed representation of the various frequency components of the perturbations.

14 total DynamicVLA models (Figure 6) were trained on the pool data with a variety of observation space combinations (see Table I), with the smallest observation space being zero (the baseline model) and the largest space being 22 observation space dims. + 39 dims. for $\Delta$ tokens. $\Delta$ token windows were also varied, from 1.2 seconds to 6.0 seconds. The goal of this ablation study is to investigate what telemetry observations best train DynamicVLA (a VLA built solely on terrestrial data) to succeed underwater.

3) Action Spaces: The action space for the models is a 6-dimensional action space defined as $a_t = [v_x, v_y, \omega_z, z_{\mathrm{ref}}, c_{\mathrm{tilt}}, e_{\mathrm{ref}}]^{T}$ where:

• Horizontal velocity commands along the body frame $x$ and $y$ axes are given by $v_x$ and $v_y$.

• Yaw rate is denoted by $\omega_z$.

• The vertical altitude setpoint is $z_{\mathrm{ref}}$, which is tracked by a closed-loop PID controller for stable depth-following (defined as a dive action).

• The camera tilt toggle command is denoted by $c_{\mathrm{tilt}}$.

• The end-effector setpoint is represented by $e_{\mathrm{ref}}$.

The stable controller compares the depth setpoint $z_{\mathrm{ref}}$ with the depth read by one-board ROV pressure-based depth sensor (a BAR30) to establish dive.

For pool deployments, the dive action was rendered dormant as the pool is shallow, and all pool tasks were performed at equivalent depths. For ocean deployments, the ”dive” action was active as opposed to dormant. Unlike a pool where the water is shallow, Coralita X127 is anywhere between 8 meters and 25 meters in depth, and both grab and moor were performed on an incline and at varying depths.

Experiment Results and Discussion

Hardware and Software

Experiments used a Blue Robotics BlueROV2 (a vector-configured mini-ROV with six T200 thrusters, see Figure 1) with the Newton Gripper end-effector installation, controlled via a Raspberry Pi 4 companion computer with visual feedback from a front-facing 1080p wide-angle camera. The BlueROV2 is configured with the BlueOS standard operating system. The BluEmbodied Kit is used to collect a LeRobot [30] dataset, hand-annotate the data, launch inference through a tethered configuration to computer, record rollout data, and hand-label model rollout successes/partials/failures. The BlueROV2 was controlled using an XBOX controller in the field and through a keyboard in the pool.

Pool Deployment Results

Overall, ten models beat the visual-only baseline when partial successes are included, (see Table I). The highest performer in both overall task success and forward pass latency, i.e. ATTITUDE trained on both control and turbulent pool episodes, achieved a 93.3% success rate including partial successes, 17.7 percentage points higher than its runner-up, ATTITUDE + IMU + PWM + $\Delta$ Tokens, at 75.6%. ATTITUDE also has a mean forward pass latency of 95.2 milliseconds, 3.5 milliseconds faster than its runner-up, ATTITUDE + IMU + PWM (see Table I). ATTITUDE, however, is not the fastest model in terms of successful task time to completion: ATTITUDE + IMU + PWM outperformed ATTITUDE by 0.4 s by a pooled task average.

More proprioception observations did not necessarily lead to better performance: for calm-water manipulation, a smaller observation space provided a better balance between prediction speed and task success, while larger observation spaces, specifically with the addition of IMU and PWM observations, led to faster overall task completion. Similarly, a 4 second $\Delta$ token window enabled better performance than other window lengths for a controlled environment. There is also found no apparent relationship between manipulation success and color confusion, which can imply that differences in manipulation success is not due to the differences in perception between models, but the observations fed into the models directly. This visual confusion in underwater policies is aligned with what was found in [3], which visual occlusion, warping, and light dissipation can degrade VLA perception underwater. While this can be said about how DynamicVLA performs in our pool in calm waters, limitations of utilizing the pool includes the depth, size, and lack of realistic currents in the pool testing environments.

Marine Deployment Results

Our results of the method described in Figure 1 can be summarized by Table II, which reports success by task and location over 508 gradable autonomous policy rollouts. To test whether a single trained policy could generalize across physically distinct instances of the same task family, one fixed ACT checkpoint was evaluated on three mooring variants without any per-site fine-tuning: on a ladder at the Marina, on a tethered rope on the Shipwreck, and on a post on the Shipwreck (see Figure 4). Success rates were 56.3%

Task	Location	Policy	$n$	SR (%)	95% CI
Grab (red rod)	Shipwreck	ACT	150	33.3	[26.3, 41.2]
Grab (red rod)	Marina	ACT	14	42.9	[21.4, 67.4]
Grab (red rod)	Marina	SmolVLA	28	0.0	[0.0, 12.1]
Grab (green rod)	Pool	ACT	22	63.6	[43.0, 80.3]
Moor (ladder)	Marina	ACT	206	56.3	[49.5, 62.9]
Moor (ladder)	Marina	SmolVLA	48	12.5	[5.9, 24.7]
Moor (rope)	Shipwreck	ACT	21	66.7	[45.4, 82.8]
Moor (post)	Shipwreck	ACT	19	36.8	[19.1, 59.0]

($n$=206) on the ladder, 66.7% ($n$=21) on rope, and 36.8% ($n$=19) on the mooring post (Table II). The comparable success rates on the ladder and rope tasks suggest that the policy learned a site-agnostic mooring strategy rather than overfitting to the Marina ladder’s specific visual features. The lower success rate observed for the post likely results from its less forgiving target geometry compared to the ladder and rope, even though both the rope and post exhibited greater occlusion from algae and marine overgrowth than the ladder.

ACT consistently outperformed SmolVLA on every task family (Table II) pooled across sites: 37.6% versus 0.0% on grab and 55.7% versus 12.5% on moor. The gap is largest on the tasks with the most training data: at the Marina ladder, ACT reached 56.3% on mooring versus SmolVLA’s 12.5%, and 42.9% versus 0.0% on grab red rod, where SmolVLA never succeeded. ACT’s best grab result was grab green rod in the pool (63.6%), which is expected given the controlled environment.

When pooled across all tasks and locations, ACT succeeded on 47.9% of rollouts and demonstrated incredible recovery trajectories during trials, versus 7.9% for SmolVLA which demonstrated no recovery capabilities. ACT produces short, temporally coherent action chunks that are replanned frequently, keeping execution close to closed-loop. This appears to grant comparative robustness to disturbances that are largely absent from the terrestrial data SmolVLA’s backbone was pretrained on. SmolVLA, a substantially heavier model, both replans more slowly and was trained on far less task-specific underwater data than our ACT model, so the performance gap likely reflects inference latency and data quantity rather than a fundamental architectural ceiling.

The inclusion of ”dive” in the action space enabled both models to perform tasks at various points on the shipwreck (along the sloping deck and at various points against the hull along the port side), by allowing the ROV to adjust its depth during execution. Sensitivity to depth results in emerging behaviors like recovery maneuvers and greater initial approach precision.

Both tasks evaluated across daylight variations show a consistent performance drop at night. The moor on ladder success rate fell from 63.9% ($n=97$) in the daytime to 49.5% ($n=109$) at night, and the grab red rod success rate fell from 42.4% ($n=66$) to 26.2% ($n=84$) respectively. The drop was larger for grasping, which is consistent with the grasping trajectories being less forgiving to disturbances: unlike mooring, which tolerates vertical error since the ROV can latch anywhere along the ladder or post, grasping the small, swaying rod leaves little margin for error, especially with the reduced contrast and noise of nighttime.

Conclusions and Future Work

This work demonstrates the applicability of VLAs and related models to underwater robot tasks. Results from both pool and marine deployments demonstrate that choices in model architecture and observation space matter when adapting ROVs to VLAs, and VLAs to underwater environments. By experimenting with novel action spaces like depth, models like ACT were able to successfully execute tasks in marine archaeological sites. Observation spaces in multi-task models, such as the addition of $\Delta$ tokens or the simplicity of ATTITUDE, can encourage emerging underwater behaviors like speediness, success rates, or forward pass latency speed. The inclusion of an expanded observation space, therefore, enabled better results than the baseline, despite a backbone trained on terrestrial data alone. This suggests that what we give a policy to observe can be just as important as the model itself, and that an underwater backbone is not necessary for multi-task VLA success. Together, these results suggest that effective underwater VLA deployment depends not only on the choice of model, but on how the model’s action and observation spaces are adapted to the physical environment.

The following is recommended for future work: Expanding the BLUEMBODIED KIT repository to include training scripts for a broader set of single and multi-task BC architectures, evaluating said architectures to determine which ones find themselves robust to marine environments, collecting additional trials across Coralita X127 and other wrecks, evaluating longer-horizon tasks such as tether untangling, and progressively extending deployments to greater depths. In addition, investigating domain generalization (whether data collected on land by terrestrial robotics can be successfully mixed with underwater data for underwater VLA training, and vice versa) is a crucial challenge to confront the difficulty of collecting real world underwater data. An investigation of the application of VLAs in untethered AUVs would also be an interesting challenge which can yield beneficial results for the underwater field robotics community. Together, these directions will move the robotics community toward the broader goal of enabling robust, generalizable, and increasingly autonomous embodied intelligence for real-world underwater robotics.

Acknowledgment

The authors would like to thank Heritage Malta for their support during real-world underwater deployments, Dr. Nana Obayashi and the Prema Lab for their generosity in pool access, and Talula Clark for their field assistance.