WorksComputer Science
MOMASPLAT: Dynamic Novel-View Demonstration Augmentation for Mobile Manipulation
Abstract
Visuomotor policies trained through imitation learning are highly sensitive to changes in camera viewpoint. This is particularly problematic for mobile manipulation (MoMa), where robots rarely execute manipulation from the exact same base pose used during demonstration collection due to navigation uncertainty, localization drift, and environmental constraints. Recent approaches use novel-view synthesis (NVS) to augment demonstrations with unseen viewpoints, but they assume static scenes and therefore fail to re-render the manipulated object during the post-contact interaction phase, leaving the most critical parts of the demonstration unaugmented. We present MOMASPLAT, a novel-view demonstration augmentation framework for mobile manipulation that synthesizes full manipulation trajectories from unseen robot base poses, including dynamic object interactions. Our key insight is that standard MoMa data collection naturally provides the multiview observations needed to reconstruct both the static scene and the manipulated object in 3D. MOMASPLAT combines 3D Gaussian Splatting, feed-forward 3D foundation models, and kinematics-based action retargeting to separately reconstruct the scene and manipulated object, render them from novel viewpoints, and generate geometrically consistent synthetic demonstrations paired with transformed actions. We evaluate MOMASPLAT on five real-world mobile manipulation tasks involving both rigid and articulated objects across three scenes, and validate the fidelity of its synthesized demonstrations against ground truth in simulation. MOMASPLAT substantially improves robustness to unseen base poses, more than doubling the average success rate of the strongest baseline and maintaining non-trivial success at large base displacements where all baselines fail, while reducing the need for largescale real-world data collection.
INTRODUCTION
Imitation learning has emerged as a powerful paradigm for training visuomotor policies that perform complex manipulation tasks directly from image observations [1], [2]. However, these policies are notoriously sensitive to changes in camera viewpoint [3]–[5]. Even small deviations in camera pose can significantly degrade performance, since the policy learns a narrow visual distribution tied to the demonstrations used during training.
This sensitivity is particularly problematic for mobile manipulation (MoMa). Unlike tabletop manipulation setups, where demonstrations are often collected from a fixed robot camera pose, mobile manipulators rarely execute tasks from the exact same location at deployment time. Navigation
uncertainty, localization drift, obstacle avoidance, and imperfect planning routinely place the robot at different base poses relative to the workspace. As a result, the robot observes the manipulation scene from viewpoints that may differ substantially from those seen during demonstration collection. Robust deployment therefore requires visuomotor policies that generalize across a distribution of robot base poses rather than a single viewpoint.
A straightforward solution would be to collect demonstrations from many different robot locations. However, large-scale real-world teleoperation is expensive and difficult to scale [2], [6], [7]. Recent work has therefore explored synthesizing additional demonstrations through novel-view synthesis (NVS), generating virtual observations from unseen viewpoints using real-world captures [8]–[10]. While promising, existing approaches assume a static scene and therefore fail during the most important part of manipulation: object interaction. Since the manipulated object moves throughout the demonstration, prior NVS methods cannot consistently re-render the object from novel viewpoints during the post-contact interaction phase, restricting augmentation primarily to pre-contact motions or introducing geometrically inconsistent generations.
In this work, we present MoMASPLAT, a demonstra-
tion augmentation framework for mobile manipulation that synthesizes complete manipulation trajectories from unseen robot base poses, including dynamic object interactions (Figure 1). Our key insight is that standard mobile-manipulation data collection naturally provides the multi-view observations needed to reconstruct both the static scene and the manipulated object in 3D. During navigation, the robot observes the scene from diverse viewpoints that enable scene reconstruction, while the manipulation demonstrations themselves provide object-centric observations that enable reconstructing the manipulated object and its motion.

Figure 1. MOMASPLAT. Starting from mobile-manipulation data collected at the original base pose (white and red robot), MoMASPLAT synthesizes demonstrations from sampled novel base poses (yellow-outlined gray robot). The top-right sequence shows the source manipulation demonstration, while the bottom-right shows the corresponding synthesized trajectory, including the dynamic object as it is moved by the robot. Augmenting the source manipulation data with these demonstrations enables execution from unseen deployment-time base poses (blue-outlined gray robot).
Building on this observation, MoMaSplat separately reconstructs the static scene and manipulated objects using 3D Gaussian Splatting [11] and feed-forward 3D foundation models [12]. Given a novel robot base pose, MoMaSplat re-renders the scene, manipulated object, and robot from the new viewpoint and retargets the original actions through kinematics-based transformations to generate geometrically consistent synthetic demonstrations.
We evaluate MoMaSplat on five real-world mobile manipulation tasks involving both rigid and articulated objects across three scenes, and validate the fidelity of its synthesized demonstrations against ground truth in simulation. Our experiments show that MoMaSplat substantially improves robustness to unseen deployment-time base poses, outperforming prior augmentation approaches across 12 unseen base poses (3 rollouts per pose) per task as the robot moves farther away from the source demonstration locations.
Our contributions are threefold:
A novel-view demonstration augmentation framework for mobile manipulation that synthesizes complete trajectories, including dynamic object interactions.
A geometry-consistent generation pipeline that separately reconstructs scenes and manipulated objects and retargets actions to unseen base poses.
Substantial improvements in robustness to deployment-time viewpoint variation across multiple real-world mobile manipulation tasks.
Related Work
Our method proposes a solution to train imitation learning visuomotor policies for mobile manipulators via data augmentation. Here, we review the most relevant related work in those areas.
Demonstration Synthesis in Simulation. A prominent line of work synthesizes new demonstrations by replaying source actions in randomized simulated scenes. The X-Gen family [7, 13, 14, 15, 16] procedurally generates data by transforming and sequencing object-centric trajectory segments, with extensions such as MoMaGen [15] addressing mobile-manipulation constraints like base reachability and visibility. However, because the generated plans are purely kinematic, they typically rely on open-loop on-robot rollouts to verify feasibility and capture visual observations, at a cost that rivals manual data collection [8, 17]. A parallel line of work uses simulators to generate data from multiple viewpoints [18, 19, 20], but requires modeling the full environment and task dynamics in simulation, suffering from the visual and physical sim-to-real gap. Our work avoids on-robot rollouts and full-scene simulation by reconstructing the real scene and manipulated object via novel-view synthesis, using simulation only to render the known robot embodiment.
Demonstration Augmentation from Real-World Captures. Beyond 2D image augmentation [21, 22, 23], recent methods synthesize observations from real-world captures. Neural simulators [24, 25, 26] use generative video models but can suffer from hallucinations and geometric inconsistencies. Alternatively, novel-view synthesis (NVS) enables viewpoint augmentation [9, 10, 17, 27, 28, 29, 30, 31]. VISTA [27] uses ZeroNVS [32], while Yang et al. [9] employ 3D Gaussian Splatting [11]; however, both focus on third-person tabletop settings requiring multi-view pre-scans. Closer to our setting, One- Thousand Trajectories [8] supports moving eye-in-hand cameras, while EgoDemoGen [33] generates paired egocentric observations and actions using a generative model. However, these methods do not explicitly reconstruct dynamic objects and restrict augmentation to pre-contact motion. In contrast, MoMaSplat jointly models the static scene and manipulated objects, enabling kinematically consistent augmentation across both pre-contact and post-contact interaction phases for mobile robots with egocentric cameras.
Mobile Manipulation. Mobile manipulation remains a key challenge in robot learning [34, 35, 36, 37, 38], particularly in selecting robot base poses. Traditional methods use geometric heuristics such as Inverse Reachability Maps [39, 40, 41], while learning-based navigation [42, 43, 44, 45] often overlooks downstream policy sensitivity to viewpoint changes. Mobiπ [3] selects base poses matching the training distribution but requires pre-scanning and test-time optimization. In contrast, we train policies robust to diverse base poses, reducing the need for precise base placement at deployment.
MoMaSplat
We aim to augment a mobile-manipulation dataset with synthetic demonstrations from unseen robot base poses. From source demonstrations at a few robot locations, MoMaSplat synthesizes observations and actions for nearby deployment-time poses never physically demonstrated.
To achieve this, MoMaSplat decomposes the environment into three components: i) a static scene model, ii) a dynamic object model for the manipulated object, and iii) the robot embodiment itself. The scene and object models are reconstructed independently and aligned into a shared metric frame. A key observation is that the navigation trajectories naturally provide the bridge between both reconstructions: during navigation, the manipulated objects remain static, allowing the object-centric and scene-centric trajectories to be consistently registered.
For augmentation, given a sampled robot base pose, MoMaSplat renders the scene, manipulated object, and robot from the corresponding viewpoint and retargets the demonstrated actions through kinematic transformations. The resulting observations and transformed actions form synthetic demonstrations that augment the dataset for policy training. Figure 2 provides an overview of the full process.
Figure 2. Overview of MOMASPLAT. 3D Reconstruction: We build 3D models of the environment from the navigation and manipulation data. We estimate camera poses via structure-from-motion and fit a 3D Gaussian Splatting model of the static scene, . For the dynamic objects, we segment each manipulated object and estimate its per-frame pose trajectory with a feed-forward 3D foundation model; since the scene and object poses lie in separate reference frames, we align them with a Sim(3) transformation and fit a separate object model . Synthesizing Demonstrations from Unseen Base Poses: Given a robot base pose, , sampled around the original demonstration poses (left), we render the scene and object from the new viewpoint, transform the source end-effector trajectory into the new base frame to render the robot from a simulator, and composite the three streams into a photorealistic novel observation. Paired with the transformed actions , this forms a synthetic demonstration; repeating across sampled poses yields the generated dataset. Imitation Learning: the policy is trained on the union of source and generated demonstrations.
Problem Formulation. We consider an imitation learning setting on a mobile manipulator operating in a scene. The robot alternates between two modes of operation: navigation, which changes the robot base pose, and manipulation, which executes arm actions from a fixed base location. We represent the robot base pose as a transform with respect to a stationary world frame , parameterized by a planar position and heading . The robot observes the scene through an egocentric RGB camera mounted on the robot body (e.g., head or torso). The camera pose relative to the robot base is determined by the robot configuration through forward kinematics, such that both changes in base pose and articulation of the robot body induce changes in viewpoint.
During data collection, the robot navigates to a predefined set of target base poses:
from which manipulation demonstrations are collected. The robot executes navigation trajectories from different starting locations toward the target base poses, either teleoperated or autonomously, and performs manipulation demonstrations from each fixed base pose without further base motion.
This process produces two complementary datasets. First, we collect a navigation dataset , where denotes the number of navigation trajectories and each trajectory contains RGB observations and robot states recorded while navigating toward one of the target base poses. These trajectories provide diverse multi-view observations of the environment and manipulated objects before interaction. Second, we collect a source manipulation dataset of demonstrations, where each trajectory contains manipulation observations and actions recorded during task execution from one of the base locations. We represent actions as 7-D commands , consisting of desired end-effector translation and rotation deltas and applied in the robot base frame, together with the gripper command .
Our goal is to augment the source manipulation dataset with a synthetic dataset of demonstrations generated at unseen base poses to obtain a base-pose robust policy trained on . To achieve this, MoMaSplat first reconstructs the scene and manipulated objects in 3D and aligns them into a shared metric frame (Sec. III-A). It then synthesizes demonstrations from sampled base poses by rendering the scene, object, and robot from the corresponding viewpoint and retargeting the actions and proprioception to remain geometrically consistent (Sec. III-B). The synthesized demonstrations are combined with the source dataset for policy training (Sec. III-C).
3D Reconstruction
To synthesize demonstrations from unseen robot base poses, MoMaSplat requires a 3D representation that can be rendered from arbitrary viewpoints. A central challenge is that manipulation involves dynamic objects whose pose changes throughout the demonstration. Unlike prior novel-view synthesis approaches that model the environment as a single static scene, MoMaSplat reconstructs the static scene and manipulated objects separately, leveraging observations naturally produced during mobile-manipulation data collection. Navigation trajectories cover the environment while the objects are static, and manipulation trajectories capture the objects in motion.
3D Scene Model Reconstruction. As the robot moves through the workspace, its camera naturally captures diverse viewpoints, providing the multi-view coverage needed for 3D reconstruction without external cameras or scanning. We therefore use to build a photorealistic 3D scene model: we recover per-image camera poses with structure-from-motion via COLMAP [46], which also produces a sparse point cloud that initializes a 3D Gaussian Splatting (3DGS) reconstruction [11].
We optimize the Gaussians by minimizing a photometric loss between rendered and observed images. Since the scene is not perfectly static across navigation trajectories, transient content may appear between demonstrations; we adopt the uncertainty-aware formulation of WildGaussians [47], which down-weights inconsistent regions during optimization to yield a robust model . We additionally mask out pixels belonging to the manipulated object and the robot during scene reconstruction, preventing the object from being baked into and avoiding a static “ghost” when the independently reconstructed dynamic object is later composited into the scene.
Structure-from-motion reconstructs the scene only up to an unknown global scale, while our framework transforms metric actions and therefore requires a metric reconstruction. We recover the scale using the robot’s onboard depth sensor: for the frames in , we compare the measured depth maps to depth maps rendered from and take the median per-pixel ratio of measured to rendered depth as the scale factor.
3D Object Model Reconstruction. Since the object of interest moves independently from the scene, we build a separate 3DGS model for it. Conceptually, the steps mirror those for the scene: we estimate the object’s pose in each frame and use these poses to optimize a 3D Gaussian reconstruction. However, the challenges here are different. Correspondence-based methods [46] are poorly suited to small, weakly-textured objects, while instance-level pose trackers [48] require an existing 3D model of the object, which is not available to us. Instead, we find that our setting is amenable to recent feed-forward 3D foundation models, like VGGT [12, 49], which estimate camera poses directly from an unposed RGB sequence.
Specifically, for each trajectory and source demonstration , we extract the RGB frames
and , respectively, and obtain per-frame segmentation masks and for the object using an open-world segmentation pipeline [50, 51], which masks out the background and the robot.
Then, we apply FastVGGT [49] to the object-masked frames and , respectively. With everything else masked out, the model predicts a camera pose at each frame that is expressed relative to the object itself. These object-relative camera poses act as the initial pose estimates in the model’s internal coordinate frame .
Frame Alignment. The estimates lie in , with arbitrary scale, orientation, and origin relative to the metric world frame established by the Scene Reconstruction. To align the two frames, we use a set of frames from the navigation demonstrations as a bridge. During navigation the object remains static, so the only motion comes from the robot itself, i.e., the pose trajectory recovered by the object model is purely due to camera motion. This is not true for the manipulation frames: the object moves, so the recovered trajectory conflates camera and object motion. On the selected navigation frames, the same cameras are posed twice, by the feed-forward model in and by structure-from-motion in . The two trajectories must describe the same motion up to an unknown Sim(3) transformation. We recover this transformation in closed form via
the Umeyama algorithm [52]. Applying to the estimated poses yields the aligned pose trajectory in the world frame, .
Object 3DGS Optimization. Finally, we jointly optimize these aligned poses alongside the Gaussian parameters of , backpropagating the photometric loss through both. This yields a pose-disentangled representation that can be rendered at arbitrary 6-DoF poses for compositing into novel-pose observations. For articulated objects, we model the moving rigid link separately from the static remainder of the object. For example, a laptop lid or the basket of an air fryer is represented by , while the keyboard base or the air fryer’s surrounding body remains part of the static scene. The component represented by therefore undergoes a rigid 6-DoF motion over the demonstrated trajectory, and we do not estimate articulation parameters or extrapolate beyond the demonstrated motion.
Synthesizing Demonstrations from Unseen Base Poses
Given the scene and object models, our framework synthesizes new demonstrations at unseen base poses in three stages. For each original robot base pose, , we first sample a set of target base poses that cover possible deployment-time locations while remaining kinematically feasible (e.g., reachable for the arm). We then synthesize each source demonstration from each sampled pose by composing three separately rendered streams: the static scene (from the 3D scene model), the manipulated objects (from the 3D object model), and the robot body as seen from the new base location. Finally, we retarget the source action to be defined with respect to the virtual sampled base location to obtain a new pair of virtual image and action label.
Sampling Base Poses. To generate synthetic demonstrations, we sample a set of target base poses (here, we omit the superindex and subindex from for simplicity) that the policy should learn to handle around each of the original base locations, . We sample the planar position uniformly within a region surrounding the original base pose, chosen to cover the range of deployment-time poses the robot may reach. For each sampled position, we set the base heading so that the object of interest is centered in the egocentric camera’s field of view, then perturb it with uniform noise to expose the policy to a range of object-in-frame configurations. We then apply the transformation that transforms the original base frame, , to the sampled base frame, (where we lift from into with zero vertical translation and zero roll/pitch when transforming 3D poses and vectors). We retain a sampled base pose, , only if the source arm trajectory, where the end-effector remains at the same pose with respect to the manipulated object, admits a valid inverse-kinematics solution at several probe timesteps spanning the demonstration; infeasible poses are rejected and resampled. To obtain the end-effector pose in the sampled base frame, we apply to the source end-effector pose , obtained from through forward kinematics. The resulting pose, , preserves the end-effector pose in the world frame.
Novel View Synthesis. We virtually place the egocentric camera at its mounted offset relative to using forward kinematics and the robot’s recorded configuration, , and matching the physical camera’s intrinsics. We then render three elements from the new view: i) the scene, ii) the manipulated object, and iii) the robot. From the scene model, , we obtain a virtual RGB image , the depth map and the accumulated opacity . For the manipulated object, we transform to its source-demonstration pose and rasterize it from the same camera, yielding per-timestep renders , depth maps and opacity image . For the robot, we use inverse kinematics on the end-effector pose for the sampled base location, , to recover a joint configuration, , that would place the end-effector at the same location with respect to the object. Then, we apply the joint configuration to the robot and render it using OmniGibson [53] to obtain a robot-only RGB image , depth map and opacity mask .
To get a single RGB observation from the three virtual images, we need to combine them together, while respecting the depths and the opacities at each pixel. To achieve this, we use per-pixel alpha compositing. Specifically, at each pixel, we sort the three layers by depth, letting denote the color and opacity of the -th layer, with being the nearest. Then, we combine them in a front-to-back manner:
so that nearer surfaces occlude farther ones. This generates virtual RGB images as would be seen from a physical camera at sampled location .
Robot Action and Proprioception Retargeting. A change in base pose moves the coordinate frame in which both the robot’s end-effector proprioception and actions are expressed, so both must be retargeted to the sampled base frame . Because the policy conditions on end-effector proprioception in the robot base frame (consisting of the end-effector pose and gripper state), we transform the end-effector pose into the sampled base frame via (as above) while keeping the gripper state unchanged. Robot actions consist of desired end-effector translation deltas , rotation deltas , and gripper commands , where both translation and rotation deltas are defined and applied in the robot base frame (i.e., ). To retarget the desired end-effector translation delta, we apply the rotation component of so that expresses the same world-frame displacement in the sampled base frame. Because rotation deltas are applied in the base frame, conjugating by rotates its axis of rotation by while preserving the rotation angle; in axis-angle format , the retargeted rotation delta is therefore . The gripper command remains unchanged (), yielding the retargeted action . Pairing the virtual RGB images with the retargeted proprioception and actions across the trajectory provides a synthetic demonstration at .
Constructing the Augmented Dataset. For each of the source manipulation demonstrations in , we generate synthetic demonstrations from sampled novel base poses, producing a synthetic dataset of demonstrations that we combine with to form for policy training (Sec. III-C).
Imitation Learning
We train a visuomotor policy on the union of source demonstrations and synthetic demonstrations using Diffusion Policy [1], which models the conditional action distribution as a denoising diffusion process and is trained by minimizing the standard noise prediction loss:
where is the ground-truth action sequence noised at denoising step with Gaussian noise . The policy conditions on the RGB observation and end-effector proprioception, and outputs an action sequence of end-effector pose deltas. To balance the two datasets during training, we co-train with a sampling ratio of 0.25 for and 0.75 for , anchoring the policy to the source data while leveraging the broader pose coverage of the synthetic data.
Experimental Evaluation
We evaluate MoMaSplat on five real-world mobile manipulation tasks across three scenes, comparing against four baselines: no augmentation and three augmentation methods. Our experiments focus on a) whether MoMASPLAT improves policy success at unseen base poses, b) how performance varies with the distance between the deployment-time base pose and the closest demonstration pose, and c) how policy success changes with the number of source demonstrations available for augmentation. Additionally, we evaluate the quality of the generated data itself. This requires ground truth observations at poses where no demonstration was collected, which is impractical in the real world. A human teleoperator produces a different motion on every attempt, so even a re-collected trajectory cannot serve as ground-truth for the generated demonstration. Therefore, we develop one task in simulation, where camera and object poses are known and source trajectories can be replayed from arbitrary base poses.
Experimental Setup
Tasks. The robot performs (i) PICK-PLACE BOTTLE and (ii) OPEN DRAWER in the desk scene; (iii) PICK-PLACE REMOTE and (iv) CLOSE LAPTOP in the living room; and (v) OPEN AIR-FRYER in the kitchen (Figure 3). We additionally build (vi) SIM PICK-PLACE BOTTLE in OmniGibson [53], a simulated scene modeled after the desk environment that follows the same task structure as (i) without replicating its exact geometry. The tasks span rigid pick-place and single-DoF articulated manipulation. We collect 75 manipulation demonstrations for CLOSE LAPTOP and 90 for each other task across three base poses, forming . We additionally collect nine navigation trajectories from different starting poses to the target base poses.

Figure 3. Tasks and scenes used in our experiments. We evaluate MoMASPLAT on five real-world manipulation tasks across three scenes (top: scenes; bottom: per-task close-ups), covering manipulation of both rigid and articulated objects. The simulated scene and its task, SIM PICK-PLACE BOTTLE (right of the dashed lines), are used to measure the fidelity of MoMASPLAT’s generated demonstrations against ground truth.
Evaluation Protocol. For each task, we deploy the trained policy at 12 unseen base poses across the scene, with heading centered on the object, and perform 3 rollouts per pose.
Baselines. We compare against four baselines. Source Only (labeled “Original data only” in Figure 4) trains Diffusion Policy [1] without augmentation. Action Augmentation [54] adds noise to proprioceptive states and actions while preserving observations. VISTA [27] generates novel-view images without transforming actions. One-Thousand Trajectories [8] reconstructs the static scene using 3DGS; we additionally provide our action transformation to account for changes in the robot base frame. All methods train on the same source demonstrations. For PICK-PLACE BOTTLE, we restrict synthetic generation for One-Thousand Trajectories to demonstrations whose initial object pose matches the reconstructed scene, since its static reconstruction cannot represent variation in this pose.
Figure 4. Policy success across base poses. For each task, we evaluate 12 unseen base locations distributed across categorical distance bins that vary per task, with 3 trials per pose, and report success rate by distance to the nearest demonstration pose. The shaded gray region denotes the original base poses; locations 5–50 cm away are unseen deployment poses. MoMaSplat maintains substantially higher success as the base pose shifts away from the source demonstrations (where Source Only is labeled “Original data only”).
Data Augmentation Setup. Each method generates 10 synthetic demonstrations per source demonstration, yielding 750–900 synthetic demonstrations per task except One-Thousand Trajectories on PICK-PLACE BOTTLE, which generates 400. For MoMASPLAT and One-Thousand Trajectories, we sample base poses within a m region around each source pose (Sec. III-B). Frame alignment uses 50 frames from one navigation demonstration.
Results
Performance at Unseen Base Poses. Our main experimental results for policy success using MoMASPLAT are summarized in Figure 4, where success rates are binned by their minimum distance to the nearest demonstration pose. We additionally evaluate the 3 original demonstration base poses with 15 rollouts per pose. At the original base poses, all methods perform comparably. The picture changes sharply at unseen poses. When we average the results over all five tasks, MoMASPLAT succeeds on 37% of unseen evaluations, more than twice the rate of the strongest baseline (One-Thousand Trajectories at 17%) and four to six times that of the remaining baselines (Source Only 6%, VISTA 8%, Action Augmentation 6%). Critically, Source Only and Action Augmentation collapse to near-zero success beyond 10 cm on most tasks, confirming that perturbing actions without changing the observation gives little robustness to base-pose shifts. VISTA performs poorly throughout; since it is designed for static-base settings, it does not transform actions to match the synthesized viewpoints. Moreover, its lack of temporal consistency produces jittery camera trajectories that further degrade the augmented data. One-Thousand Trajectories degrades more gracefully but still falls off at large distances, since it augments only the free-space approach motion. MoMASPLAT achieves the highest overall unseen-pose success rate on every task, outperforming or matching all baselines across every unseen distance bin while sustaining substantial performance at the largest evaluated distances. For example, MoMASPLAT achieves 33% on PICK-PLACE BOTTLE at 30 cm and 50% on CLOSE LAPTOP at 40 cm, where every baseline has dropped to zero success rates. Finally, to better contextualize the geometry of our tasks and scenes, Figure 5 shows the spatial layout of these evaluation locations across each workspace.

Figure 5. Spatial distribution of policy evaluation poses. Top-down view of each task showing the table and target-object locations, the original demonstration base poses, and the test base poses used for policy evaluation. Original demonstration poses are marked with circles, while evaluation poses are marked with squares; arrows indicate the robot heading at each pose. Evaluation poses are colored by MoMaSplat policy success rate, providing spatial context for the distance-based results in Figure 4.
Effect of Number of Source Demonstrations. To evaluate how the number of source demonstrations affects MoMASPLAT, we use the PICK-PLACE BOTTLE task and train policies using 10, 50, and 90 source demonstrations. For each source demonstration, we generate 10 synthetic demonstrations. Across four test locations at a distance of 10–20 cm from the demonstration poses, we observe that success degrades gracefully with fewer source demonstrations.
Specifically, we achieve success rates of 33%, 42%, and 50% for 10, 50, and 90 source demonstrations, respectively. For reference, training on the full set of 90 source demonstrations without augmentation yields only an 8% success rate at the same test poses. These results show that MoMaSplat substantially improves base-pose robustness even with a limited number of source demonstrations.
Generation Quality. We first evaluate generation quality in simulation, where the ground-truth counterpart of a generated demonstration is obtained by executing its transformed actions in the simulator and rendering the on-board camera at every timestep. We use the SIM PICK-PLACE BOTTLE task and evaluate 75 generated demonstrations, taking every tenth frame for a total of 2,486 frames. Unlike the policy evaluations above, which measure distance between base poses, here we measure how far each generated view is from the views that were used to build the reconstruction. For the 75 demonstrations, we find this distance to be between 0.4 and 27.3 cm. Since the static reconstruction does not change over a demonstration, we score the scene separately by rendering the scene splat and the empty simulated scene from the same camera pose once per demonstration, rather than considering every frame. We report PSNR, SSIM, and LPIPS over the full image, the scene, and over the robot and object regions using the simulator’s segmentation masks. Table I reports the detailed results. The reconstructed scene achieves the highest fidelity at 26.40 dB, followed by the manipulated object at 18.18 dB and the robot at 17.43 dB. For reference, the original 3DGS work [11] reports roughly 23 dB PSNR on Tanks & Temples [55]. This places our scene reconstruction in the range of standard novel-view synthesis, while the lower object PSNR reflects the harder setting of reconstructing a moving object with limited viewpoint coverage. The robot error is mainly caused by lighting and shadow differences introduced during compositing. We also evaluate object-pose accuracy to measure how well our estimated object trajectory matches the true object motion of the observations. On 30 source demonstrations, including poses directly registered by VGGT and interpolated poses, the predicted object pose has a mean translation error of 1.46 ± 0.71 cm and rotation error of 4.01° ± 4.66° relative to the simulator ground truth.
On real data, we evaluate scene reconstruction by masking out the robot and the manipulated object, and we obtain 24.00 dB PSNR on 110 held-out frames. For the manipulated object, we use its bounding-box crop and we mask out the background. At poses directly registered by VGGT, we obtain 23.63 dB, while at interpolated poses we obtain 22.59 dB. We do not report full-image view-synthesis metrics on real data because the simulated robot differs in appearance from the real robot and this appearance mismatch dominates the pixel-wise error that the view synthesis metrics evaluate.
| Region | PSNR (dB) | SSIM | LPIPS |
| Full image | |||
| Scene | |||
| Object | |||
| Robot |
Table 1. Generated-observation fidelity against simulator ground truth. Mean std over 75 generated demonstrations spanning viewpoints with distances of 0.4–27.3 cm from the reconstruction views. Full image, object, and robot use every tenth frame (2486 total frames), with object and robot restricted to their masks; scene PSNR evaluation uses one frame per demonstration (75 in total).
Overall, the simulation results show that MOMASPLAT produces observations that closely match the ground truth, while the real-world results show high-fidelity scene reconstruction and consistent object reconstruction across both directly estimated and interpolated poses.
Conclusion and Limitations
Conclusion. We propose MOMASPLAT, a framework for novel-view augmentation of mobile manipulation demonstrations that reconstructs the static scene and manipulated object in 3D, then re-renders full demonstrations, including the post-contact interaction phase, from unseen base poses. Unlike prior NVS augmentation methods that model only the static scene, MOMASPLAT produces geometrically consistent observations across the entire trajectory. Across five real-world tasks in three scenes, this improves policy robustness, outperforming prior augmentation methods and maintaining performance beyond the distances where baselines fail. We will publicly release our code to support further research.
Limitations. MOMASPLAT assumes mostly static scenes and rigid, visible objects, making deformable objects, heavy occlusions, and texture-poor surfaces challenging. Furthermore, because is reconstructed from pre-interaction navigation views with object masks excluded, regions occluded by the object in its initial state (such as the surface beneath a lifted bottle or the interior of an opened drawer or air-fryer basket) are never observed during scene reconstruction and remain unmodeled when disoccluded. It also depends on consistent open-world segmentation and produces non-physical demonstrations by replaying source object trajectories without modeling contact forces or dynamics. Finally, view quality depends on navigation-data coverage. Addressing dynamic backgrounds, deformable objects, physical contact, and cluttered scenes remains future work.
Acknowledgment
This work was conducted in the Robot Interactive Intelligence (RobIn) Lab at The University of Texas at Austin and was supported in part by the National Science Foundation (CAREER Award 2543745), the NVIDIA Academic Grant Program, Amazon, and DARPA TIAMAT program (HR0011-24-9-0428). G. Pavlakos acknowledges support by NSF-2504906 and 2544200, and the NVIDIA Academic Grant Program. S. Kumar is supported in part by the Amazon AI PhD Fellowship program. The opinions, findings, and conclusions expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.
Appendix
Baseline Details
In this section, we provide more details about the baselines we are using in our evaluation.
Source Only. This baseline trains the same Diffusion Policy [1] as we do, but using only the source demonstrations , without any augmentation. This allows us to quantify how well a policy trained only on the original base poses generalizes to unseen poses.
Action Augmentation. Following [54], we augment each demonstration by perturbing the proprioceptive state and the action labels with zero-mean Gaussian noise while leaving the image observations unchanged. This exposes the policy to small action variations, but because the images remain the same, this augmentation provides no information about how the scene looks from a different base pose. We use this data to train the same Diffusion Policy and use the same training schedule as we do with all other baselines.
VISTA. VISTA [27] augments a demonstration by re-rendering its observations from perturbed camera viewpoints with ZeroNVS [32], a zero-shot single-image novel-view-synthesis diffusion model, while leaving the actions unchanged. We run the publicly released ZeroNVS model and checkpoint.1 For each source demonstration, we sample a small set of viewpoint perturbations: a planar camera offset drawn uniformly within cm on each axis (the first sample is the unperturbed view), together with a single per-demonstration yaw that re-aims the camera at the object centroid, clipped to rad; beyond this perturbation range, we observe large artifacts in image generation and worse imitation learning results. We then process each frame independently: conditioning on the original egocentric RGB image (center-cropped and resized to ) and the relative perturbation, ZeroNVS synthesizes the frame from the novel viewpoint. We sample with DDIM (50 steps, ), a classifier-free guidance scale of 7.5, and a 70° field of view matching our camera. Because ZeroNVS is a single-image model applied frame by frame, the synthesized views are not temporally consistent across a trajectory. The generated frames are paired with the original, unmodified actions and are used to train the same Diffusion Policy as all other methods.
One-Thousand Trajectories. One-Thousand Trajectories [8] reconstructs the static scene as a 3D Gaussian Splatting model and re-renders observations from novel viewpoints. In our implementation, we reconstruct the static scene using the same pipeline and navigation data as in MoMaSplat, but without masking out the manipulated object, so that the static scene model contains the manipulated object at its initial pose.
We use its novel-view augmentation component without the additional trajectory augmentation proposed in [8]. To account for changes in the robot base frame, we apply the same delta-based action retargeting used by MoMaSplat (Sec. III-B). This baseline relies on a static scene reconstruction rather than explicitly reconstructing the dynamic manipulated object. For Pick-Place Bottle, we restrict synthetic generation to the 40 demonstrations whose initial object pose matches the reconstructed scene, producing 400 synthetic demonstrations. All 90 source demonstrations remain in the policy-training set. For the other tasks, the initial object pose is fixed, so all source demonstrations are used for synthetic generation.
Method Details
3D Scene Model Reconstruction
Structure from motion and Gaussian Splatting. We reconstruct the static scene from the navigation frames . We run structure-from-motion using COLMAP [46] with a sequential matcher over the navigation frames, together with a small set of context frames sampled just before each manipulation. This step returns per-image camera poses and a sparse point cloud. The sparse points are used to initialize a 3D Gaussian Splatting reconstruction that we train with WildGaussians [47].2 We use the nerfbaselines framework and train for 30,000 iterations. WildGaussians augments 3DGS with per-image appearance and exposure embeddings and a DINOv2-based uncertainty head that down-weights transient or inconsistent regions, such as the robot or moving objects, during optimization. We use a DSSIM weight of 0.2 in the photometric loss and hold out every eighth frame for evaluation.
Metric scaling. Structure-from-motion recovers the scene only up to an unknown global scale. However, our action retargeting is metric, so we rescale the reconstruction using the robot’s onboard depth sensor measurements. For each navigation frame we render a depth map from the scene model and compare it to the measured depth from the head-mounted RGB-D camera (stored in millimeters and converted to meters). We consider only pixels with valid measured depth (in m, at least 1000 per frame) and for each frame, we take the median of the per-pixel ratios of measured to rendered depth. Eventually, the scale factor is set as the median of these per-frame median values, making it robust to outliers. This factor allows us to convert rendered depth and Gaussian coordinates into meters.
Scene pose interpolation. For the navigation frames that have not been registered by COLMAP, we estimate their scene camera poses using the same strategy as we do with objects (see Sec. B-2). This includes SLERP-based interpolation, smoothness filtering, and cyclic-outlier rejection, applied per demonstration.
3D Object Model Reconstruction
Segmentation. To obtain the per-frame object masks , we use Grounded-SAM [50], prompting an open-world detector, GroundingDINO [56], with a per-object text label to localize the manipulated object, and propagating the detected box through the clip using SAM 2 [51].3 We use a forward and a reverse pass from a per-object seed frame. When multiple objects are tracked, pixels claimed by more than one object are removed from all of them (exclusive masks). Object reconstruction uses the per-object masks to exclude the background and robot. For scene reconstruction, we additionally obtain a robot mask by rendering the robot in the simulator at the recorded configuration and camera pose. We exclude the union of the object masks and this robot mask when training .
Frame Selection Strategy. A source manipulation demonstration contains several hundred RGB frames recorded at video rate, far more than the feed-forward model can process. Simultaneously, near-duplicate frames add little to the pose estimate while inflating compute. We therefore subsample each demonstration to a compact, motion-diverse set of roughly 10–20 frames before passing it to the model, using the per-frame object masks .
We first discard frames whose object mask is small or poorly segmented, removing views where the object is distant or occluded. From the remaining frames we drop near-duplicate, stationary views: scanning in temporal order, we keep a frame only when its mask bounding-box IoU with the previously kept frame falls below 0.8 (the same threshold for all tasks), or when more than 20 frames have elapsed since the last kept frame. The first and last frame of every demonstration are always retained to anchor the reconstruction to the endpoints of the clip. Additionally, 50 frames from one navigation demonstration are included for frame alignment, and the combined set is capped at the feed-forward model’s 400-frame input budget.
Bundle Adjustment. For our feed-forward model, we use VGGT [12]4, as implemented by the training-free FastVGGT [49] acceleration5. For each selected frame, it predicts an object-relative camera pose together with a per-frame intrinsics matrix and a dense depth map. These predictions are accurate but not multi-view consistent, so we refine them with a bundle-adjustment step. We first unproject the predicted depth into a set of 3D points and predict dense 2D feature tracks across the selected frames. Initializing a reconstruction with the predicted poses, intrinsics, and points, we filter tracks by reprojection error, using an adaptive threshold that grows until a sufficient fraction of frames retain enough inliers, and by track visibility, requiring every 3D point to be observed in at least two frames. We then run bundle adjustment, jointly optimizing the camera poses and 3D points (and, when intrinsics are not supplied, the focal length and principal point) to minimize reprojection error. The refined, multi-view-consistent poses replace as the input to the Sim(3) alignment.
Frame Alignment. The poses estimated above lie in the object model’s internal frame with arbitrary scale, orientation, and origin, whereas the scene lives in the world frame . We align them using the navigation frames, where the object is static so its recovered trajectory is explained purely by the camera motion. Effectively, the same cameras are posed twice: by the object model in and by structure-from-motion in . Matching the navigation cameras by frame, we fit the similarity transform from their camera centers with the Umeyama algorithm [52], solving for scale, rotation, and translation in closed form. To be robust to mis-registered frames, we wrap Umeyama in RANSAC: we repeatedly fit from minimal 5-camera subsets, reject hypotheses whose scale leaves a plausible range, score each by the number of camera centers whose residual is below 0.2 m, keep the highest-consensus hypothesis, and refit on its inliers followed by a Huber-robust nonlinear refinement restricted to those inliers. We report the alignment quality as the camera-center residual (mean and median) and the RMS of the aligned sparse-point residuals. Applying maps the object poses into the world frame, .
Object Gaussian Optimization. Given the registered frames and their aligned poses , we fit a 3D Gaussian Splatting model [11] of the object with the gsplat [57] trainer6 for 60,000 steps, supervising only the segmented object through its RGBA images, where the alpha channel is the object mask composited over a white background. We minimize an plus SSIM photometric loss (SSIM weight 0.2) together with an term on the rendered opacity against the mask. The per-frame poses are optimized jointly with the Gaussian parameters at a small learning rate of , so the photometric loss is backpropagated through both appearance and pose, yielding the pose-disentangled model that can be rendered at arbitrary 6-DoF poses.
Pose Interpolation. Because of frame selection, the feed-forward model only returns camera poses at the selected keyframes. In contrast to that, synthesizing a demonstration requires an object pose at every timestep of the source trajectory. We therefore densify the keyframe poses by interpolation: rotations are interpolated with spherical linear interpolation (SLERP) and translations are interpolated linearly, between consecutive keyframes. To suppress residual estimation noise we add three refinements before interpolating. A smoothness filter drops keyframes whose translation and rotation relative to the previous keyframe both fall below small thresholds, removing jitter from near-stationary stretches. A cyclic-outlier filter removes back-tracking keyframes, where an excursion to is large enough to be meaningful yet the trajectory returns close to , a signature of transient pose-estimation errors. An ease-in parameterization places interpolated samples so that pose changes are concentrated toward the end of each keyframe interval, better matching the observed motion profile than uniform piecewise-linear filling. The densified trajectory is what we rasterize along when compositing novel-view observations.
Base Pose Sampling Strategy
Here, we expand on the procedure used to draw the sampled base poses around each original base pose (Sec. III-B). Sampling operates per source demonstration about an anchor pose recovered for that demonstration, i.e., the original base position and heading expressed in the world frame, as it is established by the scene reconstruction.
Planar position. We draw the planar offset from a distribution centered at the anchor. By default this is uniform over a square region around the original base pose (the region size used in our experiments is reported in Sec. IV-A); we also support an isotropic Gaussian with magnitude bounds for annulus-shaped coverage.
Heading. For each sampled position we set the base heading so that the egocentric camera points at the object’s 3D centroid (accounting for the camera’s fixed offset from the base), then perturb it with uniform noise (default ) so the policy sees the object at a range of positions in the camera view. The robot head is fixed, so the object is kept in view through the base pose alone; we discard samples that would place the object near the image edge or out of view before running the costlier IK check.
IK feasibility. Following Sec. III-B, a sampled base pose is retained only if the source arm trajectory, with the end effector held at the same pose relative to the manipulated object, admits a valid inverse-kinematics solution. We verify reachability at several probe timesteps spanning the demonstration and accept the sample only if all of them are reachable, querying the curobo inverse-kinematics solver for the TIAGo robot inside the OmniGibson simulator [53]. Infeasible candidates are rejected and resampled, up to a fixed draw budget per demonstration, and the first accepted samples are kept.
Imitation Learning
We provide implementation details for the policy trained with Diffusion Policy [1] on the union of source and generated demonstrations (Sec. III-C).
Architecture. The policy is a conditional 1D U-Net denoiser. The egocentric head-camera RGB image is encoded by a ResNet-18 trained from scratch followed by a spatial-softmax pooling layer (32 keypoints), and the resulting visual features are concatenated with the low-dimensional proprioception to form the conditioning vector.
Observations and actions. At each step the policy conditions on the egocentric head-camera RGB image (resized to 224 224) and the end-effector proprioception, stacked over the most recent steps. It predicts a chunk of future actions, each a 7-dimensional command consisting of an end-effector translation delta, a rotation delta, and a gripper command expressed in the robot base frame, normalized to zero mean and unit variance. During training we apply random crop and translation together with color-jitter augmentation to the RGB input.
Diffusion process. We use a DDIM noise scheduler with 50 training timesteps, a cosine-based schedule, and (noise) prediction, optimizing the mean-squared denoising loss of (1). At inference, actions are produced by running the reverse process for the same number of denoising steps.
Optimization. We optimize with AdamW (learning rate , weight decay ) under a cosine schedule with 1000 warm-up steps and a batch size of 32 for 100,000 gradient steps.
Source and synthetic co-training. We train on the union of the source demonstrations and the generated demonstrations using a weighted sampler that draws each training sequence from with probability 0.25 and from with probability 0.75. This anchors the policy to the source data while exploiting the broader base-pose coverage of the synthetic data.
Hardware and data. We use a TIAGo mobile manipulator with a head-mounted RGB-D camera (70° field of view). We collect 75 source demonstrations for CLOSE LAPTOP and 90 for each other task across three base poses, and generate 10 synthetic demonstrations per source demonstration within a 120 120 cm region around each source base pose. We evaluate each policy at 12 unseen base poses with 3 rollouts per pose. Table II summarizes the main hyperparameters across the pipeline.
| Stage | Parameter | Value |
| Scene SfM | matcher | sequential (COLMAP) |
| Scene 3DGS | method / iterations | WildGaussians / 30,000 |
| Scene 3DGS | DSSIM weight | 0.2 |
| Metric scale | valid-depth range | (0, 9] m, px/frame |
| Segmentation | detector / tracker | GroundingDINO / SAM2 |
| Frame selection | IoU thresh. / max gap | 0.8 / 20 frames |
| Frame selection | VGGT input budget | 400 frames per object |
| Object BA | visibility per 3D point | views |
| Sim(3) RANSAC | subset / trials / inlier | 5 / 8000 / 0.2 m |
| Object 3DGS | steps / SSIM weight | 60,000 / 0.2 |
| Object 3DGS | pose-opt learning rate | |
| Base sampling | region / yaw noise | 120 120 cm / |
| Generation | synthetic per source demonstration | 10 |
| Policy | obs / action horizon | 2 / 8 |
| Policy | DDIM train timesteps | 50 |
| Policy | optimizer / learning rate | AdamW / |
| Policy | batch size / training steps | 32 / 100,000 |
| Policy | source : generated ratio | 0.25 : 0.75 |
Table II. Key hyperparameters across the MoMaSPLAT pipeline.
References
- [1]C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion Policy: Visuomotor policy learning via action diffusion,” in RSS, 2023.
- [2]T. Zhao, V. Kumar, S. Levine, and C. Finn, “Learning fine-grained bimanual manipulation with low-cost hardware,” in RSS, 2023.arxiv.org/abs/2304.13705
- [3]J. Yang, I. Huang, B. Vu, M. Bajracharya, R. Antonova, and J. Bohg, “Mobi-π: Mobilizing your robot learning policy,” in CoRL, 2025.
- [5]S. Fei, S. Wang, J. Shi, Z. Dai, J. Cai, P. Qian, L. Ji, X. He, S. Zhang, Z. Fei et al., “LIBERO-Plus: In-depth robustness analysis of vision-language-action models,” arXiv preprint arXiv:2510.13626, 2025.arxiv.org/abs/2510.13626
- [6]Z. Fu, T. Z. Zhao, and C. Finn, “Mobile ALOHA: Learning bimanual mobile manipulation with low-cost whole-body teleoperation,” in CoRL, 2024.arxiv.org/abs/2401.02117
- [7]A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y. Narang, L. Fan, Y. Zhu, and D. Fox, “MimicGen: A data generation system for scalable robot learning using human demonstrations,” in CoRL, 2023.arxiv.org/abs/2310.17596
- [8]C. Pan, L. Liang, D. Bauer, E. Cousineau, B. Burchfiel, S. Feng, and S. Song, “One demo is worth a thousand trajectories: Action-view augmentation for visuomotor policies,” in CoRL, 2025.arxiv.org/abs/2606.19586
- [9]S. Yang, W. Yu, J. Zeng, J. Lv, K. Ren, C. Lu, D. Lin, and J. Pang, “Novel demonstration generation with Gaussian splatting enables robust one-shot manipulation,” arXiv preprint arXiv:2504.13175, 2025.arxiv.org/abs/2504.13175
- [10]A. Zhou, M. J. Kim, L. Wang, P. Florence, and C. Finn, “NeRF in the palm of your hand: Corrective augmentation for robotics via novel-view synthesis,” in CVPR, 2023, pp. 17 907–17 917.arxiv.org/abs/2301.08556
- [11]B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis, “3D Gaussian splatting for real-time radiance field rendering,” ACM Trans. Graph., vol. 42, no. 4, pp. 139:1–139:14, 2023.
- [12]J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny, “VGGT: Visual geometry grounded transformer,” in CVPR, 2025, pp. 5294–5306.arxiv.org/abs/2503.11651
- [13]Z. Jiang, Y. Xie, K. Lin, Z. Xu, W. Wan, A. Mandlekar, L. J. Fan, and Y. Zhu, “DexMimicGen: Automated data generation for bimanual dexterous manipulation via imitation learning,” in ICRA, 2025, pp. 16 923–16 930.arxiv.org/abs/2410.24185
- [14]C. Garrett, A. Mandlekar, B. Wen, and D. Fox, “SkillMimicGen: Automated demonstration generation for efficient skill learning and deployment,” in CoRL, 2024.arxiv.org/abs/2410.18907
- [15]C. Li, M. Xu, A. Bathey, H. Yin, Y. Jiang, H. Huang, J. Wong, S. Garlanka, C. Gokmen, R. Zhang et al., “MoMaGen: Generating demonstrations under soft and hard constraints for multi-step bimanual mobile manipulation,” in ICLR, 2026.arxiv.org/abs/2510.18316
- [16]K. Lin, V. Ragunath, A. McAlinden, A. Prasad, J. Wu, Y. Zhu, and J. Bohg, “Constraint-preserving data generation for visuomotor policy learning,” in CoRL, 2025.arxiv.org/abs/2508.03944
- [17]Z. Xue, S. Deng, Z. Chen, Y. Wang, Z. Yuan, and H. Xu, “DemoGen: Synthetic demonstration generation for data-efficient visuomotor policy learning,” in RSS, 2025.arxiv.org/abs/2502.16932
- [18]F. Sadeghi, A. Teshn, E. Jang, and S. Levine, “Sim2Real viewpoint invariant visual servoing by recurrent control,” in CVPR, 2018, pp. 4691–4699.DOI
- [19]N. Hirose, D. Shah, A. Sridhar, and S. Levine, “ExAug: Robot-conditioned navigation via geometric experience augmentation,” arXiv preprint arXiv:2210.07450, 2022.arxiv.org/abs/2210.07450
- [20]Y. Seo, J. Kim, S. James, K. Lee, J. Shin, and P. Abbeel, “Multi-view masked world models for visual robotic manipulation,” in ICML, 2023, pp. 30 613–30 632.arxiv.org/abs/2302.02408
- [21]M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas, “Reinforcement learning with augmented data,” in NeurIPS, vol. 33, 2020, pp. 19 884–19 895.
- [22]D. Yarats, I. Kostrikov, and R. Fergus, “Image augmentation is all you need: Regularizing deep reinforcement learning from pixels,” in ICLR, 2021.
- [23]N. Hansen, R. Jangir, Y. Sun, G. Alenyà, P. Abbeel, A. A. Efros, L. Pinto, and X. Wang, “Self-supervised policy adaptation during deployment,” arXiv preprint arXiv:2007.04309, 2020.arxiv.org/abs/2007.04309
- [24]J. Jang, S. Ye, Z. Lin, J. Xiang, J. Bjorck, Y. Fang, F. Hu, S. Huang, K. Kundalia, Y.-C. Lin et al., “DreamGen: Unlocking generalization in robot learning through video world models,” arXiv preprint arXiv:2505.12705, 2025.arxiv.org/abs/2505.12705
- [25]J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mayankalnar, R. Steigerwald, C. Apps et al., “Genie: Generative interactive environments,” in ICML, 2024.arxiv.org/abs/2402.15391
- [26]N. Agarwal, A. Ali, M. Bala, Y. Balaji, E. Barker, T. Cai, P. Chattopadhyay, Y. Chen, Y. Cui, Y. Ding et al., “Cosmos world foundation model platform for physical AI,” arXiv preprint arXiv:2501.03575, 2025.arxiv.org/abs/2501.03575
- [27]S. Tian, B. Wulfe, K. Sargent, K. Liu, S. Zakharov, V. Guizilini, and J. Wu, “View-invariant policy learning via zero-shot novel view synthesis,” in CoRL, 2024.arxiv.org/abs/2409.03685
- [28]A. Tagliabue and J. P. How, “Tube-NeRF: Efficient imitation learning of visuomotor policies from MPC using tube-guided data augmentation and NeRFs,” arXiv preprint arXiv:2311.14153, 2023.arxiv.org/abs/2311.14153
- [29]L. Y. Chen, C. Xu, K. Dharmarajan, M. Z. Irshad, R. Cheng, K. Keutzer, M. Tomizuka, Q. Vuong, and K. Goldberg, “RoVi-Aug: Robot and viewpoint augmentation for cross-embodiment robot learning,” in CoRL, 2024.arxiv.org/abs/2409.03403
- [30]S. Wang, H. Dong, J. Tian, J. Li, Z. Yang, T. Cao, A. Chen, S. Wu, L. Wang, and S. Zhou, “Efficient camera pose augmentation for view generalization in robotic policy learning,” arXiv preprint arXiv:2603.29192, 2026.arxiv.org/abs/2603.29192
- [31]X. Zhang, M. Chang, P. Kumar, and S. Gupta, “Diffusion meets DAgger: Supercharging eye-in-hand imitation learning,” in RSS, 2024.arxiv.org/abs/2402.17768
- [32]K. Sargent, Z. Li, T. Shah, C. Herrmann, H.-X. Yu, Y. Zhang, E. R. Chan, D. Lagun, L. Fei-Fei, D. Sun et al., “Zero-shot 360-degree view synthesis from a single image,” in CVPR, 2024, pp. 9420–9429.arxiv.org/abs/2310.17994
- [33]Y. Xu, J. Yang, X. Wang, Y. Chen, Z. Zhu, B. Fang, G. Huang, X. Chen, Y. Ye, Q. Zhang et al., “EgoDemoGen: Novel egocentric demonstration generation enables viewpoint-robust manipulation,” arXiv preprint arXiv:2509.22578, 2025.
- [34]A. Bahety, A. Balaji, B. Abbate mat teo, and R. Martín-Martín, “SafeMimic: Towards safe and autonomous human-to-robot imitation for mobile manipulation,” in RSS, 2025.arxiv.org/abs/2506.15847
- [35]J. Wu, R. U. An, K. Lan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “TidyBot: Personalized robot assistance with large language models,” Autonomous Robots, 2023.DOI
- [36]P. Liu, Y. Orru, C. Paxton, N. M. M. Shafiullah, and L. Pinto, “OK-Robot: What really matters in integrating open-knowledge models for robotics,” in RSS, 2024.DOI
- [37]Y. Jiang, R. Zhang, J. Wong, C. Wang, Y. Ze, H. Yin, C. Gokmen, S. Song, J. Wu, and L. Fei-Fei, “BEHAVIOR robot suite: Streamlining real-world whole-body manipulation for everyday household activities,” arXiv preprint arXiv:2503.05652, 2025.arxiv.org/abs/2503.05652
- [38]S. Yenamandra, A. Ramachandran, K. Yadav, A. Wang, M. Khanna, T. Gervet, T.-Y. Yang, V. Jain, A. W. Clegg, J. Turner et al., “HomeRobot: Open-vocabulary mobile manipulation,” in CoRL, 2023.arxiv.org/abs/2306.11565
- [39]F. Stulp, A. Fedrizzi, L. Mösenlechner, and M. Beetz, “Learning and reasoning with action-related places for robust mobile manipulation,” Journal of Artificial Intelligence Research, vol. 43, pp. 1–42, 2012.DOI
- [40]S. Jauhri, J. Peters, and G. Chaltzaki, “Robot learning of mobile manipulation with reachability behavior priors,” IEEE Robotics and Automation Letters, vol. 7, no. 3, pp. 8399–8406, 2022.DOI
- [41]E. Rosen, S. James, S. Orozco, V. Gupta, M. Merlin, S. Tellex, and G. Konidaris, “Synthesizing navigation abstractions for planning with portable manipulation skills,” in CoRL, 2023, pp. 2278–2287.
- [42]N. Hirose, C. Glossop, A. Sridhar, D. Shah, O. Mees, and S. Levine, “LeLaN: Learning a language-conditioned navigation policy from in-the-wild videos,” arXiv preprint arXiv:2410.03603, 2024.arxiv.org/abs/2410.03603
- [43]N. Yokoyama, S. Ha, D. Batra, J. Wang, and B. Bucher, “VLFM: Vision-language frontier maps for zero-shot semantic navigation,” in ICRA, 2024, pp. 42–48.arxiv.org/abs/2312.03275
- [44]N. Yokoyama, A. Clegg, J. Truong, E. Undersander, T.-Y. Yang, S. Arnaud, S. Ha, D. Batra, and A. Rai, “ASC: Adaptive skill coordination for robotic mobile manipulation,” IEEE Robotics and Automation Letters, vol. 9, no. 1, pp. 779–786, 2023.arxiv.org/abs/2304.00410
- [45]N. M. M. Shafiullah, A. Rai, H. Etukuru, Y. Liu, I. Misra, S. Chintala, and L. Pinto, “On bringing robots home,” arXiv preprint arXiv:2311.16098, 2023.arxiv.org/abs/2311.16098
- [46]J. L. Schonberger and J.-M. Frahm, “Structure-from-motion revisited,” in CVPR, 2016, pp. 4104–4113.DOI
- [47]J. Kulhanek, S. Peng, Z. Kukelova, M. Pollefeys, and T. Sattler, “WildGaussians: 3D Gaussian splatting in the wild,” in NeurIPS, 2024.arxiv.org/abs/2407.08447
- [48]B. Wen, W. Yang, J. Kautz, and S. Birchfield, “FoundationPose: Unified 6D pose estimation and tracking of novel objects,” in CVPR, 2024.arxiv.org/abs/2312.08344
- [49]Y. Shen, Z. Zhang, Y. Qu, and L. Cao, “FastVGGT: Training-free acceleration of visual geometry transformer,” arXiv preprint arXiv:2509.02560, 2025.arxiv.org/abs/2509.02560
- [50]T. Ren, S. Liu, A. Zeng, J. Lin, K. Li, H. Cao, J. Chen, X. Huang, Y. Chen, F. Yan et al., “Grounded SAM: Assembling open-world models for diverse visual tasks,” arXiv preprint arXiv:2401.14159, 2024.arxiv.org/abs/2401.14159
- [51]N. Ravi, V. Gabeur, Y.-T. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, E. Mintun, J. Pan, K. V. Alwala, N. Carion, C.-Y. Wu, R. Girshick, P. Dollár, and C. Feichtenhofer, “SAM 2: Segment anything in images and videos,” in ICLR, 2025.arxiv.org/abs/2408.00714
- [52]S. Umeyama, “Least-squares estimation of transformation parameters between two point patterns,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 4, pp. 376–380, 1991.DOI
- [53]C. Li, R. Zhang, J. Wong, C. Gokmen, S. Srivastava, R. Martín-Martín, C. Wang, G. Levine, M. Lingelbach, J. Sun et al., “BEHAVIOR-1K: A benchmark for embodied AI with 1,000 everyday activities and realistic simulation,” in CoRL, 2023, pp. 80–93.arxiv.org/abs/2403.09227
- [54]P. Florence, L. Manuelli, and R. Tedrake, “Self-supervised correspondence in visuomotor policy learning,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 492–499, 2019.arxiv.org/abs/1909.06933
- [55]A. Knappitsch, J. Park, Q.-Y. Zhou, and V. Koltun, “Tanks and Temples: Benchmarking large-scale scene reconstruction,” ACM Trans. Graph., vol. 36, no. 4, pp. 78:1–78:13, 2017.
- [56]S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection,” in ECCV, 2024.arxiv.org/abs/2303.05499
- [57]V. Ye, R. Li, J. Kerr, M. Turkulainen, B. Yi, Z. Pan, O. Seiskari, J. Ye, J. Hu, M. Tancik, and A. Kanazawa, “gsplat: An open-source library for Gaussian splatting,” Journal of Machine Learning Research, vol. 26, no. 34, pp. 1–17, 2025.arxiv.org/abs/2409.06765

