betaFeedback

WorksComputer Science

Homeward: No Place Like the Training View

Abstract

Vision-language-action (VLA) policies implicitly assume that third-person cameras remain fixed at their data-collection poses; once a camera is bumped, remounted, or swapped at deployment, task success degrades precipitously even though the underlying task and robot remain unchanged. We present Homeward, a training-free, plug-and-play observation adapter that canonicalizes observations from an uncalibrated deployment camera back into the training view for frozen VLA policies. Our design stems from a closed-loop diagnostic paradox in simulation: while re-rendering perturbed views into the training camera with oracle geometry recovers much of the lost performance, substituting an off-the-shelf feed-forward 3D foundation model can leave the policy worse off than applying no correction at all. This degradation concentrates on the thin, articulated robot arm, which feed-forward depth prediction tears and scatters - yet the arm's exact 3D state is already known from robot kinematics. Homeward exploits this asymmetry: it estimates what is unknown - camera parameters and static scene depth - from the live RGB frame and a single training-view reference, and renders what is known - the moving robot links - directly from the kinematic model. Evaluated across ∼\sim300,000 closed-loop episodes spanning four frozen VLAs, Homeward boosts success across all 1,599 camera variants of LIBERO-Plus from 62.5%, 76.5%, 68.0%, and 3.6% to 77.6%, 95.1%, 91.2%, and 26.4% for OpenVLA-OFT, π0.5π_{0.5}, π0π_0, and OpenVLA, respectively. Without any scene fine-tuning or deployment camera calibration, Homeward extends to 6-DoF random placements, focal-length changes, and, with per-step pose estimation, continuous intra-episode camera drift, while matching or surpassing a calibrated, scene-trained view-synthesis baseline on three of four policies on LIBERO-Plus.

Introduction

Vision-language-action (VLA) models have rapidly advanced generalist robotic manipulation by mapping raw visual observations and natural language instructions directly to actions [2, 3, 20, 21, 29, 30, 33, 49]. Yet beneath their broad semantic generalization lies a brittle geometric assumption inherited from data collection: at deployment, the third-person camera must remain precisely where it stood during demonstrations. In practice, this assumption breaks routinely when cameras are bumped, re-mounted, or replaced with different lenses. Although neither the task nor the robot has changed, the unseen viewpoint severely disrupts action prediction [44]. Indeed, LIBERO-Plus [8] identifies camera viewpoint shifts as the single most damaging perturbation for frozen VLAs; across its 1,599 camera variants, OpenVLA-OFT [21] drops to 62.5% overall success and plummets to 22.0% under the largest shifts (Figure 3).

Figure 3. Closed-loop success across camera-shift severity levels, pooled across all four LIBERO-Plus suites (levels 1–5 comprise 199, 360, 325, 228, and 487 variants). While frozen policies degrade steeply with viewpoint shift, HOMEWARD degrades far less up to Level 5—lifting OpenVLA-OFT from 22.0% to 53.8%, π0.5\pi_{0.5} from 55.6% to 92.6%, and π0\pi_0 from 48.3% to 80.5%. OpenVLA benefits most at Level 1 (10.6% to 47.2%).

One natural remedy is to bake viewpoint invariance into the policy itself via multi-view data augmentation [5, 37], cross-view consistency objectives [16, 25], 3D-aware architectures [1, 43], or few-shot adaptation in the target view [24]. However, retraining or modifying large pre-trained VLAs sacrifices modularity and must be repeated for every policy family. An appealing orthogonal paradigm is test-time observation canonicalization: leaving the policy completely untouched and transforming incoming images back into the training viewpoint. Yet existing test-time adapters merely shift the engineering burden to deployment—requiring extrinsic calibration of the moved camera, metric depth sensors, or generative view-synthesis networks fine-tuned on target scenes [12, 14, 31, 32, 35]. Because only the observer’s coordinate frame has changed, we ask: How far can a frozen VLA be recovered from arbitrary camera shifts when the deployment camera is uncalibrated and nothing is trained?

Closed-loop simulation reveals a striking paradox. Reprojecting perturbed observations into the training view using privileged simulator depth and ground-truth camera poses restores OpenVLA-OFT on LIBERO-Goal from 79.7% to 90.4% overall—and from 35.5% to 82.9% under the severest shifts—confirming that viewpoint failure resides predominantly in visual input space. Yet when we replace privileged geometry with an off-the-shelf 3D foundation model (VGGT [39]), closed-loop success drops to 72.8%—worse than feeding the unaligned view directly to the policy (Figure 1).

Figure 1. Naive 3D scene restoration can leave a frozen VLA worse off than no correction—unless paired with kinematic robot rendering. (a) Displacing the third-person camera (34∘34^\circ azimuth, 15∘15^\circ elevation) degrades frozen OpenVLA-OFT. (b) Reprojecting the observation with a frozen 3D foundation model tears and scatters the moving arm (red box), lowering success from 79.7% to 72.8%. (c) HOMEWARD composites moving links rendered from measured joint states (blue box) over the restored scene, boosting success to 91.4%. (d) The canonical training-camera view, for reference. Percentages report closed-loop OpenVLA-OFT success across all 408 LIBERO-Goal camera variants.

Diagnostic inspection pinpoints the culprit: while feed-forward 3D models recover static background layout reasonably well, they tear and scatter the thin, specular, moving robot arm. Prior work noticed this reconstruction bottleneck and responded by fine-tuning 3D backbones on robot datasets [12]. Our key insight is simpler: the robot arm does not need to be visually reconstructed at all, because its exact 3D geometry is already given at every timestep by the kinematic model and joint proprioception.

Building on this asymmetric decomposition between unknown scene geometry and known robot kinematics, we introduce HOMEWARD, a training-free test-time observation adapter. Given a single clean reference frame from the training camera, HOMEWARD leverages a frozen 3D foundation model to estimate the deployment camera parameters and current scene depth, reprojects the workspace into the training view, inpaints disocclusions from the reference frame, and composites the articulated robot links rendered directly from measured joint states. While kinematic robot cross-painting has previously enabled cross-embodiment transfer under calibrated setups [4, 22], HOMEWARD demonstrates that pairing kinematic rendering with feed-forward monocular geometry unlocks reliable view restoration with a frozen policy, a frozen geometry backbone, an uncalibrated deployment camera, and zero adaptation data—requiring only the robot model, one reference image, and a one-time training-camera calibration. Restoring this kinematic layer elevates OpenVLA-OFT on LIBERO-Goal from 72.8% to 91.4%, matching privileged oracle re-rendering (90.4%).

Evaluated over roughly 300,000 closed-loop rollouts, our contributions are threefold:

  • Closed-Loop Viewpoint Diagnostic. We quantify how much frozen VLA failure under camera shifts is recoverable purely in observation space, and demonstrate that naive restoration with off-the-shelf feed-forward 3D models can underperform uncorrected inputs due to foreground reconstruction artifacts on the moving arm (§ 3.2).

  • Training-Free Kinematic–Geometric Canonicalization. We propose HOMEWARD, which decouples observation restoration into monocular scene reprojection and forward-kinematic link rendering, with initial-pose caching and motion gating for low-overhead deployment without deployment-camera calibration (§ 3.3 to § 3.5).

  • Large-Scale Evaluation & Regime Characterization. Across four frozen VLAs and five camera perturbation families—including 1,599 LIBERO-Plus variants (+15.1 to +23.2 points), 6-DoF random poses, continuous drift, and focal-length changes—we show where zero-shot geometric restoration excels, how it complements fine-tuned policies out-of-distribution, and where wide-baseline calibrated view synthesis remains ahead (§ 4).

Related Work

Training-Time Viewpoint Robustness for Robot Policies. Generalization benchmarks probe camera-pose perturbations [34, 44], and LIBERO-Plus [8] identifies third-person viewpoint shifts as a primary failure mode of modern VLAs. Most existing solutions intervene during policy training: synthesizing novel viewpoints for data augmentation [5, 37], fine-tuning on multi-view perturbed trajectories [8], lifting observations into calibrated 3D/RGB-D representations [1, 10, 11, 18, 36, 43, 46], designing view-invariant architectures [25], or enforcing cross-view action consistency [16]. Alternatively, FTM [24] adapts visual encoders at test time using a demonstration collected in the new camera view. While effective within their target distributions, all such methods either modify policy weights or require new-view demonstrations, whereas HOMEWARD operates strictly in input space for off-the-shelf frozen policies.

Test-Time Observation Canonicalization. A complementary line of work transforms test-time observations back to the training camera frame (Tab. 1). Neural view-synthesis adapters rely on scene- or robot-specific training: AnyCamVLA [14] fine-tunes a large view-synthesis model [17] on LIBERO-Plus renders conditioned on known third-person and wrist camera parameters, while GS-VLA [32] canonicalizes views via a trained Gaussian-splatting [19] module given deployment poses and depth. VistaBot [12] pairs VGGT [39] with a video diffusion synthesizer; notably, it also observes that feed-forward 3D models struggle on robot arms and grippers, addressing this by fine-tuning both VGGT and downstream policy features on robot data. Geometric reprojection and cross-painting approaches avoid generative hallucination by leveraging known sensor calibration: CoRe-VLA [31] reprojects calibrated RGB-D frames followed by a learned restoration network, ARGUS [35] fuses two calibrated views for policies trained on canonicalized inputs, and Mirage [4] masks target robots and renders source-robot meshes from joint states across calibrated cameras for cross-embodiment transfer. HOMEWARD unifies monocular estimated geometry with kinematic robot rendering under a strictly weaker deployment assumption: zero model training or fine-tuning, and an uncalibrated monocular deployment camera. We identify the design choices that make this combination viable in closed loop (§ 4.6).

Feed-Forward 3D Foundation Models. Beyond single-image depth and geometry estimators [15, 40, 45], multi-view transformers such as DUSt3R [41], MASt3R [23], VGGT [39], and π3\pi^{3} [42] infer camera intrinsics, extrinsics, and dense aligned depth from uncalibrated RGB images in a single forward pass without bundle adjustment. In manipulation, however, a specular arm that moves between the reference and live frames violates the static-scene assumption of two-view geometry [13, 47]. Rather than fine-tuning

trained or fine-tunedmoved camerarobot region
AnyCamVLA [14]synthesis modelcalibratedsynthesized
VistaBot [12]VGGT, policyintrinsicsreconstructed
Mirage [4]dynamics modelcalibratedrendered
HOMEWARDnoneuncalibratedrendered

caption: Deployment assumptions of test-time viewpoint adapters. Trained or fine-tuned lists modules trained beyond public weights; Moved camera indicates calibration required at deployment. All but VistaBot assume one-time calibration of the training camera C⋆C^{\star} (Mirage targets cross-embodiment transfer).

:::

3D foundation models on robot data, we characterize how their failure modes impact closed-loop VLA control and bypass foreground errors via forward kinematics.

Method

Problem Formulation

We consider a frozen VLA policy π\pi that maps a multimodal observation ot=(It,Wt,qt)o_t = (I_t, W_t, q_t) and a language instruction ℓ\ell to actions, where ItI_t is the third-person RGB image, WtW_t is the wrist-camera image, and qtq_t denotes the robot joint configuration. During deployment, the third-person camera is displaced from its training configuration C⋆C^{\star} to an unknown configuration C′C^{\prime}, yielding a perturbed image It′I_t^{\prime}. We assume access to three standard quantities available prior to deployment: (i) a single clean reference image IrefI_{\mathrm{ref}} of the workspace captured from C⋆C^{\star}, (ii) the robot’s kinematic mesh model (e.g., a URDF), and (iii) the extrinsic calibration of the original training camera C⋆C^{\star} relative to the robot base, recorded once during data collection. Without calibrating C′C^{\prime} or modifying π\pi, our objective is to reconstruct an estimate I^t\hat{I}_t of the canonical training view and feed (I^t,Wt,qt,ℓ)(\hat{I}_t, W_t, q_t, \ell) directly into the frozen policy.

Closed-Loop Viewpoint Diagnostic

We first conduct a controlled closed-loop study on the LIBERO-Plus camera variants of LIBERO-Goal and LIBERO-Long to isolate what input-view canonicalization can—and cannot—recover (Tab. 2; § 4.1).

Recoverable via 3D geometry, not 2D warping. Re-rendering every displaced observation It′I_t^{\prime} into C⋆C^{\star} using privileged simulator depth and ground-truth camera poses (oracle re-rendering) boosts closed-loop success on LIBERO-Goal by +10.7, +4.4, and +24.5 percentage points for OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0, respectively, and by +25.1 to +40.8 points on LIBERO-Long. Action-level inspection reveals why: under a shifted camera, the policy’s initial planned motion still points in roughly the right direction (1–4° angular error), but misjudges step magnitude by 17–28%; oracle

OFTπ0.5\pi_{0.5}π0\pi_0OpenVLA
GLGLGLGL
frozen policy79.735.384.352.765.244.98.11.2
oracle re-rendering†90.4*76.1*88.777.8*89.7*74.7*27.5*15.0*
oracle homography†70.8*38.774.8*51.360.841.513.7*4.5*
scene restoration only72.8*51.3*76.0*60.4*82.6*62.5*22.8*7.2*
HOMEWARD91.4*72.3*94.4*94.0*93.4*78.3*39.2*11.2*

caption: Closed-loop diagnostic study (success rate, %) on LIBERO-Plus camera variants of Goal (G, 408) and Long (L, 419). †Privileged, non-deployable (simulator depth and/or cameras). *Exact McNemar p<0.05p < 0.05 vs. frozen policy.

:::

3D re-rendering compresses this magnitude error to 3–10%. Conversely, warping It′I_t^{\prime} via a ground-truth plane-induced homography [13] (oracle homography) degrades success on Goal below the uncorrected policy for three of four models, since a single plane leaves off-plane parallax unresolved.

Off-the-shelf 3D estimation fails on the moving arm. Replacing privileged geometry with estimates from a frozen VGGT [39] given [Iref,It′][I_{\mathrm{ref}}, I_t^{\prime}] (scene restoration only) causes OpenVLA-OFT (72.8% vs. 79.7%) and π0.5\pi_{0.5} (76.0% vs. 84.3%) on Goal to drop below the uncorrected frozen policy, while leaving π0\pi_0 far short of the oracle on both suites. Visual inspection exposes a sharp dichotomy (Figure 1b): VGGT re-projects the static tabletop and objects plausibly, whereas the thin, moving robot arm is severely torn and scattered across depth planes. Crucially, the articulated arm is the exact component whose 3D geometry is fully determined by qtq_t and the kinematic model. HOMEWARD therefore decomposes I^t\hat{I}_t into two complementary layers (Figure 2): a scene layer StS_t canonicalized via estimated monocular geometry, and a robot layer (Rt,Mt)(R_t, M_t) rendered from exact forward kinematics.

Figure 2. Overview of HOMEWARD. (a) Monocular scene restoration: At the first policy query, frozen VGGT estimates and caches the deployment camera transform (C′→C⋆C^{\prime} \rightarrow C^{\star}) from [Iref,I0′][I_{\mathrm{ref}}, I^{\prime}_{0}]; subsequent queries infer single-frame depth DtD_t aligned on static pixels. Depth-edge filtering, z-buffer splatting, and disocclusion filling from IrefI_{\mathrm{ref}} yield the scene layer StS_t, which retains artifacts on the moving arm (inset). (b) Kinematic robot layer & policy execution: Forward kinematics and the one-time training camera calibration C⋆C^{\star} render moving-link RGB RtR_t and silhouette mask MtM_t, composited via I^t=Mt⊙Rt+(1−Mt)⊙St\hat{I}_t = M_t \odot R_t + (1 - M_t) \odot S_t and passed with the unchanged WtW_t, qtq_t, and ℓ\ell to the frozen VLA π\pi.

Monocular Scene Restoration

Given the uncalibrated image pair [Iref,It′][I_{\mathrm{ref}}, I_t^{\prime}], frozen VGGT predicts camera intrinsics (Kref,Kt)(K_{\mathrm{ref}}, K_t), extrinsics (Eref,Et)(E_{\mathrm{ref}}, E_t), and a dense depth map DtD_t for It′I_t^{\prime} within a shared metric-up-to-scale coordinate frame. We back-project each homogeneous pixel u~\tilde{u} of It′I_t^{\prime} into 3D space and reproject it into the canonical training view C⋆C^{\star}:

u⋆=Π(KrefErefEt−1[Dt(u)Kt−1u~]),(1)u^{\star} = \Pi\left(K_{\mathrm{ref}} E_{\mathrm{ref}} E_t^{-1} \left[D_t(u) K_t^{-1} \tilde{u}\right]\right), \tag*{(1)}

where Π(⋅)\Pi(\cdot) is perspective projection and rigid transformations act on homogeneous coordinates. Reprojected colors are forward-splatted at policy resolution with a zz-buffer.

To suppress boundary “flying pixels” where feed-forward depth blends foreground and background across occlusion boundaries, we reject pixels whose depth deviates from any immediate neighbor by more than 3% prior to splatting. Isolated single-pixel rasterization cracks are interpolated from local neighbors, and remaining disoccluded regions are inpainted from IrefI_{\mathrm{ref}}. Across all LIBERO-Plus camera variants, live reprojected pixels cover 70% of the canonical view on average—ranging from 88% under mild estimated rotations (<10∘<10^\circ) to 59% under wide ones (>30∘>30^\circ)—with IrefI_{\mathrm{ref}} supplying static background context for the remainder to form the restored scene layer StS_t.

Kinematic Robot Rendering and Compositing

Because StS_t retains severe depth artifacts along thin, specular, moving manipulator links, we overwrite the foreground robot using forward kinematics. Given joint state qtq_t and the one-time training-camera calibration C⋆C^\star, a lightweight mesh renderer generates canonical robot RGB appearance RtR_t and binary silhouette mask MtM_t, which are composited over StS_t:

I^t=Mt⊙Rt+(1−Mt)⊙St.(2)\hat{I}_t = M_t \odot R_t + (1-M_t) \odot S_t. \tag*{(2)}

Selective moving-link rendering. (Rt,Mt)(R_t, M_t) includes only the articulated moving links, leaving the static robot base and mounting pedestal to the scene layer StS_t. Because the mesh renderer operates without full-scene occlusion geometry, rendering the entire robot—including the lower pedestal normally occluded by the tabletop in C⋆C^\star—paints over workspace pixels and collapses OpenVLA-OFT success on Goal to 39.0% (Table 7). Furthermore, this layer tolerates imperfect calibration and appearance: perturbing C⋆C^\star by (10 mm,1∘)(10\ \mathrm{mm}, 1^\circ) costs at most 3.2 percentage points and (20 mm,2∘)(20\ \mathrm{mm}, 2^\circ) leaves every evaluated cell above its uncorrected baseline, while drawing the arm with the color error measured for a real-robot render (≈30\approx30 gray levels) costs at most 8.8 points on Goal and Long, with every cell still significantly above the frozen policy (supplement).

OFTπ0.5\pi_{0.5}π0\pi_0OpenVLA
GoalLongGoalLongGoalLongGoalLong
frozen policy79.735.384.352.765.244.98.11.2
scene restoration only72.8∗72.8^{*}51.3∗51.3^{*}76.0∗76.0^{*}60.4∗60.4^{*}82.6∗82.6^{*}62.5∗62.5^{*}22.3∗22.3^{*}7.2∗7.2^{*}
+ whole robot incl. pedestal39.0∗39.0^{*}31.072.3∗72.3^{*}53.974.3∗74.3^{*}43.44.4∗4.4^{*}0.0
+ moving links, pose per step88.2∗88.2^{*}70.6∗70.6^{*}93.6∗93.6^{*}83.1∗83.1^{*}90.7∗90.7^{*}74.0∗74.0^{*}36.3∗36.3^{*}12.2∗12.2^{*}
+ moving links, pose once90.0∗90.0^{*}71.4∗71.4^{*}93.4∗93.4^{*}92.6∗92.6^{*}92.9∗92.9^{*}73.5∗73.5^{*}35.8∗35.8^{*}11.0∗11.0^{*}
+ depth-edge filter (HOMEWARD)91.4∗91.4^{*}72.3∗72.3^{*}94.4∗94.4^{*}94.0∗94.0^{*}93.4∗93.4^{*}78.3∗78.3^{*}39.2∗39.2^{*}11.2∗11.2^{*}
π3\pi^3 instead of VGGT88.0∗88.0^{*}74.5∗74.5^{*}94.6∗94.6^{*}89.5∗89.5^{*}94.9∗94.9^{*}74.5∗74.5^{*}––
reference frame + robot, no restoration70.8∗70.8^{*}41.578.2∗78.2^{*}63.0∗63.0^{*}85.5∗85.5^{*}69.2∗69.2^{*}11.30.0
HOMEWARD (per-step), no hole filling84.674.5∗74.5^{*}95.6∗95.6^{*}80.0∗80.0^{*}90.2∗90.2^{*}74.2∗74.2^{*}29.2∗29.2^{*}12.4∗12.4^{*}
HOMEWARD (per-step), other reference frame89.0∗89.0^{*}72.6∗72.6^{*}94.1∗94.1^{*}82.3∗82.3^{*}93.1∗93.1^{*}75.9∗75.9^{*}36.3∗36.3^{*}13.1∗13.1^{*}
HOMEWARD (per-step), reference of another task87.3∗87.3^{*}49.2∗49.2^{*}95.3∗95.3^{*}67.3∗67.3^{*}91.4∗91.4^{*}66.8∗66.8^{*}34.3∗34.3^{*}8.6∗8.6^{*}

Table 7. Ablation study. Success rate (%) on the LIBERO-Plus camera variants of Goal (408) and Long (419). Rows marked (per-step) change one component of the per-step variant. – : not run. * McNemar p<0.05p < 0.05 against the frozen policy.

Pose Caching, Motion Gating, and Runtime

Pose caching. By default, we estimate and cache (Kt,Et)(K_t, E_t) at the first policy query (t=0t=0), which keeps correspondence matching away from later arm motion. Subsequent queries run single-frame depth inference on It′I'_t alone and rescale the prediction Dˉt\bar{D}_t to the cached initial depth D0D_0:

Dt=stDˉt,st=median⁡u∈ΩtD0(u)Dˉt(u),(3)D_t = s_t \bar{D}_t,\qquad s_t = \operatorname{median}_{u \in\Omega_t}\frac{D_0(u)}{\bar{D}_t(u)}, \tag*{(3)}

where Ωt\Omega_t collects the static pixels whose mean absolute color difference from I0′I'_0 is below 12 of 255 intensity levels—decoupling depth scaling from arm articulation. For scenarios involving continuous intra-episode camera motion, a per-step variant re-estimates two-view pose at every query.

Do-no-harm motion gating. A motion gate checks the initial estimate at t=0t=0: if estimated rotation is below 0.5∘0.5^\circ, translation is under 1% of median scene depth, and focal length differs by less than 10%, HOMEWARD bypasses restoration and forwards raw frames untouched for the rest of the episode (the per-step variant re-tests at every query).

Runtime overhead. Using VGGT-1B (518 px, bfloat16) on a single NVIDIA A100 GPU, cached scene restoration takes 0.16 s per policy query and kinematic robot rendering requires 0.01 s (per-step pose estimation adds 0.04 s). Estimated from the wall-clock difference between restored and frozen closed-loop rollouts, total overhead is ∼\sim0.21 s per query—amortizing to 27 ms per control step for the 8-step action-chunking policies [6, 48]. The simulator pauses during inference; letting the robot instead hold its pose for 5 control steps (0.25 s) at every query changes success by −3.7 to +1.2 points on Goal and Long (supplement).

Experiments

Experimental Setup

Benchmarks and Camera Perturbations. We evaluate five camera-shift regimes alongside unperturbed controls. (1) LIBERO-Plus Camera [8] perturbs the third-person camera across the four LIBERO suites [26] in azimuth, elevation, distance, and in-place roll across five severity levels; we evaluate all 1,599 variants (one episode per variant from initial state 0). (2) AnyCamVLA Orbits [14] orbit the camera around the workspace across small, medium, and large offsets (up to 15 cm radial distance and 60∘60^\circ azimuth), evaluated across 50 initial states per task on Long and 10 on the remaining suites. (3) CamRand samples random 3D camera positions within spheres of three radii around C⋆C^\star, re-aiming at a jittered workspace target with random roll (300 episodes per suite; 1,200 total). (4) Camera Drift continuously translates and rotates the camera from C⋆C^\star to an AnyCamVLA orbital pose over the first 120 control steps (1,200 episodes). (5) Lens Change alters the field of view (FoV) from 45∘45^\circ to 55∘55^\circ or 35∘35^\circ at fixed extrinsics (800 episodes). (6) Unmoved Camera Controls comprise 100 clean-camera episodes per suite and all 2,400 non-camera perturbation variants of LIBERO-Plus.

Evaluated Policies and Statistical Protocol. We test official LIBERO checkpoints of OpenVLA-OFT [21], π0.5\pi_{0.5} [33], π0\pi_0 [2], and OpenVLA [20] with strictly frozen weights, rendered via the GPU EGL backend of MuJoCo [38] (reproducing [8] under its official prompt convention; see supplement). LIBERO-Plus, CamRand, and clean episodes use standard task instructions, while AnyCamVLA-orbit, drift, and lens-change benchmarks use LIBERO-Plus paraphrased instructions identically across all methods. Each task’s IrefI_{\mathrm{ref}} is rendered once from an initial state other than the evaluated one, so its object and arm poses differ from the test episode. All comparisons are paired on identical episodes using exact McNemar tests [28], supplemented by 95% task-level cluster bootstrap [7] confidence intervals (reported in brackets) that resample tasks (40 on LIBERO-Plus) to account for shared scene geometry. Because the gate never triggered in 4,797 displaced-camera and 600 lens-change trials, we report ungated cached execution and ungated per-step execution under drift. For OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0, per-query gating changes drift success by −1.8-1.8 to +0.5+0.5 points and stays within 0.8 points of frozen policies on clean cameras (supplement).

Main Results on LIBERO-Plus

Consistent recovery across architectures and severities. Across all 1,599 camera variants of LIBERO-Plus (Table 3), HOMEWARD improves aggregate task success across every VLA—boosting OpenVLA-OFT, π0.5\pi_{0.5}, π0\pi_0, and OpenVLA by +15.1+15.1, +18.6+18.6, +23.2+23.2, and +22.8+22.8 points (p<10−31p < 10^{-31} each). Out of 16 suite–policy combinations, 14 improve significantly and none degrades significantly; the largest gains emerge on long-horizon manipulation (LIBERO-Long, +33.4+33.4 to +41.3+41.3 points across the first three policies). Resampling the 40 tasks yields 95% confidence intervals of [7.6, 22.9], [10.3, 27.1], [17.1, 29.7], and [17.7, 28.4] points, with per-task success improving on 26, 23, 35, and 38 of 40 tasks (and decreasing on only 5, 9, 1, and 1; task-level Wilcoxon signed-rank p<0.01p < 0.01 for every policy).

SuiteVariantsOpenVLA-OFT frozenOpenVLA-OFT + HOMEWARDπ0.5\pi_{0.5} frozenπ0.5\pi_{0.5} + HOMEWARDπ0\pi_0 frozenπ0\pi_0 + HOMEWARDOpenVLA frozenOpenVLA + HOMEWARD
Spatial37691.897.3*76.998.1*79.598.7*3.232.2*
Object39645.750.393.293.9*84.695.7*2.023.7*
Goal40879.791.4*84.394.4*65.293.4*8.139.2*
Long41935.372.3*52.794.0*44.978.3*1.211.2*
All1,59962.577.6*76.595.1*68.091.2*3.626.4*
Gain+15.1+18.6+23.2+22.8

Table 3. LIBERO-Plus camera dimen­sion. Success rate (%) of frozen policies on all 1,599 camera variants, one episode per variant, without and with HOMEWARD. * exact McNemar p<0.05p < 0.05 on the paired episodes; for “All”, p<10−31p < 10^{-31} for every policy. No policy is retrained and no calibration of the moved camera is given.

Graceful degradation under extreme camera shifts. Stratifying results across perturbation severity levels 1–5 (Figure 3) shows that while frozen OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0 suffer catastrophic drops of 42–72 points from Level 1 to Level 5, HOMEWARD restricts losses to 5–34 points. At Level 1, where perturbations are minimal, restoration costs OpenVLA-OFT 6.1 points (87.4% vs. 93.5%). Non-3D alternatives fall short: temporal ensembling and 2D registration to IrefI_{\mathrm{ref}} move OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0 by at most 2.7 points. With the same robot layer, 2D registration and oracle homography trail HOMEWARD by 3.4–26.5 points on Goal and Long (all 12 paired comparisons against HOMEWARD significant; supplement).

Generalization to Diverse Camera Shifts

One pipeline for 6-DoF shifts, lens swaps, and drift. Under CamRand (Table 4), HOMEWARD lifts the first three VLAs by +3.3+3.3 to +9.8+9.8 points and more than doubles OpenVLA (30.5% → 62.8%). On AnyCamVLA-style orbits across Goal, Spatial, and Object, it yields +8.9+8.9 to +20.4+20.4 point gains. Under Lens Change (45∘→35∘/55∘45^\circ\to35^\circ/55^\circ FoV at fixed pose), VGGT’s per-frame intrinsics KtK_t correct perspective scaling, adding +5.7+5.7 to +12.7+12.7 points on the stronger policies and +28.9+28.9 points on OpenVLA. Under Camera Drift, cached poses grow stale, whereas per-step pose tracking follows the moving camera to recover +13.7+13.7 to +15.0+15.0 points on OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0 (OpenVLA does not improve under drift). Finally, pairing large random camera shifts with LIBERO-Plus’s 2,400 non-camera perturbations (including object layout) yields +11+11 to +21+21 point improvements per policy, with gains in all six perturbation categories (supplement).

OFTπ0.5\pi_{0.5}π0\pi_0OpenVLA
CamRand
frozen83.893.988.230.5
+ HOMEWARD93.6*97.2*93.0*62.8*
AnyCamVLA-style, other suites
frozen62.059.343.35.9
+ HOMEWARD70.9*79.7*62.3*12.6*
Camera drift
frozen58.167.550.311.4
+ HOMEWARD65.0*66.155.0*8.5*
+ HOMEWARD, per-step pose72.5*82.5*64.0*12.4
Lens change
frozen82.482.453.25.1
+ HOMEWARD88.4*88.1*65.9*34.0*

Table 4. Other camera changes. Success rate (%) pooled over suites: CamRand on all four (1,200 episodes), AnyCamVLA on Goal, Spatial and Object (900), drift and lens change on all four (1,200 and 800). * McNemar p<0.05p < 0.05.

Comparison with Calibrated View Synthesis

We compare against AnyCamVLA [14], which fine-tunes a view-synthesis network on LIBERO-Plus renders and requires the true deployment camera.

Competitive or superior on LIBERO-Plus. Across all 1,599 LIBERO-Plus variants (Table 5), training-free, uncalibrated HOMEWARD outperforms AnyCamVLA—even when AnyCamVLA is given ground-truth camera poses—by +9.9+9.9 points on π0.5\pi_{0.5} ([3.5, 16.8]) and +3.8+3.8 points on π0\pi_0 ([0.3, 7.5]), matches it on OpenVLA (+0.4+0.4, [−3.2, 4.1]), and trails by only 3.1 points on OpenVLA-OFT ([−11.3, 4.8]); Figure 4 compares the restored views. Under an equal uncalibrated setting where AnyCamVLA also consumes VGGT-estimated poses, HOMEWARD leads by +8.1+8.1 to +12.9+12.9 points across π0.5\pi_{0.5}, π0\pi_0, and OpenVLA (and is within −1.9-1.9 points, [−9.2, 5.3], on OpenVLA-OFT).

scene trainingmoved cameraOFTπ0.5\pi_{0.5}π0\pi_0OpenVLA
frozen policy––62.576.568.03.6
AnyCamVLA‡fine-tunedtrue80.785.187.526.0
AnyCamVLA§fine-tunedestimated79.582.183.114.1
HOMEWARDnoneestimated77.695.191.226.4

Table 5. Comparison with calibrated view synthesis on all 1,599 LIBERO-Plus camera variants (success rate, %). AnyCamVLA runs with the true (‡) or VGGT-estimated (§) parameters of the moved camera. Best per column in bold; per-suite results are in the supplementary material.

Qualitative canonicalization at matched simulator states for a large AnyCamVLA orbit and an FoV narrowing

Figure 4. Qualitative canonicalization at matched simulator states for a large AnyCamVLA orbit (Long) and an FoV narrowing (45∘→35∘45^\circ\to35^\circ). In frames from episodes where the frozen policy fails and HOMEWARD succeeds, view synthesis blurs or omits fine objects, whereas HOMEWARD preserves sharp scene geometry and crisp manipulator links.

Characterizing the wide-orbit boundary. On AnyCamVLA’s native wide-orbit benchmark on LIBERO-Long, HOMEWARD improves frozen policies by +28+28 to +33+33 points, yet trails ground-truth-calibrated AnyCamVLA by 15.2, 4.8, and 5.8 points on OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0, with the gap widening at larger orbits. Two controlled experiments isolate the mechanism (supplement): (i) without its fine-tuning on the benchmark scenes, AnyCamVLA’s synthesis pipeline collapses below the uncorrected policy; and (ii) supplying oracle depth and poses to HOMEWARD closes the gap to within 3 points—or surpasses AnyCamVLA—even at the widest orbit. Diagnostics confirm that while VGGT’s median rotation error stays near ∼1∘\sim1^\circ, its median translation error grows from 2 cm at small orbits to 7 cm at large orbits, pointing to wide-baseline translation drift as the bottleneck.

Complementarity with Policy Fine-Tuning

We next evaluate OpenVLA-OFT+ [8], obtained by fine-tuning OpenVLA-OFT on >20,000>20{,}000 perturbed LIBERO-Plus trajectories (Table 6).

OFTOFT+ (fine-tuned)
frozen+ oursfrozen+ ours
LIBERO-Plus camera62.577.6{sup}*`95.791.1*
AnyCamVLA, Long33.761.7{sup}*`68.283.3{sup}*`
AnyCamVLA-style, other suites62.070.9{sup}*`84.287.4{sup}*`
CamRand83.893.6{sup}*`91.295.4{sup}*`
Drift (per-step pose)56.074.7{sup}*`76.586.7{sup}*`
Lens change80.282.283.086.0
Clean camera94.894.896.296.0

Table 6. Restoration on top of a fine-tuned policy. Success rate (%) pooled as in Tabs. 3 and 4, except Drift (Goal, Long) and Lens change (Goal, Spatial), the suites released for OpenVLA-OFT+; drift uses the per-step variant, the clean camera the gated default. * McNemar p<0.05p < 0.05 against the same policy frozen.

Complementary gains on out-of-distribution shifts. On the LIBERO-Plus camera split—which lies close to OpenVLA-OFT+’s fine-tuning distribution (95.7% base success)—test-time restoration is unnecessary and reduces success by 4.7 points ([−10.1, −0.1], to 91.1%). However, across all five out-of-distribution camera perturbation regimes absent from fine-tuning, plugging in HOMEWARD consistently boosts OpenVLA-OFT+ by +3.0+3.0 to +15.1+15.1 points—including +15.1+15.1 points on Long orbits ([8.1, 23.7]), +10.2+10.2 points under camera drift ([2.7, 19.2]), and +4.2+4.2 points on CamRand ([1.2, 7.0]), while leaving clean-camera performance intact (−0.2 points, [−1.2, 0.5]).

Ablation Study

Necessity of selective kinematic link rendering. Adding moving-link rendering and the 3% depth-edge filter to scene restoration only (Tab. 7) yields dramatic jumps (+18.4 to +33.6 points for OpenVLA-OFT and π0.5\pi_{0.5}; the filter alone contributes +0.5 to +4.8 points) and matches or exceeds oracle re-rendering (Tab. 2) in 6 of 8 suite–policy cells—because kinematic rendering eliminates arm disocclusion shadows that afflict even privileged point-cloud reprojection.

Impact of estimated vs. oracle scene geometry. Pairing our kinematic robot layer with simulator scene geometry raises OpenVLA-OFT across all 1,599 variants from 77.6% to 90.2% (driven largely by LIBERO-Object: 94.9% vs. 50.3%, a suite where the same estimated geometry already carries π0.5\pi_{0.5} and π0\pi_0 to 93.9% and 95.7%), whereas π0.5\pi_{0.5} and π0\pi_0 change by only 1.3 and 1.2 points overall (supplement).

Robustness to the geometry backbone. Replacing VGGT with π3\pi^3 [42] alters performance across the six primary cells by only −4.5-4.5 to +2.2+2.2 points, achieving 76.7% vs. 77.6% overall on OpenVLA-OFT across all 1,599 variants (p=0.41p = 0.41). Sampled pixel MAE against ground-truth training frames is 4.5–6.0 intensity levels—matching oracle re-projection (4.6–6.2) and below uncorrected views (32–55).

Live reprojection, pose caching, and reference choice. Compositing the kinematic arm onto a static IrefI_{\mathrm{ref}} without live scene reprojection underperforms HOMEWARD in 14 of 16 cells (13 significantly; e.g., 41.5% vs. 72.3% for OFT on Long). Caching initial pose (Kt,Et)(K_t, E_t) performs within 2.2 points of per-step estimation in 5 of 6 cells while surging +9.5 points ahead for π0.5\pi_{0.5} on Long (92.6% vs. 83.1%), where per-step pose noise accumulates. Varying IrefI_{\mathrm{ref}} initial states or disabling hole filling shifts success by ≤4\le4 points; using a cross-task reference from the same room costs ≤0.9\le0.9 points on Goal, though 7–21 points on Long where scene object layouts change across tasks.

Zero-Shift Safety and Seed Stability

Do-no-harm under unmoved cameras. On 400 clean-camera episodes per policy, the motion gate passes raw observations through in 99.25% (1,985/2,000) of rollouts, keeping success within ±0.5\pm0.5 points ([−2.2-2.2, 1.8]). Without the gate, restoring clean views alters success by at most 1.8 points ([−6.8-6.8, 3.8]), and on LIBERO-Plus’s 2,400 non-camera variants by −0.2-0.2 to +3.8+3.8 points (all significant deltas positive). Over three seeds on all 1,599 LIBERO-Plus variants, π0.5\pi_{0.5} rises from 76.2 ±\pm 0.5% to 94.6 ±\pm 0.5% and π0\pi_0 from 67.7 ±\pm 1.6% to 90.5 ±\pm 0.9% (mean ±\pm s.d.; supplement).

Conclusion

We investigated how far frozen VLA policies can be recovered from uncalibrated third-person camera shifts without any training. Closed-loop diagnostics revealed that while viewpoint degradation is largely recoverable in observation space, feed-forward 3D models falter on thin, moving manipulator links—the foreground structure already known via robot kinematics. By pairing monocular scene geometry with kinematic moving-link rendering, HOMEWARD delivers consistent gains for OpenVLA-OFT, π0.5\pi_{0.5}, and π0\pi_0 across five camera perturbation regimes and for OpenVLA in all but drift, matching or exceeding calibrated, scene-trained view synthesis on three of four policies on LIBERO-Plus.

Limitations and Future Work. Scene-trained synthesis remains an advantage under wide orbital baselines, where monocular translation estimation drifts, and restoration is unnecessary on viewpoints a fine-tuned policy has mastered. Extending HOMEWARD from simulation to physical robots is an important next step—requiring robustness to URDF-to-real photometric gaps, foreground occlusions in front of the manipulator, and contact-rich gripper–object interactions.

LLM Usage Statement

LLM-based tools were used extensively in this work, under the authors’ direction and review. Tools. Claude Code (Anthropic) was the main coding and writing assistant; OpenAI Codex was used for review, auditing and manuscript preparation. Research. The tools searched and summarized related work; wrote and debugged most of the restoration, evaluation, experiment-orchestration and analysis code; launched and monitored the simulation runs; proposed controls, ablations and analyses, among which the authors chose; and audited the consistency of claims, statistics and tables. Writing and figures. The paper and its supplement were drafted and revised with these tools from the authors’ instructions. Plots and tables were produced with AI-assisted scripts. Figures 1 and 2 were assembled in LaTeX/TikZ from recorded simulator images and MuJoCo-rendered robot illustrations; the camera icon in Figure 2 is from 3dicons. Results. All reported success rates and statistical comparisons are computed from logged closed-loop episodes; no simulator observation or trial outcome was generated by an LLM. The authors directed the work, decided which experiments to run and which claims to make, and are responsible for all content.

Qualitative results

Figure S1 shows restored views for all five kinds of camera change; view synthesis blurs or omits scene details, whereas HOMEWARD preserves the observed structure and redraws the arm. Every row shows one simulator state, mid-episode, as the moved camera sees it, as AnyCamVLA synthesizes the training view from it with estimated and with true camera parameters, as HOMEWARD restores it, and as the training camera sees it. Figure S2 follows three episodes through time from the training camera, without and with HOMEWARD. All episodes are drawn at random (fixed seed) among those where the frozen OpenVLA-OFT fails and HOMEWARD succeeds, one per camera-shift level and benchmark; images are at the policy’s input resolution.

Restored views showing five camera-change conditions across moved camera (policy input), AnyCamVLA (estimated cameras), AnyCamVLA (true cameras), ViewBack (ours), and training camera (ground truth), for LIBERO-Plus Goal level 5, LIBERO-Plus Long level 5, AnyCamVLA orbit large, CamRand large radius, and lens change 45° 35°

Figure S1. Restored views of the same simulator state for all five kinds of camera change, OpenVLA-OFT driven by HOMEWARD.

Episodes seen from the training camera, frozen OpenVLA-OFT and with HOMEWARD

Figure S2. Episodes seen from the training camera, frozen OpenVLA-OFT (top row of each pair) and with HOMEWARD (bottom row), at evenly spaced queries of each episode; one query is eight control steps. Top: LIBERO-Plus Goal, level 5 (the scene of the main paper’s first figure). Middle: LIBERO-Plus Long, level 5. Bottom: AnyCamVLA large orbit on Long.

Controls on all four suites

The ablation table of the main paper covers Goal and Long; Table S1 runs its controls on all four LIBERO-Plus camera suites. For OpenVLA, oracle re-rendering without the robot layer lifts Spatial from 3.2% to 29.8%, against 32.2% for HOMEWARD. The reference frame with the current robot drawn on it is below HOMEWARD in 14 of the 16 suite–policy cells, significantly in 13. It is above HOMEWARD for π0.5\pi_{0.5} on Object (98.2% against 93.9%, p=0.003p = 0.003) and, not significantly, for π0\pi_{0} on Object (97.0% against 95.7%); on Object the oracle with the robot layer reaches 94.9–99.0% for the first three policies, so the shortfall there lies in the estimated geometry. Against AnyCamVLA with the true cameras, HOMEWARD is ahead on Goal and Long for the first three policies (significantly in four of the six cells) and on Spatial for all but OpenVLA-OFT, where the two are on par (97.3% against 98.1%); it trails on Object for OpenVLA-OFT and π0.5\pi_{0.5} and on Long for OpenVLA (11.2% against 15.5%, p=0.025p = 0.025). With VGGT-estimated cameras, the information HOMEWARD itself works from, AnyCamVLA is below HOMEWARD in 13 of the 16 cells, significantly in 11;

Suitefrozenref. + robotHOMEWARDAnyCam.‡AnyCam.§oracle + robot
OpenVLA-OFT
Spatial91.888.397.3*98.1*99.5*91.5
Object45.712.6*50.383.6*76.5*94.9*
Goal79.770.8*91.4*81.485.0*91.4*
Long35.341.572.3*61.6*58.9*83.3*
All62.552.8*77.6*80.7*79.5*90.2*
π0.5\pi_{0.5}
Spatial76.993.4*98.1*78.274.298.1*
Object93.298.2*93.998.5*97.2*99.0*
Goal84.378.2*94.4*91.7*89.2*97.1*
Long52.763.0*94.0*72.3*68.0*91.9*
All76.582.7*95.1*85.1*82.1*96.4*
π0\pi_{0}
Spatial79.597.9*98.7*95.2*93.6*97.9*
Object84.697.0*95.7*97.2*94.9*96.5*
Goal65.285.5*93.4*85.5*79.4*95.8*
Long44.969.2*78.3*73.3*66.1*80.2*
All68.087.0*91.2*87.5*83.1*92.4*
OpenVLA
Spatial3.222.6*32.2*25.8*13.8*62.0*
Object2.00.0*23.7*23.7*19.4*61.1*
Goal8.111.339.2*39.2*19.4*52.7*
Long1.20.011.2*15.5*4.1*25.1*
All3.68.2*26.4*26.0*14.1*49.7*

Table S1. Controls on all four suites. Success rate (%) on the LIBERO-Plus camera variants (376, 396, 408 and 419 per suite). Ref. + robot: the reference frame with the current robot drawn on it, no restoration. AnyCam.: AnyCamVLA’s view synthesis with the true (‡) or VGGT-estimated (§) parameters of the moved camera, as in the main paper. Oracle + robot: re-rendering with the simulator’s depth and true cameras plus the kinematic robot layer; not deployable. * McNemar p<0.05p < 0.05 against the frozen policy.

in 11; it is above on Object for OpenVLA-OFT and π0.5\pi_{0.5} (76.5% against 50.3%, 97.2% against 93.9%) and on Spatial for OpenVLA-OFT (99.5% against 97.3%).

On AnyCamVLA’s own protocol, run in our harness so both methods see the same episodes, the released model reaches 76.9% with OpenVLA-OFT and 84.9% with π0.5\pi_{0.5}, close to the reported 78.9% and 88.6% [14]. Table S2 breaks LIBERO-Long down by orbit. At the small orbit, HOMEWARD is ahead of AnyCamVLA with true cameras for π0.5\pi_{0.5} (92.0% against 86.4%) and π0\pi_{0} (69.4% against 68.4%), and behind for OpenVLA-OFT (88.4% against 89.6%) and OpenVLA (8.0% against 19.6%); the calibrated model, fine-tuned on LIBERO-Plus renders, leads at the medium and large orbits. Without that fine-tuning the same pipeline (zero-shot LVSM) falls below the frozen policy, whereas HOMEWARD is trained on no scene. A different 3D model does not change the picture: with π3\pi^{3} [42] the large orbit moves by at most a few points, and with the simulator’s depth and cameras the same re-rendering reaches 70.0%, 81.0% and 63.6% there, near or above AnyCamVLA, so the limit at wide orbits is the estimated geometry rather than content the moved camera never observed.

LIBERO-Longsmallmediumlargeall
OpenVLA-OFT
frozen80.69.611.033.7
+ HOMEWARD88.462.634.061.7*
+ AnyCamVLA‡89.679.461.676.9*
+ AnyCamVLA§85.678.857.874.1*
+ zero-shot LVSM‡40.818.22.620.5*
+ zero-shot LVSM§41.618.64.621.6*
π0.5\pi_{0.5}
frozen86.455.69.050.3
+ HOMEWARD92.079.069.280.1*
+ AnyCamVLA‡86.484.683.684.9*
+ AnyCamVLA§86.883.280.883.6*
+ zero-shot LVSM‡70.017.814.834.2*
+ zero-shot LVSM§68.617.413.433.1*
π0\pi_0
frozen66.016.65.429.3
+ HOMEWARD69.465.451.462.1*
+ AnyCamVLA‡68.469.665.667.9*
+ AnyCamVLA§64.466.865.665.6*
+ zero-shot LVSM‡49.610.87.022.5*
+ zero-shot LVSM§50.812.07.223.3*
OpenVLA
frozen7.60.00.02.5
+ HOMEWARD8.02.60.03.5
+ AnyCamVLA‡19.65.41.48.8*
+ AnyCamVLA§3.00.40.41.3*

Table S2. AnyCamVLA protocol on LIBERO-Long, 500 episodes per offset (10 tasks × 50 initial states). Our frozen OpenVLA-OFT and π0.5\pi_{0.5} reach 33.7% and 50.3%, within 1.3 points of the 32.9% and 49.0% reported in [14]. ‡ AnyCamVLA’s released view-synthesis model run in our harness with the true parameters of the moved third-person camera, the wrist camera and the training camera; it is fine-tuned on LIBERO-Plus renders. § The same model with the moved camera’s pose and intrinsics estimated by VGGT against a training-view reference frame, as HOMEWARD estimates them; the metric scale comes from the training view’s depth, and the wrist and training cameras keep their true parameters. Zero-shot LVSM: the same pipeline with LVSM’s public RealEstate10K weights [17], without AnyCamVLA’s fine-tuning. HOMEWARD requires no supplied calibration of the moved or wrist camera. * McNemar p<0.05p < 0.05 against the frozen policy.

Ghost arm and camera refinement

HOMEWARD reprojects every pixel of the moved frame, the real arm included. When the estimated camera is off, this warped copy of the real arm appears beside the arm that the robot layer draws. On the AnyCamVLA protocol it pulls π0.5\pi_{0.5} on Object at the medium orbit below the frozen policy (61.4% against 76.0%); with the simulator’s depth and camera the same episodes reach 89.6%. We tested two fixes. Erasing the real arm before warping, with a mask of the moving links rendered from the estimated camera, lifts that cell to 88.4% but costs π0.5\pi_{0.5} 8.3 points on LIBERO-Plus Long, where the misplaced mask cuts objects next to the arm. Refining the camera first, by a cross-entropy search that matches the pixels the moving links change over up to seven frames of the episode, raises the median mask overlap on Goal and Long from 0.18 and 0.42 to 0.71 and 0.76. With the refined camera the erasure keeps the gain for π0.5\pi_{0.5} on Object but does not raise success in general (Table S3): it helps π0.5\pi_{0.5} on the AnyCamVLA protocol, leaves π0\pi_0 unchanged and lowers OpenVLA on Object. HOMEWARD therefore keeps the camera estimated from the first frame, and Object remains its weakest suite.

All suitesObject
PolicyHOMEWARD+ refined eraseHOMEWARD+ refined erase
LIBERO-Plus camera variants
π0.5\pi_{0.5}95.195.493.997.0*
π0\pi_091.290.395.797.0
OpenVLA31.6†27.7*23.712.1*
AnyCamVLA protocol, all orbits
π0.5\pi_{0.5}78.582.7*76.889.3*
π0\pi_061.061.974.976.7
OpenVLA––8.50.3*

Table S3. Erasing the real arm with a refined camera. Success rate (%) of HOMEWARD without and with erasure of the real arm under a camera refined from the arm’s motion, on the same episodes. All suites: the 1,599 LIBERO-Plus camera variants or the 6,000 AnyCamVLA episodes; † Goal and Object only (804). * McNemar p<0.05p < 0.05 against HOMEWARD.

Compound perturbations

LIBERO-Plus perturbs each suite along six factors other than the camera, with 100 variants each. We combine each of these 2,400 variants with a CamRand camera at the largest radius, so that the policy faces a moved camera and a second perturbation at once. HOMEWARD raises all four policies in every category (Table S4); it restores the camera, and the other factor stays as LIBERO-Plus sets it.

CategorynnOFT frozenOFT + HOMEWARDπ0.5\pi_{0.5} frozenπ0.5\pi_{0.5} + HOMEWARDπ0\pi_0 frozenπ0\pi_0 + HOMEWARDOpenVLA frozenOpenVLA + HOMEWARD
Background Textures40062.280.5*83.292.8*69.084.8*4.831.2*
Language Instructions40058.278.0*75.288.5*53.065.2*3.023.8*
Light Conditions40069.889.0*88.296.2*75.589.8*3.224.8*
Objects Layout40046.567.2*76.584.5*58.275.5*4.534.2*
Robot Initial States40023.234.2*66.580.5*36.248.2*2.511.8*
Sensor Noise40053.871.8*79.593.5*69.089.5*0.819.0*
All240052.370.1*78.289.3*60.275.5*3.124.1*

Table S4. Compound perturbations. Success rate (%) on the 2,400 non-camera variants of LIBERO-Plus (600 per suite), each under a CamRand camera at the largest radius, by perturbation category; one episode per variant. * McNemar p<0.05p < 0.05 against the frozen policy.

Calibration error of the training camera

HOMEWARD draws the moving links at the training camera, which it takes as calibrated once, before deployment. Perturbing that calibration keeps every cell above its frozen baseline (Table S5); the margin is smallest for OpenVLA-OFT on Goal at 20 mm and 2° (86.0% against 81.6%, not significant). Up to 10 mm and 1° of error no cell loses more than 3.2 points against the exact calibration. At 20 mm and 2° the largest losses are on Long: OpenVLA-OFT drops from 71.1% to 58.9% and π0.5\pi_{0.5} from 92.4% to 86.6%, both still far above their frozen 34.8% and 52.7%.

OFT GoalOFT Longπ0.5\pi_{0.5} Goalπ0.5\pi_{0.5} Longπ0\pi_0 Goalπ0\pi_0 Long
frozen policy81.634.883.852.765.244.9
HOMEWARD, exact training camera87.3*71.1*95.6*92.4*92.9*73.5*
camera off by 5 mm, 0.5°89.0*69.5*94.9*89.7*92.9*74.2*
camera off by 10 mm, 1°88.7*68.0*93.4*90.9*89.7*75.9*
camera off by 20 mm, 2°86.058.9*93.4*86.6*90.7*75.2*

Table S5. Calibration error of the training camera. Success rate (%) on the LIBERO-Plus camera variants of Goal (408) and Long (419) when the training camera that the robot is drawn from is off by the given translation and rotation (random direction and axis, fixed per episode). All rows run on one machine, whose renderer differs slightly from that of the ablation table of the main paper, and without the depth-edge filter. * McNemar p<0.05p < 0.05 against the frozen policy.

Generic test-time fixes

Table S6 gives the numbers behind the comparison with temporal ensembling and 2D registration in the main paper; Sec. G adds the robot layer to 2D registration.

OFTπ0.5\pi_{0.5}π0\pi_0
frozen policy62.576.568.0
+ temporal ensembling of action chunks64.7*74.867.3
+ 2D registration to the reference frame62.474.1*70.7*
+ HOMEWARD77.6*95.1*91.2*

Table S6. Generic test-time fixes. Success rate (%) on all 1,599 LIBERO-Plus camera variants, pooled over the four suites. Temporal ensembling queries the policy every 4 control steps and averages the overlapping action chunks [48]. 2D registration warps every frame onto the reference frame with a homography fitted once per episode to SIFT matches by RANSAC [9, 27]; like HOMEWARD, it requires no supplied camera calibration. * McNemar p<0.05p < 0.05 against the frozen policy.

2D alignment with the robot layer

Table S7 asks whether the 3D scene layer is needed once the robot is drawn from kinematics. It gives two 2D alignments the same kinematic robot layer as HOMEWARD: the SIFT/RANSAC homography of Table S6, fitted once per episode, and the plane-induced homography computed from the true cameras. The robot layer helps both alignments in 11 of 12 cells, by up to 27.2 points; 2D registration loses 6.9 points for OpenVLA-OFT on Goal. HOMEWARD stays ahead in all 12 comparisons, by 3.4 to 19.9 points on Goal and by 14.8 to 26.5 points on Long (McNemar p≤0.044p \le0.044).

OFT GoalOFT Longπ0.5\pi_{0.5} Goalπ0.5\pi_{0.5} Longπ0\pi_0 Goalπ0\pi_0 Long
frozen policy79.735.384.352.765.244.9
2D registration78.444.6*84.856.170.8*49.6*
+ robot layer71.6*51.3*89.7*67.5*87.5*61.8*
oracle homography†70.8*38.774.8*51.360.841.5
+ robot layer79.7*46.3*90.9*78.5*86.0*63.5*
scene restoration only72.8*51.3*76.060.4*82.6*62.5*
HOMEWARD91.4*72.3*94.4*94.0*93.4*78.3*

Table S7. 2D alignment with the kinematic robot layer. Success rate (%) on the LIBERO-Plus camera variants of Goal (408) and Long (419). † uses the true cameras; not deployable. * McNemar p<0.05p < 0.05 against the frozen policy.

Colour error of the robot layer

A mesh render of a real robot differs in colour from the camera image. We draw the robot layer with a per-episode gain and gamma error of size SS per channel, separately for light and dark parts, times a smooth shading field of amplitude 0.4S0.4S; the colour-fit rows refit the colours once per episode on the reference frame. S=50S = 50 matches the error we measure for a real-robot render: about 30 grey levels in the robot region before the fit and 17 after it. Success falls with SS for OpenVLA-OFT on Long (72.3% to 57.3% at S=100S = 100) and more slowly for π0.5\pi_{0.5} (94.0% to 88.3%); π0\pi_0 on Long does not fall. The colour fit does not recover the loss consistently (Table S8).

OFTπ0.5\pi_{0.5}π0\pi_0†OpenVLA
GoalLongGoalLongGoalLongSpatialObject
frozen policy79.735.384.352.764.545.63.22.0
exact render91.4*72.3*94.4*94.0*91.7*73.3*32.2*23.7*
S=25S = 2591.9*67.1*92.6*91.4*94.6*78.5*31.6*20.7*
S=50S = 5089.2*64.2*92.2*90.2*91.9*77.1*28.7*15.4*
S=100S = 10087.5*57.3*90.7*88.3*90.7*81.4*28.7*13.4*
S=50S = 50, colour fit87.3*64.9*92.6*86.6*92.4*80.7*26.1*–
S=100S = 100, colour fit89.7*59.2*90.2*85.2*90.4*80.4*19.4*14.6*

Table S8. Colour error of the robot layer. Success rate (%) on the LIBERO-Plus camera variants. † noise seed 9, paired with the frozen policy under the same seed. OpenVLA ran this study on Spatial and Object first. – : not run. * McNemar p<0.05p < 0.05 against the frozen policy.

On the Goal and Long camera variants used in the ablation table of the main paper, Table S9 lists the same control at S=50S = 50 for all four policies. For the first three policies the colour error costs at most 2.2 points on Goal; on Long it costs OpenVLA-OFT 8.1 points and π0.5\pi_{0.5} 3.8 and does not lower π0\pi_0.

OFTπ0.5\pi_{0.5}π0\pi_0OpenVLA
GoalLongGoalLongGoalLongGoalLong
HOMEWARD, exact render91.4*72.3*94.4*94.0*93.4*78.3*39.2*11.2*
robot drawn with a real-robot colour error89.2*64.2*92.2*90.2*91.7*79.5*30.4*10.0*
+ colour fit on the reference frame87.3*64.9*92.6*86.6*93.9*79.7*21.8*13.6*

Table S9. Colour error on the suites of the main ablation table. Success rate (%) on the LIBERO-Plus camera variants of Goal (408) and Long (419), in the layout of the ablation table of the main paper. * McNemar p<0.05p < 0.05 against the frozen policy.

Unchanged camera

Table S10 gives the results behind the unchanged-camera paragraph of the main paper: 400 clean-camera episodes per policy, with the motion gate (default) and with the gate disabled, and the 2,400 non-camera variants of LIBERO-Plus.

OFTπ0.5\pi_{0.5}π0\pi_0OpenVLA
clean camera, 400 episodes
frozen94.896.892.575.2
+ HOMEWARD94.896.292.874.8
without motion gate95.897.092.873.5
non-camera variants of the four suites, 2,400 episodes
frozen74.590.776.034.1
+ HOMEWARD76.2*90.577.5*37.9*
without motion gate76.2*91.275.936.8*

Table S10. Performance without a camera shift. Success rate (%) on 400 clean-camera episodes (100 per suite) and on the 2,400 LIBERO-Plus non-camera variants of the four suites (lighting, background, noise, layout, robot initial state, language; 600 per suite). * McNemar p<0.05p < 0.05.

Motion gate under drift

In the cached variant the motion gate is decided once, at the first policy query, and kept for the rest of the episode. Every displaced-camera result of the main paper, drift included, is run with the gate disabled. The drift camera starts at the training pose at reset and reaches its target after 120 control steps; the first query follows 10 settling steps, when the camera has covered 8% of its path. Table S11 compares three settings of the per-step variant on drift. First query: from the first-query estimates logged by the ungated runs, a gate decided once (rotation and translation criteria) would fire in 95–96% of the 400 small-drift episodes, whose target is a 10∘10^\circ orbit, and in none of the 800 others; these columns replace the outcome of each such episode by that of the frozen policy and are computed, not run. Every query: a separate run in which the gate, with all three criteria, is decided again at each query. On small drifts it passes the first query through in 96% of the episodes and 8–10% of all queries, after which restoration takes over; on larger drifts it never fires. Against the ungated runs it changes pooled drift success by +0.5 [−0.6, 1.7], −1.8 [−3.5, −0.2] and −0.2 [−2.6, 1.8] points for OpenVLA-OFT, π0.5\pi_{0.5} and π0\pi_0 (intervals resample tasks). On the 400 clean-camera episodes the same per-query gate passes 92–93% of the queries through and scores 94.8%, 96.0% and 92.0%, against 94.8%, 96.8% and 92.5% for the frozen policies; no difference is significant.

small drift (400)all drift (1,200)
frozenno gatefirst queryevery queryfrozenno gatefirst queryevery query
OpenVLA-OFT85.285.284.886.858.172.572.373.0
π0.5\pi_{0.5}84.890.084.888.067.582.580.880.8
π0\pi_064.866.864.269.050.364.063.263.8
OpenVLA30.823.830.2–11.412.414.6–

Table S11. Motion gate under drift. Success rate (%) of the per-step variant. First query: computed from the ungated and the frozen runs of the same episodes. Every query: run with the gate decided at each query; OpenVLA was not run. On medium and large drifts the first-query gate never fires.

Computation delay

The simulation is synchronous: physics advances only when the environment is stepped, after the policy and the restoration have returned, so compute time delays no observation in the experiments of the main paper. Table S12 adds a delay of 5 control steps, 0.25 s at the 20 Hz control rate and above the 0.21 s that HOMEWARD adds per query, in two ways. Hold: inference blocks, as in a controller that waits for its policy; after each query the robot holds its pose for 5 steps (zero pose command, gripper command unchanged) while objects and, under drift, the camera keep moving, and then executes the chunk. These steps are not counted against the episode’s step limit. Old observations: the robot never waits; at each query the policy receives the third-person image, wrist image and joint state of 5 steps earlier, and HOMEWARD draws the robot layer for that moment, with no compensation. Holding changes HOMEWARD by −3.7-3.7 to +1.2+1.2 points on Goal and Long; the only significant change is π0.5\pi_{0.5} on Goal, and every cell stays significantly above the frozen policy. Under drift, where the camera moves on during the pause, it changes the per-step variant by −3.3-3.3, −1.3-1.3 and +0.3+0.3 points. Acting on old observations fails for every policy, with or without HOMEWARD: frozen policies lose 15 to 48 points and restored ones 38 to 70. A chunk planned from a state that the robot has already left repeats motion it has executed, so this regime needs latency compensation in the policy; HOMEWARD does not provide it.

OFTπ0.5\pi_{0.5}π0\pi_0drift (1,200)
GoalLongGoalLongGoalLongOFTπ0.5\pi_{0.5}π0\pi_0
frozen policy79.735.384.352.765.244.958.167.550.3
acting on 0.25 s-old observations31.9*10.7*55.1*34.6*50.2*13.4*–––
HOMEWARD91.472.394.494.093.478.372.582.564.0
robot holds 0.25 s per query92.473.590.7*92.192.977.369.2*81.264.3
acting on 0.25 s-old observations21.3*24.3*56.9*37.9*47.5*26.5*–––

Table 12. A 0.25 s delay per policy query. Success rate (%) on the LIBERO-Plus camera variants of Goal (408) and Long (419) and on the drift benchmark (per-step variant). – : not run. * McNemar p<0.05p < 0.05 against the same arm without delay.

Camera estimate and the Object suite

Table S13 compares the camera that VGGT estimates at the first query, from the reference frame and the first moved-camera frame, with the simulator’s camera on all 1,599 LIBERO-Plus camera variants. Positions are put in metres with the depth of the training view; HOMEWARD itself never needs this scale. The median error is about 1∘1^\circ in rotation and 5 cm in position in every suite. The exception is a group of 53 Object variants in which the camera is only pulled back, to 148–200% of its training distance: there the position error exceeds 20 cm. The 51 to 53 variants with the same pull-back in each other suite have a median error of 1.6 to 7.2 cm and none above 12.5 cm.

variantsrotation (deg)position (cm)focal (%)position >20 cm
Spatial3761.2 / 3.05.2 / 9.45.2 / 14.10
Object3960.9 / 1.75.8 / 64.43.5 / 7.953
Goal4081.2 / 2.35.9 / 8.86.6 / 12.80
Long4190.9 / 1.74.5 / 7.54.7 / 13.60
All1,5991.0 / 2.35.4 / 9.14.5 / 12.553

Table 13. First-query camera estimate against the simulator’s camera on the LIBERO-Plus camera variants: median / 90th percentile of the rotation error, the position error and the relative focal-length error, and the number of variants with a position error above 20 cm.

Table S14 splits the Object suite by this error. On the 53 pulled-back variants HOMEWARD does not help OpenVLA-OFT (5.7%), lowers π0.5\pi_{0.5} from 100.0% to 67.9%, and still lifts π0\pi_0 from 39.6% to 90.6%. On the other 343 variants the camera estimate is as good as elsewhere and π0.5\pi_{0.5} and π0\pi_0 reach 98.0% and 96.5%, yet OpenVLA-OFT gains only 5.2 points, to 57.1%, against 98.8% with the simulator’s depth and cameras under the same robot layer and the same reference fill. The gap of OpenVLA-OFT on Object is therefore not a matter of larger camera errors or of the reference frame: both are the same as in the suites, and for the policies, where restoration works. It is specific to this checkpoint, which is far more sensitive to the residual error of the estimated geometry than the two flow-matching policies. It appears at small shifts as well: at LIBERO-Plus levels 1 and 2 restoration lowers OpenVLA-OFT on Object from 88.9% and 84.5% to 66.7% and 77.5%.

position error ≤20\leq20 cm (343)position error > 20 cm (53)
frozenHOMEWARDoracle geometry + robot layer†frozenHOMEWARDoracle geometry + robot layer†
OpenVLA-OFT51.957.198.8*5.75.769.8*
π0.5\pi_{0.5}92.198.0*98.8*100.067.9*100.0
π0\pi_091.596.5*96.5*39.690.6*96.2

Table 14. Object suite by camera-estimate error. Success rate (%) on the LIBERO-Plus camera variants of Object. † simulator depth and cameras; not deployable. * McNemar p<0.05p < 0.05 against the frozen policy.

Failure cases

Figure S3 shows four LIBERO-Plus episodes in which OpenVLA-OFT fails with HOMEWARD and succeeds with the simulator’s depth and cameras under the same robot layer. In the first row the camera is pulled back to 182% of its training distance, one of the 53 Object variants of Sec. L: the estimate misses most of the pull-back (the camera position is off by 64 cm), so the objects stay too small and too high while the robot layer is drawn at its true scale. In the second row the camera estimate is accurate (0.4°, 1.7 cm), the restored view is as close to the training view as the oracle re-rendering (mean absolute error 7.0 against 7.4 grey levels) and the frozen policy succeeds on this variant, yet the policy fails with HOMEWARD; what remains is a warped copy of the real arm beside the drawn one and softened small objects. The third and fourth rows are rotations of 68° and 60° on Long: the moved camera sees the mugs from the other side, so their restored surfaces are torn and partly filled from the reference frame, and the warped real arm (Sec. C) covers the object in the gripper.

Level 1 and wider motion gates

The 6.1-point loss of OpenVLA-OFT at Level 1 in the main paper falls on the Object suite. On its 63 Level-1 variants restoration lowers success from 88.9% to 66.7% (McNemar p=0.007p = 0.007); on the 136 Level-1 variants of the other three suites success goes from 95.6% to 97.1%. Pooled over the 199 variants the change is not significant (p=0.058p = 0.058). At the same level the other policies change by +0.5 (π0.5\pi_{0.5}), +6.5 (π0\pi_0, p=0.007p = 0.007) and +36.6 points (OpenVLA).

Level 1 is the mildest level of the benchmark, not an almost unmoved camera: by the first-query estimate its variants have a median rotation of 13.8° and a median translation of 28% of the scene depth (10th to 90th percentile: 0.9–41.0° and 1–56%). The motion gate tests for an unmoved camera and passes none of the 1,599 variants. tbl:S15 emulates wider gates from the logs: an episode whose first-query estimate is below both the rotation and the translation threshold takes the outcome of the frozen run, every other episode the outcome of HOMEWARD. Up to four times the default thresholds no episode passes. At ten times (5°, 10%) 35 episodes pass. On them restoration raises success from 42.9% to 51.4% for OpenVLA-OFT, from 60.0% to 91.4% for π0\pi_0 (p=0.001p = 0.001) and from 20.0% to 25.7% for OpenVLA, and lowers it from 94.3% to 80.0% for π0.5\pi_{0.5} (2 episodes gained, 7 lost, p=0.18p = 0.18). Wider gates pass a third to a half of the Level-1 variants. They leave OpenVLA-OFT at 85.9% and 86.4% there, and they lower pooled success significantly for every policy but π0.5\pi_{0.5}, which stays within 0.3 points. The observed Level-1 decline is concentrated on Object for this checkpoint (Sec. L) and is not eliminated by the tested camera-motion gates.

Oracle re-rendering and the training camera

Oracle re-rendering is not the training camera’s view. It reprojects the moved camera’s frame with the simulator’s depth and cameras and fills what that camera does not see from the reference frame, as HOMEWARD does; only the geometry is privileged. What the training camera itself renders of the same scene is the clean-camera condition, and Tab. S16 sets the two side by side on Goal and Long. With OpenVLA-OFT, π0.5\pi_{0.5} and π0\pi_0, the frozen policy at the training camera scores 5.3 to 14.9 points above oracle re-rendering (41.0 and 46.5 points with OpenVLA), so exact geometry alone does not return the training view. The robot layer over the same oracle geometry comes within 7.7 points of the training camera for these three policies. HOMEWARD, with estimated geometry, is within 6.6 points in five of the six cells and 18.7 points below the training camera for OpenVLA-OFT on Long; OpenVLA stays 34.8 and 44.8 points below. The clean-camera episodes use ten initial states per task and the camera variants one, so the first row is not paired with the others.

gate thresholdspassed (%) L1passed (%) allOFT L1OFT allπ0.5\pi_{0.5} L1π0.5\pi_{0.5} allπ0\pi_0 L1π0\pi_0 allOpenVLA L1OpenVLA all
no gate (HOMEWARD as reported)0087.477.698.095.197.091.247.226.4
0.5°, 1% (default)0087.477.698.095.197.091.247.226.4
2°, 4% (4×)0087.477.698.095.197.091.247.226.4
5°, 10° (10×)3286.977.498.095.496.590.646.726.3
10°, 20% (20×)311485.976.0*98.095.496.088.6*27.6*19.8*
15°, 30% (30×)452486.474.2*98.095.093.586.0*22.1*14.8*
frozen policy (all passed)10010093.562.597.576.590.568.010.63.6

Table 15. Wider motion gates, emulated on the LIBERO-Plus camera variants. Success rate (%) at Level 1 (L1, 199 variants) and over all 1,599 variants when episodes whose first-query estimate is below both thresholds (rotation; translation relative to the median scene depth) are passed through unrestored. Computed from the frozen and HOMEWARD runs of the same episodes; no threshold was tuned. Passed: share of episodes below the thresholds in the OpenVLA-OFT runs (the other runs differ by at most 0.5 points). * McNemar p<0.05p < 0.05 against no gate.

OFT GoalOFT Longπ0.5\pi_{0.5} Goalπ0.5\pi_{0.5} Longπ0\pi_0 Goalπ0\pi_0 LongOpenVLA GoalOpenVLA Long
training camera, frozen policy†^\dagger98.091.096.091.095.084.074.056.0
moved camera, frozen policy79.735.384.352.765.244.98.11.2
oracle re-rendering‡^\ddagger90.476.188.777.889.774.727.515.0
oracle re-rendering + robot layer‡^\ddagger91.483.397.191.995.880.252.725.1
HOMEWARD91.472.394.494.093.478.339.211.2

Table 16. Training camera and oracle re-rendering. Success rate (%). †^\dagger Clean-camera episodes: the unperturbed scene seen by the training camera, 100 per suite (10 tasks × 10 initial states). Other rows: the LIBERO-Plus camera variants of Goal (408) and Long (419), as in the ablation table of the main paper. ‡^\ddagger Privileged: simulator depth and cameras.

Inference cost

Table S17 lists what the two view-restoration methods add to a policy query on one A100. Times are medians over the 1,599 π0.5\pi_{0.5} episodes of the LIBERO-Plus camera variants (827 for the per-step variant); memory is the peak that the restoration models allocate in the evaluator process, measured in separate three-episode runs with the policy served from another process. In the default variant the first query of an episode runs the two-frame estimate, the computation of a per-step query; later queries run VGGT on the current frame alone. At inference AnyCamVLA is the cheaper method: its view synthesis is about five times faster than HOMEWARD’s scene layer and, with the true cameras, takes a sixth of the memory. What it needs instead is a synthesis model fine-tuned on renders of the benchmark scenes and the parameters of the moved camera; when it estimates that camera with VGGT, as HOMEWARD does, its memory is the larger of the two.

modelstime per query (ms)peak GPU memory (GiB)
HOMEWARDVGGT-1B164 + 106.6
HOMEWARD, per-stepVGGT-1B191 + 106.6
AnyCamVLA, true cameras{sup}‡`LVSM321.1
AnyCamVLA, estimated cameras{sup}§`LVSM, VGGT-1B317.2

Table S17. Inference cost of view restoration, excluding the policy. HOMEWARD: scene layer + robot layer. {sup}§` VGGT runs once per episode, on the first frame; its time is not included.

Failure cases of HOMEWARD with OpenVLA-OFT

Figure S3. Failure cases of HOMEWARD with OpenVLA-OFT. Columns: the moved camera’s frame, HOMEWARD’s output, the oracle re-rendering from the simulator’s depth and cameras (no robot layer), and the training camera’s view of the same simulator state. Rows 1 and 3 show the first policy query, rows 2 and 4 the fifth and ninth. Rotations are those estimated at the first query.

Prompt convention

The LIBERO-Plus camera variants, CamRand and clean-camera episodes use base-task instructions; the other camera protocols use paraphrased instructions, as detailed in Sec. V. The client released with LIBERO-Plus builds the prompt from the variant’s file name, which appends text such as “view 0 0 100 2 352 initstate 0”; with that prompt our frozen OpenVLA-OFT, π0.5\pi_{0.5} and π0\pi_0 score 58.2%, 69.4% and 14.8% on the camera variants instead of 62.5%, 76.5% and 68.0%, reproducing the 56.4% and 13.8% reported for OpenVLA-OFT and π0\pi_0 [8].

Renderer

Software rendering

Simulated images depend slightly on the rasterizer. Rendering the same π0\pi_0 episodes in software instead of on the GPU changes success by up to 4.5 points on Long (Table S18), but the gain of HOMEWARD is the same under both renderers: 27.7 and 28.4 points on Goal, 28.6 and 30.0 on Long. Over all 37 π0\pi_0 arms we ran under both renderers, one differs at p<0.05p < 0.05, about what chance predicts. We nevertheless compare arms only within one renderer and report GPU rendering throughout.

frozen GPUfrozen softwareHOMEWARD GPUHOMEWARD softwareper-step GPUper-step software
π0\pi_0
Spatial79.579.597.697.397.997.1
Object84.685.196.096.296.796.2
Goal65.264.792.993.190.791.9
Long44.948.073.578.0*74.076.1

Table S18. Software rendering. Success rate (%) of π0\pi_0 on the LIBERO-Plus camera variants with MuJoCo rendered on the GPU (NVIDIA EGL, used for every other result) and in software (Mesa llvmpipe), on the same episodes. * the two renderers differ, McNemar p<0.05p < 0.05.

CamRand by radius

On LIBERO-Long, HOMEWARD raises OpenVLA-OFT from 87% to 92%, from 72% to 91% and from 43% to 82% at the small, medium and large CamRand radius (100 episodes each).

Seeds

On all 1,599 LIBERO-Plus camera variants, π0.5\pi_{0.5} rises from 76.2±0.5%76.2 \pm0.5\% to 94.6±0.5%94.6 \pm0.5\% and π0\pi_0 from 67.7±1.6%67.7 \pm1.6\% to 90.5±0.9%90.5 \pm0.9\% over three seeds (mean ±\pm s.d.; p<10−50p < 10^{-50} for each seed). Over three seeds on AnyCamVLA’s orbits of all four suites, π0.5\pi_{0.5} rises from 57.6±0.1%57.6\pm0.1\% to 78.8±0.3%78.8\pm0.3\% and π0\pi_0 from 39.6±0.2%39.6\pm0.2\% to 61.2±0.2%61.2\pm0.2\% with HOMEWARD (mean ±\pm s.d.; p<10−50p < 10^{-50} for each seed), compared with 80.8±0.5%80.8 \pm0.5\% and 60.0±0.5%60.0 \pm0.5\% for AnyCamVLA with estimated cameras.

Task-level resampling

The camera variants of one LIBERO task share its scene and instruction, so the episodes of a task are not independent. Table S19 therefore resamples tasks instead of episodes on the main result: each bootstrap draw samples the 40 tasks (10 per suite) with replacement and keeps every variant of a sampled task (6 to 50 per task); intervals are the 2.5th and 97.5th percentiles of 20,000 draws. Drawing the tasks within each suite gives slightly narrower intervals. The sign test compares, over tasks, how many gain and how many lose episodes (ties dropped, two-sided); the Wilcoxon signed-rank test is taken over the 40 per-task differences in success rate (two-sided).

Policygain (points)95% CI, tasks95% CI, tasks within suitetasks up/down/tiedsign test ppWilcoxon pp
OpenVLA-OFT+15.1[7.6, 22.9][9.0, 21.7]26 / 5 / 91.9×10−41.9 \times10^{-4}4.5×10−44.5 \times10^{-4}
π0.5\pi_{0.5}+18.6[10.3, 27.1][11.7, 25.6]23 / 9 / 80.00530.0053
π0\pi_0+23.2[17.1, 29.7][17.5, 28.9]35 / 1 / 41.1×10−91.1 \times10^{-9}3.2×10−73.2 \times10^{-7}
OpenVLA+22.8[17.7, 28.4][18.4, 27.7]38 / 1 / 11.5×10−101.5 \times10^{-10}8.4×10−88.4 \times10^{-8}

Table S19. Task-level resampling of the main result. Gain of HOMEWARD over the frozen policy on all 1,599 LIBERO-Plus camera variants, with bootstrap intervals that resample the 40 tasks, and the number of tasks whose success count rises, falls or is unchanged.

The same resampling applies to the other comparisons of the main paper (Table S20); for AnyCamVLA’s orbit protocol on LIBERO-Long there are 10 tasks.

Comparison (difference in points)diff.95% CI, tasks
HOMEWARD minus AnyCamVLA, LIBERO-Plus camera variants
OpenVLA-OFT−3.1/−1.9-3.1 / -1.9[−11.3,4.8]/[−9.2,5.3][-11.3, 4.8] / [-9.2, 5.3]
π0.5\pi_{0.5}+9.9/+12.9+9.9 / +12.9[3.5,16.8]/[6.4,19.7][3.5, 16.8] / [6.4, 19.7]
π0\pi_0+3.8/+8.1+3.8 / +8.1[0.3,7.5]/[4.7,12.5][0.3, 7.5] / [4.7, 12.5]
OpenVLA+0.4/+12.3+0.4 / +12.3[−3.2,4.1]/[8.1,16.7][-3.2, 4.1] / [8.1, 16.7]
HOMEWARD minus AnyCamVLA (true cameras), orbit protocol on Long
OpenVLA-OFT−15.2-15.2[−22.7,−7.6][-22.7, -7.6]
π0.5\pi_{0.5}−4.8-4.8[−10.9,1.9][-10.9, 1.9]
π0\pi_0−5.8-5.8[−10.3,−1.1][-10.3, -1.1]
OpenVLA−5.3-5.3[−8.3,−2.4][-8.3, -2.4]
HOMEWARD minus frozen policy, orbit protocol on Long
OpenVLA-OFT / π0.5\pi_{0.5}+27.9/+29.7+27.9 / +29.7[18.4,37.0]/[19.8,39.0][18.4, 37.0] / [19.8, 39.0]
π0\pi_0 / OpenVLA+32.7/+41.0+32.7 / +41.0[23.5,42.2]/[−2.4,54.4][23.5, 42.2] / [-2.4, 54.4]
OpenVLA-OFT+ with HOMEWARD minus OpenVLA-OFT+
LIBERO-Plus camera−4.7-4.7[−10.1,−0.1][-10.1, -0.1]
AnyCamVLA, Long+15.1+15.1[8.1,23.7][8.1, 23.7]
AnyCamVLA-style, other suites+3.2+3.2[−1.3,8.3][-1.3, 8.3]
CamRand+4.2+4.2[1.2,7.0][1.2, 7.0]
Drift (per-step pose)+10.2+10.2[2.7,19.2][2.7, 19.2]
Lens change+3.0+3.0[−0.8,7.0][-0.8, 7.0]
Clean camera−0.2-0.2[−1.2,0.5][-1.2, 0.5]
Clean camera, with the motion gate / gate disabled
OpenVLA-OFT0.0/+1.00.0 / +1.0[−0.8,0.8]/[−1.2,3.8][-0.8, 0.8] / [-1.2, 3.8]
π0.5\pi_{0.5}−0.5/+0.2-0.5 / +0.2[−1.5,0.5]/[−1.8,2.5][-1.5, 0.5] / [-1.8, 2.5]
π0\pi_0+0.2/+0.2+0.2 / +0.2[−1.2,1.8]/[−2.5,3.2][-1.2, 1.8] / [-2.5, 3.2]
OpenVLA−0.5/−1.8-0.5 / -1.8[−2.2,1.0]/[−6.8,3.0][-2.2, 1.0] / [-6.8, 3.0]

Table S20. Task-level intervals for the other comparisons. Differences in success rate (points, from unrounded rates) with 95% bootstrap intervals that resample tasks (20,000 draws). Paired entries list true / estimated camera parameters.

Protocol notes

The main evaluations use NVIDIA EGL; the software-rendering control in Sec. R.1 additionally uses Mesa llvmpipe. Reported results combine automatically regenerated tables and separately maintained controls. Methods were run on the same machine where possible, but 30 evaluation groups include runs from multiple machines. Task resampling does not account for systematic differences between machines. The camera-change suites of OpenVLA exist as two complete runs on two machines, which agree on 85–100% of the episodes and differ by at most 0.5 points in pooled success; we report the run of the machine that holds every arm of the camera suites. The LIBERO-Plus camera variants, CamRand and the clean-camera episodes are prompted with the instruction of the base task. The AnyCamVLA-protocol, drift and lens-change episodes are built on LIBERO-Plus files with paraphrased instructions and use those instructions; within each benchmark, all compared methods receive identical prompts. Reference frames are rendered once per task from initial state 1, after 10 settling steps (from the next stored state when the episode itself starts from state 1). The LIBERO-Plus camera variants start from state 0 and the other benchmarks from states 0, 5, . . . , 45 or from all 50, so the objects and the arm in the reference frame are placed differently from the test episode. The simulation is synchronous, so the compute time below delays no observation; Sec. K measures a delay of this size. Scale alignment in the cached variant: the static pixels are found at VGGT’s input resolution against the first moved-camera frame, and the scale stays at 1 when no more than 100 static pixels remain. Timing (A100, VGGT-1B at 518 px, bfloat16, medians over the LIBERO-Plus camera episodes): the scene layer alone, that is VGGT inference, depth-edge filtering, reprojection and reference fill, takes 0.15 s per query with cached camera geometry and 0.19 s when the camera is re-estimated at every query; the robot layer takes 0.01 s. The 0.21 s in the main paper is an indirect estimate from the restored-minus-frozen wall-clock time per policy query under matched episode conditions (0.20 to 0.22 s for the three policies that act in chunks). It also includes evaluation-harness overhead that is not timed separately; different rollout lengths introduce noise, so it is not a direct measurement of restoration latency. The 27 ms amortization applies only to OpenVLA-OFT, π0.5\pi_{0.5} and π0\pi_0, which query once every eight steps.

References

  1. [1]Ali Abouzeid, Malak Mansour, Zezhou Sun, and Dezhen Song. GeoAware-VLA: Implicit geometry aware vision-language-action model. arXiv preprint arXiv:2509.14117, 2025.arxiv.org/abs/2509.14117
  2. [2]Kevin Black, Noah Brown, Danny Driess, et al. π₀: A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024.arxiv.org/abs/2410.24164
  3. [3]Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT-1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), 2023.arxiv.org/abs/2212.06817
  4. [4]Lawrence Yunliang Chen, Kush Hari, Karthik Dharmarajan, Chenfeng Xu, Quan Vuong, and Ken Goldberg. Mi-rage: Cross-embodiment zero-shot policy transfer with cross-painting. In Robotics: Science and Systems (RSS), 2024.
  5. [5]Lawrence Yunliang Chen, Chenfeng Xu, Karthik Dharmarajan, Muhammad Zubair Irshad, Richard Cheng, Kurt Keutzer, Masayoshi Tomizuka, Quan Vuong, and Ken Goldberg. RoVi-Aug: Robot and viewpoint augmentation for cross-embodiment robot learning. In Conference on Robot Learning (CoRL), 2024.arxiv.org/abs/2409.03403
  6. [6]Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion Policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems (RSS), 2023.
  7. [7]Bradley Efron. Bootstrap methods: Another look at the jack-knife. The Annals of Statistics, 7(1):1–26, 1979.
  8. [8]Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoyue Fei, Jinlan Fu, Jingjing Gong, and Xipeng Qiu. LIBERO-Plus: In-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626, 2025.arxiv.org/abs/2510.13626
  9. [9]Martin A. Fischler and Robert C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
  10. [10]Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox. RVT: Robotic view transformer for 3D object manipulation. In Conference on Robot Learning (CoRL), 2023.arxiv.org/abs/2306.14896
  11. [11]Ankit Goyal, Valts Blukis, Jie Xu, Yijie Guo, Yu-Wei Chao, and Dieter Fox. RVT-2: Learning precise manipulation from few demonstrations. In Robotics: Science and Systems (RSS), 2024.arxiv.org/abs/2406.08545
  12. [12]Songen Gu, Yuhang Zheng, Weize Li, Yupeng Zheng, Yating Feng, Xiang Li, Yilun Chen, Pengfei Li, et al. VistaBot: View-robust robot manipulation via spatiotemporal-aware view synthesis. arXiv preprint arXiv:2604.21914, 2026.arxiv.org/abs/2604.21914
  13. [13]Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2nd edition, 2004.DOI
  14. [14]Hyeongjun Heo, Seungyeon Woo, Sang Min Kim, Junho Kim, Junho Lee, Yonghyeon Lee, and Young Min Kim. Any-CamVLA: Zero-shot camera adaptation for viewpoint robust vision-language-action models. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2026.DOI
  15. [15]Mu Hu, Wei Yin, Chi Zhang, Zhipeng Cai, Xiaoxiao Long, et al. Metric3D v2: A versatile monocular geometric foundation model for zero-shot metric depth and surface normal estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10579–10596, 2024.DOI
  16. [16]Bingqi Huang, Bingchuan Wei, Xuan Wang, Yingkai Cai, and Zhaokui Wang. Cross-view action consistency for camera-robust vision-language-action policies. arXiv preprint arXiv:2608.06965, 2026.arxiv.org/abs/2608.06965
  17. [17]Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, and Zexiang Xu. LVSM: A large view synthesis model with minimal 3D inductive bias. arXiv preprint arXiv:2410.17242, 2024.arxiv.org/abs/2410.17242
  18. [18]Tsung-Wei Ke, Nikolaos Gkanatsios, and Katerina Fragkiadaki. 3D Diffuser Actor: Policy diffusion with 3D scene representations. In Conference on Robot Learning (CoRL), 2024.arxiv.org/abs/2402.10885
  19. [19]Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), 2023.arxiv.org/abs/2308.04079
  20. [20]Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024.arxiv.org/abs/2406.09246
  21. [21]Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success. In Robotics: Science and Systems (RSS), 2025.
  22. [22]Marion Lepert, Ria Doshi, and Jeannette Bohg. SHADOW: Leveraging segmentation masks for cross-embodiment policy transfer. In Conference on Robot Learning (CoRL), 2024.arxiv.org/abs/2503.00774
  23. [23]Vincent Leroy, Yohann Cabon, and Jérôme Revaud. Grounding image matching in 3D with MAS3R. In European Conference on Computer Vision (ECCV), 2024.arxiv.org/abs/2406.09756
  24. [24]Weiqi Li, Quande Zhang, Ruifeing Zhai, Lialang Lin, and Guangrun Wang. VLA models are more generalizable than you think: Revisiting physical and spatial modeling. arXiv preprint arXiv:2512.02902, 2025.arxiv.org/abs/2512.02902
  25. [25]Wenhao Li, Xueying Jiang, Qunhao Qian, Deli Zhao, Shijian Lu, Gongjie Zhang, and Ran Xu. From fixed to free cameras: Calibration-free view-robust vision-language-action model. arXiv preprint arXiv:2607.05396, 2026.arxiv.org/abs/2607.05396
  26. [26]Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2306.03310
  27. [27]David G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110, 2004.DOI
  28. [28]Quinn McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, 1947.pubmed.ncbi.nlm.nih.gov/20254758
  29. [29]Octo Model Team, Dibya Ghosh, Homer Walke, et al. Octo: An open-source generalist robot policy. In Robotics: Science and Systems (RSS), 2024.arxiv.org/abs/2405.12213
  30. [30]Open X-Embodiment Collaboration. Open X-Embodiment: Robotic learning datasets and RT-X models. In IEEE International Conference on Robotics and Automation (ICRA), 2024.
  31. [31]Tianhang Pan, Xuanhao Wang, Yiwen Pang, Bo Zhou, Jun Yang, Min-Ling Zhang, and Shimin Di. CoRe-VLA: Preserving cross-view coordination in VLAs under camera shifts. arXiv preprint arXiv:2609.37150, 2026.arxiv.org/abs/2609.37150
  32. [32]Yechan Park and HyunJin Kim. GS-VLA: Plug-and-play viewpoint canonicalization for frozen VLA policies via Gaussian splatting. arXiv preprint arXiv:2608.19066, 2026.arxiv.org/abs/2608.19066
  33. [33]Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, et al. π₀.₅: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025.arxiv.org/abs/2504.16054
  34. [34]Wilbert Pumacay, Ishika Singh, Jiafei Duan, Ranjay Krishna, Jesse Thomason, and Dieter Fox. THE COLOSSEUM: A benchmark for evaluating generalization for robotic manipulation. In Robotics: Science and Systems (RSS), 2024.arxiv.org/abs/2402.08191
  35. [35]Rishik Sathua, Haonan Chen, and Katherine Driggs-Campbell. ARGUS: Aligning robot scene geometry under shifting views with large 3D vision models. arXiv preprint arXiv:2608.05579, 2026.arxiv.org/abs/2608.05579
  36. [36]Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-Actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning (CoRL), 2022.arxiv.org/abs/2209.05451
  37. [37]Stephen Tian, Blake Wulfe, Kyle Sargent, Katherine Liu, Sergey Zakharov, Vitor Guizilini, and Jiajun Wu. View-invariant policy learning via zero-shot novel view synthesis. In Conference on Robot Learning (CoRL), 2024.arxiv.org/abs/2409.03685
  38. [38]Emanuel Todorov, Tom Erez, and Yuval Tassa. MuJoCo: A physics engine for model-based control. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2012.DOI
  39. [39]Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. VGGT: Visual geometry grounded transformer. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.arxiv.org/abs/2503.11651
  40. [40]Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu Deng, Xin Tong, and Jiaolong Yang. MoGe: Unlocking accurate monocular geometry estimation for open-domain images with optimal training supervision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.arxiv.org/abs/2410.19115
  41. [41]Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. DUSt3R: Geometric 3D vision made easy. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.arxiv.org/abs/2312.14132
  42. [42]Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π³: Permutation-equivariant visual geometry learning. In International Conference on Learning Representations (ICLR), 2026.
  43. [43]Albert Wilcox, Mohamed Ghanem, Masoud Moghani, Pierre Barroso, Benjamin Joffe, and Animesh Garg. Adapt3R: Adaptive 3D scene representation for domain transfer in imitation learning. arXiv preprint arXiv:2503.04877, 2025.arxiv.org/abs/2503.04877
  44. [44]Annie Xie, Lisa Lee, Ted Xiao, and Chelsea Finn. Decomposing the generalization gap in imitation learning for visual robotic manipulation. arXiv preprint arXiv:2307.03659, 2023.arxiv.org/abs/2307.03659
  45. [45]Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Anything V2. In Advances in Neural Information Processing Systems (NeurIPS), 2024.arxiv.org/abs/2406.09414
  46. [46]Yanjie Ze, Gu Zhang, Kangning Zhang, Chenyuan Hu, Muhan Wang, and Huazhe Xu. 3D Diffusion Policy: Generalizable visuomotor policy learning via simple 3D representations. In Robotics: Science and Systems (RSS), 2024.arxiv.org/abs/2403.03954
  47. [47]Junyi Zhang, Charles Herrmann, Junhwa Hur, Varun Jampani, Trevor Darrell, Forrester Cole, Deqing Sun, and Ming-Hsuan Yang. MonST3R: A simple approach for estimating geometry in the presence of motion. In International Conference on Learning Representations (ICLR), 2025.
  48. [48]Tony Z. Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware. In Robotics: Science and Systems (RSS), 2023.arxiv.org/abs/2304.13705
  49. [49]Brianna Zitkovich, Tianhe Yu, Sichun Xu, et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Proceedings of the 7th Conference on Robot Learning (CoRL), 2023.arxiv.org/abs/2307.15818

Paper details

vlnc:W_3yh2azqs
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Claude Code, Codex

View statement
Looking for related papers…
Contents