WorksComputer Science
AuroraWorld: Structured 4D World Modeling for Exo-to-Ego Video Generation in Robot Manipulation
Abstract
Egocentric observations provide a robot-centric view of manipulation processes, capturing contact interactions and object-state changes that are important for visuomotor policy learning but difficult to observe from external cameras. However, collecting such data at scale remains expensive due to additional hardware requirements and platform-specific deployment costs. Synthesizing egocentric observations from large-scale exocentric manipulation videos provides a scalable alternative. However, existing methods struggle to synthesize physically consistent and policy-relevant egocentric videos under substantial viewpoint transformations, severe occlusions, complex object interactions, and inherently sparse observations from available exocentric viewpoints. Code will be made publicly available at https://github.com/xueha0/AuroraWorld
Introduction
Embodied intelligence seeks to develop intelligent agents that can perceive, reason about, and interact with the physical world, representing an important direction toward more capable artificial intelligence systems [50, 22]. A fundamental challenge in embodied intelligence is learning generalizable robot policies that can transform sensory observations into reliable actions under diverse and complex physical interactions. Recent advances in large-scale robot learning, particularly vision-language-action models [25, 24, 51, 54] (Zitkovich et al. 2023), have demonstrated the potential of data-driven approaches for acquiring increasingly generalizable manipulation skills.
However, scaling embodied policy learning remains limited by the availability of diverse and high-quality robot interaction data, which is costly and time-consuming to collect in real-world environments. A critical aspect of this data bottleneck is the limited availability of egocentric observations in robot manipulation, such as those captured by wrist-mounted cameras. These observations provide a robot-centric view of manipulation and reveal fine-grained cues, including gripper-object contact and object states, that are valuable for precise control [12, 24]. However, egocentric videos remain less consistently available across robot platforms and tasks, as they require additional sensing setups and dedicated data collection efforts [38, 31]. This limited availability of robot-centric observations can hinder the learning of robust and generalizable embodied policies.
To alleviate this data bottleneck, researchers have explored synthesizing egocentric videos from exocentric manipulation ones. First, some works perform frame-based view synthesis to infer egocentric observations from exocentric views [36, 45, 29, 44, 42, 37] (Hu et al. 2025a). However, these approaches mainly synthesize individual observations, without capturing temporal evolution and robot-object interaction dynamics. Consequently, they are insufficient for generating temporally coherent manipulation videos involving evolving object states and interactions. Second, recent advances in video generation models enable the synthesis of temporally coherent videos from exocentric observations [31, 34, 23] (Xu et al. 2025). Nevertheless, these approaches primarily focus on visual appearance generation, while lacking explicit modeling of the underlying embodied world state required for physically grounded manipulation. Recently, emerging robotic world models [38] aim to model latent embodied states and the underlying dynamics of physical environments for predicting robot-centric observations. However, most video generation approaches and world models often require additional observational constraints, such as an initial egocentric frame or rich exocentric observations, and do not explicitly model the underlying embodied dynamics [34, 31, 38] (Xu et al. 2025), to resolve viewpoint ambiguity. However, under sparse exocentric observations, they struggle with large viewpoint changes, severe occlusions, and complex robot-object interactions, making it difficult to generate physically plausible egocentric observations.
To address this challenge, we formulate exo-to-ego synthesis as structured inference of the hidden 4D embodied world representation from sparse exocentric observations. To this end, we propose AuroraWorld, a 4D world model that infers and models the evolving embodied environment through complementary spatial, temporal, and motion components (Figure 1). Specifically, AuroraWorld captures spatial structures through 3D geometric scene representations, temporal evolution through motion-stabilized keyframes, and interaction dynamics through motion trajectories. These components characterize different aspects of the underlying embodied world and serve as structured world priors for egocentric video generation. By conditioning generation on the inferred 4D representation rather than appearance-level correlations alone, AuroraWorld enables physically consistent and temporally coherent egocentric video synthesis. We evaluate AuroraWorld on visual generation and downstream embodied policy learning tasks in simulated and real robotic environments, demonstrating improved generation quality and policy performance under sparse exocentric observations.

Figure 1. AuroraWorld proposes to models the underlying spatial, temporal and motion dynamics, enabling physically plausible exo-to-ego video generation.
Related Work
Cross-view Generation. Cross-view generation aims to synthesize observations from unseen viewpoints given available visual inputs. For static scenarios, recent approaches model scene structures from visual inputs to enable consistent generation from unseen viewpoints, including 3D reconstruction [45, 29, 44, 32, 18, 42], and diffusion-based synthesis [37]. Extending to dynamic scenarios, recent exo-to-ego approaches [31, 34, 23, 41, 35, 33] (Xu et al. 2025) leverage conditional video generation models with multi-view, pose-aware, and other forms of conditioning. However, these approaches are primarily developed for general visual scenarios, and perceptual realism is the dominant objective. In contrast, robotic manipulation requires physically grounded egocentric videos that preserve fine-grained geometric relationships, contact interactions, and object-state transitions essential for downstream policy learning. Qian et al. (Qian et al. [38] represent an initial effort toward egocentric video synthesis by exploring geometric information. However, their method lacks explicit modeling of key manipulation states or motion dynamics, limiting its applicability under sparse exocentric observations.
Video Generation and Embodied World Models
Recent advances in large-scale video generation have achieved significant progress in synthesizing temporally coherent and realistic videos. Latent diffusion models and Transformer-based architectures have become widely adopted paradigms [5, 43, 27, 14, 47] (Li et al. 2026). To improve controllability, conditional video generation methods incorporate auxiliary visual signals, such as reference images, depth, camera poses, and geometric structures, to enhance consistency across generated sequences [28, 3, 6, 7] (Chen et al. 2026; Hu et al. 2026). Building upon the advances, video-generation-based embodied world models are proposed to learn latent environment dynamics and predict future states conditioned on actions [52, 10] (Hu et al. 2025b; Jiang et al. 2025; Ye et al. 2026). While conventional video generation models primarily optimize visual plausibility and temporal consistency, embodied world models seek to capture latent physical states and action-conditioned dynamics. Recent studies explore strategies, including geometric modeling [52, 10], memory mechanisms (Guo et al. 2025), and action conditioning [55] to improve physical plausibility, consistency, and long-horizon prediction. Embodied world models have been explored as simulators for generating interactive training environments [1] (Guo et al. 2025), data engines for scalable synthetic video generation [20] (Ye et al. 2026), and planners for decision making [1] (Bi et al. 2026).
Embodied Policy Learning and Vision-Language-Action Models
Embodied policy learning aims to learn generalizable policies that enable robots to perform diverse tasks from sensor observations [9, 26, 24, 49]. Vision-language-action models have emerged as a promising paradigm by jointly modeling visual observations, language instructions, and robot actions, enabling scalable learning of manipulation policies from heterogeneous datasets [25, 24, 49, 51, 54] (Zitkovich et al. 2023). However, the performance and generalization of these models remain strongly dependent on the availability and diversity of robot interaction data, particularly egocentric observations that capture contact-rich manipulation processes. Collecting such videos at scale remains challenging, motivating approaches that can synthesize policy-relevant egocentric videos from more readily available exocentric observations.
Method
Overview
AuroraWorld aims to infer a structured 4D embodied world representation from sparse exocentric observations for egocentric video generation. Specifically, given exocentric videos and camera configurations , AuroraWorld constructs a 4D world representation:
where denotes the 4D representation construction process, and consists of key temporal information (), spatial structures (), and motion dynamics (), respectively.
The inferred 4D representation, together with exocentric observations, guides a latent diffusion model for egocentric video generation:
where denotes the conditional video generation distribution parameterized by .
Structured 4D Embodied World Representation
Motion Trajectory Inference. Robot-object interactions exhibit continuous temporal evolution that characterizes the underlying motion state of embodied environments. In many practical settings, exocentric videos lack recorded ground-truth end-effector trajectories, motivating inference from visual observations. Accordingly, AuroraWorld infers end-effector trajectories from external-view observations, establishing motion anchors for guiding egocentric video generation. Specifically, we adapt an inverse dynamics model (IDM) [20] for the exocentric observation setting to infer the 8-D end-effector trajectory:
where denotes the adapted inverse dynamics model parameterized by . Each trajectory state is represented as:
where denotes the position, denotes the unit quaternion, and denotes the gripper state. The inferred end-effector trajectory serves as a motion anchor that captures the temporal evolution of the embodied manipulation process. It also guides temporal anchor construction and provides motion guidance for constructing the action- and view-aware 3D noise prior.
Motion-Stabilized Temporal Anchor Construction. Temporal anchors capture critical manipulation states while preserving the embodied environment’s evolution during egocentric video generation. Rather than densely synthesizing all intermediate viewpoints, introducing redundant constraints and potentially accumulating temporal inconsistencies over long horizons, AuroraWorld selects sparse, informative temporal anchors based on motion transitions and post-transition stabilization. Specifically, the temporal anchor score considers two complementary cues: (1) view transition, measuring whether a candidate frame captures a meaningful change in the end-effector configuration, and (2) motion stabilization, ensuring the selected frame represents a settled manipulation state rather than an intermediate motion phase.
Specifically, we define a transition score and a stabilization score , which are combined to obtain the temporal anchor score:
where denotes robust per-clip normalization to .
The transition score measures the accumulated end-effector pose change over a preceding temporal interval , indicating whether the candidate frame represents a new and informative viewpoint:
where denotes Euclidean distance, denotes geodesic rotation distance, and balances translation and rotation changes.
The stabilization score evaluates whether the end-effector has reached a stable configuration after the transition. To measure it, we first compute the one-step translation and rotation changes:
To reduce frame-level motion fluctuations, we apply temporal smoothing to the translation and rotation changes, obtaining the smoothed changes and . They are then aggregated to obtain the instantaneous motion signal:
The stabilization score is then computed over a short future temporal window :
where higher values indicate that the end-effector has reached a locally stable configuration after the transition.
Based on the temporal anchor scores , we select top- interior frames and always include the first and last frames, yielding an anchor index set of frames. These locations identify critical states of the evolving embodied environment for constructing the structured 4D embodied representation. Since exocentric observations lack the egocentric visual information required for embodied observation generation, AuroraWorld adapts LagerNVS [42] to transform the corresponding exocentric observations into egocentric keyframes. For each selected timestep , a lightweight state-conditioned camera head first predicts the target ego-view camera pose from the inferred end-effector state. The adapted LagerNVS then synthesizes the corresponding egocentric keyframe conditioned on the exocentric observation, its camera configuration, and the predicted target-view pose:
Figure 2. An overview of AuroraWorld. The proposed framework consists of two stages. In the first stage, AuroraWorld infers a structured 4D world representation from sparse exocentric observations. Specifically, it identifies informative temporal keyframes, captures spatial structures, and models motion dynamics from exocentric views. The selected keyframes are then transferred to the egocentric viewpoint and, together with spatial and motion representations, serve as structural priors for video generation. In the second stage, these priors condition the diffusion model to synthesize coherent and physically consistent egocentric videos.
Here, denotes the state-conditioned camera head, and denotes the adapted LagerNVS model. The synthesized keyframes constitute .
Geometry-Aware Spatial Anchor Construction
Large exo-to-ego viewpoint transformations introduce substantial spatial ambiguity, making geometry-consistent scene understanding essential for generating physically plausible egocentric videos. To construct the spatial component of the structured 4D representation, AuroraWorld extracts latent geometry-aware scene representations from sparse exocentric observations. Specifically, we leverage LagerNVS [42] equipped with camera tokens to obtain latent 3D scene tokens from sparse exocentric views. These latent representations encode viewpoint-consistent scene structures and preserve geometric relationships among objects and the environment without requiring explicit 3D reconstruction. The resulting 3D tokens constitute the spatial component , enabling synthesized egocentric videos to maintain consistent scene structures under large viewpoint changes.
Structured-Prior Guided Egocentric Video Generation
Given the inferred structured 4D embodied representation , AuroraWorld uses it as a structured prior to guide egocentric video synthesis through a latent diffusion-based video generation model. Specifically, the temporal, spatial, and motion components provide complementary constraints for video generation through temporal observation conditioning and spatial-motion-aware latent initialization during the denoising process.
Temporal Anchor State Conditioning
Given constructed keyframes , we place each anchor at its original temporal location and construct a sparse conditioning video:
Meanwhile, we introduce a binary anchor mask to indicate valid temporal locations:
where indicates the presence of an egocentric temporal anchor at the corresponding temporal location, while denotes an unconditioned slot.
The sparse temporal observation condition is encoded using a VAE encoder and concatenated with the aligned mask:
where denotes a latent VAE encoder, represents channel-wise concatenation, and broadcasts and temporally packs the frame-level mask to match the VAE latent layout.
Spatial-Motion Anchor-Guided Latent Initialization
Current generative approaches for embodied world modeling typically initialize the generation process from unstructured Gaussian noise, where the initial state contains no explicit information about the underlying environment or embodied interaction. This would lead to ambiguous and physically inconsistent generation under large viewpoint changes and sparse observations. To address this limitation, AuroraWorld constructs a spatial-motion guided latent prior from the inferred spatial and motion components and and injects it into the initial diffusion state.
Specifically, given the latent spatial representation extracted from sparse exocentric observations, we first transform it into the spatial feature space:
Meanwhile, given the inferred component , we construct a spatial-motion conditioned query for each latent timestep and spatial location :
The query encodes the desired observation conditioned on spatial location and embodied motion, while the end-effector pose within the motion representation implicitly provides the corresponding ego-view configuration.
The spatial-motion query retrieves relevant geometric information from through cross-attention:
This operation associates motion evolution with corresponding spatial structures, producing a view- and manipulation-aware latent prior:
where projects each retrieved feature into the VAE latent channels. Stacking over all latent positions and standardizing the resulting tensor per sample over all latent elements gives . We define the structured terminal latent as
where has i.i.d. standard-normal entries with the same tensor shape and controls the amplitude of the structured component. The resulting initialization guides diffusion sampling from a structured latent state that incorporates scene geometry and embodied motion, rather than from an unconstrained stochastic latent.
Training. The structured 4D world representation provides the initial state prior and observation constraints for egocentric video generation. Specifically, the spatial and motion components and are first integrated into a structured terminal latent , which replaces the conventional Gaussian initialization in the flow-matching process. During training, the intermediate latent at timestep is obtained as:
where denotes the clean egocentric video latent.
The diffusion model is optimized to predict the corresponding velocity field:
where denotes the time-aligned exocentric visual feature sequence, and denotes the text prompt embedding. denotes the flow-matching velocity field parameterized by the video generation backbone. denotes the continuous flow timestep that controls the interpolation between the clean video latent and the structured terminal latent.
Experiments
Experimental Setup
Datasets. To comprehensively evaluate AuroraWorld from video generation to embodied policy learning and physical-world validation, we conduct experiments on three datasets. DROID (Khazatsky et al. [24]) is used for large-scale real-world exo-to-ego video generation, where we select 17k successful manipulation episodes with reliable camera calibration and reserve an additional 200 episodes for evaluation. RLBench (James et al. [19]) is used for downstream policy learning evaluation under controlled simulation on an eight-task subset of the PerAct suite (Shridhar et al. [40]). We further collect an in-house ARX X5 robot dataset for physical-world validation, which contains 600 real-world manipulation episodes across six tabletop tasks.
Baselines. Several representative approaches are selected for comparison: (1) frame-wise synthesis methods, including VGGT (Wang et al. [44]) and LagenNVS (Szymanowicz et al. [42]); (2) general video generation and world modeling approaches, including SVD (Blattmann et al. [5]) and BWM (Boundless World Model) (Ji et al. 2026); and (3) exo-to-ego generation methods, including Exo2Ego-V (Liu et al. [31]) and WristWorld (Qian et al. [38]). Boundless World Model is a 5B-parameter action-conditioned video generation model. We provide it with the ground-truth first egocentric frame and condition it on the dense trajectory inferred by our IDM from the source exocentric observations, providing a strong baseline to evaluate the effectiveness of our structured 4D world modeling.
Evaluation Metrics. For video generation quality, we adopt PSNR, SSIM, LPIPS, FID, and FVD as quantitative metrics. PSNR and SSIM measure pixel-level fidelity and structural similarity between synthesized and ground-truth frames, rerespectively, while LPIPS evaluates perceptual similarity based on deep visual features. FID measures the distributional distance between generated and real image features, reflecting visual realism, whereas FVD evaluates the distributional discrepancy between generated and real video features to assess temporal dynamics and consistency. For downstream policy learning, we use task success rate as the evaluation metric to assess the effectiveness of synthesized egocentric observations for visuomotor control.

Figure 3. Qualitative comparison on DROID. Rows correspond to exocentric left/right ground-truth views, egocentric ground-truth views, and generated egocentric views from AuroraWorld and competing methods, including WristWorld, BWM, and Exo-Ego-V. The results show that AuroraWorld generates egocentric videos with improved geometric consistency, appearance fidelity, and temporal coherence compared with existing approaches.
Implementation Details. We implement AuroraWorld based on the 1.3B Wan-Fun-InP backbone (Wan et al. [43]). For parameters, we set , , , , and .
Experimental Results and Analysis
Video Generation Evaluation. We first compare the video generation quality of different methods, with results on DROID reported in Table 1. The results lead to the following observations. First, although VGGT and LagerNVS leverage geometric information for view synthesis, they perform frame-wise generation and lack explicit modeling of temporal dependencies and motion continuity across consecutive frames, leading to inferior video generation performance compared with video-based approaches. Second, the proposed method outperforms existing models on most evaluated metrics by jointly modeling spatial structures, temporal evolution, and motion dynamics, thereby generating more coherent and physically consistent egocentric videos. In addition, although BWM benefits from a larger backbone and uses ground-truth initial egocentric frames as conditioning inputs, the proposed method still achieves better performance. This highlights the advantage of explicitly modeling structured 4D world representations.
| Method | PSNR | SSIM | LPIPS | FID | FVD |
| VGGT | 8.93 | 0.206 | 0.784 | 208.98 | 834.02 |
| LagenNVS | 15.05 | 0.5130 | 0.4332 | 40.991 | 245.97 |
| SVD | 12.84 | 0.4971 | 0.5294 | 31.874 | 363.84 |
| BWM | 15.77 | 0.5012 | 0.3309 | 25.632 | 70.073 |
| Exo2Ego-V | 15.43 | 0.4909 | 0.4020 | 22.731 | 130.11 |
| WristWorld | 13.68 | 0.4204 | 0.4527 | 30.267 | 141.98 |
| AuroraWorld | 17.87 | 0.5800 | 0.2648 | 15.437 | 60.171 |
Table 1. Quantitative comparison of egocentric video generation on DROID. Best results are highlighted in bold.
Downstream Policy Evaluation in Simulation. We further evaluate whether synthesized egocentric videos provide policy-relevant observations for downstream visuomotor learning using RVT-2 (Goyal et al. [11]) and (Black et al. [4]) as downstream policy backbones. In the experiments, we consider 8 manipulation tasks on the RLBench benchmark (James et al. [19]), including non-prehensile manipulation, pick-and-place, and precision insertion. Policies are trained from expert demonstrations and evaluated over 10 runs, with 25 trials per task in each run. The results are reported in Table 2. In the table, WristWorld, BWM, and AuroraWorld denote training with synthesized egocentric videos generated by the corresponding methods. For a controlled comparison, all three synthesized-view settings use the same action labels derived from the robot state trajectories inferred by our IDM from the source exocentric demonstrations; therefore, they differ only in the synthesized egocentric observations used for policy training. From
| Policy | WristWorld | BWM | AuroraWorld |
| RVT-2 | 55.60 | 61.10 | 64.00 |
| 36.85 | 37.70 | 40.15 |
Table 2. RLBench policy success rates (%) trained with different egocentric observations.
| Task | w/o Ego | WW | BWM | AuroraWorld | Real Ego |
| Fold Towel | 2/25 | 11/25 | 16/25 | 15/25 | 17/25 |
| Pull Drawer | 0/25 | 9/25 | 12/25 | 15/25 | 15/25 |
| Stack Cups | 0/25 | 0/25 | 1/25 | 0/25 | 12/25 |
| Push Ball | 8/25 | 8/25 | 9/25 | 11/25 | 14/25 |
| Push Cube | 7/25 | 13/25 | 13/25 | 15/25 | 18/25 |
| Erase Whiteboard | 7/25 | 7/25 | 9/25 | 11/25 | 12/25 |
| Average | 16.0% | 32.0% | 40.0% | 44.7% | 58.7% |
caption: Real-robot policy evaluation on the ARX X5. WW denotes Wristworld.
:::
the results, we observe that AuroraWorld achieves the highest average success rate among the synthesized-view methods with both policy backbones. With RVT-2, AuroraWorld achieves 64.00%, outperforming BWM and WristWorld by 2.90 and 8.40 percentage points, respectively. With , AuroraWorld achieves 40.15%, exceeding BWM and WristWorld by 2.45 and 3.30 percentage points, respectively. These results demonstrate that, under the same IDM-inferred action supervision, egocentric observations synthesized by AuroraWorld provide more effective training data for downstream visuomotor policies.
Real-Robot Policy Evaluation. To evaluate the real-robot applicability of AuroraWorld, we conduct experiments on the ARX X5 platform, with the results summarized in Table 3 and 4. Table 3 shows that AuroraWorld achieves the best video generation performance. Furthermore, Table 4 evaluates policies trained under five observation settings: without egocentric observations (w/o Ego), with egocentric observations synthesized by WristWorld, BWM, or AuroraWorld, and with real egocentric observations (Real Ego). Across all five settings, policy training uses action targets derived from IDM-inferred trajectories. Compared with w/o Ego, training with synthesized egocentric observations consistently improves policy performance, indicating that the generated views recover visual information relevant to downstream manipulation. Among the synthesized-view settings, AuroraWorld achieves the highest success rate, outperforming the second-best method by 4.7 percentage points. Together, these results demonstrate that synthesized egocentric observations can effectively recover policy-relevant visual information and improve visuomotor policy learning, while higher-quality synthesis further translates into better downstream policy performance. Nevertheless, AuroraWorld still exhibits a performance gap compared with the ground-truth setting on certain tasks, such as cup stacking. This task requires highly accurate object pose estimation and contact-state understanding, for which synthesized observations may not yet provide sufficient precision for fine-grained spatial alignment. We leave addressing this limitation for future work.
| Method | PSNR↑ | SSIM↑ | LPIPS↓ | FID↓ | FVD↓ |
| WristWorld | 18.24 | 0.654 | 0.414 | 22.71 | 306.34 |
| BWM | 19.88 | 0.709 | 0.422 | 39.99 | 373.72 |
| AuroraWorld | 20.01 | 0.681 | 0.375 | 16.91 | 149.93 |
Table 3. Quantitative comparison of egocentric video generation on ARX X5. Best results are highlighted in bold.
Ablation Studies
We ablate the spatial scene representation , temporal keyframe anchors , and motion trajectory prior in Table 5. Among individual components, provides the
| PSNR | SSIM | LPIPS | FID | FVD | |||
| — | — | — | 12.96 | 0.43 | 0.50 | 30.78 | 192.93 |
| ✓ | — | — | 13.09 | 0.50 | 0.49 | 24.69 | 168.45 |
| — | ✓ | — | 16.69 | 0.55 | 0.31 | 16.95 | 76.12 |
| — | — | ✓ | 14.52 | 0.49 | 0.42 | 21.99 | 129.28 |
| ✓ | — | ✓ | 14.38 | 0.49 | 0.42 | 20.47 | 110.43 |
| — | ✓ | ✓ | 17.45 | 0.57 | 0.28 | 15.79 | 64.66 |
| ✓ | ✓ | — | 17.10 | 0.56 | 0.29 | 15.68 | 65.22 |
| ✓ | ✓ | ✓ | 17.87 | 0.58 | 0.26 | 15.44 | 60.17 |
caption: Ablation study of different components.
:::
| Interior anchors | PSNR | SSIM | LPIPS | FID | FVD |
| 0 | 14.38 | 0.49 | 0.42 | 20.47 | 110.43 |
| 1 | 15.60 | 0.52 | 0.35 | 18.04 | 78.35 |
| 3 | 17.87 | 0.58 | 0.26 | 15.44 | 60.17 |
| 8 | 17.85 | 0.58 | 0.26 | 17.86 | 63.44 |
| All | 18.09 | 0.59 | 0.26 | 17.37 | 73.09 |
caption: Ablation study on the number of interior anchors.
:::
largest baseline gain, improving PSNR from 12.96 to 16.69 and reducing FVD from 192.93 to 76.12. also improves all metrics, whereas primarily benefits distributional quality, reducing FID and FVD with limited changes in reconstruction metrics. Adding either or to further improves all five metrics. The full model combines all three components and achieves the best results across all five metrics. These results indicate that temporal anchors provide the strongest target-view constraint, while the spatial and motion priors contribute complementary geometric and dynamic guidance.
We further study the effect of temporal anchor number in Table 6. Results show that increasing the number of anchors generally improves reconstruction fidelity and temporal consistency through additional egocentric view constraints. However, conditioning on all frames yields marginal improvements in PSNR, SSIM, and LPIPS but degrades the distributional and temporal metrics. We attribute this to the over-constrained nature of dense conditioning, which introduces strong appearance-level supervision, encouraging the model to rely on frame-wise correspondence rather than infer latent temporal evolution. As a result, generation becomes closer to appearance reconstruction than predictive synthesis.
Conclusion
In this work, we introduce AuroraWorld, a structured 4D embodied world model for generating egocentric robot videos from sparse exocentric observations. Unlike existing approaches that primarily rely on appearance-level correspondence, AuroraWorld infers a structured representation of the embodied environment, capturing temporal state transitions, spatial scene structures, and motion evolution. Based on this representation, we construct an anchor-guided video generation framework, where synthesized egocentric keyframes, spatial representations, and motion trajectories jointly guide the generation process. Extensive experiments on real-world and simulated robotic benchmarks demonstrate that Aurora-World achieves improved exo-to-ego video generation quality and enhances downstream policy learning performance. These results highlight the importance of structured world representations for scalable embodied video generation and robot learning.
Implementation and Training Details
AuroraWorld implementation follows the two-stage framework described in the main text. In the first stage, AuroraWorld constructs a structured embodied world representation from sparse exocentric observations by modeling temporal, spatial, and motion components. In the second stage, this representation conditions a PAI/Wan2.1-Fun-InP diffusion generator (Wan et al. [43]) to synthesize coherent and physically consistent egocentric videos. Table 7–Table 9 summarize the module-specific configurations on DROID (Khazatsky et al. [24]). For trajectory inference, we adapt the IDM to the available exocentric views in each dataset: two synchronized views for DROID and RLBench, and one fixed view for ARX X5.
| Parameter | Setting |
| Backbone | GR00T-Dreams IDM |
| Batch size | 64 |
| Optimizer | AdamW, cosine schedule |
| Learning rate | |
| Training steps | 150k steps |
| Chunk horizon | 16 |
| Chunk stride | 8 |
| Clip length | 81 frames |
| Hardware | 8 × L40 GPUs |
Table 7. Trajectory predictor configuration on DROID.
| Parameter | Setting |
| Backbone | PAI/Wan2.1-Fun-1.3B-InP |
| Batch size | 64 |
| Clip length | 81 frames |
| Height & Width | 180 × 320 |
| Interior anchors | 3 |
| Scene tokens | 512 |
| Structured-noise weight | 0.5 |
| Trainable modules | Full |
| Optimizer | AdamW, lr = |
| Training epochs | 10 |
| CFG | 5.0 |
| Hardware | 8 × A800 GPUs |
Table 9. Generator configuration on DROID.
Video Generation Evaluation
Baseline Adaptation
On DROID, we adapt the GR00T-Dreams IDM (Jang et al. [20]) to predict an -D end-effector state trajectory from two synchronized exocentric views. For the action-conditioned models, the same IDM-inferred trajectory is used to condition both BWM and AuroraWorld. The remaining baselines follow their respective native input interfaces, as detailed below.
Frame-wise synthesis. VGGT (Wang et al. [44]) and LagerNVS (Szymanowicz et al. [42]) synthesize each target frame from the two synchronized exocentric frames and the corresponding target egocentric camera pose provided by the dataset. VGGT first reconstructs the scene and projects it into the target view, whereas LagerNVS directly renders the target-view image.
Video generation and world modeling. We fine-tune SVD (Blattmann et al. [5]) on DROID and enable cross-attention to both exocentric videos. SVD is initialized with the first recorded egocentric frame and receives no robot action or trajectory condition. For BWM (Ji et al. 2026), we adopt its three-view input setting. The model receives
| Parameter | Setting |
| Batch size | 64 |
| Height | 180 |
| Width | 320 |
| Optimizer | AdamW, cosine schedule |
| Grad clip | 3 |
| Learning rate | |
| Training steps | 100k steps |
| Attention type | bidirectional cross attention |
| Source views | 2 |
| Hardware | 8 × L40 GPUs |
Table 8. LagerNVS configuration on DROID.
the complete sequences from both exocentric views together with the first recorded egocentric frame, while supervision is applied only to the target egocentric video. BWM is conditioned on the -D state trajectory inferred by our IDM from the source exocentric observations, which is converted to its native action-conditioning representation.
Exo-to-ego video generation. Exo2Ego-V (Liu et al. [31]) and WristWorld (Qian et al. [38]) receive the complete synchronized video sequences from both exocentric views without access to recorded target egocentric frames. Following its pose-conditioned interface, Exo2Ego-V is additionally provided with the synchronized target egocentric camera-pose trajectory. WristWorld estimates the target-camera pose trajectory from the source observations and uses it to project the reconstructed scene into the target egocentric view.
WorldArena Evaluation
Adapted WorldArena protocol. We apply the official WorldArena protocol (Shang et al. [39]) using a DROID adapter, with the same metadata, instructions, evaluators, and normalization procedure for all methods.
The protocol comprises six groups: Visual Quality (Image Quality, Aesthetic Quality, and JEPA), Motion Quality (Dynamic Degree, Flow, and Motion Smoothness), Content Consistency (Subject, Background, and Photometric Consistency), Physics Adherence (Interaction and Trajectory Accuracy), 3D Accuracy (Depth and Perspectivity), and Controllability (Instruction, Semantic, and Action Following). Qwen3-VL-8B-Instruct evaluates Interaction, Perspectivity, and Instruction Following using the instruction and 16 uniformly sampled frames under deterministic decoding. All remaining metrics are computed using the released evaluators.
| Method | Subj. | Aes. | Img. | BG | Dyn. | Inter. | Persp. | Instr. | Sem. | Flow | Depth | Photo. | Smooth. | JEPA |
| WristWorld | 79.5 | 29.1 | 52.8 | 90.5 | 58.7 | 55.1 | 72.6 | 53.3 | 86.3 | 32.1 | 87.2 | 1.8 | 82.5 | 91.4 |
| Exo2Ego-V | 77.7 | 27.9 | 57.4 | 89.1 | 49.7 | 52.7 | 69.0 | 45.2 | 87.0 | 24.5 | 88.6 | 0.7 | 80.0 | 57.2 |
| BWM | 79.2 | 29.4 | 49.9 | 88.4 | 59.9 | 54.3 | 73.4 | 57.7 | 88.9 | 34.6 | 91.9 | 2.8 | 82.9 | 92.6 |
| SVD | 79.0 | 31.1 | 47.7 | 88.7 | 30.1 | 58.2 | 69.2 | 53.5 | 86.1 | 31.4 | 86.7 | 1.1 | 72.8 | 76.5 |
| LagerNVS | 75.1 | 25.1 | 41.7 | 85.7 | 31.8 | 44.4 | 63.4 | 41.4 | 86.3 | 17.9 | 87.9 | 1.8 | 67.6 | 81.0 |
| VGGT | 73.6 | 29.5 | 60.7 | 87.7 | 98.1 | 25.9 | 53.7 | 21.1 | 79.9 | 100.0 | 81.9 | 0.1 | 56.3 | 63.8 |
| AuroraWorld | 79.9 | 28.2 | 53.2 | 91.0 | 63.1 | 59.7 | 74.7 | 57.4 | 88.8 | 33.4 | 93.1 | 3.1 | 80.5 | 97.6 |
Table 11. Results for the 14 retained normalized WorldArena metrics on the same DROID videos and input conditions as Table 10. Subj.: subject consistency; Aes.: aesthetic quality; Img.: image quality; BG: background consistency; Dyn.: dynamic degree; Inter.: interaction quality; Persp.: perspectivity; Instr.: instruction following; Sem.: semantic alignment; Photo.: photometric consistency; Smooth.: motion smoothness.
We exclude Action Following because it requires counterfactual action prompts and Trajectory Accuracy because it assumes globally visible motion. Neither metric is well-defined for a single egocentric view. After excluding these two metrics, we retain 14 metrics. Let denote this set of metrics and denote the dataset-level normalized score of method on metric . We compute the egocentric-view aggregate as
Table 10 shows that AuroraWorld achieves the highest 14-metric aggregate score of 64.6, exceeding BWM and WristWorld by 1.3 and 2.2, respectively. It ranks first on 7 of the 14 retained metrics and on four category-level scores: Visual Quality, Content Consistency, Physics Adherence, and 3D Accuracy. In particular, the gains in subject consistency, perspectivity, depth, and JEPA are consistent with improved preservation of scene identity and geometry. AuroraWorld does not lead Motion Quality or Controllability; VGGT obtains the highest Motion Quality score, whereas BWM performs best on Controllability. These results, together with the anchor ablations, suggest that state-conditioned anchors improve geometric and interaction consistency without uniformly improving every aspect of video dynamics.
| Method | EWMScore | Visual Quality | Motion Quality | Content Consistency | Physics Adherence | 3D Accuracy | Controllability |
| WristWorld | 62.4 | 57.7 | 57.8 | 57.3 | 55.1 | 79.9 | 69.8 |
| Exo2Ego-V | 57.6 | 47.5 | 51.4 | 55.9 | 52.7 | 78.8 | 66.1 |
| BWM | 63.3 | 57.3 | 59.2 | 56.8 | 54.3 | 82.7 | 73.3 |
| SVD | 58.0 | 51.8 | 44.7 | 56.3 | 58.2 | 78.0 | 69.8 |
| LagerNVS | 53.7 | 49.3 | 39.1 | 54.2 | 44.4 | 75.7 | 63.9 |
| VGGT | 59.4 | 51.3 | 84.8 | 53.8 | 25.9 | 67.8 | 50.5 |
| AuroraWorld | 64.6 | 59.7 | 59.0 | 58.0 | 59.7 | 83.9 | 73.1 |
Table 10. WorldArena scores for DROID egocentric videos. EWMScore averages the 14 applicable metrics defined in Appendix § 9.2.
Additional Ablations
Anchor selection. We compare uniform sampling with motion-stabilized anchor selection under an identical budget of five anchors, including the first and last frames. Uniform sampling selects frames , whereas our method retains both endpoints and selects three interior anchors near motion-transition and subsequent stabilization events. As shown in Table 13, motion-stabilized selection consistently improves all five evaluation metrics, reducing FVD from 69.634 to 60.171. These results suggest that event-aligned anchors better capture task-relevant manipulation phases, while uniform sampling may oversample stationary intervals or miss brief motion transitions. Figure 4 provides a qualitative comparison and shows that the selected anchors are better aligned with the target manipulation phase.

Figure 4. Qualitative comparison between uniform sampling and motion-stabilized temporal-anchor selection. Under the same five-anchor budget, motion-stabilized selection better aligns the generated sequence with task-relevant manipulation phases.
| Setting | PSNR ↑ | SSIM ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
| Anchor selection | |||||
| Uniform | 17.39 | 0.572 | 0.280 | 15.754 | 69.634 |
| Motion-stab. | 17.87 | 0.580 | 0.265 | 15.437 | 60.171 |
| Number of interior anchors | |||||
| 0 | 14.38 | 0.492 | 0.425 | 20.466 | 110.430 |
| 1 | 15.60 | 0.524 | 0.354 | 18.039 | 78.348 |
| 3 | 17.87 | 0.580 | 0.265 | 15.437 | 60.171 |
| 8 | 17.85 | 0.584 | 0.262 | 17.856 | 63.442 |
| All frames | 18.09 | 0.585 | 0.261 | 17.369 | 73.094 |
Table 13. Additional ablations on DROID. The two blocks compare temporal-anchor selection and the number of interior anchors.
Anchor count. We further study the effect of the number of temporal anchors on video quality. All sparse settings retain the first and last frames while varying the number of interior anchors. As shown in Table 13, increasing the number of anchors substantially improves performance over endpoint-only conditioning, but the gains are not monotonic. Three interior anchors achieve the best FID and FVD, while using eight anchors or all frames provides only marginal improvements in frame-level metrics and degrades temporal quality. These results suggest that a small number of informative anchors provides a better balance between visual fidelity and temporal consistency than dense conditioning.
Additional Visual Evaluation
Standard metrics on RLBench. We further evaluate egocentric-view generation on the RLBench Ego-Withheld split using standard reconstruction and perceptual metrics. As shown in Table 12, AuroraWorld achieves the best PSNR and LPIPS, while obtaining an FVD nearly identical to WristWorld.
| Method | PSNR | SSIM | LPIPS | FID | FVD |
| VGGT | 10.74 | 0.33 | 0.60 | 184.46 | 1451.10 |
| LagerNVS | 18.77 | 0.56 | 0.25 | 53.04 | 401.24 |
| SVD | 15.88 | 0.51 | 0.45 | 70.54 | 450.37 |
| BWM | 20.40 | 0.60 | 0.24 | 47.38 | 146.05 |
| Exo2Ego-V | 20.37 | 0.58 | 0.19 | 31.59 | 269.39 |
| WristWorld | 19.55 | 0.56 | 0.19 | 22.75 | 59.68 |
| AuroraWorld | 20.89 | 0.57 | 0.14 | 28.32 | 59.67 |
Table 12. Standard visual evaluation results on the RLBench Ego-Withheld Split.
Qualitative comparisons. Figures 5 and 6 compare task-relevant geometry and temporal consistency on RLBench and DROID.

Figure 5. Qualitative comparison on RLBench.
Effect of Target-Condition Source on Video Generation
By default, AuroraWorld infers an 8-D end-effector state trajectory from synchronized exocentric observations. The inferred trajectory guides temporal-anchor selection, target-view prediction, and structured latent initialization. We evaluate how trajectory accuracy affects video generation by replacing the inferred trajectory with synchronized ground-truth states while keeping the generator weights, visual inputs, and evaluation protocol fixed. This controlled setup isolates the effect of trajectory quality on video generation.
Within-method comparison. Table 14 first compares the two trajectory sources for AuroraWorld on DROID, RLBench, and ARX X5. Ground-truth trajectories consistently improve PSNR, SSIM, and LPIPS across all three datasets. On DROID, replacing inferred trajectories with ground-truth trajectories improves PSNR from 17.87 to 19.17 and reduces FVD from 60.17 to 52.11. Similar improvements in frame-level reconstruction and perceptual similarity are observed on RLBench and ARX X5, demonstrating that more accurate motion trajectories provide stronger guidance for target-view synthesis. The distribution-level metrics exhibit different behavior across datasets: FID and FVD improve on DROID but increase on RLBench and ARX X5. These results indicate that improved frame-aligned reconstruction does not necessarily produce a closer match to dataset-level image and video feature distributions, which also reflect generation dynamics and dataset-specific statistics.
| Trajectory | Source | PSNR ↑ | SSIM ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
| DROID | Inferred | 17.87 | 0.580 | 0.265 | 15.44 | 60.17 |
| Ground-truth | 19.17 | 0.6218 | 0.2185 | 14.16 | 52.11 | |
| RLBench | Inferred | 20.89 | 0.5708 | 0.1401 | 28.32 | 59.67 |
| Ground-truth | 21.84 | 0.5845 | 0.1216 | 28.85 | 72.11 | |
| ARX X5 | Inferred | 20.01 | 0.6808 | 0.3747 | 16.91 | 149.93 |
| Ground-truth | 20.25 | 0.6827 | 0.3706 | 20.91 | 200.32 |
Table 14. Video-generation metrics of AuroraWorld under inferred and synchronized ground-truth end-effector state trajectories.
Comparison with other methods. We further compare the effect of target-condition quality across BWM, WristWorld, and AuroraWorld on DROID. BWM and AuroraWorld are conditioned on end-effector state trajectories, while WristWorld projects reconstructed 3D scenes into the egocentric image plane using frame-wise target ego-camera poses. For BWM, the state trajectory is converted to its native action-conditioning representation. In the inferred-condition setting, BWM and AuroraWorld use the dense state trajectory inferred by our IDM from the source exocentric observations, while WristWorld uses its corresponding inferred target-view camera poses. In the ground-truth-condition setting, each method receives its synchronized ground-truth target condition. The generator weights and evaluation protocol remain fixed within each method, and BWM receives the ground-truth first egocentric frame in both settings.

Figure 6. DROID examples: exocentric left/right and egocentric-view ground truth, Exo2Ego-V, WristWorld, BWM, SVD, LagerNVS, VGGT, and AuroraWorld. AuroraWorld better preserves target geometry, appearance, and temporal layout.
Table 15 shows that ground-truth target conditions improve all five metrics for BWM, WristWorld, and AuroraWorld. In particular, ground-truth conditions improve PSNR by 0.71, 1.54, and 1.30 for BWM, WristWorld, and AuroraWorld, respectively, while reducing FVD by 11.18, 38.05, and 8.06. These consistent within-method improvements confirm that accurate target states and camera poses provide stronger motion and viewpoint guidance.
| Target Condition | PSNR ↑ | SSIM ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
| WristWorld | |||||
| Inferred | 13.68 | 0.4204 | 0.4527 | 30.27 | 141.98 |
| Ground-truth | 15.22 | 0.5161 | 0.3981 | 25.90 | 103.93 |
| BWM | |||||
| Inferred | 15.77 | 0.5012 | 0.3309 | 25.63 | 70.07 |
| Ground-truth | 16.48 | 0.5319 | 0.2933 | 24.28 | 58.89 |
| AuroraWorld | |||||
| Inferred | 17.87 | 0.5800 | 0.2648 | 15.44 | 60.17 |
| Ground-truth | 19.17 | 0.6218 | 0.2185 | 14.16 | 52.11 |
Table 15. DROID egocentric video generation under inferred and ground-truth target conditions. Bold marks the best result overall; underlining marks the best result with inferred target conditions.
Across both target-condition settings, AuroraWorld achieves the best results on all five metrics. With inferred conditions, AuroraWorld obtains 17.87 PSNR and 60.17
FVD, outperforming BWM by 2.10 PSNR and 9.90 FVD points and WristWorld by 4.19 PSNR and 81.81 FVD points. With ground-truth conditions, AuroraWorld further improves to 19.17 PSNR and 52.11 FVD, exceeding BWM by 2.69 PSNR and 6.78 FVD points and WristWorld by 3.95 PSNR and 51.82 FVD points. Moreover, AuroraWorld achieves these results without receiving a recorded egocentric frame, whereas BWM additionally receives the first recorded egocentric frame. The consistent ranking under both inferred and ground-truth conditions demonstrates the effectiveness of structured 4D modeling in translating target trajectories into geometrically accurate and temporally coherent egocentric videos.
RLBench Policy Evaluation
Task Suite and Data Splits
We use an eight-task subset of the PerAct suite (Shridhar et al. [40]) on RLBench (James et al. [19]). Table 16 maps the abbreviations in the per-task results to canonical task names.
| Abbr. | Task | Abbr. | Task |
| PID | Put item in drawer | RAD | Reach and drag |
| SBCT | Slide block to color target | TT | Turn tap |
| PMS | Put money in safe | OD | Open drawer |
| MOG | Meat off grill | PB | Push buttons |
Table 16. Abbreviations and names of the eight RLBench evaluation tasks.
Each task has 100 training demonstrations split equally into Ego-Available and Ego-Withheld. Generators adapt only on Ego-Available demonstrations with recorded egocentric views. On Ego-Withheld demonstrations, AuroraWorld and
WristWorld synthesize views without recorded egocentric input, while BWM additionally receives the first recorded egocentric frame of the same demonstration. The resulting Ego-Withheld demonstrations are used for Replacement policy training.
Policy Backbones and Training Settings
All egocentric-source variants share this partition, architecture, initialization, and training budget.
RVT-2. We follow the official RLBench recipe for RVT-2 (Goyal et al. [11]), fixing architecture, optimizer, augmentation, and policy hyperparameters.
. We fine-tune the public checkpoint (Black et al. [4]) with OpenPI. LeRobot examples use RVT-style 9-D actions: position, quaternion, gripper, and ignore-collisions flag. We supervise horizon-6 chunks, replan every high-level step, and train 40k steps with global batch 64 at . No-egocentric training uses front/overhead views plus a masked zero wrist slot; other settings retain OpenPI defaults.
Evaluation Protocol and Results
Protocol. Generated egocentric videos are used only for offline policy training on both RLBench and ARX X5, never at test time. During evaluation, ego-enabled policies receive online egocentric observations—simulator-rendered views on RLBench and live wrist-camera RGB on ARX X5—whereas w/o Ego uses exocentric observations only.
Per-task results. Each checkpoint is evaluated over 10 repeats of the same 25 test episodes per task, with at most 25 steps per rollout. This gives 250 rollouts per task, or 2,000 rollouts per setting on RLBench. Table 17 details the RLBench results summarized in Table 4, with task abbreviations defined in Table 16.
| Task | RVT-2 WW | RVT-2 BWM | RVT-2 Ours | WW | BWM | Ours |
| PID | 69.2 | 79.2 | 73.6 | 5.2 | 8.8 | 15.6 |
| RAD | 69.6 | 55.6 | 68.0 | 77.6 | 63.2 | 65.6 |
| TT | 56.8 | 67.6 | 76.8 | 24.8 | 27.2 | 28.0 |
| SBCT | 33.6 | 37.2 | 49.6 | 75.6 | 65.2 | 70.4 |
| OD | 50.4 | 47.6 | 55.6 | 13.6 | 10.8 | 17.2 |
| PMS | 40.0 | 43.6 | 52.0 | 28.8 | 43.6 | 38.0 |
| PB | 69.6 | 86.4 | 82.0 | 31.6 | 41.2 | 36.0 |
| MOG | 55.6 | 71.6 | 54.4 | 37.6 | 41.6 | 50.4 |
| Avg. | 55.60 | 61.10 | 64.00 | 36.85 | 37.70 | 40.15 |
Table 17. Per-task RLBench policy success (%) for the eight-task inferred-action Replacement experiment (250 rollouts per task). WW: WristWorld; BWM: Boundless World Model.
Real-Robot ARX X5 Evaluation
Real-World Setup and Data Collection
Hardware setup. Our real-world experiments use the ARX X5 single-arm robot equipped with two Intel RealSense D435i cameras. One camera provides a fixed exocentric view, while the other is wrist-mounted to capture egocentric observations. The IDM uses this fixed exocentric view to infer end-effector trajectories.
Data collection. We collect expert demonstrations using leader–follower teleoperation. Both cameras record synchronized RGB observations at 640 × 480 and 15 Hz, together with robot states. We collect 100 demonstrations for each of six tasks, yielding 600 episodes. For each task, the demonstrations are divided into disjoint 50-episode Ego-Available
and Ego-Withheld splits. The former adapts the video generator, which synthesizes egocentric observations for the latter without accessing its recorded egocentric videos.
Real-world tasks. The six tasks cover deformable-object manipulation, articulated-object interaction, stacking, pushing, and surface contact: Fold towel, Pull drawer, Stack cups, Push yellow ball, Push cube, and Erase whiteboard. Figure 8 shows representative trajectories.

Figure 8. ARX X5 performing six real-world manipulation tasks.
Policy and Evaluation Protocol
We evaluate MOTIF [53] on all six tasks using exocentric RGB, robot state, language, and optionally wrist RGB. Architecture and training remain fixed across conditions. All ARX X5 policies use training action targets derived from IDM-inferred end-effector trajectories. Each condition receives 25 physical trials per task under identical initialization, termination, and success criteria.
Additional Evaluation Results
Data augmentation. Beyond the egocentric-replacement experiment, we augment the 300 Ego-Available demonstrations with the 300 Ego-Withheld demonstrations. Table 18 compares WristWorld, BWM, and AuroraWorld with Ego-Available-only and Real-Ego training. Evaluation follows Section D.2.
| Task | Avail. only | +WW | +BWM | +Ours | Real Ego |
| Fold Towel | 16/25 | 17/25 | 23/25 | 21/25 | 22/25 |
| Pull Drawer | 14/25 | 15/25 | 16/25 | 20/25 | 20/25 |
| Stack Cups | 13/25 | 6/25 | 10/25 | 8/25 | 16/25 |
| Push Ball | 14/25 | 17/25 | 17/25 | 18/25 | 20/25 |
| Push Cube | 16/25 | 22/25 | 19/25 | 22/25 | 23/25 |
| Erase Whiteboard | 10/25 | 10/25 | 13/25 | 17/25 | 16/25 |
| Average | 55.3% | 58.0% | 65.3% | 70.7% | 78.0% |
Table 18. ARX X5 policy success under data augmentation. Each entry reports successful trials out of 25 physical trials. WW denotes WristWorld.
Qualitative video comparison. Figure 7 compares generated egocentric views on ARX X5.

Figure 7. Qualitative comparison of egocentric video generation on ARX X5.
Limitations
AuroraWorld relies on several upstream predictions whose errors may propagate through the generation pipeline. In particular, inaccuracies in the IDM trajectory can affect temporal-anchor selection, target-camera prediction, and structured-prior construction, as indicated by the trajectory-source comparison. Sparse exocentric observations may also be insufficient to resolve the target view under severe occlusion, rapid contact dynamics, or tasks requiring fine-grained spatial alignment, causing rendered-anchor errors to introduce incorrect geometry or temporal phase information into the generator. Consequently, policies trained with synthetic egocentric observations do not consistently match the performance of those trained with recorded egocentric views. Although AuroraWorld improves average policy performance in both simulation and real-robot experiments, noticeable gaps remain on tasks involving precise contact, occlusion, or multi-object spatial reasoning.
References
- [1]N. Agarwal, A. Ali, J. Allen, M. Antolini, A. Aubame, A. Azzolini, J. Bai, M. Bala, Y. Balaji, J. Bapst, et al. Cosmos 3: Omnimodal world models for physical AI, 2026.arxiv.org/abs/2606.02800
- [3]Y. Bian, Z. Zhang, X. Ju, M. Cao, L. Xie, Y. Shan, and Q. Xu. VideoPainter: Any-length video inpainting and editing with plug-and-play context control. In SIGGRAPH, pages 1–12, 2025.arxiv.org/abs/2503.05639
- [4]K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. R. Equi, C. Finn, N. Fusai, M. Y. Galliker, et al. π₀.₅: a vision-language-action model with open-world generalization. In CoRL, pages 17–40, 2025.arxiv.org/abs/2504.16054
- [5]A. Blattmann, T. Dockhorn, S. Kullal, D. Mendelevitch, et al. Stable Video Diffusion: Scaling latent video diffusion models to large datasets, 2023.arxiv.org/abs/2311.15127
- [6]M. Cai, Q. Wang, Z. Ye, W. Liu, Q. Liu, W. Ye, X. Wang, P. Wan, K. Gai, and X. Yue. VideoCanvas: Unified video completion from arbitrary spatiotemporal patches via in-context conditioning. In CVPR, pages 4475–4485, 2026.arxiv.org/abs/2510.08555
- [7]W. Cao, H. Zhang, F. Tian, Y. Wu, Y. Li, S. Wang, N. Yu, and Y. Liu. FreeOrbit4D: Training-free arbitrary camera redirection for monocular videos via foreground-complete 4D reconstruction. In SIGGRAPH, pages 1–12, 2026.DOI
- [9]C. Chi, Z. Xu, S. Feng, E. Cousineau, Y. Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion Policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 44(10-11):1684–1704, 2025.
- [10]K. Dharmarajan, W. Huang, J. Wu, L. Fei-Fei, and R. Zhang. Dream2Flow: Bridging video generation and open-world manipulation with 3D object flow, 2025.arxiv.org/abs/2512.24766
- [11]A. Goyal, V. Blukis, J. Xu, Y. Guo, Y.-W. Chao, and D. Fox. RVT-2: Learning precise manipulation from few demonstrations. In RSS, 2024.arxiv.org/abs/2406.08545
- [12]K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu, et al. Ego4D: Around the world in 3,000 hours of egocentric video. In CVPR, pages 18995–19012, 2022.arxiv.org/abs/2110.07058
- [14]Y. HaCohen, N. Chiprut, B. Brazowski, D. Shalem, D. Moshe, E. Richardson, E. Levin, G. Shiran, N. Zabari, O. Gordon, P. Panet, S. Weissbuch, V. Kulikov, Y. Bitterman, Z. Melumian, and O. Bibi. LTX-Video: Realtime video latent diffusion, 2024.arxiv.org/abs/2501.00103
- [18]R. Huang, W. Luo, Y. Mao, and K. Mikolajczyk. From none to all: Self-supervised 3D reconstruction via novel view synthesis. In CVPR, pages 37358–37369, 2026.arxiv.org/abs/2603.27455
- [19]S. James, Z. Ma, D. R. Arrojo, and A. J. Davison. RL-Bench: The robot learning benchmark & learning environment. IEEE Robotics and Automation Letters, 5(2):3019–3026, 2020.
- [20]J. Jang, S. Ye, Z. Lin, J. Xiang, J. Björck, Y. Fang, F. Hu, S. Huang, K. Kundalia, Y.-C. Lin, et al. DreamGen: Unlocking generalization in robot learning through video world models. In CoRL, pages 5170–5194, 2025.arxiv.org/abs/2505.12705
- [22]J. Jiang, J. Zhang, Z. Yin, R. Chen, S. Wang, Z. Yu, P. Zeng, X. Cao, X. Wang, J. Song, et al. WSA₁: A 3D-centric world-spatial-action model for generalizable robot control, 2026.arxiv.org/abs/2607.03941
- [23]T. Kang, K. Kim, D. Kim, M. Park, J. Hyung, and J. Choo. EgoX: Egocentric video generation from a single exocentric video. In CVPR, pages 11116–11126, 2026.arxiv.org/abs/2512.08269
- [24]A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al. DROID: A large-scale in-the-wild robot manipulation dataset. In RSS, 2024.arxiv.org/abs/2403.12945
- [25]M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketi, Q. Vuong, et al. OpenVLA: An open-source vision-language-action model. In CoRL, pages 2679–2713, 2025.arxiv.org/abs/2406.09246
- [26]M. J. Kim, Y. Gao, T.-Y. Lin, Y.-C. Lin, Y. Ge, G. Lam, P. Liang, S. Song, M.-Y. Liu, C. Finn, et al. Cosmos Policy: Fine-tuning video models for visuomotor control and planning, 2026.arxiv.org/abs/2601.16163
- [27]W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang, et al. HunyuanVideo: A systematic framework for large video generative models, 2024.arxiv.org/abs/2412.03603
- [28]M. Lee, S. Cho, C. Shin, J. Lee, S. Yang, and S. Lee. Video diffusion models are strong video inpainting. In Proceedings of the Association for the Advancement of Artificial Intelligence Conference, pages 4526–4533, 2025.arxiv.org/abs/2408.11402
- [29]V. Leroy, Y. Cabon, and J. Revaud. Grounding image matching in 3D with MAST3R. In ECCV, pages 71–91, 2024.arxiv.org/abs/2406.09756
- [31]J.-W. Liu, W. Mao, Z. Xu, J. Keppo, and M. Z. Shou. Exocentric-to-egocentric video generation. In NeurIPS, pages 136149–136172, 2024.DOI
- [32]Y. Liu, C. Luo, Z. Tang, J. Peng, and Z. Zhang. VGGT-X: When VGGT meets dense novel view synthesis, 2025.arxiv.org/abs/2509.25191
- [33]M. Luo, Z. Xue, A. Dimakis, and K. Grauman. Put myself in your shoes: Lifting the egocentric perspective from exocentric videos. In ECCV, pages 407–425, 2024.arxiv.org/abs/2403.06351
- [34]M. Mahdi, Y. Fu, N. Savov, J. Pan, D. P. Paudel, and L. Van Gool. Exo2EgoSyn: Unlocking foundation video generation models for exocentric-to-egocentric video synthesis, 2025.
- [35]M. Mahdi, N. Savov, D. P. Paudel, and L. Van Gool. From synchrony to sequence: Exo-to-ego generation via interpolation, 2026.arxiv.org/abs/2604.13793
- [36]B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. NeRF: Representing scenes as neural radiance fields. Communications of the ACM, 65(1):99–106, 2021.
- [37]J. Park, A. S. Ye, and T. Kwon. EgoWorld: Translating exocentric view to egocentric view using rich exocentric observations. In ICLR, 2026.
- [38]Z. Qian, X. Chi, Y. Li, S. Wang, Z. Qin, X. Ju, S. Han, and S. Zhang. WristWorld: Generating wrist-views via 4D world models for robotic manipulation, 2025.
- [39]Y. Shang, Z. Li, Y. Ma, W. Su, X. Jin, Z. Wang, L. Jin, X. Zhang, Y. Tang, H. Su, et al. WorldArena: A unified benchmark for evaluating perception and functional utility of embodied world models, 2026.
- [40]M. Shridhar, L. Manuelli, and D. Fox. Perceiver-Actor: A multi-task transformer for robotic manipulation. In CoRL, pages 785–799, 2023.
- [41]Q. Song, Y. Song, K. Peng, Y. Gao, and M. Z. Shou. World-Wander: Bridging egocentric and exocentric worlds in video generation, 2025.
- [42]S. Szymanowicz, M. Chen, J. Wang, C. Rupprecht, and A. Vedaldi. LagerNVS: Latent geometry for fully neural real-time novel view synthesis. In CVPR, pages 15443–15453, 2026.
- [43]T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, et al. Wan: Open and advanced large-scale video generative models, 2025.
- [44]J. Wang, M. Chen, N. Karaev, A. Vedaldi, C. Rupprecht, and D. Novotny. VGGT: Visual geometry grounded transformer. In CVPR, pages 5294–5306, 2025.arxiv.org/abs/2503.11651
- [45]S. Wang, V. Leroy, Y. Cabon, B. Chidlovskii, and J. Revaud. DUSt3R: Geometric 3D vision made easy. In CVPR, pages 20697–20709, 2024.arxiv.org/abs/2312.14132
- [47]H. Yang, Z. Tan, J. Gong, L. Qin, H. Chen, X. Yang, Y. Sun, Y. Lin, M. Yang, and H. Li. Omni-Video 2: Scaling MLLM-conditioned diffusion for unified video generation and editing, 2026.
- [49]Z. Yin, X. Wang, J. Jiang, K. Deng, P. Chen, S. Li, C. Liu, X. Xu, J. Song, L. Gao, et al. MiVLA: Towards generalizable vision-language-action model with human-robot mutual imitation pre-training. In CVPR, pages 1535–1545, 2026.
- [50]P.-F. Zhang, Y. Cheng, X. Sun, S. Wang, F. Li, L. Zhu, and H. T. Shen. A step toward world models: A survey on robotic manipulation, 2025.arxiv.org/abs/2511.02097
- [51]W. Zhang, H. Liu, Z. Qi, Y. Wang, X. Yu, J. Zhang, R. Dong, J. He, H. Wang, Z. Zhang, et al. DreamVLA: a vision-language-action model dreamed with comprehensive world knowledge. NeurIPS, 38:24195–24228, 2026.
- [52]H. Zhen, Q. Sun, H. Zhang, J. Li, S. Zhou, Y. Du, and C. Gan. Tesseract: Learning 4D embodied world models, 2025.arxiv.org/abs/2504.20995
- [53]H. Zhi, W. Tan, L. Zhu, F. Li, J. Li, G. Yang, and H. T. Shen. MOTIF: Learning action motifs for few-shot cross-embodiment transfer, 2026. URL https://arxiv.org/abs/2602.13764.
- [54]L. Zhong, Y. Liu, Y. Wei, Z. Xiong, S. Liu, and G. Ren. ACoT-VLA: Action chain-of-thought for vision-language-action models. In CVPR, pages 8152–8162, 2026.arxiv.org/abs/2601.11404
- [55]F. Zhu, H. Wu, S. Guo, Y. Liu, C. Cheang, and T. Kong. IRASim: A fine-grained world model for robot manipulation. In ICCV, pages 9834–9844, 2025.
