WorksComputer Science
World-in-Context: Interactive Video World Modeling with In-Context Conditioning
Abstract
Interactive video world models require coordinated control over viewing perspectives, character actions, and camera motion. Existing methods typically leave view modes implicit in text captions, conflate action with camera motion, and inject controls through fragmented architectural pathways that capture interdependencies only indirectly. In this work, we present World-in-Context, an interactive world modeling framework powered by In-Context Conditioning (ICC). Rather than relying on specialized conditioning pathways, World-in-Context unifies view, action, and camera controls into explicit tokens contextualized directly within a shared self-attention space. Our formulation reveals that (i) explicit view tokens disentangle conflicting dynamics and unlock emergent view switching from single-view data, while (ii) action and camera inputs play complementary roles in governing local responsiveness and global stability. To preserve temporal continuity across execution intervals, we further introduce Interaction-Transport RoPE (IT-RoPE), parameterizing temporal attention with accumulated control transitions. On WBench, World-in-Context-5B achieves navigation performance on par with 22B EchoWM ( vs. ) and attains a perspective-switching success rate, substantially outperforming existing 5B baselines (). Controlled analyses further demonstrate compounding advantages of World-in-Context under richer conditioning mixtures, favorable data efficiency, and natural applicability to causal rollouts.

Figure 1. World-in-Context supports versatile interactive video generation across FREE, FPP, and TPP regimes, supporting dynamic in-video view switching under joint action and camera control.
abstract: Interactive video world models require coordinated control over viewing perspectives, character actions, and camera motion. Existing methods typically leave view modes implicit in text captions, conflate action with camera motion, and inject controls through fragmented architectural pathways that capture interdependencies only indirectly. In this work, we present World-in-Context, an interactive world modeling framework powered by In-Context Conditioning (ICC). Rather than relying on specialized conditioning pathways, World-in-Context unifies view, action, and camera controls into explicit tokens contextualized directly within a shared self-attention space. Our formulation reveals that (i) explicit view tokens disentangle conflicting dynamics and unlock emergent view switching from single-view data, while (ii) action and camera inputs play complementary roles in governing local responsiveness and global stability. To preserve temporal continuity across execution intervals, we further introduce Interaction-Transport RoPE (IT-RoPE), parameterizing temporal attention with accumulated control transitions. On WBench, World-in-Context-5B achieves navigation performance on par with 22B EchoWM (81.5 vs. 81.6) and attains a 93.8% perspective-switching success rate, substantially outperforming existing 5B baselines (18.8%). Controlled analyses further demonstrate compounding advantages of World-in-Context under richer conditioning mixtures, favorable data efficiency, and natural applicability to causal rollouts.
Introduction
Video generation has rapidly advanced from passive video generation [10, 22, 41] toward interactive world models [9, 38, 40, 45], empowering users to actively steer dynamic visual environments. In this paradigm, the model acts as an interactive simulator where users provide control inputs to navigate and explore the environment, while the model dynamically predicts the evolving visual observations in response. At its core, effectively simulating such an interactive world hinges on two fundamental dimensions [28, 43]: ➀ how the world is viewed across distinct perspectives, and ➁ how user inputs concurrently govern character dynamics and camera viewports. Together, these dimensions define how an agent actively perceives and intervenes in an unfolding generative world.
Despite recent progress [15, 39, 40, 55, 59], current interactive pipelines often overlook the systematic formulation of these two essential dimensions: ❶ view-mode ambiguity: first- (FPP) and third-person (TPP) perspectives increasingly appear in interactive data, yet view mode is often specified only implicitly through text captions [9, 38, 39, 55], while free-camera (FREE) roaming is frequently conflated with FPP control despite their different motion semantics; and ❷ action-camera entanglement: most pipelines rely primarily on continuous camera trajectories alone for control [13, 31, 39, 55], overlooking the distinct role of discrete actions, especially in third-person interaction where character and camera motion form separate degrees of freedom.
A naive remedy would be simply attaching auxiliary condition channels for each missing signal. However, existing paradigms [9, 19, 33, 40, 45, 56] typically accommodate interactive controls through separate modules (e.g., AdaLN modulation), which tends to isolate different control signals and leaves their cross-dependencies to be captured only indirectly, as shown in Figure 2 (Left). As interaction becomes richer and more compositional, coordinating these modular branches grows increasingly intricate, especially when the interpretation of one signal inherently depends on concurrent inputs and the active viewport.

Figure 2. (Left) Prior methods use implicit view descriptions and separate control pathways. (Right) World-in-Context unifies controls through ICC and models their temporal evolution with IT-RoPE.
In this work, we propose World-in-Context, which unifies multi-dimensional interaction through an In-Context Conditioning (ICC) paradigm. Rather than routing controls into isolated architectural branches, World-in-Context casts diverse interactive signals into token sequences and prepends them directly into the shared self-attention space alongside visual latents. Within this unified interface, we revisit the aforementioned limitations along two fronts: ♣ Explicit view-mode modeling: Rather than leaving perspectives implicit in text captions [9, 38, 40, 59], we explicitly incorporate discrete view tokens (FREE, FPP, and TPP) into the context. Joint training across these regimes distinguishes their motion semantics, while enabling in-video view switching to emerge naturally. ♣ Joint action-camera modeling: We jointly incorporate discrete actions and continuous camera trajectories into the context. Crucially, beyond their necessary separation in TPP interaction, we further study the less-examined FPP and FREE regimes, where camera trajectories often stand in for both forms of control [15, 31, 39]. Our analysis reveals complementary roles: action governs local command fidelity, whereas camera anchors global trajectory stability.
Built upon ICC, World-in-Context natively captures cross-condition dependencies within a unified attention space. While in-context tokens specify what controls are present at individual frames, naive token conditioning can still leave the dynamic evolution between distant moments to be modeled implicitly. To this end, we further introduce IT-RoPE (Interaction-Transport RoPE), a lightweight positional encoding that incorporates pairwise control transitions directly into temporal attention. By jointly modeling control conditions and their temporal evolution, World-in-Context supports coherent interaction across viewing regimes and makes effective use of interactive supervision, with increasing benefits as controls become richer. More broadly, it reframes multi-condition interaction as context modeling, offering a unified interface for interactive world modeling.
To summarize, this work contributes threefold:
We introduce In-Context Conditioning (ICC) for interactive world modeling and present World-in-Context, which represents view, action, and camera controls as tokens within a shared attention space, providing a unified interface for diverse interaction signals.
We investigate the often-overlooked roles of viewing regimes and joint action-camera control, revealing distinct view-dependent dynamics, emergent view switching, and complementary action-camera roles in command fidelity and trajectory stability.
Extensive experiments on WBench demonstrate that World-in-Context-5B matches the 22B EchoVWM in navigation (81.5 vs. 81.6) and substantially outperforms existing 5B baselines in perspective switching (93.8% vs. 18.8%). Controlled studies further confirm compounding gains of World-in-Context under richer controls, favorable sample efficiency, and causal applicability.
Related Work
Interactive Video World Models. Interactive world models extend video generation toward closed-loop interaction, where future observations evolve according to user-steered conditions [3, 6, 32, 36, 45]. Following recent formulations [28, 43], existing models mainly navigate two core dimensions: (i) How the world is viewed. Interactive environments span FREE, FPP, and TPP regimes. Prior works [9, 38, 39, 55] typically subsume them into a shared camera formulation, leaving perspective implicit in text captions and often conflating FREE roaming with FPP motion. We instead explicitly condition on discrete view modes, distinguishing their physical motion semantics and naturally enabling dynamic view switching. (ii) How user inputs affect its evolution. Existing systems rely almost exclusively on continuous camera trajectories for navigation [9, 13, 31, 39, 55, 59] for all view modes, which inevitably entangles character and camera dynamics in TPP. To incorporate richer signals, recent approaches either formulate a unified action interface across different regimes [19], or inject actions and camera poses through distinct architectural pathways [36]. Differing from these paradigms, we explicitly maintain view modes and represent character actuation alongside camera motion as jointly contextualized controls within the same attention space, effectively treating them as coexisting yet independent degrees of freedom.
In-Context Conditioning. In-context learning, originally developed for language models [7], has increasingly been explored in visual generation. Prompt Diffusion [44] first enables diffusion models to infer visual tasks from contextual examples, followed by works that exploit shared attention contexts for task-agnostic or unified image generation [16, 17, 37, 46, 50]. This paradigm has recently extended to video through spatial or temporal in-context generation [8, 21], unified multimodal conditioning with full attention [12, 20], and reference videos as semantic prompts [2]. IC-World [47] further explores in-context generation for shared world modeling, focusing on consistency across multiple observations of the same world. In this work, we extend in-context conditioning to interactive video world modeling.
Preliminary: Problem Formulation
Let denote a video clip of frames, and the initial contextual specification (e.g., a text prompt or a starting frame). In interactive video world modeling, the model generates future visual observations
conditioned on temporally aligned user interaction sequences , parameterizing the conditional distribution:
Modern video diffusion and flow-matching backbones typically instantiate this conditional process within a latent space. Specifically, let denote the latent video representations at diffusion timestep . Existing models typically inject interactive conditions via modular, condition-specific pathways:
where commonly instantiates modulation for camera trajectories [9, 18], additive token injection for discrete actions [5, 19], cross-attention for text-form controls [30], or geometry-aware positional attention and warping for camera control [39, 53, 55]. While effective for isolated signals, such architectural separation isolates diverse controls into disparate parameter spaces, hindering the model from capturing cross-condition dependencies directly. In this work, instead of introducing dedicated condition branches, we reframe multi-modal interactive conditioning as a unified in-context sequence modeling problem:
where diverse controls are recast as unified condition tokens and natively contextualized with visual latents within the shared attention manifold.
World-in-Context
Overview
To avoid the scalability bottleneck of fragmented side channels (e.g., AdaLN or separate cross-attention layers), World-in-Context casts diverse interactive signals directly into sequence tokens within a unified attention space, as shown in Figure 2 (Right). The world modeling space is organized around two complementary pillars: (i) how the world is viewed across perspectives, and (ii) how user inputs steer dynamics through concurrent character action and camera viewport movement.
Unified In-Context Conditioning. Instead of introducing specialized parameter pathways for each control modality, all interactive signals are projected into a unified sequence of condition tokens , matching the backbone hidden dimension :
where denotes concatenation along the sequence dimension. These condition tokens are prepended directly to the visual latents at diffusion timestep . The flow-matching generative backbone then predicts the velocity field over this shared sequence:
Ingesting all controls into the shared self-attention allows cross-condition dependencies to be captured natively without ad-hoc modules, reframing interactive expansion into an in-context modeling problem.
Decomposing Interactive Context. The condition sequence of World-in-Context explicitly decouples the interactive degrees of freedom:
View Mode (): Disentangles FREE, FPP, and TPP regimes via learned categorical tokens, mitigating view ambiguity and naturally supporting dynamic switching.
Action (): Encodes hybrid keyboard-action inputs to govern local character actuation, providing responsive command dynamics.
Camera (): Embeds relative 6-DoF camera poses and Plücker coordinates to ensure continuous trajectory tracking and viewpoint stability.
To complement in-context conditioning, we further incorporates geometric transport into the temporal attention mechanism to preserve continuous physical transitions across frames.
the subsequent sections elaborate on the explicit view-mode formulation (§ 4.2), examine joint action-camera decoupling (§ 4.3), and introduce IT-RoPE for parameterized temporal attention (§ 4.4).
Explicit View-Mode Modeling
Most existing models [9, 31, 38, 40, 56] convey viewing regimes either through implicit text descriptions or directly via camera poses. However, identical geometric trajectories inherently entail different motion semantics across regimes, for example, distinguishing unconstrained FREE navigation from embodied FPP motion. Leaving such distinctions to implicit text conditioning makes it ambiguous to resolve regime-dependent motion characteristics from identical geometric inputs.
We therefore restrict strictly to world-state specifications and introduce explicit, time-aligned view tokens into the shared context:
By attending directly to visual latents, acts as a contextual switch that modulates the shared attention space toward regime-appropriate physical motion fields.
To validate the efficacy of explicit view conditioning, we compare our formulation against a baseline that specifies view modes through text alone (“w/o view modeling”). Figure 4 evaluates trajectory oscillations under FPP and FREE conditioning across 100 scenes, while Figure 3 presents qualitative transitions alongside quantitative evaluations on the WBench FPP↔TPP suite. Detailed results are provided in Section B.2; key observations are summarized:

Figure 3. Emergent view switching. (Left) Explicit view modeling enables switching with residual artifacts; fine-tuning on 500 switching clips (“500ft”) improves transitions. (Right) Success rates on WBench FPP↔TPP set [54].

Figure 4. View-dependent motion dynamics. Oscillation RMS across 100 scenes. Explicit view modeling distinguishes FPP oscillations from smoother FREE motion.
Obs.❶ Explicit view modeling distinguishes regime-dependent motion dynamics. As shown in Figure 4, text-based conditioning yields closely overlapping oscillation profiles for FPP and FREE, conflating their respective physical semantics. In contrast, explicit view modeling encourages a distinct separation: FPP exhibits pronounced periodic oscillations characteristic of embodied locomotion, while FREE preserves smoother camera trajectories. This indicates that explicit view tokens effectively mitigate motion ambiguity across viewing regimes.
Obs.❷ In-video view switching emerges naturally from explicit view tokens. Because view tokens act as modular contextual switches within the shared attention space, training exclusively on single-view clips already gives rise to perspective transitions simply by varying the view token sequence during rollout, as shown in Figure 3. On the WBench FPP↔TPP set, our base model without switching supervision achieves a 37.5% success rate, substantially outperforming the text-based baseline (6.25%). Incorporating a minimal set of 500 transition sequences further consolidates this emergent behavior to 93.8%, demonstrating that explicit view conditioning establishes a receptive interface that is readily steerable and sample-efficient to tune.
Joint Action-Camera Modeling
While existing interactive models typically rely on camera trajectories as the primary control signal [9, 38, 49] for all view modes, character actuation and camera viewpoint constitute separate degrees of freedom, particularly in TPP. Rather than mapping them to a single control interface [19] or routing them into disparate conditioning branches [36], we model character actions and camera trajectories as coexisting, decoupled conditions, contextualizing both directly within the shared transformer sequence.
Given an action sequence and a camera trajectory, World-in-Context encodes both modalities into time-aligned context tokens. For action conditioning, each input is decomposed into semantic subslots separating discrete key states from continuous motion increments. The corresponding action tokens are given by:
where , indexes the within-step tokens, denotes a slot-specific encoder (e.g., MLP), and is a zero-initialized residual branch for continuous magnitude cues.
For camera conditioning, World-in-Context converts each normalized relative pose into a tokenized Plücker ray field, following [11, 20]. For a sampled ray with camera center and unit direction , its Plücker coordinate is denoted by . The corresponding camera token is encoded as:
where denotes the local rigid-body increment, and the three projections encode ray geometry, global relative pose, and local camera motion, respectively.
Beyond their essential decoupling in TPP, we further investigate the less-examined FPP and FREE regimes, where camera paths alone are typically assumed to suffice. Figure 5 compares action-only, camera-only, and joint training under our setting on the WBench NAVIGATION set (excluding TPP scenes). Our key observations are:

Figure 5. Complementary action-camera control. (Top) Action-only training exhibits artifacts; camera-only training yields insufficient displacement. (Bottom) Joint training balances motion accuracy and consistency on WBench NAVIGATION (excluding TPP).
Obs.❸ Action and camera signals play complementary roles. As shown in Figure 5, action-only conditioning yields higher translation and rotation accuracy (+6.5 and +3.9 points over camera-only), but can introduce visual artifacts. Conversely, camera-only conditioning improves geometric (+2.1) and photometric (+1.9) consistency, though often at the cost of insufficient travel displacement. Joint training naturally harmonizes these properties, achieving the highest rotation accuracy (90.6) and geometric consistency (90.8) while maintaining strong translation fidelity (89.3). In effect, action commands govern local responsiveness, whereas camera trajectories anchor global stability.
Interaction-Transport Geometric Encoding
While in-context tokens expose control inputs to the shared attention space, standard 3D-RoPE encodes only relative spatiotemporal coordinates, leaving the intervening control progression between distant frames implicit. To this end, World-in-Context further introduces Interaction-Transport RoPE (IT-RoPE), which complements spatiotemporal positional encoding by parameterizing temporal attention with control-induced transport operators accumulated along the execution sequence.
Ordered Interval Transport. While input-dependent transforms [52] and geometric encoding [23] demonstrate the promise of data-driven attention biases, our interactive setting differs in that cross-temporal transitions are governed by temporally ordered user inputs. We thus maintain dual, order-preserving transport paths for character action and camera motion . Specifically, each control input induces a step transformation , which accumulates along continuous valid segments to define the relative transport across interval :
where , and . This noncommutative composition natively preserves control order, allowing the model to distinguish trajectories that share identical endpoints but differ in execution sequences.
View-Gated Centered Attention. To incorporate these accumulated transitions into cross-frame reasoning, we augment temporal attention with a centered, transport-corrected score:
where denotes standard spatiotemporal attention with 3D-RoPE, are control-specific sub-channels, and is a learnable scale initialized to zero. The gate dynamically modulates control influence across viewing regimes and token types, while restricts the correction to contiguous valid segments. Crucially, subtracting centers the correction, ensuring it vanishes for static intervals or , while zero-initializing preserves native backbone behavior at the start of training. Further details of transport accumulation and attention computation are provided in Section A.3.
Training of World-in-Context
Training Data. We train World-in-Context on a compact set of 100K video clips covering FRE, FPP, and TPP viewing regimes, accompanied by temporally aligned view, action, and camera annotations when available. While joint training across these single-regime clips naturally facilitates view-mode transitions (as analyzed in § 4.2), we additionally incorporate a small fraction of cross-view switching clips (500 sequences) with dynamic view schedules to reinforce transition stability.
Training Protocol. Following the conditional flow-matching objective, World-in-Context undergoes bidirectional finetuning where video and control tokens jointly interact in the shared attention manifold. This bidirectional setup serves as our default configuration to benchmark the core in-context conditioning paradigm. In § 6, we further investigate World-in-Context under broader operational settings, including causal rollout with in-context history and expanded game-control signals.
Experiments
Experimental Settings
Baselines. World-in-Context is built upon the widely adopted Wan2.2-TI2V-5B [41] backbone. For direct comparisons, our primary baselines comprise models sharing this foundation: YUME 1.5 [30], ABot-
World [19], DreamX-World [39], and Zing-0.5 [5]. We also include larger models as reference: EchoWM [55], LingBot-World v2 [9], Alaya-EVOKE [38], and HY-World 1.5 [18]. Since training data scales are rarely disclosed across works, we also implement public architectures and conduct controlled comparisons under our identical data for analyses.
Evaluations. We evaluate World-in-Context on the widely adopted WBench [54]. Alongside the standard NAVIGATION split for trajectory tracking, we further adopt its PERSPECTIVE SWITCHING - suite to systematically assess dynamic view transitions. Our framework natively supports both action-only and camera-only inference; we default to action-only control for FREE and FPP, while utilizing joint action and camera signals for character and viewpoint in TPP. More details are in Section B.
Main Results
Quantitative Results. Table 1 summarizes performance on WBench. Among 5B models, World-in-Context achieves the highest NAVIGATION
| Model | Size | Supported Controls | Conditioning Strategy | Quality | Setting | Interaction | Consistency | Physical | Average | Switching |
| EchoWM [55] | 22B | 🎮 | Positional Attn. | 81.5 | 79.4 | 86.6 | 89.8 | 70.6 | 81.6 | 32.0 |
| LingBot-World v2 [9] | 14B | 📷 | AdaLN | 81.8 | 76.8 | 82.8 | 86.5 | 69.1 | 79.4 | 21.9 |
| Alaya-EVOKE [53] | 14B | 📷 | Warp Condition | 82.8 | 83.8 | 78.6 | 86.9 | 72.1 | 80.8 | 18.8 |
| HY-World 1.5 [18] | 8.3B | 📷 | Modulat.+PRoPE | 78.1 | 72.2 | 86.8 | 86.9 | 66.3 | 78.1 | 12.5 |
| Wan 2.2 [41] | 5B | Text | 72.2 | 69.3 | 65.4 | 86.4 | 52.3 | 69.1 | 6.3 | |
| YUME 1.5 [30] | 5B | 📷 | Cross-Attn. | 77.6 | 72.4 | 71.4 | 80.1 | 65.2 | 73.3 | 12.5 |
| ABot-World [19] | 5B | 🎮📷 | Additive Inject. | 76.8 | 71.4 | 84.0 | 79.5 | 61.7 | 74.7 | 9.4 |
| DreamX-World [39] | 5B | 📷 | Positional Attn. | 77.5 | 80.8 | 78.6 | 74.9 | 63.3 | 75.0 | 18.8 |
| Zing-0.5 [5] | 5B | 🎮 | Additive Inject. | 80.6 | 77.8 | 84.2 | 88.5 | 73.8 | 81.0 | 12.5 |
| World-in-Context (Ours) | 5B | 🎮📷 | In-Context | 80.9 | 81.0 | 86.0 | 87.2 | 72.3 | 81.5 | 93.8 |
Table 1. Evaluation on WBench [54] NAVIGATION and FPP↔TPP sets. We highlight the best and second best.

Figure 8. Qualitative results of World-in-Context on FPP mode.
average (81.5), performing on par with Zing-0.5 (81.0) while improving the INTERACTION score from 84.2 to 86.0 via unified in-context conditioning. Its navigation capability also rivals larger models like 22B EchoWM (81.6). Crucially, World-in-Context reaches a 93.8% success rate on FPPTPP, significantly outperforming the best 5B baseline (18.8%) and confirming its capacity for dynamic viewpoint transitions within a single unified model.
Qualitative Results. Figure 6 further compares our World-in-Context with Zing-0.5. While both generate plausible environments, Zing-0.5 tends to incur exaggerated motion scale, whereas ours maintains more measured, balanced trajectories aligned with its higher INTERACTION score. Furthermore, World-in-Context smoothly executes FPPTPP switches where the baseline fails to transition. Extended rollouts across diverse scene layouts and artistic styles under FREE, FPP, and TPP modes are detailed in Figure 7 to Figure 9. Notably, Figure 10 also demonstrates coherent multi-stage switching (e.g., TPPFPPTPP) while preserving consistent scene structure throughout.

Figure 6. Qualitative comparisons of World-in-Context.

Figure 7. Qualitative results of World-in-Context on FREE mode.

Figure 9. Qualitative results of World-in-Context on TPP mode.

Figure 10. Qualitative results of World-in-Context on FPP ↔ TPP.
Analysis of In-Context Conditioning
To further examine how conditioning strategies scale with input complexity, we compare ICC with additive injection [19] and AdaLN modulation [40] under progressively richer inputs with our same data: view (V), action (A), and camera (C). We give the following observations:
Obs.4 ICC’s advantage scales with conditioning richness. As shown in Figure 11, with camera control alone, ICC performs on par with alternative strategies (76.3 vs. 76.3–77.0). Its gains emerge under joint action-camera inputs (+0.6 in NAVIGATION, +1.1 in INTERACTION) and expand further when adding explicit view tokens (+1.0 and +1.4, reaching 81.2 and 86.8). These widening margins indicate that a shared attention space coordinates diverse controls more effectively than separate injection pathways as interaction becomes compositional.

Figure 11. Conditioning strategy analysis. ICC increasingly benefits the coordination of diverse controls as conditioning becomes richer. C/A/V denote camera/action/view.
Analysis of IT-RoPE
To evaluate how trajectory-aware attention affects temporal consistency, we ablate IT-RoPE here against standard 3D-RoPE, single-branch variants, and an endpoint-only alternative under the fixed V+A+C setup, as shown in Figure 12.

Figure 12. IT-RoPE ablation. Accumulating intervening control transitions improves navigation and control fidelity beyond endpoint-only conditioning.
Obs.5 Tracking intervening control evolution improves interactive attention. Compared to 3D-RoPE, full IT-RoPE enhances both NAVIGATION (80.5 81.2) and INTERACTION (86.2 86.8). While action transport sharpens command responsiveness and camera transport stabilizes spatial tracking, their combination delivers the most balanced performance. Moreover, full IT-RoPE clearly outperforms the endpoint-only variant (80.5/86.6), confirming that accumulating the intervening control sequence provides critical dynamics missed by endpoint offsets alone.
Analysis of Data Efficiency
Our default interaction-training set comprises a modest 100K navigation clips and 500 switching sequences. Since training-data scales are not consistently disclosed by existing methods, we assess data efficiency through controlled comparisons in Figure 13: 50K+250, 100K+500, and 125K+750. We summarize our observations as follows:

Figure 13. Data scaling analysis. ICC consistently outperforms additive injection across training budgets.
Obs.6 ICC demonstrates data efficiency across scaling regimes. ICC consistently outperforms additive injection across all data regimes, especially in perspective switching. With only 50K+250 clips, ICC achieves a 75.0% switching rate, already surpassing additive injection trained on twice the data (71.9% at 100K+500). At 100K+500, ICC exceeds the largest-budget additive baseline (125K+750) in both NAVIGATION (81.5 vs. 80.3) and SWITCH (93.8% vs. 78.1%), showing that shared in-context tokens make substantially more efficient use of interactive supervision.
Applicability of World-in-Context
Causal Modeling with In-Context History
To examine the architectural extensibility of in-context conditioning beyond bidirectional generation, World-in-Context is naturally adapted to streaming causal rollouts. Following common causal training paradigms in interactive generation [60], the model is trained autoregressively to predict future frame latents conditioned on historical context. Rather than relying on external KV cache retrieval mechanisms, WiC-Causal directly incorporates historical latents into the shared self-attention sequence. Specifically, the condition sequence in Eq. (4) is augmented with historical latent tokens:
where preserves initial context tokens as global semantic anchors, and maintains a sliding window of recent visual tokens to enforce continuity.
As shown in Table 2, WiC-Causal exhibits competitive interactive performance, attaining an average NAVIGATION score of 81.2 and a 90.6% FPP↔TPP success rate on WBench. Compared to the bidirectional counterpart (81.5 and 93.8%), causal modeling incurs negligible degradation while retaining a better performance margin over existing 5B baselines. Furthermore, qualitative rollouts in Figure 15 demonstrate that WiC-Causal preserves long-horizon spatial coherence across diverse viewing regimes. Notably, in loop-closure and viewpoint reappearance scenarios (e.g., returning to previously explored landscapes or room corners), WiC-Causal consistently reproduces previously observed visual landmarks and layouts.
| Method | Qual. | Set. | Inter. | Consi. | Phys. | Avg. | Switch. |
| Wan 2.2 [41] | 72.2 | 69.3 | 65.4 | 86.4 | 52.3 | 69.1 | 6.3 |
| YUME 1.5 [30] | 77.6 | 72.4 | 71.4 | 80.1 | 65.2 | 73.3 | 12.5 |
| ABot-World [19] | 76.8 | 71.4 | 84.0 | 79.5 | 61.7 | 74.7 | 9.4 |
| DreamX-World [39] | 77.5 | 80.8 | 78.6 | 74.9 | 63.3 | 75.0 | 18.8 |
| Zing-0.5 [5] | 80.6 | 77.8 | 84.2 | 88.5 | 73.8 | 81.0 | 12.5 |
| World-in-Context | 80.9 | 81.0 | 86.0 | 87.2 | 72.3 | 81.5 | 93.8 |
| WiC-Causal | 80.5 | 80.4 | 85.7 | 87.6 | 72.0 | 81.2 | 90.6 |
Table 2. Evaluation of WiC-Causal on WBench.

Figure 15. Qualitative results of WiC-Causal.
Scaling to Rich Game Controls
To further demonstrate the extensibility and token-level composability beyond canonical navigation commands, we extend World-in-Context to accommodate richer, high-frequency gameplay actions. Experiments are conducted on Counter-Strike 2 gameplay sequences from EWM [29]. To assess sample efficiency and generalizability, WiC-Game is trained on a compact subset of a single map (i.e., DUST II), and tested on unseen layouts as well as real-world scenarios. Specifically, the condition sequence is augmented with discrete game-control tokens:
corresponding to firing (weapon dynamics and muzzle flashes), crouching (viewport descent), and slow walking (motion attenuation), respectively.
As illustrated in Figure 14, WiC-Game demonstrates sensitive, decoupled control execution across three complementary regimes: (i) In-domain: new control trajectories within the training map (DUST II) exhibit instant actuation responses; (ii) Out-of-distribution: when applied to unseen game maps (e.g., MIRAGE, NUKE), the model preserves scene consistency while faithfully following concurrent actions; (iii) Real-world transfer: using synthetic starting images, the learned actuation dynamics generalize naturally to realistic visual environments.

Figure 14. Qualitative results of WiC-Game.
Conclusion and Positioning
Conclusion. In this work, we present World-in-Context, a video world modeling framework based on In-Context Conditioning (ICC) that unifies view mode, character action, and camera trajectory as tokens within a shared attention space. We systematically investigate core interactive dimensions, revealing that explicit view tokens disentangle conflicting motion dynamics while enabling emergent in-video switching, and that action and camera signals complement each other in local responsiveness
and global stability. Augmented with IT-RoPE to encode accumulated control progression in temporal attention, World-in-Context achieves competitive navigation on WBench (81.5) matching the 22B EchoWM, while establishing strong perspective-switching capability (93.8%). Controlled analyses further demonstrate compounding advantages under richer conditioning, favorable sample efficiency, and natural applicability to causal rollouts, highlighting shared context modeling as a scalable interface for interactive world simulation.
Positioning of This Report. This work focuses primarily on the algorithmic formulation of interactive conditioning, rather than training an all-encompassing system artifact, with World-in-Context serving as a concrete instantiation of this design. The overall capacity of an interactive world model naturally depends on data quality, model size, and training scale. We use a modest interaction-training set of 100K navigation clips and 500 switching sequences. Since comparable training-data statistics are not consistently disclosed by existing methods, we omit cross-method data-scale comparisons from the main table. Instead, we prioritize controlled comparisons that isolate the architectural advantages of shared context modeling and control-aware temporal attention, including their effectiveness across training-data budgets. Specifically, this work centers around three foundational control modalities, i.e., view mode, character action, and camera motion, as they directly address the core dimensions governing interactive navigation: how the environment is viewed across perspectives and how user inputs steer character actuation alongside viewport evolution. These algorithmic findings complement concurrent advances in scaling, presenting an extensible interface for coordinating diverse interactive controls.
Contributions
Contributors. Harold Haodong Chen*, Rui Peng†, Rongjin Guo, Lingteng Qiu, Kejie Qiu, Chao Xu, Zilong Dong
†Corresponding Author
*Work done during Harold H. Chen’s internship at ATH@Alibaba.
For questions or feedback, feel free to contact [email protected] or open a GitHub issue.
References
Training Details of World-in-Context
Training Data
Data Sources. The training mixture comprises a default collection of 100K single-regime navigation clips alongside a subset of 500 FPP↔TPP switching sequences. All qualitative evaluation scenes presented in this work are strictly held out from the training set.
Navigation Data (100K): Clips are collected across widely-adapted open-source video datasets, including DL3DV [27], RealEstate [58], Sekai [25], SpatialViD [42], EgoExo [34], and ABot-World-Explorer [19].
Switching Data (500): Viewpoint-transition sequences are collected from EgoExo [34] and internal data.
Quality Pipeline. The raw video corpus is filtered through a standard three-stage automated filtering pipeline:
Scene/Shot Continuity: Following LingBot-World [40], TransNetV2 [35] is deployed to detect abrupt scene transitions and shot boundaries; segmented multi-shot videos are removed to retain strictly continuous navigation segments.
Motion Filtering: Following Open-Sora [57] and SANA-Video [4], UniMatch [51] optical flow and VMAF metrics are evaluated to quantify dynamic motion magnitude. Clips displaying static stagnation or excessive motion blur/discontinuities are discarded.
Aesthetic Assessment: Dover [48] is utilized to score perceptual aesthetics, eliminating clips compromised by severe compression artifacts, overexposure, or low visual fidelity.
Control Annotation. The control signals mainly used in World-in-Context, i.e., view, camera, and action, are annotated as follows:
View Mode: A vision-language foundation model (i.e., Qwen3-VL-32B [1]) classifies clips into FREE, FPP, and TPP regimes. FREE is characterized by unconstrained, smooth aerial or camera roaming (e.g., drone cinematography); FPP captures embodied locomotion or in-cockpit perspectives; TPP encompasses external character-following camera arrangements. Ambiguous instances at boundaries are cross-verified and resolved using reconstructed camera trajectory statistics.
Camera Trajectories: Following common practice, continuous 6-DoF camera trajectories and Plücker ray parameters are reconstructed using ViPE [14].
Action Inversion: While authentic character actuation natively originates from game-engine telemetry, obtaining massive synchronized game recordings remains challenging. Therefore, discrete character actions are mainly inversely resolved from ViPE-estimated continuous camera trajectories. Crucially, for FPP sequences, body-induced periodic oscillations and head-bobbing motions are decoupled and filtered out before discrete action discretization.
Conditioning Schema
World-State Captioning. Unlike existing world-modeling frameworks (e.g., LingBot-World [40], Matrix-Game 3.0 [45]) that synthesize verbose temporal narratives or dense temporal captions, World-in-Context adopts a simplified world-state captioning formulation. Under the premise that generative world models should decouple scene constitution from action-driven exploration, captions in this pipeline exclusively summarize the static visual constitution of the environment (e.g., atmospheric conditions, spatial geometry, and layout entities). All temporal progression, dynamic steering, and viewport navigation are delegated entirely to the interactive control tokens.
| DL3DV | SpatialViD | ABot | EgoExo | Sekai | RealEstate |
| 29.22% | 27.5% | 25.24% | 8.76% | 5.45% | 3.83% |
Table 3. Navigation data composition distribution.
Data Distribution. The distribution of finally adopted navigation data sources is illustrated in Table 3.
Two-Stage Training Curriculum. As outlined in § 4.5, training proceeds in two sequential phases (Figure 16):

Figure 16. View distribution across two stages.
① Stage-1 (General Navigation): World-in-Context is trained on 100K single-regime navigation clips to learn fundamental physical dynamics and interactive responsiveness across FREE, FPP, and TPP modes.
@page 19/22 starts_mid=0 ends_mid=1
Training Details of World-in-Context
Training Data
Data Sources. The training mixture comprises a default collection of 100K single-regime navigation clips alongside a subset of 500 FPP↔TPP switching sequences. All qualitative evaluation scenes presented in this work are strictly held out from the training set.
Navigation Data (100K): Clips are collected across widely-adapted open-source video datasets, including DL3DV [27], RealEstate [58], Sekai [25], SpatialViD [42], EgoExo [34], and ABot-World-Explorer [19].
Switching Data (500): Viewpoint-transition sequences are collected from EgoExo [34] and internal data.
Quality Pipeline. The raw video corpus is filtered through a standard three-stage automated filtering pipeline:
Scene/Shot Continuity: Following LingBot-World [40], TransNetV2 [35] is deployed to detect abrupt scene transitions and shot boundaries; segmented multi-shot videos are removed to retain strictly continuous navigation segments.
Motion Filtering: Following Open-Sora [57] and SANA-Video [4], UniMatch [51] optical flow and VMAF metrics are evaluated to quantify dynamic motion magnitude. Clips displaying static stagnation or excessive motion blur/discontinuities are discarded.
Aesthetic Assessment: Dover [48] is utilized to score perceptual aesthetics, eliminating clips compromised by severe compression artifacts, overexposure, or low visual fidelity.
Control Annotation. The control signals mainly used in World-in-Context, i.e., view, camera, and action, are annotated as follows:
View Mode: A vision-language foundation model (i.e., Qwen3-VL-32B [1]) classifies clips into FREE, FPP, and TPP regimes. FREE is characterized by unconstrained, smooth aerial or camera roaming (e.g., drone cinematography); FPP captures embodied locomotion or in-cockpit perspectives; TPP encompasses external character-following camera arrangements. Ambiguous instances at boundaries are cross-verified and resolved using reconstructed camera trajectory statistics.
Camera Trajectories: Following common practice, continuous 6-DoF camera trajectories and Plücker ray parameters are reconstructed using ViPE [14].
Action Inversion: While authentic character actuation natively originates from game-engine telemetry, obtaining massive synchronized game recordings remains challenging. Therefore, discrete character actions are mainly inversely resolved from ViPE-estimated continuous camera trajectories. Crucially, for FPP sequences, body-induced periodic oscillations and head-bobbing motions are decoupled and filtered out before discrete action discretization.
Conditioning Schema
World-State Captioning. Unlike existing world-modeling frameworks (e.g., LingBot-World [40], Matrix-Game 3.0 [45]) that synthesize verbose temporal narratives or dense temporal captions, World-in-Context adopts a simplified world-state captioning formulation. Under the premise that generative world models should decouple scene constitution from action-driven exploration, captions in this pipeline exclusively summarize the static visual constitution of the environment (e.g., atmospheric conditions, spatial geometry, and layout entities). All temporal progression, dynamic steering, and viewport navigation are delegated entirely to the interactive control tokens.
| DL3DV | SpatialViD | ABot | EgoExo | Sekai | RealEstate |
| 29.22% | 27.5% | 25.24% | 8.76% | 5.45% | 3.83% |
Table 3 (PDF p. 19). Navigation data composition distribution.
Data Distribution. The distribution of finally adopted navigation data sources is illustrated in Table 3.
Two-Stage Training Curriculum. As outlined in § 4.5, training proceeds in two sequential phases (Figure 16):

Figure 16 (PDF p. 19). View distribution across two stages.
① Stage-1 (General Navigation): World-in-Context is trained on 100K single-regime navigation clips to learn fundamental physical dynamics and interactive responsiveness across FREE, FPP, and TPP modes.
➁ Stage-2 (Switching Consolidation): Rather than fine-tuning solely on switching data, World-in-Context constructs an 8K curriculum by mixing regular navigation clips with the 500 switching sequences, where switching clips are oversampled to reinforce transition stability while preserving base navigation capabilities.
Mixed Action-Camera Conditioning. In § 4.3, “joint training” refers precisely to the composite conditioning mixture illustrated in Figure 17, rather than strictly enforcing concurrent dual signals on every training instance. In real-world data, simultaneously obtaining robust annotations for both modalities is often impractical; for instance, loose camera-following in TPP sequences complicates clean action discretization, and synthetic scenes can introduce camera pose estimation artifacts. Therefore, World-in-Context trains across a composite conditioning mixture of action-only, camera-only, joint, and unconditioned samples.

Figure 17. Action/camera distribution across two stages.
Distribution Transparency. The mixture schedules in Figure 16 and Figure 17 are reported for empirical transparency rather than claimed as an optimal configuration, reflecting practical trade-offs against data availability. Crucially, all comparative baselines (e.g., additive injection and AdaLN modulation in § 5) follow this identical conditioning schema to guarantee strictly controlled architectural comparisons.
More Details of IT-RoPE
In this section, we further detail additional mathematical formulations, implementation properties, and efficient computation of Interaction-Transport RoPE (IT-RoPE).
Transport Accumulation and Prefix Formulation. IT-RoPE maintains dual, decoupled transport paths for character action and camera motion (), aligned with the latent temporal grid. Within each continuous valid segment anchored at initial time , the orthogonal step transformations accumulate into prefix operators:
which recovers the ordered interval transport () defined in Eq. (9) and supports both forward and backward temporal relations. Because the step transformations need not commute, the accumulated transport retains order-dependent information about the execution sequence rather than reflecting only endpoint states.
Gating, Centering, and Initialization. The two transport corrections act additively on the base 3D-RoPE attention scores. Specifically, gates conditioned on view mode and token type dynamically adjust the relative influence of each control branch, while interval masks restrict corrections to token pairs within contiguous valid execution segments. Centering the transport operator by subtracting the identity matrix ensures that the correction vanishes whenever . This holds for same-timestep pairs () via the prefix formulation, as well as for static intervals where the control parameterization yields identity step transformations (). Furthermore, zero-initializing the branch scale factors preserves the base attention scores at initialization.
Factorized Attention Computation. By exploiting the orthogonality of the prefix operators , the centered inner product expands into:
Within each continuous valid segment, the correction decomposes into the difference between prefix-transformed and untransformed query-key dot products. This factorization avoids explicitly constructing pairwise transformation matrices for all frame intervals, allowing the prefix accumulation and corresponding query-key transforms to scale linearly with the temporal length and token count.
Experimental Details
Evaluation Benchmark
We evaluate World-in-Context on WBench [54], a benchmark for interactive video world models comprising 289 cases and 1,058 interaction turns. Each case provides an initial image, a world description, and multi-turn control sequences. We use its NAVIGATION split and FPP↔TPP perspective-switching subset.
Navigation Suite. WBench benchmarks trajectory-following navigation across five core dimensions:
Quality: Assesses perceptual aesthetics, visual fidelity, motion dynamics, temporal flickering, and inter-frame motion smoothness.
Setting: Measures contextual adherence to the specified prompt, including scene appearance preservation, environmental style consistency, and subject persistence.
Interaction: Evaluates motion responsiveness and command fidelity by comparing estimated camera trajectories with action-derived reference trajectories under translational, rotational, and symmetric steering commands.
Consistency: Quantifies structural and geometric coherence, capturing character identity stability, absence of shot cuts, and photometric/geometric consistency across viewpoint transitions.
Physical: Assesses adherence to real-world physics, commonsense dynamics, and causal coherence throughout interactive rollouts.
In addition to individual dimension scores, the overall NAVIGATION average is reported.
Perspective Switching. For each requested FPP↔TPP transition, WBench evaluates the visual rollout to determine whether viewpoint transition smoothness, terminal perspective, and framing geometry satisfy the target specifications. Binary success decisions are aggregated across all evaluation turns and reported as an independent metric to isolate dynamic viewpoint adaptability from trajectory tracking.
All evaluations follow the official WBench protocol, employing its standardized scoring functions, decision thresholds, and aggregation rules without modification.
Setup of Analysis
This section details the experimental configurations, evaluation datasets, and diagnostic metrics employed in the analyses of § 4.
View-Dependent Motion Dynamics (Figure 4). To isolate regime-dependent motion characteristics, 100 first-person navigation scenarios are randomly sampled from WBench. For each scene, the world-state caption is held static, and the action command sequence is fixed uniformly to continuous forward movement (W). The model is then prompted under contrasting view modes (FREE vs. FPP) to observe the resulting physical trajectories.
Camera trajectories are reconstructed using ViPE [14]. Positions are expressed relative to the initial frame coordinates , with representing transverse (lateral and vertical) coordinates. The smooth trajectory trend is isolated via a second-order Savitzky-Golay filter with a 33-frame window, yielding the transverse residual:
The oscillation amplitude is quantified via normalized Root Mean Square error (RMS):
where denotes the net forward displacement, and is the interior interval excluding 0.5 seconds at sequence boundaries. This metric captures transverse body-sway and vertical bobbing while remaining invariant to global forward drift.
Emergent View Switching (Figure 3). The quantitative validation evaluates viewpoint transition fidelity on the official WBench FPP↔TPP split, adhering strictly to the VLM-based transition criteria and aggregation protocol described in Section B.1.
Action vs. Camera Control Disentanglement (Figure 5). The comparative analysis of action and camera control is conducted on the WBench NAVIGATION suite, with third-person (TPP) instances excluded to isolate egocentric/free dynamics. Evaluation spans four metrics: two local command-fidelity diagnostics and two global consistency measures.
Translation Accuracy: Turn sequences are segmented into non-overlapping 0.5-second windows . Camera poses are tracked via MegaSam [24]. For each window, the generated local displacement is compared against the action-integrated reference . A window is classified as accurate if the directional deviation satisfies and the displacement exceeds a minimum motion threshold , where is the median initial positive depth and is the window duration.
Rotation Accuracy: Using identical temporal windows, relative rotation vectors and references are extracted in . A window is correct if the angular orientation deviation is within and the rotational magnitude satisfies . Accuracy scores are averaged over valid active windows and scaled to a percentage.
Geometric Consistency: Following WBench, Depth Anything 3 [26] provides predicted depths , intrinsics , and poses . Source pixels are back-projected and reprojected into target frame : , measuring relative depth disparity . The consistency score is computed as .
Photometric Consistency: Following WBench, colors from source frames are splatted into target frame via depth ordering. Denoting as the splatted image and as valid pixels, the average PSNR is evaluated across frames with . Photo-
metric consistency is aggregated as .
Setup of Analysis in sec:experiments
Additive injection and AdaLN modulation are reimplemented following the conditioning strategies of ABot-World [19] and LingBot-World [9], respectively. For explicit view conditioning in these baselines, the categorical view mode is embedded at each latent timestep, spatially broadcast, and fused with the available action and camera representations. The resulting composite features are added directly to the visual patch tokens for additive injection, or projected to predict affine modulation parameters for AdaLN. In § 5.3 and § 5.4, the default World-in-Context is evaluated prior to stage-2 switching consolidation to demonstrate the core architectural advantages of in-context conditioning and transport modeling.
References
- [1]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. 1
- [2]Yuxuan Bian, Xin Chen, Zenan Li, Tiancheng Zhi, Shen Sang, Linjie Luo, and Qiang Xu. Video-as-prompt: Unified semantic control for video generation. In International Conference on Learning Representations, pages 114558–114583, 2026. 3arxiv.org/abs/2510.20888
- [3]Harold Haodong Chen, Rongjin Guo, Disen Lan, Wen-Jie Shu, Hongfei Zhang, Hanzhe Hu, Shengtao Yao, Zixin Zhang, Guibin Zhang, Zhefan Rao, et al. The past frames the future: Memory for autoregressive video generation. arXiv preprint arXiv:2609.028466, 2026. 2
- [4]Junsong Cheng, Yuyang Zhao, Jincheng Yu, Ruihang Chu, Junyu Chen, Shuai Yang, Xianbang Wang, Yicheng Pan, Zhou Daquan, Huan Ling, et al. Sana-video: Efficient video generation with block linear diffusion transformer. In International Conference on Learning Representations, pages 38973–38997, 2026. 1
- [5]Mingyang Chen, Shendong Chen, Xiaoxiao Fu, Bosheng Gong, Haoyuan Guo, Bowen Li, Jiawen Li, Kejun Li, Tianpeng Li, Yin Liu, et al. Zing-0.5: Toward playable worlds with real-time joint action and text control. arXiv preprint arXiv:2609.17909, 2026. 3, 6, 7, 13arxiv.org/abs/2609.17909
- [6]Etched Decart, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen. Oasis: A universe in a transformer. URL: https://oasis-model. github. io, 2(3):6, 2024. 2
- [7]Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, et al. A survey on in-context learning. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 1107–1128, 2024. 3
- [8]Zhengcong Fei, Di Qiu, Debang Li, Changqian Yu, and Mingyuan Fan. Video diffusion transformers are in-context learners. arXiv preprint arXiv:2412.10783, 2024. 3arxiv.org/abs/2412.10783
- [9]Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, Jiahao Wang, Yufeng Yuan, Hanlin Wang, Yicheng Lou, et al. Infinite worlds with versatile interactions. arXiv preprint arXiv:2607.07534, 2026. 1, 2, 3, 4, 5, 6, 7arxiv.org/abs/2607.07534
- [10]Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103, 2024. 1
- [11]Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for video diffusion models. In The Thirty-ninth International Conference on Learning Representations, 2025. 5arxiv.org/abs/2503.10592
- [12]Xuanhua He, Quande Liu, Zixuan Ye, Weicai Ye, Qilun Wang, Xintao Wang, Qifeng Chen, Pengfei Wan, Di Zhang, and Kun Gai. Fulldit2: Efficient in-context conditioning for video diffusion transformers. arXiv preprint arXiv:2506.04213, 2025. 3arxiv.org/abs/2506.04213
- [13][13*] Hongrui Huang, Junke Wang, Quanhao Li, Yu-Gang Jiang, and Zuxuan Wu. Disco: World models with discrete camera motion control. arXiv preprint arXiv:2606.07967, 2026. 2, 3arxiv.org/abs/2606.07967
- [14]Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepchev, Chen-Hsuan Lin, et al. Vipe: Video pose engine for 3d geometric perception. arXiv preprint arXiv:2508.10934, 2025. 1, 3arxiv.org/abs/2508.10934
- [15]Junchao Huang, Guijian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, et al. Solarm: Open data and scalable training for long-horizon world models. arXiv preprint arXiv:2609.02886, 2026. 2arxiv.org/abs/2609.02886
- [16]Lianghua Huang, Wei Wang, Zhi-Fan Wu, Huanzhang Dou, Yupeng Shi, Yutong Feng, Chen Liang, Yu Liu, and Jingren Zhou. Group diffusion transformers are unsupervised multitask learners. 2024. 3arxiv.org/abs/2410.15027
- [17]Lianghua Huang, Wei Wang, Zhi-Fan Wu, Yupeng Shi, Huanzhang Dou, Chen Liang, Yutong Feng, Yu Liu, and Jingren Zhou. In-context lora for diffusion transformers. arXiv preprint arXiv:2410.23775, 2024. 3arxiv.org/abs/2410.23775
- [18]Team HunyuanWorld. Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency. arXiv preprint, 2025. 3, 6, 7
- [19]Fan Jiang, Zhaoxu Sun, Mengchao Wang, Ziyu Zhu, Chiyu Wang, Yunpeng Zhang, Wenlin Liu, Yun Wang, Xue Zheng, Rui Sun, et al. Abot-world-0: Infinite interactive world rollout on a single desktop gpu. arXiv preprint arXiv:2607.19191, 2026. 2, 3, 5, 6, 7, 12, 13, 1, 4arxiv.org/abs/2607.19191
- [20]Xuan Ju, Weicai Ye, Quande Liu, Qiulin Wang, Xintao Wang, Pengfei Wan, Di Zhang, Kun Gai, and Qiang Xu. Fulldit: Multi-task video generative foundation model with full attention. arXiv preprint arXiv:2503.19907, 2025. 3, 5arxiv.org/abs/2503.19907
- [21]Kinam Kim, Junho Hyung, and Jaegul Choo. Temporal in-context fine-tuning with temporal reasoning for versatile control of video diffusion models. arXiv preprint arXiv:2506.00996, 2025. 3arxiv.org/abs/2506.00996
- [22]Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 1arxiv.org/abs/2412.03603
- [23]Ruilong Li, Brent Yi, Junchen Liu, Hang Gao, Yi Ma, and Angjoo Kanazawa. Cameras as relative positional encoding. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 6arxiv.org/abs/2507.10496
- [24]Zhengqi Li, Richard Tucker, Forrester Cole, Qianqian Wang, Linyi Jin, Vickie Ye, Angjoo Kanazawa, Aleksander Holynski, and Noah Snavely. Megasam: Accurate, fast, and robust structure and motion from casual dynamic videos. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10486–10496. IEEE, 2025. 3arxiv.org/abs/2412.04463
- [25]Zhen Li, Chuanhao Li, Xiaofeng Mao, Shaoheng Lin, Ming Li, Shitian Zhao, Zhaopan Xu, Xinyue Li, Yukang Feng, Jianwen Sun, et al. Sekai: A video dataset towards world exploration. Advances in Neural Information Processing Systems, 38, 2026. 1arxiv.org/abs/2506.15675
- [26]Haotong Lin, Sili Chen, Junhao Liew, Donny Y Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647, 2025. 3arxiv.org/abs/2511.10647
- [27]Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zuxin Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22160–22169. IEEE, 2024. 1arxiv.org/abs/2312.16256
- [28]Jiuming Liu, Chaojun Ni, Mengmeng Liu, Chensheng Peng, Fangjinhua Wang, Sitian Shen, Marc Pollefeys, Masayoshi Tomizuka, Ayush Tewari, and Per Ola Kris-tensson. Towards interactive video world modeling: Frontiers, challenges, benchmarks, and future trends. arXiv preprint arXiv:2606.01164, 2026. 2arxiv.org/abs/2606.01164
- [29]Weiyu Ma, Zeyuan Gong, Xuhui Liu, Wenxuan Zhang, Jian Ding, Xinyu Cui, Jian Zhao, and Mohamed El-hoseiny. Espresso world model dataset. arXiv preprint arXiv:2610.16166, 2026. 14arxiv.org/abs/2610.16166
- [30]Xiaofeng Mao, Zhen Li, Chuanhao Li, Xiaojie Xu, Kaining Ying, and Kaipeng Zhang. Yume1. 5: A text-controlled interactive world generation model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7752–7761, 2026. 3, 6, 13
- [31]Jisu Nam, Yicong Hong, Chun-Hao Paul Huang, Feng Liu, JoungBin Lee, Jiyoung Kim, Siyoon Jin, Yunsung Lee, Jaeyoon Jung, Suhwan Choi, et al. Worldcam: Interactive autoregressive 3d gaming worlds with camera pose as a unifying geometric representation. arXiv preprint arXiv:2603.16871, 2026. 2, 3, 4arxiv.org/abs/2603.16871
- [32]Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavhari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, Vlad Mnih, Harris Chan, Maxime Gazeau, Bonnie Li, Fabio Pardo, Luyu Wang, Lei Zhang, Frederic Besse, Tim Harley, Anna Mitenkova, Jane Wang, Jeff Clune, Demis Hassabis, Raia Hadsell, Adrian Bolton, Satinder Singh, and Tim Rocktäschel. Genie 2: A large-scale foundation world model. 2024. 2
- [33]Runjia Qian, Zile Wang, Jihai Zhang, Kai Zou, Wei Yu, Jiaxing Li, Zexiang Liu, Yaokun Li, Fei Kang, Kaichen Huang, et al. Matrix-game 3.5: Enhancing real-time streaming interactive world models with patch memory. arXiv preprint arXiv:2608.29910, 2026. 2arxiv.org/abs/2608.29910
- [34]Qianjian Song, Yiren Song, Kelly Peng, Yuan Gao, and Mike Zheng Shou. Worldrender: Bridging egocentric and exocentric worlds in video generation. In European Conference on Computer Vision, pages 612–630. Springer, 2026. 1
- [35]Tomáš Soucek and Jakub Lokoc. Transnet v2: An effective deep network architecture for fast shot transition detection. In Proceedings of the 32nd ACM international conference on multimedia, pages 11218–11221, 2024. 1arxiv.org/abs/2008.04838
- [36]Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. World-play: Towards long-term geometric consistency for real-time interactive world modeling. arXiv preprint arXiv:2512.14614, 2025. 2, 3, 5arxiv.org/abs/2512.14614
- [37]Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Omnicontrol: Minimal and universal control for diffusion transformer. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 14940–14950. IEEE, 2025. 3arxiv.org/abs/2411.15098
- [38]AlayaWorld Team, Kaipeng Zhang, Chuanhao Li, Yifan Zhan, Yongtao Ge, Yuanyang Yin, Jiamtang Tan, Kang He, Liaoyuan Fan, Mingliang Zhai, et al. Alaya-world: Interactive long-horizon world modeling—full technical report. arXiv preprint arXiv:2607.18367, 2026. 1, 2, 4, 5, 7
- [39]DreamX Team, Yancheng Bai, Rui Chen, Xiangxiang Chu, Rujing Dang, Hao Dou, Bingjie Gao, Qiwen Gu, Siyu Hong, Jiachen Lei, et al. Dreamx-world 1.0: A general-purpose interactive world model. arXiv preprint arXiv:2606.16993, 2026. 2, 3, 6, 7, 13arxiv.org/abs/2606.16993
- [40]Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, Yixuan Li, Hanlin Wang, Yinghao Xu, Shuailei Ma, et al. Advancing open-source world models. arXiv preprint arXiv:2601.20540, 2026. 1, 2, 4, 12
- [41]Team Wang, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 1, 6, 13arxiv.org/abs/2503.20314
- [42]Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Chang Zeng, Yanxi Zhou, Xiao-Xiao Long, et al. Spatialvid: A large-scale video dataset with spatial annotations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 42592–42603, 2026. 1arxiv.org/abs/2509.09676
- [43]Luozhou Wang, Zhifei Chen, Yihua Du, Dongyu Yan, Wenhang Ge, Guibao Shen, Xinli Xu, Leyi Wu, Man Chen, Tianshuo Xu, et al. A mechanistic view on video generation as world models: State and dynamics. arXiv preprint arXiv:2601.17067, 2026. 2arxiv.org/abs/2601.17067
- [44]Zhendong Wang, Yifan Jiang, Yadong Lu, Pengcheng He, Weihzu Chen, Zhangyang Wang, Mingyuan Zhou, et al. In-context learning unlocked for diffusion models. Advances in Neural Information Processing Systems, 36:854–8562, 2023. 3arxiv.org/abs/2305.01115
- [45]Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, Peiyu Wang, Biao Jiang, Yichen Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory. arXiv preprint arXiv:2604.08995, 2026. 1, 2arxiv.org/abs/2604.08995
- [46]Chenyuan Wu, Pengfei Zheng, Ruiran Xuan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, et al. Omnigen2: Exploration to advanced multimodal generation. arXiv e-prints, pages arXiv–2506, 2025. 3
- [47]Fan Wu, Jiacheng Wei, Ruibo Li, Yi Xu, Junyou Li, Deheng Ye, and Guosheng Lin. IC-world: In-context generation for shared world modeling. In European Conference on Computer Vision, pages 483–501. Springer, 2026. 3arxiv.org/abs/2512.02793
- [48]Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 20087–20097. IEEE, 2023. 1arxiv.org/abs/2211.04894
- [49]Ruiqi Wu, Xuanhua He, Meng Cheng, Tianyu Yang, Yong Zhang, Zhuoliang Kang, Xunliang Cai, Xiaoming Wei, Chunle Guo, Chongyi Li, et al. Infinite-world: Scaling interactive world models to 1000-frame horizons via pose-free hierarchical memory. arXiv preprint arXiv:2602.02393, 2026. 5
- [50]Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13294–13304. IEEE, 2025. 3arxiv.org/abs/2409.11340
- [51]Lihe Yang, Zhen Zhao, and Hengshuang Zhao. Unimatch v2: Pushing the limit of semi-supervised semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):3031–3048, 2025. 1arxiv.org/abs/2410.10777
- [52]Songlin Yang, Yikang Shen, Kaiyue Wen, Shawn Tan, Mayank Mishra, Liliang Ren, Rameswar Panda, and Yoon Kim. PaTH attention: Position encoding via accumulating household transformations. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. 6arxiv.org/abs/2505.16381
- [53]Yuanyang Yin, Gongxuan Wang, Yifan Zhan, Chuanhao Li, Kaipeng Zhang, and Feng Zhao. Alaya-evoke: From linear-scaling supervision to endless world. arXiv preprint arXiv:2608.13546, 2026. 3, 6arxiv.org/abs/2608.13546
- [54]Kaining Ying, Hengrui He, Suyu Ren, Jiamu Li, Fengjiao Chen, Ziwen Wang, Xuezhi Cao, Xunliang Cai, and Henghui Ding. Wbench: A comprehensive multi-turn benchmark for interactive video world model evaluation. arXiv preprint arXiv:2605.25874, 2026. 4, 6, 7, 2arxiv.org/abs/2605.25874
- [55]Songchun Zhang, Yaowei Li, Junhao Zhuang, Weiyang Jin, Haoyu Wang, Xin Lu, Yilang Sun, Shiyi Zhang, Haoran Li, Xiaoxiao Ma, et al. Echowm: Open and enterable omnmodial world models. arXiv preprint arXiv:2608.23189, 2026. 2, 3, 6, 7arxiv.org/abs/2608.23189
- [56]Min Zhao, Hongzhou Zhu, Bokai Yan, Zihan Zhou, Yimin Chen, Wenqiang Sun, Kaiwen Zheng, Guande He, Xiao Yang, Chongxuan Li, et al. minwm: A full-stack open-source framework for real-time interactive video world models. arXiv preprint arXiv:2605.30263, 2026. 2, 4arxiv.org/abs/2605.30263
- [57]Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 1arxiv.org/abs/2412.20404
- [58]Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view synthesis using multiplane images. arXiv preprint arXiv:1805.09817, 2018. 1arxiv.org/abs/1805.09817
- [59]Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, and Enze Xie. Sana-wm: Efficient minute-scale world modeling with hybrid linear diffusion transformer. arXiv preprint arXiv:2605.15178, 2026. 2, 3arxiv.org/abs/2605.15178
- [60]Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, and Jun Zhu. Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation. arXiv preprint arXiv:2602.02214, 2026. 13arxiv.org/abs/2602.02214