WorksComputer Science
Wizard: Manipulate Any World in Real Time via Agentic Control
Abstract
Recent advances in video world models have enabled increasingly realistic and interactive virtual environments, opening new possibilities for player-customizable gameplay. Agentic frameworks further advance this direction by decoupling explicit game logic and controllable world representations from visual generation. However, existing approaches support only simple game settings and predefined motion patterns, generalize poorly to unseen environments and characters, and offer limited real-time interactivity. In this paper, we introduce Wizard, an agentic game world model that enables real-time, precise control over character motions while generalizing zero-shot across novel scenes, characters, and actions. To achieve this, we construct a diverse dataset spanning 805 scenes and 178 characters, incorporating skeleton and geometry information as structured proxies to condition video generation. Our agentic framework explicitly tracks and updates game states to support character-aware adaptive masking and world-anchored spatial conditioning, improving character identity preservation and spatial consistency. Experiments on WBench show that our method achieves higher gameplay fidelity with comparable video quality and spatiotemporal consistency, while using fewer parameters and less training data than state-of-the-art world models. These advantages are also retained in our two-step real-time variant. Overall, Wizard combines agentic control with structured proxy-based generation to enable player-driven game creation with real-time, generalizable, and mechanically consistent world simulation.
achieves higher gameplay fidelity with comparable video quality and spatiotemporal consistency, while using fewer parameters and less training data than state-of-the-art world models. These advantages are also retained in our two-step real-time variant. Overall, Wizard combines agentic control with structured proxy-based generation to enable player-driven game creation with real-time, generalizable, and mechanically consistent world simulation. Project Page: https://wyzardwu.github.io/wizard/.
Introduction
World models aim to predict future observations from visual histories and current actions, attracting growing interest in gaming applications [2, 11, 13, 31, 40]. By conditioning video generation on players’ mouse and keyboard inputs, video world models offer a natural foundation for interactive and personalized gameplay [16, 26, 51, 54, 60].
Despite recent progress, existing game world models face limitations in control flexibility, generalization, and real-time interaction. Action-conditioned models [15, 43, 46] typically restrict players to predefined action sets, while prompt-conditioned models [4, 8, 61] often depend on motion and skill patterns seen during training, limiting their ability to follow unseen instructions.
To improve state consistency and spatial controllability, recent agentic frameworks maintain explicit world states [8, 25] and translate them into structured proxies, such as depth maps [32], semantic maps [5], blockouts [27], or oriented 3D bounding boxes [17], to guide video generation. However, their demonstrations largely focus on basic movement, such as vehicle driving [21, 30] and character locomotion [50, 56], with limited support for articulated character actions, as their proxies do not explicitly encode joint-level motion. Engine-based approaches [18, 55] offer another route to structured control, but their reliance on predefined scenes and additional processing limits scene flexibility and increases latency.
To address this gap, we introduce Wizard (Figure 1), an agentic game world model that combines fine-grained character control, zero-shot generalization, real-time interaction and mechanically consistent gameplay, enabling players to customize their game worlds with novel scenes, characters, and motions specified through reference images or free-form language. At its core, Wizard employs an agentic framework to decouple scene construction, character animation, and gameplay logic from visual generation through structured spatial conditioning.
Figure 1. Overview of Wizard. Our method supports diverse gameplay experiences, including world exploration, character summoning, character transformation, and scene manipulation. By combining agentic control with geometric scene proxies and articulated character skeletons, Wizard achieves fine-grained control, zero-shot generalization, and mechanics-consistent gameplay with real-time interaction.
To enable fine-grained camera and character control, we construct a unified visual proxy (Figure 1) comprising scene geometry that encodes depth and height, and articulated character skeletons. We reconstruct an approximate traversable 3D environment from a user-provided scene image and convert character references into controllable articulated skeletons, with channel coding distinguishing character instances and their camera-relative orientations. The agent renders view-dependent geometric maps and animates skeletons according to camera and action inputs, providing precise spatial and motion conditioning for the causal video generator. For novel characters, a vision-language model retrieves compatible skeletons and animations based on morphology and intended motion style from a high-quality motion library, then selected motions are retargeted to support diverse, realistic movements. The agent also tracks health, relative distance, and orientation to enforce gameplay rules and adapt boss behavior to evolving combat situations.
In addition, we specifically construct 56.1 hours of aligned proxy–video pairs from diverse sources, including data specifically designed for skeletal control, to train the video model to faithfully follow structured proxies and generalize across scenes, characters, and motions. We further introduce character-aware adaptive masking that restricts reference-image attention to the corresponding character regions, strengthening appearance alignment and preserving character identity during motion.
To improve spatial consistency, we propose world-anchored spatial conditioning that uses agent-maintained exploration states to distinguish known and unknown regions in the proxies and injects per-pixel world coordinates only into known regions, preserving consistency in previously generated areas while allowing the model to freely synthesize unexplored content. Finally, we apply few-step distillation [59] to enable real-time video generation at 20 FPS, bringing detailed proxy control and consistent visual rendering into an interactive gameplay loop.
In summary, our contributions are as follows.
We propose Wizard, an agentic game world model that enables real-time, zero-shot generalization across unseen scenes, characters, and motions with precise control, substantially expanding the possibilities for player-driven game creation.
We build a diverse dataset covering 805 scenes and 178 characters from public videos, game footage, and synthetic sources, with aligned character skeleton and scene geometry to enhance model generalization.
We introduce an agentic framework that leverages structured visual proxies to enable accurate character control and mechanics-consistent gameplay.
We leverage character positions in visual proxies and agent-maintained exploration states to enable character-aware adaptive masking and world-anchored spatial conditioning, improving reference-to-video identity alignment and spatial consistency in explored regions while allowing flexible generation of unseen content.
Related Work
Video World Model
Recent advances in video generation have enabled action-conditioned world models that allow users to interactively explore generated environments [2, 7, 13, 31]. In games, these models translate keyboard and mouse inputs into dynamic gameplay, with emerging capabilities including NPC control in ReactiveGWM [46], fine-grained FPS interactions in SCOPE [43], and multiplayer experiences in WorldCast [51] and Multics [16]. However, relying primarily on pixel-level supervision makes it difficult to maintain persistent game states, retain long-term history, and preserve spatial consistency [17]. To address these limitations, agentic frameworks such as WorldMind [8] and Marionette [32] decouple explicit state management and gameplay logic from visual generation. Despite these advances, existing methods often remain constrained to the game settings seen during training. In contrast, Wizard combines fine-grained skeletal control with a diverse training dataset to enable zero-shot generalization to unseen scenes, characters, and motions.
Agentic World Model
Agentic world models decouple world-state management and action planning from visual generation, enabling explicit control over world dynamics while video models focus on rendering. Related simulation-guided approaches, such as PhysDreamer [58], RealWonder [28], and RoboRender [14], leverage explicit physical dynamics to guide video generation. Moving beyond simulator-defined dynamics, GameDirector [25] maintains combat states and enforces gameplay rules to support player-configurable game settings, while Code World Model [5] and Programmable World Model [17] employ agent-generated programs to maintain persistent world states and guide visual generation through structured proxies. However, these methods primarily emphasize state consistency and entity-level control [50, 56], leaving fine-grained character animation underexplored. Concurrent work, SolarRenderer [9], enables real-time video rendering from diverse proxies, yet seamlessly integrating this capability into a closed-loop gameplay experience remains a challenge. To bridge this gap, Wizard extends agentic control to scene geometry and articulated skeletons, enabling flexible scene manipulation and precise character animation. A diverse, multi-source training dataset enhances generalization, while causal adaptation and few-step distillation enable real-time video generation for responsive gameplay.
Data
Training a video world model with fine-grained control and zero-shot generalization requires diverse visual data paired with complementary proxies tailored to characters and backgrounds, namely articulated skeletons for character motion and geometry maps for scene structure. Existing public datasets for video world models either lack these complementary conditions [3, 35, 53] or provide them within a single game with limited gameplay diversity [22].
To this end, we construct 56.1 hours of aligned proxy– video pairs spanning 805 scenes and 178 characters. As shown in Figure 2, our dataset combines gameplay recordings, synthetic data, Unreal Engine simulations, and public dataset, covering diverse environments, character morphologies, and motion patterns. Synthetic scenarios further support character entry, multi-character coexistence, disappearance and reappearance, and rapid turns under moving cameras.
Figure 2. Dataset Overview and Processing Pipeline. We construct 56.1 hours of data spanning 805 scenes and 178 characters from four complementary sources. Scene geometry and character skeletons are jointly encoded into unified visual proxies, followed by filtering and captioning to produce aligned proxy–video training pairs.
For each source, we obtain camera parameters, scene depth, and character skeletons to construct unified visual proxies. Scene geometry and articulated skeletons are jointly rendered under a shared camera, preserving their spatial relationships and mutual occlusion. The RGB channels encode height, character slot and facing direction, and camera-forward depth, respectively. Each character slot is linked to its reference image, decoupling control signals from fixed appearances and supporting generalization to unseen characters. Finally, we filter invalid renders and physically implausible configurations, and use Qwen3-VL-8B [1] to generate structured captions. Each training sample contains an aligned proxy video, an RGB clip and character reference images. Please refer to Appendix § 7 for more dataset details.
Method
As illustrated in Figure 3, Wizard combines an agentic framework for explicit world-state control with a video generator that renders the evolving game through structured visual proxies. Before each gameplay session, an initialization stage taking less than five minutes reconstructs the scene and characters, rigs character skeletons, and builds a target-specific motion set through motion retrieval and retargeting. Once initialized, the framework supports real-time gameplay by updating character motions, scene geometry, and combat states based on player inputs and boss decisions, while the video generator renders the related visual observations.
Figure 3. Inference Pipeline of Wizard. (a) Character animation is driven by motions retrieved from a motion library and retargeted from compatible source skeletons to the target character’s skeleton. (b) Scene manipulation is performed by a world agent that edits 3D scene geometry and renders visual proxies using OpenGL. (c) State-aware combat control adapts boss behavior to the current game state and enforces explicit mechanics. (d) Real-time generation uses spatial memory with retrieved frames and world coordinates.
Agentic Framework
Character Animation via Skeletal Control. Given a user-provided character image, Qwen3-VL [1] detects characters and infers their body structures, including posture (upright or horizontal), the number and arrangement of legs, arms, wings, tails, and heads, and their anatomical left–right assignments. We use the predicted bounding boxes to guide SAM2 [38] in segmenting each character, then reconstruct its mesh with TripoSG [20]. UniRig [57] and our medial-axis fitting procedure generate candidate skeletons, from which we select the one that best matches the inferred body structure. We assign semantic labels to its joints and limb chains, identifying body parts such as the left arm or right leg. These labels allow corresponding body parts to be matched across characters with different joint counts and proportions, providing a shared interface for motion retargeting and control.
To enable realistic combat movements for novel characters, we build a reference motion library from CombatMotion [23], WildWorld [22], and animation assets from Elden Ring [10]. For each target character, we retrieve motions from source skeletons with compatible body structures and use a VLM to assess whether the source characters’ movement styles, such as aggression, grace, or power, suit the target’s appearance. The selected motions can come from multiple sources, allowing each target to inherit a varied moveset. We retarget these motions by matching corresponding limb chains, adapting motion to the target’s limb lengths, and applying inverse kinematics to maintain ground contact. We then filter out clips with poor ground contact, body penetration, or unnatural limb crossing. The remaining clips form a motion set tailored to the target character, enabling the controller to retrieve and execute motions in real time without custom animation scripts.
Scene Manipulation via Geometry Control. As with character control, users can customize and manipulate scenes according to their preferences. Given a scene image, we use MoGe-2 [45] to recover a 3D point map and camera parameters, remove character regions, and construct a background mesh supplemented by a height field. An OpenGL-based offscreen renderer, implemented with ModernGL and EGL, renders this geometry from the current camera viewpoint using the same proxy encoding as in training: the R and B channels encode world height and depth, respectively. As the camera moves, the world agent re-renders the persistent scene geometry to produce a spatially consistent visual proxy. We explain later how the G channel distinguishes visited regions from unknown regions (Figure 3(d)).
During gameplay, the world agent, powered by Qwen3-VL-8B, translates users’ natural-language instructions into executable programs that compose predefined geometric operators and specify their locations, scales, and deformation parameters. These programs modify the underlying scene geometry, enabling transformations such as raising a platform from the ground or turning the world upside down. The modified geometry is then re-rendered into the proxy video, supporting interactive scene manipulation while maintaining consistent camera projection and spatial structure.
State-Aware Combat Control. Our framework maintains a shared world state that tracks both characters’ positions, poses, health, action phases, and skill cooldowns. Based on this state, a behavior planner selects movements and attacks from the previously constructed target-specific motion set. When the player is out of reach, the planner selects a motion that moves the boss toward the player. Within attack range, it evaluates candidate attacks by simulating their swings against the player’s current geometry and prioritizes those likely to connect, subject to action-phase and cooldown constraints. This state-aware planning enables the response to the player’s movements and the evolving combat situation.
The selected motion is then executed in the simulation, where a rule-based controller enforces gameplay constraints and resolves combat outcomes for both characters. It detects hits by checking whether the space swept by an attack intersects the target’s body, excluding periods when the target cannot be hit, such as during a dodge. Valid hits reduce the target’s health, update its health bar, and trigger hit reactions. Skills become available only after their cooldowns expire, and combat ends when either character’s health reaches zero. The updated world state informs the planner’s next decision and is rendered into visual proxies that condition the video generator. Together, state-aware planning and rule-based execution support responsive, mechanics-consistent combat independently of video generation.
Real-Time Generation with Spatial Memory. As illustrated in Figure 3(d), the video generator takes character images, a scene image, and visual proxies to generate video chunks at 20 FPS. Generated frames are lifted into 3D using MapAnything [19] to update a persistent world cache, which guides subsequent proxy rendering and memory retrieval.
To enable exploration beyond the input view, the proxy marks unknown regions with a reserved color in the G channel and provides no world coordinates there, allowing the generator to synthesize unseen scenery. Once generated and reconstructed, these regions become known geometry, with height and depth encoded in the R and B channels. This design preserves previously observed geometry while supporting open-ended generation. For memory retrieval, we select up to three historical frames that best cover the region visible from the current viewpoint. These frames, together with the updated proxies, character and scene images, and world coordinates, condition next-chunk generation. World-coordinate injection is detailed in § 4.2.
Model Training
As shown in Figure 4, we build our video generator on Wan2.2-TI2V-5B [44]:
Figure 4. Model Training. (a) Character-aware adaptive masking aligns generated characters with their references, while (b) world-anchored spatial conditioning anchors memory and proxy tokens to a shared spatial frame. During inference, the agent maintains the corresponding masks and world coordinates to support consistent generation.
where , , and denote the visual proxy sequence, the initial scene image, and the reference image for character , respectively. denotes the world-coordinate maps associated with the current proxy sequence and memory frames, while denotes the context from previously generated frames.
their positions, reducing cross-character appearance leakage. During character transformation, we additionally block attention to cached tokens associated with the previous identity while preserving access to background history and the first-frame sink. By linking appearance guidance to agent-controlled character states, this strategy supports flexible summoning, removal, transformation, and multi-character gameplay while maintaining scene continuity.
Character-Aware Adaptive Masking. Character skeletons provide explicit spatial information during both training and inference: training data include ground-truth skeletons, while our agentic framework maintains and updates them at runtime. We leverage this shared structure to bind each reference image to its intended character. Each character reference image is VAE-encoded and appended as tokens with a distinct temporal positional offset. For each frame, we derive a bounding region from each character’s skeleton and expand it by 25% to include its surrounding context. In every DiT self-attention layer, we gate attention from video token to reference token using an additive mask:
where identifies the character associated with reference token , and denote the spatial position and frame index of video token , and is character ’s expanded region on the token grid at frame .
These spatial masks are updated from the explicit character states maintained by our agentic framework, allowing players to control when and where bosses appear or disappear while directing reference guidance to the corresponding regions. As characters move, the masks follow
World-Anchored Spatial Conditioning
Beyond memory frames introduced previously, our spatial memory incorporates world coordinates from an explicit scene map maintained and updated by the agent during exploration. We project this map into the viewpoints of both memory frames and the current proxy video to provide spatial conditioning for the video model. For each location , let indicate whether the corresponding scene point has been observed by time . We retain coordinates only for known locations:
where is the world-coordinate position of the scene point projected to . The coordinate maps are embedded into spatially aligned tokens and added element-wise to the corresponding memory tokens and channel-concatenated proxy-video tokens before sequence concatenation. This shared world-coordinate conditioning anchors memory and current generation to the same persistent map, encouraging consistency in previously observed regions while leaving unexplored regions free for synthesis.
Experiments
Experimental Setup
Implementation. We first fine-tune Wan2.2-TI2V-5B [44] using 56.1 hours of clips bidirectionally for 35K steps. Following Causal Forcing++ [59], we adapt it to causal generation and distill it into a real-time 2-step student via consistency distillation and DMD on its own rollouts. As teacher forcing provides a clean history, we corrupt it with the model’s one-step reconstruction of a noised copy, forcing the scene to be recovered from the world-anchored memory. Further implementation details are provided in Appendix § 8.
Benchmarks and Metrics. For fair comparison, we evaluate video and spatiotemporal quality using the metrics and 158 navigation cases provided by WBench [52]. To assess gameplay fidelity, we construct a Gameplay test set of 50 clips featuring characters generated by Qwen-Image [48]. Interaction Adherence measures how accurately generated actions follow control inputs, while Mechanics Consistency evaluates adherence to specified game rules. Please refer to Appendix § 9 and Appendix § 10 for more details on evaluation.
Baselines. We compare with end-to-end world models (YUME 1.5 [31], Matrix-Game 3.0 [47], LingBot-World [41]) and agentic approaches (CWM [5], Marionette [32]), each driven through its own control inputs (Appendix § 11). The agentic baselines release only bidirectional models, so our main comparison uses our bidirectional model (30 sampling steps). We also report our distilled causal model, which runs in real time with 2 steps.
Main Results
Table 1 evaluates general world modeling and gameplay capabilities. On WBench, Wizard achieves competitive video quality with a 5B backbone and 56.1 hours of diverse training data. In particular, the results of motion smoothness highlights the value of articulated skeleton proxies, while spatial memory supports competitive spatiotemporal quality.
| Method | Params | Data | Steps | Imaging Quality ↑ | Motion Smoothness ↑ | Background Consistency ↑ | Spatial Consistency ↑ | Interaction Adherence ↑ | Mechanics Consistency ↑ |
| YUME 1.5 [31] | 5B | >400 h | 50 | 63.3 | 97.0 | 90.3 | 71.4 | 45.3 | 88.0 |
| Matrix-Game 3.0 [47] | 5B | 4.8M clips | 50 | 70.0 | 95.4 | 85.7 | 80.4 | 0.0 | 15.2 |
| Marionette (Wan2.2-Fun) [32] | 5B | 674 h | 40 | 61.8 | 94.6 | 83.2 | 64.3 | 50.7 | 46.4 |
| LingBot-World (Base) [41] | 28B | n/r | 70 | 67.9 | 96.9 | 96.9 | 67.1 | 37.3 | 78.8 |
| CWM (MiniMax-H3) [5] | 33B | 5.6 h | 20 | 64.6 | 98.0 | 92.3 | 85.0 | 81.3 | 96.7 |
| Wizard (Ours) | 5B | 56.1 h | 30 | 64.7 | 98.0 | 89.9 | 83.6 | 86.7 | 96.1 |
| Wizard (Ours, causal) | 5B | 56.1 h | 2 | 64.4 | 96.9 | 88.2 | 77.4 | 81.3 | 96.3 |
Table 1. Comparison with existing world models on WBench [52] (158 navigation cases) and our Gameplay testset (50 clips). We report parameter counts, training data scale, and denoising steps; “n/r” means not reported. Bold and underline indicate the best and second-best results. Wizard and its real-time variant (20 FPS) achieve strong gameplay fidelity with competitive video and spatiotemporal quality.
Beyond these general capabilities, agentic control provides strong gameplay fidelity: both Wizard and CWM exceed 95% mechanics consistency, but our articulated pose control achieves higher interaction adherence than CWM’s fixed skeletal pose across actions. These advantages largely persist after distillation, enabling two-step generation at 20 FPS with competitive visual quality and gameplay fidelity. We further demonstrate scene manipulation and character transformation in the qualitative results, as few baselines support the precise zero-shot control required for a meaningful quantitative comparison.
Ablation Studies
Proposed Modules. We use 69 Entry & Revisit cases (initial frame from Wbench) to test reference binding between characters with identical skeletons but different references, and spatial consistency after a 150° camera turn and return. In Table 2(a), the character-aware adaptive mask raises reference binding from 52.8 to 76.4, while spatial memory improves spatial consistency by up to 4.5 points. Combining both preserves strong character binding and achieves the highest spatial consistency, demonstrating complementary benefits for identity preservation and scene recall.
| Character-Aware Adaptive Mask | Spatial Memory | Reference Binding ↑ | Spatial Consistency ↑ |
| (a) Proposed modules | |||
| ✗ | ✗ | 52.8 | 77.9 |
| ✗ | ✓ | 52.6 | 81.0 |
| ✓ | ✗ | 76.4 | 77.6 |
| ✓ | ✓ | 73.5 | 82.1 |
| Spatial condition | Revisit Error ↓ | Copy Bias ↓ | |
| (b) World coordinates in the spatial memory | |||
| Memory frames only | 29.43 | 6.61 | |
| + World coordinates (Ours) | 27.44 | 2.84 |
Table 2. Ablation Studies. (a) Character-aware adaptive mask and spatial memory. (b) Spatial memory w/ and w/o world coordinates.
World Coordinates. We evaluate 100 revisit windows from held-out SceneFly scenes [35], where previously observed regions are revisited from different angles. In Table 2(b), adding world coordinates reduces revisit error from 29.43 to 27.44 and copy bias from 6.61 to 2.84. This suggests that world-coordinate conditioning helps re-render visited content from the current viewpoint rather than directly copy the memory frame. Evaluation metric details are provided in Appendix § 10.
Qualitative Visualization
Figure 5 presents unseen user-provided references on the left and paired agent-controlled proxies and generated frames on the right, all using the same model without fine-tuning. Figure 5(a) shows zero-shot generalization to novel scenes, characters, and motions. Figure 5(b) shows character transformation and scene manipulation, including environment inversion and new geometry. Figure 5(c) illustrates mechanics-consistent combat: the agent tracks game states, terminates combat when a character’s health reaches zero, and displays the outcome. Together, these examples highlight flexible player control with consistent visual rendering and explicit rule enforcement. Please refer to Appendix § 12 for qualitative comparisons with baselines and ablated variants.
Figure 5. Qualitative Visualization of Wizard. For each example, reference images are shown on the left, with visual proxies in the top row and the corresponding video rollouts in the bottom row. Please zoom in for better visibility.
Conclusions
In this paper, we introduce Wizard, an agentic game world model that decouples character control, scene manipulation, and gameplay logic from visual generation. Articulated skeletons and geometry maps enable fine-grained character and scene control with mechanics-consistent, real-time interaction. Trained on our diverse dataset, Wizard adopts character-aware adaptive masking and world-anchored spatial conditioning to improve identity preservation and scene consistency. Experiments show strong performance with fewer denoising steps and zero-shot generalization to unseen characters, scenes, and motions. These capabilities enable players to create and manipulate customized game worlds.
Wizard: Manipulate Any World in Real Time via Agentic Control
Supplementary Material
Dataset
This section details the data summarized in Figure 2 of the main paper, following its three stages: data acquisition (Sec. A.1), unified proxy encoding (Sec. A.2), and filtering and annotation (Sec. A.3).
Data Sources
Our training set contains 50,115 clips (56.1 hours) spanning 805 scenes and 178 characters. As shown in Figure 2, it comes from four complementary sources: gameplay recordings, public datasets, UE simulations and synthetic data. We detail each component in Table 3. These data sources offer complementary strengths: gameplay recordings provide high-quality appearance and motion but cover a limited range of characters and events; public datasets expand the diversity of creature morphologies and camera movements; and simulated and synthetic data fill gaps with events rarely captured in recordings, such as character entry and disappearance, as well as rapid camera turns.
| Source | Content | Clips | Hours | Share |
| Gameplay recordings | 11,286 | 12.6 | 22% | |
| Elden Ring | combat, exploration | 11,286 | 12.6 | |
| Public datasets | 26,055 | 29.2 | 52% | |
| WildWorld | creature combat | 23,654 | 26.5 | |
| SceneFly | camera fly-through | 2,401 | 2.7 | |
| UE simulations | 2,966 | 3.3 | 6% | |
| Unreal Engine | scripted boss fights | 2,966 | 3.3 | |
| Synthetic data | 9,808 | 11.0 | 20% | |
| Procedural rendering | multi-character scenes | 9,386 | 10.5 | |
| LTX-2.5 generation | world editing | 422 | 0.5 | |
| Total | 50,115 | 56.1 | 100% |
Table 3. Overview of the training dataset. We summarize the content, number of clips, duration, and proportion of data from each source. All training clips contain 121 frames at 30 fps, with proportions measured by duration.
Gameplay Recordings (12.6 h). We record Elden Ring [10]: 8.8 hours of combat against six bosses with diverse body structures and 3.8 hours of exploration across 11 regions, with more than 20 player outfits. Camera poses and skeletons are read from the game process; scene depth is captured once per second, fused into scene geometry and re-rendered at every frame.
Public Datasets (29.2 h). WildWorld [22] adds diverse creature morphologies; its annotations are converted to our proxy representation. SceneFly [35] provides character-free camera fly-throughs that return to earlier viewpoints, which supervises scene memory; its depth is estimated with MapAnything [19].
UE Simulations (3.3 h). We script boss fights in Unreal Engine with reconstructed characters and weapons [42]. Since the engine state is known, the skeletons, cameras and scene geometry of these clips are exact.
Synthetic Data (11.0 h). This source has two parts. First, we procedurally render 162 character assets (133 bosses and 29 players) from Sketchfab [39], Quaternius [37] and KayKit [29] in scenes built from 742 Poly Haven [36] scene assets. The clips cover four scenarios that recordings rarely contain: novel-character entry, multi-character coexistence, disappearance and reappearance, and rapid turns under a following camera. All conditions come directly from the renderer. Second, for world editing we generate videos with LTX-2.5 [12] in which the terrain in front of the player is reshaped (e.g., stairs, pillars, walls) or effects with clear geometry appear (e.g., water columns, ice spikes). Their conditions are estimated with Depth Anything 3 [24], SAM 2 [38] and ViTPose [49].
Unified Proxy Encoding
Every source is converted to the same visual proxy through the two branches of Figure 2. In the scene branch, characters are removed from the depth and the region behind them is completed, which gives the scene geometry. In the character branch, the 3D joint positions of each character are drawn as a skeleton. Both are rendered into one conditioning video under the shared camera.
Following Marionette [32], the RGB channels encode geometry rather than color: R encodes the height and B the camera-forward depth of both the scene and the characters. The G channel has a different meaning in the two branches. For characters, it gives the slot of each character, separating the player from the others, and its facing direction. For the scene, it gives how reliable the geometry is: unknown regions take a reserved dark green, coarse geometry a constant level, and exact geometry a brighter level with Lambertian shading. The proxy does not indicate which reference image a character corresponds to; this binding is left to the character-aware adaptive mask.
Filtering and Annotation
Sequences are split into non-overlapping 121-frame windows at 30 FPS, resized to 832 × 480 (references to 480 × 480), and encoded with the Wan2.2 VAE. Invalid clips, i.e., those whose state information is incomplete or whose proxy does not align with the video, are discarded. Qwen3-VL-8B [1] then writes a caption for each remaining clip from its first frame and character references, which yields the aligned proxy–video pairs used for training.
Implementation Details
Training Configuration
We initialize from Wan2.2-TI2V-5B [44] and keep its VAE and UMT5 text encoder [6] frozen. The bidirectional model is trained for 35K steps on 8 NVIDIA H200 GPUs with one clip per GPU. We use AdamW with a learning rate of for the pretrained DiT weights and for the new conditioning modules, and drop text with probability 0.1. At inference, the bidirectional model uses 30 denoising steps and a guidance scale of 3.
World-Coordinate Tokens
Each location of the proxy and memory frames carries its world coordinate, a flag for whether it has geometry, and the geometry level of the proxy. Coordinates are encoded with sinusoids at 8 frequencies and multiplied by the flag, so unknown regions, sky and characters contribute no coordinates. The features are packed to the video-token resolution, mapped by a zero-initialized linear layer, and added to the patch embeddings of the video and memory tokens. Characters are hidden in memory frames, so that memory supplies the scene only.
World Cache and Memory Retrieval
At inference, the proxy is rendered from the world cache at every frame. Generated keyframes are lifted into 3D with MapAnything [19] and fused into the cache by a background worker, so generation never waits for reconstruction. For each chunk of 4 latent frames, up to three memory frames are selected greedily from earlier frames by how much of the currently visible region each newly covers. The same retrieval function builds the training data, so the model is trained on the kind of memory it receives at inference.
Few-Step Distillation
Training Stages. Following Causal Forcing++ [59], we obtain the causal model in three stages. All stages read a sliding window of 20 latent frames plus the first frame as a persistent attention sink.
Causal adaptation (11K steps). The bidirectional model is trained with teacher forcing and independent per-frame noise levels.
Consistency distillation (10K steps). A guided causal teacher takes one Euler step along a 48-level discretization, and the student matches the prediction of its EMA at the next level.
Distribution matching distillation (480 steps). The student generates clips autoregressively from its own history and is updated with the score difference between the frozen bidirectional model and a critic trained on the student’s samples.
History Corruption. Under teacher forcing, the clean history already contains almost all of the scene, so the model does not learn to rely on its memory when the history drifts at inference. In the first two stages we therefore replace the clean history, with probability , by the model’s one-step reconstruction of a noised copy with per-frame noise level , while the sink frame, the memory frames and the target stay clean. We use , in causal adaptation and , in consistency distillation. The corruption is applied only to clips that require memory (revisits and SceneFly).
Deployment. We use the EMA student after distillation. It streams with 2 denoising steps and a KV cache of 20 latent frames plus the sink frame.
Benchmarks
Benchmark Design
Existing world-model benchmarks such as WBench [52] evaluate visual quality, navigation and consistency when revisiting a view, with a single controllable subject. They do not test whether new characters can be brought into the world, whether each one keeps its specified identity, or whether scripted interactions and their consequences actually take place. We therefore keep WBench for visual quality and navigation and add three test sets for these abilities Table 4. All three follow the same principles, which also underlie the fairness of the comparison in Sec. D.
| Set | Cases | Tests | Metrics |
| WBench navigation | 158 | visual quality, navigation | official |
| Gameplay | 50 | scripted multi-character combat | interaction adh., mechanics cons. |
| Entry & Revisit | 69 | identity binding, scene memory | reference binding, spatial cons. |
| Held-out revisit | 100 | memory under large view change | revisit error, copy bias |
Table 4. Test Sets. WBench evaluates navigation; our three sets evaluate multi-character gameplay, identity binding and memory.
Unseen Content. All sets are built on WBench first frames, and all characters are specified by newly generated reference images; neither is seen during training.
Engine Ground Truth. Every case is produced by our engine from a script, so the position and state of each character at each frame are known exactly. Cases are filtered only by engine-side checks, independently of any generated video.
Model-Agnostic. The same script is rendered into every control format (text, skeleton video, depth and semantic maps), so that all methods receive equivalent control instructions through inputs they are designed to support.
WBench
We evaluate the performance of baseline methods and Wizard on all 158 cases, comprising 74 third-person and 84 first-person scenarios, following the official evaluation protocol for both perspectives.
Gameplay Set
The set contains 50 clips of 121 frames on 25 third-person first frames from WBench. Characters are specified by reference images generated with Qwen-Image [48] (humanoids, giants and quadrupeds), automatically rigged, and animated with combat motions from Elden Ring [10] and WildWorld [22]. In each clip, two characters enter from both sides (35 humanoid and 15 quadruped pairs) and follow one of four scripts, which cover the basic mechanics of a combat game: the player attacks a designated character (13 clips); a character attacks and the player evades (13); a character is knocked down and stays down (12); two characters attack the player in turn (12). Every script involves three characters and requires a specific one to act, so a model succeeds only if it introduces both characters, keeps them distinct, and executes the interaction between the right pair.
Entry & Revisit Set
The set is built in the same way on 69 first frames (56 third-person and 13 first-person). Each first frame has two sequences, and each sequence isolates one of our two modules. In the entry sequence (121 frames), the two entering characters share one skeleton and differ only in their reference images, so the control signal carries no identity cue and the characters can only be told apart through reference binding. In the revisit sequence (241 frames), the camera turns away by and returns, so the original view leaves the frame entirely and must be restored from memory.
Held-Out Revisit Windows
The world-coordinate ablation needs ground truth for the revisited content, which generated sequences do not have. It therefore uses 100 windows of 121 frames from SceneFly dataset [35] of three scenes held out from training, with ground-truth depth, camera poses and video. To test memory under a large view change, the memory frames of each window are chosen by geometry from other windows of the same scene: a frame qualifies if it covers at least 10% of the current view and its camera is rotated by – from it. Both variants receive the same memory frames, seed and sampling steps.
Metrics
We use three groups of metrics: the official WBench metrics for visual quality and spatiotemporal quality, two gameplay-fidelity metrics for the comparison with baselines, and four metrics for the ablations.
WBench Metrics
All metrics follow the official WBench [52] implementation. For spatial consistency we report the gated version, which compares the frames revisiting the initial view with the first frame and penalizes videos whose camera does not move as instructed.
Gameplay Fidelity
The WBench metrics score how a video looks, not what happens in it: a video in which no character enters, or the wrong character attacks, can still obtain high quality and consistency scores. We therefore measure gameplay fidelity with two metrics. Both are defined as the accuracy of closed-form questions whose correct answers are given by the engine state, so they require neither a subjective rating nor a reference video.
Protocol. Every query is answered by GPT-5.5 [33] and consists of (i) frames sampled from the generated video at times fixed by the script, (ii) a crop of the player from the first frame, (iii) a text description of each of the two characters, and (iv) one multiple-choice question whose options are shuffled. Let be a set of queries, the judge’s answer to query , and the answer given by the engine state. Each metric is an accuracy
pooled over all queries of the 50 clips.
Interaction Adherence measures whether the specified character performs the intended action in the generated rollout. Its query set contains one query for each of the 75 scripted actions. For an action, we sample 8 frames around its scripted time and ask who is involved, e.g., “Which character attacks the player character?” or “Which character rolls or dodges away?”, with the candidate characters and “nobody” as options. The correct answer is the character that the script assigns to the action. An answer is incorrect if the intended action is absent or performed by the wrong character. Generating an arbitrary attack is therefore insufficient: the model must show both characters, preserve their distinct identities, and have the designated character perform the specified action.
Mechanics Consistency measures whether the state of the world follows the script over time, inspired by StatePlay [26]. It is the mean of two accuracies,
(, 352 queries) measures whether the generated rollout preserves the engine-maintained character states, including their presence, number, and defeat status. In temporal windows away from entry and knock-down events, the judge determines whether each character is absent, standing, or lying defeated, and whether it appears more than once. An answer is correct if it matches the character’s state in the engine at that time. This penalizes missing characters, unintended disappearances, duplicates, and defeated characters standing up again. Event Realization (, 112 queries) measures whether the generated rollout follows game rules by realizing their expected visual consequences, such as a character falling defeated when its HP reaches zero. After each entry event, the judge checks whether the entering character appears in the scene; after each knock-down event, the judge checks whether the defeated character is lying on the ground. This verifies that rule-driven state transitions are reflected in the generated video.
Ablation Metrics
The ablations compare variants of our own model that receive identical inputs, seeds and sampling steps, so each metric below isolates the effect of one component. The module ablation uses two metrics on the Entry & Revisit set, and the world-coordinate ablation two metrics on the held-out windows.
Reference Binding measures whether each entering character takes the appearance of its own reference image. For a frame in which character is visible, let be the crop of the generated frame at the box of , and the two reference images. With the DINOv2 [34] feature and the other character,
where contains the frames, sampled at 2 fps, in which the character covers at least 0.2% of the image. The boxes come from the engine and are the same for every variant, so the metric does not depend on a detector. Because the two characters of an entry sequence share one skeleton, a model that ignores the references, or gives both characters one appearance, scores the chance level of 50%.
Spatial Consistency in the ablation study is evaluated using the metrics defined in WBench. The scores in Table 2 differ from those in Table 1 because the ablation uses a subset of revisit-intensive cases to assess how spatial memory preserves scene consistency when previously observed regions are revisited.
Revisit Error measures how accurately remembered content is restored from different viewpoints. On the held-out windows the ground-truth video is available. Let be the memory region, i.e., the pixels that are covered by a memory frame but not visible in the first frame of the window; these pixels can only be restored from memory. With the output,
in intensity levels (0–255); lower is better. is computed from the ground-truth depth and cameras only, so it is identical for both variants and independent of their outputs. We introduce a new metric because the trajectories revisit previously observed regions from different viewing angles, rather than returning to the same viewpoints required by the spatial consistency metric. This metric therefore evaluates scene consistency across revisits with viewpoint changes.
Copy Bias evaluates whether remembered regions are correctly re-rendered from a new viewpoint rather than directly copied from the memory frame, helping assess the benefit of world-coordinate conditioning. Let denote the raw memory frame without reprojection, the ground-truth current view, and the generated view. We define
where is the mean absolute difference over the revisited region . The first term measures the expected visual difference between the current view and the memory frame, while the second measures the corresponding difference for the generated view. Directly copying makes the second term zero, yielding a bias equal to the first term. Conversely, excessive deviation from can make the second term larger than the first, which is also penalized. A perfect output yields zero bias; lower is better. We evaluate viewpoint changes of 45–60°, where direct copying and correct re-rendering produce clearly different appearances.
Baseline Settings
Model Versions
We use the released checkpoints: YUME 1.5 (5B, 720p), Matrix-Game 3.0 (base model), LingBot-World-Base (camera-controlled), Marionette’s observation model on Wan2.2-Fun-5B-Control, and CWM’s adapter on MiniMax-H3. Model sizes and training data in Table 1 are taken from the original papers and exclude base-model pretraining. The Steps column lists the sampling steps of our runs: 50 for YUME and Matrix-Game, and the official defaults for Marionette (40), LingBot-World (70) and CWM (20).
Control Conditioning
All models start from the same first frames and receive the control signals through their supported inputs. Text-conditioned baselines share one prompt per clip: Qwen3.6-27B describes the scene, the player and each character, and the scripted events are written with their times. CWM [5] is run in its official setting, with depth and semantic maps rendered from the same scripts and a prompt in its released format. Marionette [32] is conditioned on rendered skeleton videos. Only Wizard receives reference images; as described in Sec. D.2, the judge does not see them.
WBench Evaluation Protocol
Results of YUME, Matrix-Game and LingBot-World are taken from the official leaderboard, which evaluates the same releases; the other methods are evaluated by us with the official code. For the 16-second videos, bidirectional models (CWM, Marionette and Wizard) generate window by window, each conditioned on previously generated frames, and our causal model generates a continuous stream.
Qualitative Results
Comparison with Baseline Methods
Figure 6 compares all methods on two Gameplay cases. In (a), the script asks Character 1 to strike the player: Wizard follows it, in CWM and YUME the other character attacks, and in LingBot-World and YUME the player is never knocked down. Marionette performs the right action but loses the appearance of both characters. In (b), the player attacks Character 2: Wizard and CWM both execute the attack, whereas no attack happens in LingBot-World, the characters are not recognizable in Marionette, and one character appears twice in YUME. Matrix-Game accepts only keyboard and mouse actions, so no character enters.

Figure 6. Qualitative Comparison with the Baseline Methods. (a) Script: at 1.6 s, Character 1 strikes the player, who is knocked down. First frame; Character 1; Character 2; s; s; s. Wizard (Ours): ✓ Character 1 strikes. CWM: ✗ Character 2 strikes. LingBot-World: ✗ the player is never knocked down. Marionette: ✓ Character 1 strikes (appearance lost). YUME 1.5: ✗ Character 2 strikes; the player is never knocked down. Matrix-Game 3.0: ✗ no characters appear. (b) Script: at 1.6 s, the player attacks Character 2, which flinches from the hit. First frame; Character 1; Character 2; s; s; s. Wizard (Ours): ✓ the player hits Character 2. CWM: ✓ the player hits Character 2. LingBot-World: ✗ no attack. Marionette: ✗ characters not recognizable. YUME 1.5: ✗ a character appears twice. Matrix-Game 3.0: ✗ no characters appear. Each case shows the first frame, the two character references and the script; frames are taken at the same times for all methods, and the line below each row gives the judge’s outcome.
Qualitative Ablation
Figure 7 illustrates the effect of each component. (a) Without the character-aware adaptive mask, the two entering characters take a mixture of both references or share one appearance; with the mask, each character follows its own reference. (b) Without the world-anchored spatial condition, the model generates a different scene when the camera returns; with it, the original view is restored. (c) The memory frame shows the scene from a viewpoint – away from the current one. The variant without world coordinates reproduces the content of the memory frame at its original image position (orange box), whereas the full model re-renders it under the current view, close to the ground truth. Additional qualitative examples are provided in Figure 8.

Figure 7. Qualitative Ablation. (a) Entry sequences without and with the mask. (b) Revisit sequences without and with the spatial condition: the camera turns away and returns. (c) Held-out windows without and with world coordinates; orange boxes mark content that matches the memory frame, and green boxes mark the same area rendered as in the ground truth.
Figure 8. More Qualitative Examples. For each example, reference images are shown on the left, with visual proxies in the top row and the corresponding video rollouts in the bottom row. Please zoom in for better visibility.
Limitations and Future Work
Wizard still has several limitations.
Motion coverage. Character motion is retrieved from a motion library and retargeted, so it is difficult to find compatible motions for characters with uncommon body structures. Generating motion directly for arbitrary skeletons would remove this dependence.
Long rollouts. The streaming model reads a bounded history and relies on retrieved memory frames for everything older, so the consistency between the scene proxies and the generated video may decline over longer rollouts.
Upstream perception. The world state is built with off-the-shelf models for rigging and 3D reconstruction, and their errors propagate: a wrongly rigged limb or a misplaced surface produces a proxy that the generator follows faithfully. Correcting the world state from the generated frames could reduce this sensitivity.
Future work will focus on more flexible and realistic character motion that better reflects actual gameplay, consistent scene appearance over long rollouts, and richer interactions among more characters. These improvements will bring agentic video generation closer to fully playable, closed-loop game worlds with greater player freedom and more precise, nuanced control.
References
- [1]Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Overview of vision-language-action model. arXiv preprint arXiv:2511.21631, 2025. 4, 2
- [2]Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first International Conference on Machine Learning, 2024. 2, 3arxiv.org/abs/2402.15391
- [3]Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gam egen: Interactive open-world game video generation. In International Conference on Learning Representations, pages 37546–37593, 2025. 3arxiv.org/abs/2411.00769
- [4]Danze Chen, Zeqing Wang, Ziyue Lin, Xingyi Yang, and Yeying Jin. H3-world: Turning language understanding into world control. arXiv preprint arXiv:2609.01560, 2026. 2arxiv.org/abs/2609.01560
- [5]Yiwen Chen, Guosheng Lin, and Chi Zhang. Code world model: Coding agent as world brain. arXiv preprint arXiv:2608.25927, 2026. 2, 3, 7, 5arxiv.org/abs/2608.25927
- [6]Hyung Won Chung, Noah Constant, Xavier Garcia, Adam Roberts, Yi Tay, Sharan Narang, and Orhan Firat. Unimax: Fairer and more effective language sampling for large-scale multilingual pretraining. arXiv preprint arXiv:2304.09151, 2023. 3
- [7]Zhiyang Deng, Yuzhe Wu, Zijun Lin, and Yeying Jin. Actionrope: action as coordinates for spatially consistent control in planar-camera game world models. arXiv preprint arXiv:2610.25861, 2026. 3
- [8]Zhiyang Deng, Boran Zhang, Danze Chen, and Yeying Jin. Worldmind: Decoupled game world model for state-aware npc behavior. arXiv preprint arXiv:2608.21439, 2026. 2, 3arxiv.org/abs/2608.21439
- [9]Guian Fang, Shengju Qian, Dongyue Lu, Junchao Huang, Lutao Jiang, Xin Wang, Yiren Song, Li Jiang, and Mike Zheng Shou. Solarrenderer: Few-step generative reference rendering. arXiv preprint arXiv:2610.28630, 2026. 3arxiv.org/abs/2610.28630
- [10]FromSoftware. Elden Ring. Video game. Bandai Namco Entertainment, 2022. 5, 2, 4
- [11]Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, Jiahao Wang, Yufeng Yuan, Hanlin Wang, Yichong Lu, et al. Infinite worlds with versatile interactions. arXiv preprint arXiv:2607.07534, 2026. 2arxiv.org/abs/2607.07534
- [12]Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvocko, Avishai Berkowitz, Daniel Shalem, Daphne Lifschitz, Dudu Moshe, Eitan Porat, Eitan Richardson, Guy Shiran, Itay Chachy, Jonathan Chetboun, Michael Finkelson, Michael Kupchick, Nir Zabari, Nitzan Guetta, Noa Kotler, Ofir Bibi, Ori Gordon, Poriya Panet, Roi Benita, Shahar Armon, Victor Kulikov, Yaron Inger, Yonatan Shiftan, Zeev Melumian, and Zeev Farbman. LTX-2: Efficient joint audio-visual foundation model. arXiv preprint arXiv:2601.03233, 2026. 2arxiv.org/abs/2601.03233
- [13]Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang, Yifan Zhang, Qi Cui, Fei Kang, Biao Jiang, Mengyin An, Yangyang Ren, et al. Matrix-game 2.0: An open-source real-time and streaming interactive world model. arXiv preprint arXiv:2508.13009, 2025. 2, 3arxiv.org/abs/2508.13009
- [14]Huang Huang, Wensi Ai, Ziyu Chen, Youhui Wang, Zijian Du, Yang Liu, Jiaolong Yang, Li Fei-Fei, and Jiajun Wu. Roborender: Robot-oriented video generation for visual sim-to-real transfer. arXiv preprint arXiv:2610.09254, 2026. 3arxiv.org/abs/2610.09254
- [15]Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, et al. Solarwm: Open data and scalable training for long-horizon video world models. arXiv preprint arXiv:2609.02886, 2026. 2
- [16]Kaiyi Huang, Wangbo Wu, Yukun Huang, Jiwen Yu, Kunhao Liu, Donglin Yang, Wenbo Hu, Ying Shan, and Xihui Liu. Multics: Generative multi-player world model with hierarchical communication. arXiv preprint arXiv:2610.17201, 2026. 2, 3arxiv.org/abs/2610.17201
- [17]Zheng-Hui Huang, Guixu Lin, Jiacheng Lin, Yi-Chuan Huang, Ruihan Yu, Muyao Niu, Siqi Yang, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, et al. Programmable world model. arXiv preprint arXiv:2609.10540, 2026. 2, 3arxiv.org/abs/2609.10540
- [18]Zheng-Hui Huang, Zhixiang Wang, Jiaming Tan, Ruihan Yu, Yidan Zhang, Bo Zheng, Yu-Lun Liu, Yung-Yu Chuang, and Kaipeng Zhang. Generative world renderer. arXiv preprint arXiv:2604.02329, 2026. 2arxiv.org/abs/2604.02329
- [19]Nikhil Keetha, Norman Müller, Johannes Schönberger, Lorenzo Porzi, Yuchen Zhang, Tobias Fischer, Arno Knapitsch, Duncan Zauss, Ethan Weber, Nelson Antunes, Jonathon Luiten, Manuel Lopez-Antequera, Samuel Rota Bulò, Christian Richardt, Deva Ramanan, Sebastian Scherer, and Peter Kontschieder. MapAnything: Universal feed-forward metric 3D reconstruction. arXiv preprint arXiv:2509.13414, 2025. 5, 2, 3arxiv.org/abs/2509.13414
- [20]Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. 5
- [21]Yitong Li, Junsong Chen, Haopeng Li, Haozhe Liu, Jincheng Yu, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Sol video inference engine: Agent-native full-stack acceleration framework for efficient video generation, 2026. 2arxiv.org/abs/2606.23743
- [22]Zhen Li, Zian Meng, Shuwei Shi, Wenshuo Peng, Yuwei Wu, Bo Zheng, Chuanhao Li, and Kaipeng Zhang. Wildworld: A large-scale dataset for dynamic world modeling with actions and explicit state toward generative world arpg. arXiv preprint arXiv:2603.23497, 2026. 3, 5, 2, 4
- [23]Yihao Liao, Yiyu Fu, Ziming Cheng, and Jiangfeiyang Wang. Animationgpt: An aigc tool for generating game combat motion assets. https://github.com/fyyakaxy/AnimationGPT, 2024. 5
- [24]Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y. Chen, Zhenyu Li, Yang Zhao, Sida Peng, Hengkai Guo, Xiaowei Zhou, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views. In International Conference on Learning Representations, 2026. 2arxiv.org/abs/2511.10647
- [25]Zijun Lin, Zhiyang Deng, Yuzhe Wu, Bihan Wen, and Yeying Jin. Gamedirector: Decoupling gameplay logic from ren- dering for player-configurable game world models. arXiv preprint arXiv:2609.25652, 2026. 2, 3
- [26]Zijun Lin, Zeqing Wang, Cheston Tan, Bihan Wen, and Yeying Jin. Stateplay: State-aware game world models for mechanics-consistent generation. arXiv preprint arXiv:2607.26754, 2026. 2, 4arxiv.org/abs/2607.26754
- [27]Ruicong Liu, Liangyang Ouyang, Fengbo Lan, Zhixiang Wang, and Kaipeng Zhang. Anygamerenderer: Entity-consistent generative rendering across games and an entity-level benchmark. arXiv preprint arXiv:2610.31466, 2026. 2arxiv.org/abs/2610.31466
- [28]Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, and Jiajun Wu. Realwonder: Real-time physical action-conditioned video generation. arXiv preprint arXiv:2603.05449, 2026. 3arxiv.org/abs/2603.05449
- [29]Kay Lousberg. KayKit: Game assets. https://kaylousberg.com/game-assets. Accessed: 2026-10-16. 2
- [30]Yanzuo Lu, Tian Ye, Yitong Li, Shuchen Xue, Haozhe Liu, Song Han, and Enze Xie. Streamrender-h3: Interactive coding worlds with real-time streaming diffusion render. https://nvlabs.github.io/Sana/Sol-Engine/StreamRender-H3/, 2026. Project page. 2
- [31]Xiaofeng Mao, Zhen Li, Chuanhao Li, Xiaojie Xu, Kaining Ying, and Kaipeng Zhang. Yumel.5: A text-controlled interactive world generation model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7752–7761, 2026. 2, 3, 7
- [32]Zian Meng, Zhen Li, Chuanhao Li, Qiang Li, and Kaipeng Zhang. Marionette: Predicting world states, rendering geometry, painting appearance. arXiv preprint arXiv:2608.14530, 2026. 2, 3, 7, 5arxiv.org/abs/2608.14530
- [33]OpenAI. GPT-5.5 system card. https://openai.com/index/gpt-5-5-system-card/, 2026. Accessed: 2026-07-28. 4
- [34]Maxime Ouqab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaeldin El-Nouby, et al. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. 5
- [35]Zhan Peng, Jie Ma, Huiqiang Sun, Chong Gao, Zhijie Xue, Zhiyu Pan, Zhiguo Cao, Jun Liang, and Jing Li. Compression and retrieval: Implicit memory retrieval for video world models. arXiv preprint arXiv:2606.23105, 2026. 3, 8, 2, 4arxiv.org/abs/2606.23105
- [36]Poly Haven. Poly Haven: The public 3d asset library. https://polyhaven.com/. Accessed: 2026-10-16. 2
- [37]Quaternius. Quaternius: Free game assets. https://quaternius.com/. Accessed: 2026-10-16. 2
- [38]Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. In International Conference on Learning Representations, pages 28085–28128, 2025. 4, 2arxiv.org/abs/2408.00714
- [39]Sketchfab. Sketchfab. https://sketchfab.com/. Accessed: 2026-10-16. 2
- [40]Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling. arXiv preprint arXiv:2512.14614, 2025. 2arxiv.org/abs/2512.14614
- [41]Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, Yixuan Li, Hanlin Wang, Yinghao Xu, Shuailei Ma, et al. Advancing open-source world models. arXiv preprint arXiv:2601.20540, 2026. 7
- [42]Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image, 2024. 2
- [43]Zizhao Tong, Yeying Jin, Hongfeng Lai, Zeqing Wang, Zhaohu Xing, Kexu Cheng, Haoran Xu, Zhao Pu, Shangwen Zhu, Ruili Feng, et al. Scope: Simulating cross-game operations in playable environments for fps world models. arXiv preprint arXiv:2605.23345, 2026. 2, 3arxiv.org/abs/2605.23345
- [44]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 5, 7, 3arxiv.org/abs/2503.20314
- [45]Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. Advances in Neural Information Processing Systems, 38:395928–395959, 2026. 5arxiv.org/abs/2507.02546
- [46]Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, and Yeying Jin. Reactivegwm: Steering npc in reactive game world models. arXiv preprint arXiv:2605.15256, 2026. 2, 3
- [47]Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, Peiyu Wang, Biao Jiang, Yichen Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory. arXiv preprint arXiv:2604.08995, 2026. 7arxiv.org/abs/2604.08995
- [48]Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02324, 2025. 7, 4arxiv.org/abs/2508.02324
- [49]Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. ViTPose: Simple vision transformer baselines for human pose estimation. In Advances in Neural Information Processing Systems, 2022. 2arxiv.org/abs/2204.12484
- [50]Xindi Yang, Baolu Li, Liam Lee, Zhenfei Yin, Songxin Zhang, Zhuoyang Song, Xu Jia, Jianfei Cai, Tien-Tsin Wong, Bingyi Jing, and Mengyue Yang. Oneira: From open-ended generation to open-world interaction in video world models. arXiv preprint arXiv:2610.01614, 2026. 2, 3arxiv.org/abs/2610.01614
- [51]Ziyang Ye, Junchao Huang, Evelyn Zhang, Zhihao Xie, Ruicheng Zhang, Boyao Han, Litao Ban, Ziye Wang, Xinting Hu, Shaoshuai Shi, et al. Worldcast: Distributed multiplayer world models. arXiv preprint arXiv:2610.12412, 2026. 2, 3arxiv.org/abs/2610.12412
- [52]Kaining Ying, Hengrui Hu, Siyu Ren, Jiamu Li, Fengjiao Chen, Ziwen Wang, Xuezhi Cao, Xunliang Cai, and Henghui Ding. Wbench: A comprehensive multi-turn benchmark for interactive video world model evaluation. arXiv preprint arXiv:2605.25874, 2026. 7, 3, 4arxiv.org/abs/2605.25874
- [53]Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. Gamefactory: Creating new games with generative interactive videos. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11590–11599. IEEE, 2025.arxiv.org/abs/2501.08325
- [54]Wangbo Yu, Kunhao Liu, Wenbo Hu, Shenghai Yuan, Chaoran Feng, Haiyang Zhou, Yukun Huang, Yiran Wang, Wang Zhao, Yingmin Luo, et al. Worldcrafter: Consistent video world model with implicit 3d-aware memory. arXiv preprint arXiv:2609.24984, 2026.arxiv.org/abs/2609.24984
- [55]Xiaoyu Zhang, Xinyu Wang, Xiaohong Zhang, Huanjie Zhu, Tengjiao Sun, Pengcheng Fang, Jiaxing Yu, Yanwen Guo, and Dongjie Fu. Magpie: Real-time world renderer for interactive games. arXiv preprint arXiv:2608.27168, 2026.arxiv.org/abs/2608.27168
- [56]Jihai Zhang, Runjia Qian, Kaiqin Kong, Biao Jiang, Puyi Wang, Siru Chen, Tianle Li, Guanjie Chen, Junyi Wu, Chubin Chen, et al. Matrix-game 4.0: Harnessing video models for real-time playable game worlds. arXiv preprint arXiv:2610.27999, 2026.arxiv.org/abs/2610.27999
- [57]Jia-Peng Zhang, Cheng-Feng Pu, Meng-Hao Guo, Yan-Pei Cao, and Shi-Min Hu. One model to rig them all: Diverse skeleton rigging with unrig. ACM Transactions on Graphics (ToG), 44(4):1–18, 2025.
- [58]Tianyuan Zhang, Hong-Xing Yu, Rundi Wu, Brandon Y Feng, Changxi Zheng, Noah Snavely, Jiajun Wu, and William T Freeman. Physdreamer: Physics-based interaction with 3d objects via video generation. In European Conference on Computer Vision, pages 388–406. Springer, 2024.arxiv.org/abs/2404.13026
- [59]Min Zhao, Hongzhou Zhu, Kaiwen Zheng, Zihan Zhou, Bokai Yan, Xinyuan Li, Xiao Yang, Chongxuan Li, and Jun Zhu. Causal forcing++: Scalable few-step autoregressive diffusion distillation for real-time interactive video generation. arXiv preprint arXiv:2605.15141, 2026.arxiv.org/abs/2605.15141
- [60]Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, and Enze Xie. Sanawm: Efficient minute-scale world modeling with hybrid linear diffusion transformer. arXiv preprint arXiv:2605.15178, 2026.arxiv.org/abs/2605.15178
- [61]Shangwen Zhu, Qianyu Peng, Zhao Pu, Zhilei Shu, Xiangrui Ke, Zhaowen Xing, Yizhao Tong, Zeqing Wang, Xinyu Cui, Zian Zheng, et al. Incantation: Natural language as the action interface for multi-entity video world models. arXiv preprint arXiv:2605.18601, 2026.arxiv.org/abs/2605.18601





