WorksComputer Science
Fight World Model: Customizable Combat through Visual Interaction Proxies
Abstract
A playable, programmable game world must maintain persistent state, enforce executable rules, and support extensible gameplay mechanics. Existing video world models can generate plausible combat, yet struggle to keep hit reactions and health updates consistent with game rules. We introduce Fight World Model (FWM), which separates rule-governed combat execution from video rendering. An executable engine maintains state and resolves interactions. A deterministic compiler encodes these outcomes and attempted actions in a spatially and temporally aligned interaction proxy. The proxy aligns action and skill windows, brief hit reactions, and persistent health by fighter identity and timing. These cues can be controlled separately to render interactions under modified rules. Identity-bound references preserve character appearance across side switches. We train FWM on FightWorld Corpus, spanning 52 skill effects, 187 character-appearance variants, and 15 arenas. Without retraining the renderer, FWM supports damage scaling, limited-hit immunity, lifesteal, and multi-stage skill mechanisms beyond the native game. It also allows character and arena customization. Experiments demonstrate the effectiveness of FWM, which outperforms existing game world models in action fidelity, hit reactions, health-state consistency, and character appearance preservation. Our project is available at https://fight-world-model.github.io.
Introduction
Interactive world models aim to simulate how environments evolve under user actions [2, 10, 27, 46, 47]. Recent video world models learn from gameplay recordings to generate playable sequences from controller or text inputs [7, 15, 30, 33, 39, 43]. Customizing such worlds requires changing both gameplay rules and visual appearances. In multi-fighter combat, the same attack may cause different health loss under different rules. A protected fighter may react to a hit without losing health, while lifesteal may restore the attacker’s health. When only character or arena appearance changes, the specified interactions should remain the same. As illustrated in Figure 1, a customizable system must allow interaction outcomes to follow edited rules while preserving those interactions across appearance changes.
Figure 1. Customizable Combat with FWM. Character and arena customization is shown on the left, and controllable skills on the right. The central pipeline connects rule execution to video rendering through an interaction proxy.
These requirements are difficult to meet when state transitions are inferred within the video generator [5, 7, 15, 33, 37, 39]. The model learns how actions relate to reactions and state changes from recorded gameplay. StatePlay further predicts game-state variables, such as health, alongside video frames [21], but predicting state does not itself provide an editable rule for updating it. Multi-entity action binding, identity-aware instructions, and NPC planning extend control over characters [8, 28, 37, 38, 51]. They do not by themselves determine how interaction outcomes should change under a new rule configuration. When rules change, a generator may retain familiar action–consequence associations from training and deviate from the externally specified outcomes.
Code-driven world models offer a way to execute world-state transitions outside the video generator and render the resulting spatial proxies [6, 16, 42, 44]. Correct state execution alone, however, does not ensure that the generated interaction follows those results. The renderer must identify the acting and affected fighters, depict each event at the intended time, and preserve the resulting state across subsequent exchanges. Spatial layouts alone do not specify action windows, hit reactions, or persistent health changes. Text and state tokens can carry these quantities, but the renderer must map them to the appropriate body regions, heads-up display (HUD) elements, and video frames. Appearance customization adds a related difficulty: actions and consequences must remain associated with the intended fighters as their appearances change or they exchange positions. The interface must therefore convey attempted actions, events, and persistent state with consistent identity and timing.
We introduce Fight World Model (FWM), a customizable world-model framework that connects executable combat rules to video generation through a visual interaction proxy. As shown in Figure 2, an executable engine resolves interactions and maintains state. A deterministic compiler rasterizes fighter states, action and skill windows, and transient hit-reaction cues onto a shared canvas and timeline. Reactions and health updates are encoded separately, allowing a hit to trigger a visible response without necessarily reducing health. Rule edits change the executed records and the resulting proxy, while the proxy format and renderer remain unchanged. Character and arena references specify appearance, with identity binding associating character references with the intended fighters across side switches. This design lets the same trained renderer depict interactions under different rules and visual appearances.
Figure 2. Overview of FWM. The Proxy Engine executes gameplay rules, and the resulting state trajectories, action windows, and events are compiled into a visual interaction proxy. The proxy and appearance references are encoded with a shared VAE and concatenated with video tokens. Matching spatiotemporal rotary positions align the proxy with the video, while shared identity embeddings bind character references to the corresponding regions in the first video latent and proxy sequence. The DiT processes the combined sequence, and only video tokens are decoded to generate the continuation.
We instantiate and evaluate FWM in a fixed-view two-fighter combat setting. FightWorld Corpus pairs gameplay videos with interaction proxies across 52 skill effects, 187 character-appearance variants, and 15 arenas. FWM outperforms adapted game world models in action fidelity, hit reactions, persistent health accuracy, and character appearance preservation. Under matched executed records, reference binding, and training budgets, the interaction proxy also achieves higher joint fidelity than spatial proxies augmented with text or state tokens. Without retraining the renderer, FWM supports damage scaling, limited-hit immunity, lifesteal, and multi-stage skill mechanisms beyond native Street Fighter II/III. Appearance and side-switch evaluations test whether specified interactions and reference-to-entity associations are preserved under visual changes. We make the following contributions:
We present FWM, a customizable combat world-model system that supports editable interaction rules and character or arena appearances without retraining the renderer.
We design a visual interaction proxy that separately encodes action windows, hit reactions, and persistent health with consistent identity and timing. Identity binding connects appearance references to the fighters participating in these interactions.
We construct FightWorld Corpus and evaluate the system through comparisons with existing methods, controlled encoding studies, rule interventions, and appearance customization.
Related Work
Game World Models. Action-conditioned video world models predict future frames from user inputs [1, 2, 33]. Recent systems extend this approach toward real-time streaming, longer horizons, and broader environments [5, 7, 9, 11, 30, 39, 43, 48]. Beyond navigation and player-centric interaction, multi-entity control introduces the need to associate each input with the intended character. Per-player keyboard streams can be paired with character references [15], while language conditions, persistent subject tokens, and synchronized inputs support multiple subjects or players [28, 31, 51]. Compact state representations also support NPC planning [8]. ReactiveGWM v1 separates player keyboard control from NPC strategy conditioning [37], and ReactiveGWM v2 uses identity handles to direct action and rule text to either fighter [38]. StatePlay additionally predicts health and skill meters alongside video through a dedicated state branch [21]. These methods improve action specification, entity control, and state prediction, but interaction outcomes remain largely governed by learned video dynamics. We focus on rendered reactions and persistent state under externally edited rules.
World State and Visual Proxies. Spatial and trajectory conditioning provides explicit control over object placement and motion in video generation [25, 35, 36, 45]. Recent world models separate state execution or maintenance from video rendering [3, 6, 16, 42, 44]. MASS maintains a learned authoritative state shared across players [3], while Magpie translates white-box engine frames into generated video [44]. Oneira represents geometry and coarse interaction states through colored 3D boxes [42]. Code World Model compiles depth, semantic-ID, and instance maps from executable code [6], and Programmable World Model projects 3D boxes into identity, semantic-category, and motion-direction maps [16]. These methods establish explicit interfaces between world state and visual generation. Our focus is on the interaction information needed when edited rules change the relationship between attempted actions, hit reactions, and persistent health.
Reference-Guided Video Generation. Reference-guided video generators condition diffusion models on reference images to preserve subject appearance across frames [13, 17, 23, 40, 49]. Multi-subject methods use spatial masks, semantic alignment from vision-language models, or binding slots to distinguish appearances across entities [20, 41, 50]. Character animation and motion-transfer methods drive reference subjects through explicit pose sequences [14] or in-context spatiotemporal tokens aligned with the output video [26, 41]. In our setting, appearance references must remain associated with the fighters performing each action and receiving each consequence, including through side switches. We study this association jointly with rule-controlled interaction outcomes.
Method
FWM supports rule and appearance customization by coupling executable state updates with conditional video generation. As shown in Figure 2, an engine executes the active rules, and a deterministic compiler converts the resulting states and events into a visual interaction proxy. Rule edits change the executed interaction and proxy, while appearance edits retain the proxy and change the references. Both use the same trained renderer.
Rule Execution and Customization
Let be the executable state at engine update , the available action inputs, and the rule configuration. The engine computes
where contains the events produced together with the updated state . The engine computes entity motion, resolves hit events, and applies the configured state updates. includes persistent entity attributes and the counters or timers needed to execute the rules. The configuration can be edited in code [6, 16].
We reserve for output video frames. Engine states and event records are aligned to this frame timeline before compilation. Let denote the aligned state and its visible projection. From the aligned event history, we form active records that specify entity identities, onsets, and durations for action windows and reaction events. An event onset is paired with the state update it produces. The compiler then generates each proxy frame as .
A rule edit replaces with and reruns the engine from the same initial state and submitted action sequence. Entity trajectories, hit times, and rule-dependent state are recomputed under . Multi-stage skill mechanisms are expressed through rules that schedule displacement, impacts, and status windows. The resulting records are compiled and rendered without changing the renderer parameters. For damage scaling, limited-hit immunity, and lifesteal, the appearance references and text conditions also remain fixed. Changes in executed outcomes reach the renderer only through the recomputed proxy.
An appearance edit changes the character or arena references while keeping the executed records and proxy fixed. Character references retain their target entity identities. Rule and appearance edits can also be combined by rendering recomputed records with new appearance conditions. For successive action intervals, the engine carries its state forward, while visual context supports continuity between generated segments.
Interaction Proxy
The proxy separates attempted actions, transient reactions, and persistent state because editable rules can change their relationship. These signals share entity identities and frame indices, allowing their values and durations to vary without changing the conditioning format. We describe the concrete combat encoding used in our experiments below.
For this instantiation, the visible state at video frame is
where is a persistent fighter identity, and , , , , and denote its bounding box, facing direction, stance, coarse action label, and health. Each fighter’s box and HUD health bar share an identity color. Facing and stance marks encode orientation and posture, and a separate band encodes the coarse action label, including an unknown value for unannotated actions.
Action and Skill Windows. For the fighter with logged inputs, six attack classes combine punches and kicks with light, medium, and heavy strengths. Horizontal stripes encode punches and vertical stripes encode kicks, with coverage indicating strength. The marks remain active during the command window and supplement the coarse labels available for both fighters. They encode attempted actions, with hit outcomes specified separately. At inference, the engine can assign command windows to either fighter through its persistent identity.
A magenta block in front of the caster follows its position and facing direction to mark a skill’s casting window. Text describes the effect’s type and appearance, so the corpus’s 52 visual skill effects share the same casting marker.
Reactions and Persistent State. Reaction onset is marked by briefly inverting the affected fighter’s box fill and health-bar color. Bar length independently encodes health. This supports a protected hit with a reaction cue but no health decrease, or lifesteal with an additional update to the attacker’s health. Identity colors associate each reaction and state update with the affected fighter across side switches.
The same compiler processes video-derived annotations during training and engine-generated records at inference. Training reaction onsets are inferred from observed health decreases, while the engine specifies reaction events and health under the edited rules. The shared format therefore permits rule-dependent cue combinations beyond those implied by the training annotations. Rule-intervention experiments test whether the trained renderer realizes these combinations. Broad command stripes and four-frame inversions are designed to retain cues through spatial and temporal compression by the video variational autoencoder (VAE) [18, 19]. Detailed motion, textures, and effect appearance are supplied by the renderer.
Video Rendering and Identity Binding
We condition a pretrained video diffusion transformer on the proxy and appearance inputs. Video, reference, and proxy inputs share a VAE and patch embedding. Their tokens are concatenated in video–reference–proxy order, with learned channel-wise gates on the reference and proxy streams. The initial video context, proxy, and references remain clean, while future video latents are denoised. Text descriptions are encoded by the text encoder and supplied through cross-attention. Only video tokens are decoded.
Let be the initial context frame and the number of future video frames. The renderer samples
where contains the available character or arena references and the target identity associated with each character reference. The text condition describes appearance and skill-effect type. The proxy supplies executed interaction states and event schedules.
Proxy tokens share spatiotemporal rotary positions with the corresponding video tokens [32]. Reference tokens use positions outside the video grid, distinguishing appearance conditions from video-aligned interaction cues.
As illustrated in Figure 2, shared identity embeddings bind character references to persistent entity identities. For each training sample, we assign the fighters distinct slots sampled without replacement from a learned codebook. Each fighter’s slot embedding is added to its corresponding tokens in the first video latent and throughout the proxy sequence, weighted by soft bounding-box coverage. The same embedding is added to all tokens of that fighter’s reference. Random slot assignment discourages a fixed mapping between codebook entries and appearances. At inference, the target identity attached to a character reference selects its slot embedding. Arena references do not use an entity slot. A training sample for character appearance editing contains a source initial image, a reference crop, and an edited target video. The source image establishes the scene and entity locations, while the reference specifies the target appearance in the continuation.
Training
We optimize the diffusion transformer, conditioning gates, and identity codebook with flow matching [22, 24]. The objective in Eq. (5) supervises only future video latents, while the VAE and text encoder remain frozen. We apply condition dropout [12] to proxy, reference, and text inputs. For reference-conditioned samples, appearance-description dropout encourages reliance on the reference image.
FightWorld Corpus
We construct FightWorld Corpus by pairing Street Fighter II [4] gameplay videos with interaction proxies and optional appearance references. A frozen video editor augments native recordings with edits to skill effects, character appearances, and arenas, yielding 112,618 training samples. As summarized in Figure 3, the corpus spans 52 skill effects, 187 character-appearance variants, and 15 arenas.
Figure 3. FightWorld Corpus. Native gameplay is expanded with skill effects, character-appearance edits, and scene edits, yielding 112,618 training samples.
Character and arena edits vary visual appearance while retaining the source interaction. We reuse source state annotations only after checking consistency in fighter trajectories, event timing, and health. Each accepted video is paired with a character crop or an arena reference, as illustrated for scene edits in Figure 4.
Figure 4. Scene Editing in FightWorld Corpus. Each scene pairs a clean reference with an edited training frame. References exclude fighters and HUD elements and may differ in background details.
Skill edits change the visual effect associated with a casting event and may also alter the visible interaction. We therefore extract state and event annotations from both native and skill-edited videos, retaining only clips that pass quality screening.
Native and edited videos share the same training format, with interaction proxies, text descriptions, and optional character or arena references as conditions. We use these samples to train a shared renderer across skill effects and appearances.
Experiments
We evaluate appearance control, interaction fidelity, and rule customization. We compare FWM with adapted game world models and conduct controlled studies to examine the contributions of interaction encoding and identity binding.
Evaluation Setup
Implementation. We evaluate FWM on two-fighter interactions in the fixed 2D view of Street Fighter II [4]. Our renderer uses Wan2.2-TI2V-5B [34] as its backbone and is trained on FightWorld Corpus.
Evaluation Protocol. For recorded interactions, held-out video annotations specify the target interactions. For rule interventions, we fix the initial state and submitted actions, then rerun the engine under each rule configuration to compute trajectories, hit events, and state updates. FWM’s reference images and text condition remain unchanged for damage scaling, limited-hit immunity, and lifesteal. Appearance evaluations change the references while retaining the executed interaction. Each controlled encoding comparison supplies identical interaction records to all variants. Action and Reaction Metrics. Move Accuracy (MoveAcc) measures agreement between fighter motion and directional commands. Attack Accuracy (AttackAcc) is the fraction of detected punch or kick events supported by the corresponding input commands within their temporal windows. Skill Accuracy (SkillAcc) requires the correct skill class and an onset within the command-relative evaluation window. Reaction measures the fraction of expected hit responses visibly performed by the designated fighter. Qwen judges these responses from body motion in nearby frames, with the HUD excluded.
Health and Persistence. Health MAE measures the mean absolute error between generated and target HUD health, normalized by maximum health and reported in percentage points. The joint health check requires correct updates for both fighters and persistence of the resulting health values between events.
Joint Interaction Fidelity. Joint interaction fidelity () treats an attempt as successful only if all applicable action, entity, timing, reaction, and health checks pass. We average attempt-level results within each episode and give episodes equal weight. For rule interventions, requires the applicable reaction and health checks to pass together.
Appearance and Entity Assignment. Adherence uses a VLM to judge whether the requested appearance is applied to the designated character or arena. The judge receives the target reference and a source-frame identity guide. DINO measures cosine similarity between reference and generated features for characters and arenas separately. Both metrics exclude the fixed initial context.
Baselines. We compare FWM with H3-WORLD [5], ReactiveGWM v1 [37], and ReactiveGWM v2 [38]. We adapt all three methods to FightWorld Corpus and its 52 skill effects. For skill control, we add a dedicated trigger to v1 and v2 and specify activation in H3-WORLD’s interval action text. The +Ref variants use our reference-conditioning branch for appearance editing.
Character and Scene Customization
As illustrated in Figure 5, FWM renders the same prescribed interaction with different character and arena references. We retain the executed records and proxy while changing

Figure 5. Character and Scene Reference Conditioning. Interaction rollouts across four character appearances (under default and sideswitched stances) and two arena references.
appearance, testing whether the specified actions and consequences are preserved.
Identity Association across Side Switches. We further test whether character references remain associated with their intended fighters across side switches. This requires preserving identity independently of the fighters’ screen positions. The variants in Table 1 share reference inputs and training budgets, differing only in whether identity binding is enabled. Binding improves character adherence and DINO similarity, with an adherence gain of approximately 45 percentage points on side-switch clips.
| Test Subset | Binding | Adherence ↑ | DINO Similarity ↑ |
| Non-Switch | Off | 78.43 | 0.47 |
| Non-Switch | On | 99.16 | 0.61 |
| Side-Switch | Off | 53.68 | 0.35 |
| Side-Switch | On | 98.27 | 0.60 |
Table 1. Effect of Identity Binding. Variants share reference inputs and training budgets, differing only in identity binding. Adherence is a percentage. Values are ranked within each test subset.
Control Representation Comparisons
From Commands to Executed Outcomes. Adding rule text to action commands improves joint fidelity, but the generated interactions remain inconsistent. Spatial Proxy Only accurately controls movement while showing low joint fidelity in Table 2. These conditions leave the renderer to infer hit events and health updates. The results motivate supplying executed outcomes alongside attempted actions.
Interaction Fidelity across Control Representations. We compare variants with different conditioning signals to assess the effects of interaction information and its encoding. Rates are percentages, and Health MAE is in percentage points of maximum health. Across tables, bold and underlined values mark the best and second-best results.
| Conditioning | Move / Attack / Skill | Reaction ↑ | Health MAE ↓ | ↑ |
| Actions Only | 70.27 / 56.22 / 58.64 | 74.43 | 3.42 | 34.18 |
| Actions + Rule Text | 71.71 / 68.40 / 57.35 | 80.16 | 3.07 | 46.37 |
| Spatial Proxy Only | 99.19 / 92.27 / 46.58 | 78.28 | 2.31 | 33.46 |
| Spatial Proxy + State Tokens | 94.48 / 91.23 / 82.42 | 87.17 | 1.23 | 63.28 |
| Spatial Proxy + Text | 96.72 / 93.63 / 92.43 | 92.24 | 2.78 | 76.35 |
| FWM Interaction Proxy | 99.78 / 99.24 / 99.06 | 99.79 | 0.15 | 98.48 |
Table 2.
Holding Executed Information Fixed. To isolate the effect of encoding, we compare Spatial Proxy + Text, Spatial Proxy + State Tokens, and FWM under identical executed interaction records. The records specify fighter identities, action classes, affected fighters, event onsets and durations, and frame-wise health trajectories. All variants share geometry, initial context, reference binding, backbone initialization, and training budgets.
Effect of Interaction Encoding. With identical executed information, FWM improves joint fidelity by approximately 22 percentage points over Spatial Proxy + Text, the stronger symbolic baseline in Table 2. State Tokens have lower average health error than Text but poorer joint fidelity, showing that average health accuracy does not fully capture interaction correctness.
Text misassigns damage to the attacker or misses the recipient’s health update in Figure 6. State Tokens track health more closely on average but lag at impact onset and show local limb blur during contact. Both baselines supply events and health as symbolic records that must be associated with body regions, HUD bars, and video frames. FWM places these cues in the corresponding proxy regions and frames.

Figure 6. Complete-Information Encodings on the Same Interaction. Held-out rollout under matched executed records (Table 2). Green brackets mark HUD state errors: ↓! unexpected health loss, ↓× missing health loss, delayed update, ↓̸= incomplete update, and ↑! unexpected recovery.
Comparison with Existing Methods
Action Control and State Consistency. ReactiveGWM v1 [37] achieves higher player movement and attack precision than H3-WORLD [5] in Table 3. It uses keyboard inputs for the player and coarser strategy text for the NPC. ReactiveGWM v2 [38] directs action instructions to either fighter, but still infers hit outcomes and health updates within video generation. The baselines miss actions and reactions in Figure 7, and v2 shows spurious health recovery.
| Method | Action Move / Attack / Skill | Reaction | Health MAE | Adherence Character / Scene | Character DINO | Scene DINO |
| Without Reference-Image Conditioning | ||||||
| H3-WORLD | 43.85 / 62.61 / 47.58 | 48.25 | 8.75 | — | — | — |
| ReactiveGWM v1 | 70.15 / 78.98 / 48.55 | 35.56 | 4.59 | — | — | — |
| ReactiveGWM v2 | 50.24 / 59.70 / 16.67 | 49.21 | 4.65 | — | — | — |
| With Reference-Image Conditioning | ||||||
| H3-WORLD + Ref. | 36.91 / 62.70 / 42.03 | 42.22 | 7.57 | 88.10 / 100.00 | 0.47 | 0.68 |
| ReactiveGWM v1 + Ref. | 63.16 / 85.53 / 53.86 | 33.97 | 4.79 | 89.29 / 100.00 | 0.50 | 0.64 |
| ReactiveGWM v2 + Ref. | 72.22 / 74.10 / 34.78 | 57.14 | 4.04 | 91.67 / 100.00 | 0.41 | 0.65 |
| FWM (Ours) | 99.78 / 99.24 / 99.06 | 99.79 | 0.15 | 99.12 / 100.00 | 0.60 | 0.72 |
Table 3. Comparison with Adapted Game World Models. Baselines are adapted to FightWorld Corpus. +Ref denotes the addition of our reference-conditioning branch. Rates are percentages, and Health MAE is in percentage points of maximum health. DINO measures cosine similarity. A dash indicates an inapplicable metric.
Figure 7. Dual-Character Actions and Responses. Top rails show visible action summaries for both fighters. Per-frame badges indicate realized (✓) or unrealized (✕) fighter actions and responses; ? denotes strategy-level NPC text in ReactiveGWM v1; HUD state icons follow Figure 6.
Multi-Phase Skills. The baselines struggle to synchronize skill effects with their consequences. In Figure 8, ReactiveGWM v1 and v2 deduct health before projectile impact, while H3-WORLD shows insufficient health loss after contact. ReactiveGWM v2 also renders a malformed projectile effect.

Figure 8. Skill Timing and Event Consequences. Comparison on a projectile skill commanded at ; HUD state icons follow Figure 6.
Appearance and Interaction Fidelity. Adding appearance references improves v2’s movement and skill accuracy by approximately 22 and 18 percentage points, respectively.
The +Ref baselines achieve high character adherence in Table 3, yet their hit reactions and health updates remain inconsistent. Garment blur and identity drift during rapid contact and side switches further indicate that overall adherence does not guarantee stable visual identity throughout an interaction.
Executed Outcomes as Visual Conditions. FWM improves action accuracy, hit reactions, health consistency, and character adherence over the adapted baselines in Table 3. Its Reaction score exceeds the best +Ref result by approximately 43 percentage points. The rollouts in Figure 7 and Figure 8 follow the prescribed action and reaction timing while preserving the resulting health across subsequent frames.
Rule Interventions
Rule and Skill Customization. We test whether generated interactions follow edited rules and newly composed skill mechanisms without retraining the renderer. For attack-level interventions, the initial state, submitted actions, appearance references, and FWM text condition remain fixed. The engine recomputes motion, hit timing, and state updates, and the changed outcomes reach FWM only through the proxy. FWM maintains high joint fidelity across the revised rules in Table 5, while symbolic encodings degrade particularly under immunity and lifesteal.
| Encoding | Health MAE ↓ | ↑ |
| Default Rules | ||
| Spatial Proxy + State Tokens | 1.87 | 89.26 |
| Spatial Proxy + Text | 2.42 | 76.38 |
| FWM | 0.16 | 100.00 |
| Damage Magnitude | ||
| Spatial Proxy + State Tokens | 1.63 / 2.48 | 74.62 / 65.37 |
| Spatial Proxy + Text | 3.17 / 4.86 | 56.28 / 43.71 |
| FWM | 0.18 / 0.22 | 97.76 / 98.84 |
| Limited-Hit Immunity | ||
| Spatial Proxy + State Tokens | 4.28 | 49.36 |
| Spatial Proxy + Text | 8.73 | 24.18 |
| FWM | 0.21 | 99.92 |
| Lifesteal | ||
| Spatial Proxy + State Tokens | 5.67 | 37.42 |
| Spatial Proxy + Text | 10.46 | 16.83 |
| FWM | 0.19 | 99.87 |
Table 5. Table 5: Interaction Fidelity under Rule Changes. Within each rule setting, encodings share executed records and are ranked separately. Default rules use original damage without immunity or lifesteal. Paired values denote reduced / amplified damage. Health MAE is in percentage points of maximum health, and is a percentage.
Damage Magnitude. Damage scaling changes the health loss associated with a valid hit. FWM tracks the resulting health changes under both reduced and amplified damage, with low health error and high joint fidelity.
Limited-Hit Immunity. Limited-hit immunity blocks health loss for a specified number of hits while retaining reaction events. FWM renders the corresponding body responses with unchanged health during protection, then shows the prescribed health loss once protection is exhausted.
Lifesteal. Under lifesteal, a hit reduces the recipient’s health and restores the attacker’s health. FWM renders these coupled updates for the correct fighters and preserves the resulting health values after the hit.
Multi-Stage Skill Mechanisms. We extend the rule interventions to skills that combine displacement, repeated impacts, and state constraints. In Figure 9, Earthshatter couples knockback with staged health loss and lingering ground damage. Other mechanisms cover per-projectile hits, staggered area impacts, action locks, and spatial confinement. FWM renders these combinations through the shared proxy without additional skill-specific conditioning channels or renderer retraining.
Figure 9. Skill-Level Rule Mechanisms Beyond Native Combat. Five combat mechanisms absent from native Street Fighter II/III, spanning lingering ground damage, per-projectile hits, staggered area hits, action locks, and spatial confinement (boundary locked).
Proxy Ablations
Attempted Actions and Their Consequences. We retrain each variant with one cue family removed, keeping the training setting unchanged. In Table 4, removing command marks lowers attack precision while leaving Reaction largely unchanged. Removing the skill trigger reduces skill accuracy by approximately 20 percentage points. These results support encoding attempted actions separately from their consequences.
| Variant | Move / Attack / Skill | Reaction | Health MAE | |
| FWM | 99.78 / 99.24 / 99.06 | 99.79 | 0.15 | 98.48 |
| w/o Command Marks | 99.07 / 86.42 / 87.87 | 97.93 | 0.28 | 85.26 |
| w/o Damage-Event Cues | 99.16 / 98.92 / 98.91 | 80.42 | 0.27 | 78.38 |
| w/o Health Bars | 99.84 / 99.08 / 99.12 | 98.31 | 2.18 | 57.24 |
| w/o Skill Event Trigger | 99.21 / 99.03 / 79.28 | 98.07 | 0.26 | 78.81 |
Table 4. Contributions of Proxy Cues. Each ablated variant omits one cue family and is retrained under the same training setting. Rates are percentages, and Health MAE is in percentage points of maximum health.
Events and Persistent State. Removing damage-event cues lowers the Reaction score while health error remains low. Removing health bars largely preserves reactions but degrades health accuracy and joint fidelity. These complementary effects support separate conditioning for transient reactions and persistent state. More details, comparisons and visualizations are provided in the appendix.
Conclusion
We presented Fight World Model (FWM), which connects executable gameplay rules to video generation through a visual interaction proxy. Experiments in a combat setting demonstrate improved interaction and appearance fidelity over adapted game world models. FWM supports edited rules and multi-stage skill mechanisms without retraining the renderer. Character and arena customization preserves the prescribed interactions, with identity binding maintaining character associations across side switches.
References
Implementation Details
Engine and Timeline
The combat engine exposes damage magnitude, the number of protected hits, and the lifesteal rate as editable parameters. Its state retains protection counts and active skill windows across action intervals.
Engine updates indexed by are mapped to output video frames before proxy compilation. State samples and event windows use the same frame mapping, so an event onset is paired with its resulting state update. Frame index zero denotes the initial context, and denotes the continuation. The engine carries executable state between action intervals, while visual context conditions the next generated segment.
Proxy Encoding Parameters
For light, medium, and heavy attacks, stripes cover the top one third, top two thirds, or full box interior below the coarse action band. They remain active over the command window. The skill-trigger block follows the caster for 30 output video frames, and reaction color inversions last four frames. Both durations are defined at the output video rate and resampled together with the state and event records.
Renderer and Binding Parameters
We initialize the renderer from the Wan2.2-TI2V-5B checkpoint [34]. For video latent frames, character references use temporal coordinate , and arena references use . Their spatial coordinates lie outside the video grid. Video and proxy inputs use matched temporal indices and the same resize and center-crop transform.
The identity codebook contains six learned slots. Each embedding is normalized to unit root-mean-square magnitude and scaled by 0.25 at all three injection sites. For the first video latent and proxy sequence, soft bounding-box coverage weights the corresponding entity’s embedding, with weights normalized where boxes overlap. The target entity’s embedding is added to all character-reference tokens without a spatial box mask. Persistent entity IDs in the proxy identify the reference targets. Arena references receive no identity embedding.
Training Objective and Configuration
Let denote clean video latents and . We construct noisy latents with flow matching [22, 24]:
Here is flow time, distinct from engine index and video-frame index . The first latent is replaced with the clean source-image encoding at noise level zero. Future video tokens receive noise level . With conditions , the loss is
where contains future video latent elements. Initial-context, proxy, and reference tokens are excluded from the loss. At inference, future video latents start from noise.
We use AdamW with a constant learning rate of , weight decay of 0.01, and no learning-rate warmup. Training uses bfloat16 autocast with fp32 master parameters and an effective batch size of 128 on 128 H200 GPUs. The budget is 35 epochs, corresponding to 30,765 optimizer steps.
Each clip contains 89 frames at 928 × 704 resolution. Dropout probabilities for proxy, reference, text, and appearance descriptions are 0.1, 0.1, 0.1, and 0.2, respectively. The controlled encoding variants in Table 2 share initialization, training data, and optimization budget. Only the representation and its required input layers differ.
Figure 10. Interaction-Proxy Vocabulary. Transient cues and persistent state share fighter identities.
Data Construction Details
Video Editing and Quality Control
A frozen MiniMax-H3 Ref2VA editor produces appearance and skill-effect variants using edit-specific prompts. Screening checks the requested content, temporal coherence, and interaction consistency.
Appearance-only variants are checked against the source video for fighter trajectories, event timing, and health traces. Only edits that pass these checks reuse the source state annotations. Skill-effect edits are annotated from the edited videos.
State and Event Annotation
Gemma-4-31B-it localizes fighters at regularly spaced keyframes to initialize SAM 2 video propagation [29]. Semantic descriptions help distinguish fighters from similar background characters. At each keyframe, the tracker checks the two identity assignments against appearance and trajectory continuity, retaining or swapping them to correct drift and bridge short gaps. Bounding boxes specify spatial state. Facing direction follows relative fighter positions and retains its previous value while the boxes overlap.
Health traces are estimated from HUD bar lengths and cleaned within each round. Nearby decreases are merged into damage events, each recording the affected fighter and event time. Gemma also supplies coarse action and stance labels at selected frames, including frames near damage events. Unannotated action frames use the unknown label. Logged inputs provide fine attack commands and scheduled skill triggers for the controlled fighter. The compiler converts these annotations into interaction proxies.
Evaluation Protocols
Controlled Interactions
Recorded-interaction tests use held-out video annotations, with source episodes separated between training and testing. Rule interventions rerun the engine from the same initial state and submitted actions under each configuration, recomputing motion trajectories, hit events, and state outcomes. These records define the target interactions for the compared encodings. In FWM’s damage scaling, limited-hit immunity, and lifesteal tests, reference images and text condition remain fixed. Changed outcomes reach FWM only through the proxy.
Appearance interventions retain the executed records and proxy while changing the references. Evaluation cases include no-hit attempts, repeated attacks, overlap, and side switches.
Complete-Information Encodings. The encodings use matched event-free intervals, sampled values, and temporal indices, with shared resolution and sampling settings. Each uses the input layers required by its representation, and text inputs are checked for truncation.
Action Recognition and Timing
Movement. MoveAcc compares tracked fighter motion with directional commands over valid command windows and averages across the four movement keys. Jump positives require an actual airborne jump. Grounded hurt recovery is excluded from jump positives.
Attack and Skill Recognition. We train ResNet-18-based temporal convolutional networks (TCNs) for attack and skill recognition. Attack detections are grouped into temporally contiguous events. AttackAcc measures the fraction of detected punch or kick events supported by the corresponding input command within its temporal window.
The skill classifier uses a 256-channel temporal trunk with dilation rates 1, 2, 4, and 8. It predicts onset and class, with a no-skill outcome in the onset prediction. Class logits are pooled over the predicted effect interval. SkillAcc requires the correct class and onset within the command-relative window. The same timing rule applies across methods and accommodates skill startup, release, and recovery.
An attempted attack may hit or miss, so AttackAcc and Reaction assess command-grounded action execution and received-hit consequences, respectively.
Reaction and Health
Visible Reactions. Reaction measures the fraction of expected hit responses visibly performed by the designated fighter within the evaluation window. Qwen judges body motion in surrounding gameplay frames, using a source-frame identity guide and excluding the HUD.
Brief reactions can resemble ordinary motion across appearances and effects, so we judge them using local temporal context around expected events. Computing reaction precision would additionally require locating every visible received-hit onset throughout the generated video, a dense detection task that is difficult for current VLMs.
Health and Persistence. Let contain readable fighter-frame observations and denote maximum health. Health MAE is the normalized mean absolute error, expressed in percentage points:
Observed and target health values are obtained from HUD bar lengths. The joint health check uses a one-pixel tolerance and requires correct updates for both fighters, with the resulting values preserved between events. This includes unchanged health under immunity and the attacker’s restored health under lifesteal.
Joint Interaction Fidelity
An attempt succeeds only when all applicable action, entity, timing, reaction, and health checks pass. The score averages attempts within each episode and weights episodes equally:
Here counts episodes, counts attempts in episode , and are binary checks indexed by . The applicable set follows the registered interaction. Its checks cover command-grounded actions and timing, the identities of acting and affected fighters, visible body responses, and correct health updates with persistence. For rule interventions, combines the required reaction and health checks. Both rates are expressed as percentages in the results.
Appearance and Entity Assignment
Adherence assesses appearance-edit success for the designated character or arena. A VLM judges sampled continuation frames using the target reference and a source-frame identity guide. Frame judgments are combined into a clip-level decision.
DINO measures reference-to-generation feature cosine similarity separately for characters and arenas. Proxy boxes localize fighter crops, while fighter and HUD regions are suppressed for arena scoring. The fixed initial context is excluded from both appearance metrics.
Reference-Augmented Baselines
We augment H3-WORLD [5], ReactiveGWM v1 [37], and ReactiveGWM v2 [38] with our reference-conditioning branch. Variants are evaluated on the same clips and frame indices as FWM, using explicit appearance references. The comparisons in Figure 11 to Figure 14 show character drift or garment blur during rapid contact and side switches, together with missed (↓), premature (‼), or incomplete (≠) health updates. FWM preserves appearance and the prescribed health updates more consistently in these examples.
Figure 11. Reference-Augmented Comparison (Saitama, Player, No Side Switch). Matched frames corresponding to row 1 of Figure 5; HUD state icons follow Figure 6.

Figure 14. Reference-Augmented Comparison (Superman, NPC, Side Switch). Matched frames corresponding to row 4 of Figure 5; HUD state icons follow Figure 6.
Figure 12. Reference-Augmented Comparison (Naruto, Player, Side Switch). Matched frames corresponding to row 2 of Figure 5; HUD state icons follow Figure 6.

Figure 13. Reference-Augmented Comparison (Lava Humanoid, NPC, No Side Switch). Matched frames corresponding to row 3 of Figure 5; HUD state icons follow Figure 6.
References
- [1]Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for world modeling: Visual details matter in Atari. In NeurIPS, 2024.arxiv.org/abs/2405.12399
- [2]Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In ICML, 2024.arxiv.org/abs/2402.15391
- [3]Ziqi Cai, Siqi Yang, Yimu Wang, Zixian Gao, Yunheng Liu, Shuchen Weng, Erwin Wu, Kaipeng Zhang, and Boxin Shi. MASS: Multiplayer world models with authoritative shared state, 2026.arxiv.org/abs/2608.06257
- [4]Capcom. Street fighter II. Video game, 1991.
- [5]Danze Chen, Zeqing Wang, Ziyue Lin, Xingyi Yang, and Yeying Jin. H3-World: Turning language understanding into world control, 2026.arxiv.org/abs/2609.01560
- [6]Yiwen Chen, Guosheng Lin, and Chi Zhang. Code world model: Coding agent as world brain, 2026. Project page: https://buacxyw.github.io/cwm/.
- [7]Decart and Etched. Oasis: A universe in a transformer. https://oasis-model.github.io/, 2024.
- [8]Zhiyang Deng, Boran Zhang, Danze Chen, and Yeying Jin. WorldMind: Decoupled game world model for state-aware NPC behavior, 2026.
- [9]Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, et al. Infinite worlds with versatile interactions, 2026. LingBot-World 2.0 / LingBot-World-Infinity.arxiv.org/abs/2607.07534
- [10]David Ha and Jürgen Schmidhuber. World models, 2018.DOI
- [11]Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang, Yifan Zhang, et al. Matrix-game 2.0: An open-source, real-time, and streaming interactive world model, 2025.arxiv.org/abs/2508.13009
- [12]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022.arxiv.org/abs/2207.12598
- [13]Fa-Ting Hong, Zunnan Xu, Zixiang Zhou, Jun Zhou, Xiu Li, Qin Lin, Qinglin Lu, and Dan Xu. Audio-visual controlled video diffusion with masked selective state spaces modeling for natural talking head generation. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 12549–12558. IEEE, 2025.arxiv.org/abs/2504.02542
- [14]Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, and Liefeng Bo. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In CVPR, 2024.arxiv.org/abs/2311.17117
- [15]Li Hu, Guangyuan Wang, Peng Zhang, and Bang Zhang. WanToFight: Real-time generative game engine for multiplayer game content, 2026.
- [16]Zheng-Hui Huang, Guixu Lin, Jiacheng Lin, Yi-Chuan Huang, Ruihan Yu, Muyao Niu, Siqi Yang, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang, and Zhixiang Wang. Programmable world model, 2026. v2, 13 Oct 2026. https://alaya-lab.github.io/pwm/.arxiv.org/abs/2609.10540
- [17]Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. VACE: All-in-one video creation and editing. In ICCV, 2025.arxiv.org/abs/2503.07598
- [18]Diederik P. Kingma and Max Welling. Auto-encoding variational Bayes. In ICLR, 2014.arxiv.org/abs/1312.6114
- [19]Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. HunyuanVideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024.arxiv.org/abs/2412.03603
- [20]Guannan Li, Jiaj Chen, Jingyuan Liao, Yu Geng, and Baolan Qiu. MSR: Multiple subject reference for video generation, 2026.arxiv.org/abs/2609.18393
- [21]Zijun Lin, Zeqing Wang, Cheston Tan, Bihan Wen, and Yeying Jin. StatePlay: State-aware game world models for mechanics-consistent generation, 2026.arxiv.org/abs/2607.26754
- [22]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In ICLR, 2023.arxiv.org/abs/2210.02747
- [23]Jijie Liu, Tianxiang Ma, Binchuan Li, Zhuowei Chen, Jiawei Liu, Gen Li, Siyu Zhou, Qian He, and Xinglong Wu. Phantom: Subject-consistent video generation via cross-modal alignment, 2025.arxiv.org/abs/2502.11079
- [24]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023.arxiv.org/abs/2209.03003
- [25]Ge Ya Luo, Zhi Hao Luo, Anthony Gosselin, Alexia Jolicœur-Martineau, and Christopher Pal. Ctrl-V: Higher fidelity video generation with bounding-box controlled object motion, 2024.arxiv.org/abs/2406.05630
- [26]Mingshuang Luo, Shuang Liang, Zhengkun Rong, Yuxuan Luo, Tianshu Hu, Ruibing Hou, Hong Chang, Yong Li, Yuan Zhang, and Mingyuan Gao. DreamActor-M2: Universal character image animation via spatiotemporal in-context learning, 2026.
- [27]Jack Parker-Holder and Shlomi Fruchter. Genie 3: A new frontier for world models. Google DeepMind blog, https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/, 2025.
- [28]Alexander Pondaven, Ziyi Wu, Igor Gilitschenski, Philip Torr, Sergey Tulyakov, Fabio Pizzati, and Aliaksandr Siarohin. ActionParty: Multi-subject action binding in generative video games. In ECCV, 2026.arxiv.org/abs/2604.02330
- [29]Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, et al. SAM 2: Segment anything in images and videos, 2024.arxiv.org/abs/2408.00714
- [30]Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, et al. Advancing open-source world models, 2026.arxiv.org/abs/2601.20540
- [31]Georgy Savva, Oscar Michel, Daohan Lu, Suppakit Waitlikhith, Timothy Meehan, Dhairya Mishra, Srivats Poddar, Jack Lu, and Saining Xie. Solaris: Building a multiplayer video world model in Minecraft, 2026.arxiv.org/abs/2602.22208
- [32]Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer: Enhanced transformer with rotary position embedding. Neurocomputing, 568: 127063, 2024.arxiv.org/abs/2104.09864
- [33]Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. In ICLR, 2025.arxiv.org/abs/2408.14837
- [34]Wan Team, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models, 2025.arxiv.org/abs/2503.20314
- [35]Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guoqiang Wei, Liping Yuan, and Hang Li. Boximator: Generating rich and controllable motions for video synthesis, 2024.arxiv.org/abs/2402.01566
- [36]Zhouxia Wang, Ziyang Yuan, Xintao Wang, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. MotionCtrl: A unified and flexible motion controller for video generation. In SIGGRAPH, 2024.arxiv.org/abs/2312.03641
- [37]Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, and Yeying Jin. ReactiveGWM: Steering NPC in reactive game world models, 2026.
- [38]Zeqing Wang, Danze Chen, Zhaohu Xing, Zizhao Tong, Yinhan Zhang, Xingyi Yang, and Yeying Jin. ReactiveGWM: Flexible control and NPC reactivity in game world models, 2026. Code: https://github.com/INV-Wzo/ReactiveGWM.arxiv.org/abs/2605.15256
- [39]Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, Peiyu Wang, Biao Jiang, Yichen Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory, 2026. Skywork AI. Weights: https://huggingface.co/Skywork/Matrix-Game-3.0.arxiv.org/abs/2604.08995
- [40]Zunnan Xu, Zhentao Yu, Zixiang Zhou, Jun Zhou, Xiaoyu Jin, Fa-Ting Hong, Xiaozhong Ji, Junwei Zhu, Chengfei Cai, Shiyu Tang, et al. HunyuanPortrait: Implicit condition control for enhanced portrait animation. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15909–15919. IEEE, 2025.
- [41]Wenhao Yan, Fengjia Guo, Zhuoyi Yang, and Jie Tang. SCAIL-2: Unifying controlled character animation with end-to-end in-context conditioning, 2026.arxiv.org/abs/2606.10804
- [42]Xindi Yang, Baolu Li, Liam Lee, Zhenfei Yin, Songxin Zhang, Zhuoyang Song, Xu Jia, Jianfei Cai, et al. Oneira: From open-ended generation to open-world interaction in video world models, 2026.arxiv.org/abs/2610.01614
- [43]Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Di Zhang, and Xihui Liu. GameFactory: Creating new games with generative interactive videos. In ICCV, 2025.arxiv.org/abs/2501.08325
- [44]Xiaoyu Zhan, Xinyu Wang, Xiaohong Zhang, Huanjie Zhu, Tengjiao Sun, Pengcheng Fang, Jiaxing Yu, Yanwen Guo, and Dongjie Fu. Magpie: Real-time world renderer for interactive games, 2026.arxiv.org/abs/2608.27168
- [45]Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023.arxiv.org/abs/2302.05543
- [46]Ruicheng Zhang, Mingyang Zhang, Jun Zhou, Xiaofan Liu, Zunnan Xu, Zhizhou Zhong, Puxin Yan, Haocheng Luo, and Xin Li. MIND-V: Hierarchical world model for long-horizon robotic manipulation with RL-based physical alignment. arXiv preprint arXiv:2512.06628, 2025.arxiv.org/abs/2512.06628
- [47]Ruicheng Zhang, Guangyu Chen, Zunnan Xu, Zihao Liu, Zhizhou Zhong, Mingyang Zhang, Jun Zhou, and Xiou Li. RoboStereo: Dual-tower 4D embodied world models for unified policy optimization. arXiv preprint arXiv:2603.12639, 2026.arxiv.org/abs/2603.12639
- [48]Yifan Zhang, Chunli Peng, Boyang Wang, Puyi Wang, Qingcheng Zhu, Fei Kang, Biao Jiang, Zedong Gao, Eric Li, Yang Liu, and Yahui Zhou. Matrix-game: Interactive world foundation model, 2025.arxiv.org/abs/2506.18701
- [49]Yong Zhang, Zhuoyi Yang, Jiayan Teng, Xiaotao Gu, and Chongxuan Li. Concat-ID: Towards universal identity-preserving video synthesis, 2025.
- [50]Zixiang Zhou, Zhentao Yu, Yifeng Ma, Hongmei Wang, Wenqing Yu, Cong Wang, Zilin Yang, Rui Chen, Jiarong Ou, Yezhou Liu, Yuan Zhou, and Qinglin Lu. Aura: Consistent multi-subject video generation via VLM-grounded semantic alignment, 2026.arxiv.org/abs/2607.04311
- [51]Shangwen Zhu, Qianyu Peng, Zhao Pu, Zhilei Shu, Xiangrui Ke, Zhaohui Xing, Zizhao Tong, Zeqing Wang, et al. Incantation: Natural language as the action interface for multi-entity video world models, 2026.arxiv.org/abs/2605.18601








