WorksComputer Science
TouchéWorld tech report : Towards Online Unified Camera–Hand Exploration in Generative Worlds
Abstract
Recent video world models allow users to explore generated environments through camera navigation. Continuous hand motion provides a fine-grained action channel for interacting with the people and objects encountered during exploration. We present TouchéWorld, a framework that integrates gesture-driven interaction into a camera-controlled video world model. Given an initial image and a scene-description prompt, the model generates video conditioned on temporally aligned camera and hand controls. We construct joint camera–hand training data from heterogeneous egocentric videos through annotation completion, consistency-based filtering, and temporal alignment. Building on a pretrained camera-controlled world model, we combine spatial hand-skeleton conditioning with numerical hand-motion features through parameter-efficient adaptation. Causal adaptation and few-step distillation support chunk-level online generation, allowing users to update their controls based on generated visual feedback. We further introduce TouchéBench to evaluate hand-motion following, camera–hand coordination, online interaction, and object response. Together, these components establish a framework for studying continuous exploration and hand-driven interaction within generative worlds.
Project Page Model Weights Dataset Code

Figure 1. From camera-guided exploration to continuous hand–world interaction. Camera control allows users to navigate generated environments, while hand motion provides a fine-grained action channel for interacting with encountered scene contents. In the illustrated example, the user approaches an osmanthus tree and reaches into its canopy, producing hand contact and corresponding changes in the foliage. With scene-only text conditioning, camera and hand inputs guide the unfolding interaction, while causal chunkwise generation incorporates updated controls into subsequent visual output.
Introduction
Video world models are turning generated visual content into environments that users can explore. Advances in camera-conditioned generation [15, 23, 47] and streaming inference [53, 54] allow users to move through scenes, choose their viewpoints, and explore worlds as they unfold [19, 50, 53, 54]. Exploration, however, naturally invites participation. Users may wish not only to observe what they encounter, but also to act on it. Consider walking through a generated courtyard, approaching a sweet-scented osmanthus tree, reaching out to touch its branches, and then continuing along the path. Supporting this process requires camera-guided navigation and hand-driven interaction to work together, with the generated world responding to user actions as exploration continues.
Continuous hand motion provides a natural interface for such participation. Camera control determines how users observe the world, while hand position, orientation, and articulation provide a fine-grained action channel for influencing its contents. Users can express actions such as reaching, touching, pushing, and withdrawing, and adjust their subsequent behavior based on generated visual feedback. We organize this process as online interaction: the system receives user controls while producing visual output and incorporates updated controls into subsequent video chunks, allowing users to decide their next action through ongoing observation.
Recent work provides important foundations through hand–object dynamics generation, joint hand and camera control, and streaming interaction [21, 39, 41]. The supported range of scenes and interactions nevertheless remains limited, particularly when manipulation is embedded within continuous navigation and exploration. We focus on the continuity across these activities: a user navigates toward an object, reaches out to manipulate it, withdraws their hand, and resumes exploration. Across these transitions, viewpoint control, hand motion, and scene responses must remain coordinated, while subsequent generation carries forward the state established by earlier actions. This calls for jointly considering the data, generation process, and evaluation of continuous camera–hand interaction.
Three challenges arise. First, joint control requires reliable supervision that separates viewpoint changes from action-induced scene dynamics. Camera motion, hand articulation, and their visual consequences must be accurately annotated and temporally aligned, despite incomplete or inconsistent estimates in egocentric videos. Second, online generation requires causal processing of both controls and visual history: each generated chunk must respond to available camera and hand inputs without relying on future actions. Third, interaction quality extends beyond visual similarity. A generated hand may follow its controls while the object remains unresponsive, and camera motion may produce apparent hand–object alignment without the intended manipulation. Evaluation must therefore distinguish hand following, joint control, and action-dependent scene responses.
To this end, we present a framework that integrates continuous hand interaction into a camera-controlled video world model. We first construct joint camera–hand training data from heterogeneous egocentric videos, completing missing annotations, filtering camera and hand estimates through consistency checks. Building on a pretrained camera-controlled model, we combine spatial hand-skeleton conditioning with numerical motion features through parameter-efficient adaptation. We further impose consistent causal constraints on hand conditioning and visual generation, and apply few-step distillation to accelerate chunk-wise synthesis. The resulting model incorporates updated camera and hand controls into subsequent chunks, connecting exploration and manipulation within a shared generated history.
We further introduce TouchéBench to evaluate four aspects of interactive world generation: hand-motion following, camera–hand coordination, online interaction, and object response. Reference-based evaluation on HOI4D [25] measures hand-control accuracy and object-response fidelity, while purpose-recorded scenarios assess joint camera–hand control and action-dependent continuations from shared contexts. Alongside TouchéBench, we evaluate video quality on unseen scenes from EgoStandard [24] and EgoLife [44], and on the out-of-domain HOT3D dataset [2]. Comparisons with external methods and across our bidirectional teacher, causal model, and four-step distilled model characterize visual quality and interaction performance across generation stages. Ablation studies further examine the effects of hand-weighted training on hand following and interaction outcomes.
Our contributions are threefold:
Joint camera–hand training data. We construct a data pipeline that completes missing camera and hand annotations, filters estimates through consistency checks, and aligns video and control signals to provide joint supervision for camera-guided exploration and hand-driven interaction.
A framework for online camera–hand interaction. We extend a camera-controlled video world model with spatial skeleton and numerical motion conditioning. Parameter-efficient adaptation, causal training, and few-step distillation enable chunk-level online generation with continuously updated camera and hand controls.
TouchéBench and experimental evaluation. We introduce a benchmark covering hand-motion following, camera–hand coordination, online interaction, and object response. Comparisons and ablation studies evaluate these capabilities alongside video quality and examine the effects of causal adaptation, few-step distillation, and hand-weighted training.
Related work
Action-Conditioned Video World Models
Action-conditioned video world models enable generated environments to respond to user control and evolve continuously by incorporating action signals into the video generation process. While early works primarily utilized discrete keystrokes or latent actions as control signals [15, 30, 51], recent research has introduced continuous camera trajectories to support more flexible navigation and viewpoint changes [6, 10, 18, 32, 34, 35, 50, 52]. To link camera motion with spatial changes in the generated imagery, existing methods have explored various geometric representations and conditioning mechanisms. For instance, LingBot-World 2.0 [10] employs Plücker embeddings to encode pixel-level observation rays and injects camera conditions via adaptive layer normalization; SolarWM [18] utilizes fused-PRoPE [23] to integrate projection geometry transformations, derived from camera intrinsics and extrinsics into self-attention computations; and EchoWM [50] adopts relative UCPE [47] to encode geometric relationships between local ray coordinate systems, thereby supporting continuous trajectory control.
Beyond viewpoint navigation, recent work explores control [4, 12, 13, 36] interfaces for character behavior, scene events, and object dynamics. GameGen-X [6] and Hunyuan-GameCraft-2 [33] combine instruction-based control with game interaction, enabling users to specify actions and changes to scene contents. LongLive enables interactive streaming video generation and introduces KV-recache to refresh cached states when text prompts change [45]. H3-World [7] explores language-based action control on MiniMax-H3 [26] by converting character and camera actions into temporally aligned textual instructions [7]. Complementary approaches introduce spatial control signals: MotionStream [31] supports interactive motion guidance, while Force Prompting [11] and StreamForce [37] condition generation on physical forces, with StreamForce extending force control to causal streaming generation. Code World Model explores an alternative formulation in which a coding agent maintains executable world states and compiles them into proxy videos that condition visual generation [8]. These studies broaden the ways users can influence generated worlds. Our work investigates continuous hand motion as an action interface, jointly with camera control, to connect exploration and manipulation within an ongoing generation process.
Hand-Conditioned Video Generation and Interaction Evaluation
Hand-conditioned video generation models how hand motion influences scene contents. Inter-Dyn [1] conditions video generation on driving-motion cues, including hand-mask sequences, to synthesize the resulting interaction dynamics. Dexterous World Models [21] combines hand-mesh renderings with static-scene videos rendered along a specified camera trajectory to model action-induced scene changes. Recent egocentric approaches further investigate joint hand and viewpoint control: Generated Reality [41] conditions generation on tracked head and hand poses and supports autoregressive interaction. HandsOnWorld [9] learns from unconstrained monocular videos and introduces camera-disentangled hand control. Hand2World [39] and JCV [48] primarily study hand–object interaction in tabletop manipulation settings. While these settings capture local interactions, they provide limited coverage of the transitions between navigation and manipulation. Related interaction-generation methods also investigate structural supervision. sion: SCAR jointly generates videos and structure- and contact-aware representations to improve hand–object interaction synthesis [43]. Building on these developments, we focus on connecting camera-guided exploration and hand-driven manipulation within a continuous generation process, supported by joint camera–hand data construction and causal control.
Interaction evaluation examines both control fidelity and the consequences of actions. Existing methods assess visual quality, hand-motion agreement, and generated interaction behavior under their respective conditioning settings [1, 21, 39, 41]. Complementing control-conditioned evaluation, Ego2Act evaluates goal-directed egocentric manipulation from an initial image and a high-level goal, assessing task completion and physical plausibility [20]. Continuous camera–hand interaction additionally calls for evaluating how newly introduced controls affect subsequent generation and how hand actions operate alongside navigation. TouchéBench brings these aspects together through hand-motion following, camera–hand coordination, online intervention response, and object response. Paired trajectory evaluation and action interventions from shared contexts relate control accuracy to the resulting interaction outcomes.
Method
We present a video world modeling framework that integrates continuous hand interaction with camera-guided exploration. Building on a camera-controlled image-to-video model, we introduce complementary skeleton and motion conditions, followed by causal adaptation and few-step distillation. Figure 2 illustrates joint data construction, while Figure 3 summarizes conditioning, training, and online interaction.

Figure 2. Joint camera–hand data construction through annotation completion, consistency-based filtering, and temporal alignment.

Figure 3. Overview of TouchéWorld: joint camera–hand conditioning, causal adaptation, few-step distillation, and online interaction with updated controls.
Problem Formulation
Given an initial image and a scene-description prompt , we generate a continuous video conditioned on camera controls and hand controls . Camera controls specify the viewing trajectory, while hand controls specify hand poses and motion at the corresponding video time positions. Together, they allow users to navigate the generated environment and interact with its contents.
We generate video chunks sequentially:
where and denote the controls available when generating chunk . Within each chunk, hand-control latents are temporally aligned with the corresponding video latent positions.
The system receives user controls while producing visual output and incorporates updated controls into subsequent chunks. This chunk-level online interaction allows users to adjust camera movement and hand actions based on generated visual feedback, connecting exploration and interaction within a shared video history.
Data Construction
As shown in Figure 2, we construct aligned video–camera–hand data from heterogeneous egocentric videos. We retain valid existing annotations and complete missing camera and hand annotations through separate estimation and validation branches.
For videos without camera annotations, we estimate camera poses using ViPE [17] and depth using MoGe2 [38]. We reproject source frames into target views using the estimated geometry and compare them with the observed target frames. Reprojection consistency and trajectory-quality checks are used to filter unreliable camera estimates.
For videos without hand annotations, we independently estimate hand poses using MINT [55] and HandFlow [42] and reject samples with inconsistent predictions. The retained annotations are converted into skeleton renderings and numerical hand features for joint conditioning.
We temporally align video frames, camera poses, and hand annotations, updating camera intrinsics and image-space hand coordinates consistently with resizing and cropping. Camera trajectories are expressed relative to the initial frame. Alongside interaction clips, we include clips with hands entering an initially hand-free view and static clips with and without hands, covering both interaction and periods without hand motion.
We plan to release the constructed dataset to support further research on joint camera–hand control in video world models.
Joint Camera–Hand Conditioning
Camera conditioning. We build on the LTX-2.5 [14] image-to-video backbone with a SolarWM camera adapter. Camera intrinsics and relative extrinsics are incorporated into self-attention through PRoPE. We retain this camera-conditioning pathway while introducing two complementary branches for hand control.
Hand conditioning. The spatial branch encodes rendered skeleton videos with the video VAE [22] and projects their latents into the video-token space. The motion branch encodes numerical hand features , consisting of wrist translation, joint geometry, and a validity mask, using a temporal encoder with linear projections and a residual temporal convolution. Given noisy video latents , we form the Transformer input as
where denotes patchification, and are the video and skeleton projections, and broadcasts each motion embedding across the spatial tokens at the corresponding latent time position.
The two branches provide spatially localized skeleton features and temporally contextualized motion features.
Joint adaptation. We freeze the pretrained backbone and camera LoRA[16], and jointly optimize an additional hand LoRA with the two hand-conditioning branches. The skeleton projection and the motion encoder’s output projection are bias-free and zero-initialized, so the added conditioning residual is initially zero.
To emphasize hand reconstruction while retaining scene-wide supervision, we use a weighted denoising loss:
where and are the prediction and denoising target for video token , respectively. The binary mask identifies hand-region tokens, excludes the initial conditioning latent, and controls the additional hand weight. Setting recovers uniform weighting over valid tokens.
Causal Adaptation and Asymmetric DMD
We obtain a streaming generator in two stages: teacher-forced causal adaptation, followed by asymmetric Distribution Matching Distillation (DMD) [46] from a bidirectional teacher to obtain a few-step causal generator.
Teacher-forced causal adaptation. We initialize from the bidirectional joint-control teacher, freeze the backbone, camera and hand LoRAs, and both hand-conditioning branches, and train only an additional causal-adaptation LoRA. We adopt bidirectional attention within each chunk and causal attention across chunks. Each chunk is jointly denoised with access to its own tokens and the available preceding chunks. During teacher-forced training, preceding video chunks are supplied from ground-truth sequences. We also use left-padded temporal convolutions in the motion encoder and restrict skeleton conditions to the available temporal range to prevent information leakage from future chunks.
Asymmetric DMD with self-forcing. We initialize the few-step student from the causally adapted model and distill it using the frozen bidirectional joint-control teacher. During distillation, the student performs self-forcing [19] rollouts, conditioning each chunk on its own generated history:
where denotes the complete few-step sampling procedure, denotes its sampling noise, and contains the initial image, scene-description prompt, and camera–hand controls available when generating chunk .
We apply Distribution Matching Distillation (DMD) to align the student and teacher distributions through a noise-averaged reverse KL objective. Given the generated sequence , we construct , where . A trainable fake-score network estimates the student distribution score, while the frozen teacher provides . Their difference yields the generator update:
where , denotes the conditioning information for the full distillation sequence, and incorporates the noise weighting and the perturbation factor . Here, and denote the student-distribution and teacher-distribution scores, respectively, expressed in a common score parameterization.
Online inference. The distilled student generates video sequentially using four denoising steps per chunk. Each chunk conditions on the initial frame, a bounded history of generated chunks, and the available camera and hand controls. Committed outputs remain unchanged. The interface continues to receive updated controls during generation and incorporates them into subsequent chunks, allowing users to adjust their actions based on visual feedback.
TouchéBench
We introduce TouchéBench to evaluate four aspects of camera–hand-controlled world generation: hand-motion following, camera–hand coordination, online interaction, and object response (Figure 4). The benchmark combines reference-based evaluation on 200 held-out HOI4D clips with purpose-recorded scenarios for joint camera–hand control and action-dependent continuations. Hand-motion following and object response are evaluated on the same HOI4D clips and generated outputs, allowing control accuracy and object-response fidelity to be assessed together.

Figure 4. TouchéBench evaluates hand-motion following (Q1), camera–hand coordination (Q2), online interaction (Q3), and object response (Q4).
Q1: Hand-motion following. We condition generation on the hand trajectories associated with each HOI4D clip and estimate hand keypoints from the generated frames. Following temporal alignment and hand correspondence matching, we compare detected joints with reference joints in a common normalized image coordinate system. We report the percentage of correct keypoints:
where contains valid reference joints, indicates a matched detection, and is the normalized distance threshold. We report PCK at , counting missing detections as incorrect. Joint localization error on matched detections and detection coverage are reported as supplementary measures. Scores are computed per clip and averaged across clips; the initial conditioning frame is excluded.
Q2: Camera–hand coordination. We evaluate joint camera–hand control on ten purpose-recorded clips combining camera movement with hand interaction. Each clip uses first-frame image-to-video initialization and the camera and hand inputs supported by each method; JCV receives hand conditioning without camera control. GPT-6 Astra [28], accessed through Codex [27], assesses each continuation using six binary checklist questions covering visual consistency, viewpoint attainment, interaction execution, and the final object state. We report Camera as the viewpoint-attainment pass rate and Interaction as the average pass rate of interaction-execution and final-state judgments. Scores are averaged equally across clips.
Q3: Online interaction. We evaluate action-dependent continuations using eight purpose-recorded scenarios, each containing three alternative hand-action branches following a shared approximately two-second context. The scene-description prompt remains unchanged across branches and does not specify the target action. Each branch supplies its corresponding hand controls to test whether the generated continuation realizes the intended interaction and distinguishes it from alternative outcomes. GPT-6 Astra [28], accessed through Codex [27], assesses the 24 continuations using six binary checklist questions per continuation, yielding 144 judgments. Three basic criteria assess visual and scene consistency, while three hard criteria assess action execution (), the intended final state (), and exclusion of alternative outcomes ().
We report the mean pass rates for Basic, Hard, and Overall.
Q4: Object response. Using the same HOI4D-200 evaluation set, we compare target-object responses in generated and reference videos. We obtain temporally corresponding object masks with SAM3 [5] and evaluate mask overlap and centroid displacement:
where and denote generated and reference object masks, respectively, and denotes the mask centroid in image coordinates. We report mask IoU and centroid displacement in pixels. Frame-level measurements are averaged within each clip and then equally across clips, excluding the initial conditioning frame. Target-tracking coverage is reported separately.
Experiments
Experimental Setup
Datasets. We train our model on 40,958 egocentric video clips, comprising 24,000 EgoLife [44] clips, 15,918 EgoStandard clips, and 1,040 auxiliary clips. The auxiliary set includes hand-emergence clips, in which hands are removed from the initial frame and appear in subsequent chunks, as well as static clips with and without hands. We evaluate on EgoStandard-200 [24], EgoLife-300 [44], and HOT3D-45 [2]. All three test sets contain scenes unseen during training. EgoStandard and EgoLife evaluate generalization to unseen scenes within the training domains, while HOT3D provides an out-of-domain evaluation. For HOT3D, we follow the original 81-frame configuration.
Baselines. Few existing methods closely match our joint camera–hand control setting. We compare against the following models:
LTX-2.5 I2V and the camera-only SolarWM model, which serve as baselines for video generation quality.
JCV [48] and Hand2World [39], two related hand-conditioned video generation models with publicly available weights. We use their released weights without additional training or fine-tuning, adapting the input representation, resolution, and evaluation interface to our test protocol. Each method receives the inputs supported by its original implementation. In particular, JCV does not expose a camera-control interface and therefore receives hand conditioning only. For Hand2World, we project 3D MANO meshes to construct the hand-conditioning input used by its original pipeline. We use the same evaluation examples and scoring implementation across methods, with method-specific context initialization and temporal alignment described below.
We report our bidirectional teacher, the multi-step causal model obtained through teacher-forced adaptation, and the four-step causal model obtained through self-forcing DMD to examine video quality and interaction performance across training stages.
Evaluation protocol. We evaluate video quality using PSNR, SSIM [40], LPIPS [49], DINOv2 [29] similarity, and KID [3], excluding the initial conditioning frame. The four paired metrics are averaged within each clip and then equally across clips, while KID measures distributional similarity over sampled frames. All methods are evaluated using a shared scoring implementation. We additionally assess hand-motion following, camera–hand coordination, online interaction, and object response using TouchéBench.
Video Generation Quality
Table 1 compares video generation quality across two in-domain test sets with unseen scenes and the out-of-domain HOT3D-45 benchmark. Our bidirectional teacher achieves strong results across all three datasets. Relative to camera-only SolarWM, it improves PSNR by 2.82 dB on EgoStandard-200 and 1.66 dB on EgoLife-300, while reducing LPIPS from 0.494 to 0.345 and from 0.584 to 0.488, respectively. On HOT3D-45, it achieves an LPIPS of 0.488 and a DINOv2 similarity of 0.769. These results show improved agreement with reference observations across both in-domain and out-of-domain scenes. Figure 5 further illustrates the visual results across diverse scenes, where the same camera and hand controls are reused with different initial images and scene descriptions.

Figure 5. Generation across scenes using shared camera and hand controls from an osmanthus-tree interaction. The final case compares methods on the store-shelf scene. Frames are sampled at matched relative progress.
| Method | EgoStandard-200 | EgoLife-300 | HOT3D-45 (OOD) | ||||||||||||
| PSNR↑ | SSIM↑ | LPIPS↓ | DINOv2↑ | KID↓ | PSNR↑ | SSIM↑ | LPIPS↓ | DINOv2↑ | KID↓ | PSNR↑ | SSIM↑ | LPIPS↓ | DINOv2↑ | KID↓ | |
| Bidirectional | |||||||||||||||
| LTX-2.5 I2V | 15.95 | 0.615 | 0.551 | 0.724 | 0.00887 | 13.61 | 0.473 | 0.635 | 0.537 | 0.00648 | 14.04 | 0.517 | 0.558 | 0.707 | 0.01766 |
| SolarWM (camera-only) | 16.36 | 0.643 | 0.494 | 0.728 | 0.01732 | 14.53 | 0.527 | 0.584 | 0.542 | 0.01033 | 14.77 | 0.543 | 0.513 | 0.719 | 0.01154 |
| JCV | 16.63 | 0.643 | 0.486 | 0.757 | 0.00308 | 13.47 | 0.473 | 0.647 | 0.486 | 0.01533 | 12.64 | 0.470 | 0.634 | 0.726 | 0.02330 |
| Hand2World | 13.81 | 0.619 | 0.635 | 0.402 | 0.07425 | 12.08 | 0.468 | 0.692 | 0.354 | 0.02642 | 13.63 | 0.512 | 0.553 | 0.600 | 0.07396 |
| Ours (bidirectional teacher) | 19.18 | 0.708 | 0.345 | 0.816 | 0.00265 | 16.19 | 0.572 | 0.488 | 0.652 | 0.00657 | 15.11 | 0.552 | 0.488 | 0.769 | 0.00617 |
| Causal | |||||||||||||||
| Ours (causal) | 17.60 | 0.666 | 0.410 | 0.785 | 0.00367 | 15.05 | 0.519 | 0.556 | 0.582 | 0.00876 | 14.74 | 0.525 | 0.500 | 0.750 | 0.00747 |
| Ours (causal-4 step) | 18.38 | 0.696 | 0.408 | 0.742 | 0.00852 | 16.04 | 0.578 | 0.589 | 0.437 | 0.00746 | 15.17 | 0.556 | 0.499 | 0.732 | 0.01180 |
Table 1. Video quality on unseen in-domain scenes and out-of-domain HOT3D. All our variants use hand-weighted loss. Best results are bold; KID is unscaled.
Our multi-step causal model also achieves lower LPIPS and higher DINOv2 similarity than SolarWM on all three datasets, while its PSNR improves or remains comparable. The four-step distilled model further improves PSNR and SSIM over the multi-step causal model across all three datasets, with mixed changes in LPIPS, DINOv2 similarity, and KID. These results characterize the visual-quality trade-offs introduced by causal adaptation and few-step distillation.
We next evaluate hand-motion following, camera–hand coordination, online interaction, and object response using TouchéBench to examine how these models translate visual generation quality into controllable interaction.
Evaluation on TouchéBench
Table 2 reports results on the four dimensions of TouchéBench. Our bidirectional teacher achieves the highest hand-following [email protected] of 65.8%, compared with 61.2% for Hand2World and 59.1% for JCV, while our multi-step and four-step causal models obtain 57.9% and 54.0%, respectively. Hand2World achieves the lowest localization error on matched detections at 28.2 pixels. For camera–hand coordination, our teacher achieves 100.0% Camera and 60.0% Interaction pass rates, compared with 50.0% and 15.0% for JCV, and 40.0% and 15.0% for Hand2World. Our causal models retain this advantage, with the multi-step model scoring 70.0% and 40.0%, and the four-step model scoring 60.0% and 35.0%. On online interaction, the multi-step causal model achieves the highest overall pass rate of 75.0% and hard-criterion pass rate of 55.6%, exceeding the teacher’s 71.5% and 45.8%, respectively. The four-step model obtains 68.8% overall, remaining above JCV’s 51.4% and Hand2World’s 33.3%. For object response, JCV achieves the highest mask IoU of 38.7%, while our multi-step causal model achieves the lowest target-center offset of 48.8 pixels. These results highlight the distinction between image-space alignment and interaction success: favorable keypoint or object-overlap scores can coexist with low action-execution and final-state pass rates. The qualitative case study 6 illustrates this distinction through examples of camera movement and object behavior, while the action-branch examples 7 show different continuations from a shared context. Together, the quantitative and qualitative results demonstrate stronger joint-control and action-dependent interaction performance for our framework, with reduced hand-following and camera–hand coordination scores after causal adaptation and few-step distillation.
| Method | Q1: Hand following | Q2: Camera–hand coordination | Q3: Online interaction | Q4: Object response | |||||
| Method | [email protected] | L2 | Camera | Interaction | Basic | Hard | Overall | IoU | TO |
| JCV | 59.1 | 34.3 | 50.0 | 15.0 | 79.2 | 23.6 | 51.4 | 38.7 | 54.0 |
| Hand2World | 61.2 | 28.2 | 40.0 | 15.0 | 48.6 | 18.1 | 33.3 | 28.6 | 189.3 |
| Ours (Teacher) | 65.8 | 48.2 | 100.0 | 60.0 | 97.2 | 45.8 | 71.5 | 36.6 | 49.3 |
| Ours (Causal) | 57.9 | 46.8 | 70.0 | 40.0 | 94.4 | 55.6 | 75.0 | 34.9 | 48.8 |
| Ours (Causal-4 step) | 54.0 | 58.9 | 60.0 | 35.0 | 93.1 | 44.4 | 68.8 | 35.0 | 49.3 |
Table 2. TouchéBench results. All our variants use hand-weighted loss. PCK, checklist scores, and IoU are percentages; L2 and target-center offset (TO) are in pixels. Q2 Interaction averages action-execution and final-state pass rates. Best results are bold.
| Model | Hand loss | PSNR | SSIM | LPIPS | DINO | KID |
| EgoStandard-200 | ||||||
| Teacher | w/o | 19.13 | 0.705 | 0.350 | 0.812 | 0.00280 |
| Teacher | w/ | 19.18 | 0.708 | 0.345 | 0.816 | 0.00265 |
| Causal | w/o | 17.61 | 0.668 | 0.416 | 0.780 | 0.00354 |
| Causal | w/ | 17.60 | 0.666 | 0.410 | 0.785 | 0.00367 |
| EgoLife-300 | ||||||
| Teacher | w/o | 15.97 | 0.564 | 0.503 | 0.633 | 0.00630 |
| Teacher | w/ | 16.19 | 0.572 | 0.488 | 0.652 | 0.00657 |
| Causal | w/o | 14.85 | 0.516 | 0.568 | 0.558 | 0.01215 |
| Causal | w/ | 15.05 | 0.519 | 0.556 | 0.582 | 0.00876 |
| HOT3D-45 | ||||||
| Teacher | w/o | 15.14 | 0.552 | 0.489 | 0.759 | 0.00704 |
| Teacher | w/ | 15.11 | 0.552 | 0.488 | 0.769 | 0.00617 |
| Causal | w/o | 14.77 | 0.515 | 0.500 | 0.743 | 0.01005 |
| Causal | w/ | 14.74 | 0.525 | 0.500 | 0.750 | 0.00747 |
| HOI4D-200 | ||||||
| Teacher | w/o | 15.74 | 0.663 | 0.438 | 0.792 | 0.00210 |
| Teacher | w/ | 15.76 | 0.665 | 0.441 | 0.792 | 0.00240 |
| Causal | w/o | 15.55 | 0.660 | 0.445 | 0.787 | 0.00260 |
| Causal | w/ | 15.39 | 0.656 | 0.450 | 0.786 | 0.00248 |
Table 3. Table 3: Effect of hand-weighted loss on video quality. Better results within each Teacher or Causal pair are bold, including ties.
Case Study
Figure 5 illustrates how a shared camera–hand control sequence produces scene-dependent interactions. Starting from the osmanthus-tree example, we reuse the controls with four different scene initializations. The hand settles on the tabletop in the cafe scene, moves out of view after reaching through free space in the gallery, and grasps a package at the store shelf. These examples show that the generated hand behavior adapts to the surrounding scene while retaining the input motion pattern. In the shelf comparison, JCV produces substantial viewpoint changes toward the ceiling, while Hand2World generates hand motion in front of a largely unchanged shelf; our model instead produces a reach-and-lift interaction. Figure 6 further examines the distinction between spatial agreement and object interaction. In the reference, the hand moves the bottle relative to the cup. JCV retains the bottle near its initial position through much of the sequence, whereas our teacher and causal model generate more pronounced hand-associated object movement. Retaining an object near its initial location can preserve overlap with reference masks over parts of the sequence, illustrating why mask IoU should be interpreted alongside action execution and object-state changes. Finally, Figure 7 evaluates action-dependent continuations from a shared approximately two-second context. Different hand inputs ask the model to rotate a box, lift it above another box, or slide it along the tabletop. Our teacher and causal model produce distinct changes in box orientation, height, and position across these branches, while the baselines show less consistent separation of the intended outcomes. Together, these cases illustrate joint camera–hand control, scene-dependent contact behavior, and distinct object responses to actions introduced after a shared context.

Figure 6. Object-response comparison on HOI4D. Magenta overlays show SAM3-tracked masks; green contours show reference masks. Image-space overlap can differ from action-consistent object motion.

Figure 7. Action-dependent continuations from a shared two-second source context under rotating, lifting/stacking, and sliding controls. Branch labels describe evaluation goals and are not supplied as action-text prompts.
| Model | Hand loss | Q1: Hand following | Q2: Camera–hand coordination | Q3: Online interaction | Q4: Object response | ||||
| [email protected] ↑ | L2 ↓ | Camera ↑ | Interaction ↑ | Overall ↑ | Hard ↑ | BBox IoU ↑ | Mask IoU ↑ | ||
| Teacher | w/o | 58.7 | 60.0 | 100.0 | 40.0 | 65.3 | 37.5 | 68.4 | 32.5 |
| Teacher | w/ | 65.8 | 48.2 | 100.0 | 60.0 | 71.5 | 45.8 | 69.5 | 36.6 |
| Causal | w/o | 55.0 | 55.0 | 90.0 | 45.0 | 72.2 | 50.0 | 68.5 | 33.6 |
| Causal | w/ | 57.9 | 46.8 | 70.0 | 40.0 | 75.0 | 55.6 | 69.0 | 34.9 |
Table 4. Effect of hand-weighted loss on TouchéBench. Scores are percentages except L2 (pixels). Weighted variants match the main-table models. Better results within each model pair are bold, including ties.
Ablation Studies and Analysis
Hand-weighted loss. Tables 3, 5, and 4 examine the effect of hand-weighted training on video quality and interaction. Hand weighting improves keypoint accuracy for both the teacher and causal models across all three PCK thresholds. For the teacher, [email protected] increases from 58.7% to 65.8%, while localization error decreases from 60.0 to 48.2 pixels. The causal model improves from 55.0% to 57.9% PCK, with localization error decreasing from 55.0 to 46.8 pixels. These improvements also extend to online interaction: Q3 Overall increases from 65.3% to 71.5% for the teacher and from 72.2% to 75.0% for the causal model. For camera–hand coordination, the teacher’s Interaction score increases from 40.0% to 60.0%, while its Camera score remains at 100.0%. The causal model exhibits a trade-off, with Camera decreasing from 90.0% to 70.0% and Interaction from 45.0% to 40.0%. Object-mask IoU improves for both model types. Changes in global video quality are dataset-dependent: all five metrics improve for the teacher on EgoStandard and for the causal model on EgoLife, while the remaining evaluation sets show mixed changes. Overall, hand-weighted supervision strengthens hand following and online interaction, with a coordination trade-off in the causal model.
Causal adaptation and few-step distillation. We examine the generation stages through the teacher, multi-step causal, and four-step causal results in Table 1 and Table 2. Moving from the bidirectional teacher to the causal model reduces reference-based video quality and hand-following accuracy, with [email protected] decreasing from 65.8% to 57.9%. Q2 Camera and Interaction also decrease from 100.0% and 60.0% to 70.0% and 40.0%, respectively. On Q3, the causal model achieves higher Overall and Hard pass rates, increasing from 71.5% and 45.8% to 75.0% and 55.6% under the method-specific context initialization used in our evaluation. This contrast shows that stronger reference reconstruction and keypoint alignment do not directly imply better action-dependent continuations. Four-step distillation improves PSNR and SSIM over the multi-step causal model on all three video-quality test sets, while changes in LPIPS, DINOv2 similarity, and KID are mixed. Hand-following PCK decreases to 54.0%, Q2 Camera and Interaction decrease to 60.0% and 35.0%, and Q3 Overall decreases to 68.8%. Object-response measurements remain close, with mask IoU changing from 34.9% to 35.0% and target-center offset from 48.8 to 49.3 pixels. These results characterize the transition to four-step causal generation: object-response fidelity remains similar under the reported spatial metrics, while hand following, joint control, and online interaction show larger performance reductions.
Conclusion and Future Work
We presented TouchéWorld, a video world modeling framework that integrates continuous hand interaction with camera-guided exploration. We constructed joint camera–hand training data through annotation completion, consistency-based filtering, and temporal alignment, and combined spatial skeleton conditioning with numerical hand-motion features through parameterization. efficient adaptation. Causal adaptation and few-step distillation enable chunk-level online generation with updated camera and hand controls. We further introduced TouchéBench to evaluate hand-motion following, camera–hand coordination, online interaction, and object response. Experiments demonstrate scene-dependent interaction across diverse environments and action-dependent continuations from shared contexts, while revealing distinct trade-offs among visual quality, hand following, and interaction performance.
| Model | Hand loss | [email protected] | [email protected] | [email protected] | L2 (px) |
| Teacher | w/o | 10.0 | 29.3 | 58.7 | 60.0 |
| Teacher | w/ | 13.7 | 36.6 | 65.8 | 48.2 |
| Causal | w/o | 8.8 | 26.2 | 55.0 | 55.0 |
| Causal | w/ | 10.1 | 28.9 | 57.9 | 46.8 |
Table 5. Detailed hand-loss ablation. Scores are percentages except L2 (pixels). , , , and denote viewpoint attainment, action execution, final-state correctness, and alternative-outcome exclusion. Better results within each model pair are bold, including ties. (a) Q1: Hand-motion following on HOI4D-200
| Model | Hand loss | Overall | Basic | Hard | Camera () | Interaction | ||
| Teacher | w/o | 80.0 | 100.0 | 60.0 | 100.0 | 40.0 | 40.0 | 40.0 |
| Teacher | w/ | 86.7 | 100.0 | 73.3 | 100.0 | 60.0 | 60.0 | 60.0 |
| Causal | w/o | 80.0 | 100.0 | 60.0 | 90.0 | 45.0 | 40.0 | 50.0 |
| Causal | w/ | 68.3 | 86.7 | 50.0 | 70.0 | 40.0 | 40.0 | 40.0 |
Table 5b. (b) Q2: Camera–hand coordination
| Model | Hand loss | Overall | Basic | Hard | |||
| Teacher | w/o | 65.3 | 93.1 | 37.5 | 29.2 | 58.3 | 25.0 |
| Teacher | w/ | 71.5 | 97.2 | 45.8 | 37.5 | 66.7 | 33.3 |
| Causal | w/o | 72.2 | 94.4 | 50.0 | 41.7 | 66.7 | 41.7 |
| Causal | w/ | 75.0 | 94.4 | 55.6 | 33.3 | 66.7 | 33.3 |
Table 5c. (c) Q3: Action-dependent online interaction. All checklist columns are higher-is-better. Q3 causal models use an approximately two-second context; teachers are initialized from the context endpoint. Loss effects are compared within each model type.
Our qualitative results also reveal early signs of gesture-mediated interaction with people in generated scenes. This opens a direction for extending interactive world models beyond object manipulation toward human interaction. Future work will explore how generated people perceive and respond to user gestures, enabling richer exchanges between users and the inhabitants of generative worlds.
Acknowledgements
This work is partially supported by a public grant overseen by the French National Research Agency (ANR) as part of the program Investissements d’Avenir (ANR-10-LABX-0083).
References
- [1]Rick Akkerman, Haiwen Feng, Michael J Black, Dimitrios Tzionas, and Victoria Fernández Abrevaya. Interdyn: Controllable interactive dynamics with video diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12467–12479. IEEE, 2025.arxiv.org/abs/2412.11785
- [2]Prithviraj Banerjee, Sindi Shkodrani, Pierre Moulon, Shreyas Hampali, Shangchen Han, Fan Zhang, Linguang Zhang, Jade Fountain, Edward Miller, Selen Basol, Richard Newcombe, Robert Wang, Jakob Julian Engel, and Tomas Hodan. Hot3d: Hand and object tracking in 3d from egocentric multi-view videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7061–7071, 2025.arxiv.org/abs/2411.19167
- [3]Mikolaj Bińkowski, Danica J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018.arxiv.org/abs/1801.01401
- [4]Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. In Forty-first international conference on machine learning, 2024.arxiv.org/abs/2402.15391
- [5]Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris Coll-Vinent, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, et al. Sam 3: Segment anything with concepts. In International conference on learning representations, pages 138846–138923, 2026.arxiv.org/abs/2511.16719
- [6]Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open-world game video generation. In International Conference on Learning Representations, pages 37546–37593, 2025.arxiv.org/abs/2411.00769
- [7]Danze Chen, Zeqing Wang, Ziyue Lin, Xingyi Yang, and Yeying Jin. H3-world: Turning language understanding into world control. arXiv preprint arXiv:2609.01560, 2026.arxiv.org/abs/2609.01560
- [8]Yiwen Chen, Guosheng Lin, and Chi Zhang. Code world model: Coding agent as world brain, 2026.arxiv.org/abs/2608.25927
- [9]Yushuo Chen, Xiaoyu Shi, Xiaoshi Wu, Xintao Wang, Pengfei Wan, and Yebin Liu. Handsworld: Unconstrained egocentric video generation with camera-disentangled hand control. arXiv preprint arXiv:2607.02075, 2026.arxiv.org/abs/2607.02075
- [10]Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, Jiahao Wang, Yufeng Yuan, Hanlin Wang, Yichong Lu, Ka Leong Cheng, Haojie Zhang, Jian Gao, Tianrui Feng, Yuzheng Liu, Yao Yao, Yinghao Xu, Xing Zhu, Yujun Shen, and Hao Ouyang. Infinite worlds with versatile interactions, 2026.arxiv.org/abs/2607.07534
- [11]Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, and Chen Sun. Force prompting: Video generation models can learn and generalize physics-based control signals. Advances in Neural Information Processing Systems, 38:103174–103201, 2026.arxiv.org/abs/2505.19386
- [12]Bohai Gu, Taiyi Wu, Yueyang Yuan, Jian Liu, Xiaocheng Lu, Dazhao Du, Jie Zhang, Jinxiang Lai, Shuai Yang, Xiaotong Zhao, et al. Worldcraft: From camera navigation to object manipulation in interactive video world models. arXiv preprint arXiv:2605.25077, 2026.arxiv.org/abs/2605.25077
- [13]Junliang Guo, Yang Ye, Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, and Jiang Bian. Mineworld: a real-time and open-source interactive world model on minecraft. arXiv preprint arXiv:2504.08388, 2025.arxiv.org/abs/2504.08388
- [14]Yoav HaCohen, Benny Brazowski, Nisan Chiprut, Yaki Bitterman, Andrew Kvochko, Avishai Berkowitz, Daniel Shalem, Daphna Lifschitz, Dudu Moshe, Eitan Porat, Eitan Richardson, Guy Shiran, Itay Chachy, Jonathan Chetboun, Michael Finkelson, Michael Kupchick, Nir Zabari, Nitzan Guetta, Noa Kotler, Ofir Bibi, Ori Gordon, Poriyа Panet, Roi Benita, Shahar Armon, Victor Kulikov, Yaron Inger, Yonatan Shiftan, Zeev Melumian, and Zeev Farbman. Ltx-2: Efficient joint audio-visual foundation model, 2026.
- [15]Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang, Yifan Zhang, Qi Cui, Fei Kang, Biao Jiang, Mengyan An, Yangyang Ren, Baixin Xu, Hao-Xiang Guo, Kaixiang Gong, Size Wu, Wei Li, Xuchen Song, Yang Liu, Yangguang Li, and Yahui Zhou. Matrix-game 2.0: An open-source, real-time, and streaming interactive world model, 2026.arxiv.org/abs/2508.13009
- [16]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022.arxiv.org/abs/2106.09685
- [17]Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, Jiawei Ren, Kevin Xie, Joydeep Biswas, Laura Leal-Taixe, and Sanja Fidler. Vipe: Video pose engine for 3d geometric perception, 2025.arxiv.org/abs/2508.10934
- [18]Junchao Huang, Guian Fang, Shengju Qian, Xianghao Kong, Zhuoran Zhao, Wei Huang, Yihua Du, Zixin Zhang, Justin Cui, Yuchao Gu, Yukang Chen, Xinting Hu, Tianyu He, Shaoshuai Shi, Zhuotao Tian, Xin Wang, Mike Zheng Shou, and Li Jiang. Solarmm: Open data and scalable training for long-horizon video world models, 2026.arxiv.org/abs/2609.02886
- [19]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. Advances in Neural Information Processing Systems, 38:167283–167308, 2026.arxiv.org/abs/2506.08009
- [20]Patrick Amadeus Irawan, Iskandar Muda Rizky Parlamaanka, Rava Maulana, Qinrong Cui, Erland Hilman Fuadi, Zayd M. K. Zuhri, Nanda Ryaas Absar, Ahmed Elshabrawy, Wilfried Ariel Mulyawan, Shoubin Yu, Yue Zhang, Mohit Bansal, and Alham Fikri Aji. Ego2act: Evaluating goal-directed manipulation in egocentric video generation, 2026.arxiv.org/abs/2610.01092
- [21]Byungjun Kim, Taeksoo Kim, Junyoung Lee, and Hanbyul Joo. Dexterous world models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 29663–29673, 2026.arxiv.org/abs/2512.17907
- [22]Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.arxiv.org/abs/1312.6114
- [23]Ruilong Li, Brent Yi, Junchen Liu, Hang Gao, Yi Ma, and Angjoo Kanazawa. Cameras as relative positional encoding. Advances in Neural Information Processing Systems, 38:15984–16009, 2026.arxiv.org/abs/2507.10496
- [24]Lightwheel. EgoSuite-Open100K: A Fully Annotated Open Egocentric Human Dataset. https://huggingface.co/collections/LightwheelAI/egosuite-open100k, 2026. Dataset.
- [25]Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interaction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21013–21022, 2022.arxiv.org/abs/2203.01577
- [26]MiniMax. MiniMax H3: An open model breaking the boundaries between tasks and modalities. https://www.minimax.io/blog/minimax-h3, 2026.
- [27]OpenAI. Code generation with openai models and codex. https://developers.openai.com/api/docs/guides/code-generation, 2026. Accessed: 2026-11-03.
- [28]OpenAI. Gpt-6 astra: A new generation of intelligence. https://openai.com/index/gpt-6-astra/, 2026. Accessed: 2026-11-03.
- [29]Maxime Ouqab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024.
- [30]Shaohao Rui, Xiaofeng Mao, Zhanyu Zhang, Peijia Lin, Yansong Zhu, Yibo Zhang, Haibin Wan, Zhangrui Zhao, and Weijie Ma. Biwm: Advancing open-source interactive video world models with bidirectional autoregression, 2026.arxiv.org/abs/2606.10135
- [31]Joonghyuk Shin, Zhengqi Li, Richard Zhang, Jun-Yan Zhu, Jaesik Park, Eli Shechtman, and Xun Huang. Motionstream: Real-time video generation with interactive motion controls. In International Conference on Learning Representations, pages 7381–7406, 2026.arxiv.org/abs/2511.01266
- [32]Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling. arXiv preprint arXiv:2512.14614, 2025.arxiv.org/abs/2512.14614
- [33]Junshu Tang, Jiacheng Liu, Jiaqi Li, Longhuang Wu, Haoyu Yang, Penghao Zhao, Sirius Gong, Xiang Yuan, Shuai Shao, Linfeng Zhang, et al. Hunyuan-gamecraft-2: Instruction-following interactive game world model. arXiv preprint arXiv:2511.23429, 2025.arxiv.org/abs/2511.23429
- [34]AlayaWorld Team, Kaipeng Zhang, Chuanhao Li, Yifan Zhan, Yongtao Ge, Yuanyang Yin, Jiaming Tan, Kang He, Laoyuan Fan, Ruicong Liu, et al. Alayaworld: Long-horizon and playable video world generation. arXiv preprint arXiv:2607.06291, 2026.arxiv.org/abs/2607.06291
- [35]DreamX Team, Yancheng Bai, Rui Chen, Xiangxiang Chu, Rujing Dang, Hao Dou, Bingjie Gao, Qiwen Gu, Siyu Hong, Jiachen Lei, et al. Dreamx-world 1.0: A general-purpose interactive world model. arXiv preprint arXiv:2606.16993, 2026.arxiv.org/abs/2606.16993
- [36]Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. In International Conference on Learning Representations, pages 73754–73776, 2025.arxiv.org/abs/2408.14837
- [37]Hanhui Wang, Yiming Xie, Haiwen Feng, Zhaoyang Lv, Shenlong Wang, and Huaizu Jiang. Streaming video generation with streaming force control. arXiv preprint arXiv:2606.07508, 2026.arxiv.org/abs/2606.07508
- [38]Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong Yang. Moge-2: Accurate monocular geometry with metric scale and sharp details. In Advances in Neural Information Processing Systems, pages 35928–35959. Curran Associates, Inc., 2025.arxiv.org/abs/2507.02546
- [39]Yuxi Wang, Wenqi Ouyang, Tianyi Wei, Yi Dong, Zhiqi Shen, and Xingang Pan. Hand2world: Autoregressive egocentric interaction generation via free-space hand gestures, 2026.arxiv.org/abs/2602.09600
- [40]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.pubmed.ncbi.nlm.nih.gov/15376593
- [41]Linxi Xie, Lisong C. Sun, Ashley Neall, Tong Wu, Shengqu Cai, and Gordon Wetzstein. Generated reality: Human-centric world simulation using interactive video generation with hand and camera control, 2026.arxiv.org/abs/2602.18422
- [42]Mingxi Xu, Bowen Duan, Yi Gu, Zhengyang Shen, Renjing Xu, and Yutao Yue. Handflow: Fully generative 4d hand recovery with flow matching, 2026.arxiv.org/abs/2607.11221
- [43]Haodong Yan, Hang Yu, Zhide Zhong, Weilin Yuan, Xin Gong, Zehang Luo, Chengxi Heyu, Junfeng Li, Wenxuan Song, Shunbo Zhou, and Haoang Li. Open-world hand-object interaction video generation based on structure and contact-aware representation, 2025.arxiv.org/abs/2512.01677
- [44]Jingkang Yang, Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang, Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, Bei Ouyang, Zhengyu Lin, Marco Cominelli, Zhongang Cai, Bo Li, Yuanhan Zhang, Peiyuan Zhang, Fangzhou Hong, Joerg Widmer, Francesco Gringoli, Lei Yang, and Ziwei Liu. Egolife: Towards egocentric life assistant. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28885–28900, 2025.arxiv.org/abs/2503.03803
- [45]Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, et al. Longlive: Real-time interactive long video generation. arXiv preprint arXiv:2509.22622, 2025.arxiv.org/abs/2509.22622
- [46]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6613–6623, 2024.arxiv.org/abs/2311.18828
- [47]Cheng Zhang, Boying Li, Meng Wei, Yan-Pei Cao, Camilo Gambaradella, Dinh Phung, and Jianfei Cai. Unified camera positional encoding for controlled video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 38027–38037, 2026.arxiv.org/abs/2512.07237
- [48]Chenyangguang Zhang, Botao Ye, Boqi Chen, Alexandros Delitzas, Fangjinhua Wang, Marc Pollefeys, and Xi Wang. Controllable egocentric video generation via occlusion-aware sparse 3D hand joints. In European Conference on Computer Vision (ECCV), 2026.arxiv.org/abs/2603.11755
- [49]Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 586–595, 2018.arxiv.org/abs/1801.03924
- [50]Songchun Zhang, Yaowei Li, Junhao Zhuang, Weiyang Jin, Haoyu Wang, Xin Lu, Yilang Sun, Shiyi Zhang, Haoran Li, Xiaoxiao Ma, et al. Echowm: Open and enterable omnimodal world models. arXiv preprint arXiv:2608.23189, 2026.
- [51]Yifan Zhang, Chunli Peng, Boyang Wang, Puyi Wang, Qingcheng Zhu, Fei Kang, Biao Jiang, Zedong Gao, Eric Li, Yang Liu, and Yahui Zhou. Matrix-game: Interactive world foundation model, 2025.arxiv.org/abs/2506.18701
- [52]Min Zhao, Hongzhou Zhu, Bokai Yan, Zihan Zhou, Yimin Chen, Honglie Wang, Wenqiang Sun, Zhengwei Fang, Zizheng Xun, Zihao Li, et al. minwm: A full-stack open-source framework for real-time interactive video world models. arXiv preprint arXiv:2605.30263, 2026.arxiv.org/abs/2605.30263
- [53]Min Zhao, Hongzhou Zhu, Kaiwen Zheng, Zihan Zhou, Bokai Yan, Xinyuan Li, Xiao Yang, Chongxuan Li, and Jun Zhu. Causal forcing++: Scalable few-step autoregressive diffusion distillation for real-time interactive video generation, 2026.arxiv.org/abs/2605.15141
- [54]Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, and Jun Zhu. Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation. arXiv preprint arXiv:2602.02214, 2026.arxiv.org/abs/2602.02214
- [55]Zijie Zhu, Weiren Cai, Yizhou Wang, Zhenjie Yang, Yide Liu, Jiahao Chen, and Guanqi He. Mint: A unified model for world-space camera and hand motion estimation from scalable egocentric pipeline supervision, 2026.arxiv.org/abs/2609.04958