Compression Forcing: Hierarchical History Tokenization Enables Flexible World Models

Introduction

World models [15, 31] have emerged as a cornerstone of intelligent decision-making, enabling agents to plan and act by simulating future trajectories conditioned on past observations and actions [16, 18]. Recently, by adopting architectural designs and training paradigms from video generation models, world models have achieved unprecedented generative and predictive capabilities across diverse domains [23, 14, 38, 55, 28]. Despite remarkable progress, the objectives of the two paradigms are fundamentally distinct: video generation optimizes for perceptual fidelity and visual realism, whereas world modeling demands faithfully capturing environment dynamics to support accurate downstream prediction and planning. This distinction raises a natural question: From first principles, how should we design a sequential generative model specifically tailored to the requirements of world model applications?

The use of historical information (e.g. frames) is a central consideration. Standard autoregressive approaches treat history in a binary manner: frames within the context window are fully conditioned upon, while those outside are entirely discarded. Diffusion Forcing [8, 44] introduces a more flexible alternative by enabling conditioning on partially noised histories at inference time. By interpolating between isotropic Gaussian noise and clean observations, this essentially acts as a frequency-based low-pass filter [11] of history information, treating partial noising as partial masking. While this mechanism aligns well with perceptual objectives in video generation, where low-frequency structure dominates human sensitivity, it is less aligned with the predictive purposes of world modeling.

Instead, we introduce an alternative principle: partial compression as partial masking. Rather than relying on human-imposed prior for masking, we learn an ordered tokenization where hierarchy is induced by a task-specific objective. By conditioning on variable-length prefixes of this learned sequence, truncation naturally serves as a structured form of partial masking, aligning information decomposition in a data-centric fashion. As a concrete realization of this principle in the context of world modeling, we introduce Dynamics-Aware Tokenization (DAT), a tokenization method compressing historical information into a hierarchical sequence ordered by dynamic saliency. Through training via a task-specific objective, DAT prioritizes dynamic motions in earlier tokens, whereas static visual information is encapsulated in later tokens.

Building upon this structured tokenization, we further introduce Compression Forcing (CF), a training and sampling framework that operationalizes partial compression as partial masking within diffusion-based world models. Specifically, CF trains a diffusion-based backbone to condition on variable-length prefixes of DAT tokens, enabling prediction under partial history compression (Figure 1), used as a residual stream to make dynamics-relevant information accessible to network attention queries (Figure 3, § 3.2). Crucially, this architectural design unlocks a novel inference-time capability: Hierarchical History Guidance (HG-H, § 3.3). By composing score functions derived from the full history sequence and DAT’s leading subset, the model is equipped with the inference-time flexibility to selectively amplify dynamic-relevant history, effectively suppressing visual nuisances to ensure robust world modeling.

Through extensive experiments, we demonstrate that Compression Forcing achieves high-fidelity future prediction across a diverse set of environments and action spaces, including dynamic video forecasting [7], 3D game simulation [40], and real-world robotic manipulation [9, 13]. Moreover, beyond perceptual metrics, we show that our world model translates to stronger downstream policy evaluation [32] performance, verifying Compression Forcing’s effectiveness in aiding downstream decision-making.

Our main contributions are as follows: (1) We propose partial compression as partial masking, a general principle that replaces noise-based history conditioning with learned, task-driven hierarchical compression, applicable beyond world modeling. (2) We instantiate this principle through Compression Forcing, a framework comprising Dynamics-Aware Tokenization (DAT) for saliency-ordered history compression, a diffusion-based backbone for prefix-flexible conditioning, and Hierarchical History Guidance (HG-H) for inference-time amplification of dynamics-critical information. (3) We validate the effectiveness of our approach across various domains, demonstrating improved prediction fidelity and effective downstream decision-making performance.

Preliminaries and Related Work

World Models

World Models [31, 15] are predictive models learned by agents to capture the dynamics of the environment, commonly formulated as Partially-Observable Markov Decision Processes (POMDPs) [26]. Concretely, a POMDP is specified by the tuple $(\mathcal{S}, \mathcal{O}, \Omega, \mathcal{A}, \mathcal{T}, r, \gamma)$, where the environment evolves over an underlying Markovian state $s_t \in\mathcal{S}$ according to a transition function $\mathcal{T} : \mathcal{S} \times\mathcal{A} \to\Delta(\mathcal{S})$. Since the states $s_t$ are inaccessible, agents rely on observations $o_t \sim\Omega(\cdot\mid s_t)$ to perceive the environment. Consequently, the primary objective of world models is to approximate the environment dynamics by estimating the distribution of future observations conditioned on history observations and actions: $p_\theta(o_{t+1} \mid o_{\leq t}, a_{\leq t})$. In this paper, we focus on pixel-space observations $\mathcal{O} \equiv\mathbb{R}^{H \times W \times C}$, and denote each frame as $x_t$, following conventions in diffusion-based generation.

Diffusion Models

Diffusion Models [21, 43] generate high-fidelity samples by learning to reverse a gradual noise-injection process. Formally, the forward process gradually converts data $x^0 \sim q$ into noise $x^K \sim\mathcal{N}(0,I)$ through a Markov Chain, defined as

where $\beta_{0:K}$ is a pre-defined schedule of noise levels. Sampling is implemented via a reverse process, given by

Rather than explicitly modeling the transition densities, modern approaches often reparameterize this by learning to approximate the noise component added at each step $\epsilon^k = (\sqrt{1-\bar{\alpha}_k})^{-1}(x^k-\sqrt{\bar{\alpha}_k}x^0)$, where $\alpha_k = 1-\beta_k$ and $\bar{\alpha}_k = \prod_{i=1}^{k}\alpha_i$. The training objective is then simplified to minimizing the mean square loss:

Sampling is performed by iterative ancestral denoising:

Sequential Video Generation and History Guidance

Sequential video generation predicts future frames $x^0_{\mathcal{F}}$ conditioned on history observations $x^0_{\mathcal{H}}$, enabling long-context video generation with flexible horizon [25, 10] and on-the-fly streaming [56, 54]. Beyond standard teacher forcing approaches [52], a prevailing paradigm utilizes Diffusion Forcing [8] to enable flexible history conditioning compatible with diffusion architectures. By sampling independent noise levels for each frame during training, it allows for diffusion-based auto-regressive rollouts conditioned on clean history. Building on this, DFoT [44] proposes to treat the history sequence as an external control signal [20], steering future generation by composing score functions conditioned on history interpolated with isotropic Gaussian noise of varying levels $k_{\mathcal{H}}$:

By operationalizing guidance through noised history tokens $x^{k_{\mathcal{H}}}_{\mathcal{H}}$, this history-conditioning mechanism acts as a frequency-based low-pass filter to enforce conditioning on low-frequency components in history observations. While this frequency-based prior is well-suited for general video synthesis, it is less suited for world modeling, where context differentiation should be governed by predictive salience (i.e., dynamics) rather than spatial frequency.

Vision Tokenization

Visual tokenization serves as a foundational component of modern generative architectures, compressing high-dimensional observations into computationally tractable representations for backbone processing. Classical generative approaches, such as SD-VAE [42] and VQGAN [12], focus on optimizing perceptual fidelity through continuous regularization or discrete codebooks. Concurrently, emerging research [58] demonstrates that representation encoders commonly seen in VLMs [37] (Tschannen et al. 2025), originally designed for semantic understanding, are also highly effective tokenizers for generative tasks. On the structural front, approaches like Semanticist [50] and ElasticTok [53] explore hierarchical tokenization via nested dropout, enabling adaptive reconstruction with variable token counts. However, while recent world models have shown impressive results [5, 3], they predominantly rely on tokenizers inherited from video generation models, allocating a fixed, uniform number of tokens per frame regardless of their predictive importance. The potential of vision tokenization methods specifically tailored for the predictive needs of world models remains largely unexplored.

Compression Forcing

In this section, we introduce Compression Forcing, a general framework for constructing flexible world models via hierarchical history tokenization. Departing from standard approaches [3, 5, 23] that employ history tokenization methods which allocates fixed, uniform budgets for each frame, Compression Forcing compresses history into sequences ordered by dynamic saliency, enabling world modeling with flexible amplification of dynamics-salient history information at inference time.

We first introduce Dynamics-Aware Tokenization (DAT), a novel tokenization method that compresses historical observations into hierarchical token sequences ordered by dynamic saliency (§ 3.1). Building on this structured representation, we detail the design of our Diffusion-based World Model Backbone, designed to condition on arbitrary-length prefixes of the DAT compressed token sequences (§ 3.2). Finally, we demonstrate that these designs unlock a novel inference-time capability: Hierarchical History Guidance (HG-H), a guidance mechanism that allows selective amplification of the role of dynamics-critical history information in generation guidance to facilitate flexible future prediction (§ 3.3).

Dynamics-Aware Tokenizer

Standard vision tokenizers in video generation models [47, 12, 42] are optimized for pixel-level reconstruction objectives, resulting in representations that allocate uniform budgets per frame. Especially combined with history guidance methods that interpolate between isotropic Gaussian noise and clean images, this creates a “frequency-based bias” [11] where guidance is performed by enforcing conditioning on low-frequency components, i.e. treating partial noising as partial masking. While this human-imposed prior aligns well with perceptual objectives in video generation, it is misaligned with the predictive demands of world modeling. We propose a more general alternative: partial compression as partial masking. Rather than relying on a fixed frequency-based decomposition, we learn a tokenization whose hierarchy is induced by dynamic saliency, so that truncating the token sequence naturally serves as a task-aligned form of partial masking that effectively retains dynamics-critical information.

Dynamics-Aware Objective. To realize this principle, we design Dynamics-Aware Tokenization (DAT), a novel tokenization method that compresses history frames into a sequence ordered by dynamic saliency, disentangling dynamic motions from static visual nuisances. We achieve this through two complementary decoding objectives: static reconstruction and dynamic prediction, whose relative weighting across variable-length prefixes induces the desired hierarchy. Concretely, an encoder $\mathcal{E}$ compresses the history frames $x_{t:t+\tau}$ into a latent sequence $z_{1:L} = \mathcal{E}(x_{t:t+\tau})$, and a diffusion-based decoder $\mathcal{D}$ is trained to decode variable-length prefixes $z_{1:m}$ ($m \le L$) under two distinct objectives:

• (a) Static Reconstruction: Given only the prefix $z_{1:m}$ and no external context, the decoder is trained to reconstruct the full clip $x_{t:t+\tau}$. Since the prefix alone must account for all visual content, this compels tokens to capture a lossless compression of information:

• (b) Dynamic Prediction: Given the prefix $z_{1:m}$ and the initial frame $x_t$ as additional context, the decoder is trained to predict target frames $x_{t+1:t+\tau}$ given $z_{1:m}$. Essentially, $x_t$ allows the model to act as a residual learner, enabling the prefix to only focus on dynamic changes:

We enforce the desired hierarchy through a schedule-weighted combination of these two objectives. During training, we randomly sample prefix lengths $m \in[1,L]$ and minimize:

where $\omega(m)$ is a monotonic increasing function. When $m$ is small, the objective is dominated by dynamic prediction $\mathcal{L}_{\mathrm{dyn}}$, compelling leading tokens to prioritize motion dynamics; as $m$ grows, $\mathcal{L}_{\mathrm{static}}$ takes over, shifting focus toward full visual reconstruction. This yields a naturally ordered sequence: earlier tokens encode dynamic changes, while later tokens progressively capture static visual details.

Tokenizer Architecture. The tokenizer $\mathcal{E}$ is built based on the Video Vision Transformer (ViViT) [4] architecture. To produce the ordered sequence $z_{1:L}$, we append $L$ learnable Slot Tokens at the end of the patchified video sequence. Information flow is regulated through a Hybrid Attention Mechanism: video patch tokens attend bidirectionally to each other but are masked from attending to slot tokens, whereas the $i$-th slot token $z_i$ attends globally to all video patches but causally to preceding slots $\mathbf{z}_{<i}$. This causal design is essential for ensuring that prefix-truncation is well-structured: since $\mathbf{z}_i$ is computed without access to succeeding slots $\mathbf{z}_{>i}$, any prefix $\mathbf{z}_{1:m}$ remains a self-consistent representation regardless of the truncation point.

For the diffusion decoder $\mathcal{D}$, we adopt a video diffusion transformer architecture [34]. Notably, rather than maintaining separate networks for each objective, we implement the Static and Dynamic Decoders as a single model with shared parameters. The distinction between tasks is achieved through noise conditioning [8]: for Static Reconstruction, we apply identical noise levels to all frames; for Dynamic Prediction, we set the noise level of the condition frame $\mathbf{x}_t$ to zero and identical noise levels to other frames.

Prefix-Conditioned World Model Backbone

Backbone Architecture. Building upon the hierarchical history representation produced by DAT, we instantiate the world model backbone $\mathcal{W}_{\theta}$ as a conditional diffusion transformer [41, 5]. The backbone predicts the next frame $\mathbf{x}_{t+1}$ conditioned on the agent’s recent actions $\mathbf{a}_t$ and observations $\mathbf{x}_{t-n+1:t}$. Within each transformer block, we employ a dual-conditioning mechanism: the fine-grained history representation $\mathbf{H}$ is injected via cross-attention. Meanwhile, the low-dimensional conditions (action $\mathbf{a}_t$ and diffusion noise level $k$) are projected into a joint embedding $\boldsymbol{\xi} = \operatorname{MLP}_{\mathrm{act}}(\mathbf{a}_t) + \operatorname{MLP}_{\mathrm{noise}}(k)$, modulating intermediate features $\mathbf{h}$ via AdaLN [51]:

where $\gamma(\cdot)$ and $\beta(\cdot)$ are learned scale and shift parameters regressed from $\boldsymbol{\xi}$.

History Prefix Conditioning. Since DAT accepts a fixed-length input of $\tau$ frames, a discrepancy exists between its limited receptive field and the backbone’s need for broader history context (where $n > \tau$). To bridge this gap, we encode the history context $\mathbf{x}_{t-n+1:t}$ using a sliding window approach with a temporal stride $s$. We extract $\mathcal{I} = \left\lfloor\frac{n-\tau}{s} \right\rfloor+ 1$ overlapping frame sequences. Each window $\mathbf{x}^{(i)}$ is independently tokenized by the frozen DAT encoder, and the resulting latents are concatenated to form an encoded global history context $\mathbf{H}^{\mathrm{enc}}$:

To fully leverage the saliency ordering within $\mathbf{H}$, the world model backbone must learn to condition on variable-length prefixes of each history window. During training, for each encoded window $\mathcal{E}(\mathbf{x}^{(i)})$, we independently sample a truncation budget $l_i \sim\mathcal{U}[0,L]$ and construct a binary mask $\mathbf{M}_{l_i}$ that preserves the leading $l_i$ tokens and masks out the rest. The training objective is defined as:

where $\mathbf{l} = [l_1, \ldots, l_{\mathcal{I}}]$ is the budget vector and $\mathbf{M}_{\mathbf{l}} = \operatorname{Concat}(\mathbf{M}_{l_1}, \ldots, \mathbf{M}_{l_{\mathcal{I}}})$. This training objective equips the backbone with the ability to condition on any prefix length of the the DAT-encoded history, laying the foundation for controllable guidance of history information at inference time detailed in Section 3.3.

Hierarchical History Guidance (HG-H)

Given the world model backbone’s flexibility to condition on arbitrary-length prefixes of the DAT-encoded history, we introduce Hierarchical History Guidance (HG-H), an inference-time guidance mechanism that amplifies dynamics-salient history information through score composition [20].

General Guidance Formulation. Let $H^{(1)}$ denote the encoded history context masked by a budget vector $\boldsymbol{l}$. The diffusion backbone functions as a learned score estimator $s_{\theta}(x_{t+1},H^{(1)},a_t,k) \approx\nabla\log p_k(x_{t+1}\mid H^{(1)},a_t)$. Given a set of budget vectors $\{\boldsymbol{l}_j\}_{j=1}^{\mathcal{J}}$, we can guide generation via composing $\widetilde{\nabla}\log p_k(x_{t+1}^k)$ as a weighted sum of conditional scores:

where $\omega_j$ controls the guidance strength for each component. Essentially, this formulation allows flexible amplification on specific historical information components based on their predictive utility.

Hierarchical History Guidance. As the primary instantiation of this guidance formulation, we propose Hierarchical History Guidance (HG-H), prioritizing high-saliency dynamics over visual details. For every $j \in\{1,\ldots,\mathcal{J}\}$, we assign a uniform saliency budget $\boldsymbol{l}^{\mathrm{dyn}}_j=[m_j,\ldots,m_j]$, where $m_j\leq L$ represents the truncation threshold across all history windows. The guidance objective simplifies to:

By guiding generation with truncated prefixes of each history window, HG-H amplifies the influence of motion dynamics encoded in the leading tokens, while suppressing stochastic visual nuances captured by later tokens. Essentially, this operationalizes the partial compression as partial masking principle: the saliency ordering imposed by DAT ensures that prefix truncation serves as a task-aligned form of partial masking, with $\omega_j$s controlling the degree of amplification at inference time.

Experiments

In this section, we evaluate Compression Forcing under three distinct world modeling environments, and compare their performance against state-of-the-art baselines. We demonstrate competitive performance against existing methods in a variety of tasks (§ 4.2, § 4.3 and § 4.4) and conduct in-depth analysis on tokenization and generation error distribution to understand the effects of DAT and HG-H (§ 4.5). Experimental Details can be found in Appendix § 8.

Experimental Setting

Environments. We evaluate Compression Forcing across three domains: general video prediction, game simulation, and robot manipulation. For video prediction, we use Kinetics-600 [7], a large-scale benchmark with diverse, motion-rich videos, utilizing it for history-conditioned video prediction. For game simulation, we adopt Counter-Strike: Global Offensive (CS:GO), using a 5.5M-frame online gameplay dataset [40]. For robot manipulation, we use RT-1 [9], comprising real-world manipulation tasks such as picking and placing. The action spaces differ across domains: Kinetics-600 is action-free; CS:GO involves discretized keyboard and mouse inputs; and RT-1 uses a 7D continuous end-effector action space. Together, these environments span a broad spectrum of dynamics complexity, action dependence, and visual diversity, enabling a comprehensive evaluation of Compression Forcing across both virtual and real-world settings. Further details can be found in Appendix § 8.§ 1.

Evaluation Metrics. We evaluate video prediction performance using a variety of widely-adopted metrics, measuring pixel-level fidelity and perceptual similarity. For pixel-level fidelity, we utilize PSNR [22], and for perceptual similarity, we report SSIM [49], FVD [46], FID [19] and LPIPS [57]. Details are shown in Appendix § 8.§ 2. Finally, we evaluate fidelity to physical dynamics through two downstream metrics deferred to Appendix § 9: (1) object tracking fidelity, where we sample query points within SAM-generated object masks, track them with CoTracker3 across ground-truth and predicted videos, and measure point-trajectory deviations and penalties for lost tracks, and (2) policy evaluation correlation via Real2Sim evaluation protocol of Li et al. [ [32], capturing how reliable Compression Forcing is in predicting and ranking policy performance.

Baselines. We compare Compression Forcing against two distinct sets of baselines. The first shares our backbone architecture but conditions on alternative latent spaces for history compression: SD-VAE [42], which compresses each frame into a fixed-budget representation, and ElasticTok [53], which compresses video segments into a 1D sequence with leading tokens prioritizing reconstruction fidelity, akin to a learned PCA. The second comprises strong existing world models, including DIAMOND [3] in CS:GO and DreamerV4 [18] in RT-1. Baseline details are shown in Appendix § 8.§ 3.

Compression Forcing for Dynamic Video Generation

We first evaluate Compression Forcing on action-free video prediction using Kinetics-600 [7], a large-scale benchmark featuring diverse, motion-rich natural videos. This setting is particularly informative as it isolates our DAT from action conditioning: the hierarchical structure must emerge purely from visual dynamics, instead of relying on actions to aid dynamic prediction. Across all baselines, we train the diffusion-based backbone for 160k gradient steps. We train the DAT tokenizer for 120k gradient steps and the ElasticTok tokenizer for 175k gradient steps. During inference, we condition our model on 16 history frames, and autoregressively rollout 8 frames. Further training and inference details are provided in Appendix § 8.§ 5.

As shown in Table 1, Compression Forcing consistently outperforms all existing baselines in all metrics evaluated. Even without guidance, where the model is exposed to all DAT tokens, the performance of our model consistently outperforms SD-VAE and ElasticTok, showcasing the efficacy of leveraging dynamics-aware tokenization for precise prediction. Performance gains yielded by HG-H further demonstrate that partial compression serves as an effective mechanism for targeted probability steering during inference.

Tokenizer	Guidance	SSIM$\uparrow$	PSNR$\uparrow$	FID$\downarrow$	LPIPS$\downarrow$
SD-VAE	✗	0.543	18.53	90.57	0.351
ElasticTok	✗	0.544	18.55	91.23	0.351
ElasticTok	✓	0.546	18.60	90.74	0.349
DAT	✗	0.551	18.77	89.26	0.344
DAT	HG-H	0.553	18.83	88.13	0.343

Compression Forcing for Game Simulation

We next evaluate Compression Forcing on action-conditioned game simulation using Counter-Strike: Global Offensive (CS:GO), a popular 3D first-person shooting game. This environment is particularly challenging due to its highly stochastic dynamics and partial observability, including rapid viewpoint changes, occlusions, and complex player-environment interactions. We adopt the online gameplay dataset of Pearce and Zhu [ [40] (5.5M frames, 95 hours) and compare against DIAMOND [3], a strong autoregressive diffusion-based world model. All other baselines share the same backbone architecture and training budget of 550k gradient steps. Both tokenizers ElasticTok and DAT were trained for 100k gradient steps, for the same token dimensions and count per window. During inference, we condition on 4 context frames and autoregressively roll out 12 predicted frames. We use HG-H for models with ElasticTok and DAT tokenizers. Further details are provided in Appendix § 8.6.

As shown in Table 2, DAT outperforms the existing popular baseline DIAMOND and other tokenization methods using the same diffusion-backbone architecture and training budget. The results for models with ElasticTok and DAT showcase the effectiveness of HG-H, and its amplification when combined with DAT.

Method	SSIM$\uparrow$	PSNR$\uparrow$	FVD$\downarrow$	FID$\downarrow$	LPIPS$\downarrow$
DIAMOND-Fast	0.449	18.2	577.1	115.6	0.547
DIAMOND-HQ	0.447	18.3	368.5	87.2	0.510
SD-VAE	0.496	17.29	231.9	70.25	0.514
ElasticTok(HG-H)	0.494	17.22	197.1	74.2	0.519
DAT(HG-H)	0.479	17.24	123.0	53.6	0.493

Compression Forcing for Robot Manipulation

Robot Manipulation is another great task to measure a world model’s capability, especially in real environments and physical constraints provided by the RT-1 dataset [9]. The RT-1 dataset contains 130k episodes and 700+ distinct tasks. We use the same setup for baselines as in CS:GO and train DAT and the other baselines for 80k gradient steps. Both tokenizers ElasticTok and DAT were trained for 40k gradient step, with same number of tokens and their dimension. For evaluation we roll out 16 generated frames from a context of 8 frames. We compare our method against two world-modeling paradigms: DIAMOND [3] a diffusion-based world model and Dreamer 4 [18], a latent dynamics RL agent. Further details and extended visualizations are provided in Appendix § 8.7 and Appendix § 9.§ 5.

As shown in Table 3, we observe a substantial performance gap between these established baselines and our diffusion-backbone variants. While DIAMOND and Dreamer 4 struggle with structural drift and temporal blurring in the RT-1 domain, Compression Forcing maintains high-fidelity reconstructions and motion continuity.

Method	SSIM$\uparrow$	PSNR$\uparrow$	FVD$\downarrow$	FID$\downarrow$	LPIPS$\downarrow$
SD-VAE	0.722	20.47	120.4	42.82	0.267
ElasticTok(HG-H)	0.721	20.42	105.4	44.86	0.267
DIAMOND	0.6130	14.05	382.4	104.50	0.3267
Dreamer 4	0.2718	16.27	984.1	365.67	0.6050
DAT(HG-H)	0.721	20.43	101.5	42.77	0.266

Analysis

Emergence of Hierarchical Features in Tokenization. To verify whether our Dynamics-Aware Tokenization (DAT) learns hierarchical representations ordered by dynamic saliency, we visualize cross-attention maps between DAT tokens and image patches in the dynamic decoder. As shown in Figure 2, the attention maps exhibits a clear saliency-ordered hierarchy. Leading tokens assign high attention weights to regions of active motion (the athlete in motion), while higher-index slots progressively shift toward static background elements such as the court boundaries. This confirms that our joint schedule-weighted objective induces DAT to decouple dynamic interactions from visual nuisances, effectively encoding historical information into hierarchical representations.

Spatial Distribution of Prediction Errors. To assess whether this hierarchical tokenization translates to improved prediction in dynamics-critical regions, we analyze per-pixel prediction error across autoregressive rollouts on RT-1 (Figure 6). While baseline tokenizers (SD-VAE and ElasticTok) accumulate substantial and progressive errors around the manipulated target object and the robot’s end-effector, regions most critical for accurate physical interaction, Compression Forcing maintains markedly low error in these dynamic regions throughout the prediction horizon, showcasing practical benefits of aligning history compression with predictive saliency rather than perceptual fidelity. Additional visualizations and further analysis can be found in Appendix § 9.6.

Conclusion

We introduce Compression Forcing, a novel world modeling framework for built upon the principle of partial compression as partial masking. Through the co-design of Dynamics-Aware Tokenization (DAT), a prefix-conditioned diffusion backbone, and Hierarchical History Guidance (HG-H), our approach compresses history into sequences ordered by dynamic saliency and selectively amplifies dynamics-critical information at inference time. Extensive experiments across real-world video forecasting, 3D game simulation, and robotic manipulation demonstrate that Compression Forcing achieves improved prediction fidelity and downstream decision-making performance compared to prior methods, showcasing the broad applicability of our method.

While we believe this work marks significant progress in rethinking world model design principles in a task-centric manner, there is substantial room for improvement. Promising directions include extending prediction beyond single next-frame to multi-step [5] or variable-horizon [23, 24] futures, and differentiating truncation budgets between recent and distant history to better capture their distinct roles in prediction. More broadly, we hope that the principle of partial compression as partial masking serves as a useful foundation for future exploration in world models and beyond.

Theoretical Analysis

We provide a linear motivation for DAT’s residual formulation by comparing low-rank projections of dynamics residuals and raw future sequences.

Setup. Let $\mathbf{x}_t \in\mathbb{R}^d$ denote the current observation and $\mathbf{Y} \triangleq\mathbf{x}_{t+1:t+\tau} - \mathbf{x}_t \in\mathbb{R}^{d\times\tau}$ the ground-truth dynamics residual (stacked as columns). We model the raw future as $\mathbf{F} \triangleq\mathbf{x}_{t+1:t+\tau} = \mathbf{x}_t\mathbf{1}^{\mathsf{T}} + \mathbf{Y}$, i.e. a static component plus a dynamics component. We assume access to $N$ i.i.d. samples $\{(\mathbf{x}_t^{(n)},\mathbf{F}^{(n)},\mathbf{Y}^{(n)})\}_{n=1}^{N}$, and define the empirical uncentered second-moment matrices:

where $\operatorname{vec}(\cdot)$ denotes column-wise vectorization. We then denote the static uncentered second-moment matrix as $\mathbf{C}_{S} \triangleq\frac{1}{N}\sum_{n=1}^{N}\operatorname{vec}(\mathbf{x}_t^{(n)}\mathbf{1}^{\mathsf{T}})\operatorname{vec}(\mathbf{x}_t^{(n)}\mathbf{1}^{\mathsf{T}})^{\mathsf{T}}$.

Two projection problems. In the projection objectives and errors below, $\mathbf{F}$ and $\mathbf{Y}$ denote their vectorized forms, and expectations are taken over the empirical sample distribution. We consider learning a rank-$m$ orthogonal projector $\mathbf{P} \in\mathbb{R}^{d\tau\times d\tau}$ (with $\mathbf{P}^{\mathsf{T}} = \mathbf{P}$, $\mathbf{P}^{2} = \mathbf{P}$, $\operatorname{rank}(\mathbf{P}) = m$) that best preserves the data under ridge regularization:

• Raw projection (analogous to ElasticTok / standard tokenization):

• Residual projection (analogous to DAT’s dynamic prediction objective):

To formally prove Proposition A.1, we first establish two supporting lemmas regarding the properties of orthogonal projectors and second-moment matrices.

Proof. Using the property $\lVert v\rVert^2 = \operatorname{tr}(vv^{\mathsf{T}})$ and the idempotence of the orthogonal projector $(I-P)^{\mathsf{T}}(I-P)=(I-P)$, we have:

Proof. For any rank-$m$ orthogonal projector, the Frobenius norm squared is constant:

Thus, the regularization term $\lambda m$ does not affect the optimization landscape. Then by Lemma A.2, the problem reduces to:

By the Eckart-Young-Mirsky theorem, this trace is minimized when $P$ projects onto the subspace spanned by the eigenvectors corresponding to the $m$ largest eigenvalues of $C$.

Empowered with these Lemmas, we then complete the proof of Proposition A.1.

Proof of Proposition A.1. Let $\{(\sigma_i^Y,u_i^Y)\}_{i=1}^{d_\tau}$ denote the eigendecomposition of $C_Y$, with eigenvalues sorted in descending order.

By Lemma A.3, the residual projector $P_Y^*$ is the global minimizer for the reconstruction error of $Y$ among all rank-$m$ projectors. Its column space $U_m^Y$ exactly spans the top-$m$ eigenvectors of $C_Y$. Applying Lemma A.2, the expected error achieves the theoretical minimum:

Now consider the raw projector $P_F^*$, which projects onto $U_m^F$ (the top-$m$ eigenspace of $C_F$). Using Lemma A.2, its expected reconstruction error on the dynamics residual $Y$ is:

Since $P_Y^*$ is the global minimizer of $\operatorname{tr}((I-P)C_Y)$ over all rank-$m$ projectors, any other rank-$m$ projector, including $P_F^*$, must yield error no less than $P_Y^*$, establishing the inequality:

The weak inequality follows directly from residual-projector optimality and does not require Assumption A.1; the decomposition only provides an interpretation of the difference between the two objectives.

Under the eigengap assumption, the residual-optimal projector is unique. Equality holds if and only if $\operatorname{span}(U_m^F)=\operatorname{span}(U_m^Y)$, which requires $\operatorname{span}(U_m^Y)$ to be simultaneously an invariant subspace of both $C_Y$ and $C_S$.

Connection to DAT. Proposition A.1 provides a linear illustration of the intuition behind DAT’s dual-objective design (§ 3.1). The static reconstruction loss $\mathcal{L}_{\mathrm{static}}$ (Eq. (6)) is analogous to the raw projection objective, whose optimal projector $\mathbf{P}_{F}^{*}$ is aligned with the full second-moment matrix $\mathbf{C}_{F}$. The dynamic prediction loss $\mathcal{L}_{\mathrm{dyn}}$ (Eq. (7)), which conditions on $\mathbf{x}_{t}$ and supports residual learning, is analogous to the residual projection objective with optimal projector $\mathbf{P}_{Y}^{*}$. By weighting $\mathcal{L}_{\mathrm{dyn}}$ more heavily for small prefix lengths $m$ (via the schedule $\omega(m)$), DAT encourages the leading tokens to prioritize residual changes. As the prefix grows and $\omega(m)$ increases, $\mathcal{L}_{\mathrm{static}}$ receives greater weight, encouraging later tokens to recover visual details needed for full reconstruction.

Extended Discussions

Partial Compression as Partial Masking

While in this paper, we mainly instantiate the principle of partial compression as partial masking for world modeling, the applicability of this approach is more general, offering an alternative to existing mechanisms for controlling the information content of conditioning signals in generative models.

General formulation. Consider a conditional generative model $p_{\theta}(\mathbf{x}\mid\mathbf{c})$, where $\mathbf{c}$ denotes an arbitrary conditioning signal. Suppose we have access to an ordered encoder $\mathcal{E}$ that compresses $\mathbf{c}$ into a hierarchical latent sequence $\mathbf{z}_{1:L}=\mathcal{E}(\mathbf{c})$, where the hierarchy is induced by a task-specific objective such that truncating to a prefix $\mathbf{z}_{1:m}$ retains the information most relevant to the downstream task. Then, by training $p_{\theta}(\mathbf{x}\mid\mathbf{z}_{1:m})$ to condition on variable-length prefixes $\mathbf{z}_{1:m}$ for $m\in[0,L]$, truncation naturally serves as a structured form of partial masking. Essentially, the flexibility of conditioning on an arbitrary-lengthed prefix, the model is capable of generation with task-aligned partial information rather than information filtered by a fixed, human-imposed prior (e.g., frequency-based decomposition in Diffusion Forcing Chen et al. [ [8], Dieleman [ [11]).

This formulation immediately enables score-based guidance in the diffusion setting. Treating $p_{\theta}$ as a learned score estimator, one can compose score functions across different prefix lengths to selectively amplify specific information components:

Task-specific tokenization, task-agnostic guidance. A key property of this formulation is the clean separation between two design choices. The tokenizer $\mathcal{E}$ is necessarily task-specific: the ordering it learns must reflect the information hierarchy relevant to the downstream objective. In world modeling, DAT orders tokens by dynamic saliency (§ 3.1); in other domains, the appropriate hierarchy would differ, for instance, key frame is visual retrieval tasks, function groups in protein folding or high-entropy decision-making moments in planning.

In contrast, the guidance mechanism above is entirely task-agnostic: once a hierarchical tokenization exists, score composition over variable-length prefixes provides a universal mechanism for inference-time amplification, regardless of what the hierarchy encodes. This separation means that the Compression Forcing framework can serve as inspiration for tasks not limited to world models, where prefix-conditioned generation with hierarchical tokenization can facilitate flexible amplification of control signals for generation.

Hierarchical History Guidance

Notably, our framework can also be integrated with history guidance methods along the time axis [44] by setting different budget $l_i$ for each history segment, which we leave for future work. The general guidance formulation in Eq. (11) subsumes several existing conditioning strategies as special cases, and its relationship to prior work merits clarification.

Connection to DFoT history guidance. DFoT [44] steers future generation by composing score functions conditioned on history observations interpolated with isotropic Gaussian noise of varying levels $k_{\mathcal{H}}$. While in the paper, we mainly discuss the fractional history guidance, which can be viewed as a special case of our formulation with

where $I_k(\mathbf{x}_{t:t+\tau}) \triangleq(1-k)\cdot\mathbf{x}_{t:t+\tau}+k\cdot\mathbf{w}, \mathbf{w}\sim\mathcal{N}(0,\mathbf{I})$, as in flow-matching and stochastic interpolants literature [33] (Albergo and Vanden-Eijnden 2023; Albergo et al. 2023).

Composability along the temporal axis. While for the sake of simplicity we focus on hierarchical history guidance with uniform budgets for each history window, i.e. $l_j^{\mathrm{dyn}}=[m_j,\ldots,m_j]$ across all $\mathcal{J}$ windows, our framework naturally extends to heterogeneous truncation budgets across history windows. This is analogous to temporal history guidance as in Song et al. [ [44]. We leave systematic exploration of such temporally heterogeneous guidance strategies to future work.

Connection to classifier-free guidance. Classic Classifier-free guidance (CFG) Ho and Salimans [ [20], in for instance text-conditioned image generation, amplifies the influence of a conditioning signal $\mathbf{c}$ by extrapolating between the conditional and unconditional score estimates: $\widetilde{\nabla}=(1+\omega)\nabla\log p(\mathbf{x}|\mathbf{c})-\omega\nabla\log p(\mathbf{x})$. This can be viewed as applying a binary mask of the control signal. HG-H generalizes this by decomposing $\mathbf{c}$ itself into a hierarchy of sub-conditions $\{\mathbf{H}^{(l_j)}\}$ and amplifying specific components. When $\mathcal{J}=2$ with $l_1=0$ (empty history) and $l_2=[L,\ldots,L]$ (full history), HG-H reduces to standard CFG. Building upon hierarchical structures tokenized via DAT, Compression Forcing enables finer-grained control that is not possible for methods treating the conditioning signal as monolithic.

Tokenization in World Models

While most recent pixel-space world models [14, 45, 23] inherit visual tokenization from image and video generation models [6, 48], earlier work in latent-state and discrete-token-based world modeling has explored representation designs with dynamic awareness. Following the line of works with RSSM-style backbones [16, 17], Iso-Dream [39] proposes three separate branches during tokenization, explicitly disentangling controllable, noncontrollable, and static components from visual data. Following discrete token representation methods in world modeling [47, 35], $\Delta$-IRIS [36] shows improvements by conditioning the tokenizer on previous frames and actions, intuitively encoding only stochastic deltas between timesteps. However, to the best of our knowledge, DAT is the first to learn a hierarchical order over tokens by dynamic saliency, rather than a binary separation into predefined categories. This ordering ensures that any truncation point produces a valid representation that smoothly interpolates between dynamics-only and full-detail conditioning, directly enabling variable-rate compression and the inference-time guidance mechanism (HG-H) introduced in § 3.3.

Experiment Details

Environment Details

Kinetics-600 [7] is a well-known video generation benchmark with $\sim400\mathrm{k}$ human action videos labeled into 600 classes. We use $128 \times128$ pixel resolution, following other works with this dataset. This dataset is difficult because of the significant diversity among videos. We use it to test the efficacy of our DAT, backbone, and HG-H in an action-free setting.

CS:GO [40] is a dataset of gameplay videos from the popular video game CS:GO, with actions represented per-timestep as a list of discrete values. We use the largest dataset subset, hdf5_dm_july2021, with about 95 hours of diverse online human gameplay on the Dust II map.

RT-1 [13] is a widely used real-world robot experience dataset, with multiple manipulation tasks executed by an embodied robot arm. Episodes are sampled at 3 frames per second.

Metrics Details

Peak Signal-to-Noise Ratio (PSNR): PSNR [22] measures pixel-level fidelity by comparing the maximum possible pixel intensity to the MSE between a generated and target frame. With $L$ being the maximum possible pixel value, PSNR is calculated with

Structural Similarity Index Measure (SSIM): SSIM [49] measures perceived visual qualities. It jointly compares luminance, contrast, and structural similarity rather than raw pixel error alone. For generated and target patches $x$ and $y$, SSIM is defined as

Here, $\mu_x$ and $\mu_y$ denote the mean intensities of the two patches, $\mu_x^2$ and $\mu_y^2$ are their variances, and $\sigma_{xy}$ is their covariance. $L$ is dynamic range of the pixel values, and $C_1=(K_1L)^2$ and $C_2=(K_2L)^2$ are constants for numerical stability. SSIM is particularly useful when two images are similar at a high level but differ in local contrast or structural details, with higher SSIM values indicating greater similarity. Following [23], we compute SSIM over sliding $11 \times11$ Gaussian windows and average those local values into an image-level SSIM.

Learned Perceptual Image Patch Similarity (LPIPS): LPIPS [57] measures similarity in a learned feature space. Instead of comparing images through raw intensity differences, it compares deep feature representations extracted from a pretrained network, making it more sensitive to perceptually meaningful differences in texture, structure, and appearance. For generated and target images $x$ and $x'$, LPIPS is computed as

where $\hat{y}^l,\hat{y}^{\prime l}\in\mathbb{R}^{H_l\times W_l\times C_l}$ are the channel-wise normalized feature maps of $x$ and $x'$ at layer $l$, $H_l$, $W_l$, and $C_l$ are the spatial height, spatial width, and number of channels of that layer, and $w_l\in\mathbb{R}^{C_l}$ is a learned channel-wise weighting vector. The metric averages squared feature differences across spatial locations and sums them across layers. Lower LPIPS values indicate greater perceptual similarity.

Fréchet Inception Distance (FID): FID [19] is a distribution-level metric for evaluating the realism of generated images. Rather than comparing each generated sample to a specific target image, it compares the feature distributions of real and generated images after embedding them with a pretrained Inception network. Let $(\mu_r, \Sigma_r)$ and $(\mu_g, \Sigma_g)$ denote the empirical mean and covariance of the real and generated feature embeddings, respectively. FID is defined as

Here, $\mu_r$ and $\mu_g$ are the means of the real and generated feature distributions, $\Sigma_r$ and $\Sigma_g$ are their covariance matrices, and $\operatorname{Tr}(\cdot)$ denotes the matrix trace. FID is useful because it measures similarity at the distribution level, capturing both sample quality and diversity rather than only pixel-wise agreement. Lower FID values indicate that the generated samples more closely match the real data distribution.

Fréchet Video Distance (FVD): FVD [46] extends the idea of FID from images to videos by comparing distributions of real and generated videos in a learned spatiotemporal feature space. Instead of evaluating frames independently, it embeds entire video clips using a pretrained Inflated 3D ConveNet (I3D) video recognition network, allowing it to account for temporal coherence and motion in addition to visual quality. FVD is formulated identically to FID except that it uses I3D activations instead of Inception activations.

Baselines Details

We compare against two groups of baselines. The first group is designed to isolate the effect of the history representation while keeping the world-model backbone as close as possible to ours. Concretely, these models use the same conditional diffusion backbone, action conditioning, and auto-regressive rollout protocol as Compression Forcing, and differ only in how the history observations are compressed before being provided to the backbone. The second group consists of strong external world-model baselines, namely DIAMOND and DreamerV4, which represent established alternative paradigms for action-conditioned future prediction.

SD-VAE: This baseline uses the Stable Diffusion VAE [42] as the history encoder. Each context frame is encoded independently into a spatial latent map with a fixed per-frame budget, and the diffusion backbone conditions on the stack of these latent tensors directly. As a result, every frame contributes the same amount of unmasked conditioning information regardless of how informative it is for prediction, and the representation is optimized for perceptual reconstruction rather than video-predictive utility.

ElasticTok: ElasticTok [53] is a variable-rate tokenizer that maps visual inputs to an ordered one-dimensional token sequence. In contrast to SD-VAE, it allows the history to be represented by a prefix of varying length, where earlier tokens are intended to carry the most important reconstruction information. In our experiments, we use a frozen pretrained ElasticTok encoder and feed its output tokens to the same diffusion world-model backbone used by our method. Depending on the environment, the history is encoded either frame-wise or over short sliding windows, after which the resulting token sequence is consumed by the backbone through the same cross-attention interface as our tokenizer-based models. ElasticTok thus provides an adaptive and prefix-truncatable representation, but its token ordering is learned from reconstruction objectives, not from the future-prediction objective that drives DAT. Hierarchical guidance is applied to ElasticTok in the same way as DAT.

DIAMOND: DIAMOND [3] is an autoregressive diffusion world model that predicts future observations conditioned on recent observations and actions. We use the official public implementation¹ for this baseline. Unlike DAT, DIAMOND does not expose a learned hierarchical history representation that can be truncated or selectively amplified at inference time.

DreamerV4: DreamerV4 [18] learns a world model consisting of a causal tokenizer and an action-conditioned dynamics model. Future trajectories are generated by rolling the latent dynamics forward under the action sequence and decoding predicted observations from the latent representations. Since the official implementation is not open-sourced, we use a popular and well-tested community-contributed DreamerV4 implementation². DreamerV4 is included because it is a strong recent world-model baseline, but it differs substantially from our approach in both training objective and architecture; its latent dynamics model compresses each frame into a flat discrete token sequence with no explicit ordering by predictive importance, while Compression Forcing’s DAT learns to order tokens by dynamic saliency.

Shared Tokenizer and Backbone configurations.

The following hyperparameters are shared across the Kinetics, CS:GO, and RT-1 datasets. Note that the DAT has an input-independent null token prepended to 255 data-dependent tokens, bringing the total number of tokens to 256. When making unconditional generations from the DAT decoder, we only condition on the null token. The DAT decoder is implemented using $\varepsilon$-prediction in the VAE latent space; the $x$-prediction form in Eqs. (6)–(7) denotes clean-latent prediction with the corresponding timestep-dependent weighting left implicit, since the two squared-error objectives are equivalent under the standard output reparameterization and SNR-dependent reweighting [29].

Shared ElasticTok tokenizer configuration. Across all datasets, the ElasticTok tokenizer used hidden size 1024, feed-forward size 2048, 10 encoder layers, 10 decoder layers, 8 attention heads, and RoPE $\theta= 5 \times10^{6}$. Frame windows were encoded into 256 tokens of dimension 16. We trained in FP32 using AdamW with learning rate $1 \times10^{-4}$, weight decay $1 \times10^{-4}$, and 2000 learning-rate warmup steps, with KL weight $1 \times10^{-8}$ and LPIPS loss ratio 0.1. We trained each ElasticTok tokenizer on 8 GPUs.

Shared CDiT backbone configuration. Across all datasets, the SDVAE, ElasticTok, and DAT-conditioned backbones used 12 layers, patch size 2, MLP ratio 4.0, and 4 latent input channels. Diffusion used 1000 timesteps with a linear noise schedule, $\beta_1 = 1 \times10^{-4}$, $\beta_T = 2 \times10^{-2}$, $\varepsilon$-prediction, and learned variance. Backbone training used AdamW with learning rate $8 \times10^{-5}$, weight decay 0, and gradient clipping with max norm 10.0. We trained each backbone on a single GPU.

Details of Compression Forcing for Dynamics Video Generation

Training. We trained the ElasticTok and DAT tokenizers to tokenize 8 frame windows. We trained both models to convergence, so we stopped DAT after 120k steps and ElasticTok after 175k steps.

Hyperparameter	Value
Encoder Architecture
Model	ViViT
Depth	12
Hidden size	768
Patch size	16
Attention heads	12
Tokens per window	256
Token dimension	16
Decoder Architecture
Model	Latte
Depth	12
Hidden size	384
Patch size	2
Attention heads	6
Diffusion
Timesteps	1000
Noise schedule	Linear
$\beta_{1}$	1e-4
$\beta_{T}$	0.02
Prediction target	$\epsilon$
Latent diffusion	Yes
Latent spatial downsample	$8 \times$
Latent channels	4
Latent scaling factor	0.18215
Training
Optimizer	AdamW
Learning rate	1e-4
Weight decay	0
Gradient clip max norm	0.1
Gradient clip start iteration	20 000
GPUs	8
Float precision	FP32
Data Processing
Normalization	$[-1, 1]$
Frame stride	1

The backbones had hidden size 768 and 12 attention heads, and we trained all backbones for 160k steps.

Evaluation. We condition on 16 context frames and autoregressively roll out 8 future frames. All metrics are computed over 256 evaluation videos. For both our DAT(HG-H) and ElasticTok with guidance, we use the general guidance formulation with $\vec{\omega} = (-0.5, 1.5)$ and $\vec{l} = (128, 256)$.

Details of Compression Forcing for Game Simulation

Training. We trained the ElasticTok and DAT tokenizers to tokenize 4 frame windows. We trained both for 100k steps with batch size 128. The backbones had hidden size 1024 and 16 attention heads, and we trained the backbones with SDVAE, ElasticTok and DAT history conditioning for 550k steps each.

Evaluation. We condition on 4 context frames and autoregressively roll out 12 future frames. All metrics are computed over 500 evaluation videos. For both our DAT(HG-H) and ElasticTok(HG-H) evaluations, we used the Hierarchical History Guidance formulation with $\vec{\omega} = (2, 4, 6)$ and $\vec{l} = (1, 16, 128, 256)$.

Details of Compression Forcing for Robot Manipulation

Training. We trained the ElasticTok and DAT tokenizers to tokenize 4 frame windows. We trained both for 40k steps with batch size 512. The backbones had hidden size 768 and 12 attention heads, and we trained the backbones with SDVAE, ElasticTok, and DAT history conditioning for 80k steps each.

Evaluation. We condition on 8 context frames and autoregressively roll out 16 future frames. All metrics are computed over 500 evaluation videos. For both our DAT(HG-H) and ElasticTok(HG-H) evaluations, we used the Hierarchical History Guidance formulation with $\vec{\omega} = (0.4, 0.8, 1.2)$ and $\vec{l} = (1, 64, 128, 256)$.

Analysis Details

Details on Tokenization Attention Visualization. To produce the attention maps shown in Figure 2, we visualize the cross-attention weights between DAT tokenized slot tokens and image patches within the dynamic decoder $\mathcal{D}_{\mathrm{dyn}}$. Concretely, we provide the decoder with a clean video sequence and set the denoising timestep to zero, and visualize the first cross-attention layer’s attention weights. The query Q corresponds to the patchified video latent tokens inside the decoder, and the key-value pair (K, V) corresponds to the slot tokens $z_{1:L} = \mathcal{E}(x_{t:t+\tau})$ produced by the ViViT encoder followed by the encoder-to-slot projection. For a given slot index $i$, we extract the attention weights from all decoder patches to slot $z_i$, reshape them back to the spatial grid, and bilinearly upsample to the original frame resolution. The resulting heatmaps are overlaid on the raw video frames. As shown in Figure 2, earlier slot indices (e.g., slot 16) focus attention on regions of active motion, while later indices (e.g., slot 254) shift attention toward static background elements, confirming the dynamic-to-static ordering induced by DAT.

Details on Prediction Error Spatial Distribution. To produce the error maps shown in Figure 6, we compute per-pixel absolute error between predicted and GT frames across all compared methods in the RT-1 domain. For each method, we compute the mean absolute error across color channels at every pixel, yielding a single-channel error map per frame. This pixel-wise error is mapped to an RGB heatmap and blended onto the predicted frame, so that the spatial distribution of errors can be interpreted in the context of the generated scene content.

Extended Experiments and Visualizations

Real2Sim Policy Evaluation

Background. A world model that faithfully captures environment dynamics should not only produce visually plausible rollouts but also aid downstream decision-making. Real2Sim policy evaluation Li et al. [ [32] provides a principled protocol for this purpose: by using simulation or world models as a surrogate simulator, one can assess whether it correctly distinguishes successful from unsuccessful manipulation trajectories and, crucially, whether it preserves the absolute success rate and relative performance ranking among policies observed in the real world. This interactive evaluation is particularly demanding because prediction errors compound over long horizons; a reliable world model must maintain both visual coherence and physical plausibility throughout the entire rollout.

Task Setup. We follow the evaluation protocol of Li et al. [ [32] as adopted in Huang et al. [ [23]. We select the open drawer task from the RT-1 dataset and evaluate four policies at different training stages: RT-1 (Begin), RT-1 (15%), RT-1 (Converged) Google [ [13], and RT-1-X. For each policy, we sample $N=50$ trajectories and extract their initial frames. These frames are provided to the policy to produce actions, which are then fed into the world model to predict the next observation. The policy subsequently acts on the predicted frames in a closed-loop, interactive manner. We autoregressively generate each new frame conditioned on the last $L=8$ frames. This process continues until a rollout of length $H=40$ is reached. We then apply a trajectory-level success verifier $\psi$ to each completed rollout and compute the overall success rate as $\frac{1}{N}\sum_{n=1}^{N}\psi(o_{0:H}^{(n)})$. The full procedure is described in Algorithm 1.

Baselines and Policy Models. We compare CF against the same two tokenization baselines used in our main experiments: SD-VAE and ElasticTok, both equipped with the same diffusion backbone and rollout protocol. For each world model, we run the full interactive evaluation loop described above, producing 50 rollouts per policy. We evaluate four RT-1 policies that span a broad range of training progress: RT-1 (Begin), an early-stage checkpoint with near-zero real-world success; RT-1 (15%), trained on 15% of the data; RT-1 (Converged), the fully trained model; and RT-1-X, the cross-embodiment variant. Their real-world success rates (Table 5) increase monotonically from Begin to Converged, with RT-1-X falling between Begin and 15%. This monotonic spread provides a clear ground-truth ranking: a faithful world model should recover this ordering in its predicted success rates.

	RT-1 (Begin)	RT-1 (15%)	RT-1 (Conv.)	RT-1-X
Ground Truth	0	70.4	81.5	51.9
CF (Ours)	2	54	64	40
ElasticTok	6	40	48	36
SD-VAE	2	36	32	30

Algorithm 1 Real2Sim Policy Evaluation

Require: World model $P(\mathbf{o}_{t+1} \mid\mathbf{o}_{\leq t}, \mathbf{a}_{\leq t})$, policy $\pi(\mathbf{a}_t \mid\mathbf{o}_t)$, task $\kappa$, initial-frame set $\mathcal{D}_{\kappa}$, verifier $\psi(\mathbf{o}_{0:H}) \to\{0,1\}$.

1. success_count $\leftarrow0$

2. for $n = 1,\ldots,N$ do

3. &nbsp;&nbsp;&nbsp;&nbsp;Sample initial frame $\mathbf{o}_0$ from $\mathcal{D}_{\kappa}$

4. &nbsp;&nbsp;&nbsp;&nbsp;for $t = 0,\ldots,H-1$ do

5. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Sample action $\mathbf{a}_t \sim\pi(\cdot\mid\mathbf{o}_t)$

6. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;if $t < L$ then

7. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$\mathbf{o}_{t+1} \sim P(\mathbf{o}_{t+1} \mid\mathbf{o}_{\leq t}, \mathbf{a}_{\leq t})$

8. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;else

9. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;$\mathbf{o}_{t+1} \sim P(\mathbf{o}_{t+1} \mid\mathbf{o}_{t-L:t}, \mathbf{a}_{t-L:t})$

10. &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end if

11. &nbsp;&nbsp;&nbsp;&nbsp;end for

12. &nbsp;&nbsp;&nbsp;&nbsp;success_count $\leftarrow$ success_count $+ \psi(\mathbf{o}_{0:H})$

13. end for

14. return success_rate $= \frac{1}{N} \cdot$ success_count

Experimental Results. We assess each world model by comparing its predicted policy success rates against the real-world ground-truth rates (Table 5). As shown in Figure 7, CF (Ours) tracks real-world performance most accurately across all four policies, preserving both the correct ranking and approximate magnitude of success rates. In particular, SD-VAE inverts the ranking between RT-1 (15%) and RT-1 (Converged), predicting a higher success rate for the weaker policy, whereas both ElasticTok and CF preserve the correct ordering. CF further stands out by maintaining the tightest alignment with the $y=x$ diagonal, indicating that its predicted success rates most closely approximate absolute real-world performance rather than merely recovering the relative ranking.

Human Verification. In practice, the trajectory success verifier $\psi$ is realized through human evaluation. Annotators watch each generated rollout depicting the robot attempting to open a drawer and label the outcome according to the following criteria:

• Success: By the final frame, the drawer is visibly pulled out from the cabinet, with a clear gap between the drawer face and the cabinet frame, and the drawer remains stationary in the open position.

• Failure: The drawer remains closed or barely displaced, the robot fails to grasp the handle, the drawer slides back after being pulled, the robot jams or stops prematurely, or the viewpoint prevents the annotator from confirming that the drawer has been opened.

Object Tracking

Beyond perceptual metrics, we evaluate whether generated videos preserve the physical dynamics of manipulated objects by measuring object tracking fidelity in RT-1 domain. While metrics such as FVD and LPIPS capture global perceptual similarity, they do not directly assess whether the model faithfully reproduces object motion trajectories, a property critical for downstream decision-making in robotic manipulation.

Setup. We evaluate on 64 episodes from the RT-1 dataset. For each episode, we first obtain class-agnostic object masks on the ground-truth frame at generation onset using SAM (ViT-Base) Kirillov et al. [ [30], filtering by area (0.5%–50% of image) and deduplicating by IoU $> 0.5$. We then sample 10 query points uniformly within each detected mask and track them using CoTracker3 [27] (offline mode) across 16 generated frames. The trackers starts at the first conditioned frames, and tracks object positions throughout generation. We compare our model with baselines in the main paper, with the same backbone and different tokenizers: SD-VAE and ElasticTok.

Metric. We propose a two-component tracking error that decomposes into base (accuracy of survived points) and penalty (cost for lost points):

where $g_t^i$ and $p_t^i$ are the ground-truth and predicted positions of point $i$ at frame $t$, $v_t^i$ denotes ground-truth visibility, and $c_t^i$ is a cumulative penalty that accumulates the ground-truth frame-to-frame displacement for each frame the point remains lost:

This design ensures that the penalty is proportional to the motion importance of lost points: losing a static background point incurs negligible cost, while losing a point undergoing significant motion incurs a large, accumulating penalty.

All metrics are computed per episode and then averaged equally across episodes, including the penalty per lost point (set to zero for episodes with no lost points).

Results. As shown in Table 6, Compression Forcing achieves the the highest per-episode win rate (48.4%). While ElasticTok achieves the lowest base error (0.871 px), it suffers from the highest point loss rate (18.7%), indicating poor structural consistency in generated videos. SD-VAE maintains good point survival (91.3%) but incurs higher penalty per lost point (21.1 px), suggesting that its lost points tend to lie in high-motion regions. In contrast, Compression Forcing balances tracking accuracy with point survival: when points are lost, they tend to be in lower-motion regions (19.7 px per lost point), indicating that our model better preserves object dynamics precisely where it matters most, at contact points and regions of active manipulation. This finding is consistent with the per-pixel error analysis in Figure 6, where Compression Forcing’s error accumulation is less pronounced at dynamic contact regions compared with baselines.

Method	Base$\downarrow$	Penalty$\downarrow$	Total$\downarrow$	Tracked%	Win Rate
ElasticTok	0.871	3.854	4.725	81.3%	6.2%
SD-VAE	1.473	2.624	4.097	91.3%	45.3%
DAT (Ours)	1.462	2.451	3.914	90.8%	48.4%

Generation Results on Kinetics

We present extended qualitative results on the Kinetics-600 video prediction domain. All rollouts are conditioned on 16 context frames and autoregressively generate 8 predicted frames across diverse motion-rich natural videos.

As shown in Figure 10, Compression Forcing produces rollouts that preserve both the structure of the scene and the continuity of motion, maintaining coherent body poses and object interactions throughout the prediction horizon.

Generation Results on CS:GO

We present extended qualitative results of Compression Forcing on the CS:GO environment. All rollouts are conditioned on 4 context frames and autoregressively generate 8 predicted frames.

General Generation Quality. Figure 11 showcases diverse gameplay scenarios including firefights, aiming through lenses, and open-area traversal. Across all examples, Compression Forcing produces temporally coherent rollouts that preserve scene geometry and maintain consistent weapon positioning. Even under viewpoint changes, the model faithfully renders environmental structures such as walls, crates, and doorways without structural drift.

Action Following. Figure 12 illustrates the model’s fidelity to action-conditioned transitions. In the first example, the player exits the scope view, and the generated frames correctly capture the transition from the zoomed-in scope overlay to the standard field of view. In the second example, the player enters scope mode, and Compression Forcing accurately renders the narrowing of the visual field and the appearance of the scope reticle. Both examples confirm that the model has learned to associate discrete action inputs with their corresponding visual effects.

Environment Stochasticity. Figure 13 illustrates Compression Forcing’s capability of capturing environment stochasticity, generating feasible yet different plausible respawn positions.

Generation Results on RT-1

We present extended qualitative results on the RT-1 robot manipulation domain. All rollouts are conditioned on 8 context frames and autoregressively generate 16 predicted frames across diverse manipulation tasks including picking, placing, and drawer interactions.

Figure 14 presents extended qualitative results on the RT-1 domain. Compression Forcing produces rollouts that accurately track the robot arm’s trajectory and preserve fine-grained contact interactions with the manipulated objects throughout the prediction horizon.

Extended Visualizations on Analysis Experiments

We provide extended visualizations for the two main analysis experiments discussed in § 4.5. Figure 15 presents additional attention map samples from DAT across different video sequences, and Figure 16 extends the per-pixel prediction error comparison to additional RT-1 episodes.