WorksComputer Science
Beckmann World Models: Direct Terminal-Map Learning for One-Step Action-Conditioned Prediction
Abstract
Action-conditioned world models support planning and policy evaluation by predicting future states. However, learning an accurate one-step predictor can require substantial training computation. We introduce Beckmann World Models (BWM), which learn a direct map from noise, visual history, and actions to future frames without a pretrained generative teacher. We adapt the terminal-map conservation law of Beckmann Transport Models to action-conditioned video prediction, training directly from noise-data pairs without conditional populations of generated samples required by DriftWorld. However, transport consistency on interpolated inputs does not ensure accurate prediction from the pure noise used at inference. We address this gap with endpoint-aligned training that directly supervises the source prediction, emphasizes motion and spatial structure, and teaches recovery from generated histories. All components train the same map and retain one network call per future chunk. On PushT and Robomimic Can, the final model reduces 64-frame prediction MSE by 68% and 57%, respectively, relative to same-backbone regression baseline. For policy evaluation, BWM achieves the lowest mean IoU error among the evaluated models, 0.0100, at 24.8 ms per four-frame PushT prediction. On Robomimic Can, the BWM stage trained with motion supervision and generated histories achieves 3.3 times higher throughput and 78% lower peak memory than local one-step DriftWorld. In a latent-space extension to RT-1 and Bridge-V2, equal-update controls show that feature supervision under generated histories improves perceptual prediction. These findings show how terminal-map learning can combine efficient training with accurate one-step visual prediction.
Introduction
An action-conditioned visual world model predicts future states given past observations and a sequence of actions. Such predictions support visual planning, policy evaluation, and learning from imagined interaction [10, 8, 15, 16, 17]. Recent video simulators extend these capabilities to richer environments and manipulation tasks (Yang et al., 2024, Bruce et al., 2024, Zhu et al., 2025b,a, Guo et al., 2026). Their practical value depends on both accuracy and inference cost.
Diffusion and flow models offer expressive conditional generators, but standard samplers repeatedly evaluate a neural network to predict each future state [19, 38, 25, 20, 41]. Their sampling cost motivates learning a direct map from noise to future frames with one-step prediction. Distillation obtains such predictors from pretrained generators [32, 46 a], while direct training learns them from data [39, 37, 12, 13]. Drifting models provide another route to one-step generation [7]. DriftWorld applies drifting to action-conditioned video prediction [27]. These methods enable one-step prediction, but their training objectives can have substantially different computational costs. We study terminal-map learning as a way to train accurate action-conditioned predictors efficiently.
We introduce Beckmann World Models (BWM), which adapt the terminal-map formulation of Beckmann Transport Models [24] to action-conditioned visual prediction (Figure 1).
Figure 1. Conditional terminal-map prediction and training. (a) Observed history and aligned actions condition a one-call prediction. The endpoint image shows a representative prediction on Robomimic Can. (b) Pair-based training uses and the target boundary . Source anchoring supervises at . (c) Target frames define motion masks and within-chunk differences. Prediction quality is evaluated in § 4.
For fixed history and actions, a terminal map assigns each noise input the endpoint of an autonomous transport. The terminal map is constant along each autonomous transport trajectory. This conservation property yields a training objective with a stop-gradient target constructed from pairs of noise samples and ground-truth future frames. The terminal map is conditioned on observation history and actions, without an explicit transport-time input. This construction requires neither a pretrained generative teacher nor a population of generated futures for each condition. Unlike feature-space drifting, the terminal-map objective does not require a pretrained similarity encoder.
Turning this construction into an accurate world model requires supervising the inputs used at inference. The transport objective constrains the map on interpolated inputs, while boundary reconstruction teaches it to reproduce the ground-truth future frames. Neither directly supervises the prediction obtained from pure noise. In Robomimic Can, a stabilized map has higher prediction MSE at the source than at the target boundary. Thus, accurate boundary reconstruction can coexist with an inaccurate prediction from initial noise.
We address this gap with endpoint-aligned Beckmann training, which directly supervises predictions from initial noise. Because much of a given scene is static, uniform pixel error can underweight the changes that distinguish action outcomes. We therefore emphasize moving pixels and temporal differences, then supervise spatial structure with frozen visual features. To prepare the map for autoregressive use, we also train subsequent predictions under detached generated histories. Together, these objectives address the discrepancy between transport training and inference under source noise and self-generated histories, without increasing inference cost. All components train the same map and preserve one forward pass per future chunk.
On PushT and Robomimic Can, the final staged model reduces 64-frame MSE by 68% and 57% relative to same-backbone regression. Improvements are also seen in moving regions and perceptual metrics, showing that the gains extend beyond static background reconstruction. The displayed qualitative results illustrate these gains in object pose and robot configuration (Figure 6).
Figure 6. Autoregressive RGB predictions. Reference and predicted frames at positions 24, 44, and 64 in two selected evaluation sequences, with the same actions and times across rows. BWM uses the final spatial continuation. Local baselines use their five-epoch fits. Appendix § 10.7 gives sample selection and additional baseline rows.
The downstream benefit is clearest in policy evaluation. BWM estimates simulator policy scores with mean IoU error 0.0100, compared with 0.0306 for local DriftWorld and 0.0203 for its released checkpoint, while predicting four frames in 24.8 ms.
Compared with DriftWorld, which also uses single-pass inference, BWM substantially reduces training-step cost. On Robomimic Can, BWM achieves lower perceptual error in less cumulative training time than DriftWorld (Figure 4). Complete-update profiling gives higher throughput and 78% lower peak allocated memory with equal batches of windows. These measurements cover the stage trained with motion supervision and generated-history recovery, including generated prefixes, and precede the final stage with VGG feature supervision.
Figure 4. Can prediction quality and training resources. Left: evaluated checkpoints against cumulative training-loop time, counting all preceding BWM stages. The BWM curve ends at the checkpoint trained with motion supervision and generated-history recovery, before spatial supervision. Right: complete-update profiling on two A800 GPUs, with one conditioning window per GPU. Throughput counts these primary windows, including the cost of auxiliary recovery predictions. Peak memory is allocated memory per GPU. Both models use one call at inference.
We further evaluate generated-history feature supervision on RT-1 and Bridge-V2 in the latent space. The hybrid objective combines terminal-map training with feature drifting. Adding feature supervision to predictions conditioned on generated histories reduces LPIPS by 34% and 24% against equal-update continuations from the same initial checkpoints. This extension retains one forward pass per frame and uses generated samples during training.
Our contributions are summarized below.
Conditional terminal-map learning. We adapt Beckmann training to action-conditioned prediction and reduce the gap between target-boundary reconstruction and prediction from initial noise.
Endpoint-aligned supervision. We integrate source supervision with motion, spatial, and generated-history signals to improve one-step prediction without additional network evaluations.
Training efficiency and downstream evaluation. We evaluate complete training recipes, demonstrate improved rollouts and policy-score estimation, and test generated-history feature supervision through controlled experiments on real videos.
Related Work
Visual prediction for decision making. Visual world models predict future observations for candidate action sequences, enabling model-based planning ([10, 8]). Latent world models learn compact dynamics where a policy can be trained through imagined interactions ([15, 16, 17]). Explicit visual prediction instead allows existing visual policies and outcome readers to interact with predicted observations. In this setting, small errors in object configuration can change the apparent consequence of an action. DIAMOND demonstrates the importance of such details for decision making and studies how diffusion sampling affects autoregressive stability ([1]). BWM follows this explicit prediction setting and studies how to train accurate predictions while retaining a single model evaluation per chunk.
Controllable video simulation. Expressive video generation provides a foundation for visual simulation ([20, 41, 2]), but useful robot prediction also requires a close correspondence between controls and visual outcomes. UniSim brings heterogeneous interaction data into a common action-conditioned generation framework ([44]). IRASim develops fine-grained robot action conditioning ([49]), while Ctrl-World addresses partial observability and sustained interaction through joint camera prediction, frame-level pose conditioning, and retrieval from visual history ([14]). These methods improve action conditioning, history conditioning, and consistency across camera views. BWM instead investigates how one-step generative training objectives affect prediction accuracy and training cost for single-view robotic dynamics. Our evaluations concern prediction, training cost, and policy-score estimation within the reported benchmarks.
Learning predictors with few sampling steps. Diffusion and Flow Matching learn local generative dynamics that are integrated during sampling ([19, 25]). Improved samplers and rectified flow reduce the cost of this integration [36, 22, 26]. Distillation moves more computation into training by transferring a pretrained generative process to a faster student. Step-distillation, distribution matching, and adversarial supervision provide different ways to conduct that transfer (Salimans & Ho, 2022, Yin et al., 2024b,a, Sauer et al., 2024). For robot video, concurrent DIDO combines distribution matching distillation with supervision of gripper and object trajectories and DINO feature alignment before coupling the predictor to an action expert [28]. This highlights the need to preserve interaction details as sampling is accelerated. BWM trains the predictor without a pretrained generative teacher or additional object annotations, using frozen visual features for spatial supervision.
Direct map learning. Consistency Models learn a common endpoint for inputs along a probability-flow trajectory [39, 37]. Finite-time maps and transitions across step sizes provide related ways to represent a completed portion of a flow [3, 11]. MeanFlow instead models average velocity and derives a training identity relating it to instantaneous velocity [12]. BTM supplies the principle used here. Its autonomous transport defines a terminal map whose value is constant along each trajectory, motivating a stationary conservation objective without an explicit transport-time input [24]. BWM instantiates that map for conditional visual dynamics and addresses the accuracy of its prediction from initial noise. We inherit BTM’s transport formulation and stop-gradient training objective.
Conditional distribution updates. Drifting learns a direct generator by evolving its output distribution during optimization. A kernelized field attracts generated samples toward data and repels them from the current generated population [7]. DriftWorld applies this construction separately to each visual history and action sequence, using a ground-truth future chunk and a population of generated futures [27]. It shares BWM’s one-step interface and several endpoint regularizers, making it a close comparison for the training objective. BWM instead constructs its stop-gradient training target from noise–data pairs and a Jacobian-vector product, avoiding DriftWorld’s per-condition populations of generated samples during training. This changes the computation needed for training, although derivatives, boundary evaluations, and generated histories still incur cost. We therefore compare complete training steps and prediction quality over cumulative training time. The latent-video extension retains feature drifting and examines its combination with terminal-map training.
Supervision under generated histories. A predictor trained on ground-truth context encounters different inputs when its own outputs become the context. Self Forcing addresses this mismatch by generating autoregressive contexts during training and applying a distributional loss to the resulting video [21]. DriftWorld also trains with detached generated histories [27]. BWM uses a detached prefix and supervises the subsequent prediction against the corresponding ground-truth future frames. This teaches recovery from context errors, without supplying physical labels for counterfactual states. Motion weighting and frozen feature losses also build on existing world-model and perceptual supervision [27, 47]. We study their integration with the terminal map and test the addition of feature supervision at generated-history endpoints through controlled continuations. Source anchoring addresses a separate mismatch at the noise input. Its squared-error supervision can favor concentrated predictions when futures are ambiguous, so paired accuracy alone does not establish calibrated conditional diversity [1].
Evaluating world-model utility. Different downstream tests assess different properties of a predictor. GPC uses a world model to rank or refine action proposals from a frozen generative policy [30]. WorldGym evaluates policies through imagined rollouts, while Ctrl-World distinguishes instruction following from successful low-level execution [31, 14]. We accordingly measure agreement with simulator policy scores separately from executed planning performance. AVDC addresses another setting, generating task-conditioned video and recovering actions through dense correspondences without action labels [23]. Our AVDC-derived control follows the action-conditioned adaptation used in GPC and tests that video diffusion recipe within our protocol.
Beckmann World Models
Problem Formulation
Let denote four observed frames, the aligned controls, and the ground-truth future chunk. We write and predict
For pixel-space prediction, we use an action-conditioned residual U-Net. We study direct pixel-space prediction on PushT and Robomimic Can videos, and latent-space prediction using an SD3 VAE on RT-1 and Bridge-V2 videos. We concatenate the observed frames with the interpolated future input and condition the residual blocks on frame-aligned actions. No interpolation-time variable is supplied to the network. For a future tensor with scalar coordinates, write .
For a fixed condition, the BTM construction associates an autonomous field with a terminal map . Under the support and regularity assumptions of Lee et al. [24], the endpoint obeys
Transport time is distinct from video time. The auxiliary transport evolves the transport state while keeping the observation history and actions fixed. We learn its endpoint directly. Equation (2) is inherited from BTM. We develop an action-conditioned terminal-map predictor and address two challenges in one step video prediction: directly supervising the map from pure source noise and reducing error accumulation under self-generated histories.
Pair-Based Terminal-Map Training
Sample and form a pair of noise and a ground-truth future chunk,
Following BTM, we construct a target using a Jacobian-vector product (JVP) and apply stop-gradient,
Here sg stops reverse-mode gradients. The JVP differentiates only the future input . and are fixed. The displacement defines the JVP direction along the noise–data interpolant, without requiring a separate velocity network. Although contains a derivative, the stopped target avoids backpropagating through that derivative during the parameter update.
The transport residual uses detached per-example weights, boundary and near-boundary reconstruction, and an output-gradient balance, yielding (Equations (15) through 17 in Appendix § 9). These weighting and boundary terms stabilize training.
Source and Endpoint Supervision
The one-step prediction from initial noise is . Our fixed-window diagnostic shows that accurate reconstruction at the data boundary does not ensure an accurate (Figure 2). We therefore add source anchoring,
Figure 2. Source anchoring improves prediction from initial noise. Measured Can endpoint MSE on 90 fixed windows and four shared source draws per window. The stabilized map has low error at the target boundary () but much higher error at the source (). Source anchoring reduces source MSE by 71%. The continuation with motion supervision and generated-history recovery reduces it further. Points denote measured input locations. Connecting lines guide the eye. These checkpoints precede spatial training.
Source anchoring directly supervises this prediction. It penalizes conditional-mean error and variation induced by source noise. This is useful for paired manipulation prediction but may suppress valid conditional diversity. Appendix D gives the exact conditional-variance decomposition in Equation (22).
Endpoint regularization. Uniform pixel error can be dominated by static backgrounds. We emphasize pixels that change relative to the last observation, match within-chunk temporal differences, and softly penalize out-of-range RGB values,
Figure 3 illustrates these endpoint and recovery signals. The moving mask is target-defined. The final training stage matches three levels of frozen ImageNet VGG16 features at ([35]), with detached target features and channel normalization (Equation (18), Appendix § 9). We reuse the source prediction to compute this loss and discard the feature encoder at inference.
Figure 3. Endpoint supervision. (a) A target-defined mask weights changed pixels, normalized per example. The notation denotes averaging over tensor coordinates. (b) Temporal differences are compared within a future chunk. Color shows channel-averaged signed RGB differences on a shared scale. (c) A frozen VGG16 encoder matches three levels of normalized features, detaching only target features. (d) A detached generated prefix conditions a new prediction using actions and targets from the dataset. Fixed PushT crops and saved Can rollout frames illustrate the losses, rather than a captured training batch. Appendix A gives exact definitions and schedules.
Generated-history recovery. Every fourth update, a detached generated prefix replaces the visual history for an additional prediction using actions and targets from the dataset. The weight is 0.25 on active updates, with a depth curriculum. We condition the model on its own detached predictions while retaining the corresponding ground-truth future as the target, training it to tolerate accumulated prediction errors (Equation (19), Appendix § 9).
After the source-loss weight reaches its final value, the pixel-space training objective is
where and . Training introduces source anchoring, motion supervision and generated-history recovery, and spatial supervision in successive stages. The spatial stage retains the mature recovery schedule. Algorithm 1 summarizes the computation.
Latent-Space Video Prediction
For RT-1 and Bridge-V2 video, one forward pass predicts one SD3 VAE latent frame ([9]). Transport and source losses operate in the latent space and while VGG perceptual supervision operates on decoded frames. We retain conditional drifting in frozen DINOv3 and latent features ([34, 27]), using samples and coefficient 0.01, and add the same term at generated-history recovery endpoints. Equation (21) and Appendix § 11 give the exact hybrid objective. Its generated samples are used only in training. Inference requires one forward pass per frame.
Experiments
We evaluate BWM through the cost of learning its predictor, the accuracy of its autoregressive predictions, and its ability to estimate policy performance at low inference cost. A separate latent-video study tests whether feature supervision under generated histories improves perceptual prediction.
Experimental Setup
RGB dynamics. We evaluate PushT ([6]) and Robomimic Can ([29]) at resolution. Each 64-frame evaluation starts from four frames and predicts the remaining 60 frames autoregressively from actions. This benchmark includes 388 PushT and 30 Can episodes. Full-episode evaluation includes 400 and 30 episodes respectively. Pixel MSE measures paired prediction error, while LPIPS, SSIM, and PSNR characterize appearance. Metrics are averaged within each episode and then across episodes. Appendix § 10 gives the metric conventions and episode-bootstrap intervals.
Baselines and training budgets. Local DriftWorld and MSE regression use the corresponding action-conditioned backbone. The MSE U-Net is trained directly with squared prediction error. The AVDC-derived control uses a time-conditioned U-Net with an adapted diffusion recipe. Local baselines use five capped epochs on our split. Released GPC and DriftWorld checkpoints are additional references. All reported scores are obtained under our evaluation protocol.
Policy evaluation and timing. Following DriftWorld’s PushT protocol, we evaluate seven frozen diffusion-policy checkpoints ([6, 27]). Each policy is evaluated on 300 trials paired with simulator trials. We measure the mean absolute difference between the seven policy-level mean IoUs and the Pearson correlation across those means. These metrics assess how closely predicted policy performance agrees with simulator performance. Model timings measure batch-one inference after warmup on an A800 and exclude the policy and the pose-estimation networks used to score predictions. All RGB predictors produce four frames per prediction in this comparison. Table 1 reports their numbers of network evaluations.
Table 1: PushT policy evaluation and inference cost. Error and correlation use seven policy means, each from 300 paired trials. Calls and latency refer to four predicted frames. Local baselines are five-epoch fits.
| World model | Training | Calls | ms ↓ | IoU error ↓ | ↑ |
| DriftWorld | Local | 1 | 21.5 | .0306 | .8565 |
| GPC | Local | 12 | 140.7 | .5127 | .3659 |
| AVDC | Local | 100 | 1866.2 | .3951 | -.0801 |
| DriftWorld | Released | 1 | 21.5 | .0203 | .9255 |
| GPC | Released | 12 | 139.1 | .0860 | .8438 |
| BWM | Local | 1 | 24.8 | .0100 | .9280 |
Table 1.
Training Efficiency
We compare BWM with local DriftWorld, which already has the same single-pass prediction interface, through profiling of complete training steps and prediction quality over cumulative training time. Figure 4 examines complete-update resources and prediction quality over cumulative training time.
Training-step cost. The BWM training procedure has a lower measured update cost even after accounting for recovery predictions. At equal batches of windows, BWM achieves higher throughput with 78% lower peak allocated memory (Figure 4). Appendix § 10.§ 3 gives the absolute throughput and memory measurements.
Policy Evaluation
Accurate image prediction is useful for policy evaluation when the imagined outcomes preserve differences in policy performance. We test this directly by comparing predicted policy scores with simulator scores. Figure 5 compares policy-score estimation error against per-chunk inference latency.
Figure 5. Policy-score accuracy versus inference speed on PushT. Error is the mean absolute difference between model and simulator policy-level mean IoUs. Latency is for four predicted frames. Training histories and sampler calls are given in Table 1. Timings exclude policy execution and reward estimation.
BWM achieves a mean absolute policy evaluation error of 0.0100, compared with 0.0306 for local DriftWorld and 0.0203 for its released checkpoint. BWM and released DriftWorld have similar correlations across policy means, but BWM more closely reproduces their absolute levels (Table 1). The result shows why absolute error provides information beyond correlation when evaluating a predictive simulator.
This accuracy is obtained at 24.8 ms per four-frame prediction, close to the other one-step predictors. Against released GPC, BWM has lower policy-score error while requiring 5.6× less inference time. The comparison therefore supports accurate policy-performance estimation without iterative sampling at inference. Table 1 reports the evaluated generative models. The AVDC-derived control underestimates policy IoU, as detailed in Appendix § 10.§ 4.
Model-based planning. Planning provides a separate test of whether predictions improve executed actions. With fixed GPC-RANK proposals ([30]), BWM achieves mean executed IoU 0.721, compared with 0.709 for local DriftWorld and 0.695 for the MSE U-Net. The paired difference between BWM and DriftWorld is +0.012, with a 95% bootstrap interval of [−0.047, 0.071]. BWM achieves the highest mean executed IoU, while its clearest downstream advantage is more accurate policy-score estimation.
Autoregressive Prediction
We next examine whether accurate policy scores are accompanied by faithful visual dynamics over repeated predictions. On the 64-frame benchmark, the final BWM model reduces pixel MSE by 68% on PushT and 57% on Can relative to same-backbone regression. Moving-region error also decreases, and perceptual quality improves on both datasets (Table 2). These complementary measurements show that the improvement extends to changing robot and object pixels. Full-episode evaluation retains lower MSE than the local baselines (Appendix § 10).
Table 2: RGB prediction on episode-held-out splits. MSE, moving-region MSE, and LPIPS for 60 autoregressively predicted frames after the initial context. All metrics are lower-is-better. BWM is the final spatial-stage model. Baselines are local five-epoch fits. These rows compare different total training budgets.
| Model | PushT MSE | PushT Moving MSE | PushT LPIPS | Robomimic Can MSE | Robomimic Can Moving MSE | Robomimic Can LPIPS |
| DriftWorld | .0314 | .4093 | .0790 | .0104 | .0769 | .0682 |
| MSE U-Net | .0303 | .4008 | .0804 | .0051 | .0544 | .0104 |
| GPC | .0630 | 1.4241 | .1609 | .0276 | .3103 | .0589 |
| AVDC | .0684 | 1.1840 | .1860 | .0400 | .2644 | .1194 |
| BWM | .0098 | .1247 | .0280 | .0022 | .0242 | .0056 |
Table 2.
The qualitative rollouts illustrate the structure behind these aggregate errors. In Figure 6, BWM more closely preserves the moving object’s pose in PushT and the configuration of the robot arm and object in Can. Appendix § 10.7 provides additional baseline rows and describes sample selection.
The released DriftWorld checkpoint has lower PushT pixel MSE and LPIPS than BWM under our evaluation (Appendix § 10.6). It has a much longer training history and was not trained with our episode split. Thus the final RGB comparisons establish improvements over our local controls, while the released-reference comparison identifies lower policy-score error as BWM’s advantage. Visual fidelity and policy-score agreement assess different aspects of world-model quality.
Ablation Analysis
The source diagnostic tests whether boundary reconstruction is an adequate proxy for prediction from Gaussian noise. On fixed Can windows, the stabilized map has small boundary error but substantially larger source error (Figure 2). Source anchoring reduces source MSE from 0.002831 to 0.000823, a 71% reduction. This directly supports supervising the prediction from Gaussian noise used at inference.
The subsequent training stages improve the quality of repeated predictions. Adding motion supervision and generated-history recovery reduces full-episode Can MSE from 0.0940 to 0.0112, with a smaller improvement on PushT. Continued spatial supervision further improves LPIPS on both datasets.
Real-World Video Prediction
We examine whether extending feature supervision to generated histories improves a fixed BWM checkpoint on RT-1 ([4]) and Bridge-V2 ([42]). The models use videos and latents encoded by the frozen SD3 VAE. Each evaluation window starts from four context frames and predicts eight frames autoregressively. Context is restored at the start of the next window. References are VAE reconstructions. This evaluates conditioned video prediction separately from the RGB benchmark’s free-running rollouts.
Both fine-tuning conditions begin from the same epoch-70 EMA checkpoint trained on the full dataset, retain the history feature loss, and use fresh optimizers. The added term applies feature drifting to predictions conditioned on generated histories. The additional training comprises 1,000 updates on RT-1 and 500 on Bridge-V2. Because the control already has history feature supervision, this comparison tests the benefit of extending that supervision to generated contexts.
The resulting improvement is clearest in perceptual prediction. LPIPS decreases by 34% on RT-1 and 24% on Bridge-V2 relative to their controls. FID and FVD also improve on both datasets, while PSNR decreases slightly (Table 3). Thus the added supervision improves feature-based measures of frame appearance and video distribution without a corresponding pixelwise improvement. Figure 7 provides matched qualitative examples, including the residual gripper blur.
Figure 7. Native-video prediction with generated-history feature supervision. Reference, control, and recovery-refined predictions at matched positions in selected RT-1 and Bridge-V2 episodes. References are VAE reconstructions. Each eight-frame prediction window starts from four frames. Separated columns span several such windows. Residual gripper blur remains visible.
Feature supervision under generated histories. Full-data BWM parents and matched-length continuations. RT-1 adds 1,000 updates. Bridge-V2 adds 500. The control retains clean-history feature drifting. Recovery adds it at generated-history endpoints. Both continuations use the same parent and one training seed.
| Dataset | Model | PSNR ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
| RT-1 | Parent | 27.35 | .0917 | 19.77 | 167.50 |
| RT-1 | Control | 27.34 | .0933 | 20.56 | 172.89 |
| RT-1 | + recovery | 27.27 | .0612 | 8.05 | 104.91 |
| Bridge-V2 | Parent | 25.00 | .1113 | 31.86 | 175.53 |
| Bridge-V2 | Control | 25.01 | .1106 | 30.98 | 171.56 |
| Bridge-V2 | + recovery | 24.95 | .0841 | 21.34 | 117.20 |
Table 3.
Discussion and Conclusion
Beckmann World Models learn action-conditioned futures as terminal maps. The central empirical finding is that a pair-based RGB training procedure can retain single-pass inference while reaching accurate Can predictions earlier, with higher training throughput and lower peak memory than local DriftWorld. Source anchoring and endpoint supervision also produce accurate policy-score predictions at a similar inference cost to the other one-step models. On real robotics videos, feature supervision under generated histories …
Reproducibility Statement
Appendices A through E specify the losses, tensor alignment, training lineage, evaluation protocols, and additional results. Episode-bootstrap intervals quantify evaluation-cohort uncertainty. The reported experiments use one training seed.
Ethics Statement
This work studies visual prediction for manipulation using existing datasets. Model-generated outcomes can be inaccurate, and the reported evaluations do not establish suitability for safety-critical robotic control.
Statement of AI Use
Generative AI tools assisted with experiment execution, and debugging. Quantitative results come from model evaluation records, and empirical visualizations use dataset observations and model outputs. The authors are responsible for the final paper.
Implementation Details
Gradient Balancing and Optimization
Let denote parameters of the output spatial projection. Define
We average each gradient vector across distributed ranks before taking its Euclidean norm. When both norms exceed , the detached balance is
When either finite norm is at most , the coefficient is one. Nonfinite gradients cause a failure rather than a silent substitution. The fallback prevents an initially zero boundary gradient from disabling transport. Source and endpoint auxiliary terms are not inserted into this original balancing group.
The stabilized map uses Muon for hidden weights and AdamW for output and auxiliary parameters. The original target learning rates are and . The completed motion-aware continuation uses one quarter of these rates. Warmup lasts 500 updates from one percent of the target rate. The global gradient norm is clipped at two, and EMA decay is 0.999. Training and the full-update comparison use full precision. Each continuation explicitly initializes from its parent’s EMA and resets the continuation’s optimizer and scheduler. Resuming the same continuation instead restores its own model, optimizer, scheduler, EMA, RNG states, sampler position, and update clock.
The matched DriftWorld backbone does not imply identical optimization. Its training runs use AdamW, smaller window batches, and 16 generated negatives per PushT condition or 32 per Robomimic Can condition. Kernel temperatures are 0.02, 0.05, and 0.2. These differences are part of the complete implementation comparison and prevent attributing every speed or quality difference to a single operation.
Exact Endpoint Regularizers
For example , let be the last observed frame before the target chunk. The target-defined moving mask is
The RGB channel index is , future position is , and spatial coordinates are . Broadcast the mask over channels and let mean average the indicated tensor coordinates. The training loss is
The floor limits the additional per-coordinate weight induced by the coefficient 0.1 to at most two. The other terms are
The temporal term concerns within-chunk differences. It does not force a repaired endpoint to preserve the offset of an erroneous generated preceding frame.
The recovery branch uses a separate valid 16-frame training sequence. Robomimic Can generates between one and seven two-frame prefix chunks before a two-frame endpoint. PushT generates between one and three four-frame prefix chunks and alternates four-frame and three-frame endpoints. The depth curriculum expands over 1,000 completed updates. Prefix generation is detached, raw, and conditioned on aligned dataset actions. No blend with the true history or inference clamp is introduced. The branch is active every fourth update, with weight 0.25 on active updates. It is not multiplied by four to compensate for its frequency. All losses use one final backward pass and one optimizer update.
The clean-window dataset retains short episodes that cannot support a 16-frame sequence. The recovery sequence is obtained from a separate training-only valid-sequence index. No sample crosses an episode boundary. This regularizer does not create labeled counterfactual simulator transitions.
Frozen Spatial Endpoint Features
Spatial supervision uses ImageNet-pretrained VGG16 convolutional weights [35] and no classifier. Each predicted video tensor is reshaped from to , mapped from to , and normalized by the standard ImageNet channel statistics. We extract relu1_2, relu2_2, and relu3_3 at their native spatial resolutions. For feature tensor , channel normalization is
The loss at each level is the mean squared Euclidean distance between normalized channel vectors, summed over channels and averaged over batch, frame, and spatial locations. The three level losses receive equal weight. Target features are computed without gradients. Predictions are not clipped before feature extraction, so out-of-range outputs still contribute gradients.
The spatial coefficient ramps linearly from zero to 0.05 over 1,000 child updates. The VGG module is frozen and is stored outside the world model, optimizer, EMA, and checkpoint state. The source prediction already required by is reused, so spatial supervision introduces no additional world-model forward pass. Training fails if the declared feature asset or checksum is unavailable rather than silently substituting random features. At inference the feature module is absent.
Continuation schedule. The base map first learns . Source supervision ramps to one over 500 updates. Motion supervision and generated-history recovery terms ramp over 1,000 updates from the source-anchored EMA. The final spatial stage continues the motion-aware EMA, ramps only its feature coefficient, and retains the parent recovery clock.
Stabilized Transport Objective
We stabilize the transport residual using a detached per-example weight and boundary supervision:
The detached balances the transport and boundary gradient norms on the output projection. These stabilizers and the optimizer recipe are empirical adaptations, not consequences of the transport theorem. Appendix A gives their exact implementation.
Spatial Feature Supervision
The moving mask comes from the target, so prediction error cannot define where supervision is applied. A final continuation adds frozen ImageNet VGG16 features [35]. With denoting channel normalization at each spatial location,
The three levels are relu1_2, relu2_2, and relu3_3. The target features are detached. Gradients pass through the frozen encoder to the prediction. Reusing adds no world-model forward pass, and the encoder is discarded at inference.
Training Algorithm
Algorithm 1 RGB terminal-map training and inference
Require: Training windows , active training stage, and EMA parameters
Sample and
, , and
Compute with a JVP holding fixed
Construct and by (4) and 15 through 17
if source supervision is active then
and add and the active endpoint losses
end if
if recovery is active on this update then
Generate a detached prefix on a valid training sequence
Predict its next endpoint and add from (19)
end if
Backpropagate the combined loss once and update and its EMA
Inference: sample and return
Generated-History Endpoint Objective
Recovery from generated histories. Every fourth update, a detached model-generated prefix replaces the visual history for a separate training endpoint. The aligned actions and target are taken from the dataset. If , then
The prefix exposes the predictor to its own context errors. Stopping it prevents backpropagation through a long rollout. The dataset endpoint remains the recovery target, not the physical consequence of a counterfactual generated state.
Evaluation, Training Lineage, and Additional Results
RGB Evaluation
We use EMA weights and evaluation seed 314. For the 64-frame protocol, an episode must contain the four-frame context and 60 future frames. This leaves 388 PushT episodes and 30 Can episodes. Full-episode evaluation uses all 400 PushT episodes and the same 30 Can episodes. Predictions are fed back without an inference clamp. Targets and controls are taken from the dataset episode, and a sequence never crosses an episode boundary.
Pixel MSE is computed on raw RGB predictions. LPIPS [47] uses clipped images. PSNR and SSIM [43] use the clipped, quantized image convention of the evaluator. Moving-region MSE selects target pixels that differ from the last context frame by more than 0.05 in at least one channel. The metric is averaged within episodes and then equally across episodes. This evaluation statistic is distinct from the floored motion-loss normalization used during training.
Table 4 reports the complete final BWM metrics. We recomputed 95% percentile intervals by resampling complete episodes 10,000 times (resampling seed 943), separately for each task and protocol. These intervals describe evaluation-cohort variation for one trained model, not uncertainty across training seeds.
| Protocol | Metric | PushT | Robomimic Can |
| 64 frames | MSE | 0.0098 [0.0086, 0.0111] | 0.0022 [0.0016, 0.0028] |
| Moving MSE | 0.1247 [0.1098, 0.1401] | 0.0242 [0.0177, 0.0319] | |
| LPIPS | 0.0280 [0.0242, 0.0320] | 0.0056 [0.0045, 0.0068] | |
| SSIM | 0.9754 [0.9731, 0.9776] | 0.9843 [0.9812, 0.9871] | |
| PSNR | 31.21 [30.82, 31.61] | 34.75 [33.79, 35.69] | |
| Full episode | MSE | 0.0154 [0.0137, 0.0171] | 0.0082 [0.0057, 0.0112] |
| Moving MSE | 0.1646 [0.1475, 0.1829] | 0.0517 [0.0379, 0.0680] | |
| LPIPS | 0.0412 [0.0362, 0.0465] | 0.0238 [0.0160, 0.0338] | |
| SSIM | 0.9662 [0.9634, 0.9689] | 0.9560 [0.9442, 0.9658] | |
| PSNR | 28.67 [28.26, 29.08] | 30.17 [29.00, 31.34] |
Table 4. Final BWM prediction metrics and episode-bootstrap intervals. Each cell is a mean with its 95% interval. The spatial stage has 30 epochs.
The cumulative Can curve sums logged training-loop intervals across the stabilized-map, source-anchoring, and motion and recovery runs. The respective increments are 1.14, 1.50, and 1.68 hours, totaling 4.32 hours on two A800 GPUs. Local DriftWorld uses 10.19 hours on the same GPU count. The curves show evaluated checkpoints without smoothing or interpolation of an unobserved threshold crossing. They exclude evaluation waits and data preparation.
Training-Cost Comparison
DriftWorld is the key comparison for training efficiency because it already shares BWM’s one-call prediction interface. We evaluate two distinct consequences of replacing its conditional generated population with pair-based RGB terminal-map training: the resources needed for complete updates, and the quality reached over cumulative wall-clock time. Figure 4 presents both. Neither can be inferred from the number of inference calls alone.
Resource and Inference Measurements
The complete-update comparison uses two A800 80GB GPUs and one primary conditioning window per GPU. It includes the forward and JVP computations, all active losses, backward pass, distributed synchronization, clipping, optimizer, scheduler, and EMA update. The mature recovery schedule is exercised across its prefix depths. Peak memory is the maximum PyTorch allocated memory across ranks, including model, optimizer, and EMA state. The spatial encoder is absent from this particular measurement. Throughput counts primary windows rather than treating auxiliary generated samples as additional independent training examples.
| Recipe | Mean s | 95th percentile s | Windows s | Peak GiB |
| Local DriftWorld | 1.841 | 1.843 | 1.09 | 36.07 |
| Source-anchored map | .496 | .509 | 4.03 | 6.12 |
| Motion and recovery | .553 | .739 | 3.62 | 7.77 |
Table 5. Complete-update cost for the Can RGB recipes. Equal batches of training windows in full precision.
Inference timings measure batch-one model execution on an A800 after warmup, with the model resident in GPU memory. Local baseline, final BWM, and newly evaluated released GPC and AVDC-derived profiles were measured in separate runs, so small latency differences among one-call models should not be interpreted as an architectural speed advantage. All reported latencies are means: BWM averages 40 synchronized measurements across eight contexts (24.82 ms, median 24.69 ms), local baselines use mean inference time over 20 iterations, and released GPC and AVDC each average 20 measurements. BWM measures the terminal-map call with preallocated noise. Baseline profiles include sampler-side noise creation. The separately measured profiles therefore compare model execution cost, rather than a common end-to-end serving stack. GPU exclusivity was not independently verified for the final BWM profile. Native VAE decoding and the policy and reward networks are outside the RGB generator comparison.
Policy Evaluation and Planning
For policy evaluation, the seven frozen diffusion-policy checkpoints are epochs 50, 100, 150, 200, 250, 300, and 650. Each uses 300 paired world-model and simulator trials with base seed 100000, up to 300 environment steps, and eight executed controls per policy prediction. For policy , let and be its trial-mean IoUs. The reported error is
Pearson correlation is computed over the seven pairs of means, not over all individual trials. Within each model evaluation, paired trials use corresponding initializations. Random-number consumption differs across samplers, so simulator score arrays are not asserted to be identical across all models.
Table 6 gives the policy-level means for released GPC and AVDC-derived, recomputed from their paired trial arrays. Correlation is descriptive because it summarizes only seven policy checkpoints.
| Policy epoch | Released GPC Model | Released GPC Simulator | AVDC-derived Model | AVDC-derived Simulator |
| 50 | 0.5105 | 0.6190 | 0.1800 | 0.6103 |
| 100 | 0.5100 | 0.5929 | 0.1770 | 0.5922 |
| 150 | 0.4939 | 0.6029 | 0.1829 | 0.6042 |
| 200 | 0.4862 | 0.5630 | 0.1768 | 0.5609 |
| 250 | 0.4798 | 0.5536 | 0.1857 | 0.5507 |
| 300 | 0.4723 | 0.5617 | 0.1758 | 0.5624 |
| 650 | 0.4809 | 0.5423 | 0.1813 | 0.5448 |
Table 6. PushT baseline policy estimates. World-model and simulator mean IoUs for each frozen policy. Each entry averages 300 trials.
The released-GPC adapter was checked against the official sampler across consecutive chunks, including the past-control history. The AVDC-derived baseline is the locally trained action-conditioned adaptation, not a released AVDC policy simulator. Its final checkpoint contains 9,840 updates.
GPC-RANK uses a fixed proposal distribution, frozen outcome readers, and 50 environment seeds. Model predictions rank the action proposals. The selected controls are then executed in the simulator.
The paired planning interval in the main text resamples environment seeds. It does not quantify variation from retraining the world models.
Checkpoint Choice and Policy Calibration
The final spatial checkpoint is selected for the joint visual-prediction recipe, not by taking the best policy score from each stage. Earlier policy results remain useful evidence (Table 7). The spatial-five checkpoint has error 0.0098 and , slightly better on these policy metrics than the final spatial-thirty model, although its PushT LPIPS is worse (0.0407 versus 0.0280). The learned-direction branch has the highest correlation among these entries but much larger absolute score error. Correlation can preserve policy ordering despite biased scores.
| Checkpoint | Policy error ↓ | Pearson ↑ | LPIPS ↓ |
| Spatial, 5 epochs | .0098 | .9462 | .0407 |
| Spatial, 30 epochs (main) | .0100 | .9280 | .0280 |
| Learned-direction alternative | .0355 | .9743 | .0501 |
Table 7. Separate PushT policy and visual checkpoints. Seven frozen policies and 300 paired trials each. LPIPS uses AR64. Entries are descriptive stage results, not independently selected best metrics of one model.
Released-Model Prediction References
The released checkpoints were trained under different data and optimization settings and may have seen episodes in our evaluation split. We report them separately to make the comparison with stronger released models visible without claiming equal held-out exposure. The released Can DriftWorld checkpoint also uses different renders, which confounds its static appearance error.
| Task | Checkpoint | MSE | Moving MSE | LPIPS |
| PushT | Released DriftWorld | .0085 | .1182 | .0163 |
| PushT | Released GPC | .0192 | .2446 | .0324 |
| PushT | Final BWM | .0098 | .1247 | .0280 |
| Can | Released DriftWorld | .0525 | .0662 | .0676 |
| Can | Final BWM | .0022 | .0242 | .0056 |
Table 8. Released references on the RGB evaluator. All errors are lower-is-better. These are not equal-training-data comparisons.
Qualitative Comparisons
The RGB figures show two evaluation sequences: PushT saved sample 2 and Can saved sample 3, at frame indices 23, 43, and 63 (zero-based). They are illustrative selected sequences, not random samples of the evaluation distribution. The same frame indices and actions are used for all rows. Source-noise draws may differ across models. Figures use the displayed evaluation frames. Quantitative metrics are computed independently on the full episode cohorts. Figure 8 includes the local GPC and AVDC-derived predictions omitted from the more compact main-text comparison.
Figure 8. Additional RGB baseline comparisons. The same two evaluation sequences as Figure 6, with local GPC and AVDC-derived rows included. BWM uses its final spatial continuation.
Native Video Implementation and Evaluation
Representation and Losses
The native backbone is DriftWorld’s action-conditioned U-Net (175.4M parameters for Bridge-V2 and 160.3M for RT-1). Each call predicts one SD3 latent frame from a four-frame latent history and the corresponding action history. The VAE is frozen. The terminal map uses Gaussian source noise with standard deviation 1.35 and the same JVP-based stop-gradient target and stabilization as the RGB model. Source MSE operates on latents. Target-defined RGB motion masks are pooled to the latent grid. The range penalty acts on decoded RGB. Spatial VGG supervision compares decoded predictions with decoded target latents. There is no within-call temporal difference for a single predicted frame.
Recovery uses a valid eight-frame training sequence and one to three generated prefix frames. Prefixes are detached and appended to the history before predicting the next target. The recovery branch occurs every fourth update, with the same pixel and latent endpoint weighting as the parent. Feature drifting at recovery endpoints uses the preceding ground-truth frame to compute motion weights, so an erroneous generated history cannot redefine the target motion mask.
Feature drifting. For each condition, independent one-call samples are contrasted with one target using DriftWorld’s retained drifting objective. The feature spaces are blocks 2, 5, and 8 of DINOv3 ViT-B with 16-pixel patches (a spatial grid of 768-dimensional features) and the VAE latent grid (a grid of 16-dimensional features). DINO and VAE parameters remain frozen, with gradients propagated to generated inputs. We retain the feature normalization, field normalization, multi-temperature aggregation, and motion weighting of the native implementation. The drifting coefficient is 0.01, both on clean histories and, in the refined arm, on recovery histories. Action accentuation is disabled and the inference guidance input is one.
The clean-history and recovery drifting terms each use new -sample draws. The recovery term reuses the already-generated detached prefix, adds no second prefix rollout, and ramps from zero to 0.01 over 100 child updates. It is additional to the existing recovery reconstruction term. Activation checkpointing in the frozen VAE decoder reduces training memory. Neither feature encoder is required at inference, although the VAE decoder is needed to display latent predictions.
Native Terminal-Map Objective
For 256 × 256 RT-1 and Bridge-V2 videos, we predict one SD3 VAE latent frame per call [9], using the native action-conditioned DriftWorld backbone. Transport and source losses operate in latent space. Motion masks are pooled from ground-truth RGB differences, and spatial supervision compares decoded predictions with decoded target latents. Because each call predicts one frame, the within-chunk temporal loss is zero in this extension.
The richer appearance of these datasets motivates additional perceptual supervision. We use DriftWorld’s conditional drifting loss in frozen DINOv3 and latent feature spaces [34, 27], with generated samples per condition and coefficient 0.01. The final refinement applies the same feature loss at existing generated-history recovery endpoints:
where is the retained conditional feature-drifting loss and . The recovery feature term is added only when recovery is active and is not multiplied by the RGB recovery coefficient 0.25. This is a hybrid training objective: it preserves the terminal map and one-call deployment, but includes a generated population during training. The RGB population-free cost measurements therefore do not describe this extension.
Training and Controlled Continuations
The native model first trains for 30 epochs on a common subset (2,078 Bridge-V2 episodes and 2,041 RT-1 episodes), then continues spatial training on full data through epoch 60. The full-data caches contain 52,969 Bridge-V2 and 86,634 RT-1 training episodes. Feature drifting is added through epoch 70. The selected recovery refinements start from these epoch-70 EMA weights and add 500 Bridge-V2 or 1,000 RT-1 updates. Their controls use the same parents and added update counts with recovery-feature weight zero.
Both arms use fresh Muon and AdamW optimizer states, a 100-update warmup, batch four per GPU, EMA decay 0.999, gradient clipping at two, and learning rates scaled to one quarter of the original learning rates of and . The existing source, motion, recovery, and spatial ramps have completed before the refinement. The new recovery-feature coefficient is the changed loss term. The same seed and data setup do not imply identical later random draws because the added population consumes randomness. Added feature computation also makes these equal-update rather than equal-compute controls.
Bridge-V2 development compared several refinement checkpoints and coefficients. The 500-update full-weight result was selected after validation review. The RT-1 transfer selected the 1,000-update checkpoint. Both use one training seed. These results support the chosen continuation on the observed validation cohorts. They are not a preregistered, untouched-test estimate of its general effect.
Native Evaluation Protocol
The evaluation cohorts contain 215 Bridge-V2 episodes with 7,079 predicted frames and 198 RT-1 episodes with 8,409 predicted frames. Each eight-frame autoregressive window starts from four ground-truth latent frames. Longer displayed videos concatenate such windows. Reference images are obtained by decoding the target latents, and pixel and perceptual comparisons therefore include the VAE representation in both arms.
SSIM, PSNR, and LPIPS are averaged per episode and then equally across episodes. FID (Heusel et al. [18]) aggregates all generated and reference frames. FVD (Unterthiner et al. [40]) uses the StyleGAN-V fvd2048_16f evaluator on the first 16 generated frames of eligible episodes, with the same mirrored evaluation convention across arms. There are 201 eligible Bridge-V2 and 193 eligible RT-1 videos. FID and FVD values are retained evaluator outputs. They were not recomputed from compressed display videos. Table 3 and Table 9 share the same episode cohorts within each dataset.
| Dataset | Arm | SSIM ↑ | PSNR ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
| RT-1 | Parent | 0.8840 | 27.35 | 0.0917 | 19.77 | 167.50 |
| RT-1 | Control | 0.8836 | 27.34 | 0.0933 | 20.56 | 172.89 |
| RT-1 | Recovery | 0.8861 | 27.27 | 0.0612 | 8.05 | 104.91 |
| Bridge-V2 | Parent | 0.8555 | 25.00 | 0.1113 | 31.86 | 175.53 |
| Bridge-V2 | Control | 0.8556 | 25.01 | 0.1106 | 30.98 | 171.56 |
| Bridge-V2 | Recovery | 0.8580 | 24.95 | 0.0841 | 21.34 | 117.20 |
Table 9. Complete native continuation metrics. Same cohorts and models as Table 3.
Paired perceptual differences. Aligning the control and selected-refinement records by episode identity gives 198 RT-1 and 215 Bridge-V2 pairs. We resample these pairs 10,000 times (seed 20261014), using each episode’s mean LPIPS with equal weight. Recovery minus control LPIPS is −0.03217 on RT-1 (95% percentile interval [−0.03497, −0.02952]), and −0.02652 on Bridge-V2 (95% percentile interval [−0.02816, −0.02500]). Both descriptive intervals are below zero. They measure variation within the observed validation cohorts, not uncertainty from training seeds or checkpoint selection. The selected refinement remains without its own untouched-test confirmation. FID and FVD are distributional evaluator outputs, so we do not obtain intervals by resampling their aggregate scores.
Common-Subset Baselines
Table 10 retains the separate 30-epoch subset comparison. All methods see the same eligible training episodes and one deterministic crop per episode per epoch, use SD3 latents and the same native generator, and evaluate on the cohorts above. Their objectives, optimizer recipes, and batch sizes differ. These rows cannot be compared as equal-compute runs or pooled with the full-data continuations.
| Dataset | Method | Calls per frame | PSNR ↑ | LPIPS ↓ | FID ↓ | FVD ↓ |
| RT-1 | DriftWorld | 1 | 22.42 | 0.1295 | 27.48 | 237.45 |
| MSE U-Net | 1 | 23.03 | 0.1788 | 62.41 | 419.16 | |
| GPC-derived | 3 | 20.69 | 0.2937 | 124.18 | 884.52 | |
| AVDC-derived | 100 | 20.27 | 0.1751 | 94.87 | 719.10 | |
| BWM spatial | 1 | 23.15 | 0.1659 | 75.53 | 535.59 | |
| Bridge-V2 | DriftWorld | 1 | 21.92 | 0.1042 | 14.18 | 188.13 |
| MSE U-Net | 1 | 22.44 | 0.1663 | 61.96 | 405.61 | |
| GPC-derived | 3 | 19.93 | 0.3350 | 132.93 | 756.50 | |
| AVDC-derived | 100 | 19.59 | 0.1590 | 79.44 | 657.46 | |
| BWM spatial | 1 | 22.13 | 0.1550 | 68.86 | 592.24 |
Table 10. Thirty-epoch common-subset native comparison. The same training episode subsets and evaluation cohorts are used within each dataset. Batches and optimizer recipes differ. These models precede BWM’s full-data and recovery continuations.
Figure 9. Real robotics video prediction. RT-1 drawer manipulation and Bridge-V2 gripper motion from the final recovery-refined checkpoints. Columns show separated positions in each displayed video (positions 13, 49, and 85 for RT-1 and positions 4, 25, and 46 for Bridge-V2). Predictions are generated within the conditioned windows described in § 4.6. References are VAE reconstructions.
The BWM spatial model attains higher PSNR than DriftWorld on both tasks, but DriftWorld has better LPIPS, FID, and FVD. MSE training is also competitive in PSNR. This experiment demonstrates the trade-off of paired endpoint supervision and motivates the later feature-drifting extension. It does not support a claim that the BWM spatial recipe alone dominates population-based perceptual learning at native resolution.
Selected Recovery Video Frames
Figure 9 retains the original selected recovery display. Figure 7 augments the same episode positions with matched control predictions.
Separate Paired-Feature Recovery and Independent Test
A subsequent fidelity experiment replaces the added generated-history population term with paired frozen-feature appearance and displacement supervision. The original clean-history drifting loss remains active. This is a different recipe from the selected recovery-feature refinements in Table 3. Its results cannot be pooled with those gains.
Bridge-V2 selects the 1,000-update EMA using validation gates before testing. The untouched test cohort has 213 episodes and 7,044 predicted frames, with the same eight-frame conditioned windows, evaluation seed 314, VAE-decoded references, and detector conventions for parent and child. Evaluation settings and reference frames match across arms. Table 11 retains both validation and independent-test outcomes. The test LPIPS improvement is 4.5%, while FVD worsens by 7.8%. FID improves 2.8% and PSNR decreases slightly. The RT-1 candidate fails the 400-update validation gate and is not promoted to a test evaluation.
| Split | Model | PSNR | SSIM | LPIPS | FID | FVD |
| Validation | Parent | 25.00 | .8555 | .11125 | 31.86 | 175.53 |
| Validation | Paired-feature child | 24.95 | .8577 | .10642 | 31.36 | 188.04 |
| Test | Parent | 25.53 | .8631 | .10439 | 32.69 | 160.06 |
| Test | Paired-feature child | 25.47 | .8651 | .09964 | 31.77 | 172.55 |
Table 11. Paired-feature variant: validation and independent test. Bridge-V2 parent versus the separately selected 1,000-update EMA. This is not the 500-update recovery-population variant in Table 3. FID and FVD are retained evaluator outputs.
All 16 frames in each of four fixed diagnostic windows were reviewed at the retained Bridge checkpoints. Changes are small, with late-block blur around the gripper and manipulated object and reset recovery persisting. No convincing flicker fix is established. This mixed result sharpens the interpretation of the main refinement: paired appearance improvements alone do not ensure improved video-level distributional quality. It does not identify the population term as the sole cause, because the variant also changes recovery features and selection history. The larger selected LPIPS, FID, and FVD gains still need their own untouched-test confirmation.
Terminal-Map and Endpoint Diagnostics
Scope of the Terminal-Map Formulation
The BTM transport theorem selects a terminal map through the autonomous flow and its terminal behavior [24]. Merely imposing a pointwise identity on the target and conservation away from it does not prove uniqueness without an appropriate terminal trace condition. Likewise, the population consistency argument for the unweighted stopped update is not a convergence theorem for our adaptively weighted, source-regularized neural objective. Our claims about the learned map are empirical.
Fixed-Window Source Diagnostic
The Can probe evaluates 90 fixed evaluation windows and four shared source draws per window at . All conditions and targets are fixed across the compared models. Table 12 gives the retained endpoint MSEs. Source anchoring primarily improves prediction from source noise, reducing its MSE by 70.9% from the stabilized model. The continuation with motion supervision and generated-history recovery reduces it by 82.2% from that same baseline. These checkpoints precede the final spatial continuation.
| Model | 0 | .25 | .50 | .75 | 1 |
| Stabilized BTM | .002831 | .002027 | .001437 | .000810 | .000105 |
| + source | .000823 | .001429 | .001178 | .000687 | .000096 |
| + motion and recovery | .000503 | .000868 | .000747 | .000498 | .000025 |
| Learned direction | .001170 | .002241 | .002476 | .002513 | .000112 |
Table 12. Endpoint MSE across input locations on Can. The source is and the target boundary is .
Endpoint Supervision
Figure 3 gives the four complementary signals used around the deployed endpoint: changed pixels, temporal differences, frozen spatial features, and recovery under generated histories. Its image tiles illustrate the loss construction using real evaluation crops. They are not a captured training branch or an isolated loss ablation.
Conditional Variance of the Source Loss
This loss gives direct supervision to the model’s one-call prediction. For source noise independent of the ground-truth future frames conditional on , let . Then
Here denotes the mean conditional variance per coordinate. Source anchoring consequently penalizes both conditional-mean error and variation induced by source noise. This is useful for paired prediction in largely deterministic manipulation settings, but can suppress valid conditional diversity. We evaluate prediction accuracy rather than claim calibrated multimodal coverage.
Fresh RGB Training and Failure Analysis
Matched Clean-Window Comparison
To check how much of the final comparison depends on inherited training, we also train a fresh BWM generator for the same five capped epochs and clean-window exposure as the audited local MSE and DriftWorld fits. All warmup fits inside the budget. The predeclared final EMA is evaluated without checkpoint selection. Recovery adds supervision and the optimizer differs, so this controls clean exposure rather than total computation. Table 13 reports the 64-frame results.
| Task | Model | MSE | Moving MSE | LPIPS |
| PushT | Local DriftWorld | .0314 | .4093 | .0790 |
| MSE U-Net | .0303 | .4008 | .0804 | |
| Fresh BWM | .0211 | .2832 | .0980 | |
| Can | Local DriftWorld | .0104 | .0769 | .0682 |
| MSE U-Net | .0051 | .0544 | .0104 | |
| Fresh BWM | .0048 | .0475 | .0154 |
Table 13. Fresh generators at matched clean-window exposure. 64-frame evaluation after five capped epochs. BWM has zero inherited updates. Recovery targets, optimizer, and compute differ. The predeclared final checkpoint is used, separately from Table 2.
Relative to MSE, fresh BWM lowers PushT pixel and moving-region MSE by 30% and 29%, but has 22% worse LPIPS. Can pixel MSE improves by 6% over 60 predicted frames, whereas full-episode MSE is 14% worse and LPIPS is 104% worse. The controlled result supports useful PushT prediction at limited clean exposure, while showing that one-call training alone does not guarantee sharp, sustained dynamics.
Budget and Selection
This separate experiment initializes the generator, EMA, and optimizer from scratch at seed 1, with zero inherited updates. Frozen pretrained VGG16 remains an auxiliary training loss. It is not a pretrained generator. Can completes 20,000 updates at global batch two (40,000 clean windows, 10,000 additional recovery targets). PushT completes 39,350 updates at global batch four (157,400 clean windows, 39,348 additional recovery targets). These are five capped epochs, corresponding to 0.719 and 0.409 full-corpus clean pass equivalents, respectively.
The sampler, split, backbone, four-frame history, action alignment, and clean-window exposure match the audited local five-epoch baselines. Existing baseline final checkpoints are reevaluated with the same numerical protocol. The two-rank seed-1 BWM runs use Muon at 0.00025 plus AdamW at 0.000025, whereas the baselines use AdamW (0.000025 on Can, 0.0000125 on PushT). Source weight is one from the first update. Optimizer warmup takes 500 updates, and recovery, prefix-depth, and spatial ramps take 1,000, all within the stated budget. No held-out tuning, early stopping, or best-checkpoint promotion follows these runs: the final epoch-five EMA is predeclared. Thus the comparison matches clean exposure, with additional BWM endpoint supervision and different optimization. Historical baseline per-update timing and exact training-code revisions were not retained, so this experiment supplies no controlled training-speed estimate.
References
- [1]Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and François Fleuret. Diffusion for World Modeling: Visual Details Matter in Atari. In Advances in Neural Information Processing Systems, volume 37, pp. 58757–58791, 2024. URL https://arxiv.org/abs/2405.12399.
- [2]Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models. In IEEE and CVF Conference on Computer Vision and Pattern Recognition, 2023. URL https://arxiv.org/abs/2304.08818.
- [3]Nicholas Matthew Boffi, Michael Samuel Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic interpolants: A mathematical framework for consistency models. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview.net/forum?id=cqDH0e6ak2.openreview.net/forum?id=cqDH0e6ak2
- [4]Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., Ibarz, J., Ichter, B., Irpan, A., Jackson, T., Jesmonth, S., Joshi, N., Julian, R., Kalashnikov, D., Kuang, Y., … Zitkovich, B. (2023). RT-1: Robotics Transformer for Real-World Control at Scale. In Proceedings of Robotics: Science and Systems. https://doi.org/10.15607/RSS.2023.XIX.025
- [6]Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. In Robotics: Science and Systems, 2023. URL https://arxiv.org/abs/2303.04137.
- [7]Mingyang Deng, He Li, Tianhong Li, Yilun Du, and Kaiming He. Generative Modeling via Drifting. arXiv preprint arXiv:2602.04770v2, 2026. URL https://arxiv.org/abs/2602.04770.
- [8]Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control. arXiv preprint arXiv:1812.00568, 2018. URL https://arxiv.org/abs/1812.00568.
- [9]Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 12606–12633. PMLR, 2024. URL https://proceedings.mlr.press/v235/esser24a.html.proceedings.mlr.press/v235/esser24a.html
- [10]Chelsea Finn, Ian Goodfellow, and Sergey Levine. Unsupervised Learning for Physical Interaction through Video Prediction. In Advances in Neural Information Processing Systems, 2016. URL https://arxiv.org/abs/1605.07157.
- [11]Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One Step Diffusion via Shortcut Models. In International Conference on Learning Representations, 2025. URL https://arxiv.org/abs/2410.12557.
- [12]Zhengyang Geng, Mingyang Deng, Xingjian Bai, J. Zico Kolter, and Kaiming He. Mean Flows for One-step Generative Modeling. In Advances in Neural Information Processing Systems, volume 38, pp. 75460–75482, 2025. doi: 10.52202/085713-2534. URL https://doi.org/10.52202/085713-2534.
- [13]Zhengyang Geng, Yiyang Lu, Zongze Wu, Eli Shechtman, J. Zico Kolter, and Kaiming He. Improved Mean Flows: On the Challenges of Fastforward Generative Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 30467–30476, 2026. URL https://arxiv.org/abs/2512.02012v2.
- [14]Yanjiang Guo, Lucy Xiaoyang Shi, Jianyu Chen, and Chelsea Finn. Ctrl-World: A Controllable Generative World Model for Robot Manipulation. In International Conference on Learning Representations, 2026. URL https://arxiv.org/abs/2510.10125.
- [15]David Ha and Jürgen Schmidhuber. World Models. arXiv preprint arXiv:1803.10122, 2018. URL https://arxiv.org/abs/1803.10122.
- [16]Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to Control: Learning Behaviors by Latent Imagination. In International Conference on Learning Representations, 2020. URL https://arxiv.org/abs/1912.01603.
- [17]Danijar Hafner, Jurģis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, 640(8059):647–653, 2025. doi: 10.1038/s41586-025-08744-2. URL https://www.nature.com/articles/s41586-025-08744-2.DOI
- [18]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. In Advances in Neural Information Processing Systems, 2017. URL https://arxiv.org/abs/1706.08500.
- [19]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, 2020. URL https://arxiv.org/abs/2006.11239.
- [20]Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video Diffusion Models. In Advances in Neural Information Processing Systems, 2022. URL https://arxiv.org/abs/2204.03458.
- [21]Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. In Advances in Neural Information Processing Systems, 2025. URL https://arxiv.org/abs/2506.08009.
- [22]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the Design Space of Diffusion-Based Generative Models. In Advances in Neural Information Processing Systems, 2022. URL https://arxiv.org/abs/2206.00364.
- [23]Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B. Tenenbaum. Learning to Act from Actionless Videos through Dense Correspondences. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2310.08576.
- [24]Cheuk-Kit Lee, Florentin Coeurdoux, Yuyuan Chen, Sophia Tang, Peter Potaptchik, Yilun Du, Michael S. Albergo, and Eric Vanden-Eijnden. Beckmann Transport Models: From Autonomous Flows to One-Step Maps. arXiv preprint arXiv:2608.01692v3, 2026. URL https://arxiv.org/abs/2608.01692.
- [25]Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow Matching for Generative Modeling. In International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2210.02747.
- [26]Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. In International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2209.03003.
- [27]Susie Lu, Haonan Chen, Weirui Ye, and Yilun Du. DriftWorld: Fast World Modeling through Drifting. arXiv preprint arXiv:2607.15065v2, 2026. URL https://arxiv.org/abs/2607.15065.
- [28]Lyu, J., Bai, S., Xiao, R., Liao, Z., Tan, W., Tang, Z., Shi, R., Peng, C., Ji, Y., Wang, Y., Chen, B., Wang, P., Wang, Z., & Zhao, X. (2026). DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models. https://arxiv.org/abs/2609.15570
- [29]Mandlekar, A., Xu, D., Wong, J., Nasiriany, S., Wang, C., Kulkarni, R., Fei-Fei, L., Savarese, S., Zhu, Y., & Martín-Martín, R. (2022). What Matters in Learning from Offline Human Demonstrations for Robot Manipulation. Proceedings of Machine Learning Research, 164, 1678–1690. https://proceedings.mlr.press/v164/mandlekar22a.htmlproceedings.mlr.press/v164/mandlekar22a.html
- [30]Han Qi, Haocheng Yin, Aris Zhu, Yilun Du, and Heng Yang. Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling. IEEE Robotics and Automation Letters, 11(5):5534–5541, 2026. doi: 10.1109/lra.2026.3673995. URL https://doi.org/10.1109/lra.2026.3673995.
- [31]Julian Quevedo, Ansh Kumar Sharma, Yixiang Sun, Varad Suryavanshi, Percy Liang, and Sherry Yang. WorldGym: World Model as An Environment for Policy Evaluation. In International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=hDBHy1CAw.openreview.net/forum?id=hDBHy1CAw
- [32]Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=TIdIXIpzhoI.openreview.net/forum?id=TIdIXIpzhoI
- [34]Siméoni, O., Vo, H. V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Eun Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., … Bojanowski, P. (2026). DINOv3. Transactions on Machine Learning Research. https://openreview.net/forum?id=2NIGyqNjnsopenreview.net/forum?id=2NIGyqNjns
- [35]Karen Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In *International Conference on Learning Representations*, 2015. URL https://arxiv.org/abs/1409.1556.
- [36]Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models. In *International Conference on Learning Representations*, 2021a. URL https://arxiv.org/abs/2010.02502.
- [37]Yang Song and Prafulla Dhariwal. Improved Techniques for Training Consistency Models. In *International Conference on Learning Representations*, 2024. URL https://arxiv.org/abs/2310.14189.
- [38]Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. In *International Conference on Learning Representations*, 2021b. URL https://arxiv.org/abs/2011.13456.
- [39]Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency Models. In *Proceedings of the 40th International Conference on Machine Learning*, volume 202 of *Proceedings of Machine Learning Research*, pp. 32211–32252. PMLR, 2023. URL https://proceedings.mlr.press/v202/song23a.html.proceedings.mlr.press/v202/song23a.html
- [40]Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards Accurate Generative Models of Video: A New Metric & Challenges. *arXiv preprint arXiv:1812.01717*, 2019. URL https://arxiv.org/abs/1812.01717.
- [41]Vikram Voleti, Alexia Jolicoeur-Martineau, and Christopher Pal. MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation. In *Advances in Neural Information Processing Systems*, 2022. URL https://arxiv.org/abs/2205.09853.
- [42]Walke, H. R., Black, K., Zhao, T. Z., Vuong, Q., Zheng, C., Hansen-Estruch, P., He, A. W., Myers, V., Kim, M. J., Du, M., Lee, A., Fang, K., Finn, C., & Levine, S. (2023). BridgeData V2: A Dataset for Robot Learning at Scale. In Proceedings of The 7th Conference on Robot Learning (Vol. 229, pp. 1723–1736). https://proceedings.mlr.press/v229/walke23a.htmlproceedings.mlr.press/v229/walke23a.html
- [43]Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image Quality Assessment: From Error Visibility to Structural Similarity. *IEEE Transactions on Image Processing*, 13(4): 600–612, 2004. doi: 10.1109/TIP.2003.819861. URL https://doi.org/10.1109/TIP.2003.819861.
- [44]Sherry Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Leslie Kaelbling, Dale Schuurmans, and Pieter Abbeel. Learning Interactive Real-World Simulators. In *International Conference on Learning Representations*, 2024. URL https://arxiv.org/abs/2310.06114.
- [46]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T. Freeman, and Taesung Park. One-step Diffusion with Distribution Matching Distillation. In *IEEE and CVF Conference on Computer Vision and Pattern Recognition*, 2024b. URL https://arxiv.org/abs/2311.18828.
- [47]Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In *IEEE Conference on Computer Vision and Pattern Recognition*, 2018. URL https://arxiv.org/abs/1801.03924.
- [49]Fangqi Zhu, Hongtao Wu, Song Guo, Yuxiao Liu, Chilam Cheang, and Tao Kong. IRASim: A Fine-Grained World Model for Robot Manipulation. In IEEE and CVF International Conference on Computer Vision, pp. 9834–9844, 2025b. URL https://arxiv.org/abs/2406.14540.








