WorksComputer Science
pico-WAM: Evaluating Counterfactual Prediction and Planning Error in a Compact World Action Model
Abstract
When a World Action Model fails at control, is the bottleneck proposal generation or trajectory scoring? We introduce pico-WAM, a compact model with supplied physical state for tracing policy-dynamics interactions. Using 206 Push-T demonstrations and 15 training seeds, we compare data regimes and search configurations. Counterfactual branch training improves short-horizon cost ranking and enlarges the coverage advantage of policy-restarted over warm-started search by 4.8 percentage points; restart wins under both data regimes. Replacing learned trajectory scores with simulator costs improves coverage by 12.2 points while retaining learned proposals. Substituting simulator states in proposal generation alone shows no detected gain. Component diagnostics and a separate door-opening study show why better ranking need not yield better control or outperform imitation. These controlled interventions identify scoring as an effective improvement target in the tested setting and provide a reproducible protocol for diagnosing compact world models.
Code
Introduction
pico-WAM is a compact architecture for isolating how a World Action Model predicts outcomes and selects actions. A single shared representation feeds the policy, transition model, and an explicit state readout, making every decision step traceable: observe, propose actions, imagine consequences, score, and execute. We use this transparent setup to study how training data coverage and search choices govern closed-loop behavior.
A world model trained strictly on demonstrations only observes the actions experts chose, yet a planner must evaluate counterfactual actions they did not choose. Can a compact model score these off-manifold proposals accurately enough to guide control? Figure 1 illustrates this gap between predicting state transitions and selecting the best action sequence.
Figure 1. Comparing candidate choices with their consequences. Two models select 12-step plans from the same pool of 256 candidates. Wireframe outlines show predicted and simulated physical states rather than decoded images. Neither chosen sequence moves the block; the lower simulator cost here reflects a closer agent approach.
We investigate two controlled questions under strictly matched action labels, initial parameters, and training updates. First, does adding counterfactual simulator branches change the benefit of restarting search from the policy versus warm-starting from the previous plan? Second, does replacing learned trajectory scoring with ground-truth simulator costs improve control when retaining learned policy proposals? We find that branch data sharpens cost ranking and favors policy-restarted search, while exact simulator scoring substantially boosts task coverage using the learned proposals. These paired interventions form our core contribution. Component ablations explain the behavior of our shared representation, a supplementary door-opening study extends our ranking findings, and imitation baselines demonstrate why better trajectory ranking alone does not guarantee stronger closed-loop control.
Related Work
PETS and MBPO address model error through uncertainty-aware search and restricted model rollouts Chua et al. [4]; Janner et al. [5]. Our added transitions instead come from the true simulator. DINO-WM plans in visual features, PILOT starts from a policy, and SufficientPlan reduces search cost Zhou et al. [15]; ); Liu et al. [9]; ); Li et al. [7]. Generative Predictive Control ranks diffusion-policy proposals with a learned model Qi et al. [10]. These establish policy-guided planning; we study its components using supervised state and a known geometric objective.
AD-WM and Action–Consequence Alignment train models for action discrimination Qiu et al. [11]; ); Wang et al. [12]; AD-WM already measures ranking and selection regret. Decision-relevant error, model exploitation and planning-horizon effects are established concerns Wang et al. [13]; ); Bhamidipaty et al. [2]; ); Alrasheed et al. [1]. DreamTrue expands action coverage using generated videos Li et al. [8]; our simulator supplies paired ground-truth outcomes. Our paired experiments measure how branch data changes the warm/restart gap and how exact scoring helps while retaining learned proposals. These results explain interactions among existing components. Diffusion Policy supplies an imitation reference Chi et al. [3]; our adapted task does not reproduce published visual-goal benchmark scores.
Model and Planner
The policy answers “what action should I imitate?” The dynamics answers “what happens if I take this action?” A state readout exposes the prediction, and an external planner uses it to compare actions (Figure 2).
Figure 2. pico-WAM. (a) Images and supplied physical state enter a shared latent feeding the policy and readout. Reactive control executes four actions from each eight-action chunk. (b) Actions advance the latent; future latents reuse the same heads. Expert actions, state targets and future encodings supervise the model through Eq. (2). Branches never supply expert action labels. (c) Planning executes one action before observing again. Simulator queries are reserved for diagnostics, except in the explicitly exact planners.
Shared Representation
Inputs are a 64 × 64 image, a fixed instruction and an eight-dimensional state: agent position/velocity and block position/orientation (). A three-layer CNN with 16 spatial-softmax keypoints, a GRU and a state MLP feed a fusion MLP, producing . The policy has three 512-unit hidden layers and predicts eight two-dimensional actions. Residual dynamics with two 256-unit hidden layers advance the latent:
A linear readout maps latents to standardized physical state. Applying the policy to imagined latents supplies an initial plan. The optional image decoder is disabled; full state is available at inference and the instruction is constant. This isolates policy/dynamics interaction rather than visual perception or language generalization.
Training
For a window of transitions,
Each term is squared error averaged over its batch and all output coordinates; future losses also average over . BC regresses eight-action expert chunks. State targets use training-demonstration means and standard deviations, floored at . Future latent targets are observation encodings with gradients stopped; prediction gradients reach the dynamics, readout and initial encoder. Action and auxiliary batches are sampled separately (§ 4.1). A branch perturbs eight demonstration actions from every fourth recorded state with Gaussian noise (SD 0.3), clipped to . The 6,013 branches yield 48,104 transitions, inherit their source episode’s split and supply prediction targets only. Branching changes both state and action coverage.
Planning
The policy rolls forward for 12 steps to form a proposal. The 256 candidates comprise the policy proposal, a search center and 254 noisy perturbations (SD 0.1), clipped to . We score them using
Here , increases linearly from 0.5 to 1.5, and averages distances between corresponding block and target corners in pixels. With agent position and block center of mass , subtracts the agent radius. One search iteration selects the cheapest plan’s first action before replanning. Restart centers search on the current policy proposal (duplicated in the bank); warm uses the shifted previous plan, appending the proposal’s final action. The first call uses the proposal in both cases.
Experimental Protocol
Data and Matched Training
Push-T requires pushing a T-shaped block onto a target. The 206 LeRobot demonstrations LeRobot contributors provide 25,444 transitions at 10 Hz, with keypoints supplying block pose and controller replay recovering agent velocity. We retain the original geometry and 100 Hz physics, but use relative actions (up to 100 px per axis) and rebuild physics each step for exact replay; replayed block transitions agree within 0.53 px at the 99th percentile. Coverage measures maximum target overlap during an episode (including reset), and success requires exceeding 95%. Because the recordings contain no successes under this threshold (Appendix § 8), control success requires completion beyond recorded outcomes.
BC learns actions only, Current adds current-state reconstruction, and WAM optimizes all losses at horizon . Current and WAM train on either demonstration-only windows or equal mixtures of demonstrations and counterfactual branches. Within each training run, all five conditions share initial parameters, action minibatches, 30,000 Adam updates, and batch size 64. Demonstration-only conditions resample windows to match auxiliary sample counts, matching updates and labels rather than compute. Lacking trained dynamics, BC and Current act reactively.
Episodes split into 164 training, 21 validation, and 21 test demonstrations. Each of the 15 training runs uses 50 development and 200 final evaluation starts shared across conditions, with warm and restart search sharing checkpoints. Our predeclared primary contrast tests whether branch training increases the advantage of restart over warm search:
where is mean coverage and denote branch and demonstration-only training. Four additional coverage contrasts and all success contrasts are secondary. All checkpoints were frozen before final evaluation.
Measurements and Interventions
Prediction tests roll forward from one encoded observation under recorded or random actions. Normalized mean squared error (nMSE) divides each state-coordinate error by its training standard deviation before squaring and averaging. Random-action tests sample eight sequences in from starts spaced five steps apart, using zero-action outcomes as a reference. Every ten control steps, a non-intervening diagnostic replay compares predicted and true progress of the selected 12-step plan.
For candidate ranking, every model scores the same 256 candidates across 1,911 states: 470 recorded states and 1,441 states from 50 fresh noisy-expert episodes (action noise 0.2). Half the candidates are uniform in , while half perturb recorded actions (noise 0.1). Selection regret is the chosen sequence’s true cost minus the best true cost in the pool.
The proposal/scoring study crosses learned (L) and exact (E) components—LL, LE, EL, and EE (proposals first)—reusing 15 frozen branch-trained checkpoints across 256 candidates, a 12-step horizon, and 100 fresh starts per run, with LE−LL coverage as primary. Exact scoring evaluates candidate actions in the simulator via Eq. (3), whereas learned scoring uses predicted states. Exact proposals render and re-encode simulator states before querying the policy, replacing the full latent pipeline. Candidate random streams are paired per episode, though closed-loop states naturally diverge. Secondary tests substitute a two-layer, 256-unit state MLP or simulator dynamics while retaining policy and search settings.
Architecture ablations retrain five runs with a single modification: blocked auxiliary encoder gradients, masked image or state inputs (retaining parameter counts), removed instructions, or omitted individual prediction losses. Blocked-gradient minus full-model reactive coverage is primary; gradient blocking tests representation shaping rather than separate encoders.
Additional checks repeat training at with 16-step branches, evaluate frozen models in original gym-pusht 0.1.8 via an action adapter, and compare imitation baselines. Tuned BC sweeps 16 configurations of learning rate, head width, history, and chunk length on separate starts. Diffusion Policy follows its state-based recipe (two observations, 16 predicted/eight executed actions, 100 denoising steps, 5,000 epochs, and EMA weights) across five runs. Comparisons on the dynamics study’s starts were selected post-hoc and are exploratory.
Door Opening
A separate study on Meta-World door-open-v3 Yu et al. [14] (39-D state, 4-D actions) trains two-layer, 256-unit state-space dynamics models on 80 scripted-expert episodes alone or mixed equally with 80 noisy-expert episodes (noise 0.3), totaling 5,840 five-step windows per condition. Models share initialization, expert-data normalization, and 10,000 updates at batch size 128, paired with a separate BC policy.
Primary regret is evaluated across 60 states from 20 fresh expert episodes using 64 shared five-step candidates, with cost defined as handle-to-goal distance plus hand-to-handle distance. Planners perturb a repeated BC action and execute the best sequence’s first step. Secondary control tests use 50 shared starts and 150-step limits across ten runs. Fixed data and evaluation banks mean intervals reflect training-run variance rather than dataset sampling, testing the data-to-ranking mechanism rather than pico-WAM architecture transfer.
Uncertainty
Main-study runs use run-specific start banks paired across conditions, while door-opening runs share a single bank; intervals do not capture dataset or task uncertainty. Secondary intervals are unadjusted, and containing zero does not imply equivalence. Prospective follow-ups use locally timestamped protocols and frozen banks, excluding earlier exploratory runs from confirmatory estimates. Appendix A details reused demonstrations versus fresh evaluation episodes.
What the Components Learn
Broader Data Helps Search
Adding counterfactual branches raises policy-restarted search coverage from 43.8% to 78.4% (Table 1). While restart search already outperforms warm search when trained on demonstrations alone, branch training widens this advantage from 22.5 to 27.3 percentage points—yielding a primary interaction of 4.8 points that is positive in 12 of 15 runs (Figure 3). Without branch data, both planners underperform the same checkpoint’s reactive policy. Overall, these results favor policy-restarted search under both data regimes rather than reversing the preferred initialization strategy.
Figure 3. Branches help restart search more than warm search. (a) Coverage across shared initial states; thin lines show individual training runs, markers and error bars indicate means and 95% confidence intervals, and the dashed line denotes BC. (b) Paired coverage contrasts, with ticks marking individual runs. The data-by-search interaction is primary; remaining intervals are secondary and unadjusted.
| Training | Control | Coverage, % | Success, % |
| BC | Reactive | 23.3 | |
| Current, demos | Reactive | 22.2 | |
| Current, + branches | Reactive | 21.6 | |
| WAM, demos | Reactive | 22.6 | |
| Warm search | 0.0 | ||
| Restart search | 1.8 | ||
| WAM, + branches | Reactive | 20.4 | |
| Warm search | 5.6 | ||
| Restart search | 32.8 |
Table 1. Matched study across 15 training runs and 200 initial states per condition. Coverage is reported as mean ± SD across runs; success is mean percentage. BC and Current act reactively. Per-run counts are provided in the numerical appendix.
On held-out test demonstrations, prediction errors under recorded expert actions are comparable across models. Under random actions, however, the demonstration-only model’s error exceeds the zero-action baseline on average, whereas branch training reduces block prediction error roughly threefold. This advantage persists on fresh noisy-expert episodes (Table 2).
| Measure | Demos | + branches | Difference |
| Prediction, 21 unseen test demonstrations | |||
| State nMSE, recorded, 15 steps | 0.011 | 0.012 | 0.000 [−0.001, 0.002] |
| Block nMSE, random, 10 stepsa | 0.025 | 0.007 | −0.019 [−0.023, −0.015] |
| Block RMSE, px, random, 10 steps | 14.8 | 7.1 | −7.8 [−9.3, −6.2] |
| Prediction, 50 fresh scripted-expert episodes | |||
| State nMSE, recorded, 15 steps | 0.042 | 0.020 | −0.022 [−0.030, −0.014] |
| Block nMSE, random, 10 stepsa | 0.041 | 0.015 | −0.026 [−0.032, −0.020] |
| Block RMSE, px, random, 10 steps | 17.3 | 10.6 | −6.7 [−8.5, −4.9] |
| Candidate ranking, 1,911 common states | |||
| Rank correlation with true cost | 0.49 | 0.80 | 0.32 [0.30, 0.34] |
| Overlap of top 32 | 0.40 | 0.65 | 0.26 [0.24, 0.27] |
| Selection regretb | 7.56 | 2.77 | −4.79 [−5.11, −4.47] |
| Selection optimismb | 5.93 | 2.27 | −3.66 [−4.72, −2.60] |
| Gain over policy priorb,c | 4.67 | 7.01 | 2.34 [2.20, 2.47] |
Table 2. Prediction and ranking diagnostics: means across 15 training runs and paired differences (+ branches minus demos) with 95% intervals. aZero-action reference: 0.023 on test demonstrations, 0.027 on fresh episodes. bCost units from Eq. (3), approximately in pixels; regret is the selected sequence’s true cost minus the best candidate’s true cost, and optimism is true minus predicted cost. cEvaluated on each model’s own restart candidates.
Figure 4. Prediction, ranking, and control reveal distinct error modes. (a) Ten-step block prediction error; thin lines show individual runs and thick lines denote means. (b) Selection regret across 1,911 shared states; paired lines denote runs and black markers indicate means. (c) Predicted versus realized plan progress per run, with the dashed line marking exact agreement. Panels evaluate distinct state populations and do not establish causal mediation.
Across shared evaluation states, branch training increases candidate rank correlation from 0.49 to 0.80 and lowers selection regret from 7.56 to 2.77, improving all 15 runs. Even so, selected plans remain systematically optimistic: branch-trained restart search predicts 10.9 px of progress but achieves only 1.2 px. Because regret evaluates Eq. (3)—which combines block-to-goal distance with an agent-approach penalty—this aggregate improvement does not separate approach gains from actual block manipulation, underscoring that ranking a fixed candidate pool differs from completing a full episode.
Separate Proposals from Scores
Replacing the entire learned model with the ground-truth simulator raises restart coverage to 89.9%, but simultaneously alters both proposal generation and trajectory scoring. Our crossed factorial evaluation disentangles the two (Figure 5): pairing exact simulator scoring with learned policy proposals boosts coverage by 12.2 points, with positive gains in every run. Conversely, supplying exact proposals under learned scoring yields no detectable improvement. Exact scoring improves coverage while retaining learned proposals; substituting simulator states during proposal generation alone shows no detected gain. This identifies scoring as an effective intervention in the tested planner.
Figure 5. Exact scoring improves control even with learned proposals retained. Evaluated across 15 frozen checkpoints on 100 fresh starts each. L and E denote learned and exact computations (proposals listed first). (a) Mean coverage and 95% intervals. (b) Paired differences; * marks the primary contrast, while others are secondary and unadjusted. Exact proposals include simulator rendering and re-encoding.
Notably, even exact simulator dynamics leave warm search at 49.4% coverage, failing to close the gap with policy-restarted search; the relative roles of planning horizon, cost shaping, and search breadth remain open questions. Moreover, a separately trained state-space MLP shows no detectable coverage difference from the shared latent dynamics model while achieving a higher binary success rate (Table 4), confirming that a shared representation is a diagnostic design choice rather than a guaranteed performance advantage.
| Study, seeds | Reference | Coverage | Success |
| Matched, 15 | BC | 0.1 [−3.1, 3.4] | 9.5 [3.4, 15.6] |
| Horizon-matched, 15 | BC | 0.2 [−4.0, 4.4] | 10.0 [4.7, 15.3] |
| Original gym-pusht, 15 | BC | −1.4 [−4.6, 1.7] | 8.1 [1.7, 14.5] |
| Tuned BC, 15 | tuned BC | 2.7 [−0.6, 6.0] | −0.1 [−7.3, 7.1] |
| Diffusion Policy, 5 | DP | −1.0 [−7.6, 5.5] | −16.7 [−27.9, −5.5] |
| State dynamics, 15 | latent | −1.9 [−4.7, 1.0] | 12.0 [5.7, 18.3] |
| Oracle dynamics, 15 | latent | 13.1 [10.6, 15.5] | 45.3 [38.5, 52.0] |
Table 4. Branch-trained restart search minus imitation baselines (top), and alternative dynamics minus latent dynamics (bottom), reported in percentage points with paired 95% intervals. Tuned-BC and Diffusion Policy rows use imitation-study starts, whereas Figure 6 uses dynamics-study starts. Success contrasts are secondary.
Which Architectural Components Matter?
Stopping prediction-loss gradients into the encoder leaves the primary reactive-policy contrast inconclusive, but alters policy-restarted search coverage by −44.3 points in our secondary analysis (Table 3); representations shaped solely by behavior cloning do not support equally effective search. Masking physical state inputs and relying on images alone sharply degrades both reactive and planned control, confirming that our compact implementation depends heavily on explicit state. Omitting remaining auxiliary losses or instruction inputs yields no detectable change, though confidence intervals remain too wide to prove they are redundant. Comparing WAM against Current similarly shows no detectable reactive gain from jointly adding multi-step state and latent-consistency losses.
| Variant minus full | Reactive | Restart |
| Auxiliary gradients blocked1 | 5.1 [−5.1, 15.3] | −44.3 [−50.2, −38.4] |
| Image only | −31.5 [−37.9, −25.1] | −41.6 [−46.7, −36.4] |
| State only | 0.7 [−8.0, 9.4] | −3.3 [−7.1, 0.5] |
| No instruction | 1.5 [−6.3, 9.3] | −4.1 [−8.6, 0.5] |
| No future-state loss | 0.4 [−5.2, 5.9] | −6.4 [−16.6, 3.8] |
| No latent loss | 0.5 [−8.2, 9.2] | −5.2 [−14.3, 3.9] |
| Full model, coverage % | 71.6 | 81.7 |
Table 3. Ablation variants minus the full model: coverage differences with paired 95% intervals across five runs. 1Reactive coverage with blocked auxiliary gradients is primary; other contrasts are secondary and unadjusted.
Better Choices Need Not Beat Imitation
Training with a matched 12-step horizon preserves a positive data-by-search interaction (11.8 points). Evaluating in the original gym-pusht (v0.1.8) environment confirms the individual benefits of branch data and policy-restarted search, though omitting demonstration-only warm search precludes testing the full interaction. Across both checks, branch-trained restart search shows no detectable coverage advantage over base BC (Table 4). In our matched study, restart search improves binary success over untuned BC, but shows no detectable advantage over tuned BC and trails Diffusion Policy in overall success rate (Figure 6). Because reactive policies execute four-action chunks between observations while planners replan at every step, comparing them conflates search with feedback frequency; by contrast, our primary warm/restart and oracle-scoring contrasts hold replanning frequency constant.
Figure 6. Imitation baselines separate better trajectory ranking from better closed-loop control. Exploratory planner-minus-imitation contrasts with unadjusted paired 95% intervals; positive values favor planning. Comparisons share training runs and initial states (five runs for Diffusion Policy). Exact restart is a diagnostic that changes both proposal generation and scoring.
In Meta-World door opening, augmenting demonstrations with noisy rollouts reduces selection regret across all ten runs, dropping from to cost units (paired difference , 95% CI ). Planning with mixed-data dynamics succeeds in 88.2% of trials compared to 78.6% for demonstration-only dynamics, though both trail the reactive BC policy at 98.6% (Figure 7). While cost units are not directly comparable to Push-T, adding noisy-expert rollouts improves candidate ranking and planner success in this door-opening study, while both planners remain below reactive BC in success rate.
Figure 7. Broader transition data also improves cost ranking in door opening. Evaluated across ten training runs with fixed datasets and candidate banks. (a) Paired selection regret, highlighting the mean. (b) Secondary success rate differences with unadjusted paired 95% intervals. Standalone state-space models test the data-to-ranking mechanism rather than shared-representation transfer.
Discussion and Conclusion
pico-WAM makes a WAM’s computation visible: one representation supports imitation, imagined transitions and state prediction. The experiments turn that architecture into practical lessons. Test predictions on actions the planner might choose, rather than only recorded actions. Separate proposal generation from scoring when diagnosing a plan. Measure the data-by-search interaction and retain a direct-action control.
Branches enlarge an existing restart advantage; they do not change which initialization wins. The scoring substitution identifies an effective intervention conditional on learned proposals. Lower regret can reflect agent approach before any block movement (Figure 1), so it need not imply immediate target progress. Branch-trained plans still overestimate realized progress; identifying which contact or state errors drive each failure remains open. These interventions explain behavior in a controlled setting; they do not establish a new architecture, hardware transfer, or general superiority of planning. The shared-model tests use supervised state in Push-T, and the door experiment uses a different architecture. This scope lets a small teaching implementation support reproducible questions about how acting and predicting interact, without requiring it to compete with large models.
Reproduction Details
Study history and scope A five-seed development study triggered a predeclared precision rule; its final bank was never evaluated. The 15-seed study used new final starts and frozen checkpoints. Test demonstrations had appeared in earlier exploration; fresh control starts and prediction episodes provide separate checks. Reports retain protocols, hashes and per-run outcomes.
Retrospective sensitivity Omitting one run at a time gives primary interaction estimates of 4.0–5.6 points and scoring effects of 11.4–12.8. All corresponding paired- intervals remain above zero. The supplement also reports 50,000 paired-run bootstrap draws. These checks reuse the original runs; they do not extend dataset/task generality or replace the primary analyses.
Executable supplement reproduction-review.zip contains source, configurations, archived reports and pinned review dependencies. A fresh Python 3.11 CPU environment regenerated all eight figures and matching numerical files, replayed the saved Figure 1 actions, and passed a miniature factorial pipeline. Full retraining, checkpoints and original data downloads are separate; commands and upstream revisions are included. These checks do not establish full numerical reproduction.
Resources On a V100-SXM2, 30,000 updates took 195 s for BC and 465 s for WAM; restart control took about 13 ms/step including diagnostic replay. Main-study work used 21.3 GPU-hours, horizon-matched training 16.0, architecture tests 16.9 and Diffusion Policy 44.7. The dynamics study used 1,390 allocated CPU-hours, including reruns. These are implementation-specific measurements, not an efficiency comparison or a complete compute total.
Data checks All 25,650 source frames agree across the demonstration and keypoint repositories in indices, rewards and flags. Reconstructed corners agree within px; next-state coverage reproduces rewards within across 25,444 transitions. No recorded frame succeeds (Figure 8); the human recording termination policy is unknown. The study therefore concerns learning from incomplete demonstrations.
Figure 8. Recorded demonstrations remain below the 95% success threshold (mean maximum coverage 84.8%). Original gym-pusht and reconstructed coverage agree within . Source rewards and success flags confirm that incomplete demonstrations precede conversion.
Door implementation Meta-World source tag v3.0.0 is paired with MuJoCo 3.6.0; its package metadata reports 2.0.0. Restored simulator continuations passed the registered tolerance. Fifty final starts are shared across ten seeds, not 500 unique starts. Source hashes and package versions are retained in the supplementary reports. Simulator branching requires arbitrary-state replay and is not a hardware collection procedure.
References
- [1]Ali Alrasheed, Basim Azam, and Naveed Akhtar. The planning limits of latent world models. arXiv preprint arXiv:2609.39235, 2026.arxiv.org/abs/2609.39235
- [2]Logan Mondal Bhamidipaty, Esmeralda S. Whittamer, David Abel, Mykel J. Kochenderfer, and Subramanian Ramamoorthy. Imperfect world models are exploitable. arXiv preprint arXiv:2605.15960, 2026.arxiv.org/abs/2605.15960
- [3]Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. In Robotics: Science and Systems, 2023. doi: 10.15607/RSS.2023.XIX.026.DOI
- [4]Kurtland Chua, Roberto Calandra, Rowan McAllister, and Sergey Levine. Deep reinforcement learning in a handful of trials using probabilistic dynamics models. In Advances in Neural Information Processing Systems, volume 31, 2018.arxiv.org/abs/1805.12114
- [5]Michael Janner, Justin Fu, Marvin Zhang, and Sergey Levine. When to trust your model: Model-based policy optimization. In Advances in Neural Information Processing Systems, volume 32, 2019.arxiv.org/abs/1906.08253
- [7]Changbai Li, Sirui Li, Yichen Yang, Tongfei Chen, Zichao Feng, Shuwei Shao, and Huobin Tan. How much planning is enough? reducing search and computation in world-model planning. arXiv preprint arXiv:2610.08350, 2026a.arxiv.org/abs/2610.08350
- [8]Junyan Li, Ruizhi Li, Yu Liu, Xiangshuo Liu, Mingchao Sun, Hongyu Pan, Mu Xu, Lue Fan, and Zhaoxiang Zhang. Dreamture: Action-faithful robot world model with counterfactual post-training. arXiv preprint arXiv:2610.12468, 2026b.arxiv.org/abs/2610.12468
- [9]Yang Liu, Hanchen Cui, and Karthik Desingh. PILOT: Guiding long-horizon world-model planning for robot manipulation with visual subgoals and policy priors. arXiv preprint arXiv:2610.14335, 2026.arxiv.org/abs/2610.14335
- [10]Han Qi, Haocheng Yin, Aris Zhu, Yilun Du, and Heng Yang. Inference-time enhancement of generative robot policies via predictive world modeling. IEEE Robotics and Automation Letters, 2025.arxiv.org/abs/2502.00622
- [11]Jiabin Qiu, Zixuan Chen, Hongye Cao, Jieqi Shi, Jing Huo, and Yang Gao. AD-WM: Action-discriminative world models for counterfactual model predictive control. arXiv preprint arXiv:2609.30264, 2026.arxiv.org/abs/2609.30264
- [12]Jinping Wang, Zhiqiang Gao, Xiantong Zhen, and Ling Shao. Action-sequence alignment for reliable planning and self-improving in latent world models. arXiv preprint arXiv:2610.04539, 2026a.
- [13]Linhao Wang, Yiyan Fan, and Dongjin Huang. Not all errors matter: Decision-relevant prediction error predicts planning quality. arXiv preprint arXiv:2609.32322, 2026b.arxiv.org/abs/2609.32322
- [14]Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-World: A benchmark and evaluation for multi-task and meta reinforcement learning. arXiv preprint arXiv:1910.10897, 2021.arxiv.org/abs/1910.10897
- [15]Gaoyue Zhou, Hengkai Pan, Yann LeCun, and Lerrel Pinto. DINO-WM: World models on pre-trained visual features enable zero-shot planning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 79115–79135, 2025.arxiv.org/abs/2411.04983







