betaFeedback

WorksComputer Science

Plausible Videos Can Lead to Poor Decisions: Diagnosing and CURE-ing World Models for Planning

Abstract

Action-conditioned world models are increasingly used as world models for planning, yet a world model that makes plausible, realistic-looking videos that are consistent with the training data can produce poor decisions. We address two key questions: (1) when can we trust a world model for planning, and (2) what should we do when we cannot? We introduce CURE (Certify, Upweight, Rollout, Explore), a certificate-guided technique that assesses whether a world model can be trusted for planning and determines what to do next: whether to upweight the training data to debias the model; roll out more samples at test time; or explore by collecting new data where support (how well the training data cover the planner's decisions) is missing. We deploy CURE across multiple domains, spanning simulated physics based on PhyWorld, MuJoCo arm control, and Language-Table pushing in simulation and on real videos. The decisions our certificate trusts have lower regret than those it does not (e.g., 0.06 vs. 0.38 on Language-Table sim and 0.07 vs. 0.38 on MuJoCo contact), whereas video perceptual quality does not identify good decisions. By focusing on decision quality rather than perceptual quality, CURE lowers regret over the best pessimistic decision rule by 10-25% across support levels, and over world-action models, search and policy learning by 41-64%, given the same data.

Introduction

Video generators are widely used as world models: condition on what the camera has seen and an action of interest, then roll out the generator to see which action to take [7, 8, 52, 81, 82]. The reasoning seems defensible: a correct world model should produce convincing video, so convincing video may provide evidence that a world model predicts the consequence of an action well. However, there is considerable evidence that this is not the case: video generators that produce such videos can still violate basic physics [4, 5, 6, 32, 49] and they do not necessarily lead to

good decisions [14, 42, 61, 79].

Figure 1 shows a concrete example of the potential disagreement between video quality and decision quality. Here, the planner needs to move the red tip on a robot arm toward the target position (the MuJoCo benchmark [74], details in § 5.1), and a world model is trained to generate a video of the robot arm’s movement under a specific action, i.e., an imagined rollout. For each candidate action, the model predicts its regret by comparing the red tip’s final position in the imagined rollout with the target position, and the planner favors the action with lower predicted regret. However, in our example, the first row shows a model generating a more convincing video whose imagined rollout leads to the wrong decision. The second row shows the same scene after applying CURE, the technique we introduce in this paper. The imagined rollout is visibly less convincing, with a blurred and ghosted arm, yet the action the planner selects from it is the best one available (regret 0). Nothing in the first row signals the problem: the video is sharp and shows the tip reaching the goal. This is not an isolated case: across decisions and domains, video quality does not identify the decisions with low regret (§ 5.2).

Task (MuJoCo): move the red tip to the goal

Figure 1. Convincing video does not mean a better action. World models make decisions by generating videos as imagined rollouts, predicting the regret from the videos, and picking the action with the lowest predicted regret. A more convincing video may still predict regret badly, and choose an action that fails in the true rollout (top). A less convincing video, despite the visible artifacts (ghosting, blurring and smearing of the arm), can predict accurately and select a better action (bottom).

Why does this happen? Prior work suggests several reasons. At test time, a planner may score actions on too few rollouts being misled by noise [33, 35], or search too few candidate actions, missing better ones [16, 23, 43, 47]. The model may also lack the capacity to learn the dynamics [25]. Finally, the training data may teach correlations rather than general laws, so the model fails to perform out of distribution [12, 32, 36]. These explanations for planning failure reduce to two quantities: model error and support (how well the training data covers the planner’s decisions). Indeed, planning error, or regret, is bounded by the product of these two quantities, plus regularization bias [31, 36, 75]. Standard world model training fits the training data and rewards visually compelling videos, but controls neither error source. As a result, a world model can produce convincing video, but plan poorly.

Existing solutions focus on different error sources. Scaling test-time compute, with more rollouts [33, 35] or more candidates [16, 23, 47], reduces rollout noise and error. However, if support is low, the model’s error cannot be fixed unless support is addressed. Debiased training and retargeting [36] lowers the model’s error where the training data has a few clips of the rarely recorded pairs, but cannot help when there is effectively no support. Collecting more data helps, but only if the new clips are collected where support is missing: more data from the same training source is limited by coverage, not its size [34, 41, 83]. Exploration that targets the model’s uncertainty [27, 56, 68] could find those gaps, but it relies on uncertainty estimates that are themselves unreliable out of distribution [51, 53].

Each solution addresses a different root cause, and sometimes none is needed. We address two fundamental questions:

(Q1) When can we trust our world model for planning?

(Q2) When we cannot trust our model for planning, what can we do about it?

Contributions. We introduce CURE (Certify, Upweight, Rollout, Explore).

  1. When to trust a world model: Certify. We produce a theoretically justified certificate for when to trust a world model for planning. The certificate is easily computed from the training data, using estimates of the model’s error and of its support. It guarantees the planner is as good as any fixed action, up to these two quantities, connecting each error source to planning error.

  1. What to do about it: Upweight, Rollout, Explore. We introduce CURE, a unified technique that turns the certificate into go/no-go criteria for fixing a world model until its decisions pass the certificate (Figure 2). These criteria say when to upweight the training data to debias the model, when to roll out more samples, and when, and where, to explore by collecting new data.

CURE decision pipeline

Figure 2. CURE. We estimate the behavior policy π^b\hat{\pi}_b and the weights α^\hat{\alpha} on the training data, and the model error ε^\hat{\varepsilon} on a validation set. We then train the candidate models (standard, debiased, retargeted) and certify each one (D1). If none passes, we collect new clips where the certificate fails and re-certify, until the clip budget is spent (D2). We keep the model with the largest certificate margin (D3), and spend extra rollouts only on contended actions, those too close to the top score to tell apart, that have support (D4). Full procedure in the supplementary material.

  1. Experimental evidence across multiple domains. On simulated physics, robot arm control, and tabletop pushing in simulation and on real videos, the certificate separates good decisions from bad, whereas video quality does not (Figure 4). Given the same data, CURE lowers regret over the best pessimistic decision rule by 10–25% across support levels, and over world-action models, search and policy learning by 41–64% (Table 1).

Figure 4. The certificate predicts decision error while video quality does not. Setting: Language-Table sim. (a) Regret against the percentile of each signal (smoothed; 95% bands). LPIPS is negated, so higher means better video. (b) Regret of trusted and withheld decisions. PSNR and LPIPS trust the same share as the certificate.

lowpartialhigh
One fix alone (no certificate)
Standard (no fix)0.693±0.0240.693 \pm0.0240.257±0.0160.257 \pm0.0160.126±0.0070.126 \pm0.007
Upweight only0.626±0.0400.626 \pm0.0400.253±0.0630.253 \pm0.0630.173±0.0500.173 \pm0.050
Rollout only0.646±0.0240.646 \pm0.0240.240±0.0160.240 \pm0.0160.115±0.0060.115 \pm0.006
Explore only0.436±0.0190.436 \pm0.0190.257±0.0160.257 \pm0.0160.126±0.0070.126 \pm0.007
Ablation: CURE without one step
without Explore0.605±0.0190.605 \pm0.0190.178±0.0080.178 \pm0.0080.107±0.0050.107 \pm0.005
without Upweight0.436±0.0190.436 \pm0.0190.240±0.0160.240 \pm0.0160.115±0.0060.115 \pm0.006
without Rollout†0.385±0.0210.385 \pm0.0210.192±0.0100.192 \pm0.0100.121±0.0040.121 \pm0.004
Baselines†
Behavior policy0.991±0.0040.991 \pm0.0040.959±0.0090.959 \pm0.0090.985±0.0050.985 \pm0.005
World-action model0.845±0.0350.845 \pm0.0350.790±0.0250.790 \pm0.0250.458±0.0100.458 \pm0.010
Search with a verifier0.705±0.0100.705 \pm0.0100.520±0.0120.520 \pm0.0120.250±0.0060.250 \pm0.006
TD3+BC0.657±0.0130.657 \pm0.0130.497±0.0140.497 \pm0.0140.337±0.0050.337 \pm0.005
AWR0.751±0.0100.751 \pm0.0100.710±0.0100.710 \pm0.0100.410±0.0090.410 \pm0.009
BCQ0.909±0.0050.909 \pm0.0050.834±0.0090.834 \pm0.0090.677±0.0060.677 \pm0.006
SPIBB0.656±0.0090.656 \pm0.0090.573±0.0100.573 \pm0.0100.487±0.0060.487 \pm0.006
Count lower bound0.495±0.0170.495 \pm0.0170.297±0.0120.297 \pm0.0120.152±0.0050.152 \pm0.005
Mean − λ·sd0.462±0.0180.462 \pm0.0180.276±0.0150.276 \pm0.0150.146±0.0050.146 \pm0.005
Worst rollout0.460±0.0170.460 \pm0.0170.279±0.0150.279 \pm0.0150.146±0.0050.146 \pm0.005
Ensemble lower bound‡0.428±0.0110.428 \pm0.0110.236±0.0030.236 \pm0.0030.127±0.0030.127 \pm0.003
CURE0.385±0.0210.385 \pm0.0210.178±0.0080.178 \pm0.0080.107±0.0050.107 \pm0.005

Table 1. CURE has the lowest regret at every support level. Regret (lower is better), averaged over Kikk2C, Language-Table sim and real, and MuJoCo contact, with 95% intervals. All rows share the model, data (incl. CURE’s 300 clips at low support) and evaluation. Rollout uses 16 rollouts per action; †4, as for the baselines. Gray: step not called, so equal to CURE. ‡Uses ten world models instead of one. Baselines are described in § 3.2; per-domain results are in the supplement.

Related Work

Video world models and their evaluation. Video models are increasingly used for planning by imagining each candidate action’s outcome [8, 13, 17, 19, 52, 55, 81]. Benchmarks grade the generated video itself [4, 28, 32, 42, 49], yet visual quality does not guarantee task success [85]. Policy evaluators [62, 72] and action-following benchmarks [9] judge a model with something outside its training data, such as simulators or real trials. None asks whether, from the training data alone, a model can be trusted to rank actions, or what to do when it cannot.

Policy learning with world models. Policies are learned by planning in a world model [25] or training on its rollouts [2, 24, 30, 70, 86]. Online methods collect fresh data [18, 22] or simulated outcomes of non-expert actions [54, 80] where the policy exploits model errors. Offline methods are limited by how well the training data covers the best policy [63, 75], and stay close to the data [37, 38, 84] or penalize model uncertainty [34, 83]. Uncertainty penalties trust the model to know where it is wrong, but generative models can be confident far from their data [44, 51]. We instead treat support as a property of the training data, measured with importance weights.

Debiased estimation and safe policy improvement. Doubly robust estimation corrects for the behavior policy by combining an outcome model with importance weights [10, 11, 15, 64], and recent work moves such corrections into model training [26, 45, 48]. We adopt ADIGen [36] as our debiasing step. Where weights grow large, estimators cap them and fall back on the model [69, 77], and where the behavior policy never takes an action, there is nothing to reweight [65]: debiasing repairs low-support but not absent support. Safe policy improvement accepts a new policy only as far as the training data supports it [40, 71, 73]. CURE brings this test to world models, and collects data where it fails.

Imagination and data collection. Retraining a generator on its own rollouts adds no evidence and risks model collapse [1, 67], so missing support must be filled with executed clips. Active learning and exploration collect where the model is uncertain [27, 56, 66, 68], which misleads when a generator is confident without data. Our collection rule targets missing support instead.

Setup and Background

Problem

The objective is to learn a world model from training data, and use the world model for planning. Our data are nn i.i.d. clips (Xi,Ai,Yi)(X_i, A_i, Y_i), comprised of a context XX or the conditioning frames from a video, an action A∈AA \in\mathcal{A}, and an outcome video YY with utility U(Y)U(Y), under a behavior policy πb(a∣x)\pi_b(a \mid x). A world model GG is a conditional video model: it takes a scene and an action as input to produce a generated video, approximating P(Y∣X=x,A=a)P(Y \mid X = x, A = a). We assume standard identifiability conditions [57] so that this equals the interventional law P(Y(a)∣X=x)P(Y(a) \mid X = x), where Y(a)Y(a) is the video that would result from setting A=aA = a. Planning chooses one action, rolled out over the whole video.

A planner π\pi is a map from scenes to action distributions over an action set B⊂A\mathcal{B} \subset\mathcal{A}. With μ(x,a)=E[U(Y(a))∣X=x]\mu(x,a) = \mathbb{E}[U(Y(a)) \mid X = x] the expected utility of action aa and a∗(x)=arg⁡max⁡a∈Bμ(x,a)a^*(x) = \arg\max_{a \in\mathcal{B}} \mu(x,a) the best action, we minimize regret:

Reg⁡(π)=EX[μ(X,a∗(X))−∑a∈Bπ(a∣X)μ(X,a)].\operatorname{Reg}(\pi) = \mathbb{E}_X \left[\mu(X,a^*(X)) - \sum_{a \in\mathcal{B}} \pi(a \mid X)\mu(X,a)\right].

A summary of the notation used throughout the paper is provided in the Supplement.

Planners

We plan with an estimated utility function u^(x,a)=E[U(G^(x,a))]\hat{u}(x,a) = \mathbb{E}[\mathit{U}(\hat{G}(x,a))] derived from rollouts under the estimated world model G^\hat{G}, and with π^b\hat{\pi}_b, a probability distribution over B\mathcal{B} that estimates πb\pi_b. The primary planner we use is KL-regularized, so that decisions poorly supported by our data are penalized πβ(a∣x)∝π^b(a∣x)exp⁡(u^(x,a)/β)\pi_\beta(a \mid x) \propto\hat{\pi}_b(a \mid x)\exp(\hat{u}(x,a)/\beta). β→0\beta\to0 yields the greedy choice arg⁡max⁡a∈Bu^(x,a)\arg\max_{a \in\mathcal{B}}\hat{u}(x,a), while β→∞\beta\to\infty gives the fitted behavior policy π^b\hat{\pi}_b. KL regularization is widely used [29, 41, 50, 59, 60, 78, 87]. We additionally evaluate greedy, advantage-weighted [59], behavior-cloning-regularized [20], batch-constrained [21] and baseline-bootstrapped [40] planners. We also compare against the behavior policy’s default action, search with a verifier, the rules that world-action models use [13, 54], and pessimistic rules that score each action by a lower bound on its utility (the standard remedy for limited support [31, 34, 39, 83]).

Three Solutions for Building Decision-Ready World Models

When a world model plans poorly, three solutions are available — each targets a different root cause of planning error:

Upweight: Debiased training and retargeting. A standard world model fits the training data as is, so it learns the scene–action pairs it sees often well and extrapolates poorly under low-support. The debiased loss of ADIGen [36] corrects this with a weight α^(x,a)\hat{\alpha}(x,a) estimating the density ratio ν(a∣x)/πb(a∣x)\nu(a \mid x)/\pi_b(a \mid x) between a uniform interventional policy ν\nu over B\mathcal{B} and the behavior policy πb\pi_b. Rather than divide by an estimate of πb\pi_b, which is unstable, the weight is ‘automatically’ learned by directly fitting a Riesz regression [10] on the training data D0\mathcal{D}_0, and is capped at MM for stability — the full loss is restated in the supplementary material. Retargeting [36] then continues training with ν\nu replaced by the planner’s own action distribution, so that the model is most accurate where the planner acts.

Rollout: Scaling test-time compute. At decision time, test-time compute can be scaled in two ways: scoring more candidate actions, or sampling more rollouts per candidate. For the first, draw KK candidates from B\mathcal{B}, roll out each, and choose the one with the highest u^\hat{u} [16, 25, 47]. This can improve planning, but a larger KK makes model over-optimism more likely, especially on actions the training data rarely contains. For the second, more rollouts per candidate reduce the variance of u^(x,b)\hat{u}(x,b) but not its bias. Both help only when the model is accurate on the actions being compared.

Explore: Data collection. The alternative is to execute an action, add the clip to the data, and retrain the world model. Given a budget of mm clips, a collection rule maps the current model, training data, and pool of scenes to mm scene–action pairs to execute, giving a dataset to augment with, Dm\mathcal{D}_m.

Cure: When to Trust a World Model, and What to Do About It

Each solution of § 3.3 targets a different root cause, and sometimes none is needed. We first derive a certificate for assessing whether a world model can be trusted for planning, answering Q1, and then introduce CURE, a certificate-guided framework that determines what to do next, answering Q2.

The Certificate

Throughout, for any ff that acts on clips and actions, we let ∥f∥2=ED0[f(x,a)2]\lVert f \rVert^2 = \mathbb{E}_{D_0}[f(x,a)^2]. We require two pieces: a weight measuring support, and an estimate of model error. First, for a planner π\pi over B\mathcal{B}, the weight $\alpha_\pi(x,a) = \pi(a \mid π(a∣x)/πb(a∣x)\pi(a \mid x)/\pi_b(a \mid x) is the density ratio between π\pi and the behavior policy, measuring how far π\pi is from the behavior policy. Second, the model’s error for planning is ε=∥δˉ∥\varepsilon= \lVert\bar{\delta}\rVert, where δ(x,a)=μ(x,a)−u^(x,a)\delta(x,a) = \mu(x,a) - \hat{u}(x,a) and δˉ\bar{\delta} centers δ\delta over the actions in B\mathcal{B}. Together, they bound the regret of a planner built from a world model:

Proposition 1 (Restated from Kim et al. [36]). For any planner π\pi over B\mathcal{B}, let π⋆\pi^\star be the optimal planner that always takes a⋆(x)a^\star(x), and u^π(x)=∑b∈Bπ(b∣x)u^(x,b)\hat{u}_\pi(x) = \sum_{b\in\mathcal{B}} \pi(b \mid x)\hat{u}(x,b) the model’s predicted utility of π\pi. Then,

Reg⁡(π)≤ε∥απ⋆−απ∥+E[u^(x,a⋆)−u^π(x)].\operatorname{Reg}(\pi) \le\varepsilon\lVert\alpha_{\pi^\star} - \alpha_\pi\rVert+ \mathbb{E}\left[\hat{u}(x,a^\star) - \hat{u}_\pi(x)\right].

The first term combines the model’s centered utility error on the training distribution, ε\varepsilon, with the difference between the optimal and learned planners relative to the behavior policy. This term can be large when the model is inaccurate or when the planners differ on actions poorly supported by the training data. The second term measures how much predicted utility we lose by following the learned planner instead of the optimal action. For our KL-regularized planner, this can reflect regularization bias.

From the regret bound to a certificate

A small bound in Proposition 1 would support trusting the planner, but the bound depends on the optimal policy π⋆\pi^\star, which is unknown. We therefore apply the same reasoning to each fixed action b∈Bb \in\mathcal{B}: how much do we lose by following our planner instead of always taking bb? Equivalently, we lower-bound the improvement: how much might our planner improve over a constant-action planner bb?

To answer this, we substitute estimates into Proposition 1: the weights α^\hat{\alpha} follow from the fitted behavior policy π^b\hat{\pi}_b, and ε^\hat{\varepsilon} is estimated on validation clips: a cross-fitted regression [11] of the residual U(Y)−u^(X,A)U(Y)-\hat{u}(X,A) on scene and action gives a bias δ^(x,b)\hat{\delta}(x,b), which we center over B\mathcal{B}. This gives the certificate for action bb:

Cert⁡b(G^,π)=E[u^π(x)−u^(x,b)]−ε^∥α^b−α^π∥.(1)\operatorname{Cert}_b(\hat{G},\pi) = \mathbb{E}\left[\hat{u}_\pi(x)-\hat{u}(x,b)\right] - \hat{\varepsilon}\lVert\hat{\alpha}_b-\hat{\alpha}_\pi\rVert. \tag*{(1)}

The first term is how much the model predicts our planner will improve over bb. The second accounts for how much model error and limited support could change this comparison. If Cert⁡b>0\operatorname{Cert}_b > 0, the estimated bound favors our planner over bb, even after accounting for model error and limited support. If this holds for every b∈Bb \in\mathcal{B} up to a tolerance τ\tau, that is, min⁡b∈BCert⁡b(G^,π)>−τ\min_{b\in\mathcal{B}}\operatorname{Cert}_b(\hat{G},\pi) > -\tau, we certify our world model: no fixed action is estimated to outperform our planner by more than τ\tau. This certification holds with high probability (Supplement).

Per decision. For a single scene xx, dropping the expectation gives Cert⁡b(x)=u^π(x)−u^(x,b)−ε^∥α^b−α^π∥x\operatorname{Cert}_b(x) = \hat{u}_\pi(x) - \hat{u}(x,b) - \hat{\varepsilon}\lVert\hat{\alpha}_b-\hat{\alpha}_\pi\rVert_x with the norm over actions at xx. If Cert⁡b(x)>−τ\operatorname{Cert}_b(x) > -\tau for every covered bb (α^b≤M\hat{\alpha}_b \le M), the decision has regret at most τ\tau (Supplement); Sec. 5.2 uses this.

When the certificate fails, the pieces indicate why: the model’s error is too large, the planner strays too far from the support of the training data, or the predicted gain is too small. Each root cause calls for a different fix, which CURE makes precise.

From Certificate to Actions

We summarize our pipeline in Figure 2, read from top to bottom. We use three candidate world models G^={G^standard,G^debias,G^retarget}\hat{G} = \{\hat{G}_{\mathrm{standard}},\hat{G}_{\mathrm{debias}},\hat{G}_{\mathrm{retarget}}\} of Sec. 3.3 (standard, debiased, and debiased + retargeted) for planning. We first check whether any world model can be certified (D1). If none is certified, we check whether low support calls for collecting new clips (D2). Otherwise, we select the certified model with the largest margin (D3). Finally, for each scene, we decide whether to act, use more rollouts, or collect new clips (D4).

D1 (Certify): Certifying a world model. Score every candidate world model. A world model is certified when Cert⁡b>−τ\operatorname{Cert}_b > -\tau for every b∈Bb \in\mathcal{B}: no fixed action can beat the planner by more than τ\tau, even after accounting for the model error and support. If no candidate is certified, the training data cannot support the decision.

Rule: if min⁡b∈BCert⁡b(G^)≤−τ\min_{b \in B} \operatorname{Cert}_{b}(\hat{G}) \le-\tau for every G^∈G\hat{G} \in\mathcal{G} and clip budget remains, go to D2; otherwise go to D3.

D2 (Explore): Collect mm clips, Dm\mathcal{D}_m, where the certificate fails. Low support may explain the failure in D1: when the planners differ on poorly supported actions, ∥α^b−α^π∥\lVert\hat{\alpha}_b - \hat{\alpha}_{\pi} \rVert can be large, so the predicted improvement may not be enough to account for model error and limited support. In D2, we propose to: (1) Check low support: Identify actions bb with α^(x,b)>M\hat{\alpha}(x,b) > M; we denote this set of actions by Bls(x)\mathcal{B}_{\mathrm{ls}}(x) (defined in the Supplement). (2) Check predicted utility: Check whether u^π(x)−u^(x,b)>0\hat{u}_{\pi}(x) - \hat{u}(x,b) > 0 for these actions, i.e., the model predicts a gain over bb that the training data cannot certify. (3) Calculate the deficit: An action leaves Bls(x)\mathcal{B}_{\mathrm{ls}}(x) once its weight falls back under MM, which for the uniform target ν\nu means π^b(b∣x)≥1/(∣B∣M)\hat{\pi}_b(b \mid x) \ge1/(|B|M). The deficit counts how many of the nn training clips are missing to reach that level,

d(x,b)=n[1∣B∣M−π^b(b∣x)]+d(x,b) = n \left[\frac{1}{|B|M} - \hat{\pi}_b(b \mid x)\right]_+

so it is zero for supported actions and largest, about n/(∣B∣M)n/(|B|M) clips, for actions the behavior policy never took. (4) Collect and retrain: If the check in (2) passes, draw mm pairs (x,b)(x,b), b∈Bls(x)b \in\mathcal{B}_{\mathrm{ls}}(x), in proportion to d(x,b)d(x,b), execute each, and retrain with Dm\mathcal{D}_m.

Rule: If some b∈Bls(x)b \in\mathcal{B}_{\mathrm{ls}}(x) has u^π(x)−u^(x,b)>0\hat{u}_{\pi}(x) - \hat{u}(x,b) > 0, the model predicts a gain over a low-support action that the training data cannot certify: collect Dm\mathcal{D}_m there, and return to D1.

D3 (Upweight): Among the certified G^standard\hat{G}_{\mathrm{standard}}, G^debias\hat{G}_{\mathrm{debias}}, and G^retarget\hat{G}_{\mathrm{retarget}}, which should we choose? They all share the same architecture, training data, training procedure.

Rule: Among the certified world models C⊆G\mathcal{C} \subseteq\mathcal{G}, keep the one with the largest certified margin, that is,

G^⋆=arg⁡max⁡G^∈C min⁡b∈B Cert⁡b(G^,π^G^)\hat{G}^{\star} = \underset{\hat{G} \in\mathcal{C}}{\arg\max}\ \underset{b \in B}{\min}\ \operatorname{Cert}_{b}(\hat{G},\hat{\pi}_{\hat{G}})

This rule says whether the training data’s support suffices (G^standard\hat{G}_{\mathrm{standard}}), whether low-support can be corrected without more data (G^debias\hat{G}_{\mathrm{debias}}), and whether training should follow the planner to where it acts (G^retarget\hat{G}_{\mathrm{retarget}}).

D4 (Rollout): Test-time compute? With G^⋆\hat{G}^{\star} selected in D3, we now ask whether the model can clearly distinguish between actions for a given scene xx. For a scene xx with KK candidate actions, each scored by averaging RR rollouts, and preferred action a^(x)=arg⁡max⁡bu^(x,b)\hat{a}(x) = \arg\max_b \hat{u}(x,b), the contended actions Bcon(x)\mathcal{B}_{\mathrm{con}}(x) are those the model cannot separate from a^(x)\hat{a}(x) within its model error, u^(x,a^(x))−u^(x,b)≤2ϵ^\hat{u}(x,\hat{a}(x)) - \hat{u}(x,b) \le2\hat{\epsilon}. In D4 we therefore propose to: (1) Check contended actions: If ∣Bcon(x)∣=1|\mathcal{B}_{\mathrm{con}}(x)| = 1, the preferred action a^(x)\hat{a}(x) can be clearly distinguished from other actions within the model error, and therefore, we can select the preferred action. (2) Check low support: If Bcon(x)∩Bls(x)≠∅\mathcal{B}_{\mathrm{con}}(x) \cap\mathcal{B}_{\mathrm{ls}}(x) \ne\varnothing, the contended actions include actions poorly supported by the training data, so we need to return to D2 to collect more data in support for them. (3) Rollout: Otherwise, double the rollouts on the contended actions (R→2RR \to2R) and the number of candidate actions (K→2KK \to2K, adding only supported actions), recompute Bcon(x)\mathcal{B}_{\mathrm{con}}(x), and repeat until (1) or (2) applies or the compute budget is spent; if still undecided, return to D2.

Rule: Double the rollouts per contended action (R→2RR \to2R) and the number of candidate actions (K→2KK \to2K, adding only actions with support), then recompute Bcon(x)\mathcal{B}_{\mathrm{con}}(x). Repeat until one action remains or the compute budget is spent. If still undecided, send xx to D2.

Examples. Figure 3 walks through CURE on two test scenes.

Figure 3. CURE in action. Top: MuJoCo contact. Bottom: Language-Table sim. Left: the standard world model’s chosen action misses the goal in the real rollout (true outcome shown). Middle: CURE’s certificate decides the fix. Top: the certificate is below zero because the training data has too little support for the needed action, so CURE collects new clips (Explore). Bottom: the certificate passes, so CURE acts, with the upweighted model and more rollouts per action (Upweight, Rollout). Right: CURE’s chosen action reaches the goal. The regret numbers are normalized such that 0 is the best action in the bank, and 1 is a random action on average.

Experiments

We test each step of CURE: Certify (§ 5.2), Explore (Sec. 5.3), Upweight (Sec. 5.4) and Rollout (Sec. 5.5). In § 5.6 we run our experiments where we follow all four calls of CURE, and compare CURE with other families.

Setup

Domains. We test CURE in settings commonly used to evaluate physical prediction and control. kick2c [36] is a simulated physics task closely related to PHYRE [3] and PhyWorld [32], varied to control how far decisions fall out of distribution: a 2D ball is kicked toward a goal line while a gust, varying from scene to scene, pushes it sideways. MuJoCo [74], a standard benchmark for model-based control, is a planar arm in two variants: (i) in contact, the arm chooses a push to move a block onto a goal spot, a contact-rich setting where outcomes change abruptly with the action; (ii) in smooth, the same arm reaches for a goal point with no block, serving as a control where outcomes vary smoothly. Language-Table [46], widely used to train and evaluate video world models [81], is a tabletop task in which a 2D push moves a named block toward a target, in simulation (sim) and on real robot video (real). We simplify by focusing on the continuous action without language. On real video untaken actions cannot be executed, so their outcomes are scored by a separate model. Figures in this section focus on Language-Table sim; Table 1 summarizes results over different domains with further details in the supplement.

Support. To simulate training data with low-support, we control how often the behavior policy takes the actions the test scenes need. In each task the behavior policy has a default action for the common case, such as the usual kick under the common gust. A fraction η\eta of its clips instead takes other actions inducing low-support. We control support to separate the failure modes it causes, and test the findings in both simulated and real-robot data. We call η=0.25\eta= 0.25 high support, η=0.05\eta= 0.05 partial support and η=0.005\eta= 0.005 low support.

Models. The primary world model is a 28-million-parameter latent video Diffusion Transformer (DiT) [58], the architecture behind today’s leading video generators and world models [7, 52, 82]. We call this DiT, trained on the training data with the unweighted loss, the standard model; every comparator that needs a world model uses it, so differences between methods reflect how they decide, not which model they use. We use a compact model because each result is repeated over 20 seeds, three support levels and three training losses, with collection and retraining on top; we check that the findings hold at other sizes (7M and 115M) and on other backbones as detailed in the supplement. We compare against world-action models and policy learning on world models (§ 5.6).

Evaluation. Decisions use the planner πβ\pi_\beta of § 3.2, and regret is normalized so that a random action scores 1 and the best action in the bank 0. Each step of CURE is evaluated with the training data, test set, seeds and planner held fixed. For Certify, we test whether the certificate of each decision predicts its regret, and compare it with video quality scores. For Upweight, we compare the regret of each world model

across support levels. For Rollout, we compare regret as the number of candidate actions scored grows, at each support level. For Explore, we give each collection rule the same budget of new clips, retrain the model, and compare the resulting regret. Implementation details, per-domain choices, and results are in the supplementary material.

Certify

We test whether the certificate from Eq. (1) separates good decisions from bad ones, and whether video quality does too. In Figure 4a, we sort decisions by each signal (certificate Cert, PSNR, LPIPS), from least to most trusted (low to high Cert and PSNR, high to low LPIPS). We then plot the mean regret of decisions at each percentile of each signal. We find that regret falls steadily as the certificate rises (Spearman ρ=−0.67\rho= -0.67, 95% interval [−0.69,−0.65][-0.69, -0.65]), meaning a higher certificate reliably identifies a better decision. Video quality does not: among the decisions of any one trained model, PSNR is unrelated to regret (ρ≈−0.03\rho\approx-0.03), while the certificate’s correlation is −0.40-0.40 (negative for every model).

In Figure 4b, we use each signal to plan. Decisions that pass the certificate are almost always good: they make up 22% of decisions, with regret 0.06, compared to 0.38 for the decisions it does not certify. If these decisions were made based on the equivalent top 22% of videos according to PSNR, those decisions are worse than the rest, with regret 0.49 against 0.26.

Across domains1, the certificate finds where the training data lacks support and ranks world models by regret, which video quality does not. The certificate also separates good decisions from bad: those it certifies have lower regret than those it withholds. Across whole models, Fréchet Video Distance (FVD) [76] tells a similar story: on held-out clips of logged actions, averaged over support levels, the standard model’s videos are more realistic than CURE’s (FVD 22 vs. 80), but regret is 3.1× higher.

Explore

How to collect. We compare collection strategies at low support with a fixed budget of 300 new clips. CURE collects where support is missing and more than halves regret (0.73 to 0.29). Here clips are allocated across support levels, but any grouping of scenes, such as by instruction or wind condition, can be used. Rules that follow the behavior policy do worse: more clips from the same behavior policy give 0.38, and clips sampled from the planner πβ\pi_\beta with β>0\beta> 0, which stays close to the behavior policy, give 0.41, against 0.30 for CURE’s rule in this comparison. Rules that follow the model also do worse: collecting the action it ranks best in each scene gives 0.34, and collecting where it is most uncertain gives 0.38. Collecting uniformly over actions does well here (0.31), but only because at low support almost every action needs data. Uniform collection does not know where support is missing, which matters once the budget must be split across support levels as shown below.

Sample efficiency. CURE spends its clip budget where the certificate is weakest first, in increasing order of the mean certificate across support levels. We compare it with splitting the budget evenly across support levels (even split), at equal total clips. The even split is less efficient than CURE: with 300 clips its regret is higher by 0.020 (paired 95% interval [0.008, 0.032]), and it needs 900 clips to reach what CURE reaches with a budget of 600 (0.13).

The same story holds across domains: collecting where support is missing lowers regret at low support in every domain except MuJoCo smooth, the control with nothing to fix, and spreading the same clips evenly across support levels is worse than CURE.

Upweight

We compare the standard, debiased, and debiased with retargeting (just called retargeted) models across support levels on Language-Table sim. At high support, every model is already near zero regret — weights are stable, and there are enough demonstrations to plan well. At partial support, debiasing lowers regret by about a third (from 0.25 to 0.17), and retargeting lowers it further (0.14). At low support, every model fails equally (0.72 to 0.73), since no loss can recover actions the training data never shows; this is where new data is needed (see Sec. 5.3). Removing the Upweight step from CURE confirms this (see Table 1): averaged over domains, regret at partial support rises from 0.18 to 0.24, and from 0.11 to 0.12 at high support.

Two findings from the other domains qualify this picture. Low support is not always hopeless: at equal training data, upweighting lowers regret at low support by 0.23 on kick2c and by 0.13 on Language-Table real. Also, debiasing is not always the right call. In particular, when the weights add variance with little reduction in bias, CURE keeps the standard model, using a check on the validation set. For example, on MuJoCo contact debiasing raises regret at partial and high support (Supplement), and CURE selects the standard model.

Rollout

Test-time compute helps only where the model has support, and the certificate tells CURE where that is (Figure 5). At high support, all methods are competitive. At partial support, scoring 128 candidate actions instead of 16 halves the regret of the retargeted model (0.39 to 0.18) and helps the standard model less (0.45 to 0.30). At low support, regret stops improving after 16 candidates (0.75 to 0.77 for the standard model), since the issue is not variance, it is bias.

Figure 5. Test-time compute helps only where the model has support. Regret against number of candidates scored KK. Hollow: low; filled: partial; dotted: high support.

Averaging more rollouts per action behaves the same way: going from 1 to 16 lowers the retargeted model’s regret at partial support from 0.17 to 0.12, but leaves low-support regret above 0.68. The certificate separates these cases: it passes 4% of decisions at low support, against 25% at partial and 37% at high support. CURE therefore spends compute where the certificate finds support, and collects new clips where it does not.

Cure as a strategy

Across five domains, the certificate predicts decision error where video quality does not; collecting where support is missing is the main fix at low support, debiasing at partial support, and extra test-time compute helps only once support exists. Choosing among these fixes by the certificate lowers regret over standard world models by 31–61% at low support and up to 50% at partial support, without raising it at high support. Against baselines from every family, given the same data and compute, CURE has the lowest or tied-lowest regret in every domain. We evaluate CURE holistically, first across differing support levels, and in comparison to competing baselines mentioned in § 3.2. Recall that low, partial and high support mean that a fraction η\eta of 0.5%, 5% and 25% of the behavior policy’s clips depart from its default action.

CURE across support levels. Table 1 compares CURE with each fix applied alone at every support level, with CURE missing one step, and with baselines from each family. At low support, only new clips help; the certificate fails, and CURE collects. At partial support, upweighting and averaging more rollouts help; the certificate passes, and CURE acts with both. At high support, the model is already close to correct, and extra rollouts trim what remains. CURE has the lowest regret at every support level: 0.39, 0.18 and 0.11, against 0.69, 0.26 and 0.13 for the standard model and at least 0.44, 0.24 and 0.12 for any single fix. Removing one step at a time shows where each matters: without Explore, regret at low support rises to 0.60; without Upweight, regret at partial support rises to 0.24; and without Rollout, regret at partial and high support rises to 0.19 and 0.12. Where the certificate does not call for a step, removing it changes nothing. Upweight and Rollout help more together, since extra rollouts reduce variance but not bias.

CURE against baselines. Given the same data, CURE’s regret is below that of world-action models, search with a verifier and policy learning in the world model at every support level. The strongest baselines are decision rules that score actions pessimistically, by a lower bound or the worst imagined rollout. CURE’s regret is lower than all of them at every support level, by 16–36% for the single-model rules. An ensemble lower bound, which runs ten world models, comes closest in regret (0.428, 0.236 and 0.127 against CURE’s 0.385, 0.178 and 0.107). They act on the decision rule rather than the model or data, so they are complementary to CURE.

CURE’s regret gains, largest at low support, hold beyond our 28M model: at 115M, 340M, on DIAMOND [2], and on the pretrained CogVideoX-2B [82]. On CogVideoX-2B the certificate still tracks decision quality: its rank correlation with regret is −0.38 within each support level and seed, and certified decisions have regret 0.016 against 0.245 for the rest (Supplement).

Conclusions

World model visual quality says little about how well it can plan. While several solutions exist to close this gap, there is little guidance on which solution to use. We introduce CURE, which certifies when a world model can be trusted for planning. When it cannot, CURE recommends when to upweight the training data, when to roll out more at test time, and when and where to explore. As world models are increasingly used for planning, CURE provides a certificate-guided framework for assessing when an existing model can be trusted and deciding how to allocate limited data and compute where they matter.

LLM Usage Statement

We used Claude (Anthropic) as an assistant throughout the preparation of this paper. Writing: it drafted, edited and shortened text in all sections of the main paper and the supplement, from the authors’ outlines, notes and results; the authors revised and approved all text. Figures: it wrote plotting code for the schematic figures Figure 2 and for the layout and styling of the result figures; all plotted values come from the authors’ experiment outputs. Analysis and checking: it helped draft experiment specifications and analysis plans, checked the consistency of reported numbers against result tables, and reviewed the paper for errors. The authors take full responsibility for the content of the paper.

References

  1. [1]S. Alemohammad et al. Self-consuming generative models go MAD. In ICLR, 2024. 3arxiv.org/abs/2307.01850
  2. [2]E. Alonso, A. Jelley, V. Micheli, A. Kanervisto, A. Storkey, T. Pearce, and F. Fleuret. Diffusion for world modeling: Visual details matter in Atari. In NeurIPS, 2024. 2, 8arxiv.org/abs/2405.12399
  3. [3]Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. PHYRE: A new benchmark for physical reasoning. In Advances in Neural Information Processing Systems (NeurIPS), 2019. 5arxiv.org/abs/1908.05656
  4. [4]H. Bansal, C. Peng, Y. Bitton, R. Goldenberg, A. Grover, and K.-W. Chang. VideoPhy-2: A challenging action-centric physical commonsense evaluation in video generation. arXiv:2503.06800, 2025. 1, 2arxiv.org/abs/2503.06800
  5. [5]H. Bansal et al. VideoPhy: Evaluating physical commonsense for video generation. In ICLR, 2025.arxiv.org/abs/2406.03520
  6. [6]D. M. Bear et al. Physion: Evaluating physical prediction from vision in humans and machines. In NeurIPS Datasets and Benchmarks, 2021. 1arxiv.org/abs/2106.08261
  7. [7]T. Brooks et al. Video generation models as world simulators. OpenAI technical report, 2024. 1, 6
  8. [8]J. Bruce et al. Genie: Generative interactive environments. In ICML, 2024. 1, 2arxiv.org/abs/2402.15391
  9. [9]Sixiang Chen, Jiaming Liu, Jixian Wu, Yichen Guo, Tinghao Wang, Siyuan Qian, Hao Chen, Jiajun Cao, Jian Tang, and Shanghang Zhang. Do robotic world models really follow actions? diagnosing and aligning action-conditioned generation for policy learning. arXiv preprint arXiv:2608.24885, 2026. 2arxiv.org/abs/2608.24885
  10. [10]V. Chernozhukov, W. Newey, and R. Singh. Automatic debiased machine learning of causal and structural effects. Econometrica, 2022. 2, 3arxiv.org/abs/1809.05224
  11. [11]V. Chernozhukov et al. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 2018. 2, 4
  12. [12]P. de Haan, D. Jayaraman, and S. Levine. Causal confusion in imitation learning. In NeurIPS, 2019. 2arxiv.org/abs/1905.11979
  13. [13]Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in neural information processing systems, 36:9156–9172, 2023. 2, 3arxiv.org/abs/2302.00111
  14. [14]H. Duan, H.-X. Yu, S. Chen, L. Fei-Fei, and J. Wu. World-Score: A unified evaluation benchmark for world generation. In ICCV, 2025. 1
  15. [15]M. Dudík, J. Langford, and L. Li. Doubly robust policy evaluation and learning. In ICML, 2011. 2arxiv.org/abs/1103.4601
  16. [16]F. Ebert, C. Finn, A. X. Lee, and S. Levine. Self-supervised visual planning with temporal skip connections. In CoRL, 2017. 2, 3arxiv.org/abs/1710.05268
  17. [17]Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. Visual foresight: Model-based deep reinforcement learning for vision-based robotic control. arXiv preprint arXiv:1812.00568, 2018. 2arxiv.org/abs/1812.00568
  18. [18]Yunhai Feng, Nicklas Hansen, Ziyan Xiong, Chandramouli Rajagopalan, and Xiaolong Wang. Finetuning offline world models in the real world. arXiv preprint arXiv:2310.16029, 2023. 2arxiv.org/abs/2310.16029
  19. [19]Chelsea Finn and Sergey Levine. Deep visual foresight for planning robot motion. In 2017 IEEE international conference on robotics and automation (ICRA), pages 2786–2793. IEEE, 2017. 2arxiv.org/abs/1610.00696
  20. [20]Scott Fujimoto and Shixiang Shane Gu. A minimalist approach to offline reinforcement learning. In NeurIPS, 2021. 3arxiv.org/abs/2106.06860
  21. [21]Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In ICML, 2019. 3arxiv.org/abs/1812.02900
  22. [22]Yanjiang Guo, Tony Lee, Lucy Xiaoyang Shi, Jianyu Chen, Percy Liang, and Chelsea Finn. Vlaw: Iterative co-improvement of vision-language-action policy and world model. arXiv preprint arXiv:2602.12063, 2026. 2arxiv.org/abs/2602.12063
  23. [23]D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson. Learning latent dynamics for planning from pixels. In ICML, 2019. 2arxiv.org/abs/1811.04551
  24. [24]D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse domains through world models. Nature, 2025. 2
  25. [25]N. Hansen, H. Su, and X. Wang. TD-MPC2: Scalable, robust world models for continuous control. In ICLR, 2024. 2, 3arxiv.org/abs/2310.16828
  26. [26]Toru Hishimuna and Kei Senda. Weighted model estimation for offline model-based reinforcement learning. Advances in neural information processing systems, 34:17789–17800, 2021. 2
  27. [27]N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel. Bayesian active learning for classification and preference learning. arXiv:1112.5745, 2011. 2, 3arxiv.org/abs/1112.5745
  28. [28]Z. Huang et al. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024. 2arxiv.org/abs/2311.17982
  29. [29]Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In ICML, 2022. 3arxiv.org/abs/2205.09991
  30. [30]M. Janner et al. When to trust your model: Model-based policy optimization. In NeurIPS, 2019. 2arxiv.org/abs/1906.08253
  31. [31]Ying Jin, Zhuoran Yang, and Zhaoran Wang. Is pessimism provably efficient for offline RL? In International Conference on Machine Learning (ICML), 2021. 2, 3arxiv.org/abs/2012.15085
  32. [32]B. Kang et al. How far is video generation from world model: A physical law perspective. arXiv:2411.02385, 2024. 1, 2, 5arxiv.org/abs/2411.02385
  33. [33]M. Kearns, Y. Mansour, and A. Y. Ng. A sparse sampling algorithm for near-optimal planning in large Markov decision processes. Machine Learning, 49(2–3):193–208, 2002. 2
  34. [34]R. Kidambi, A. Rajeswaran, P. Netrapalli, and T. Joachims. MOReL: Model-based offline reinforcement learning. In NeurIPS, 2020. 2, 3arxiv.org/abs/2005.05951
  35. [35]M. J. Kim, Y. Gao, T.-Y. Lin, Y.-C. Lin, Y. Ge, G. Lam, P. Liang, S. Song, M.-Y. Liu, C. Finn, and J. Gu. Cosmos Policy: Fine-tuning video models for visuomotor control and planning. arXiv:2601.16163, 2026. 2arxiv.org/abs/2601.16163
  36. [36]Raphael C. Kim, Jingzen Zhu, Ramin Zabih, and Michele Santacatterina. Automatic, debiased, and invariant counterfactual generation under general interventions. arXiv preprint arXiv:2606.07399, 2026. 2, 3, 4, 5
  37. [37]Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021. 2arxiv.org/abs/2110.06169
  38. [38]Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33:1179–1191, 2020. 2arxiv.org/abs/2006.04779
  39. [39]Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017. 3arxiv.org/abs/1612.01474
  40. [40]R. Laroche, P. Trichelair, and R. Tachet des Combes. Safe policy improvement with baseline bootstrapping. In ICML, 2019. 2, 3
  41. [41]S. Levine, A. Kumar, G. Tucker, and J. Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv:2005.01643, 2020. 2, 3arxiv.org/abs/2005.01643
  42. [42]D. Li et al. WorldModelBench: Judging video generation models as world models. arXiv:2502.20694, 2025. 1, 2arxiv.org/abs/2502.20694
  43. [43]F. Liu, H. Wang, Y. Cai, K. Zhang, X. Zhan, and Y. Duan. Video-T1: Test-time scaling for video generation. In ICCV, 2025. 2arxiv.org/abs/2503.18942
  44. [44]Cong Lu, Philip J Ball, Jack Parker-Holder, Michael A Osborne, and Stephen J Roberts. Revisiting design choices in offline model-based reinforcement learning. arXiv preprint arXiv:2110.04135, 2021. 2arxiv.org/abs/2110.04135
  45. [45]Alex Luedtke and Kenji Fukumizu. Doublegen: Debiased generative modeling of counterfactuals. arXiv preprint arXiv:2509.16842, 2025. 2arxiv.org/abs/2509.16842
  46. [46]C. Lynch et al. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023. 5arxiv.org/abs/2210.06407
  47. [47]N. Ma, S. Tong, H. Jia, H. Hu, Y.-C. Su, M. Zhang, X. Yang, Y. Li, T. Jaakkola, X. Jia, and S. Xie. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv:2501.09732, 2025. 2, 3arxiv.org/abs/2501.09732
  48. [48]Valentyn Melnychuk and Stefan Feuerriegel. Gdr-learners: Orthogonal learning of generative models for potential outcomes. In International Conference on Learning Representations, pages 105267–105301, 2026. 2
  49. [49]S. Motamed, L. Culp, K. Swersky, P. Jaini, and R. Geirhos. Do generative video models understand physical principles? arXiv:2501.09038, 2025. 1, 2arxiv.org/abs/2501.09038
  50. [50]Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. AWAC: Accelerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.09359, 2020. 3arxiv.org/abs/2006.09359
  51. [51]Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilian Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? arXiv preprint arXiv:1810.09136, 2018. 2
  52. [52]NVIDIA. Cosmos world foundation model platform for physical AI. arXiv:2501.03575, 2025. 1, 2, 6
  53. [53]Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. V. Dillon, B. Lakshminarayanan, and J. Snoek. Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. In NeurIPS, 2019. 2arxiv.org/abs/1906.02530
  54. [54]Keivalya Pandya. pico-wam: Evaluating counterfactual prediction and planning error in a compact world action model. arXiv preprint arXiv:2611.01039, 2026. 2, 3arxiv.org/abs/2611.01039
  55. [55]J. Parker-Holder and S. Fruchter. Genie 3: A new frontier for world models. Google DeepMind blog, 2025. 2
  56. [56]D. Pathak et al. Curiosity-driven exploration by self-supervised prediction. In ICML, 2017. 2, 3arxiv.org/abs/1705.05363
  57. [57]J. Pearl. Causality. Cambridge University Press, 2 edition, 2009. 3
  58. [58]W. Peebles and S. Xie. Scalable diffusion models with transformers. In ICCV, 2023. 6arxiv.org/abs/2212.09748
  59. [59]Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv preprint arXiv:1910.00177, 2019. 3
  60. [60]Jan Peters, Katharina Mülling, and Yasemin Altun. Relative entropy policy search. In AAAI, 2010. 3DOI
  61. [61]Y. Qin et al. WorldSimBench: Towards video generation models as world simulators. arXiv:2410.18072, 2024. 1arxiv.org/abs/2410.18072
  62. [62]Julian Quevedo, Ansh Kumar Sharma, Yixiang Sun, Varad Suryavanshi, Percy Liang, and Sherry Yang. Worldgym: World model as an environment for policy evaluation. In International Conference on Learning Representations, pages 78932–78957, 2026. 2arxiv.org/abs/2506.00613
  63. [63]Paria Rashidinejad, Banghua Zhu, Cong Ma, Jiantao Jiao, and Stuart Russell. Bridging offline reinforcement learning and imitation learning: A tale of pessimism. Advances in Neural Information Processing Systems, 34:11702–11716, 2021. 2arxiv.org/abs/2103.12021
  64. [64]J. M. Robins, A. Rotnitzky, and L. P. Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89(427):846–866, 1994. 2DOI
  65. [65]Noveen Sachdeva, Yi Su, and Thorsten Joachims. Off-policy bandits with deficient support. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 965–975, 2020. 2DOI
  66. [66]Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak. Planning to explore via self-supervised world models. In International conference on machine learning, pages 8583–8592. PMLR, 2020. 3arxiv.org/abs/2005.05960
  67. [67]I. Shumailov et al. AI models collapse when trained on recursively generated data. Nature, 2024.DOI
  68. [68]P. Shyam, W. Jaśkowski, and F. Gomez. Model-based active exploration. In ICML, 2019.arxiv.org/abs/1810.12162
  69. [69]Y. Su, L. Wang, M. Santacatterina, and T. Joachims. CAB: Continuous adaptive blending for policy evaluation and learning. In ICML, 2019.
  70. [70]R. S. Sutton. Dyna, an integrated architecture for learning, planning, and reacting. SIGART Bulletin, 1991.DOI
  71. [71]A. Swaminathan and T. Joachims. Counterfactual risk minimization. In ICML, 2015.
  72. [72]Gemini Robotics Team, Krzysztof Choromanski, Coline Devin, Yilun Du, Debidatta Dwibedi, Ruiqi Gao, Abhishek Jindal, Thomas Kipf, Sean Kirmani, Isabel Leal, et al. Evaluating gemini robotics policies in a veo world simulator. arXiv preprint arXiv:2512.10675, 2025.arxiv.org/abs/2512.10675
  73. [73]Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High confidence policy improvement. In International Conference on Machine Learning, pages 2380–2388. PMLR, 2015.
  74. [74]E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In IROS, 2012.DOI
  75. [75]Masatoshi Uehara and Wen Sun. Pessimistic model-based offline reinforcement learning under partial coverage. arXiv preprint arXiv:2107.06226, 2021.arxiv.org/abs/2107.06226
  76. [76]Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.arxiv.org/abs/1812.01717
  77. [77]Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dudík. Optimal and adaptive off-policy evaluation in contextual bandits. In International Conference on Machine Learning, pages 3589–3597. PMLR, 2017.arxiv.org/abs/1612.01205
  78. [78]Yifan Wu, George Tucker, and Ofir Nachum. Behavior regularized offline reinforcement learning. arXiv preprint arXiv:1911.11361, 2019.arxiv.org/abs/1911.11361
  79. [79]C. Yang, X. Wang, J. Jiang, Q. Zhang, and X. Huang. Evaluating world models with LLM for decision making. arXiv:2411.08794, 2024. Later retitled “LLM-based world models can make decisions solely, but rigorous evaluations are needed”.
  80. [80]Jiazhi Yang, Kashyap Chitta, Shenyuan Gao, Long Chen, Yuqian Shao, Xiaosong Jia, Hongyang Li, Andreas Geiger, Xiangyu Yue, and Li Chen. Resim: Reliable world simulation for autonomous driving. Advances in Neural Information Processing Systems, 38:167710–167741, 2025.arxiv.org/abs/2506.09981
  81. [81]M. Yang et al. Learning interactive real-world simulators. In ICLR, 2024.arxiv.org/abs/2310.06114
  82. [82]Z. Yang et al. CogVideoX: Text-to-video diffusion models with an expert transformer. arXiv:2408.06072, 2024.arxiv.org/abs/2408.06072
  83. [83]T. Yu et al. MOPO: Model-based offline policy optimization. In NeurIPS, 2020.arxiv.org/abs/2005.13239
  84. [84]T. Yu et al. COMBO: Conservative offline model-based policy optimization. In NeurIPS, 2021.arxiv.org/abs/2102.08363
  85. [85]Jiahan Zhang, Muqing Jiang, Nanru Dai, Taiming Lu, Arda Uzunoglu, Shunchi Zhang, Yana Wei, Jiahao Wang, Vishal Patel, Paul Liang, et al. World-in-world: World models in a closed-loop world.arxiv.org/abs/2510.18135
  86. [86]Fangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou, Xiao Ma, and Song Guo. Wmpo: World model-based policy optimization for vision-language-action models. In International Conference on Learning Representations, pages 62486–62502, 2026.arxiv.org/abs/2511.09515
  87. [87]Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019.arxiv.org/abs/1909.08593

Paper details

vlnc:W_4venk9qm
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents