WorksComputer Science
Plausible Videos Can Lead to Poor Decisions: Diagnosing and CURE-ing World Models for Planning
Abstract
Action-conditioned world models are increasingly used as world models for planning, yet a world model that makes plausible, realistic-looking videos that are consistent with the training data can produce poor decisions. We address two key questions: (1) when can we trust a world model for planning, and (2) what should we do when we cannot? We introduce CURE (Certify, Upweight, Rollout, Explore), a certificate-guided technique that assesses whether a world model can be trusted for planning and determines what to do next: whether to upweight the training data to debias the model; roll out more samples at test time; or explore by collecting new data where support (how well the training data cover the planner's decisions) is missing. We deploy CURE across multiple domains, spanning simulated physics based on PhyWorld, MuJoCo arm control, and Language-Table pushing in simulation and on real videos. The decisions our certificate trusts have lower regret than those it does not (e.g., 0.06 vs. 0.38 on Language-Table sim and 0.07 vs. 0.38 on MuJoCo contact), whereas video perceptual quality does not identify good decisions. By focusing on decision quality rather than perceptual quality, CURE lowers regret over the best pessimistic decision rule by 10-25% across support levels, and over world-action models, search and policy learning by 41-64%, given the same data.
Introduction
Video generators are widely used as world models: condition on what the camera has seen and an action of interest, then roll out the generator to see which action to take [7, 8, 52, 81, 82]. The reasoning seems defensible: a correct world model should produce convincing video, so convincing video may provide evidence that a world model predicts the consequence of an action well. However, there is considerable evidence that this is not the case: video generators that produce such videos can still violate basic physics [4, 5, 6, 32, 49] and they do not necessarily lead to
good decisions [14, 42, 61, 79].
Figure 1 shows a concrete example of the potential disagreement between video quality and decision quality. Here, the planner needs to move the red tip on a robot arm toward the target position (the MuJoCo benchmark [74], details in § 5.1), and a world model is trained to generate a video of the robot arm’s movement under a specific action, i.e., an imagined rollout. For each candidate action, the model predicts its regret by comparing the red tip’s final position in the imagined rollout with the target position, and the planner favors the action with lower predicted regret. However, in our example, the first row shows a model generating a more convincing video whose imagined rollout leads to the wrong decision. The second row shows the same scene after applying CURE, the technique we introduce in this paper. The imagined rollout is visibly less convincing, with a blurred and ghosted arm, yet the action the planner selects from it is the best one available (regret 0). Nothing in the first row signals the problem: the video is sharp and shows the tip reaching the goal. This is not an isolated case: across decisions and domains, video quality does not identify the decisions with low regret (§ 5.2).

Figure 1. Convincing video does not mean a better action. World models make decisions by generating videos as imagined rollouts, predicting the regret from the videos, and picking the action with the lowest predicted regret. A more convincing video may still predict regret badly, and choose an action that fails in the true rollout (top). A less convincing video, despite the visible artifacts (ghosting, blurring and smearing of the arm), can predict accurately and select a better action (bottom).
Why does this happen? Prior work suggests several reasons. At test time, a planner may score actions on too few rollouts being misled by noise [33, 35], or search too few candidate actions, missing better ones [16, 23, 43, 47]. The model may also lack the capacity to learn the dynamics [25]. Finally, the training data may teach correlations rather than general laws, so the model fails to perform out of distribution [12, 32, 36]. These explanations for planning failure reduce to two quantities: model error and support (how well the training data covers the planner’s decisions). Indeed, planning error, or regret, is bounded by the product of these two quantities, plus regularization bias [31, 36, 75]. Standard world model training fits the training data and rewards visually compelling videos, but controls neither error source. As a result, a world model can produce convincing video, but plan poorly.
Existing solutions focus on different error sources. Scaling test-time compute, with more rollouts [33, 35] or more candidates [16, 23, 47], reduces rollout noise and error. However, if support is low, the model’s error cannot be fixed unless support is addressed. Debiased training and retargeting [36] lowers the model’s error where the training data has a few clips of the rarely recorded pairs, but cannot help when there is effectively no support. Collecting more data helps, but only if the new clips are collected where support is missing: more data from the same training source is limited by coverage, not its size [34, 41, 83]. Exploration that targets the model’s uncertainty [27, 56, 68] could find those gaps, but it relies on uncertainty estimates that are themselves unreliable out of distribution [51, 53].
Each solution addresses a different root cause, and sometimes none is needed. We address two fundamental questions:
(Q1) When can we trust our world model for planning?
(Q2) When we cannot trust our model for planning, what can we do about it?
Contributions. We introduce CURE (Certify, Upweight, Rollout, Explore).
When to trust a world model: Certify. We produce a theoretically justified certificate for when to trust a world model for planning. The certificate is easily computed from the training data, using estimates of the model’s error and of its support. It guarantees the planner is as good as any fixed action, up to these two quantities, connecting each error source to planning error.
What to do about it: Upweight, Rollout, Explore. We introduce CURE, a unified technique that turns the certificate into go/no-go criteria for fixing a world model until its decisions pass the certificate (Figure 2). These criteria say when to upweight the training data to debias the model, when to roll out more samples, and when, and where, to explore by collecting new data.

Figure 2. CURE. We estimate the behavior policy and the weights on the training data, and the model error on a validation set. We then train the candidate models (standard, debiased, retargeted) and certify each one (D1). If none passes, we collect new clips where the certificate fails and re-certify, until the clip budget is spent (D2). We keep the model with the largest certificate margin (D3), and spend extra rollouts only on contended actions, those too close to the top score to tell apart, that have support (D4). Full procedure in the supplementary material.
Experimental evidence across multiple domains. On simulated physics, robot arm control, and tabletop pushing in simulation and on real videos, the certificate separates good decisions from bad, whereas video quality does not (Figure 4). Given the same data, CURE lowers regret over the best pessimistic decision rule by 10–25% across support levels, and over world-action models, search and policy learning by 41–64% (Table 1).
Figure 4. The certificate predicts decision error while video quality does not. Setting: Language-Table sim. (a) Regret against the percentile of each signal (smoothed; 95% bands). LPIPS is negated, so higher means better video. (b) Regret of trusted and withheld decisions. PSNR and LPIPS trust the same share as the certificate.
| low | partial | high | |
| One fix alone (no certificate) | |||
| Standard (no fix) | |||
| Upweight only | |||
| Rollout only | |||
| Explore only | |||
| Ablation: CURE without one step | |||
| without Explore | |||
| without Upweight | |||
| without Rollout† | |||
| Baselines† | |||
| Behavior policy | |||
| World-action model | |||
| Search with a verifier | |||
| TD3+BC | |||
| AWR | |||
| BCQ | |||
| SPIBB | |||
| Count lower bound | |||
| Mean − λ·sd | |||
| Worst rollout | |||
| Ensemble lower bound‡ | |||
| CURE |
Table 1. CURE has the lowest regret at every support level. Regret (lower is better), averaged over Kikk2C, Language-Table sim and real, and MuJoCo contact, with 95% intervals. All rows share the model, data (incl. CURE’s 300 clips at low support) and evaluation. Rollout uses 16 rollouts per action; †4, as for the baselines. Gray: step not called, so equal to CURE. ‡Uses ten world models instead of one. Baselines are described in § 3.2; per-domain results are in the supplement.
Related Work
Video world models and their evaluation. Video models are increasingly used for planning by imagining each candidate action’s outcome [8, 13, 17, 19, 52, 55, 81]. Benchmarks grade the generated video itself [4, 28, 32, 42, 49], yet visual quality does not guarantee task success [85]. Policy evaluators [62, 72] and action-following benchmarks [9] judge a model with something outside its training data, such as simulators or real trials. None asks whether, from the training data alone, a model can be trusted to rank actions, or what to do when it cannot.
Policy learning with world models. Policies are learned by planning in a world model [25] or training on its rollouts [2, 24, 30, 70, 86]. Online methods collect fresh data [18, 22] or simulated outcomes of non-expert actions [54, 80] where the policy exploits model errors. Offline methods are limited by how well the training data covers the best policy [63, 75], and stay close to the data [37, 38, 84] or penalize model uncertainty [34, 83]. Uncertainty penalties trust the model to know where it is wrong, but generative models can be confident far from their data [44, 51]. We instead treat support as a property of the training data, measured with importance weights.
Debiased estimation and safe policy improvement. Doubly robust estimation corrects for the behavior policy by combining an outcome model with importance weights [10, 11, 15, 64], and recent work moves such corrections into model training [26, 45, 48]. We adopt ADIGen [36] as our debiasing step. Where weights grow large, estimators cap them and fall back on the model [69, 77], and where the behavior policy never takes an action, there is nothing to reweight [65]: debiasing repairs low-support but not absent support. Safe policy improvement accepts a new policy only as far as the training data supports it [40, 71, 73]. CURE brings this test to world models, and collects data where it fails.
Imagination and data collection. Retraining a generator on its own rollouts adds no evidence and risks model collapse [1, 67], so missing support must be filled with executed clips. Active learning and exploration collect where the model is uncertain [27, 56, 66, 68], which misleads when a generator is confident without data. Our collection rule targets missing support instead.
Setup and Background
Problem
The objective is to learn a world model from training data, and use the world model for planning. Our data are i.i.d. clips , comprised of a context or the conditioning frames from a video, an action , and an outcome video with utility , under a behavior policy . A world model is a conditional video model: it takes a scene and an action as input to produce a generated video, approximating . We assume standard identifiability conditions [57] so that this equals the interventional law , where is the video that would result from setting . Planning chooses one action, rolled out over the whole video.
A planner is a map from scenes to action distributions over an action set . With the expected utility of action and the best action, we minimize regret:
A summary of the notation used throughout the paper is provided in the Supplement.
Planners
We plan with an estimated utility function derived from rollouts under the estimated world model , and with , a probability distribution over that estimates . The primary planner we use is KL-regularized, so that decisions poorly supported by our data are penalized . yields the greedy choice , while gives the fitted behavior policy . KL regularization is widely used [29, 41, 50, 59, 60, 78, 87]. We additionally evaluate greedy, advantage-weighted [59], behavior-cloning-regularized [20], batch-constrained [21] and baseline-bootstrapped [40] planners. We also compare against the behavior policy’s default action, search with a verifier, the rules that world-action models use [13, 54], and pessimistic rules that score each action by a lower bound on its utility (the standard remedy for limited support [31, 34, 39, 83]).
Three Solutions for Building Decision-Ready World Models
When a world model plans poorly, three solutions are available — each targets a different root cause of planning error:
Upweight: Debiased training and retargeting. A standard world model fits the training data as is, so it learns the scene–action pairs it sees often well and extrapolates poorly under low-support. The debiased loss of ADIGen [36] corrects this with a weight estimating the density ratio between a uniform interventional policy over and the behavior policy . Rather than divide by an estimate of , which is unstable, the weight is ‘automatically’ learned by directly fitting a Riesz regression [10] on the training data , and is capped at for stability — the full loss is restated in the supplementary material. Retargeting [36] then continues training with replaced by the planner’s own action distribution, so that the model is most accurate where the planner acts.
Rollout: Scaling test-time compute. At decision time, test-time compute can be scaled in two ways: scoring more candidate actions, or sampling more rollouts per candidate. For the first, draw candidates from , roll out each, and choose the one with the highest [16, 25, 47]. This can improve planning, but a larger makes model over-optimism more likely, especially on actions the training data rarely contains. For the second, more rollouts per candidate reduce the variance of but not its bias. Both help only when the model is accurate on the actions being compared.
Explore: Data collection. The alternative is to execute an action, add the clip to the data, and retrain the world model. Given a budget of clips, a collection rule maps the current model, training data, and pool of scenes to scene–action pairs to execute, giving a dataset to augment with, .
Cure: When to Trust a World Model, and What to Do About It
Each solution of § 3.3 targets a different root cause, and sometimes none is needed. We first derive a certificate for assessing whether a world model can be trusted for planning, answering Q1, and then introduce CURE, a certificate-guided framework that determines what to do next, answering Q2.
The Certificate
Throughout, for any that acts on clips and actions, we let . We require two pieces: a weight measuring support, and an estimate of model error. First, for a planner over , the weight $\alpha_\pi(x,a) = \pi(a \mid is the density ratio between and the behavior policy, measuring how far is from the behavior policy. Second, the model’s error for planning is , where and centers over the actions in . Together, they bound the regret of a planner built from a world model:
Proposition 1 (Restated from Kim et al. [36]). For any planner over , let be the optimal planner that always takes , and the model’s predicted utility of . Then,
The first term combines the model’s centered utility error on the training distribution, , with the difference between the optimal and learned planners relative to the behavior policy. This term can be large when the model is inaccurate or when the planners differ on actions poorly supported by the training data. The second term measures how much predicted utility we lose by following the learned planner instead of the optimal action. For our KL-regularized planner, this can reflect regularization bias.
From the regret bound to a certificate
A small bound in Proposition 1 would support trusting the planner, but the bound depends on the optimal policy , which is unknown. We therefore apply the same reasoning to each fixed action : how much do we lose by following our planner instead of always taking ? Equivalently, we lower-bound the improvement: how much might our planner improve over a constant-action planner ?
To answer this, we substitute estimates into Proposition 1: the weights follow from the fitted behavior policy , and is estimated on validation clips: a cross-fitted regression [11] of the residual on scene and action gives a bias , which we center over . This gives the certificate for action :
The first term is how much the model predicts our planner will improve over . The second accounts for how much model error and limited support could change this comparison. If , the estimated bound favors our planner over , even after accounting for model error and limited support. If this holds for every up to a tolerance , that is, , we certify our world model: no fixed action is estimated to outperform our planner by more than . This certification holds with high probability (Supplement).
Per decision. For a single scene , dropping the expectation gives with the norm over actions at . If for every covered (), the decision has regret at most (Supplement); Sec. 5.2 uses this.
When the certificate fails, the pieces indicate why: the model’s error is too large, the planner strays too far from the support of the training data, or the predicted gain is too small. Each root cause calls for a different fix, which CURE makes precise.
From Certificate to Actions
We summarize our pipeline in Figure 2, read from top to bottom. We use three candidate world models of Sec. 3.3 (standard, debiased, and debiased + retargeted) for planning. We first check whether any world model can be certified (D1). If none is certified, we check whether low support calls for collecting new clips (D2). Otherwise, we select the certified model with the largest margin (D3). Finally, for each scene, we decide whether to act, use more rollouts, or collect new clips (D4).
D1 (Certify): Certifying a world model. Score every candidate world model. A world model is certified when for every : no fixed action can beat the planner by more than , even after accounting for the model error and support. If no candidate is certified, the training data cannot support the decision.
Rule: if for every and clip budget remains, go to D2; otherwise go to D3.
D2 (Explore): Collect clips, , where the certificate fails. Low support may explain the failure in D1: when the planners differ on poorly supported actions, can be large, so the predicted improvement may not be enough to account for model error and limited support. In D2, we propose to: (1) Check low support: Identify actions with ; we denote this set of actions by (defined in the Supplement). (2) Check predicted utility: Check whether for these actions, i.e., the model predicts a gain over that the training data cannot certify. (3) Calculate the deficit: An action leaves once its weight falls back under , which for the uniform target means . The deficit counts how many of the training clips are missing to reach that level,
so it is zero for supported actions and largest, about clips, for actions the behavior policy never took. (4) Collect and retrain: If the check in (2) passes, draw pairs , , in proportion to , execute each, and retrain with .
Rule: If some has , the model predicts a gain over a low-support action that the training data cannot certify: collect there, and return to D1.
D3 (Upweight): Among the certified , , and , which should we choose? They all share the same architecture, training data, training procedure.
Rule: Among the certified world models , keep the one with the largest certified margin, that is,
This rule says whether the training data’s support suffices (), whether low-support can be corrected without more data (), and whether training should follow the planner to where it acts ().
D4 (Rollout): Test-time compute? With selected in D3, we now ask whether the model can clearly distinguish between actions for a given scene . For a scene with candidate actions, each scored by averaging rollouts, and preferred action , the contended actions are those the model cannot separate from within its model error, . In D4 we therefore propose to: (1) Check contended actions: If , the preferred action can be clearly distinguished from other actions within the model error, and therefore, we can select the preferred action. (2) Check low support: If , the contended actions include actions poorly supported by the training data, so we need to return to D2 to collect more data in support for them. (3) Rollout: Otherwise, double the rollouts on the contended actions () and the number of candidate actions (, adding only supported actions), recompute , and repeat until (1) or (2) applies or the compute budget is spent; if still undecided, return to D2.
Rule: Double the rollouts per contended action () and the number of candidate actions (, adding only actions with support), then recompute . Repeat until one action remains or the compute budget is spent. If still undecided, send to D2.
Examples. Figure 3 walks through CURE on two test scenes.
Figure 3. CURE in action. Top: MuJoCo contact. Bottom: Language-Table sim. Left: the standard world model’s chosen action misses the goal in the real rollout (true outcome shown). Middle: CURE’s certificate decides the fix. Top: the certificate is below zero because the training data has too little support for the needed action, so CURE collects new clips (Explore). Bottom: the certificate passes, so CURE acts, with the upweighted model and more rollouts per action (Upweight, Rollout). Right: CURE’s chosen action reaches the goal. The regret numbers are normalized such that 0 is the best action in the bank, and 1 is a random action on average.
Experiments
We test each step of CURE: Certify (§ 5.2), Explore (Sec. 5.3), Upweight (Sec. 5.4) and Rollout (Sec. 5.5). In § 5.6 we run our experiments where we follow all four calls of CURE, and compare CURE with other families.
Setup
Domains. We test CURE in settings commonly used to evaluate physical prediction and control. kick2c [36] is a simulated physics task closely related to PHYRE [3] and PhyWorld [32], varied to control how far decisions fall out of distribution: a 2D ball is kicked toward a goal line while a gust, varying from scene to scene, pushes it sideways. MuJoCo [74], a standard benchmark for model-based control, is a planar arm in two variants: (i) in contact, the arm chooses a push to move a block onto a goal spot, a contact-rich setting where outcomes change abruptly with the action; (ii) in smooth, the same arm reaches for a goal point with no block, serving as a control where outcomes vary smoothly. Language-Table [46], widely used to train and evaluate video world models [81], is a tabletop task in which a 2D push moves a named block toward a target, in simulation (sim) and on real robot video (real). We simplify by focusing on the continuous action without language. On real video untaken actions cannot be executed, so their outcomes are scored by a separate model. Figures in this section focus on Language-Table sim; Table 1 summarizes results over different domains with further details in the supplement.
Support. To simulate training data with low-support, we control how often the behavior policy takes the actions the test scenes need. In each task the behavior policy has a default action for the common case, such as the usual kick under the common gust. A fraction of its clips instead takes other actions inducing low-support. We control support to separate the failure modes it causes, and test the findings in both simulated and real-robot data. We call high support, partial support and low support.
Models. The primary world model is a 28-million-parameter latent video Diffusion Transformer (DiT) [58], the architecture behind today’s leading video generators and world models [7, 52, 82]. We call this DiT, trained on the training data with the unweighted loss, the standard model; every comparator that needs a world model uses it, so differences between methods reflect how they decide, not which model they use. We use a compact model because each result is repeated over 20 seeds, three support levels and three training losses, with collection and retraining on top; we check that the findings hold at other sizes (7M and 115M) and on other backbones as detailed in the supplement. We compare against world-action models and policy learning on world models (§ 5.6).
Evaluation. Decisions use the planner of § 3.2, and regret is normalized so that a random action scores 1 and the best action in the bank 0. Each step of CURE is evaluated with the training data, test set, seeds and planner held fixed. For Certify, we test whether the certificate of each decision predicts its regret, and compare it with video quality scores. For Upweight, we compare the regret of each world model
across support levels. For Rollout, we compare regret as the number of candidate actions scored grows, at each support level. For Explore, we give each collection rule the same budget of new clips, retrain the model, and compare the resulting regret. Implementation details, per-domain choices, and results are in the supplementary material.
Certify
We test whether the certificate from Eq. (1) separates good decisions from bad ones, and whether video quality does too. In Figure 4a, we sort decisions by each signal (certificate Cert, PSNR, LPIPS), from least to most trusted (low to high Cert and PSNR, high to low LPIPS). We then plot the mean regret of decisions at each percentile of each signal. We find that regret falls steadily as the certificate rises (Spearman , 95% interval ), meaning a higher certificate reliably identifies a better decision. Video quality does not: among the decisions of any one trained model, PSNR is unrelated to regret (), while the certificate’s correlation is (negative for every model).
In Figure 4b, we use each signal to plan. Decisions that pass the certificate are almost always good: they make up 22% of decisions, with regret 0.06, compared to 0.38 for the decisions it does not certify. If these decisions were made based on the equivalent top 22% of videos according to PSNR, those decisions are worse than the rest, with regret 0.49 against 0.26.
Across domains1, the certificate finds where the training data lacks support and ranks world models by regret, which video quality does not. The certificate also separates good decisions from bad: those it certifies have lower regret than those it withholds. Across whole models, Fréchet Video Distance (FVD) [76] tells a similar story: on held-out clips of logged actions, averaged over support levels, the standard model’s videos are more realistic than CURE’s (FVD 22 vs. 80), but regret is 3.1× higher.
Explore
How to collect. We compare collection strategies at low support with a fixed budget of 300 new clips. CURE collects where support is missing and more than halves regret (0.73 to 0.29). Here clips are allocated across support levels, but any grouping of scenes, such as by instruction or wind condition, can be used. Rules that follow the behavior policy do worse: more clips from the same behavior policy give 0.38, and clips sampled from the planner with , which stays close to the behavior policy, give 0.41, against 0.30 for CURE’s rule in this comparison. Rules that follow the model also do worse: collecting the action it ranks best in each scene gives 0.34, and collecting where it is most uncertain gives 0.38. Collecting uniformly over actions does well here (0.31), but only because at low support almost every action needs data. Uniform collection does not know where support is missing, which matters once the budget must be split across support levels as shown below.
Sample efficiency. CURE spends its clip budget where the certificate is weakest first, in increasing order of the mean certificate across support levels. We compare it with splitting the budget evenly across support levels (even split), at equal total clips. The even split is less efficient than CURE: with 300 clips its regret is higher by 0.020 (paired 95% interval [0.008, 0.032]), and it needs 900 clips to reach what CURE reaches with a budget of 600 (0.13).
The same story holds across domains: collecting where support is missing lowers regret at low support in every domain except MuJoCo smooth, the control with nothing to fix, and spreading the same clips evenly across support levels is worse than CURE.
Upweight
We compare the standard, debiased, and debiased with retargeting (just called retargeted) models across support levels on Language-Table sim. At high support, every model is already near zero regret — weights are stable, and there are enough demonstrations to plan well. At partial support, debiasing lowers regret by about a third (from 0.25 to 0.17), and retargeting lowers it further (0.14). At low support, every model fails equally (0.72 to 0.73), since no loss can recover actions the training data never shows; this is where new data is needed (see Sec. 5.3). Removing the Upweight step from CURE confirms this (see Table 1): averaged over domains, regret at partial support rises from 0.18 to 0.24, and from 0.11 to 0.12 at high support.
Two findings from the other domains qualify this picture. Low support is not always hopeless: at equal training data, upweighting lowers regret at low support by 0.23 on kick2c and by 0.13 on Language-Table real. Also, debiasing is not always the right call. In particular, when the weights add variance with little reduction in bias, CURE keeps the standard model, using a check on the validation set. For example, on MuJoCo contact debiasing raises regret at partial and high support (Supplement), and CURE selects the standard model.
Rollout
Test-time compute helps only where the model has support, and the certificate tells CURE where that is (Figure 5). At high support, all methods are competitive. At partial support, scoring 128 candidate actions instead of 16 halves the regret of the retargeted model (0.39 to 0.18) and helps the standard model less (0.45 to 0.30). At low support, regret stops improving after 16 candidates (0.75 to 0.77 for the standard model), since the issue is not variance, it is bias.
Figure 5. Test-time compute helps only where the model has support. Regret against number of candidates scored . Hollow: low; filled: partial; dotted: high support.
Averaging more rollouts per action behaves the same way: going from 1 to 16 lowers the retargeted model’s regret at partial support from 0.17 to 0.12, but leaves low-support regret above 0.68. The certificate separates these cases: it passes 4% of decisions at low support, against 25% at partial and 37% at high support. CURE therefore spends compute where the certificate finds support, and collects new clips where it does not.
Cure as a strategy
Across five domains, the certificate predicts decision error where video quality does not; collecting where support is missing is the main fix at low support, debiasing at partial support, and extra test-time compute helps only once support exists. Choosing among these fixes by the certificate lowers regret over standard world models by 31–61% at low support and up to 50% at partial support, without raising it at high support. Against baselines from every family, given the same data and compute, CURE has the lowest or tied-lowest regret in every domain. We evaluate CURE holistically, first across differing support levels, and in comparison to competing baselines mentioned in § 3.2. Recall that low, partial and high support mean that a fraction of 0.5%, 5% and 25% of the behavior policy’s clips depart from its default action.
CURE across support levels. Table 1 compares CURE with each fix applied alone at every support level, with CURE missing one step, and with baselines from each family. At low support, only new clips help; the certificate fails, and CURE collects. At partial support, upweighting and averaging more rollouts help; the certificate passes, and CURE acts with both. At high support, the model is already close to correct, and extra rollouts trim what remains. CURE has the lowest regret at every support level: 0.39, 0.18 and 0.11, against 0.69, 0.26 and 0.13 for the standard model and at least 0.44, 0.24 and 0.12 for any single fix. Removing one step at a time shows where each matters: without Explore, regret at low support rises to 0.60; without Upweight, regret at partial support rises to 0.24; and without Rollout, regret at partial and high support rises to 0.19 and 0.12. Where the certificate does not call for a step, removing it changes nothing. Upweight and Rollout help more together, since extra rollouts reduce variance but not bias.
CURE against baselines. Given the same data, CURE’s regret is below that of world-action models, search with a verifier and policy learning in the world model at every support level. The strongest baselines are decision rules that score actions pessimistically, by a lower bound or the worst imagined rollout. CURE’s regret is lower than all of them at every support level, by 16–36% for the single-model rules. An ensemble lower bound, which runs ten world models, comes closest in regret (0.428, 0.236 and 0.127 against CURE’s 0.385, 0.178 and 0.107). They act on the decision rule rather than the model or data, so they are complementary to CURE.
CURE’s regret gains, largest at low support, hold beyond our 28M model: at 115M, 340M, on DIAMOND [2], and on the pretrained CogVideoX-2B [82]. On CogVideoX-2B the certificate still tracks decision quality: its rank correlation with regret is −0.38 within each support level and seed, and certified decisions have regret 0.016 against 0.245 for the rest (Supplement).
Conclusions
World model visual quality says little about how well it can plan. While several solutions exist to close this gap, there is little guidance on which solution to use. We introduce CURE, which certifies when a world model can be trusted for planning. When it cannot, CURE recommends when to upweight the training data, when to roll out more at test time, and when and where to explore. As world models are increasingly used for planning, CURE provides a certificate-guided framework for assessing when an existing model can be trusted and deciding how to allocate limited data and compute where they matter.
LLM Usage Statement
We used Claude (Anthropic) as an assistant throughout the preparation of this paper. Writing: it drafted, edited and shortened text in all sections of the main paper and the supplement, from the authors’ outlines, notes and results; the authors revised and approved all text. Figures: it wrote plotting code for the schematic figures Figure 2 and for the layout and styling of the result figures; all plotted values come from the authors’ experiment outputs. Analysis and checking: it helped draft experiment specifications and analysis plans, checked the consistency of reported numbers against result tables, and reviewed the paper for errors. The authors take full responsibility for the content of the paper.
References
- [1]S. Alemohammad et al. Self-consuming generative models go MAD. In ICLR, 2024. 3arxiv.org/abs/2307.01850
- [2]E. Alonso, A. Jelley, V. Micheli, A. Kanervisto, A. Storkey, T. Pearce, and F. Fleuret. Diffusion for world modeling: Visual details matter in Atari. In NeurIPS, 2024. 2, 8arxiv.org/abs/2405.12399
- [3]Anton Bakhtin, Laurens van der Maaten, Justin Johnson, Laura Gustafson, and Ross Girshick. PHYRE: A new benchmark for physical reasoning. In Advances in Neural Information Processing Systems (NeurIPS), 2019. 5arxiv.org/abs/1908.05656
- [4]H. Bansal, C. Peng, Y. Bitton, R. Goldenberg, A. Grover, and K.-W. Chang. VideoPhy-2: A challenging action-centric physical commonsense evaluation in video generation. arXiv:2503.06800, 2025. 1, 2arxiv.org/abs/2503.06800
- [5]H. Bansal et al. VideoPhy: Evaluating physical commonsense for video generation. In ICLR, 2025.arxiv.org/abs/2406.03520
- [6]D. M. Bear et al. Physion: Evaluating physical prediction from vision in humans and machines. In NeurIPS Datasets and Benchmarks, 2021. 1arxiv.org/abs/2106.08261
- [7]T. Brooks et al. Video generation models as world simulators. OpenAI technical report, 2024. 1, 6
- [8]J. Bruce et al. Genie: Generative interactive environments. In ICML, 2024. 1, 2arxiv.org/abs/2402.15391
- [9]Sixiang Chen, Jiaming Liu, Jixian Wu, Yichen Guo, Tinghao Wang, Siyuan Qian, Hao Chen, Jiajun Cao, Jian Tang, and Shanghang Zhang. Do robotic world models really follow actions? diagnosing and aligning action-conditioned generation for policy learning. arXiv preprint arXiv:2608.24885, 2026. 2arxiv.org/abs/2608.24885
- [10]V. Chernozhukov, W. Newey, and R. Singh. Automatic debiased machine learning of causal and structural effects. Econometrica, 2022. 2, 3arxiv.org/abs/1809.05224
- [11]V. Chernozhukov et al. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 2018. 2, 4
- [12]P. de Haan, D. Jayaraman, and S. Levine. Causal confusion in imitation learning. In NeurIPS, 2019. 2arxiv.org/abs/1905.11979
- [13]Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. Advances in neural information processing systems, 36:9156–9172, 2023. 2, 3arxiv.org/abs/2302.00111
- [14]H. Duan, H.-X. Yu, S. Chen, L. Fei-Fei, and J. Wu. World-Score: A unified evaluation benchmark for world generation. In ICCV, 2025. 1
- [15]M. Dudík, J. Langford, and L. Li. Doubly robust policy evaluation and learning. In ICML, 2011. 2arxiv.org/abs/1103.4601
- [16]F. Ebert, C. Finn, A. X. Lee, and S. Levine. Self-supervised visual planning with temporal skip connections. In CoRL, 2017. 2, 3arxiv.org/abs/1710.05268
- [17]Frederik Ebert, Chelsea Finn, Sudeep Dasari, Annie Xie, Alex Lee, and Sergey Levine. Visual foresight: Model-based deep reinforcement learning for vision-based robotic control. arXiv preprint arXiv:1812.00568, 2018. 2arxiv.org/abs/1812.00568
- [18]Yunhai Feng, Nicklas Hansen, Ziyan Xiong, Chandramouli Rajagopalan, and Xiaolong Wang. Finetuning offline world models in the real world. arXiv preprint arXiv:2310.16029, 2023. 2arxiv.org/abs/2310.16029
- [19]Chelsea Finn and Sergey Levine. Deep visual foresight for planning robot motion. In 2017 IEEE international conference on robotics and automation (ICRA), pages 2786–2793. IEEE, 2017. 2arxiv.org/abs/1610.00696
- [20]Scott Fujimoto and Shixiang Shane Gu. A minimalist approach to offline reinforcement learning. In NeurIPS, 2021. 3arxiv.org/abs/2106.06860
- [21]Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In ICML, 2019. 3arxiv.org/abs/1812.02900
- [22]Yanjiang Guo, Tony Lee, Lucy Xiaoyang Shi, Jianyu Chen, Percy Liang, and Chelsea Finn. Vlaw: Iterative co-improvement of vision-language-action policy and world model. arXiv preprint arXiv:2602.12063, 2026. 2arxiv.org/abs/2602.12063
- [23]D. Hafner, T. Lillicrap, I. Fischer, R. Villegas, D. Ha, H. Lee, and J. Davidson. Learning latent dynamics for planning from pixels. In ICML, 2019. 2arxiv.org/abs/1811.04551
- [24]D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse domains through world models. Nature, 2025. 2
- [25]N. Hansen, H. Su, and X. Wang. TD-MPC2: Scalable, robust world models for continuous control. In ICLR, 2024. 2, 3arxiv.org/abs/2310.16828
- [26]Toru Hishimuna and Kei Senda. Weighted model estimation for offline model-based reinforcement learning. Advances in neural information processing systems, 34:17789–17800, 2021. 2
- [27]N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel. Bayesian active learning for classification and preference learning. arXiv:1112.5745, 2011. 2, 3arxiv.org/abs/1112.5745
- [28]Z. Huang et al. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024. 2arxiv.org/abs/2311.17982
- [29]Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In ICML, 2022. 3arxiv.org/abs/2205.09991
- [30]M. Janner et al. When to trust your model: Model-based policy optimization. In NeurIPS, 2019. 2arxiv.org/abs/1906.08253
- [31]Ying Jin, Zhuoran Yang, and Zhaoran Wang. Is pessimism provably efficient for offline RL? In International Conference on Machine Learning (ICML), 2021. 2, 3arxiv.org/abs/2012.15085
- [32]B. Kang et al. How far is video generation from world model: A physical law perspective. arXiv:2411.02385, 2024. 1, 2, 5arxiv.org/abs/2411.02385
- [33]M. Kearns, Y. Mansour, and A. Y. Ng. A sparse sampling algorithm for near-optimal planning in large Markov decision processes. Machine Learning, 49(2–3):193–208, 2002. 2
- [34]R. Kidambi, A. Rajeswaran, P. Netrapalli, and T. Joachims. MOReL: Model-based offline reinforcement learning. In NeurIPS, 2020. 2, 3arxiv.org/abs/2005.05951
- [35]M. J. Kim, Y. Gao, T.-Y. Lin, Y.-C. Lin, Y. Ge, G. Lam, P. Liang, S. Song, M.-Y. Liu, C. Finn, and J. Gu. Cosmos Policy: Fine-tuning video models for visuomotor control and planning. arXiv:2601.16163, 2026. 2arxiv.org/abs/2601.16163
- [36]Raphael C. Kim, Jingzen Zhu, Ramin Zabih, and Michele Santacatterina. Automatic, debiased, and invariant counterfactual generation under general interventions. arXiv preprint arXiv:2606.07399, 2026. 2, 3, 4, 5
- [37]Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. arXiv preprint arXiv:2110.06169, 2021. 2arxiv.org/abs/2110.06169
- [38]Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. Advances in neural information processing systems, 33:1179–1191, 2020. 2arxiv.org/abs/2006.04779
- [39]Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017. 3arxiv.org/abs/1612.01474
- [40]R. Laroche, P. Trichelair, and R. Tachet des Combes. Safe policy improvement with baseline bootstrapping. In ICML, 2019. 2, 3
- [41]S. Levine, A. Kumar, G. Tucker, and J. Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv:2005.01643, 2020. 2, 3arxiv.org/abs/2005.01643
- [42]D. Li et al. WorldModelBench: Judging video generation models as world models. arXiv:2502.20694, 2025. 1, 2arxiv.org/abs/2502.20694
- [43]F. Liu, H. Wang, Y. Cai, K. Zhang, X. Zhan, and Y. Duan. Video-T1: Test-time scaling for video generation. In ICCV, 2025. 2arxiv.org/abs/2503.18942
- [44]Cong Lu, Philip J Ball, Jack Parker-Holder, Michael A Osborne, and Stephen J Roberts. Revisiting design choices in offline model-based reinforcement learning. arXiv preprint arXiv:2110.04135, 2021. 2arxiv.org/abs/2110.04135
- [45]Alex Luedtke and Kenji Fukumizu. Doublegen: Debiased generative modeling of counterfactuals. arXiv preprint arXiv:2509.16842, 2025. 2arxiv.org/abs/2509.16842
- [46]C. Lynch et al. Interactive language: Talking to robots in real time. IEEE Robotics and Automation Letters, 2023. 5arxiv.org/abs/2210.06407
- [47]N. Ma, S. Tong, H. Jia, H. Hu, Y.-C. Su, M. Zhang, X. Yang, Y. Li, T. Jaakkola, X. Jia, and S. Xie. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv:2501.09732, 2025. 2, 3arxiv.org/abs/2501.09732
- [48]Valentyn Melnychuk and Stefan Feuerriegel. Gdr-learners: Orthogonal learning of generative models for potential outcomes. In International Conference on Learning Representations, pages 105267–105301, 2026. 2
- [49]S. Motamed, L. Culp, K. Swersky, P. Jaini, and R. Geirhos. Do generative video models understand physical principles? arXiv:2501.09038, 2025. 1, 2arxiv.org/abs/2501.09038
- [50]Ashvin Nair, Abhishek Gupta, Murtaza Dalal, and Sergey Levine. AWAC: Accelerating online reinforcement learning with offline datasets. arXiv preprint arXiv:2006.09359, 2020. 3arxiv.org/abs/2006.09359
- [51]Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilian Gorur, and Balaji Lakshminarayanan. Do deep generative models know what they don’t know? arXiv preprint arXiv:1810.09136, 2018. 2
- [52]NVIDIA. Cosmos world foundation model platform for physical AI. arXiv:2501.03575, 2025. 1, 2, 6
- [53]Y. Ovadia, E. Fertig, J. Ren, Z. Nado, D. Sculley, S. Nowozin, J. V. Dillon, B. Lakshminarayanan, and J. Snoek. Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. In NeurIPS, 2019. 2arxiv.org/abs/1906.02530
- [54]Keivalya Pandya. pico-wam: Evaluating counterfactual prediction and planning error in a compact world action model. arXiv preprint arXiv:2611.01039, 2026. 2, 3arxiv.org/abs/2611.01039
- [55]J. Parker-Holder and S. Fruchter. Genie 3: A new frontier for world models. Google DeepMind blog, 2025. 2
- [56]D. Pathak et al. Curiosity-driven exploration by self-supervised prediction. In ICML, 2017. 2, 3arxiv.org/abs/1705.05363
- [57]J. Pearl. Causality. Cambridge University Press, 2 edition, 2009. 3
- [58]W. Peebles and S. Xie. Scalable diffusion models with transformers. In ICCV, 2023. 6arxiv.org/abs/2212.09748
- [59]Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv preprint arXiv:1910.00177, 2019. 3
- [60]Jan Peters, Katharina Mülling, and Yasemin Altun. Relative entropy policy search. In AAAI, 2010. 3DOI
- [61]Y. Qin et al. WorldSimBench: Towards video generation models as world simulators. arXiv:2410.18072, 2024. 1arxiv.org/abs/2410.18072
- [62]Julian Quevedo, Ansh Kumar Sharma, Yixiang Sun, Varad Suryavanshi, Percy Liang, and Sherry Yang. Worldgym: World model as an environment for policy evaluation. In International Conference on Learning Representations, pages 78932–78957, 2026. 2arxiv.org/abs/2506.00613
- [63]Paria Rashidinejad, Banghua Zhu, Cong Ma, Jiantao Jiao, and Stuart Russell. Bridging offline reinforcement learning and imitation learning: A tale of pessimism. Advances in Neural Information Processing Systems, 34:11702–11716, 2021. 2arxiv.org/abs/2103.12021
- [64]J. M. Robins, A. Rotnitzky, and L. P. Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89(427):846–866, 1994. 2DOI
- [65]Noveen Sachdeva, Yi Su, and Thorsten Joachims. Off-policy bandits with deficient support. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 965–975, 2020. 2DOI
- [66]Ramanan Sekar, Oleh Rybkin, Kostas Daniilidis, Pieter Abbeel, Danijar Hafner, and Deepak Pathak. Planning to explore via self-supervised world models. In International conference on machine learning, pages 8583–8592. PMLR, 2020. 3arxiv.org/abs/2005.05960
- [67]I. Shumailov et al. AI models collapse when trained on recursively generated data. Nature, 2024.DOI
- [68]P. Shyam, W. Jaśkowski, and F. Gomez. Model-based active exploration. In ICML, 2019.arxiv.org/abs/1810.12162
- [69]Y. Su, L. Wang, M. Santacatterina, and T. Joachims. CAB: Continuous adaptive blending for policy evaluation and learning. In ICML, 2019.
- [70]R. S. Sutton. Dyna, an integrated architecture for learning, planning, and reacting. SIGART Bulletin, 1991.DOI
- [71]A. Swaminathan and T. Joachims. Counterfactual risk minimization. In ICML, 2015.
- [72]Gemini Robotics Team, Krzysztof Choromanski, Coline Devin, Yilun Du, Debidatta Dwibedi, Ruiqi Gao, Abhishek Jindal, Thomas Kipf, Sean Kirmani, Isabel Leal, et al. Evaluating gemini robotics policies in a veo world simulator. arXiv preprint arXiv:2512.10675, 2025.arxiv.org/abs/2512.10675
- [73]Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High confidence policy improvement. In International Conference on Machine Learning, pages 2380–2388. PMLR, 2015.
- [74]E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In IROS, 2012.DOI
- [75]Masatoshi Uehara and Wen Sun. Pessimistic model-based offline reinforcement learning under partial coverage. arXiv preprint arXiv:2107.06226, 2021.arxiv.org/abs/2107.06226
- [76]Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018.arxiv.org/abs/1812.01717
- [77]Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dudík. Optimal and adaptive off-policy evaluation in contextual bandits. In International Conference on Machine Learning, pages 3589–3597. PMLR, 2017.arxiv.org/abs/1612.01205
- [78]Yifan Wu, George Tucker, and Ofir Nachum. Behavior regularized offline reinforcement learning. arXiv preprint arXiv:1911.11361, 2019.arxiv.org/abs/1911.11361
- [79]C. Yang, X. Wang, J. Jiang, Q. Zhang, and X. Huang. Evaluating world models with LLM for decision making. arXiv:2411.08794, 2024. Later retitled “LLM-based world models can make decisions solely, but rigorous evaluations are needed”.
- [80]Jiazhi Yang, Kashyap Chitta, Shenyuan Gao, Long Chen, Yuqian Shao, Xiaosong Jia, Hongyang Li, Andreas Geiger, Xiangyu Yue, and Li Chen. Resim: Reliable world simulation for autonomous driving. Advances in Neural Information Processing Systems, 38:167710–167741, 2025.arxiv.org/abs/2506.09981
- [81]M. Yang et al. Learning interactive real-world simulators. In ICLR, 2024.arxiv.org/abs/2310.06114
- [82]Z. Yang et al. CogVideoX: Text-to-video diffusion models with an expert transformer. arXiv:2408.06072, 2024.arxiv.org/abs/2408.06072
- [83]T. Yu et al. MOPO: Model-based offline policy optimization. In NeurIPS, 2020.arxiv.org/abs/2005.13239
- [84]T. Yu et al. COMBO: Conservative offline model-based policy optimization. In NeurIPS, 2021.arxiv.org/abs/2102.08363
- [85]Jiahan Zhang, Muqing Jiang, Nanru Dai, Taiming Lu, Arda Uzunoglu, Shunchi Zhang, Yana Wei, Jiahao Wang, Vishal Patel, Paul Liang, et al. World-in-world: World models in a closed-loop world.arxiv.org/abs/2510.18135
- [86]Fangqi Zhu, Zhengyang Yan, Zicong Hong, Quanxin Shou, Xiao Ma, and Song Guo. Wmpo: World model-based policy optimization for vision-language-action models. In International Conference on Learning Representations, pages 62486–62502, 2026.arxiv.org/abs/2511.09515
- [87]Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019.arxiv.org/abs/1909.08593


