betaFeedback

WorksComputer Science

Hierarchical World Models with Implicit Dynamics

Abstract

Planning with world models enables test-time adaptation for embodied agents, offering superior generalization over direct policy learning. However, long-horizon tasks remain hindered by cumulative errors and search complexity. Existing hierarchical methods often use rigid macro-actions or latent skills, leading to brittle representations and limited fine-grained control. We introduce ID-WM (Implicit Dynamics World Models), a hierarchical world model that decouples state-space reachability from local control. Our framework utilizes a high-level model to sample feasible transitions across the state manifold and a low-level world model as an inverse dynamics solver to ground these transitions into precise actions. This allows the planner to identify reachable subgoals without being constrained by low-level execution details. By prioritizing local physical feasibility over global behavioral patterns, ID-WM synthesizes novel long-horizon plans from short segments, transcending the specific behavioral modes seen during training. We demonstrate its effectiveness across three navigation and manipulation suites, achieving a 38.5% improvement over state-of-the-art policy and diffusion-based planners. Crucially, by leveraging this hierarchical structure to mitigate compounding errors, our approach yields a 135% performance gain over single-level world model MPC, particularly in complex, unseen configurations for long-horizon tasks. Videos can be viewed at https://id-wm.github.io

Introduction

World models (WM)  [65, 1, 2] have emerged as a powerful paradigm for embodied decision-making [61, 14, 60, 36]. By learning predictive models of environment dynamics, they enable test-time reasoning and the synthesis of novel behaviors for unseen goals by searching in the action space [4, 17, 5, 6]. However, planning with WMs faces two fundamental challenges: 1) search complexity grows exponentially with planning horizon, and 2) prediction errors accumulate over long rollouts. Therefore, most WM-based planners are restricted to short-horizon tasks [4, 17], rely on simplified action abstractions [5], or operate in high-level spaces such as language, sacrificing fine-grained control [?].

Hierarchical learning offers a principled solution to this scalability challenge by decomposing long-horizon problems into manageable abstractions [23, 54, 57]. It has been extensively studied in Reinforcement Learning (RL) [28, 51, 7, 55], where hierarchy mitigates the credit assignment problem in long trajectories, and in imitation learning [58, 53], where it enables reusable behaviors. More recently, hierarchical structure has been introduced into diffusion-based planners to avoid directly modeling entire trajectories [46, 18, 47], improving scalability and generalization beyond fixed-horizon training data.

How can we enable the same capabilities for task-agnostic world models, enabling long-horizon zero-shot planning for unseen goals? A direct approach would be to simply increase the prediction horizon or action abstraction (e.g., frame-skipping) [66]. However, previous attempts at hierarchical world models typically depend on task-specific priors to function effectively, relying on pre-trained skills and low-level policies [52, 55] or rewards and value functions to verify subgoal feasibility [63, 50]. This dependency arises because planning is fundamentally adversarial: without these anchoring constraints, a powerful optimizer will inevitably exploit deficiencies in a learned world model, discovering "wormholes" or unrealistic transitions [64, 37]. To make hierarchical world models suitable for planning, we must construct a latent space that both condenses trajectory information to minimize search complexity, and remains inherently well-behaved to prevent the planner from exploiting unrealistic transitions.

To this end, we introduce ID-WM, a hierarchical world model with implicit dynamics grounded in the insight that effective planning requires decoupling topological reachability from execution details (Figure 1). ID-WM consists of a high-level diffusion-based world model that only implicitly models the dynamics by outputting states that are within a fixed distance away from the current state, functioning as a neighboring state generator, and a deterministic low-level model conditioned on both current state and action, acting as an inverse dynamics solver. Conditioned on the current state and a Gaussian noise, the high-level model leverages the generative power of diffusion to map a tractable unit Gaussian latent space to the manifold of feasible neighboring states. This formulation allows us to treat the noise as an implicit high-level action, enabling the planner to search over a well-behaved distribution instead of complex, discontinuous skill spaces to reason purely about "where to go," while the low-level model recovers the precise executable actions for "how to get there."

We summarize our contributions as follows:

  • We introduce ID-WM, a hierarchical world model that decouples reachability from control, eliminating the need for task-specific value functions or pre-trained policies.

  • We propose a corresponding hierarchical planning algorithm that bridges high-level abstraction with local dynamics, enabling robust, long-horizon decision-making.

  • Across three navigation and manipulation environment suites, ID-WM shows 38.5% improvement for reaching arbitrary goals against powerful hierarchical offline reinforcement learning, imitation learning, and diffusion planning methods, and crucially, a 135% improvement over single-level world model and planning regimes.

Figure 1. Overview of ID-WM. Our hierarchical planning framework decouples state-space reachability from local control, eliminating the reliance on task-specific value functions or pre-trained policies. Planning proceeds in two interleaved stages: (1) high-level planning, which discovers a sequence of temporally abstract subgoals spaced Δ\Delta steps apart, and (2) low-level control, which leverages a world model as an inverse dynamics solver to extract executable actions between each subgoal.

Hierarchical Learning

Hierarchical learning has long been a foundational pursuit in artificial intelligence, decomposing long-horizon problems into manageable abstractions [23, 54, 57]. This paradigm spans policy architectures [59], reinforcement learning [28, 7, 38, 63], imitation learning [58], and planning [62, 50]. Existing work focuses on both behavioral (skills/options) [54, 52, 55, 30, 56, 27, 10] and subgoal-centric approaches [29, 50, 51]. However, these methods typically rely on coupled priors, such as rewards, learned value functions or pre-trained low-level policies [50, 38] to ensure feasibility, limiting generalization beyond the training distribution. In contrast, we eliminate this dependency, enabling robust hierarchical reasoning purely through implicit dynamics even when reliable value functions and policies are absent.

World Models for Embodied Planning

World models [65, 1, 2] have emerged as a foundational paradigm for embodied decision-making, enabling agents to reason about the future consequences of their actions. Recently, the field has shifted towards developing general-purpose world models [31, 4, 5, 32, 21, 14] capable of capturing broad environmental dynamics, diverging from traditional model-based reinforcement learning (MBRL) where learning task-specific reward or value functions is paramount [11, 12, 13, 34, 35, 39, 40]. Once these dynamics are learned, inference-time planning algorithms like the Cross-Entropy Method (CEM) [22] or gradient descent—are employed to derive behaviors zero-shot [4, 17, 15, 6]. However, the effectiveness of these approaches is often constrained to short horizons due to the computational complexity of search and the accumulation of prediction errors over time. While various strategies have been proposed to mitigate these issues, ranging from robust gradient-based optimization [16, 37] and efficient sampling routines [41] to temporal approximations for longer action sequences [5], they remain fundamentally flat in their temporal reasoning. Consequently, they struggle with long-horizon task solving, remaining susceptible to local minima and limited in their scalability to complex behaviors. Concurrently, HWM [66] builds a hierarchical latent world model by increasing the high-level frameskip and learning macro-actions; in contrast, ID-WM searches over the diffusion noise space, which is smooth and well-conditioned by construction.

Diffusion Models and Planning

Diffusion probabilistic models [24, 25, 26] excel at capturing complex, multimodal distributions via iterative denoising. Their efficacy is proven across high-dimensional modalities, including pixels [44, 45, 32, 21], language [43], and robot actions [8, 42]. Recently, diffusion models have been proposed as zero-shot planners for embodied tasks [3, 48, 49, 19]. These methods model entire trajectories directly, limiting generalization to out-of-distribution initial-goal pairs and constraining planning horizons to those seen during training. While hierarchical formulations mitigate these constraints, they rely on task-specific value functions [46, 38, 18], successful task trajectories [20], or assumed access to low-level primitives [47]. In contrast, we redefine diffusion as a local generator rather than a global planner. By delegating long-horizon reasoning to search, our method composes local jumps into novel trajectories without requiring task-specific priors.

Hierarchical World Models with Implicit Dynamics

Image not extracted

Figure 2. Architecture of ID-WM. Given an initial observation o0o_0 and a goal ogo_g, we optimize for both the sequence of subgoals s0,…,sTΔs_0, \dots, s_{T\Delta} and the executable actions ata_t in an interleaved fashion. Here, fhif_{hi} denotes the diffusion-based high-level world model and flof_{lo} denotes the deterministic, transformer-based low-level world model, both of which predict in latent space.

Overview and Problem Formulation. We consider a goal-conditioned planning problem within a Markov Decision Process (MDP) defined by the tuple M=(S,A,P)\mathcal{M} = (\mathcal{S}, \mathcal{A}, \mathcal{P}), where S\mathcal{S} is the continuous state space, A\mathcal{A} is the action space, and P(st+1∣s≤t,a≤t)\mathcal{P}(s_{t+1}|s_{\le t}, a_{\le t}) represents the transition dynamics.

Following prior world modeling methods [4, 5, 6], we learn environment dynamics from a task-agnostic offline dataset without any reward labeling via a world model, and plan to reach arbitrary unseen goals at test time. More specifically, an agent is provided with an initial state o0o_0 and a goal state ogo_g at test time. The objective is to execute a sequence of actions (a0,a1,…,aT−1)(a_0, a_1, \dots, a_{T-1}) such that the distance between the agent’s final state oTo_T and the goal ogo_g is minimized. Unlike prior work, we do not assume a fixed task horizon TT [4], focusing instead on enabling the agent to perform autonomous long-horizon reasoning and planning.

Hierarchical World Models with Implicit Dynamics

The core component of ID-WM is a hierarchical world model comprising a high-level diffusion model and a low-level inverse dynamics model. As illustrated in Figure 2, the high-level model operates as a reachable state generator, predicting a distribution of feasible future states conditioned on the current state. The low-level model acts as an inverse dynamics solver, grounding these predicted transitions into executable actions.

High-level WM:fhi(st+Δ∣st,Δ)Low-level WM:flo(st+1∣st−k:t,at−k:t)\begin{align*} \text{High-level WM:} & \quad f_{hi}(s_{t+\Delta} \mid s_{t}, \Delta) \\ \text{Low-level WM:} & \quad f_{lo}(s_{t+1} \mid s_{t-k:t}, a_{t-k:t}) \end{align*}

Specifically, the high-level world model conditions on the current state sts_t and a temporal radius Δ\Delta, generating future states that fall within a Δ\Delta-step horizon. The low-level world model adopts the formulation from [4], a transformer-based world model conditioning on a historical context of kk past states and actions to infer immediate next steps.

Both levels of the hierarchy share a unified structure, consisting of the following components:

Observation model:st∼encθ(st∣ot)Transition model:st+1∼pθ(st+1∣s≤t,c)Decoder model:o^t∼qθ(ot∣st)\begin{align*} \text{Observation model:} & \quad s_t \sim \text{enc}_\theta(s_t \mid o_t) \\ \text{Transition model:} & \quad s_{t+1} \sim p_\theta(s_{t+1} \mid s_{\leq t}, c) \\ \text{Decoder model:} & \quad \hat{o}_t \sim q_\theta(o_t \mid s_t) \\ \end{align*}

where cc is the context that the transition model takes in to make predictions. For fhif_{hi}, cc is the temporal distance Δ\Delta, instructing how far away the subgoals should be. For flof_{lo}, cc is the sequence of executable actions the agent has taken up to time tt (i.e., a≤ta_{\leq t}).

High-level Implicit Dynamics Model

To model the manifold of reachable states without restricting the agent to a fixed set of discrete skills, we formulate the high-level world model as a conditional diffusion probabilistic model (DDPM) [25]. Let x0x_0 represent the target future state st+Δs_{t+\Delta} (at a fixed temporal offset Δ\Delta). The forward diffusion process q(xj∣xj−1)q(x_j | x_{j-1}) gradually adds Gaussian noise to the data according to a variance schedule β1,…,βK\beta_1, \dots, \beta_K:

q(xj∣xj−1)=N(xj;1−βjxj−1,βjI)\begin{equation} q(x_j | x_{j-1}) = \mathcal{N}(x_j; \sqrt{1 - \beta_j} x_{j-1}, \beta_j \mathbf{I}) \tag*{(1)} \end{equation}

This allows us to sample xjx_j at any arbitrary diffusion step jj in closed form. We learn a conditional denoising network ϵθ(xj,j,st)\epsilon_\theta(x_j, j, s_t) to reverse this process, where sts_t is the current state to condition the generation. The training objective is to predict the noise ϵ\epsilon added to the clean data x0x_0:

Ldiff=Ex0,ϵ∼N(0,I),j[∥ϵ−ϵθ(xj,j,st)∥2]\begin{equation} \mathcal{L}_{\text{diff}} = \mathbb{E}_{x_0, \epsilon \sim \mathcal{N}(0, I), j} \left[ \| \epsilon - \epsilon_\theta(x_j, j, s_t) \|^2 \right] \tag*{(2)} \end{equation}

Sampling as Implicit Dynamics (DDIM). To represent the high-level decision at environment time tt, we let zt∈Rdz_t \in \mathbb{R}^d denote the initial noise vector xKx_K used for sampling the next subgoal. During inference, we employ Denoising Diffusion Implicit Models (DDIM) [26] to deterministically map this noise vector zt∼N(0,I)z_t \sim \mathcal{N}(0, \mathbf{I}) to a generated subgoal s^t+Δ\hat{s}_{t+\Delta}. The deterministic reverse step from diffusion step jj to j−1j-1 is given by:

xj−1=αˉj−1(xj−1−αˉjϵθ(xj,j,st)αˉj)⏟predicted x0+1−αˉj−1ϵθ(xj,j,st)\begin{equation} \begin{split} x_{j-1} &= \sqrt{\bar{\alpha}_{j-1}} \underbrace{\left( \frac{x_j - \sqrt{1 - \bar{\alpha}_j} \epsilon_\theta(x_j, j, s_t)}{\sqrt{\bar{\alpha}_j}} \right)}_{\text{predicted } x_0} \\ &\quad + \sqrt{1 - \bar{\alpha}_{j-1}} \epsilon_\theta(x_j, j, s_t) \end{split} \tag*{(3)} \end{equation}

where αj=1−βj\alpha_j = 1 - \beta_j and αˉj=∏i=1jαi\bar{\alpha}_j = \prod_{i=1}^j \alpha_i. This formulation effectively parameterizes the high-level transition dynamics as a deterministic function Gθ(st,zt)\mathcal{G}_\theta(s_t, z_t).

Implicit High-Level Action Space. A key insight of our approach is to treat the initial noise vector ztz_t as an implicit high-level action. Unlike discrete skill indices or learned latent codes which may suffer from mode collapse or topological discontinuities, illustrated in  § 6.13, the noise space Z=Rd\mathcal{Z} = \mathbb{R}^d is inherently smooth and well-behaved (isotropic Gaussian). By optimizing over ztz_t, the planner searches over the standard Gaussian latent space, where any sample zz maps to a physically valid reachable state on the data manifold via the deterministic DDIM mapping.

Low-level World Model

The high-level World Model (WM) provides long-horizon guidance by identifying target states, yet it lacks the granular transition dynamics necessary to prescribe executable actions. To this end, we employ a transformer-based deterministic low-level WM following [4], which functions as an inverse dynamics model through planning.

Given a history of states st−k:ts_{t-k:t} and actions at−k:ta_{t-k:t}, the low-level WM predicts the next state s^t+1=flo(st−k:t,at−k:t)\hat{s}_{t+1} = f_{lo}(s_{t-k:t}, a_{t-k:t}). The model is trained via supervised learning to minimize the prediction error:

L=∥s^t+1−st+1∥2\begin{equation} \mathcal{L} = \|\hat{s}_{t+1} - s_{t+1}\|_2 \tag*{(4)} \end{equation}

Consistent with [4], we utilize an MLP action encoder to process raw action sequences before they are input into the transformer backbone.

At inference, we leverage this model for inverse dynamics through planning. Given a target state s∗s^*, we search for an optimal action sequence (a0,…,aHlo−1)(a_0, \dots, a_{H_{lo}-1}) over a low-level planning horizon HloH_{lo} such that the planning cost ∥s^Hlo−s∗∥2\|\hat{s}_{H_{lo}} - s^*\|_2 is minimized. We detail the full hierarchical planning procedure in § 3.2.

Hierarchical Planning with ID-WM

At inference, given an initial state o0o_0 and a goal state ogo_g, we employ hierarchical planning to determine a sequence of actions that reaches ogo_g. We formulate this as a Model Predictive Control (MPC) problem, where each iteration consists of two phases: subgoal generation and subgoal realization. The overall hierarchical planning procedure is detailed in  alg:reduced. We also include a full version in alg:verbose. Throughout, Δ\Delta denotes the temporal radius of each subgoal (the maximum number of environment steps between consecutive subgoals), HhiH_{hi} the number of subgoals per high-level plan, and HloH_{lo} the low-level planning horizon used to reach each subgoal.

Subgoal Search via High-level WM. In the first phase, we identify a sequence of latent subgoals s0∗,…,sHhi∗s^*_0, \dots, s^*_{H_{hi}} such that s0∗=s0s^*_0 = s_0 and sHhi∗=sgs^*_{H_{hi}} = s_g. To ensure that each subgoal remains within the reachable manifold of the low-level WM given its search budget, we constrain the distance between consecutive subgoals si∗,si+1∗s^*_i, s^*_{i+1} to be no more than Δ\Delta environment steps. We optimize the latent noise variables z0,…,zHhi−1z_0, \dots, z_{H_{hi}-1} using the Cross-Entropy Method (CEM) to minimize the following planning cost:

L=∥s^Hhi−sg∥2+αLreg(Z)\begin{equation} \mathcal{L} = \|\hat{s}_{H_{hi}} - s_g\|_2 + \alpha \mathcal{L}_{reg}(Z) \tag*{(5)} \end{equation}

where Lreg\mathcal{L}_{reg} is a regularization term designed to keep the sampled noise within the support of the standard Gaussian distribution. This prevents the planner from exploiting adversarial regions of the latent space that would result in out-of-distribution transitions during DDIM sampling.

Subgoal Realization via Low-level WM. We similarly employ the CEM optimizer to identify the executable action sequences required to reach each intermediate subgoal. The agent plans and executes actions to reach the immediate subgoal, proceeding sequentially through the high-level trajectory. Upon completing the sequence for all subgoals generated by the high-level WM, the agent initiates a new MPC iteration to replan the high-level trajectory from its current state.

Latent Noise Regularization. While the latent space is theoretically structured as a unit Gaussian, unconstrained planning can favor adversarial latents that produce unrealistic state transitions. To mitigate this, we incorporate a regularization term in the high-level planning cost:

Lreg=Ei[(max⁡(0,∣zi∣−τ))p]\begin{equation} \mathcal{L}_{reg} = \mathbb{E}_{i} \left[ \left( \max(0, |z_{i}| - \tau) \right)^p \right] \tag*{(6)} \end{equation}

This term penalizes latent vectors whose magnitude exceeds a threshold τ\tau, ensuring the optimization stays within high-density regions of the prior. We examine the effect of noise regularization in § 6.6.

\caption{Hierarchical Planning with \textbf{\textsc{ID-WM}}{}}
\label{alg:reduced}
\begin{algorithmic}[1]
\REQUIRE Initial state $s_0$, goal state $s_g$, high/low models $f_{hi}, f_{lo}$, planning costs $C_{hi}, C_{lo}$, planning horizons $H_{hi}, H_{lo}$.
\STATE $s_t \gets s_0$
\WHILE{goal $s_g$ not reached}
    \STATE \textcolor{teal}{\ttfamily{\# High-Level Plan}}
    \STATE $(\hat{s}_1, \dots, \hat{s}_{H_{hi}}) \gets \text{Planner}(s_t, s_g, f_{hi}, C_{hi})$
    
    \STATE \textcolor{teal}{\ttfamily{\# Low-Level Plan and Execution}}
    \FOR{$i = 1$ to $H_{hi}$}
        \STATE $(a_1, \dots, a_{H_{lo}}) \gets \text{Planner}(s_t, \hat{s}_i, f_{lo}, C_{lo})$
        \STATE Execute actions $(a_1, \dots, a_{H_{lo}})$ in the environment
        \STATE $s_t \gets$ observe new state
    \ENDFOR
\ENDWHILE
\end{algorithmic}

Experiments

Our experiments are designed to answer the following key questions:

  • Subgoal Generation: Can ID-WM reliably generate diverse and feasible reachable states?

  • Generalization: Can ID-WM solve long-horizon planning tasks to reach arbitrary, zero-shot goals at test time?

  • Scaling: How does the performance of ID-WM scale across environments of increasing complexity?

  • Visual Planning: Is ID-WM effective for long-horizon planning directly from high-dimensional visual observations?

Environments

We evaluate ID-WM on three navigation and manipulation environment suites as shown in Figure 3. The environments are Wall, PushT, and Maze, with task horizon ranging from hundreds to thousands of environment steps. A full description of the environments is detailed in § 6.1.

Figure 3. We evaluate ID-WM on three environment suites (Wall, PushT, Maze); from left to right: Wall, PushT, and the four maze sizes (U-, Medium, Large, Giant).

Baselines

We compare ID-WM with five state-of-the-art offline learning methods including imitation learning (GCBC), offline RL (HIQL), diffusion planners (Diffuser, Hierarchical Diffuser), and planning with world models (Single-level WM with MPC). See § 6.4 for full details.

Generating Neighboring States with ID-WM

We first investigate the extent to which each level of ID-WM accurately captures the underlying data distribution and predicts future states across varying temporal granularities. Specifically, for the high-level WM, we evaluate whether sampling from the latent prior yields diverse and reachable future states.

In Figure 4, we sample 100 latent noise vectors z∼N(0,I)z \sim \mathcal{N}(0, I) and visualize the resulting generated states s^Δ=fhi(s0,z,Δ)\hat{s}_\Delta = f_{hi}(s_0, z, \Delta) reachable within a Δ\Delta-step horizon. For each Δ\Delta, we overlay all 100 generated samples to visualize the predicted state distribution, explicitly marking the agent’s position with a black dot for clarity.

As illustrated in Figure 4, the high-level WM generates diverse and physically reachable states for both the Wall and PushT environments. In the Wall environment, the dispersion of agent positions increases with Δ\Delta, reflecting a larger reachable radius as the temporal budget grows. Similarly, in the PushT environment, we observe an increasingly diffused distribution of both the agent and the T-block. The smooth coverage of object configurations demonstrates that the model successfully captures the complex, multi-modal interactions between the agent and the object. This high-fidelity distribution modeling serves as a necessary prerequisite for planning, enabling the system to generate diverse trajectories and generalize effectively to unseen goals in downstream tasks.

Figure 4

Figure 4. Diverse neighboring states generated with the high-level WM of ID-WM.

Hierarchical Planning with ID-WM

Can ID-WM solve long-horizon planning tasks to reach arbitrary, zero-shot goals at test time? To this end, we conduct hierarchical planning on three environment suites.

For all environments and baselines, we randomly sample 50 pairs of initial and goal states, with sampling method detailed in  § 6.2, and report the average performance across these instances. For the PushT task, we report two metrics: the success rate and the final coverage of the T-block relative to the target configuration. A trial is considered successful only if both the agent and the T-block reach their respective goal positions. In contrast, the final coverage metric provides a more granular measure of performance by focusing specifically on the object’s alignment. To further evaluate generalization capabilities, we test PushT under two settings: a standard regime and a data-augmented regime featuring random spatial transformations and rotations. All experiments are conducted with a maximum horizon of 2000 environment steps. For all experiments, ID-WM uses Δ=5\Delta = 5 (25 environment steps with frameskip) and Hhi=5H_{hi} = 5 subgoals per high-level plan, without per-task tuning; the planner replans after each executed plan (up to 15 replans), so no prior knowledge of the goal distance is required. We ablate Δ\Delta and the number of subgoals in § 6.7.

EnvironmentMetricHIQLGCBCDiffuserH-DiffuserSL-WMOurs
Medium MazeSR0.830.83±0.01\pm 0.010.800.80±0.00\pm 0.000.800.80±0.13\pm 0.130.920.92±0.03\pm 0.030.970.97±0.01\pm 0.010.99\mathbf{0.99}±0.01\mathbf{\pm 0.01}
WallSR0.620.62±0.12\pm 0.121.00\mathbf{1.00}±0.00\mathbf{\pm 0.00}0.070.07±0.08\pm 0.080.100.10±0.02\pm 0.020.880.88±0.02\pm 0.021.00\mathbf{1.00}±0.00\mathbf{\pm 0.00}
PushT (Fix)SR0.450.45±0.07\pm 0.071.00\mathbf{1.00}±0.00\mathbf{\pm 0.00}0.000.00±0.00\pm 0.000.010.01±0.01\pm 0.010.190.19±0.02\pm 0.021.00\mathbf{1.00}±0.00\mathbf{\pm 0.00}
Cov0.500.50±0.05\pm 0.050.85\mathbf{0.85}±0.04\mathbf{\pm 0.04}0.100.10±0.01\pm 0.010.350.35±0.09\pm 0.090.340.34±0.03\pm 0.030.85\mathbf{0.85}±0.01\mathbf{\pm 0.01}
PushT (Rand.)SR0.020.02±0.02\pm 0.020.210.21±0.01\pm 0.010.000.00±0.00\pm 0.000.010.01±0.01\pm 0.010.210.21±0.04\pm 0.040.77\mathbf{0.77}±0.01\mathbf{\pm 0.01}
Cov0.220.22±0.03\pm 0.030.310.31±0.01\pm 0.010.050.05±0.01\pm 0.010.200.20±0.08\pm 0.080.270.27±0.02\pm 0.020.58\mathbf{0.58}±0.01\mathbf{\pm 0.01}
PushT-Aug (Fix)SR0.870.87±0.01\pm 0.011.00\mathbf{1.00}±0.00\mathbf{\pm 0.00}0.010.01±0.02\pm 0.020.550.55±0.14\pm 0.140.100.10±0.04\pm 0.041.00\mathbf{1.00}±0.00\mathbf{\pm 0.00}
Cov0.810.81±0.05\pm 0.050.830.83±0.01\pm 0.010.310.31±0.03\pm 0.030.740.74±0.16\pm 0.160.430.43±0.01\pm 0.010.84\mathbf{0.84}±0.01\mathbf{\pm 0.01}
PushT-Aug (Rand.)SR0.320.32±0.06\pm 0.060.150.15±0.01\pm 0.010.010.01±0.01\pm 0.010.270.27±0.03\pm 0.030.100.10±0.04\pm 0.041.00\mathbf{1.00}±0.01\mathbf{\pm 0.01}
Cov0.590.59±0.03\pm 0.030.250.25±0.01\pm 0.010.140.14±0.01\pm 0.010.720.72±0.19\pm 0.190.170.17±0.01\pm 0.010.77\mathbf{0.77}±0.01\mathbf{\pm 0.01}
Figure 5

Figure 5. High-level subgoals planned by ID-WM. The planned sequence successfully guides the agent to the target goal while ensuring that consecutive subgoals satisfy local reachability constraints.

In tab:results, ID-WM outperforms all baselines, achieving a 38.5% overall improvement and 135% gain over single-level WM with MPC (SL-WM). The gap is most evident in PushT with random goals, where ID-WM is the only approach exceeding a 30% success rate without augmentation. While SL-WM performs well on shorter horizons (Maze, Wall), it fails on PushT’s long-horizon requirements (often >100>100 steps) due to exponential search complexity and compounding prediction errors. Unlike the 25-step benchmark in [4], our results show that standard WM planning cannot solve these extended tasks, a limitation our hierarchical approach effectively addresses. We visualize the resulting high-level plans in Figure 5 and examine subgoal quality and diversity in sec:neighboring,sec:convergence.

Furthermore, the remaining baselines exhibit distinct failure modes. GCBC performs well on in-distribution goals (e.g., fixed-goal PushT) and in environments with dense data coverage (Medium Maze, Wall) but degrades significantly on out-of-distribution (OOD) goals. HIQL shows potential for OOD generalization but demands dense data coverage to accurately learn value functions, reflected in its performance gap between PushT and PushT-Aug. Diffuser-based methods struggle with dataset noise; despite adding replanning (§ 6.11.8), performance remains poor. While H-Diffuser offers marginal improvements, it still relies heavily on dense data. Even with data augmentation (§ 6.3), H-Diffuser’s success rate remains low, primarily because it struggles to jointly attend to both the object and the agent’s final position.

Comparison with hierarchical latent world models. We additionally compare with HWM [66], a hierarchical world model that increases the frameskip of its high-level model and compresses action sequences into macro-actions with a transformer-based action encoder. We compare with the DINO-WM variant of HWM. As shown in Table 1, on PushT, which requires long-horizon object manipulation, ID-WM outperforms HWM by a large margin on all variants. This highlights the benefit of disentangling state reachability from fine-grained control using diffusion noise as an implicit latent action.

PushTPushT-Aug
2-3 (lr)4-5 MethodFixRand.FixRand.
HWM0.380.200.420.16
ID-WM1.000.771.001.00

Table 1. Success rate of ID-WM and HWM on PushT environments with fixed (Fix) and random (Rand.) goals.

Unlike approaches that explicitly model dataset behavior, ID-WM is not constrained by the underlying training distribution. By leveraging full planning over the environment dynamics, our method is inherently capable of finding diverse solutions for a given goal. Figure 6 demonstrates this by showing two different subgoal sequences generated by ID-WM. For instance, Plan 1 (top row) rotates the T-block counterclockwise, while Plan 2 (bottom row) rotates it clockwise. Ultimately, both planned trajectories successfully achieve the final objective via our hierarchical framework.

Figure 6. Diverse plans generated by ID-WM. The top and bottom rows illustrate distinct strategies: counterclockwise and clockwise rotations of the T-block, both resulting in successful task completion.

How does ID-WM scale with task complexity?

Figure 7. Planning performance with increasing task complexity. In a), we test ID-WM’s ability on Maze with increasing sizes. In b), we test ID-WM on increasingly longer horizon PushT tasks.

We investigate whether ID-WM scales to tasks of increasing complexity across two dimensions: environment size and goal horizon.

Scaling with Maze Size. We evaluate ID-WM across four mazes of increasing difficulty: U-Maze (3×33 \times 3), Medium Maze (6×66 \times 6), Large Maze (7×107 \times 10), and Giant Maze (13×1313 \times 13) (Figure 7a). Each dataset contains 2,000 random 100-step trajectories. Given a fixed dataset size, data coverage naturally becomes sparser as maze size increases.

While all methods achieve near-perfect success on the U-Maze, baselines degrade significantly in larger environments (Figure 7a). In contrast, ID-WM maintains the highest performance on the Large Maze. We hypothesize this is because ID-WM combines local dynamics modeling with high-level planning, effectively "stitching" together regions of sparse data. Unlike GCBC or Diffuser (which model entire trajectories simultaneously) or HIQL (which requires dense data for global value functions), ID-WM does not rely on global trajectory coverage. Furthermore, while H-Diffuser uses abstraction, it lacks an explicit dynamics model, resulting in brittle plans requiring expensive, high-frequency replanning. By grounding its hierarchy in explicit local dynamics, ID-WM generates robust, scalable plans.

Scaling with Goal Horizon. Building on the PushT planning benchmark introduced by [4] (which evaluates goals 25 steps away), we increase complexity by sampling goals from the validation set that are d∈{25,50,…,150}d \in \{25, 50, \dots, 150\} steps away.

We compare ID-WM against a single-level world model baseline that performs full-horizon CEM search and replanning at every step. While the total number of executed actions is identical, ID-WM achieves a near-perfect success rate across all horizons as shown in Figure 7b). This confirms our main PushT results (tab:results), demonstrating that ID-WM maintains precise goal-reaching capabilities even when goals are over 150 steps away.

Vision-based Planning with ID-WM

To evaluate ID-WM in high-dimensional observation spaces, we test it on a vision-based PushT task. We use the state-based PushT dataset augmented with four 90∘^\circ rotations for spatial robustness. Observations (3×64×643 \times 64 \times 64 images) are encoded into 4×8×84 \times 8 \times 8 latents using a pretrained Stable Diffusion VAE [68]. We compare against AVDC [19], a language-conditioned video planner that generates all subgoals simultaneously (details in § 6.4).

As shown in Figure 8, ID-WM generates reliable, sequential subgoals that ensure steady progress. In contrast, AVDC fails to produce meaningful plans, likely because the initial-goal pairs are out-of-distribution. This highlights the generalization advantage of iterative subgoal search over simultaneous generation—which relies heavily on trajectory-level dataset coverage—even within compressed visual spaces.

Figure 8. Subgoals generated by ID-WM on a visual PushT task. The first column denotes the starting state, and the last shaded column denotes the desired goal state. ID-WM generates reliable, sequential subgoals that drive steady progress toward the goal state without any degenerate states.

Quantitative evaluation. To complement the qualitative comparison, we report success rates on vision-based PushT in Table 2. We compare against two additional baselines: DINO-WM [4], a flat single-level world model planner, and HWM [66], a concurrent hierarchical latent world model that uses a larger frameskip for its high-level model. In this setting, ID-WM uses DINO-WM as its low-level world model, so the comparison between DINO-WM and ID-WM isolates the gain provided by the hierarchy. We evaluate four settings: for d∈{25,50,75}d \in \{25, 50, 75\}, the goal is sampled from a held-out validation set and is known to be dd steps away; for Random Goal, both the initial and goal states are sampled randomly from the environment. ID-WM achieves the best performance on the most challenging long-horizon settings (d=50d = 50, d=75d = 75, and Random Goal), demonstrating its effectiveness for long-horizon vision-based planning.

Methodd=25d = 25d=50d = 50d=75d = 75Random Goal
DINO-WM0.840.160.000.00
HWM0.960.460.140.02
ID-WM0.900.600.500.26

Table 2. Success rate on vision-based PushT. For d∈{25,50,75}d \in \{25, 50, 75\} the goal is dd steps away; Random Goal samples both the initial and goal states randomly.

Conclusion

We present ID-WM, a hierarchical world modeling and planning framework that decouples temporally extended subgoal generation from local reachability. By treating diffusion noise as implicit actions, we constrain the planning search space to a unit Gaussian, ensuring reliable subgoal generation. ID-WM significantly outperforms state-of-the-art baselines and demonstrates high-quality vision-based subgoal generation. Ultimately, grounding hierarchical abstraction in explicit local dynamics represents a promising step toward robust, long-horizon decision-making.

Limitations. First, ID-WM’s reliance on search-based optimization within a diffusion latent space incurs higher inference times than reactive policies or deterministic world models. Future work will explore amortized or gradient-based planning to mitigate this bottleneck. Second, while our hierarchy prevents planning complexity from scaling with trajectory length, it still scales with state-space dimensionality. Integrating compact, structured representation learning could further streamline this planning process for highly complex environments.

Appendix

Environments and Dataset Generation

We evaluate ID-WM on three navigation and manipulation environment suites as shown in Figure 3. The environments are Wall, PushT, and Maze, with task horizon ranging from hundreds to thousands of environment steps.

  1. Wall: Introduced in [4], this is a simple 2D navigation environment that consists of two rooms separated by a wall with a door. The agent’s goal is to navigate from a random starting location on one side of the wall to a random location on the other side, requiring the agent to pass through the door. We use the same dataset as in [4], which consists of randomly generated 1920 trajectories of length 50.

  2. Maze: This environment is introduced in [9], where a force-actuated 2-DoF ball navigates to an arbitrary location in a maze. In this work, we consider maze layout of four sizes, U (3×33 \times 3), Medium (6×66 \times 6), Large (7×107 \times 10), and Giant (13×1313 \times 13), visualized in Figure 7 a). We generate 2000 random trajectories of length 100 for each maze layout.

  3. PushT: Introduced by [8], this environment consists of a pusher agent and a T-shaped block. The objective is for both the agent and the block to reach specified target locations from random initial positions. We evaluate two variants of this task: in Fix Goal, the target state is a fixed configuration (the green T); in Random Goal, the target is randomly sampled from the environment’s state space. Both settings are significantly more challenging than the formulation in [4], where the goal is provided at a known, fixed horizon of 25 steps. In contrast, completing the task in our settings often requires hundreds of steps, rendering standard single-level planning infeasible. Following [4], we train on noisy, replayed trajectories. We further evaluate our method in a data-abundant regime by applying random translation and rotation augmentations to the existing dataset.

Task Visualizations

For the Wall, PushT, Giant Maze, Large Maze, Medium Maze, and U-Maze environments, we randomly sample 50 initial and goal states from each environment’s state space (visualizations in Figure 9). For PushT, the random-goal setting samples both the initial and goal states randomly from the environment, whereas the fixed-goal setting samples a random initial state while using the environment’s fixed goal location. For the Wall environment, to ensure sufficient task difficulty, the initial and final object locations are constrained to lie on opposite sides of the wall. For the Giant Maze, Large Maze, Medium Maze, and U-Maze environments, we follow the random state sampling procedure used in OGBench: we sample arbitrary free cells for the ball and add continuous uniform noise around the corresponding grid-cell centers.

Figure 9. Examples of sampled initial and goal state pairs for the Wall, PushT, Giant Maze, Large Maze, and U-Maze environments

Data Augmentation

Data augmentation for PushT.

For PushT we additionally apply geometric data augmentation to the offline dataset at each training step. The state vector contains end-effector position and velocity (xdot,ydot,x˙dot,y˙dot)(x_\mathrm{dot}, y_\mathrm{dot}, \dot{x}_\mathrm{dot}, \dot{y}_\mathrm{dot}), T-block position (xT,yT)(x_T, y_T), and T-block orientation (sin⁡θT,cos⁡θT)(\sin\theta_T, \cos\theta_T). One of three augmentation types is sampled per step:

  • Displacement (40%): all positions shifted by a random integer offset (δx,δy)(\delta x, \delta y) chosen so that both the end-effector and the T-block remain within the screen boundaries [50,450][50,450] and [100,400][100,400] respectively.

  • Rotation (30%): all positions and the T-block angle rotated by θ∼U(0,2π)\theta \sim \mathcal{U}(0, 2\pi) about the scene center; velocities and actions are rotated accordingly.

  • Displacement + Rotation (30%): a half-range displacement combined with a small rotation θ∼U(0,30∘)\theta \sim \mathcal{U}(0, 30^\circ).

Baselines

We compare ID-WM with five state-of-the-art offline learning methods including imitation learning (GCBC), offline RL (HIQL), diffusion planners (Diffuser, Hierarchical Diffuser), and planning with world models (Single-level WM with MPC). For qualitative vision-based experiments, we compare ID-WM with AVDC [19].

  1. Single-level WM and Planning: This baseline employs a flat world model based on [4] to perform single-level planning via the Cross-Entropy Method (CEM) without hierarchical abstractions.

  2. Goal-conditioned Behavior Cloning (GCBC): An imitation learning paradigm where a policy is trained to map the current state and a target goal observation directly to the expert action that leads toward that goal. For our implementation, we extend Diffusion Policy [8] by conditioning the policy on a future frame randomly sampled from the trajectory.

  3. HIQL [7]: An offline reinforcement learning method that learns hierarchical representations by combining high-level subgoal-as-action policy with low-level subgoal-conditioned policy for long-horizon tasks.

  4. Diffuser [3]: A trajectory-level diffusion model that performs planning by sampling entire state-action sequences. We replan after executing each action in the environment.

  5. Hierarchical Diffuser [18]: An extension of Diffuser that decomposes the planning process into two temporal scales. A high-level module synthesizes intermediate subgoals, which are subsequently used by a low-level denoising network to generate a refined trajectory. We replan both the high level and the low level after executing each action in the environment.

  6. AVDC [19]: This approach uses a video generation model to predict future states and then employs inverse kinematics (IK) to derive the necessary actions; we specifically use it to compare the quality of its generated video sequences against the high-level subgoals produced by our method.

  7. HWM [66]: A concurrent hierarchical latent world model that increases the frameskip of the high-level model and plans over learned macro-actions. We compare with the DINO-WM variant of HWM. Since the official implementation of this variant is not open-sourced, we reimplement it following the reported settings and DINO-WM for unspecified details. We compare against HWM in tab:hwm,tab:vision_quant.

Hierarchical Planning with CEM

In this section, we provide the complete pseudo-code for our hierarchical planning procedure using CEM. alg:verbose expands upon the simplified overview presented in alg:reduced, explicitly detailing the sampling and optimization loops for both high-level subgoal generation and low-level action execution.

\caption{Hierarchical Planning with \textbf{\textsc{ID-WM}}{} (CEM)}
\label{alg:verbose}
\begin{algorithmic}[1]
\REQUIRE Initial state $s_0$, goal state $s_g$, high/low models $f_{hi}, f_{lo}$, costs $C_{hi}, C_{lo}$
\REQUIRE CEM parameters: $N$ samples, $M$ iterations, $K$ elites, horizons $H_{hi}, H_{lo}$
\STATE $s_t \gets s_0$
\WHILE{goal $s_g$ not reached}
    \STATE \textcolor{teal}{\ttfamily{\# High-Level Plan (Subgoal Generation)}}
    \STATE Initialize latent distribution parameters $\theta_{hi}$
    \FOR{$m = 1$ to $M$}
        \STATE Sample $N$ latent sequences $Z^{(n)} = (z^{(n)}_1, \dots, z^{(n)}_{H_{hi}}) \sim \mathcal{D}(\theta_{hi})$
        \FOR{$n = 1$ to $N$}
            \STATE Rollout subgoals: $\hat{s}^{(n)}_t \sim f_{hi}(\hat{s}^{(n)}_{t-1}, z^{(n)}_t)$ for $t \in [1, H_{hi}]$
            \STATE Compute trajectory cost $J^{(n)}_{hi} = C_{hi}(\hat{s}^{(n)}_{1:H_{hi}}, s_g)$
        \ENDFOR
        \STATE Update $\theta_{hi}$ using top $K$ elite sequences with the lowest $J_{hi}$
    \ENDFOR
    \STATE Extract best subgoal sequence $(\hat{s}^*_1, \dots, \hat{s}^*_{H_{hi}})$
    
    \STATE \textcolor{teal}{\ttfamily{\# Low-Level Plan (Action Generation)}}
    \FOR{$i = 1$ to $H_{hi}$}
        \STATE Initialize action distribution parameters $\theta_{lo}$
        \FOR{$m = 1$ to $M$}
            \STATE Sample $N$ action sequences $A^{(n)} = (a^{(n)}_1, \dots, a^{(n)}_{H_{lo}}) \sim \mathcal{D}(\theta_{lo})$
            \FOR{$n = 1$ to $N$}
                \STATE Rollout states: $\tilde{s}^{(n)}_t \sim f_{lo}(\tilde{s}^{(n)}_{t-1}, a^{(n)}_t)$ for $t \in [1, H_{lo}]$
                \STATE Compute trajectory cost $J^{(n)}_{lo} = C_{lo}(\tilde{s}^{(n)}_{1:H_{lo}}, \hat{s}^*_i)$
            \ENDFOR
            \STATE Update $\theta_{lo}$ using top $K$ elite sequences with the lowest $J_{lo}$
        \ENDFOR
        \STATE Extract best action sequence $(a^*_1, \dots, a^*_{H_{lo}})$
        \STATE \textcolor{teal}{\ttfamily{\# Execute Planned Actions}}
        \STATE Execute actions $(a^*_1, \dots, a^*_{H_{lo}})$ in the environment
        \STATE $s_t \gets$ observe new state
    \ENDFOR
\ENDWHILE
\end{algorithmic}

Ablation: Effect of CEM Search Budget and Noise Regularization

Figure 10

Figure 10. Success rate with respect to CEM samples and noise regularization.

We investigate the impact of the CEM search budget and the noise regularization strategy introduced in § 3.2. By varying the number of CEM samples per level (maintaining a 10% elite fraction), we demonstrate that the performance of ID-WM scales effectively with test-time compute (Figure 10). Furthermore, planning with non-zero noise regularization (α>0\alpha > 0) consistently yields higher success rates compared to the unregularized baseline. Unless otherwise noted, all reported results use the same planner configuration at both levels: 300 CEM samples, 30 elites, and a horizon of 5 (Table 6).

Ablation on Δ\Delta and number of subgoals

All results in the main paper use Δ=5\Delta = 5 (25 environment steps with frameskip) and Hhi=5H_{hi} = 5 subgoals per high-level plan, without per-task tuning. State-based experiments use a 50-step noise schedule for both DDPM training and DDIM inference; vision-based experiments train with 250 steps and use 25 DDIM steps at inference.

To assess sensitivity to these choices, we sweep Δ\Delta and the number of subgoals on PushT-Aug with random goals (Table 3). Performance is robust for large Δ\Delta across a wide range of subgoal counts. For a fixed effective search horizon (Δ×\Delta \times number of subgoals), splitting it evenly between the two levels is most effective: at an effective horizon of roughly 25, the success rate increases monotonically from 0.06 (Δ=1\Delta = 1, 25 subgoals) to 0.96 (Δ=5\Delta = 5, 5 subgoals). A very small Δ\Delta pushes the burden onto a long subgoal chain while contributing little per hop.

Δ\Delta \ #Subgoals345681225
10.060.040.060.080.040.100.06
20.140.140.280.500.640.480.24
30.500.560.640.820.760.740.42
40.600.760.860.861.000.900.44
50.740.840.960.980.940.760.38

Table 3. Success rate on PushT-Aug (Random Goal) when sweeping the subgoal horizon Δ\Delta (rows) and the number of subgoals per high-level plan (columns). The default configuration (Δ=5\Delta = 5, 5 subgoals) is underlined.

Steps-to-goal metric

Success rate measures whether a goal is reached but not how efficiently. To quantify trajectory efficiency, we report the average number of environment steps to success for all methods on PushT-Aug (Random Goal) and Medium Maze in Table 4, computed over successful trajectories only (N/A if no trajectory succeeded). ID-WM is the most efficient method on PushT-Aug and the most efficient among value-function-free methods on Medium Maze. HIQL’s learned value function yields substantially shorter maze paths, reflecting a trade-off: the planning cost in § 3.2 is the distance between the last imagined state and the goal, which optimizes for reaching the goal rather than for path efficiency. Within the same framework, a self-supervised value function could be substituted for this cost to improve path optimality.

MethodPushT-Aug (Rand.)Medium Maze
HIQL307.434.1
GCBC1014.9336.8
DiffuserN/A287.8
H-Diffuser713.9169.4
SL-WM312.5229.5
ID-WM234.7163.9

Table 4. Average steps to success over successful trajectories (lower is better). N/A indicates that no trajectory succeeded.

Test-time Compute

tab:inference_time reports the wall-clock inference time to reach one goal for each method in the state-based setting, measured on a single NVIDIA L40S GPU. GCBC and HIQL are reactive policies whose compute budget cannot be adapted at test time; GCBC is comparatively slow because of its Diffusion Policy backbone. Diffuser and H-Diffuser follow their original formulation of replanning a full trajectory after every action, which is the most expensive setting. Despite running CEM at both levels, ID-WM is roughly 27×27\times faster than Diffuser and 54×54\times faster than H-Diffuser, and faster than GCBC. The only faster methods are HIQL and SL-WM, both of which fail on long-horizon tasks.

The CEM search budget for ID-WM and SL-WM is identical in all reported results (Table 6): a horizon of 5 at each of our two levels gives an effective horizon of 25, matching SL-WM’s flat horizon. ID-WM is slower only because of DDIM sampling in the high-level model.

Inference-time-matched SL-WM. Since SL-WM runs faster than ID-WM at the same CEM budget, we additionally match wall-clock time on PushT (Random Goal) by allowing SL-WM 45 replans (3×3\times). As shown in Table 7, tripling SL-WM’s replanning budget yields only a marginal improvement (0.21 to 0.28) and remains far below ID-WM (0.77). This indicates that SL-WM’s failure mode is compounding prediction error over long horizons rather than insufficient search, and that the gain comes from the hierarchy rather than from additional test-time compute.

MethodInference time
HIQL5.61 s
SL-WM16.93 s
ID-WM53.08 s
GCBC73.3 s
Diffuser24 min
H-Diffuser48 min

Table 6. CEM planner configuration shared by ID-WM and SL-WM in all reported results.

MethodSamplesElitesReplansHorizon
SL-WM300301525
ID-WM30030155 (each level)

Table 6. CEM planner configuration shared by ID-WM and SL-WM in all reported results.

MethodSamplesElitesReplansHorizonSR
SL-WM (CEM budget matched)3003015250.21
SL-WM (inference time matched)3003045250.28
ID-WM30030155 (each level)0.77

Table 7. Success rate on PushT (Random Goal) when matching either the CEM budget or the wall-clock inference time of SL-WM to ID-WM.

Compute Resources

We train our state-based world models using 1 to 8 NVIDIA H200 Tensor Core GPUs, utilizing Distributed Data Parallel (DDP) to accelerate training when scaling to 8 GPUs. All baseline models are trained to convergence on a single NVIDIA H200 Tensor Core GPU. Planning evaluations are also conducted using a single NVIDIA H200 Tensor Core GPU. For the vision-based experiments, we employ a CDiT [5] backbone and train the model using DDP across a cluster of 32 NVIDIA H200 Tensor Core GPUs.

Implementation Details

Implementations and Hyperparameters

Hyperparameters for ID-WM high-level dynamics model training and planning are listed in Table 8 and Table tab:plan_hyperparams, respectively. Hyperparameters for low-level WM training are listed in Table 9 and Table tab:wm_shared_hyperparams, while those for low-level WM planning are also included in Table tab:plan_hyperparams. The high-level dynamics model and low-level WM share the same CEM planner configurations, except that action regularization is applied only in high-level planning. We use consistent hyperparameters across tasks, except for the observation history length in the Wall environment, which follows DINO-WM.

For success metrics, we continue using all state-based evaluation metrics from DINO-WM. Success is evaluated, and the planner may stop, after each plan is fully executed.

NameValue
Emb Dim56
Down Dims[256]
Kernel3
Groups8
lr1e−41\mathrm{e}{-4}
WDecay0.1
Noise Timesteps50
lr warmup steps200

Table 8. Shared hyperparameters for training high-level dynamics model of ID-WM{} across all environments.

NameValue
Noise Reg. coefficientα\alpha{0,2}\{0,2\}
τ\tau2
pp1
Max Replan15
HL Horizon5
LL Horizon5
Top-kk30
Num Samples300
Var Scale1
Opt Steps30
ToT_oFrameskipDataset SizeTraj. Len.
PushT3518500100--300
Wall15192050
U-Maze352000100
Medium Maze352000100
Large Maze352000100
Giant Maze352000100

Table 9. Environment-dependent hyperparameters for WM training. ToT_o denotes the observation history length.

NameValue
Predictor lr5e−45\mathrm{e}{-4}
Action encoder lr5e−45\mathrm{e}{-4}
Depth6
Heads8
MLP dim64
Dropout0
Emb. dropout0
PoolMean

Baseline Implementations and Hyperparameters

Unless otherwise stated, all baselines are trained on the same offline datasets and evaluated using the same task specifications, goal definitions, and environment interaction budget as ID-WM.

Single-level WM and Planning

We implement a single-level WM with transformer-based predictor following DINO-WM [4]. The training and inference details are identical to those of the low level WM in § 3.1.2 with implementation details in § 6.11.1. During inference, the planner optimizes planning cost ∥s^Hlo−s∗∥2\|\hat{s}_{H_{lo}} - s^*\|_2. For state-based WM, we adapt DINO-WM and remove the frozen pre-trained DINOv2 encoder. This single-level WM baseline isolates the effect of hierarchy and tests whether subgoal proposal mitigates cumulative prediction error.

Goal-conditioned Behavior Cloning (GCBC)

GCBC trains a policy to imitate dataset actions conditioned on both the current state and a desired goal state. For training, given an offline trajectory {(st,at)}t=0T\{(s_t, a_t)\}_{t=0}^{T}, we randomly sample a current state sts_t and a future state st+ks_{t+k} where k>0k>0. The policy is trained to predict dataset action ata_t with the training objective:

LGCBC=E(st,at,g)∼D[∥πθ(st,g)−at∥22],\begin{equation} \mathcal{L}_{\mathrm{GCBC}}=\mathbb{E}_{(s_t,a_t,g)\sim \mathcal{D}} \left[\| \pi_\theta(s_t, g) - a_t \|_2^2\right], \tag*{(7)} \end{equation}

Specifically, we implement GCBC as a Diffusion Policy [8] conditioned on the concatenation of the current observation and goal state to predict an action chunk.

We include GCBC as a representative of direct goal-conditioned imitation learning and as a baseline for what behavioral cloning alone can achieve. Hyperparameters are listed in Table 10 and Table tab:gcbc_shared_hyperparams.

ToT_oAct. WindowPred. HorizonAct. Steps
PushT29108
Wall2192020
U-Maze2192020
Medium Maze2192020
Large Maze2192020
Giant Maze2192020

Table 10. Environment-dependent hyperparameters for GCBC training. ToT_o: observation history length; Act. Window: minimum timestep gap between current state and goal state; Pred. Horizon: action length generated per diffusion forward pass; Act. Steps: actions executed before re-planning.

NameValue
lr1e−41\mathrm{e}{-4}
Weight decay0
Diffusion steps100
Layers8
Heads4
Embedding dim256
Attn. dropout0.1
Emb. dropout0

HIQL [7]

HIQL decomposes offline goal-conditioned RL into two levels of hierarchy. A high-level policy proposes a subgoal representation ϕ([s;w])\phi([s; w]), where ww is an intermediate waypoint state sampled from the same trajectory, and a low-level policy executes primitive actions to reach that subgoal. Both policies are trained offline via Advantage-Weighted Regression (AWR) using a shared implicit value function Vθ(s,g)V_\theta(s, g).

The value function is trained with an IQL-style expectile regression objective:

LV(θV)=E(s,s′)∼D, g∼p(g∣τ)[L2τ(r(s,g)+γVθˉV(s′,g)−VθV(s,g))].\mathcal{L}_V(\theta_V) = \mathbb{E}_{(s,s') \sim \mathcal{D},\, g \sim p(g \mid \tau)} \left[ L_2^\tau \left( r(s,g) + \gamma V_{\bar{\theta}_V}(s',g) - V_{\theta_V}(s,g) \right) \right].

where L2τL_2^\tau denotes the asymmetric squared expectile loss with parameter τ\tau. Given the shared value function, both policies are updated via AWR. For the low-level policy πl\pi^l, the advantage measures progress toward subgoal ww. For the high-level policy πh\pi^h, the advantage measures progress toward the final goal gg upon reaching subgoal ww.

We use the HIQL implementation from the OGBench repository [?]. In all state-based experiments, no visual encoder is used, and ϕ\phi processes raw state vectors directly. As HIQL is stochastic, all results are averaged over 3 random seeds. Hyperparameters are listed in Table 11.

NameValue
lr3e−43\mathrm{e}{-4}
Discount factor γ\gamma0.99
Expectile τ\tau0.7
High-level α\alpha3.0
Low-level α\alpha3.0
Subgoal steps ksubk_{\mathrm{sub}}25
Actor/value hidden dims(512,512,512)(512,512,512)
Train steps10710^7

Table 11. Shared hyperparameters for HIQL across all evaluation environments.

Diffuser [3]

Diffuser frames offline goal-conditioned planning as a trajectory denoising problem. A temporal U-Net processes a full trajectory τ=(s0,a0,…,sH−1,aH−1)\tau = (s_0, a_0, \ldots, s_{H-1}, a_{H-1}) of planning horizon HH, and iterative reverse diffusion over TT steps is guided by goal conditioning to produce trajectories that reach a specified goal state. At inference, the current state is inpainted at the start of the trajectory and the goal state at the end; the remaining timesteps are filled by denoising. All environments share the same base configuration. Only the planning horizon HH is adjusted (in Table 12).

Detailed implementation choices for inference are expanded in § 6.11.8.

DiffuserHDiffuser
HHHHJJ
PushT12812525
Wall484515
U-Maze487525
Medium Maze487525
Large Maze487525
Giant Maze967525

Table 12. Environment-dependent hyperparameters for Diffuser and HDiffuser. HH: planning horizon; JJ: subsampling jump (HDiffuser high-level planner only).

NameValue
lr2e−42\mathrm{e}{-4}
Diffusion stepsTT256
EMA decay0.995
U-Net dim. mults(1,4,8)(1, 4, 8)

Hierarchical Diffuser [18]

H-Diffuser [18] extends Diffuser with a two-level hierarchy. A high-level diffusion model plans jumpy subgoals spaced JJ timesteps apart, producing a compressed sequence of length ⌈Hhl/J⌉\lceil H_{\mathrm{hl}}/J \rceil that covers a larger temporal horizon at lower computational cost. A low-level diffusion model then plans a dense trajectory of length HllH_{\mathrm{ll}} between consecutive subgoals, conditioned on the current state and the subgoal as the terminal constraint.

Hyperparameters shared with Diffuser are listed in Table tab:diffuser_shared_hyperparams. Environment-specific hyperparameters are listed in Table 12.

Detailed implementation choices for inference are expanded in § 6.11.8.

Baseline Tuning for Diffuser & H-Diffuser

The original Diffuser paper uses two execution strategies depending on the environment: for D4RL locomotion tasks, only the first action of each plan is executed and the model replans at every step (execute one action + replan); for maze2d and Kuka block-stacking, the full planned trajectory is executed open-loop without replanning (execute plan). H-Diffuser adopts the same two strategies at the subgoal level. To enable a fair comparison with our method, we additionally evaluate both models under a third strategy in which the full plan is executed and the model then replans (execute plan + replan). Results under all three strategies are reported in Table 13 and Table 14.

ModelExecutionPushTPushT-Aug
Fix GoalRandom GoalFix GoalRandom Goal
SRCovSRCovSRCovSRCov
Diffuserexecute plan + replan0.000.030.000.030.280.370.060.19
Diffuserexecute one action + replan0.000.110.000.060.040.270.020.15
Diffuserexecute plan0.000.050.000.030.000.090.000.13

Table 13. Diffuser results across execution settings and PushT variants.

ModelExecutionPushTPushT-Aug
Fix GoalRandom GoalFix GoalRandom Goal
SRCovSRCovSRCovSRCov
H-Diffuserexecute plan + replan0.380.280.000.080.560.510.460.36
H-Diffuserexecute one action + replan0.000.250.020.110.560.560.260.50
H-Diffuserexecute plan0.040.090.000.060.020.170.000.10

Table 14. H-Diffuser results across execution settings and PushT variants.

High-Level World Model Training Convergence

We further analyze the training-time convergence behavior of the high-level world model, extending the qualitative study in Figure 4. While Figure 4 visualizes the final learned neighboring-state distribution, Figure 11 shows how this distribution emerges over the course of training.

Specifically, we fix validation observations s0s_0 and periodically sample predicted future states from intermediate checkpoints of the high-level WM. The first row shows the initial observation s0s_0 sampled from the validation set. The lower rows visualize generated future states s^Δ=fhi(s0,z,Δ)\hat{s}_\Delta = f_{hi}(s_0, z, \Delta) for horizons Δ∈{1,3,5}\Delta \in \{1,3,5\}, where each prediction is obtained by sampling latent noise vectors z∼N(0,I)z \sim \mathcal{N}(0,I). Each column corresponds to a different number of training steps on a logarithmic scale: 20, 60, 200, 600, 2000, 6000, and 20000.

As training begins, the predicted distributions are highly dispersed and often fail to preserve the structure of the PushT scene. In early checkpoints, the sampled futures do not consistently respect the local geometry of the object or the physically plausible displacement induced by the agent-object interaction. As training progresses, the generated samples become increasingly concentrated around coherent future configurations. By approximately 200 to 600 training steps, the model begins to capture the coarse reachable region for each temporal horizon, and later checkpoints further sharpen the predicted distribution while preserving meaningful sample diversity.

Figure 11. Training-time convergence of predicted future-state distributions for the PushT environment

Comparison of ID-WM and naive WM with equivalent frameskip

Figure 12. Visualization of the induced state mappings by ID-WM and naive WM.

We visualize the difference between our proposed ID-WM high-level world model and a naive WM without implicit dynamics. The left column shows the sampled noise/action distribution. In Figure 12, the middle column shows the neighboring states predicted by ID-WM, and the right column shows predictions from a naive WM with frameskip matched to the ID-WM subgoal horizon. ID-WM yields a smooth and locally consistent reachable set. The naive WM produces irregular mappings: nearby samples can jump to distant states, leave the local reachable neighborhood, or form discontinuous regions. This indicates that long-skip prediction with a naive world model is prone to out-of-distribution subgoals and discontinuous action-to-state geometry, making it less suitable for planning.

References

  1. [1]Ha, D., & Schmidhuber, J. (2018). World Models. CoRR, abs/1803.10122. http://arxiv.org/abs/1803.10122
  2. [2]LeCun, Y. (2022). A Path Towards Autonomous Machine Intelligence. In OpenReview.
  3. [3]Janner, M., Du, Y., Tenenbaum, J. B., & Levine, S. (2022). Planning with Diffusion for Flexible Behavior Synthesis. International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA, 162, 9902–9915. https://proceedings.mlr.press/v162/janner22a.htmlproceedings.mlr.press/v162/janner22a.html
  4. [4]Zhou, G., Pan, H., LeCun, Y., & Pinto, L. (2025). DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning. Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, 267. https://proceedings.mlr.press/v267/zhou25t.htmlproceedings.mlr.press/v267/zhou25t.html
  5. [5]Bar, A., Zhou, G., Tran, D., Darrell, T., & LeCun, Y. (2025). Navigation World Models. IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, 15791–15801. https://doi.org/10.1109/CVPR52734.2025.01472
  6. [6]Goswami, R. G., Bar, A., Fan, D., Yang, T.-Y., Zhou, G., Krishnamurthy, P., Rabbat, M., Khorrami, F., & LeCun, Y. (2025). World Models Can Leverage Human Videos for Dexterous Manipulation. CoRR, abs/2512.13644. https://doi.org/10.48550/ARXIV.2512.13644
  7. [7]Park, S., Ghosh, D., Eysenbach, B., & Levine, S. (2023). HIQL: Offline Goal-Conditioned RL with Latent States as Actions. Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/6d7c4a0727e089ed6cdd3151cbe8d8ba-Abstract-Conference.htmlpapers.nips.cc/paper\_files/paper/2023/hash/6d7c4a0727e089ed6cdd3151cbe8d8ba-Abstract-Conference.html
  8. [8]Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B., & Song, S. (2023). Diffusion Policy: Visuomotor Policy Learning via Action Diffusion. Robotics: Science and Systems XIX, Daegu, Republic of Korea, July 10-14, 2023. https://doi.org/10.15607/RSS.2023.XIX.026
  9. [9]Fu, J., Kumar, A., Nachum, O., Tucker, G., & Levine, S. (2020). D4RL: Datasets for Deep Data-Driven Reinforcement Learning. CoRR, abs/2004.07219. https://arxiv.org/abs/2004.07219
  10. [10]Hansen, N., V, J. S., Sobal, V., LeCun, Y., Wang, X., & Su, H. (2025). Hierarchical World Models as Visual Whole-Body Humanoid Controllers. The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. https://openreview.net/forum?id=7wuJMvK639openreview.net/forum?id=7wuJMvK639
  11. [11]Hafner, D., Lillicrap, T. P., Ba, J., & Norouzi, M. (2020). Dream to Control: Learning Behaviors by Latent Imagination. ICLR.
  12. [12]Hafner, D., Lillicrap, T. P., Norouzi, M., & Ba, J. (2021). Mastering Atari with Discrete World Models. ICLR.
  13. [13]Hafner, D., Pasukonis, J., Ba, J., & Lillicrap, T. (2023). Mastering diverse domains through world models. https://arxiv.org/abs/2301.04104
  14. [14]Hafner, D., Yan, W., & Lillicrap, T. (2025). Training agents inside of scalable world models. https://arxiv.org/abs/2509.24527
  15. [15]Sobal, U., Zhang, W., Cho, K., Balestriero, R., Rudner, T. G., & LeCun, Y. (2026). Learning from reward-free offline data: A case for planning with latent dynamics models. Advances in Neural Information Processing Systems, 38, 43905–43941.
  16. [16]Parthasarathy, A., Kalra, N., Agrawal, R., LeCun, Y., Bounou, O., Izmailov, P., & Goldblum, M. (2025). Closing the Train-Test Gap in World Models for Gradient-Based Planning. CoRR, abs/2512.09929. https://doi.org/10.48550/ARXIV.2512.09929
  17. [17]Assran, M., Bardes, A., Fan, D., Garrido, Q., Howes, R., Muckley, M., Rizvi, A., Roberts, C., Sinha, K., Zholus, A., et al. (2025). V-jepa 2: Self-supervised video models enable understanding, prediction and planning. https://arxiv.org/abs/2506.09985
  18. [18]Chen, C., Deng, F., Kawaguchi, K., Gulcehre, C., & Ahn, S. (2024). Simple Hierarchical Planning with Diffusion. The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. https://openreview.net/forum?id=kXHEBK9uAYopenreview.net/forum?id=kXHEBK9uAY
  19. [19]Ko, P.-C., Mao, J., Du, Y., Sun, S.-H., & Tenenbaum, J. B. (2024). Learning to Act from Actionless Videos through Dense Correspondences. The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. https://openreview.net/forum?id=Mhb5fpA1T0openreview.net/forum?id=Mhb5fpA1T0
  20. [20]Kim, A. M., Wu, A., & Zhao, Y. (2025). Hierarchical Diffusion Motion Planning with Task-Conditioned Uncertainty-Aware Priors. https://arxiv.org/abs/2509.25685
  21. [21]Wiedemer, T., Li, Y., Vicol, P., Gu, S. S., Matarese, N., Swersky, K., Kim, B., Jaini, P., & Geirhos, R. (2025). Video models are zero-shot learners and reasoners. https://arxiv.org/abs/2509.20328
  22. [22]Rubinstein, R. Y. (1997). Optimization of computer simulation models with rare events. European Journal of Operational Research, 99(1), 89–112.
  23. [23]Schmidhuber, J. (1991). Learning to generate sub-goals for action sequences. Artificial Neural Networks, 967–972.
  24. [24]Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015). Deep Unsupervised Learning using Nonequilibrium Thermodynamics. Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, 37, 2256–2265. http://proceedings.mlr.press/v37/sohl-dickstein15.htmlproceedings.mlr.press/v37/sohl-dickstein15.html
  25. [25]Ho, J., Jain, A., & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. ArXiv, abs/2006.11239. https://api.semanticscholar.org/CorpusID:219955663api.semanticscholar.org/CorpusID:219955663
  26. [26]Song, J., Meng, C., & Ermon, S. (2021). Denoising Diffusion Implicit Models. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. https://openreview.net/forum?id=St1giarCHLPopenreview.net/forum?id=St1giarCHLP
  27. [27]Zhang, J., Springenberg, J. T., Byravan, A., Hasenclever, L., Abdolmaleki, A., Rao, D., Heess, N. M. O., & Riedmiller, M. A. (2023). Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domains. ArXiv, abs/2302.12617. https://api.semanticscholar.org/CorpusID:257206001api.semanticscholar.org/CorpusID:257206001
  28. [28]Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., & Kavukcuoglu, K. (2017). FeUdal Networks for Hierarchical Reinforcement Learning. Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, 70, 3540–3549. http://proceedings.mlr.press/v70/vezhnevets17a.htmlproceedings.mlr.press/v70/vezhnevets17a.html
  29. [29]Li, J., Tang, C., Tomizuka, M., & Zhan, W. (2022). Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learning. IEEE Robotics Autom. Lett., 7(4), 10216–10223. https://doi.org/10.1109/LRA.2022.3190100
  30. [30]Shi, L. X., Lim, J. J., & Lee, Y. (2022). Skill-based Model-based Reinforcement Learning. Conference on Robot Learning, CoRL 2022, 14-18 December 2022, Auckland, New Zealand, 205, 2262–2272. https://proceedings.mlr.press/v205/shi23a.htmlproceedings.mlr.press/v205/shi23a.html
  31. [31]Bruce, J., Dennis, M. D., Edwards, A., Parker-Holder, J., Shi, Y., Hughes, E., Lai, M., Mavalankar, A., Steigerwald, R., Apps, C., Aytar, Y., Bechtle, S., Behbahani, F. M. P., Chan, S. C. Y., Heess, N., Gonzalez, L., Osindero, S., Ozair, S., Reed, S. E., … Rocktäschel, T. (2024). Genie: Generative Interactive Environments. ICML.
  32. [32]Peng, X., Zheng, Z., Shen, C., Young, T., Guo, X., Wang, B., Xu, H., Liu, H., Jiang, M., Li, W., et al. (2025). Open-sora 2.0: Training a commercial-level video generation model in $200k. https://arxiv.org/abs/2503.09642
  33. [34]Hansen, N., Wang, X., & Su, H. (2022). Temporal Difference Learning for Model Predictive Control. https://arxiv.org/abs/2203.04955
  34. [35]Hansen, N., Su, H., & Wang, X. (2024). TD-MPC2: Scalable, Robust World Models for Continuous Control. ICLR.
  35. [36]Li, L., Zhang, Q., Luo, Y., Yang, S., Wang, R., Han, F., Yu, M., Gao, Z., Xue, N., Zhu, X., et al. (2026). Causal world modeling for robot control. https://arxiv.org/abs/2601.21998
  36. [37]Psenka, M., Rabbat, M., Krishnapriyan, A., LeCun, Y., & Bar, A. (2026). Parallel Stochastic Gradient-Based Planning for World Models. https://arxiv.org/abs/2602.00475
  37. [38]Haramati, D., Qi, C., Daniel, T., Zhang, A., Tamar, A., & Konidaris, G. D. (2026). Hierarchical entity-centric reinforcement learning with factored subgoal diffusion. International Conference on Learning Representations, 2026, 137484–137510.
  38. [39]Robine, J., Höftmann, M., Uelwer, T., & Harmeling, S. (2023). Transformer-based World Models Are Happy With 100k Interactions. ICLR.
  39. [40]Micheli, V., Alonso, E., & Fleuret, F. (2023). Transformers are Sample-Efficient World Models. ICLR.
  40. [41]Bharadhwaj, H., Xie, K., & Shkurti, F. (2020). Model-Predictive Control via Cross-Entropy and Gradient-Based Optimization. ArXiv, abs/2004.08763. https://api.semanticscholar.org/CorpusID:215827996api.semanticscholar.org/CorpusID:215827996
  41. [42]Dasari, S., Mees, O., Zhao, S., Srirama, M. K., & Levine, S. (2025). The Ingredients for Robotic Diffusion Transformers. IEEE International Conference on Robotics and Automation, ICRA 2025, Atlanta, GA, USA, May 19-23, 2025, 15617–15625. https://doi.org/10.1109/ICRA55743.2025.11127493
  42. [43]Li, X. L., Thickstun, J., Gulrajani, I., Liang, P., & Hashimoto, T. (2022). Diffusion-LM Improves Controllable Text Generation. ArXiv, abs/2205.14217. https://api.semanticscholar.org/CorpusID:249192356api.semanticscholar.org/CorpusID:249192356
  43. [44]Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang, J., Lee, J., Guo, Y., et al. (2023). Improving image generation with better captions. Computer Science. Https://cdn. Openai. Com/papers/dall-e-3. Pdf, 2(3), 8.
  44. [45]Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E. L., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., Salimans, T., Ho, J., Fleet, D. J., & Norouzi, M. (2022). Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding. ArXiv, abs/2205.11487. https://api.semanticscholar.org/CorpusID:248986576api.semanticscholar.org/CorpusID:248986576
  45. [46]Li, W., Wang, X., Jin, B., & Zha, H. (2023). Hierarchical Diffusion for Offline Decision Making. International Conference on Machine Learning. https://api.semanticscholar.org/CorpusID:260859191api.semanticscholar.org/CorpusID:260859191
  46. [47]Hao, C., Xiao, A., Xue, Z., & Soh, H. (2025). CHD: Coupled Hierarchical Diffusion for Long-Horizon Tasks. ArXiv, abs/2505.07261. https://api.semanticscholar.org/CorpusID:278502357api.semanticscholar.org/CorpusID:278502357
  47. [48]Ajay, A., Du, Y., Gupta, A., Tenenbaum, J. B., Jaakkola, T., & Agrawal, P. (2022). Is Conditional Generative Modeling all you need for Decision-Making?. ArXiv, abs/2211.15657. https://api.semanticscholar.org/CorpusID:254044710api.semanticscholar.org/CorpusID:254044710
  48. [49]Lu, H., Han, D., Shen, Y., & Li, D. (2025). What Makes a Good Diffusion Planner for Decision Making?. ArXiv, abs/2503.00535. https://api.semanticscholar.org/CorpusID:276741986api.semanticscholar.org/CorpusID:276741986
  49. [50]Nasiriany, S., Pong, V. H., Lin, S., & Levine, S. (2019). Planning with Goal-Conditioned Policies. Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:202779996api.semanticscholar.org/CorpusID:202779996
  50. [51]Nachum, O., Gu, S. S., Lee, H., & Levine, S. (2018). Data-Efficient Hierarchical Reinforcement Learning. Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:43920938api.semanticscholar.org/CorpusID:43920938
  51. [52]Xie, K., Bharadhwaj, H., Hafner, D., Garg, A., & Shkurti, F. (2020). Latent Skill Planning for Exploration and Transfer. International Conference on Learning Representations. https://api.semanticscholar.org/CorpusID:233714270api.semanticscholar.org/CorpusID:233714270
  52. [53]Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Ho, D., Hsu, J., Ibarz, J., Ichter, B., Irpan, A., Jang, E., Ruano, R. M. J., Jeffrey, K., Jesmonth, S., … Yan, M. (2022). Do As I Can, Not As I Say: Grounding Language in Robotic Affordances. Conference on Robot Learning. https://api.semanticscholar.org/CorpusID:247939706api.semanticscholar.org/CorpusID:247939706
  53. [54]Sutton, R. S., Precup, D., & Singh, S. (1999). Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning. Artif. Intell., 112, 181–211. https://api.semanticscholar.org/CorpusID:76564api.semanticscholar.org/CorpusID:76564
  54. [55]Pertsch, K., Lee, Y., & Lim, J. J. (2020). Accelerating Reinforcement Learning with Learned Skill Priors. Conference on Robot Learning. https://api.semanticscholar.org/CorpusID:225039804api.semanticscholar.org/CorpusID:225039804
  55. [56]Dalal, M., Pathak, D., & Salakhutdinov, R. (2021). Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives. Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:240070909api.semanticscholar.org/CorpusID:240070909
  56. [57]Barto, A. G., & Mahadevan, S. (2003). Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems, 13, 41–77. https://api.semanticscholar.org/CorpusID:386824api.semanticscholar.org/CorpusID:386824
  57. [58]Gupta, A., Kumar, V., Lynch, C., Levine, S., & Hausman, K. (2019). Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning. ArXiv, abs/1910.11956. https://api.semanticscholar.org/CorpusID:204955905api.semanticscholar.org/CorpusID:204955905
  58. [59]Chung, J., Ahn, S., & Bengio, Y. (2016). Hierarchical Multiscale Recurrent Neural Networks. ArXiv, abs/1609.01704. https://api.semanticscholar.org/CorpusID:1463401api.semanticscholar.org/CorpusID:1463401
  59. [60]Hansen, N., Su, H., & Wang, X. (2025). Learning Massively Multitask World Models for Continuous Control. ArXiv, abs/2511.19584. https://api.semanticscholar.org/CorpusID:283251537api.semanticscholar.org/CorpusID:283251537
  60. [61]Hu, A., Russell, L., Yeo, H., Murez, Z., Fedoseev, G., Kendall, A., Shotton, J., & Corrado, G. (2023). GAIA-1: A Generative World Model for Autonomous Driving. ArXiv, abs/2309.17080. https://api.semanticscholar.org/CorpusID:263310665api.semanticscholar.org/CorpusID:263310665
  61. [62]Nair, S., & Finn, C. (2019). Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation. ArXiv, abs/1909.05829. https://api.semanticscholar.org/CorpusID:202565422api.semanticscholar.org/CorpusID:202565422
  62. [63]Gumbsch, C., Sajid, N., Martius, G., & Butz, M. V. (2024). Learning Hierarchical World Models with Adaptive Temporal Abstractions from Discrete Latent Dynamics. International Conference on Learning Representations. https://api.semanticscholar.org/CorpusID:268859646api.semanticscholar.org/CorpusID:268859646
  63. [64]Chua, K., Calandra, R., McAllister, R. T., & Levine, S. (2018). Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models. Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:44177328api.semanticscholar.org/CorpusID:44177328
  64. [65]Nguyen, D., & Widrow, B. (1989). The truck backer-upper: an example of self-learning in neural networks. IJCNN.
  65. [66]Zhang, W., Terver, B., Zholus, A., Chitnis, S., Sutaria, H., Assran, M., Balestriero, R., Bar, A., Bardes, A., LeCun, Y., et al. (2026). Hierarchical planning with latent world models. https://arxiv.org/abs/2604.03208
  66. [68]Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, 10674–10685. https://doi.org/10.1109/CVPR52688.2022.01042

Paper details

vlnc:W_ygq38baq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents