betaFeedback

WorksComputer Science

Latent Action Policies from Suboptimal Data

Abstract

Latent Action Models (LAMs) learn control representations from action-free video, but existing pipelines typically train latent policies with behavioral cloning, without an explicit policy improvement objective. Although offline RL offers a natural approach to policy improvement, standard methods do not transfer directly to learned latent action spaces. We find that the structure of learned latent actions makes naive offline RL unreliable: directly maximizing value over latent actions or fitting a unimodal policy can produce latent actions that cannot be realized by physical execution, while the latent representation can merge physical controls that receive different rewards. To understand when latent-space optimization remains valid, we derive conditions under which policy improvement transfers to decoded execution and under which latent and physical optima align. Building on these insights, we propose LAPIS for goal-conditioned latent policy improvement, grounding value-based policy improvement in observed state-latent transitions and using a conditional flow policy to capture their multimodal structure. LAPIS outperforms baselines on most tasks, with the clearest gains in visual manipulation, and remains effective even with only 1% action labels, where competing methods largely collapse.

Introduction

Latent Action Models (LAMs) have emerged as a promising paradigm for learning control representations from action-free video by inferring latent actions that explain observed state transitions [39, 8, 7]. Because latent-action learning captures transition and outcome structure rather than task optimality, it can naturally exploit large collections of diverse and suboptimal video, including failed trajectories that would be poor demonstrations but remain informative for learning how the environment changes. Yet existing LAM pipelines typically train the resulting latent policies with behavioral cloning (BC) [36, 39, 7, 4], without explicitly optimizing latent policies for improved task return. This creates a gap between the broad data that LAMs can learn from and the policies that current pipelines can extract from it.

Offline reinforcement learning (RL) offers a natural way to improve beyond the observed behavior [23, 22, 13, 21, 26], but applying standard offline RL directly in a learned latent action space is not straightforward. We identify three properties that distinguish learned latent actions from physical actions. The first two concern realizability—whether the transition outcome represented by a latent action can be physically produced from the current state: (i) state-conditioned realizability, whereby a latent action realizable at one state can be unrealizable at another; and (ii) non-convex realizable set, whereby interpolating between realizable latent actions can yield unrealizable ones. The third, (iii) outcome-based abstraction, arises because distinct physical controls that reach the same outcome can map to the same latent action.

These properties expose two distinct failure modes of naive offline RL in latent action spaces. The first is an optimization–execution mismatch. Because the realizable latent set is state-dependent and potentially non-convex, standard offline RL can move outside this set by directly maximizing latent-action value or through unimodal policy fitting, producing latent actions that cannot be faithfully realized by physical execution. This raises the question: when does latent-space policy improvement transfer to decoded physical execution? The second is an objective mismatch. Because outcome-based abstraction can merge physical controls with different rewards, the optimal latent policy need not correspond to the optimal physical policy. This raises a separate question: when do the latent and physical policy optima align?

Figure 1. Overview of LAPIS. Left: a Latent Action Model learns latent actions from action-free trajectories and a decoder from limited action labels. Middle: naive offline RL can produce unrealizable latent actions through value maximization or unimodal policy fitting, while outcome-based abstraction can merge reward-distinct controls. Right: LAPIS performs support-aware policy improvement with critic-guided filtering and a multimodal flow policy, instantiated with an outcome-compatible goal-conditioned objective.

To answer these questions, we analyze paired physical- and latent-action batch MDPs constructed from the same observed transitions. For the first question, we show that batch-constrained latent policy improvement transfers to physical execution under deterministic dynamics and supported outcome identifiability. For the second, we show that latent and physical optima align under an outcome-determined objective within the decoder-covered, batch-constrained physical policy class. These results characterize when latent-space policy improvement remains valid and provide the principles behind our method.

Building on these insights, we introduce LAPIS (Latent Action Policy Improvement on Support), a latent policy improvement framework motivated by these failure modes. Rather than directly optimizing a latent actor toward arbitrary high-value actions, LAPIS grounds value-based policy improvement in observed state–latent transitions and uses a conditional flow policy to capture their potentially multimodal structure. LAPIS is instantiated with a goal-conditioned objective based on future states, which is naturally compatible with the outcome semantics captured by latent actions. At inference, classifier-free guidance provides a simple way to adjust the emphasis between behavior and high-value policy branches. With the same LAM and 5% action labels, LAPIS achieves the highest success among latent-space methods on all five OGBench visual manipulation tasks and outperforms prior semi-supervised offline RL baselines on nearly all tasks with only 5% or 1% action labels. Further experiments examine the contributions of these design choices.

Related Work

Latent action models from action-free video. A growing body of work learns controllable representations from action-free video by inferring latent actions that explain observed transitions (Edwards et al., 2019; Rybkin et al., [35]; Baker et al., [3]; Bruce et al., [4]; Google DeepMind, [17]). Recent approaches learn such representations using discrete or continuous latent codes and extend latent action modeling to richer video and vision-language-action settings (Schmidt & Jiang, [36]; Ye et al., [39]; Chen et al., [7]; Gao et al., [14]; Zhang et al., [40]; Chen et al., [8]). Their control pipelines predominantly train latent policies by behavioral cloning and use a small labeled set to decode latent actions into physical controls. LAPO (Schmidt & Jiang, [36]) additionally incorporates reinforcement learning by applying online PPO to fine-tune the physical action policy after latent-action pretraining. In contrast, we study policy improvement directly in the learned latent action space using offline data, with scarce action labels used for physical action decoding.

Action-free and semi-supervised reinforcement learning. A related line of work studies learning control from trajectories with missing or limited action annotations, using inferred actions, latent representations, or other forms of auxiliary supervision to exploit action-free experience (Torabi et al., [37]; Baker et al., [3]; Ghosh et al., [16]; Park et al., [29]; Zheng et al., [42]). Most closely related is LAQ (Chang et al., [6]), which performs Q-learning over latent actions from state-only data but relies on a hand-specified controller for execution. In contrast, we learn a latent action space from action-free data together with a physical-action decoder from scarce action labels, and study offline policy improvement directly in this learned latent action space. This setting makes the relationship between latent-space optimization and decoded physical execution central, motivating our analysis of when latent policy improvement transfers to physical control. Further related work is discussed in Appendix § 9.

Preliminaries

Goal-conditioned reinforcement learning We model the environment as an MDP M=(S,A,p,r,γ)\mathcal{M} = (\mathcal{S}, \mathcal{A}, p, r, \gamma), with dynamics p(s′∣s,a)p(s' \mid s, a), bounded reward rr, and initial-state distribution μ0\mu_0. We write (s,a,s′)=(st,at,st+1)(s, a, s') = (s_t, a_t, s_{t+1}) for a transition. A policy π(a∣s)\pi(a \mid s) induces the usual state-action and state value functions QMπQ^\pi_{\mathcal{M}} and VMπV^\pi_{\mathcal{M}}, with expected return JM(π)=Es0∼μ0[VMπ(s0)]J_{\mathcal{M}}(\pi) = \mathbb{E}_{s_0 \sim\mu_0}[V^\pi_{\mathcal{M}}(s_0)]. We refer to M\mathcal{M} as the ground MDP to distinguish it from the batch MDPs introduced in § 5. In the goal-conditioned setting, the policy additionally conditions on a goal gg, π(a∣s,g)\pi(a \mid s, g). Contrastive RL (Eysenbach et al., [10]) casts goal-conditioned control as density-ratio estimation, bypassing explicit modeling of transition dynamics or reward functions. The goal-reaching reward is (1−γ)p(s′=g∣s,a)(1-\gamma)p(s'=g \mid s,a), and the corresponding goal-conditioned value equals the discounted future-state occupancy:

Qgπ(s,a)=pπ(st+k=g∣s,a)=(1−γ)∑k=1∞γk−1pπ(st+k=g∣st=s,at=a),(1)Q^\pi_g(s,a) = p^\pi(s_{t+k}=g \mid s,a) = (1-\gamma)\sum_{k=1}^{\infty}\gamma^{k-1}p^\pi(s_{t+k}=g \mid s_t=s,a_t=a), \tag*{(1)}

where the right-hand side denotes the discounted occupancy of future states visited from (s,a)(s,a) under π\pi. To illustrate the contrastive principle, consider a critic parameterized as a bilinear score fϕ(s,a,g)=ϕ(s,a)Tψ(g)/df_\phi(s,a,g) = \phi(s,a)^\mathsf{T}\psi(g)/\sqrt{d} and trained by noise-contrastive binary classification, with a positive goal g+g^+ sampled from the future-state occupancy and a negative goal g−∼pBg^- \sim p_\mathcal{B} sampled from the dataset state distribution,

max⁡ϕ,ψE(s,a),g+,g−[log⁡σ(fϕ(s,a,g+))+log⁡(1−σ(fϕ(s,a,g−)))].(2)\max_{\phi,\psi}\mathbb{E}_{(s,a),g^+,g^-}\left[\log\sigma(f_\phi(s,a,g^+))+\log(1-\sigma(f_\phi(s,a,g^-)))\right]. \tag*{(2)}

For the ideal binary classification problem, the optimal score is the log density ratio, f∗=log⁡Qgπ−log⁡pB(g)f^* = \log Q^\pi_g - \log p_\mathcal{B}(g) wherever the ratio is defined. LAPIS uses a TD-InfoNCE estimator, detailed in Appendix § 13.§ 3. In the offline setting, directly maximizing this critic can drive the actor outside the support of the fixed behavior data, so offline CRL regularizes policy improvement with behavioral cloning,

LCRL(π)=E(s,a,g)∼D[−fϕ(s,π(s,g),g)+α∥π(s,g)−a∥22],\mathcal{L}_{\mathrm{CRL}}(\pi) = \mathbb{E}_{(s,a,g)\sim\mathcal{D}}\left[-f_\phi(s,\pi(s,g),g)+\alpha\left\lVert\pi(s,g)-a\right\rVert_2^2\right],

improving the actor toward high-value actions while keeping it close to the observed behavior.

Latent action models A Latent Action Model (LAM) learns a latent action space ZZ from an action-free dataset B={(si,si′)}i=1N\mathcal{B}=\{(s_i,s'_i)\}_{i=1}^{N} by training an inverse dynamics model II and a forward dynamics model FF, with z=I(s,s′)z=I(s,s') and s′=F(s,z)s'=F(s,z) with a bottlenecked latent representation,

min⁡I,FE(s,s′)∼B[∥s′−F(s,I(s,s′))∥22].(3)\min_{I,F}\mathbb{E}_{(s,s')\sim\mathcal{B}}\left[\left\lVert s'-F(s,I(s,s'))\right\rVert_2^2\right]. \tag*{(3)}

Relabeling each transition by its latent action zi=I(si,si′)z_i = I(s_i, s'_i) yields the latent dataset Bz={(si,zi,si′)}i=1N\mathcal{B}^z = \{(s_i, z_i, s'_i)\}_{i=1}^{N}. A separate small labeled dataset Bdec={(si,ai,si′)}i=1Ndec\mathcal{B}^{\mathrm{dec}} = \{(s_i, a_i, s'_i)\}_{i=1}^{N_{\mathrm{dec}}} with Ndec≪NN_{\mathrm{dec}} \ll N supplies the action supervision for the decoder.

The deployed system composes a latent policy with a decoder. The latent policy πz(z∣s)\pi_z(z \mid s) selects a latent transition at the current state, and the state-marginal decoder Ψ(a∣z)\Psi(a \mid z) maps the chosen latent to a physical action, trained on Bdec\mathcal{B}^{\mathrm{dec}} by max⁡ΨEBdec[log⁡Ψ(a∣z)]\max_{\Psi} \mathbb{E}_{\mathcal{B}^{\mathrm{dec}}}[\log\Psi(a \mid z)] and frozen thereafter. Deployment samples z∼πz(⋅∣s)z \sim\pi_z(\cdot\mid s) then a∼Ψ(⋅∣z)a \sim\Psi(\cdot\mid z), giving the deployed policy πˉ=Ψ∘πz\bar{\pi} = \Psi\circ\pi_z. Section 5 introduces an ideal state-conditioned batch decoder Ψ∗(a∣s,z)\Psi^*(a \mid s, z) only for the transfer analysis.

Latent Action Structure and Failures

A latent policy selects a latent action representing a desired transition outcome, which must ultimately be realized through physical control at deployment. Section 4.1 characterizes the realizability and abstraction structure of learned latent actions. Section 4.2 then shows how realizability can break the connection between latent-space optimization and physical execution, while outcome abstraction can misalign latent and physical objectives.

Structural properties of learned latent action spaces

To quantify whether a target latent can be physically realized from the current state, we define its realization error as the infimum of its distance to the latents of reachable outcomes. We use it as a diagnostic (Figure 2, Appendix B) and to bound the transfer gap outside the data (Appendix D.9).

Figure 2. Structural properties of learned latent action spaces. Top row: concept schematics for (a) state-conditioned realizability (the same latent can be realizable at s1s_1 but not s2s_2), (b) the non-convex realizable set (interpolating two realizable latents can pass through unrealizable ones), and (c) outcome-based abstraction (distinct physical controls reaching the same successor share one latent). Bottom row: empirical diagnostics for the experimental LAM on visual-cube-single-play-v0; the diagnostic protocol is in Appendix § 10.

Definition 1 (Latent realizability). Let R(s):=⋃a∈Asupp⁡p(⋅∣s,a)R(s) := \bigcup_{a \in\mathcal{A}} \operatorname{supp} p(\cdot\mid s, a) be the set of successor states reachable from ss. The latent realization error of a target latent zz at state ss is

δ(s,z)=inf⁡s′∈R(s)∥I(s,s′)−z∥2,\delta(s, z) = \inf_{s' \in R(s)} \lVert I(s, s') - z \rVert_2,

and zz is ϵ\epsilon-realizable at ss if δ(s,z)≤ϵ\delta(s, z) \le\epsilon, defining the realizable set Zrealϵ(s)={z:δ(s,z)≤ϵ}\mathcal{Z}_{\mathrm{real}}^{\epsilon}(s) = \{z : \delta(s, z) \le\epsilon\}.

The observed set Bz\mathcal{B}^z contains only a subset of realizable state–latent pairs; absence from Bz\mathcal{B}^z does not imply unrealizability. Figure 2 illustrates the properties discussed below.

State-conditioned realizability Whether a latent is realizable depends on the current state: the same zz can be realizable from s1s_1 but not s2s_2, so realizability is a property of the pair (s,z)(s, z), not of zz alone (Zrealϵ(s1)≠Zrealϵ(s2)\mathcal{Z}_{\mathrm{real}}^{\epsilon}(s_1) \ne\mathcal{Z}_{\mathrm{real}}^{\epsilon}(s_2)). In Figure 2(a), we take latents with known realizations at their source states and search for physical realizations of the same latents from other states. Latents have near-zero realization error at their source states, while 73% of transfers to other states exceed the diagnostic threshold under finite search. LAPO ([36 Sec. 4.1]) notes that the action behind a latent can depend on the state; this is harmless when imitating the latents observed at ss, but offline RL moves away from them and can select latents that are realizable only from other states.

Non-convex realizable set Even at a fixed state, the space between two realizable latents need not be realizable. For z1,z2∈Zrealϵ(s)z_1, z_2 \in\mathcal{Z}_{\mathrm{real}}^{\epsilon}(s), an interpolation zα=(1−α)z1+αz2z_{\alpha} = (1-\alpha)z_1 + \alpha z_2 need not be realizable, so zα∉Zrealϵ(s)z_{\alpha} \notin\mathcal{Z}_{\mathrm{real}}^{\epsilon}(s) for α∈(0,1)\alpha\in(0,1). In Figure 2(b), we select pairs of realizable endpoint latents at the same state and search for physical realizations along the interpolation between them. The endpoints admit known physical realizations, while interior errors exceed the diagnostic threshold.

Outcome-based abstraction The encoder z=I(s,s′)z = I(s, s') labels the transition outcome rather than the physical command, so distinct actions reaching the same successor from ss share a latent. In Figure 2(c), we search for substantially different action chunks that reach each target outcome within a fixed tolerance; 95% of sampled targets admit at least two such realizations.

Why the properties break naive offline RL

These properties become consequential when offline RL is applied in the learned latent space, giving rise to two distinct failure modes: an optimization–execution mismatch and an objective mismatch. Optimization–execution mismatch State-conditioned realizability and a non-convex realizable set imply that physically realizable latent actions depend on the current state, while interpolations between realizable latents can be unrealizable. As a result, directly optimizing the actor to maximize learned value can move it toward unrealizable latents despite behavior regularization, while a unimodal actor can place probability between separated realizable modes and generate unrealizable latents. Thus, even when the critic favors a high-value latent action zz, if zz is unrealizable, decoding and physical execution can reach a different successor than the outcome represented by zz. Consequently, latent-space policy improvement need not translate to improvement after decoding,

References

  1. [1]Anurag Ajay, Aviral Kumar, Pulkit Agrawal, Sergey Levine, and Ofir Nachum. OPAL: Offline primitive discovery for accelerating offline reinforcement learning. In International Conference on Learning Representations (ICLR), 2021.arxiv.org/abs/2010.13611
  2. [2]Kavosh Asadi, Dipendra Misra, and Michael L. Littman. Lipschitz continuity in model-based reinforcement learning. In International Conference on Machine Learning (ICML), 2018.arxiv.org/abs/1804.07193
  3. [3]Bowen Baker, Ilge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video PreTraining (VPT): Learning to act by watching unlabeled online videos. In Advances in Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2206.11795
  4. [4]Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feyal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, and Tim Rocktäschel. Genie: Generative interactive environments. In International Conference on Machine Learning (ICML), 2024.arxiv.org/abs/2402.15391
  5. [5]Théo Cachet, Christopher R. Dance, and Olivier Sigaud. Bridging environments and language with rendering functions and vision-language models. In International Conference on Machine Learning (ICML), volume 235 of Proceedings of Machine Learning Research, pp. 5135–5188, 2024.
  6. [6]Matthew Chang, Arjun Gupta, and Saurabh Gupta. Learning value functions from undirected state-only experience. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2204.12458
  7. [7]Xiaoyu Chen, Junliang Guo, Tianyu He, Chuheng Zhang, Pushi Zhang, Derek Cathera Yang, Li Zhao, and Jiang Bian. IGOR: Image-GOal representations are the atomic control units for foundation models in embodied AI. arXiv preprint arXiv:2411.00785, 2024.arxiv.org/abs/2411.00785
  8. [8]Xiaoyu Chen, Hangxing Wei, Pushi Zhang, Chuheng Zhang, Kaixin Wang, Yanjiang Guo, Rushuai Yang, Yucen Wang, Xinquan Xiao, Li Zhao, Jianyu Chen, and Jiang Bian. villa-X: Enhancing latent action modeling in vision-language-action models. arXiv preprint arXiv:2507.23682, 2025.arxiv.org/abs/2507.23682
  9. [10]Benjamin Eysenbach, Tianjun Zhang, Sergey Levine, and Ruslan Salakhutdinov. Contrastive learning as goal-conditioned reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2022.arxiv.org/abs/2206.07568
  10. [11]Kevin Frans, Seohong Park, Pieter Abbeel, and Sergey Levine. Diffusion guidance is a controllable policy improvement operator. arXiv preprint arXiv:2505.23458, 2025.arxiv.org/abs/2505.23458
  11. [12]Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219, 2020.arxiv.org/abs/2004.07219
  12. [13]Scott Fujimoto, David Meger, and Doina Precup. Off-policy deep reinforcement learning without exploration. In International Conference on Machine Learning (ICML), 2019.arxiv.org/abs/1812.02900
  13. [14]Shenyuan Gao, Siyuan Zhou, Yilun Du, Jun Zhang, and Chuang Gan. AdaWorld: Learning adaptable world models with latent actions. In International Conference on Machine Learning (ICML), 2025.arxiv.org/abs/2503.18938
  14. [15]Carles Gelada, Saurabh Kumar, Jacob Buckman, Ofir Nachum, and Marc G. Bellemare. DeepMDP: Learning continuous latent space models for representation learning. In International Conference on Machine Learning (ICML), 2019.arxiv.org/abs/1906.02736
  15. [16]Dibya Ghosh, Chethan Anand Bhateja, and Sergey Levine. Reinforcement learning from passive data via latent intentions. In International Conference on Machine Learning (ICML), 2023.arxiv.org/abs/2304.04782
  16. [17]Google DeepMind. Genie 2: A large-scale foundation world model. Technical report, Google DeepMind, 2024.
  17. [18]Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning. In Proceedings of the Conference on Robot Learning, volume 100 of Proceedings of Machine Learning Research, pp. 1025–1037. PMLR, 2020.arxiv.org/abs/1910.11956
  18. [19]Philippe Hansen-Estruch, Ilya Kostrikov, Michael Janner, Jakub Grudzien Kuba, and Sergey Levine. IDQL: Implicit Q-learning as an actor-critic method with diffusion policies. arXiv preprint arXiv:2304.10573, 2023.
  19. [20]Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022.arxiv.org/abs/2207.12598
  20. [21]Donghyeon Ki, JunHyeok Oh, Seong-Woong Shim, and Byung-Jun Lee. Prior-guided diffusion planning for offline reinforcement learning. In Advances in Neural Information Processing Systems (NeurIPS), 2025.
  21. [22]Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit Q-learning. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2110.06169
  22. [23]Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643, 2020.arxiv.org/abs/2005.01643
  23. [24]Lihong Li, Thomas J. Walsh, and Michael L. Littman. Towards a unified theory of state abstraction for MDPs. In Proceedings of the International Symposium on Artificial Intelligence and Mathematics, 2006.
  24. [25]Yaron Lipman, Ricky T O Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2210.02747
  25. [26]Haofei Lu, Dongqi Han, Yifei Shen, and Dongsheng Li. What makes a good diffusion planner for decision making? In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2503.00535
  26. [27]Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. Learning latent plans from play. In Conference on Robot Learning (CoRL), 2020.arxiv.org/abs/1903.01973
  27. [28]Vivek Myers, Andre He, Kuan Fang, Homer Walke, Philippe Hansen-Estruch, Ching-An Cheng, Mihai Jalobeanu, Andrey Kolobov, Anca Dragan, and Sergey Levine. Goal representations for instruction following: A semi-supervised language interface to control. In Conference on Robot Learning (CoRL), 2023.arxiv.org/abs/2307.00117
  28. [29]Seohong Park, Dibya Ghosh, Benjamin Eysenbach, and Sergey Levine. HIQL: Offline goal-conditioned RL with latent states as actions. In Advances in Neural Information Processing Systems (NeurIPS), 2023.arxiv.org/abs/2307.11949
  29. [30]Seohong Park, Kevin Frans, Benjamin Eysenbach, and Sergey Levine. OGBench: Benchmarking offline goal-conditioned RL. In International Conference on Learning Representations (ICLR), 2025a.arxiv.org/abs/2410.20092
  30. [31]Seohong Park, Qiyang Li, and Sergey Levine. Flow Q-learning. In International Conference on Machine Learning (ICML), 2025b.arxiv.org/abs/2502.02538
  31. [32]Xue Bin Peng, Aviral Kumar, Grace Zhang, and Sergey Levine. Advantage-weighted regression: Simple and scalable off-policy reinforcement learning. arXiv preprint arXiv:1910.0177, 2019.
  32. [33]Karl Pertsch, Youngwoon Lee, and Joseph J Lim. Accelerating reinforcement learning with learned skill priors. In Conference on Robot Learning (CoRL), 2020.arxiv.org/abs/2010.11944
  33. [34]Balaraman Ravindran and Andrew G. Barto. SMDP homomorphisms: An algebraic approach to abstraction in semi-markov decision processes. In Proceedings of the 18th International Joint Conference on Artificial Intelligence, pp. 1011–1016, 2003.
  34. [35]Oleh Rybkin, Karl Pertsch, Konstantinos G Derpanis, Kostas Daniilidis, and Andrew Jaegle. Learning what you can do before doing anything. In International Conference on Learning Representations (ICLR), 2019.arxiv.org/abs/1806.09655
  35. [36]Dominik Schmidt and Minqi Jiang. Learning to act without actions. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2312.10812
  36. [37]Faraz Torabi, Garrett Warnell, and Peter Stone. Behavioral cloning from observation. In International Joint Conference on Artificial Intelligence (IJCAI), 2018.arxiv.org/abs/1805.01954
  37. [38]Tongzhou Wang, Antonio Torralba, Phillip Isola, and Amy Zhang. Optimal goal-reaching reinforcement learning via quasimetr ic learning. In International Conference on Machine Learning (ICML), 2023.arxiv.org/abs/2304.01203
  38. [39]Seonghyeon Ye, Joel Jang, Byeongguk Jeon, Sejune Joo, Jianwei Yang, Baolin Peng, Ajay Mandlekar, Reuben Tan, Yu-Wei Chao, Bill Yuchen Lin, Lars Liden, Kimin Lee, Jianfeng Gao, Luke Zettlemoyer, Dieter Fox, and Minjoon Seo. Latent action pretraining from videos. In International Conference on Learning Representations (ICLR), 2025.arxiv.org/abs/2410.11758
  39. [40]Chuheng Zhang, Tim Pearce, Pushi Zhang, Kaixin Wang, Xiaoyu Chen, Wei Shen, Li Zhao, and Jiang Bian. What do latent action models actually learn? In Advances in Neural Information Processing Systems (NeurIPS), 2025.arxiv.org/abs/2506.15691
  40. [41]Chongyi Zheng, Ruslan Salakhutdinov, and Benjamin Eysenbach. Contrastive difference predictive coding. In International Conference on Learning Representations (ICLR), 2024.arxiv.org/abs/2310.20141
  41. [42]Qingqing Zheng, Mikael Henaff, Brandon Amos, and Aditya Grover. Semi-supervised offline reinforcement learning with action-free trajectories. In International Conference on Machine Learning (ICML), 2023.arxiv.org/abs/2210.06518
  42. [43]Wenxuan Zhou, Sujay Bajracharya, and David Held. PLAS: Latent action space for offline reinforcement learning. In Conference on Robot Learning (CoRL), 2020.arxiv.org/abs/2011.07213

Paper details

vlnc:W_aczn65gn
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents