betaFeedback

WorksComputer Science

Primal-Dual Representation Learning for Low-Rank Constrained MDPs

Abstract

Representation learning has substantially improved the efficiency of unconstrained continuous-state Markov Decision Processes (MDPs), but its extension to Constrained MDPs (CMDPs) is difficult because the transition representation must be learned while constraint violation is controlled. In this paper, we study representation learning for low-rank CMDPs with continuous state spaces, and propose REP-PD-TEG for the soft cumulative-violation setting. In the feature learning period, the algorithm interacts with the environment using a decaying-εε-greedy mixture execution policy, and uses the collected samples to learn transition features by maximum likelihood. Based on the learned features, it constructs uncertainty bonuses and plans with an optimistic Lagrangian objective to obtain the return policy. We prove regret and soft-violation guarantees for the return policies, and quantify the influence of the decaying-εε-greedy exploration. In particular, REP-PD-TEG achieves the optimal O~(K)\widetilde O(\sqrt K) return policy bound, and gives a O~(K3/4)\widetilde O(K^{3/4}) overall sample bound when exploratory actions are charged. For the stricter hard cumulative violation, where feasibility must be verified episode by episode, we propose REP-PD-H-TEG. It plans under a bonus-corrected lower-confidence utility certificate with the learned representation, and performs a stabilized dual search to obtain the return policy, yielding regret and hard-violation bounds. Together, these results give the first theoretically guaranteed representation-learning method for low-rank continuous-state CMDPs with both soft cumulative guarantees and certified hard-violation control.

Introduction

Constrained Markov Decision Processes (CMDPs) provide a principled framework for constrained reinforcement learning with both reward objectives and safety requirements [4, 22]. This framework is especially relevant in safety-critical domains, and a growing line of provably constrained RL algorithms has been developed within this framework [8, 9, 16, 24, 27, 39]. These works primarily focus on the tabular setting, where guarantees scale with the cardinality of the state space and become unsuitable for continuous or high-dimensional environments. Linear CMDP methods [5, 13, 17, 18, 26, 42] partially address this issue by assuming that transitions are linearly represented by known low-dimensional features, yielding state-dimension-independent bounds. However, existing tabular/linear CMDP methods all rely on a priori knowledge of the latent feature representations. In the absence of such prior feature knowledge, their guarantees no longer apply, leaving CMDPs with unknown features insufficiently addressed.

Representation learning has been effective for discovering low-dimensional transition features in unconstrained MDPs [2, 29, 31, 36, 37, 41, 44]. Extending this idea to CMDPs is more challenging because representation error now affects both reward optimization and constraint evaluation. The

Preprint. learner must identify latent dynamics from online data, use the learned representation to guide exploration, and still control cumulative constraint violation [2, 16]. This creates a coupling between representation coverage and constraint-aware policy optimization that is absent from unconstrained low-rank RL and is not handled by existing CMDP results with known features.

In this paper, we develop primal-dual representation-learning algorithms for low-rank continuous-state CMDPs with unknown transition features. During feature learning, the algorithms interact with the environment via an ϵ\epsilon-greedy mixture execution policy and use the collected samples to learn transition features by maximum-likelihood estimation (MLE). For soft violation criterion, where only the total constraint shortfall across episodes is charged, our algorithm, REP-PD-TEG, uses the learned features to construct uncertainty bonuses, and plan with the bonus-augmented Lagrangian objective that couples reward maximization, utility optimization, and exploration. The algorithm outputs the return policies with a primal-dual update, and admits both return-policy and execution-policy guarantees. For the stricter per-episode hard-violation metric, where each episode’s shortfall is charged separately, we develop REP-PD-H-TEG, which keeps the representation-learning procedure but plans under a tight lower-confidence utility certificate and conducts a stabilized dual search to obtain the return policy. Specifically, we make the following contributions:

  • We propose the first theoretically guaranteed representation-learning method for low-rank continuous-state CMDPs with unknown features. Starting from the occupancy of the current return policy, it uses ϵ\epsilon-greedy auxiliary probes for action coverage, fits transition features by MLE, and constructs a merged-covariance uncertainty bonus. We quantify the effect of decaying-ϵ\epsilon exploration for both return and execution policies.

  • For the aggregate soft-violation criterion, we propose REP-PD-TEG, which updates the return policy through an optimistic Lagrangian whose reward and dual-weighted utility terms are both bonus-augmented based on the learned features. We show that REP-PD-TEG achieves return policy regret and soft violation of order O~(K1/2+p)\widetilde{O}(K^{1/2+p}) sample complexity under the schedule ϵk=k−p\epsilon_k = k^{-p}, where ϵk\epsilon_k is the exploration level in episode kk. In particular, this recovers the optimal O~(K)\widetilde{O}(\sqrt{K}) return policy rate when p=0p = 0, which corresponds to uniform auxiliary probing over actions during the whole procedure. We also prove an execution policy overall bound: executing the ϵk\epsilon_k-greedy exploration adds O(K1−p)O(K^{1-p}) complexity, and the balanced choice p=1/4p = 1/4 yields a O~(K3/4)\widetilde{O}(K^{3/4}) execution policy regret and soft violation up to problem-dependent factors.

  • For the stricter per-episode hard-violation criterion, we propose REP-PD-H-TEG, which uses the learned features to conduct a stabilized dual search and certifies feasibility through a bonus-corrected lower-confidence utility certificate. For return policies, we obtain regret of order O~(K3/4+p)\widetilde{O}(K^{3/4+p}) and hard violation of order O~(K1/2+p)\widetilde{O}(K^{1/2+p}); taking p=0p = 0 again gives uniform probing and the clean O~(K)\widetilde{O}(\sqrt{K}) hard-violation dependence. For execution policies, we show that the cost of ϵk\epsilon_k-greedy exploration is also O(K1−p)O(K^{1-p}) and choosing p=1/8p = 1/8 gives a O~(K7/8)\widetilde{O}(K^{7/8}) regret and hard violation up to problem-dependent factors.

Appendix H provides the empirical study of REP-PD-TEG on a simulated job-scheduling task. Besides, we also implement an approximate neural network version of REP-PD-TEG on the Safety Gym task to validate the effectiveness of our representation learning procedure for a complex state-action space.

Remark on the probing schedule. In tabular and linear CMDPs, the state or feature representation is known, so the analyzed policy is the execution policy. In low-rank CMDPs, the transition features must be learned online, and auxiliary ϵk\epsilon_k-greedy probing is needed to obtain coverage over the state-action space for representation learning. This creates a distinction between the return-policy sequence and the execution policy. The latter is especially relevant in constrained RL because exploratory behavior can incur real training-time costs—for example, robot collisions or excessive contact forces—even when the final return policy is safe. The two REP-PD-TEG rows in Table 1 report, respectively, the sample complexity of the best return policy specialization (p=0p = 0) and the best execution policy specialization (p=1/4p = 1/4), where pp denotes the probing strength with ϵk=k−p\epsilon_k = k^{-p}. Similarly, the two REP-PD-H-TEG rows in Table 2 show the best return policy complexity with p=0p = 0 and the balanced execution policy complexity under the hard specialization p=1/8p = 1/8, respectively.

MDPMethodKnown FeatureKnown Safe Action SetRegretViolation
TabularEfroni et al. [16]YesYesO~(∣S∣3∣A∣H4K)\widetilde{O}\left(\sqrt{|\mathcal{S}|^{3}|\mathcal{A}|H^{4}K}\right)O~(∣S∣3∣A∣H4K)\widetilde{O}\left(\sqrt{|\mathcal{S}|^{3}|\mathcal{A}|H^{4}K}\right)
Bura et al. [9]YesYesO~(∣S∣2∣A∣H6K)\widetilde{O}\left(\sqrt{|\mathcal{S}|^{2}|\mathcal{A}|H^{6}K}\right)00
LinearAmani et al. [5]YesYesO~(d3H4K)\widetilde{O}\left(\sqrt{d^{3}H^{4}K}\right)00
Kitamura et al. [26]YesYesO~(d5H8K)\widetilde{O}\left(\sqrt{d^{5}H^{8}K}\right)00
Wei et al. [42]YesNoO~(d3H4K)\widetilde{O}\left(\sqrt{d^{3}H^{4}K}\right)O~(dH2K)\widetilde{O}\left(\sqrt{dH^{2}K}\right)
Ghosh et al. [18]YesNoO~(d3H4K)\widetilde{O}\left(\sqrt{d^{3}H^{4}K}\right)O~(d3H4K)\widetilde{O}\left(\sqrt{d^{3}H^{4}K}\right)
Low-rankREP-PD-H-TEGNoNoO~(∣A∣d2K3/4(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}K^{3/4}(1-\gamma)^{-2}\right) [return policy]<br>O~(∣A∣d2K7/8(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}K^{7/8}(1-\gamma)^{-2}\right) [execution policy]O~(∣A∣d2K(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}\sqrt{K}(1-\gamma)^{-2}\right) [return policy]<br>O~(∣A∣d2K7/8(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}K^{7/8}(1-\gamma)^{-2}\right) [execution policy]

Table 2.

Table 1: Comparison of soft-violation bounds. The two REP-PD-TEG rows report the complexity of the best return policy specialization and the best execution policy specialization, respectively.

MDPMethodKnown FeatureRegretViolation
TabularEfroni et al. [16]YesO~(∣S∣3∣A∣H4K)\widetilde{O}\left(\sqrt{|\mathcal{S}|^{3}|\mathcal{A}|H^{4}K}\right)O~(∣S∣3∣A∣H4K)\widetilde{O}\left(\sqrt{|\mathcal{S}|^{3}|\mathcal{A}|H^{4}K}\right)
Liu et al. [27]YesO~(∣S∣3∣A∣H6K)\widetilde{O}\left(\sqrt{|\mathcal{S}|^{3}|\mathcal{A}|H^{6}K}\right)00
LinearDing et al. [13]YesO~(d2H6K)\widetilde{O}\left(\sqrt{d^{2}H^{6}K}\right)O~(d2H6K)\widetilde{O}\left(\sqrt{d^{2}H^{6}K}\right)
Ghosh et al. [17]YesO~(d3H3K)\widetilde{O}\left(\sqrt{d^{3}H^{3}K}\right)O~(d3H3K)\widetilde{O}\left(\sqrt{d^{3}H^{3}K}\right)
Ghosh et al. [17]YesO~(d3H5K)\widetilde{O}\left(\sqrt{d^{3}H^{5}K}\right)00
Low-rankREP-PD-TEGNoO~(∣A∣d2K(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}\sqrt{K}(1-\gamma)^{-2}\right) [return policy]<br>O~(∣A∣d2K3/4(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}K^{3/4}(1-\gamma)^{-2}\right) [execution policy]O~(∣A∣d2K(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}\sqrt{K}(1-\gamma)^{-2}\right) [return policy]<br>O~(∣A∣d2K3/4(1−γ)−2)\widetilde{O}\left(|\mathcal{A}|d^{2}K^{3/4}(1-\gamma)^{-2}\right) [execution policy]

Table 1.

Table 2: Comparison of hard-violation bounds. The two REP-PD-H-TEG rows report the complexity of the best return policy specialization and the balanced execution policy specialization, respectively.

Related Work

Provably Constrained RL CMDPs provide a rigorous framework for safe RL, balancing reward maximization with cumulative cost constraints [4, 19]. Several practical constrained RL methods have been developed on this foundation, including policy-gradient, Lagrangian, shielding, and model-based approaches [1, 6, 7, 11, 21, 35]. On the theoretical side, there has been a growing body of work on provably safe reinforcement learning. Many works [8, 9, 14, 16, 24, 25, 27, 39] consider the tabular setting. Among the best known regret and constraint-violation guarantees are of order O~(K)\widetilde{O}(\sqrt{K}) [16, 27]. However, their polynomial dependence on the cardinality of state space ∣S∣|\mathcal{S}| makes them unsuitable for large or continuous state spaces.

References

  1. [1]Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In International conference on machine learning, pages 22–31. PMLR, 2017.arxiv.org/abs/1705.10528
  2. [2]Alekh Agarwal, Sham Kakade, Akshay Krishnamurthy, and Wen Sun. Flambe: Structural complexity and representation learning of low rank mdps. Advances in neural information processing systems, 33:20095–20107, 2020.
  3. [3]Alekh Agarwal, Sham M Kakade, Jason D Lee, and Gaurav Mahajan. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. Journal of Machine Learning Research, 22(98):1–76, 2021.
  4. [4]Eitan Altman. Constrained Markov decision processes. Routledge, 2021.DOI
  5. [5]Sanae Amani, Christos Thrampoulidis, and Lin Yang. Safe reinforcement learning with linear function approximation. In International Conference on Machine Learning, pages 243–253. PMLR, 2021.arxiv.org/abs/2106.06239
  6. [6]Yarden As, Ilnura Usmanova, Sebastian Curi, and Andreas Krause. Constrained policy optimization via bayesian world models. arXiv preprint arXiv:2201.09802, 2022.arxiv.org/abs/2201.09802
  7. [7]Arko Banerjee, Kia Rahmani, Joydeep Biswas, and Isil Dillig. Dynamic model predictive shielding for provably safe reinforcement learning. Advances in Neural Information Processing Systems, 37:100131–100159, 2024.arxiv.org/abs/2405.13863
  8. [8]Kianté Brantley, Miro Dudik, Thodoris Lykouris, Sobhan Mryoosefi, Max Simchowitz, Aleksandrs Slivkins, and Wen Sun. Constrained episodic reinforcement learning in concave-convex and knapsack settings. Advances in Neural Information Processing Systems, 33:16315–16326, 2020.arxiv.org/abs/2006.05051
  9. [9]Archana Bura, Aria HasanzadeZonouzy, Dileep Kalathil, Srinivas Shakkottai, and Jean-Francois Chamberland. Dope: Doubly optimistic and pessimistic exploration for safe reinforcement learning. Advances in neural information processing systems, 35:1047–1059, 2022.arxiv.org/abs/2112.00885
  10. [10]Yuan Cheng, Ruiquan Huang, Jing Yang, and Yingbin Liang. Improved sample complexity for reward-free reinforcement learning under low-rank mdps. arXiv preprint arXiv:2303.10859, 2023.arxiv.org/abs/2303.10859
  11. [11]Juntao Dai, Jiaming Ji, Long Yang, Qian Zheng, and Gang Pan. Augmented proximal policy optimization for safe reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7288–7295, 2023.DOI
  12. [12]Dongsheng Ding, Kaiqing Zhang, Tamer Basar, and Mihailo R Jovanovic. Natural policy gradient primal-dual method for constrained markov decision processes. In NeurIPS, 2020.
  13. [13]Dongsheng Ding, Xiaohan Wei, Zhuoran Yang, Zhaoran Wang, and Mihailo Jovanovic. Provably efficient safe exploration via primal-dual policy optimization. In International Conference on Artificial Intelligence and Statistics, pages 3304–3312. PMLR, 2021.arxiv.org/abs/2003.00534
  14. [14]Dongsheng Ding, Chen-Yu Wei, Kaiqing Zhang, and Alejandro Ribeiro. Last-iterate convergent policy gradient primal-dual methods for constrained mdps. Advances in Neural Information Processing Systems, 36:66138–66200, 2023.arxiv.org/abs/2306.11700
  15. [15]Simon Du, Akshay Krishnamurthy, Nan Jiang, Alekh Agarwal, Miroslav Dudik, and John Langford. Provably efficient rl with rich observations via latent state decoding. In International Conference on Machine Learning, pages 1665–1674. PMLR, 2019.arxiv.org/abs/1901.09018
  16. [16]Yonathan Efroni, Shie Mannor, and Matteo Pirotta. Exploration-exploitation in constrained mdps. arXiv preprint arXiv:2003.02189, 2020.arxiv.org/abs/2003.02189
  17. [17]Arnob Ghosh, Xingyu Zhou, and Ness Shroff. Provably efficient model-free constrained rl with linear function approximation. Advances in Neural Information Processing Systems, 35: 13303–13315, 2022.arxiv.org/abs/2206.11889
  18. [18]Arnob Ghosh, Xingyu Zhou, and Ness Shroff. Towards achieving sub-linear regret and hard constraint violation in model-free rl. In International Conference on Artificial Intelligence and Statistics, pages 1054–1062. PMLR, 2024.
  19. [19]Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, and Alois Knoll. A review of safe reinforcement learning: Methods, theories and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.DOI
  20. [20]Ruiquan Huang, Jing Yang, and Yingbin Liang. Safe exploration incurs nearly no additional sample complexity for reward-free rl. arXiv preprint arXiv:2206.14057, 2022.arxiv.org/abs/2206.14057
  21. [21]Weidong Huang, Jiaming Ji, Chunhe Xia, Borong Zhang, and Yaodong Yang. Safedreamer: Safe reinforcement learning with world models. arXiv preprint arXiv:2307.07176, 2023.arxiv.org/abs/2307.07176
  22. [22]Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhongshao He, Jiayi Zhou, Zhaowei Zhang, et al. Ai alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852, 2023.arxiv.org/abs/2310.19852
  23. [23]Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, John Langford, and Robert E Schapire. Contextual decision processes with low bellman rank are pac-learnable. In International Conference on Machine Learning, pages 1704–1713. PMLR, 2017.
  24. [24]Krishna C Kalagarla, Rahul Jain, and Pierluigi Nuzzo. A sample-efficient algorithm for episodic finite-horizon mdp with constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 8030–8037, 2021.arxiv.org/abs/2009.11348
  25. [25]Toshinori Kitamura, Tadashi Kozuno, Masahiro Kato, Yuki Ichihara, Soichiro Nishimori, Akiyoshi Sannai, Sho Sonoda, Wataru Kumagai, and Yutaka Matsuo. A policy gradient primal-dual algorithm for constrained mdps with uniform pac guarantees. arXiv preprint arXiv:2401.17780, 2024.arxiv.org/abs/2401.17780
  26. [26]Toshinori Kitamura, Arnob Ghosh, Tadashi Kozuno, Wataru Kumagai, Kazumi Kasaura, Kenta Hoshino, Yohei Hosoe, and Yutaka Matsuo. Provably efficient rl under episode-wise safety in linear cmdps. arXiv preprint arXiv:2502.10138, 2025.arxiv.org/abs/2502.10138
  27. [27]Tao Liu, Ruida Zhou, Dileep Kalathil, Panganamala Kumar, and Chao Tian. Learning policies with zero or bounded constraint violation for constrained mdps. Advances in Neural Information Processing Systems, 34:17183–17193, 2021.arxiv.org/abs/2106.02684
  28. [28]Dipendra Misra, Mikael Henaff, Akshay Krishnamurthy, and John Langford. Kinematic state abstraction and provably efficient rich-observation reinforcement learning. In International conference on machine learning, pages 6961–6971. PMLR, 2020.arxiv.org/abs/1911.05815
  29. [29]Aditya Modi, Jinglin Chen, Akshay Krishnamurthy, Nan Jiang, and Alekh Agarwal. Model-free representation learning and exploration in low-rank mdps. Journal of Machine Learning Research, 25(6):1–76, 2024.
  30. [30]Gergely Neu and Ciara Pike-Burke. A unifying view of optimism in episodic reinforcement learning. Advances in Neural Information Processing Systems, 33:1392–1403, 2020.arxiv.org/abs/2007.01891
  31. [31]Chengzhou Ni, Yuda Song, Xuezhou Zhang, Chi Jin, and Mengdi Wang. Representation learning for general-sum low-rank markov games. arXiv preprint arXiv:2210.16976, 2022.arxiv.org/abs/2210.16976
  32. [32]Santiago Paternain, Miguel Calvo-Fullana, Luiz FO Chamon, and Alejandro Ribeiro. Safe policies for reinforcement learning via primal-dual methods. IEEE Transactions on Automatic Control, 68(3):1321–1336, 2022.arxiv.org/abs/1911.09101
  33. [33]Shuang Qiu, Xiaohan Wei, Zhuoran Yang, Jieping Ye, and Zhaoran Wang. Upper confidence primal-dual optimization: Stochastically constrained markov decision processes with adversarial losses and unknown transitions. arXiv preprint arXiv:2003.00660, 2020.
  34. [34]Shuang Qiu, Lingxiao Wang, Chenjia Bai, Zhuoran Yang, and Zhaoran Wang. Contrastive ucb: Provably efficient contrastive self-supervised learning in online reinforcement learning. In International Conference on Machine Learning, pages 18168–18210. PMLR, 2022.arxiv.org/abs/2207.14800
  35. [35]Alex Ray, Joshua Achiam, and Dario Amodei. Benchmarking safe exploration in deep reinforcement learning. arXiv preprint arXiv:1910.01708, 7(1):2, 2019.
  36. [36]Tongzheng Ren, Chenjun Xiao, Tianjun Zhang, Na Li, Zhaoran Wang, Sujay Sanghavi, Dale Schuurmans, and Bo Dai. Latent variable representation for reinforcement learning. arXiv preprint arXiv:2212.08765, 2022.arxiv.org/abs/2212.08765
  37. [37]Tongzheng Ren, Tianjun Zhang, Lisa Lee, Joseph E Gonzalez, Dale Schuurmans, and Bo Dai. Spectral decomposition representation for reinforcement learning. arXiv preprint arXiv:2208.09515, 2022.arxiv.org/abs/2208.09515
  38. [38]Tongzheng Ren, Tianjun Zhang, Csaba Szepesvári, and Bo Dai. A free lunch from the noise: Provable and practical exploration for representation learning. In Uncertainty in Artificial Intelligence, pages 1686–1696. PMLR, 2022.arxiv.org/abs/2111.11485
  39. [39]Rahul Singh, Abhishek Gupta, and Ness B Shroff. Learning in markov decision processes under constraints. arXiv preprint arXiv:2002.12435, 2020.DOI
  40. [40]Wen Sun, Nan Jiang, Akshay Krishnamurthy, Alekh Agarwal, and John Langford. Model-based rl in contextual decision processes: Pac bounds and exponential improvements over model-free approaches. In Conference on learning theory, pages 2898–2933. PMLR, 2019.arxiv.org/abs/1811.08540
  41. [41]Masatoshi Uehara, Xuezhou Zhang, and Wen Sun. Representation learning for online and offline rl in low-rank mdps. arXiv preprint arXiv:2110.04652, 2021.arxiv.org/abs/2110.04652
  42. [42]Honghao Wei, Xin Liu, and Lei Ying. Safe reinforcement learning with instantaneous constraints: the role of aggressive exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21708–21716, 2024.arxiv.org/abs/2312.14470
  43. [43]Yunchang Yang, Tianhao Wu, Han Zhong, Evrard Garcelon, Matteo Pirotta, Alessandro Lazaric, Liwei Wang, and Simon S Du. A reduction-based framework for conservative bandits and reinforcement learning. arXiv preprint arXiv:2106.11692, 2021.arxiv.org/abs/2106.11692
  44. [44]Hongming Zhang, Tongzheng Ren, Chenjun Xiao, Dale Schuurmans, and Bo Dai. Provable representation with efficient planning for partially observable reinforcement learning. arXiv preprint arXiv:2311.12244, 2023.
  45. [45]Tianjun Zhang, Tongzheng Ren, Mengjiao Yang, Joseph Gonzalez, Dale Schuurmans, and Bo Dai. Making linear mdps practical via contrastive representation learning. In International Conference on Machine Learning, pages 26447–26466. PMLR, 2022.
  46. [46]Xuezhou Zhang, Yuda Song, Masatoshi Uehara, Mengdi Wang, Alekh Agarwal, and Wen Sun. Efficient reinforcement learning in block mdps: A model-free representation learning approach. In International Conference on Machine Learning, pages 26517–26547. PMLR, 2022.arxiv.org/abs/2202.00063
  47. [47]Chenhao Zhou and Jiyu Wei. Toward online robust zero-sum markov games with function approximation. arXiv preprint arXiv:2610.26286, 2026. Accepted at NeurIPS 2026.
  48. [48]Chenhao Zhou, Zebang Shen, Hanbin Zhao, Hui Qian, et al. Solving zero-sum markov games with continuous state via spectral dynamic embedding. Advances in Neural Information Processing Systems, 37:68622–68657, 2024.

Paper details

vlnc:W_6wbh8dhr
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents