betaFeedback

WorksComputer Science

A Separation Principle for Cooperative Multi-Agent Reinforcement Learning

Abstract

We study cooperative multi-agent reinforcement learning (MARL) problems in which agents evolve under decoupled noisy dynamics but share a population objective. A canonical example is a mobility operator that orchestrates a fleet of vehicles to meet customer demand. The stochastic and time-varying nature of these problems calls for learning approaches, yet existing MARL algorithms do not scale: learning assignment and routing jointly over the full state-action space becomes prohibitive as the number of agents grows. Formulating the problem over the space of probability measures, we prove a separation principle: the population cost-to-go function is upper bounded by an optimal transport problem whose transportation cost is the target-conditioned single-agent cost-to-go function. This simplifies the multi-agent problem to learning a single-agent policy and coordinating the agents via optimal transport. Building on this principle, we introduce SALT (Separation-based Assignment and Learning via optimal Transport), a family of algorithms that learn a target-conditioned single-agent policy using any standard reinforcement learning algorithm, and then coordinate the agents via optimal transport. Since learning occurs at the single-agent level, SALT scales to fleet sizes where MARL baselines become intractable and transfers without retraining to fleet sizes unseen during training. In a vehicle-delivery experiment on the southern Manhattan road network, where MARL baselines are computationally prohibitive, SALT improves travel time by 15-21% over a shortest-path routing heuristic. In stochastic grid worlds, SALT reduces average travel times by 23-61% relative to MARL baselines and scales to population sizes where these methods fail.

Introduction

Consider a fleet of autonomous vehicles serving delivery requests in a city. At each decision time, the operator must simultaneously decide which vehicle should serve which request and how each vehicle should navigate to its assigned destination. These decisions are tightly coupled. The best assignment depends on how long each vehicle will actually take to reach each target, while routing itself depends on the assignment. Assignments based only on nominal shortest paths can therefore be poor when travel times depend on traffic lights, congestion, and random delays. Yet solving this full joint assignment-routing problem over the fleet quickly becomes computationally prohibitive. Beyond delivery fleets, the same structure arises across a wide range of cooperative systems: inspection drones achieving coverage patterns [1], warehouse robots repositioning to incoming orders [2], emergency response units relocating to optimal positions [3], ride-hailing platforms balancing supply and demand [4], and electric-vehicle fleets sharing charging infrastructure [5]. In all these settings, the goal is to optimally coordinate a fleet of decoupled agents so that their collective terminal state matches or approximates a target configuration, such as uniform coverage, a set of delivery locations, or a demand-weighted positioning pattern. These problems are typically solved in a receding-horizon fashion to account for changing conditions and request patterns and, thus, require modular approaches that generalize to different fleet sizes or target distributions.

Fleet assignment and routing are classically addressed via combinatorial solvers [6, 7] and, more recently, learned routing heuristics [8]. Since the joint assignment-routing problem is combinatorial in the fleet size, these methods decouple the two steps using nominal travel costs, such as shortest-path distances, to solve the assignment problem. These heuristics, however, ignore the stochasticity of the costs and may favor targets that “look close” but are expensive to reach in expectation. A natural learning-based alternative is multi-agent reinforcement learning (MARL) [9, 10, 11, 12], often with centralized training and decentralized execution. Policy-gradient methods such as MAPPO [13] and MADDPG [14], and value-decomposition methods such as VDN [15], QMIX [16], and QPLEX [17] are effective in many settings [18, 19], but learn over a joint state-action space whose size grows exponentially with the number of agents [20], limiting scalability and generalization to unseen fleet sizes. Value-decomposition methods gain tractability by imposing architectural assumptions on the joint value function, such as additivity or monotonicity, which are provably too restrictive even for simple coordination tasks [21, 16]. Mean-field reinforcement learning [22, 23] instead replaces pairwise interactions with an aggregate representation of the population. Although practical in large systems [4, 24], its approximations can lose fidelity when modeling finite populations [25]. Closer to our motivation, some methods learn latent subtasks and dynamically assign them among agents [26], since in general cooperative MARL no such decomposition is available a priori. We instead study a structured subclass, with decoupled dynamics and a terminal optimal transport cost, in which the assignment-routing separation can be exploited directly.

In this work, we seek a principled separation of population-level assignment and single-agent control (Figure Figure 1). Inspired by the “deterministic” separation principle for multi-agent control in [27], we formulate the cooperative MARL problem as an optimal control problem over the space of probability measures where terminal deviation from a desired target measure is quantified via an optimal transport cost. By construction, this representation is permutation invariant and captures both finite populations (as empirical measures) and a continuum of agents (as continuous measures). In the deterministic setting, Terpin et al. [27] show that the population cost-to-go is exactly an optimal transport cost whose transportation cost is the single-agent cost-to-go, thus separating assignment from single-agent control. This equality does not extend directly to the stochastic setting, where the population measure is random and future assignments can adapt to realized noise. Nevertheless, we show that a carefully designed separation principle, where noise is “averaged out”, continues to upper bound the population cost-to-go. This analysis is non-trivial: the randomness of the population measure requires handling probability measures over probability measures, a modeling challenge in itself, compounded by delicate measurability issues. Finally, Terpin et al. [27] provide a purely theoretical analysis, whereas we leverage our separation principle to design scalable algorithms for MARL.

Overview of SALT

Figure 1. Overview of SALT. (1) Cooperative MARL learns a joint fleet policy and scales poorly with NN. (2) Formulating the problem over probability measures replaces agent labels with an empirical population μt\mu_t and target distribution ν\nu, but remains intractable. (3) The separation principle decouples assignment from routing, upper bounding the population cost-to-go by an optimal transport assignment with cost j^(s,z)\hat{j}(s,z). SALT assigns targets through this optimal transport problem and routes agents with a shared single-agent policy π\pi, yielding scalable and sample-efficient learning.

Contributions. Our contributions are as follows:

  1. We cast the cooperative multi-agent control problem with homogeneous agents, decoupled stochastic dynamics, and a terminal optimal transport formation cost as an optimal control problem over probability measures in the spirit of the work of Terpin et al. [27]. This removes agent labels, makes the formulation permutation invariant, and applies to both finite populations and a continuum of agents.

  1. We prove a separation principle for this optimal control problem (Theorem 3.1, Part I). The population cost-to-go is upper bounded by an optimal transport problem whose transportation cost is the target-conditioned single-agent cost-to-go. This separates target allocation from routing, which can be solved independently of the fleet size. The upper bound becomes exact in the deterministic setting, recovering [27 Corollary 4.2]. We then bound the gap between the two sides of Theorem 3.1, Part I in terms of the noise of the single-agent dynamics (Theorem D.4), and the loss of optimality under weak population interactions (Theorem E.6). Importantly, our proof is constructive and yields an implementable policy.

  1. This separation principle yields SALT, a family of MARL algorithms that learn a shared target-conditioned single-agent policy and coordinate agents via optimal transport. SALT is agnostic to the single-agent algorithm and generalizes to population sizes not seen during training.

  1. We validate SALT on a time-varying southern Manhattan road network with 2087 intersections and 4255 directed roads. SALT trained with N=20N = 20 vehicles transfers without retraining to fleets of 1–100 vehicles and reduces travel time by 15–21% relative to a nominal shortest-path heuristic. In controlled stochastic grid worlds, the SALT variants reduce the average travel time by 23–61% relative to the standard MARL baselines, consistently across different single-agent reinforcement learning (RL) algorithms. Additionally, SALT improves sample efficiency: with PPO [28], it reaches the same reach-rate milestones as MAPPO with up to 480×480\times fewer environment interactions, and scales to N=128N = 128, while MAPPO fails at N=8N = 8 within a 2×1092 \times10^{9}-interaction budget.

Notation. We work with Polish spaces, Borel functions, and Borel probability measures. For a Polish space XX, P(X)\mathcal{P}(X) denotes the space of probability measures on XX. We write δx\delta_x for the Dirac measure at x∈Xx \in X. The projection from X×YX \times Y onto XX is denoted by proj⁡XX×Y\operatorname{proj}_{X}^{X \times Y}. For μ∈P(X)\mu\in\mathcal{P}(X) and F:X→YF : X \to Y, the pushforward of μ\mu through FF, F#μ∈P(Y)F_{\#}\mu\in\mathcal{P}(Y), is defined by F#μ(B)=μ(F−1(B))F_{\#}\mu(B) = \mu(F^{-1}(B)) for all Borel sets B⊆YB \subseteq Y. A coupling between the probability measures μ∈P(X)\mu\in\mathcal{P}(X) and ν∈P(Y)\nu\in\mathcal{P}(Y) is a joint probability measure with marginals μ\mu and ν\nu; the set of couplings between μ\mu and ν\nu is Γ(μ,ν)\Gamma(\mu,\nu). For a nonnegative cost c:X×Y→R≥0c : X \times Y \to\mathbb{R}_{\ge0}, the optimal transport cost between μ\mu and ν\nu is

K[c](μ,ν):=inf⁡γ∈Γ(μ,ν)∫X×Yc(x,y) dγ(x,y).\mathcal{K}[c](\mu,\nu) := \inf_{\gamma\in\Gamma(\mu,\nu)} \int_{X \times Y} c(x,y)\,\mathrm{d}\gamma(x,y).

If k:X→P(Y)k : X \to\mathcal{P}(Y) is a Markov kernel, μ⊗k\mu\otimes k denotes the induced joint law on X×YX \times Y. In particular, if π(dx,dy)=μ(dx)k(x;dy)\pi(\mathrm{d}x,\mathrm{d}y) = \mu(\mathrm{d}x)k(x;\mathrm{d}y) is a disintegration, we write π=μ⊗kx\pi= \mu\otimes k^x.

Problem setting

We consider cooperative MARL problems in which NN homogeneous agents coordinate to reach a desired configuration over a finite horizon TT, while incurring agent-level stage costs and a population-level terminal cost. Let SS, AA, WW, and ZZ denote the state, action, noise, and target spaces. While Z=SZ = S for canonical target matching, we keep ZZ distinct to account for lower-dimensional targets or task-level descriptors, such as destinations when the state also includes velocity or battery level.

Agent-based formulation

Each agent ii has state sti∈Ss_t^i \in S and chooses action ati∈Aa_t^i \in A. Its noisy dynamics are st+1i=ft(sti,ati,wti)s_{t+1}^i = f_t(s_t^i, a_t^i, w_t^i), where wti∼ζt(⋅∣sti,ati)w_t^i \sim\zeta_t(\cdot\mid s_t^i, a_t^i) and ξt(⋅∣sti,ati)∈P(W)\xi_t(\cdot\mid s_t^i, a_t^i) \in\mathcal{P}(W) gives the conditional law of the noise affecting an agent in state ss taking action aa. We consider three noise models, which differ in how noise is correlated across agents: individual noise, where noise is sampled independently across agents; local noise, where agents with the same state-action pair share the same noise realization; and global noise, where all agents share a common noise realization (see Appendix § 9). Let st:=(st1,…,stN)s_t := (s_t^1,\ldots,s_t^N) and at:=(at1,…,atN)a_t := (a_t^1,\ldots,a_t^N) denote the ordered states and actions of the agents. We take the control cost to be the sum of agent-level stage costs gt:S×A×W→R‾≥0g_t:S \times A \times\mathbb{W} \to\overline{\mathbb{R}}_{\ge0} and a population-level terminal formation cost. Given a transportation cost gT:S×Z→R‾≥0g_T:S \times Z \to\overline{\mathbb{R}}_{\ge0}, the terminal cost compares the final population distribution μT=1N∑i=1NδsTi\mu_T = \frac{1}{N}\sum_{i=1}^{N}\delta_{s_T^i} to a target configuration ν∈P(Z)\nu\in\mathcal{P}(Z) through the optimal transport cost

K[gT](μT,ν)=inf⁡γ∈Γ(μT,ν)∫S×ZgT(s,z)dγ(s,z).(1)\mathcal{K}[g_T](\mu_T,\nu)=\inf_{\gamma\in\Gamma(\mu_T,\nu)}\int_{S\times Z}g_T(s,z)\mathrm{d}\gamma(s,z). \tag*{(1)}

For gT(s,z)=∥s−z∥pg_T(s,z)=\lVert s-z\rVert^p with p≥1p\ge1, K[gT](μT,ν)\mathcal{K}[g_T](\mu_T,\nu) is the pp-th power of the type-pp Wasserstein distance. Since this is a fully observed cooperative problem, it can be cast as a multi-agent Markov decision process (MDP) [29], which reduces to a single-agent MDP over the joint action space. By standard MDP theory [30, 31], we can restrict attention, without loss of optimality, to deterministic Markov policies π=(πt)t=0T−1\pi=(\pi_t)_{t=0}^{T-1}, universally measurable on general Borel state and action spaces [31 Ch. 8]. We denote this class by Π\Pi. Throughout, we assume that, given the current states and actions, the noise is independent of the past, so the law of the noise sequence (wt)t=0T−1(w_t)_{t=0}^{T-1}, wt:=(wt1,…,wtN)w_t := (w_t^1,\ldots,w_t^N), under which expectations are taken, is determined by the dynamics, the policy, and the noise model.

Problem 1. Given initial states sˉ0=(sˉ01,…,sˉ0N)\bar{s}_0=(\bar{s}_0^1,\ldots,\bar{s}_0^N) and a target distribution ν\nu, learn a policy π∈Π\pi\in\Pi minimizing the expected cumulative cost

JN(sˉ0,ν)=inf⁡π∈ΠE(wt)t=0T−1[1N∑i=1N∑t=0T−1gt(sti,ati,wti)+K[gT](μT,ν)]J^N(\bar{s}_0,\nu)=\inf_{\pi\in\Pi}\mathbb{E}_{(w_t)_{t=0}^{T-1}}\left[\frac{1}{N}\sum_{i=1}^{N}\sum_{t=0}^{T-1}g_t(s_t^i,a_t^i,w_t^i)+\mathcal{K}[g_T](\mu_T,\nu)\right]

s.t.  st+1i=ft(sti,ati,wti),s0=sˉ0.\ s_{t+1}^i=f_t(s_t^i,a_t^i,w_t^i),\quad s_0=\bar{s}_0.

Population-based formulation

We generalize Problem 1 at the population level by replacing the ordered state sts_t with the empirical measure μt=1N∑i=1Nδsti∈P(S)\mu_t=\frac{1}{N}\sum_{i=1}^{N}\delta_{s_t^i}\in\mathcal{P}(S), which discards agent labels and makes the permutation invariance in the problem explicit. We initially allow general population states μt∈P(S)\mu_t\in\mathcal{P}(S) and we will later specialize this formulation back to the finite-agent setting.

Population control. A population-level control must specify how the mass at each state is split among actions. Such a control is a state-action distribution λt≡λt(μt)∈P(S×A)\lambda_t\equiv\lambda_t(\mu_t)\in\mathcal{P}(S\times A) whose SS-marginal is the current population state: (proj⁡S)#λt=μt(\operatorname{proj}_{S})_{\#}\lambda_t=\mu_t. For example, if all agents follow a common deterministic policy πt:S→A\pi_t:S\to A, then λt=(Id⁡S,πt)#μt\lambda_t=(\operatorname{Id}_S,\pi_t)_{\#}\mu_t, that is, all the mass that μt\mu_t places at state ss is assigned to the state-action pair (s,πt(s))(s,\pi_t(s)).

References

  1. [1]Wei Jing, Di Deng, Yan Wu, and Kenji Shimada. Multi-UAV coverage path planning for the inspection of large and complex structures. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1480–1486, 2020.arxiv.org/abs/2007.13065
  2. [2]T. Lamballais, M. Merschformann, D. Roy, M. B. M. de Koster, K. Azadeh, and L. Suhl. Dynamic policies for resource reallocation in a robotic mobile fulfillment system with time-varying demand. European Journal of Operational Research, 300(3):937–952, 2022.
  3. [3]Matthew S. Maxwell, Mateo Restrepo, Shane G. Henderson, and Huseyin Topaloglu. Approximate dynamic programming for ambulance redeployment. INFORMS Journal on Computing, 22(2):266–281, 2010.DOI
  4. [4]Minne Li, Zhiwei Qin, Yan Jiao, Yaodong Yang, Jun Wang, Chenxi Wang, Guobin Wu, and Jieping Ye. Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning. In The World Wide Web Conference, pages 983–994, 2019.arxiv.org/abs/1901.11454
  5. [5]Zhongjing Ma, Duncan S. Callaway, and Ian A. Hiskens. Decentralized charging control of large populations of plug-in electric vehicles. IEEE Transactions on Control Systems Technology, 21(1):67–78, 2013.
  6. [6]Gioele Zardini, Nicolas Lanzetti, Marco Pavone, and Emilio Frazzoli. Analysis and control of autonomous mobility-on-demand systems. Annual Review of Control, Robotics, and Autonomous Systems, 5(1):633–658, 2022.
  7. [7]Javier Alonso-Mora, Samitha Samaranayake, Alex Wallar, Emilio Frazzoli, and Daniela Rus. On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment. Proceedings of the National Academy of Sciences, 114(3):462–467, 2017.
  8. [8]Wouter Kool, Herke van Hoof, and Max Welling. Attention, learn to solve routing problems! In International Conference on Learning Representations, 2019.arxiv.org/abs/1803.08475
  9. [9]Kaiqing Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A selective overview of theories and algorithms. In Handbook of Reinforcement Learning and Control, pages 321–384. Springer, Cham, 2021.arxiv.org/abs/1911.10635
  10. [10]Aleksandar Krnjaic, Raul D. Steleac, Jonathan D. Thomas, Georgios Papoudakis, Lukas Schäfer, Andrew Wing Keung To, Kuan-Ho Lao, Murat Cubuktepe, Matthew Haley, Peter Börsting, and Stefano V. Albrecht. Scalable multi-agent reinforcement learning for warehouse logistics with robotic and human co-workers. In IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 677–684, 2024.arxiv.org/abs/2212.11498
  11. [11]Kaixiang Lin, Renyu Zhao, Zhe Xu, and Jiayu Zhou. Efficient large-scale fleet management via multi-agent deep reinforcement learning. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1774–1783, 2018.DOI
  12. [12]Yi Shen, Benjamin McClosky, Joseph W. Durham, and Michael M. Zavlanos. Multi-agent reinforcement learning for resource allocation in large-scale robotic warehouse sortation centers. In IEEE Conference on Decision and Control, pages 7137–7143, 2023.DOI
  13. [13]Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of PPO in cooperative multi-agent games. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, volume 35, pages 24611–24624, 2022.arxiv.org/abs/2103.01955
  14. [14]Ryan Lowe, Yi Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. Multi-agent actor-critic for mixed cooperative–competitive environments. In Advances in Neural Information Processing Systems, volume 30, pages 6379–6390, 2017.arxiv.org/abs/1706.02275
  15. [15]Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinícius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, and Thore Graepel. Value-decomposition networks for cooperative multi-agent learning based on team reward. In International Conference on Autonomous Agents and Multiagent Systems, pages 2085–2087, 2018.DOI
  16. [16]Tabish Rashid, Mikayel Samvelyan, Christian Schroeder de Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research, 21(178):1–51, 2020.
  17. [17]Jianhao Wang, Zhizhou Ren, Terry Liu, Yang Yu, and Chongjie Zhang. QPLEX: Duplex dueling multi-agent Q-learning. In International Conference on Learning Representations, 2021.arxiv.org/abs/2008.01062
  18. [18]Georgios Papoudakis, Filippos Christianos, Lukas Schäfer, and Stefano V. Albrecht. Benchmarking multi-agent deep reinforcement learning algorithms in cooperative tasks. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2021.arxiv.org/abs/2006.07869
  19. [19]Abubakr O. Al-Abbasi, Arnob Ghosh, and Vaneet Aggarwal. DeepPool: Distributed model-free algorithm for ride-sharing using deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 20(12):4714–4727, 2019.
  20. [20]Lucian Buşoniu, Robert Babuška, and Bart De Schutter. A comprehensive survey of multi-agent reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(2):156–172, 2008.
  21. [21]Kyung hwan Son, Daewoo Kim, Wan Ju Kang, David Earl Hostallero, and Yung Yi. QTRAN: Learning to factorize with transformation for cooperative multi-agent reinforcement learning. In International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 5887–5896, 2019.arxiv.org/abs/1905.05408
  22. [22]Yaodong Yang, Rui Luo, Minne Li, Ming Zhou, Weinan Zhang, and Jun Wang. Mean field multi-agent reinforcement learning. In International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5571–5580, 2018.arxiv.org/abs/1802.05438
  23. [23]Mathieu Laurière, Sarah Perrin, Julien Pérolat, Sertan Girgin, Paul Muller, Romuald Élie, Matthieu Geist, and Olivier Pietquin. Learning in mean field games: A survey. arXiv preprint arXiv:2205.12944, 2022.arxiv.org/abs/2205.12944
  24. [24]Enshu Wang, Rong Ding, Zhaoxing Yang, Haiming Jin, Chenglin Miao, Lu Su, Fan Zhang, Chunming Qiao, and Xinbing Wang. Joint charging and relocation recommendation for e-taxi drivers via multi-agent mean field hierarchical reinforcement learning. IEEE Transactions on Mobile Computing, 21(4):1274–1290, 2022.
  25. [25]Batuhan Yardim, Artur Goldman, and Niao He. When is mean-field reinforcement learning tractable and relevant? In International Conference on Autonomous Agents and Multiagent Systems, pages 2038–2046, 2024.
  26. [26]Mingyu Yang, Jian Zhao, Xunhan Hu, Wengang Zhou, Jiangcheng Zhu, and Houqiang Li. LDSA: Learning dynamic subtask assignment in cooperative multi-agent reinforcement learning. In Advances in Neural Information Processing Systems, volume 35, pages 1698–1710, 2022.
  27. [27]Antonio Terpin, Nicolas Lanzetti, and Florian Dörfler. Dynamic programming in probability spaces via optimal transport. SIAM Journal on Control and Optimization, 62(2):1183–1206, 2024.DOI
  28. [28]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.arxiv.org/abs/1707.06347
  29. [29]Craig Boutilier. Planning, learning and coordination in multiagent decision processes. In Conference on Theoretical Aspects of Rationality and Knowledge, pages 195–210, 1996.
  30. [30]Martin L. Puterman. Markov decision processes: Discrete stochastic dynamic programming. Wiley, New York, 1994.
  31. [31]Dimitri P. Bertsekas and Steven E. Shreve. Stochastic optimal control: The discrete-time case. Athena Scientific, Belmont, MA, 1996.
  32. [32]Charalambos D. Aliprantis and Kim C. Border. Infinite dimensional analysis: A hitchhiker’s guide. Springer, Berlin, 3 edition, 2006.
  33. [33]Christopher J. C. H. Watkins and Peter Dayan. Q-learning. Machine Learning, 8(3–4):279–292, 1992.DOI
  34. [34]Hado van Hasselt, Arthur Guez, and David Silver. Deep reinforcement learning with double Q-learning. In AAAI Conference on Artificial Intelligence, pages 2094–2100, 2016.arxiv.org/abs/1509.06461
  35. [35]Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1928–1937, 2016.
  36. [36]Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems, volume 26, pages 2292–2300, 2013.
  37. [37]Christian Schroeder de Witt, Tarun Gupta, Denys Makoviichuk, Viktor Makoviychuk, Philip H. S. Torr, Mingfei Sun, and Shimon Whiteson. Is independent learning all you need in the StarCraft multi-agent challenge? arXiv preprint arXiv:2011.09533, 2020.arxiv.org/abs/2011.09533
  38. [38]Jakub Grudzien Kuba, Ruiqing Chen, Muning Wen, Ying Wen, Fanglei Sun, Jun Wang, and Yaodong Yang. Trust region policy optimisation in multi-agent reinforcement learning. In International Conference on Learning Representations, 2022.arxiv.org/abs/2109.11251
  39. [39]OpenStreetMap contributors. OpenStreetMap. https://www.openstreetmap.org, 2025. Planet dump retrieved from https://planet.osm.org.
  40. [40]Paul R. Halmos. Measure theory. Graduate Texts in Mathematics. Springer, New York, 1974.
  41. [41]R. M. Dudley. Real analysis and probability, volume 74 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 2 edition, 2002.
  42. [42]Patrick Billingsley. Probability and measure. Wiley Series in Probability and Statistics. Wiley, Hoboken, NJ, anniversary edition, 2012.
  43. [43]Kenneth L. Judd. The law of large numbers with a continuum of IID random variables. Journal of Economic Theory, 35(1):19–25, 1985.DOI
  44. [44]Yeneng Sun. The exact law of large numbers via Fubini extension and characterization of insurable risks. Journal of Economic Theory, 126(1):31–69, 2006.DOI
  45. [45]Alexander S. Kechris. Classical descriptive set theory, volume 156 of Graduate Texts in Mathematics. Springer, New York, 1995.DOI
  46. [46]Cédric Villani. Optimal transport: Old and new, volume 338 of Grundlehren der mathematischen Wissenschaften. Springer, Berlin, 2009.
  47. [47]Garrett Birkhoff. Tres observaciones sobre el álgebra lineal. Universidad Nacional de Tucumán, Revista, Serie A, 5:147–151, 1946.
  48. [48]Liviu Aolaritei, Nicolas Lanzetti, Hongruyu Chen, and Florian Dörfler. Distributional uncertainty propagation via optimal transport. IEEE Transactions on Automatic Control, 70(9):6080–6095, 2025.arxiv.org/abs/2205.00343
  49. [49]Diederik P. Kingma and Jimmy Ba. Adam: a method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015. arXiv:1412.6980.arxiv.org/abs/1412.6980
  50. [50]Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: an imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.arxiv.org/abs/1912.01703
  51. [51]John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. Scalable parallel programming with CUDA. Queue, 6(2):40–53, 2008. doi: 10.1145/1365490.1365500.DOI
  52. [52]James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Yash Katariya, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs. http://github.com/jax-ml/jax, 2018.

Paper details

vlnc:W_qw8c47vq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents