Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems
(2024) 6th Annual Conference on Learning for Dynamics and Control
Files
hoppe24a.pdf
Open Access - Adobe PDF
- 406.37 KB
Details
- Authors
- Abstract
- We study vehicle dispatching in autonomous mobility on demand (AMoD) systems, where a cen- tral operator assigns vehicles to customer requests or rejects these with the aim of maximizing its total profit. Recent approaches use multi-agent deep reinforcement learning (MADRL) to re- alize scalable yet performant algorithms, but train agents based on local rewards, which distorts the reward signal with respect to the system-wide profit, leading to lower performance. We there- fore propose a novel global-rewards-based MADRL algorithm for vehicle dispatching in AMoD systems, which resolves so far existing goal conflicts between the trained agents and the oper- ator by assigning rewards to agents leveraging a counterfactual baseline. Our algorithm shows statistically significant improvements across various settings on real-world data compared to state- of-the-art MADRL algorithms with local rewards. We further provide a structural analysis which shows that the utilization of global rewards can improve implicit vehicle balancing and demand forecasting abilities. An extended version of our paper, including an appendix, can be found at https://arxiv.org/abs/2312.08884. Our code is available at https://github. com/tumBAIS/GR-MADRL-AMoD.
- Affiliations
Citations
Hoppe, H., Enders, T., Cappart, Q., & Schiffer, M. (2024). Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems. Proceedings of Machine Learning Research 6th Annual Conference on Learning for Dynamics and Control, 242, 260-272. https://hdl.handle.net/2078.5/272653 (Original work published 2024)