skip to main content
10.1145/3308558.3314139acmotherconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Published: 13 May 2019 Publication History

Abstract

Traffic signal control is an emerging application scenario for reinforcement learning. Besides being as an important problem that affects people's daily life in commuting, traffic signal control poses its unique challenges for reinforcement learning in terms of adapting to dynamic traffic environment and coordinating thousands of agents including vehicles and pedestrians. A key factor in the success of modern reinforcement learning relies on a good simulator to generate a large number of data samples for learning. The most commonly used open-source traffic simulator SUMO is, however, not scalable to large road network and large traffic flow, which hinders the study of reinforcement learning on traffic scenarios. This motivates us to create a new traffic simulator CityFlow with fundamentally optimized data structures and efficient algorithms. CityFlow can support flexible definitions for road network and traffic flow based on synthetic and real-world data. It also provides user-friendly interface for reinforcement learning. Most importantly, CityFlow is more than twenty times faster than SUMO and is capable of supporting city-wide traffic simulation with an interactive render for monitoring. Besides traffic signal control, CityFlow could serve as the base for other transportation studies and can create new possibilities to test machine learning methods in the intelligent transportation domain.

References

[1]
Martin Fellendorf and Peter Vortisch. 2010. Microscopic traffic flow simulator VISSIM. In Fundamentals of traffic simulation. Springer, 63-93.
[2]
Wenzel Jakob, Jason Rhinelander, and Dean Moldovan. 2017. pybind11 - Seamless operability between C++11 and Python. https://github.com/pybind/pybind11.
[3]
Daniel Krajzewicz and Jakob Erdmann. 2013. Road intersection model in SUMO. In 1st SUMO User Conference-SUMO, Vol. 21. 212-220.
[4]
Stefan Krauβ. 1998. Microscopic modeling of traffic flow: Investigation of collision free vehicle dynamics. Ph.D. Dissertation. Universitat zu Koln.
[5]
Li Li, Yisheng Lv, and Fei-Yue Wang. 2016. Traffic signal timing via deep reinforcement learning. IEEE/CAA Journal of Automatica Sinica 3, 3 (2016), 247-254.
[6]
Yuxi Li. 2017. Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274(2017).
[7]
Pablo Alvarez Lopez, Michael Behrisch, Laura Bieker-Walz, Jakob Erdmann, Yun-Pang Flötteröd, Robert Hilbrich, Leonhard Lücken, Johannes Rummel, Peter Wagner, and Evamarie Wieβner. 2018. Microscopic Traffic Simulation using SUMO, In The 21st IEEE International Conference on Intelligent Transportation Systems. IEEE Intelligent Transportation Systems Conference (ITSC). https://elib.dlr.de/124092/
[8]
Martin Treiber and Venkatesan Kanagaraj. 2015. Comparing numerical integration schemes for time-continuous car-following models. Physica A: Statistical Mechanics and its Applications 419 (2015), 183-195.
[9]
Elise Van der Pol and Frans A Oliehoek. 2016. Coordinated deep reinforcement learners for traffic light control. Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016) (2016).
[10]
Hua Wei, Guanjie Zheng, Huaxiu Yao, and Zhenhui Li. 2018. IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control. In ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD). 2496-2505.
[11]
Cathy Wu, Aboudy Kreidieh, Eugene Vinitsky, and Alexandre M Bayen. 2017. Emergent Behaviors in Mixed-Autonomy Traffic. In Conference on Robot Learning. 398-407.
[12]
Zhe Xu, Zhixin Li, Qingwen Guan, Dingshui Zhang, Qiang Li, Junxiao Nan, Chunyang Liu, Wei Bian, and Jieping Ye. 2018. Large-Scale Order Dispatch in On-Demand Ride-Hailing Platforms: A Learning and Planning Approach. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 905-913.
[13]
Derek Yin and Tony Qiu. 2011. Comparison of macroscopic and microscopic simulation models in modern roundabout analysis. Transportation Research Record: Journal of the Transportation Research Board2265(2011), 244-252.

Cited By

View all
  • (2025)Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach With Attention and Hierarchical Game PriorsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.350309226:1(443-456)Online publication date: Jan-2025
  • (2025)Optimizing Traffic Flow: A Multi-agent Approach to Dynamic Signal Control Accounting for Vehicle TypesOptimization and Data Science in Industrial Engineering10.1007/978-3-031-81455-6_1(3-18)Online publication date: 26-Jan-2025
  • (2024)FightLadderProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693177(27653-27674)Online publication date: 21-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
WWW '19: The World Wide Web Conference
May 2019
3620 pages
ISBN:9781450366748
DOI:10.1145/3308558
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • IW3C2: International World Wide Web Conference Committee

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2019

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Microscopic Traffic Simulation
  2. Mobility
  3. Reinforcement Learning Platform

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

WWW '19
WWW '19: The Web Conference
May 13 - 17, 2019
CA, San Francisco, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)298
  • Downloads (Last 6 weeks)15
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Cooperative Decision-Making for CAVs at Unsignalized Intersections: A MARL Approach With Attention and Hierarchical Game PriorsIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2024.350309226:1(443-456)Online publication date: Jan-2025
  • (2025)Optimizing Traffic Flow: A Multi-agent Approach to Dynamic Signal Control Accounting for Vehicle TypesOptimization and Data Science in Industrial Engineering10.1007/978-3-031-81455-6_1(3-18)Online publication date: 26-Jan-2025
  • (2024)FightLadderProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693177(27653-27674)Online publication date: 21-Jul-2024
  • (2024)Scaling up Cooperative Multi-agent Reinforcement Learning SystemsProceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems10.5555/3635637.3663271(2737-2739)Online publication date: 6-May-2024
  • (2024)Adaptive Traffic Signal Control Based on Graph Neural Networks and Dynamic Entropy-Constrained Soft Actor–CriticElectronics10.3390/electronics1323479413:23(4794)Online publication date: 5-Dec-2024
  • (2024)Adaptive Traffic Signal Control Method Based on Offline Reinforcement LearningApplied Sciences10.3390/app14221016514:22(10165)Online publication date: 6-Nov-2024
  • (2024)DNLight: Learning Efficient Evaluation for Traffic Signal Control2024 43rd Chinese Control Conference (CCC)10.23919/CCC63176.2024.10662517(6526-6531)Online publication date: 28-Jul-2024
  • (2024)MAGT-toll: A multi-agent reinforcement learning approach to dynamic traffic congestion pricingPLOS ONE10.1371/journal.pone.031382819:11(e0313828)Online publication date: 18-Nov-2024
  • (2024)WebLight: DRL based Intersection Control in Developing Countries without Reliable CamerasProceedings of the 7th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies10.1145/3674829.3675075(201-210)Online publication date: 8-Jul-2024
  • (2024)FrugalLight : Symmetry-Aware Cyclic Heterogeneous Intersection Control using Deep Reinforcement Learning with Model Compression, Distillation and Domain KnowledgeACM Journal on Computing and Sustainable Societies10.1145/36485992:2(1-32)Online publication date: 19-Feb-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media