Dunhao Zhong
ABSTRACT
Intelligent traffic signal control is an effective way to solve the traffic congestion problem in the real world. One trend is to use Deep Reinforcement Learning (DRL) to control traffic signals based on the snapshots of traffic states. While most of the research used single numeric reward to frame multiple objectives, such as minimizing waiting time and waiting queue length, they overlooked that one reward for multiple objectives misleads agents taking wrong actions in certain states, which causes following traffic fluctuation. In this paper, we propose a DRL-based framework that uses multiple rewards for multiple objectives. Our framework aims to solve the difficulty of assessing behaviours by single numeric reward and control traffic flows more steadily. We evaluated our approach on both synthetic traffic scenarios and a real-world traffic dataset in Toronto. The results show that our approach outperformed single reward-based approaches.
- Itamar Arel, Cong Liu, T Urbanik, and AG Kohls. 2010. Reinforcement learning-based multi-agent system for network traffic signal control. IET Intelligent Transport Systems, Vol. 4, 2 (2010), 128--135.Google Scholar
Cross Ref
- Tim Brys, Tong T Pham, and Matthew E Taylor. 2014. Distributed learning and multi-objectivity in traffic light control. Connection Science, Vol. 26, 1 (2014), 65--83.Google ScholarDigital Library
- Graham Cookson and Bob Pishue. 2018. Inrix global traffic scorecard. Intelligence That Moves the World. United States: INRIX RESEARCH (2018).Google Scholar
- Samah El-Tantawy and Baher Abdulhai. 2010. An agent-based learning towards decentralized and coordinated traffic signal control. In ITSC, 2010 13th International IEEE Conference on. IEEE, 665--670.Google ScholarCross Ref
- Kunihiko Fukushima. 1979. Neural network model for a mechanism of pattern recognition unaffected by shift in position-Neocognitron. IEICE Technical Report, A, Vol. 62, 10 (1979), 658--665.Google Scholar
- Wade Genders and Saiedeh Razavi. 2016. Using a deep reinforcement learning agent for traffic signal control. arXiv preprint arXiv:1611.01142 (2016).Google Scholar
- Duan Houli, Li Zhiheng, and Zhang Yi. 2010. Multiobjective reinforcement learning for traffic signal control using vehicular ad hoc network. EURASIP journal on advances in signal processing, Vol. 2010 (2010), 7.Google Scholar
- Mohamed A Khamis and Walid Gomaa. 2014. Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework. Eng. Appl. Artif. Intell., Vol. 29 (2014), 134--151.Google ScholarDigital Library
- Mohamed A Khamis, Walid Gomaa, and Hisham El-Shishiny. 2012. Multi-objective traffic light control system based on Bayesian probability interpretation. In 2012 15th International IEEE Conference on Intelligent Transportation Systems. IEEE, 995--1000.Google ScholarCross Ref
- Xiaoyuan Liang, Xunsheng Du, Guiling Wang, and Zhu Han. 2018. Deep reinforcement learning for traffic light control in vehicular networks. arXiv preprint arXiv:1803.11115 (2018).Google Scholar
- P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. WieBner. 2018. Microscopic Traffic Simulation using SUMO. In 2018 21st ITSC. 2575--2582. https://doi.org/10.1109/ITSC.2018.8569938Google Scholar
- Patrick Mannion, Jim Duggan, and Enda Howley. 2016. An experimental review of reinforcement learning algorithms for adaptive traffic signal control. In Autonomic Road Transport Support Systems. Springer, 47--66.Google Scholar
- Ezell Stephen. 2010. Intelligent transportation systems. The Information Technology & Innovation Foundation (2010).Google Scholar
- Richard S Sutton and Andrew G Barto. 2018. Reinforcement learning: An introduction .MIT press.Google Scholar
- Toronto Open Data Team. 2018. King St. Transit Pilot$-$Detailed Traffic & Pedestrian Volumes. https://www.toronto.ca/city-government/data-research-maps/open-data/open-data-catalogue/#55a44849--90eb-ed1e-fbca-a7ad6b1025e3.Google Scholar
- Elise Van der Pol and Frans A Oliehoek. 2016. Coordinated deep reinforcement learners for traffic light control. Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016) (2016).Google Scholar
- Hua Wei, Guanjie Zheng, Huaxiu Yao, and Zhenhui Li. 2018. Intellilight: A reinforcement learning approach for intelligent traffic light control. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 2496--2505.Google ScholarDigital Library
- MA Wiering. 2000. Multi-agent reinforcement learning for traffic light control. In ICML'2000. 1151--1158.Google Scholar
- Kok-Lim Alvin Yau, Junaid Qadir, Hooi Ling Khoo, Mee Hong Ling, and Peter Komisarczuk. 2017. A survey on reinforcement learning models and algorithms for traffic signal control. ACM Computing Surveys (CSUR), Vol. 50, 3 (2017), 34.Google Scholar
Index Terms
- Traffic Signal Control Using Deep Reinforcement Learning with Multiple Resources of Rewards
Recommendations
Intelligent Traffic Signal Control with Deep Reinforcement Learning at Single Intersection
ICCAI '21: Proceedings of the 2021 7th International Conference on Computing and Artificial IntelligenceIn this paper, we apply the Proximal Policy Optimization (PPO) algorithm in intelligent traffic signal control at a single intersection with eight lanes and four signal phases. The optimization goal is to minimize the average waiting time of vehicles ...
Graph cooperation deep reinforcement learning for ecological urban traffic signal control
AbstractCooperation between intersections in large-scale road networks is critical in traffic congestion. Currently, most traffic signals cooperate via pre-defined timing phases, which is extremely inefficient in real-time traffic scenarios. Most existing ...
Meta-Reinforcement Learning for Multiple Traffic Signals Control
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge ManagementDespite the success of recent reinforcement learning (RL) in traffic signal control which has shown to outperform the conventional control methods, current RL-based methods require large amounts of samples to learn and lack the generalization ability to ...
Comments