Abstract
Traffic lights are an important controlling factor in traffic flows, and good policies will facilitate traffic congestion. A car's waiting time is highly related to the period in which the traffic lights are green or red; thus, a proper controlling policy reduces the waiting time and speeds up the car movement. Emergency vehicles always have a higher priority than other cars and need to reach their destination faster. In the last decade, a lot of research has been done to solve the traffic light scheduling issue using artificial intelligence techniques. Reinforcement learning with simulating human learning has the outstanding performance to solve complex problems, such as traffic lights control. In this study, a deep reinforcement learning traffic light control was implemented at a crossroad using real-time traffic information with an emphasis on emergency vehicles. The agent learns to adjust its policies to prioritize the emergency cars over the other cars. The results show less waiting time and speed up the cars’ movement. It also indicates the emergency vehicles will cross the intersection with the least delay. With 200 to 600 cars and one emergency vehicle, the average delay decreases 2–16% in total using reinforcement learning. Considering more emergency vehicles, there is a 27–40% decrease in average delay for emergency vehicles.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Litman T (2013) Congestion Costing Critique: Critical Evaluation of the “Urban Mobility Report”
Nellore K, Hancke GP (2016) A survey on urban traffic management system using wireless sensor networks. Sensors 16(2):157
Grégoire P-L et al (2007) Urban traffic control based on learning agents. in 2007 IEEE Intelligent Transportation Systems Conference.. IEEE
Liu Z (2007) A survey of intelligence methods in urban traffic signal control. IJCSNS Int J Computer Sci Netw Secur 7(7):105–112
Alam J, Pandey M and Ahmed H (2013) Intelligent Traffic Light Control System for Isolated Intersection Using Fuzzy Logic. in Proceedings of the Conference on Advances in Communication and Control Systems-2013. Atlantis Press
Homaei H, Hejazi S, Dehghan SAM (2015) A new traffic light controller using fuzzy logic for a full single junction involving emergency vehicle preemption. J Uncertain Syst 9(1):49–61
Ben Arbi I (2019) A Survey on Intelligent Urban Road Traffic Control Systems. (Research Proposal)
Singh L, Tripathi S, Arora H (2009) Time optimization for traffic signal control using genetic algorithm. Int J Recent Trends Eng 2(2):4
Park B, Messer CJ, Urbanik T (1999) Traffic signal optimization program for oversaturated conditions: genetic algorithm approach. Transp Res Rec 1683(1):133–142
Di Febbraro A, Sacco N (2005) Freeway traffic control based on neural network estimation. IFAC Proceedings Volumes 38(1):66–71
Dhingra G et al (2019) Traffic management using convolution neural network. Int J Eng Adv Technol 8(5S):8471
Vidali A, Crociani L, Vizzari G, Bandini S (2019) A Deep Reinforcement Learning Approach to Adaptive Traffic Lights Management. In WOA Jun (pp. 42–50).
Mousavi SS, Schukat M, Howley E (2017) Traffic light control using deep policy-gradient and value-function-based reinforcement learning. IET Intel Transport Syst 11(7):417–423
Gao J et al (2017) Adaptive traffic signal control: Deep reinforcement learning algorithm with experience replay and target network. arXiv preprint http://arxiv.org/abs/1705.02755
Genders W and Razavi S (2016) Using a deep reinforcement learning agent for traffic signal control. arXiv preprint http://arxiv.org/abs/1611.01142
Li L, Lv Y, Wang F-Y (2016) Traffic signal timing via deep reinforcement learning. IEEE/CAA J Automatica Sinica 3(3):247–254
Vidali A, Crociani L, Vizzari G, Bandini S (2019) A Deep Reinforcement Learning Approach to Adaptive Traffic Lights Management. InWOA Jun (pp. 42–50).
Lison P (2015) An introduction to machine learning. Language Technology Group: Edinburgh, UK,
Yau K-LA et al (2017) A survey on reinforcement learning models and algorithms for traffic signal control. ACM Comput Surv (CSUR) 50(3):34
Wiering M, Van Otterlo M (2012) Reinforcement learning. Adapt Learn Optim 12:3
Van der Pol E (2016) Deep reinforcement learning for coordination in traffic light control. Master's thesis, University of Amsterdam
Chin YK et al (2011) Q-learning based traffic optimization in management of signal timing plan. Int J Simul Syst Sci Technol 12(3):29–35
Lin L-J (1993) Reinforcement learning for robots using neural networks., Carnegie-Mellon Univ Pittsburgh PA School of Computer Science
Younes MB, Boukerche A (2018) An efficient dynamic traffic light scheduling algorithm considering emergency vehicles for intelligent transportation systems. Wireless Netw 24(7):2451–2463
Behrisch M et al (2011) SUMO–simulation of urban mobility: an overview. in Proceedings of SIMUL 2011, The Third International Conference on Advances in System Simulation. ThinkMind.
Chu H, Liao Y, Chang L, Lee Y (2019) Traffic Light Cycle Configuration of Single Intersection Based on Modified Q-Learning. Appl Sci 9:4558–4577
Kumar N and Rahman (2019) Deep Reinforcement Learning with Vehicle Heterogeneity Based Traffic Light Control for Intelligent Transportation System, IEEE International Conference on Industrial Internet, pp. 28–33
Wei H, Zheng G, Yao H and Li Z (2018) IntelliLight: a Reinforcement Learning Approach for Intelligent Traffic Light Control, ACM International Conference on Knowledge Discovery & Data Mining, pp. 2496–2505
Bouktif S, Cheniki A, Ouni A (2021) Traffic signal control using hybrid action space deep reinforcement learning. Sensors 21:2302–2317
Maske H, Chu T, Kalabić U (2020) Control of traffic light timing using decentralized deep reinforcement learning. IFAC-PapersOnLine 53(2):14936–14941
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Rights and permissions
About this article
Cite this article
Shamsi, M., Rasouli Kenari, A. & Aghamohammadi, R. Reinforcement learning for traffic light control with emphasis on emergency vehicles. J Supercomput 78, 4911–4937 (2022). https://doi.org/10.1007/s11227-021-04068-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-021-04068-w