Abstract
Intelligent traffic light control is a key approach to improve the efficiency of transportation system. However, existing intelligent traffic light control methods usually only adjust phase with fixed duration or just adjust duration in a fixed phase circle. In actual scenarios with complicated and dynamic traffic flow, these methods cannot give the optimal phase and duration corresponding to the current situation because of their restricted traffic control mode, which limits the potential to further improve the efficiency of traffic transportation. For this sake, we propose a novel traffic light control system that achieves completely dynamic control. The system is able to efficiently adjust both phase and duration via deep reinforcement learning and adaptive timing. Among them, the reinforcement learning model is specially used for phase decision and the adaptive timing algorithm used for duration decision is designed for effective utilization of green time in each phase. We test our system in different traffic flows and explore the relationship between optimal duration and traffic flow. We also verify the superb performance of our traffic light control system in a whole-day traffic scene.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Jin, J., Ma, X., Kosonen, I.: An intelligent control system for traffic lights with simulation-based evaluation. Control Eng. Pract. 58, 24–33 (2017)
Xiong, Z., Sheng, H., Rong, W., Cooper, D.E.: Intelligent transportation systems for smart cities: a progress review. Sci. China Inf. Sci. 55(12), 2908–2914 (2012). https://doi.org/10.1007/s11432-012-4725-1
Miller, A.J.: Settings for fixed-cycle traffic signals. J. Oper. Res. Soc. 14(4), 373–386 (1963)
Cools, S.B., Gershenson, C., D’Hooghe, B.: Self-organizing traffic lights: a realistic simulation. In: Prokopenko, M. (ed.) Advances in Applied Self-Organizing Systems. Advanced Information and Knowledge Processing., pp. 45–55. Springer, London (2013). https://doi.org/10.1007/978-1-4471-5113-5_3
Pandit, K., Ghosal, D., Zhang, H.M., Chuah, C.N.: Adaptive traffic signal control with vehicular ad hoc networks. IEEE Trans. Veh. Technol. 62(4), 1459–1471 (2013)
Zhang, R., Ishikawa, A., Wang, W., Striner, B., Tonguz, O.: Using reinforcement learning with partial vehicle detection for intelligent traffic signal control. IEEE Trans. Intell. Transp. Syst., 1–12 (2020). https://doi.org/10.1109/TITS.2019.2958859
Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Li, L., Lv, Y., Wang, F.Y.: Traffic signal timing via deep reinforcement learning. IEEE/CAA J. Autom. Sin. 3(3), 247–254 (2016)
Wei, H., Zheng, G., Yao, H., Li, Z.: Intellilight: a reinforcement learning approach for intelligent traffic light control. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496–2505 (2018)
Wei, H., et al.: Colight: learning network-level cooperation for traffic signal control. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1913–1922 (2019)
Hsieh, P.C., Chen, Y.R., Wu, W.H., Hsiung, P.A.: Timing optimization and control for smart traffic. In: 2014 IEEE International Conference on Internet of Things (iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom), pp. 9–16. IEEE (2014)
Tang, C., Xia, S., Zhu, C., Wei, X.: Phase timing optimization for smart traffic control based on fog computing. IEEE Access 7, 84217–84228 (2019)
Shen, G., Zhu, X., Xu, W., Tang, L., Kong, X.: Research on phase combination and signal timing based on improved k-medoids algorithm for intersection signal control. Wirel. Commun. Mob. Comput. 2020 (2020)
Vogel, A., Oremović, I., Šimić, R., Ivanjko, E.: Fuzzy traffic light control based on phase urgency. In: 2019 International Symposium ELMAR, pp. 9–14. IEEE (2019)
Yi-Fei, W., Zheng, G.: Research on polling based traffic signal control strategy with fuzzy control. In: 2018 IEEE 4th International Conference on Computer and Communications (ICCC), pp. 500–504. IEEE (2018)
Liang, X., Du, X., Wang, G., Han, Z.: A deep reinforcement learning network for traffic light cycle control. IEEE Trans. Veh. Technol. 68(2), 1243–1253 (2019)
Zhao, C., Hu, X., Wang, G.: Pdlight: a deep reinforcement learning traffic light control algorithm with pressure and dynamic light duration. arXiv preprint arXiv:2009.13711 (2020)
Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y., et al.: Policy gradient methods for reinforcement learning with function approximation. In: NIPs, vol. 99, pp. 1057–1063. Citeseer (1999)
Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937. PMLR (2016)
Lopez, P.A., et al.: Microscopic traffic simulation using sumo. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 2575–2582. IEEE (2018)
Codecá, L., Frank, R., Faye, S., Engel, T.: Luxembourg sumo traffic (lust) scenario: traffic demand evaluation. IEEE Intell. Transp. Syst. Mag. 9(2), 52–63 (2017)
Acknowledgments
This work was supported by Open Project of Chongqing Vehicle Test & Research Institute (No. 20AKC18) and Sanya Science and Education Innovation Park of Wuhan University of Technology (No. 2020KF0055).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wu, P., Song, B., Chen, X., Liu, B. (2021). A Traffic Light Control System Based on Reinforcement Learning and Adaptive Timing. In: Zhang, H., Yang, Z., Zhang, Z., Wu, Z., Hao, T. (eds) Neural Computing for Advanced Applications. NCAA 2021. Communications in Computer and Information Science, vol 1449. Springer, Singapore. https://doi.org/10.1007/978-981-16-5188-5_39
Download citation
DOI: https://doi.org/10.1007/978-981-16-5188-5_39
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-5187-8
Online ISBN: 978-981-16-5188-5
eBook Packages: Computer ScienceComputer Science (R0)