Abstract
The former traffic light control (TLC) system cannot effectively regulate the traffic conditions dynamically in real time due to urban growth. The Dueling Double Deep Recurrent Q-Network with Attention Mechanism (3DRQN-AM) method for TLC is proposed in this study. The proposed method is based on Deep Q-Network and employs target network, double learning method and dueling network to boost its learning efficiency. In order to integrate the past state of the vehicle’s motion trajectory with the current state of the vehicle for the best decision-making, the Long-Short Term Memory (LSTM) is introduced. While this is going on, an Attention Mechanism is introduced to help the neural network automatically focus on crucial state components and improve its capacity to represent state. According to experimental findings, the Dueling Double Deep Q-Network with Attention Mechanism (3DQN-AM), Dueling Double Deep Recurrent Q-Network (3DRQN), Dueling Double Deep Q-Network (3DQN), Fixed-Time-3DRQN-AM (FT-3DRQN-AM) signal management methods are compared. The techniques presented in this work lower the average waiting time under typical traffic flow by about 46.2%, 53.3%, 85.1%, and 30.0% respectively, and the average queue length by about 41.9%, 44.6%, 76.0%, and 21.7% respectively. Under peak traffic conditions, the average waiting time is decreased by around 20.8%, 32.1%, 36.7%, and 38.7% respectively, while the average queue is decreased by roughly 2.8%, 2.8%, 21.3%, and 44.9%.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Xu, L., Kun, W., Pengfei, L., Miaoyu, X.: Deep learning short-term traffic flow prediction based on lane changing behavior recognition. In: 2020 2nd International Conference on Information Technology and Computer Application (ITCA), pp. 760–763 (2020). https://doi.org/10.1109/ITCA52113.2020.00163
Tran-Van, N.Y., Nguyerr, X.H., Le, K.H.: Towards smart traffic lights based on deep learning and traffic flow information. In: 2022 9th NAFOSTED Conference on Information and Computer Science (NICS), pp. 1–6 (2022). https://doi.org/10.1109/NICS56915.2022.10013375
Maduka, N.C., Ajibade, I.I., Bello, M.: Modelling and optimization of smart traffic light control system. In: 2022 IEEE Nigeria 4th International Conference on Disruptive Technologies for Sustainable Development (NIGERCON), pp. 1–5 (2022). https://doi.org/10.1109/NIGERCON54645.2022.9803073
Abhishek, A., Nayak, P., Hegde, K.P., Lakshmi Prasad, A., Nagegowda, K.S.: Smart traffic light controller using deep reinforcement learning. In: 2022 3rd International Conference for Emerging Technology (INCET), pp. 1–5 (2022). https://doi.org/10.1109/INCET54531.2022.9824501
Raeisi, M., Mahboob, A.S.: Intelligent control of urban intersection traffic light based on reinforcement learning algorithm. In: 2021 26th International Computer Conference, Computer Society of Iran (CSICC), pp. 1–5 (2021). https://doi.org/10.1109/CSICC52343.2021.9420622
Wu, T., Kong, F., Peng, P., Fan, Z.: Road intersection model based reward function design in deep q-learning network for traffic light control. In: 2020 5th International Conference on Robotics and Automation Engineering (ICRAE), pp. 182–186 (2020). https://doi.org/10.1109/ICRAE50850.2020.9310858
Yang, D., Seo, S.W.: Traffic light detection using attention-guided continuous conditional random fields. In: 2022 International Conference on Electronics, Information, and Communication (ICEIC), pp. 1–3 (2022). https://doi.org/10.1109/ICEIC54506.2022.9748468
Huang, L., Huang, H., Xiao, J., Lv, X.: Research on adaptive adjustment method of intelligent traffic light based on real-time traffic flow detection. In: 2022 5th International Conference on Artificial Intelligence and Big Data (ICAIBD), pp. 644–647 (2022). https://doi.org/10.1109/ICAIBD55127.2022.9820545
Long, G., Wang, A., Jiang, T.: Traffic signal self-organizing control with road capacity constraints. IEEE Trans. Intell. Transp. Syst. 23(10), 18502–18511 (2022). https://doi.org/10.1109/TITS.2022.3152060
Zhu, Y., Cai, M., Schwarz, C., Junchao, L., Shaoping, X.: Intelligent traffic light via policy-based deep reinforcement learning. Int. J. Intell. Transport. Syst. Res. 20, 734–744 (2022). https://doi.org/10.1007/s13177-022-00321-5
Yu, P., Luo, J.: Minimize pressure difference traffic signal control based on deep reinforcement learning. In: 2022 41st Chinese Control Conference (CCC), pp. 5493–5498 (2022). https://doi.org/10.23919/CCC55666.2022.9901790
Klein, N., Prünte, J.: A new deep reinforcement learning algorithm for the online stochastic profitable tour problem. In: 2022 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), pp. 0635–0639 (2022). https://doi.org/10.1109/IEEM55944.2022.9989933
Gu, J., Fang, Y., Sheng, Z., Wen, P.: Double deep q-network with a dual-agent for traffic signal control. Appl. Sci. 10, 1622 (2020). https://doi.org/10.3390/app10051622
Huang, J., et al.: Application of deep reinforcement learning in optimization of traffic signal control. In: 2021 IEEE 23rd International Conference on High Performance Computing & Communications, pp. 1958–1964 (2021). https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys53884.2021.00293
Kodama, N., Harada, T., Miyazaki, K.: Traffic signal control system using deep reinforcement learning with emphasis on reinforcing successful experiences. IEEE Access 10, 128943–128950 (2022). https://doi.org/10.1109/ACCESS.2022.3225431
Xu, J., Li, L., Zhu, R., Lv, P.: Communication information fusion based multi-agent reinforcement learning for adaptive traffic light control. In: 2023 3rd International Conference on Neural Networks, Information and Communication Engineering (NNICE), pp. 488–492 (2023). https://doi.org/10.1109/NNICE58320.2023.10105755
Choe, C.J., Baek, S., Woon, B., Kong, S.H.: Deep q learning with LSTM for traffic light control. In: 2018 24th Asia-Pacific Conference on Communications (APCC), pp. 331–336 (2018). https://doi.org/10.1109/APCC.2018.8633520
Zhang, X., Liang, X., Zhiyuli, A., Zhang, S., Xu, R., Wu, B.: At-lstm: an attention-based lstm model for financial time series prediction. In: IOP Conference Series: Materials Science and Engineering, vol. 569, no. 5, pp. 052037 (2019). https://doi.org/10.1088/1757-899X/569/5/052037
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ni, W., Li, Z., Wang, P., Li, C. (2024). Advanced State-Aware Traffic Light Optimization Control with Deep Q-Network. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14448. Springer, Singapore. https://doi.org/10.1007/978-981-99-8082-6_14
Download citation
DOI: https://doi.org/10.1007/978-981-99-8082-6_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8081-9
Online ISBN: 978-981-99-8082-6
eBook Packages: Computer ScienceComputer Science (R0)