Abstract
Vehicle-road collaboration positively promotes vehicle development and intelligent transportation. The construction of intelligent transportation cannot be separated from optimizing traffic signal timing, which is crucial for improving traffic efficiency, reducing congestion, and minimizing accident risks. Nowadays, reinforcement learning (RL) has emerged as an effective method for traffic signal timing optimization. However, many current RL-based approaches ignore the variations among different intersections, reducing the traffic efficiency. In this paper, we propose a novel method for traffic signal timing optimization, which models the problem of traffic timing optimization as an importance-oriented decision making problem. To achieve this, we first construct a directed adjacency graph based on the real road network. Then, a graph attention neural network (GAT) is utilized to estimate the importance of each intersection. Finally, we introduce the nodes importance into the reward function to find the optimal traffic light timing scheme. Extensive experiments demonstrate that our proposed method achieves higher traffic efficiency, compared to existing RL-based traffic signal timing optimization methods which ignore the intersection importance. Moreover, our method fits well with different RL algorithms, including Q-learning, DQN, Sarsa, DDPG and A3C.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, C., et al.: Toward a thousand lights: decentralized deep reinforcement learning for large-scale traffic signal control. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 3414–3421 (2020)
Chu, T., Wang, J., Codecà, L., Li, Z.: Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Trans. Intell. Transp. Syst. 21(3), 1086–1095 (2019)
Dalal, M., Pathak, D., Salakhutdinov, R.R.: Accelerating robotic reinforcement learning via parameterized action primitives. Adv. Neural. Inf. Process. Syst. 34, 21847–21859 (2021)
Dong, Y., Liu, Q., Du, B., Zhang, L.: Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification. IEEE Trans. Image Process. 31, 1559–1572 (2022)
Duan, J., Li, D., Huang, H.J.: Reliability of the traffic network against cascading failures with individuals acting independently or collectively. Trans. Res. Part C: Emerging Technol. 147, 104017 (2023)
Ducrocq, R., Farhi, N.: Deep reinforcement q-learning for intelligent traffic signal control with partial detection. Int. J. Intell. Transp. Syst. Res. 21(1), 192–206 (2023)
Ge, L., Li, H., Liu, J., Zhou, A.: Temporal graph convolutional networks for traffic speed prediction considering external factors. In: 2019 20th IEEE International Conference on Mobile Data Management (MDM), pp. 234–242. IEEE (2019)
Guo, M., Wang, P., Chan, C.Y., Askary, S.: A reinforcement learning approach for intelligent traffic signal control at urban intersections. In: 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 4242–4247. IEEE (2019)
Huang, J., Luo, K., Cao, L., Wen, Y., Zhong, S.: Learning multiaspect traffic couplings by multirelational graph attention networks for traffic prediction. IEEE Trans. Intell. Transp. Syst. 23(11), 20681–20695 (2022)
Koh, S.S., Zhou, B., Yang, P., Yang, Z., Fang, H., Feng, J.: Reinforcement learning for vehicle route optimization in sumo. In: 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pp. 1468–1473. IEEE (2018)
Li, Q., Gama, F., Ribeiro, A., Prorok, A.: Graph neural networks for decentralized multi-robot path planning. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 11785–11792. IEEE (2020)
Li, Y., Yu, R., Shahabi, C., Liu, Y.: Diffusion convolutional recurrent neural network: data-driven traffic forecasting. In: International Conference on Learning Representations (2018)
Li, Y., Tang, J., Zhao, H., Luo, R.: Reinforcement learning method with dynamic learning rate for real-time route guidance based on sumo. In: 2022 17th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 820–824. IEEE (2022)
Li, Z., Xu, C., Zhang, G.: A deep reinforcement learning approach for traffic signal control optimization. arXiv preprint arXiv:2107.06115 (2021)
Liu, J., Qin, S., Luo, Y., Wang, Y., Yang, S.: Intelligent traffic light control by exploring strategies in an optimised space of deep q-learning. IEEE Trans. Veh. Technol. 71(6), 5960–5970 (2022)
Liu, S., Wu, G., Barth, M.: A complete state transition-based traffic signal control using deep reinforcement learning. In: 2022 IEEE Conference on Technologies for Sustainability (SusTech), pp. 100–107. IEEE (2022)
Lyu, J., Ma, X., Yan, J., Li, X.: Efficient continuous control with double actors and regularized critics. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 7655–7663 (2022)
Navarro-Espinoza, A., et al.: Traffic flow prediction for smart traffic lights using machine learning algorithms. Technologies 10(1), 5 (2022)
Silver, D., et al.: Mastering the game of go without human knowledge. Nature 550(7676), 354–359 (2017)
Tan, T., Bao, F., Deng, Y., Jin, A., Dai, Q., Wang, J.: Cooperative deep reinforcement learning for large-scale traffic grid signal control. IEEE Trans. Cybernet. 50(6), 2687–2700 (2019)
Tang, J., Zeng, J.: Spatiotemporal gated graph attention network for urban traffic flow prediction based on license plate recognition data. Comput.-Aided Civil Infrastruct. Eng. 37(1), 3–23 (2022)
Wang, M., Wu, L., Li, M., Wu, D., Shi, X., Ma, C.: Meta-learning based spatial-temporal graph attention network for traffic signal control. Knowl.-Based Syst. 250, 109166 (2022)
Wang, T., Liang, T., Li, J., Zhang, W., Zhang, Y., Lin, Y.: Adaptive traffic signal control using distributed marl and federated learning. In: 2020 IEEE 20th International Conference on Communication Technology (ICCT), pp. 1242–1248. IEEE (2020)
Wang, T., Cao, J., Hussain, A.: Adaptive traffic signal control for large-scale scenario with cooperative group-based multi-agent reinforcement learning. Trans. Res. Part C: Emerging Technol. 125, 103046 (2021)
Wang, Y., Xu, T., Niu, X., Tan, C., Chen, E., Xiong, H.: Stmarl: a spatio-temporal multi-agent reinforcement learning approach for cooperative traffic light control. IEEE Trans. Mob. Comput. 21(6), 2228–2242 (2020)
Wei, H., et al.: Colight: learning network-level cooperation for traffic signal control. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1913–1922 (2019)
Yin, R., Song, X., et al.: Mitigation strategy of cascading failures in urban traffic congestion based on complex networks. Inter. J. Mod. Phys. C (IJMPC) 34(02), 1–20 (2023)
Acknowledgment
This work is supported by the National Natural Science Foundation of China under Grant Nos. 61972025, 61802389, the Fundamental Research Funds for the Central Universities under Grant Nos. 2023JBMC055, the National Key R &D Program of China under Grant Nos. 2020YFB1005604 and 2020YFB2103802.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, P. et al. (2024). Traffic Signal Timing Optimization Based on Intersection Importance in Vehicle-Road Collaboration. In: Kim, D.D., Chen, C. (eds) Machine Learning for Cyber Security. ML4CS 2023. Lecture Notes in Computer Science, vol 14541. Springer, Singapore. https://doi.org/10.1007/978-981-97-2458-1_6
Download citation
DOI: https://doi.org/10.1007/978-981-97-2458-1_6
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-2457-4
Online ISBN: 978-981-97-2458-1
eBook Packages: Computer ScienceComputer Science (R0)