Abstract
To alleviate traffic congestion, it is a trend to apply reinforcement learning (RL) to traffic signal control in multi-intersection road networks. However, existing researches generally combine a basic RL framework Ape-X DQN with the graph convolutional network (GCN), to aggregate the neighborhood information, lacking unique collaboration exploration at each intersection with shared parameters. This paper proposes a multi-mode Light model that learns the general collaboration patterns in a road network with the graph attention network and trains simple Multilayer Perceptron for each intersection to capture each intersection’s unique collaboration pattern. The experiment results demonstrate that our model improves average by \(27.19\%\) compared with the state-of-the-art transportation method MaxPressure and average by \(4.57\%\) compared with the state-of-the-art reinforcement learning method Colight.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arel, I., Liu, C., Urbanik, T., Kohls, A.G.: Reinforcement learning-based multi-agent system for network traffic signal control. IET Intel. Transport Syst. 4(2), 128–135 (2010)
Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: A brief survey of deep reinforcement learning. arXiv preprint arXiv:1708.05866 (2017)
Casas, N.: Deep deterministic policy gradient for urban traffic light control. arXiv preprint arXiv:1703.09035 (2017)
Chu, T., Wang, J., Codecà, L., Li, Z.: Multi-agent deep reinforcement learning for large-scale traffic signal control. IEEE Trans. Intell. Transp. Syst. 21(3), 1086–1095 (2019)
Cools, S.B., Gershenson, C., D’Hooghe, B.: Self-organizing traffic lights: a realistic simulation. In: Prokopenko, M. (ed.) Advances in Applied Self-Organizing Systems. Advanced Information and Knowledge Processing, pp. 45–55. Springer, London (2013). https://doi.org/10.1007/978-1-4471-5113-5_3
Ge, H., Song, Y., Wu, C., Ren, J., Tan, G.: Cooperative deep q-learning with q-value transfer for multi-intersection signal control. IEEE Access 7, 40797–40809 (2019)
Horgan, D., et al.: Distributed prioritized experience replay. arXiv preprint arXiv:1803.00933 (2018)
Koonce, P., Rodegerdts, L.: Traffic signal timing Manual. Technical report, United States. Federal Highway Administration (2008)
McShane, W.R., Roess, R.P.: Traffic engineering (1990)
Nishi, T., Otaki, K., Hayakawa, K., Yoshimura, T.: Traffic signal control based on reinforcement learning with graph convolutional neural nets. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 877–883. IEEE (2018)
Prashanth, L., Bhatnagar, S.: Reinforcement learning with average cost for adaptive control of traffic lights at intersections. In: 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), pp. 1640–1645. IEEE (2011)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Cools, S.B., Gershenson, C., D’Hooghe, B.: Self-organizing traffic lights: a realistic simulation. In: Prokopenko, M. (eds) Advances in Applied Self-Organizing Systems. Advanced Information and Knowledge Processing, pp. 27–66. Springer, London (2013). https://doi.org/10.1007/978-1-4471-5113-5_3
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Wei, H., et al.: Colight: learning network-level cooperation for traffic signal control. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp. 1913–1922 (2019)
Wei, H., Zheng, G., Yao, H., Li, Z.: Intellilight: a reinforcement learning approach for intelligent traffic light control. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496–2505 (2018)
Xu, M., Wu, J., Huang, L., Zhou, R., Wang, T., Hu, D.: Network-wide traffic signal control based on the discovery of critical nodes and deep reinforcement learning. J. Intell. Transp. Syst. 24(1), 1–10 (2020)
Acknowledgement
This work was supported in part by the National Key R &D Program of China 2020YFB2103900, in part by the National Natural Science Foundation of China under Grant 61936014, in part by the Shanghai Municipal Science and Technology Major Project under Grant 2021SHZDZX0100, in part by the Natural Science Foundation of Shanghai under Grant 22ZR1462900 and in part by the Fundamental Research Funds for the Central Universities.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chen, Z., Zhao, S., Deng, H. (2022). Multi-mode Light: Learning Special Collaboration Patterns for Traffic Signal Control. In: Pimenidis, E., Angelov, P., Jayne, C., Papaleonidas, A., Aydin, M. (eds) Artificial Neural Networks and Machine Learning – ICANN 2022. ICANN 2022. Lecture Notes in Computer Science, vol 13530. Springer, Cham. https://doi.org/10.1007/978-3-031-15931-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-15931-2_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-15930-5
Online ISBN: 978-3-031-15931-2
eBook Packages: Computer ScienceComputer Science (R0)