Abstract
With the development of transportation, the traditional traffic signal systems being unable to provide dynamic and flexible timing schemes for urban arterial road traffic in complex lane queuing conditions. In the control of arterial traffic, to solve the problem that vehicles queuing in turning lanes of branch road and then congesting the arterial road, this paper proposed an arterial traffic optimization algorithm based on deep reinforcement learning (DRL) and green wave coordination control in complex lane queuing conditions. The proposed algorithm provides a detailed division of the arterial roads and analyzed the mutual influence between vehicles inside the roads, combines DRL algorithm with the MAXBAND algorithm to optimize the signal period, phase sequence and green signal ratio of arterial roads, creates a new reward function for Deep Q Network (DQN) algorithm for multi-agent coordination. The algorithm was validated in SUMO simulation environment. The simulation results prove that the algorithm can flexibly perform signal timing and is more effective than traditional algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Little, J.: The synchronization of traffic signals by mixed-integer linear programming. Oper. Res. 14(4), 568–594 (1966)
Shen, G.: Urban traffic trunk two-direction green wave intelligent control strategy and its application. In: 6th World Congress on Intelligent Control and Automation, pp. 8563–8567. IEEE, Dalian (2006)
Zheng, Y., Ma, D., Jin, F., Zhao, Z.: ES-band: a novel approach to coordinate green wave system with adaptation evolutionary strategies. In: 2nd ACM SIGSPATIAL International Workshop on GeoSpatial Simulation, pp. 36–39. ACM, Chicago (2019)
Cabezas, X., GarcÃa, S., Salas, S.: A hybrid heuristic approach for traffic light synchronization based on the MAXBAND. Soft Comput. Lett. 1, 100001 (2019)
Oblakova, A., Hanbali, A., Boucherie, R., Ommeren, J.: Green wave analysis in a tandem of traffic-light intersections. Memorandum Faculty of Mathematical Sciences University of Twente (2017)
Yin, M.: Multi-junction traffic light optimization during holiday based on improved green wave band control. J. Phys. Conf. Ser. 1486(7), 072012 (2020)
Mnih, V., Kavukcuoglu, K., Silver, D., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Liu, C., Chen, Z., Tang, J., Xu, J., Piao, C.: Energy-efficient UAV control for effective and fair communication coverage: a deep reinforcement learning approach. IEEE J. Sel. Areas Commun. 36(9), 2059–2070 (2018)
Wu, Y., Liao, S., Liu, X., Li, Z., Lu, R.: Deep reinforcement learning on autonomous driving policy with auxiliary critic network. IEEE Trans. Neural Netw. Learn. Syst., 1–11 (2021)
Monireh, A., Ana, B.: Hierarchical traffic signal optimization using reinforcement learning and traffic prediction with long-short term memory. Expert Syst. Appl. 171, 114580 (2021)
Hay, R.T.: SUMO: a history of modification. Mol. Cell 18(1), 1–12 (2005)
Ghanim, M.S., Shaaban, K., Allawi, S.: Operational performance of signalized intersections: HCM and microsimulation comparison. In: 2022 Intermountain Engineering, Technology and Computing (IETC), pp. 1–6. IEEE, Orem (2022)
Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: an overview. In: Innovations in Multi-Agent Systems and Applications-1, vol. 310, pp. 183–221. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14435-6_7
Little, J., Kelson, M., Gartner, N.: MAXBAND: a versatile program for setting signals on arteries and triangular networks. Transp. Res. Rec. J. Transp. Res. Board 795, 40–46 (1981)
Pedroso, J P.: Optimization with Gurobi and python. INESC Porto and Universidade do Porto, Porto, Portugal, 1 (2011)
Prashanth, L.A., Bhatnagar, S.: Reinforcement learning with function approximation for traffic signal control. IEEE Trans. Intell. Transp. Syst. 12(2), 412–421 (2010)
Noaeen, M., Naik, A., Goodman, L., et al.: Reinforcement learning in urban network traffic signal control: a systematic literature review. Expert Syst. Appl. 199, 116830 (2022)
Hawi, R., Okeyo, G., Kimwele, M.: Techniques for smart traffic control: an in-depth. Int. J. Comput. Appl. Technol. Res. 4(7), 566–573 (2015)
Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant No. 62372131.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wang, T., Liu, S., Chen, L., Ouyang, M., Gao, S., Zhang, Y. (2024). Arterial Traffic Optimization Algorithm Based on Deep Reinforcement Learning and Green Wave Coordination Control in Complex Lane Queuing Conditions. In: Sun, Y., Lu, T., Wang, T., Fan, H., Liu, D., Du, B. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2023. Communications in Computer and Information Science, vol 2013. Springer, Singapore. https://doi.org/10.1007/978-981-99-9640-7_30
Download citation
DOI: https://doi.org/10.1007/978-981-99-9640-7_30
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-9639-1
Online ISBN: 978-981-99-9640-7
eBook Packages: Computer ScienceComputer Science (R0)
