Abstract
In this paper traffic signal control strategies for T-shaped intersections in urban road networks using deep Q network (DQN) algorithms are proposed. Different DQN networks and dynamic time aggregation were used for decision-makings. The effectiveness of various strategies under different traffic conditions are checked using the Simulation of Urban Mobility (SUMO) software. The simulation results showed that the strategy combining the Dueling DQN method and dynamic time aggregation significantly improved vehicle throughput. Compared with DQN and fixed-time methods, this strategy can reduce the average travel time by up to 43% in low-traffic periods and up to 15% in high-traffic periods. This paper demonstrated the significant advantages of applying Dueling DQN in traffic signal control strategies for urban road networks.
This work was supported in part by JiangXi Education Department under Grant No. GJJ191688.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liang, X., Du, X., Wang, G., Han, Z.: A deep reinforcement learning network for traffic light cycle control. IEEE Trans. Veh. Technol. 68(2), 1243–1253 (2019)
Chin, Y.K., Bolong, N., Yang, S.S., Teo, K.: Exploring q-learning optimization in traffic signal timing plan management. In: Third International Conference on Computational Intelligence (2011)
Abdoos, M., Mozayani, N., Bazzan, A.: Holonic multi-agent system for traffic signals control. Eng. Appl. Artif. Intell. 26(5–6), 1575–1587 (2013)
Balaji, P.G., German, X., Srinivasan, D.: Urban traffic signal control using reinforcement learning agents. IET Intel. Transport Syst. 4(3), 177–188 (2010)
Varaiya, P.: The max-pressure controller for arbitrary networks of signalized intersections. In: Ukkusuri, S., Ozbay, K. (eds.) Advances in Dynamic Network Modeling in Complex Transportation Systems. Complex Networks and Dynamic Systems, vol. 2. Springer, New York (2013). https://doi.org/10.1007/978-1-4614-6243-9_2
Bazzan, A.: Opportunities for multiagent systems and multiagent reinforcement learning in traffic control. Auton. Agent. Multi-Agent Syst. 18(3), 342 (2009)
Abdulhai, B., Pringle, P.: Autonomous multiagent reinforcement learning5 GC urban traffic control. In: Annual Transportation Research Board Meeting (2003)
Hamilton, A., Waterson, B., Cherrett, T., Robinson, A., Snell, I.: The evolution of urban traffic control: changing policy and technology. Transp. Plan. Technol. 36, 24–43 (2013)
Zhang, J., Wang, F.Y., Wang, K., Lin, W.H., Xu, X., Chen, C.: Data-driven intelligent transportation systems: a survey. IEEE Trans. Intell. Transp. Syst. 12(4), 1624–1639 (2011)
Wu, X., Liu, H.X.: Using high-resolution event-based data for traffic modeling and control: an overview. Transp. Res. Part C Emerg. Technol. 42(2), 28–43 (2014)
El-Tantawy, S., Abdulhai, B.: Towards multi-agent reinforcement learning for integrated network of optimal traffic controllers (Marlin-OTC). Transp. Lett. 2(2), 89–110 (2010)
Genders, W., Razavi, S.: Using a deep reinforcement learning agent for traffic signal control (2016)
Abdulhai, B., Pringle, R., Karakoulas, G.J.: Reinforcement learning for true adaptive traffic signal control. J. Transp. Eng. 129(3), 278–285 (2003)
Watkins, C., Dayan, P.: Q-learning. In: Machine Learning (1992)
Thorpe, T.L.: Vehicle traffic light control using SARSA (1997)
Mikami, S., Kakazu, Y.: Genetic reinforcement learning for cooperative traffic signal control. In: IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence (1994)
Volodymyr, M., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Mnih, V., et al.: Playing Atari with deep reinforcement learning. Computer Science (2013)
Li, L., Lv, Y., Wang, F.Y.: Traffic signal timing via deep reinforcement learning. IEEE/CAA J. Automatica Sinica 3, 247–254 (2016)
Hasselt, H.V., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: National Conference on Artificial Intelligence (2016)
Freitas, N.D., Lanctot, M., Hasselt, H.V., Hessel, M., Schaul, T., Wang, Z.: Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ni, W., Li, C., Wang, P., Li, Z. (2024). Traffic Signal Optimization at T-Shaped Intersections Based on Deep Q Networks. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Lecture Notes in Computer Science, vol 14449. Springer, Singapore. https://doi.org/10.1007/978-981-99-8067-3_22
Download citation
DOI: https://doi.org/10.1007/978-981-99-8067-3_22
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8066-6
Online ISBN: 978-981-99-8067-3
eBook Packages: Computer ScienceComputer Science (R0)