Abstract
Today, Traffic congestion has become an increasingly serious problem. Efficient adaptive traffic signal control (ATSC) is a challenging issue in the road network. The existing multi-agent reinforcement learning (MARL) schemes do not have satisfactory performance due to the difficulty of coordination between agents and the delay consequence of reward function. In this paper, we present an novel adaptive traffic signal control scheme in the urban road network based on MARL. In the scheme, we adopt a delay time estimation model with network-wide coordination to estimate the total delay time of vehicles for each road link, and control traffic signals adaptively based on the estimated delay time with traffic flow data. We conduct comprehensive simulations using large-scale data collected from real world systems to evaluate the performance of our design, especially under heavy traffic pressure. The results show that our scheme can significantly alleviate the road congestion as well as improving the road network throughput and reducing the vehicles delay time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, J., Yuan, Q., Du, R., Wu, J.: Mucar: A greedy multi-flow-based coding-aware routing in wireless networks. In: Proceedings of the 12th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), Seattle, USA, 22–25 June, pp. 310–318 (2015)
Yin, B., Dridi, M., El Moudni, A.: Traffic control model and algorithm based on decomposition of mdp. In: Proceedings of the 2014 International Conference on Control, Decision and Information Technologies (CoDIT), Metz, France, 3–5 November, pp. 78–89. IEEE (2014)
Yin, B., Dridi, M., El Moudni, A.: Markov decision process for traffic control at an isolated intersection. In: Proceedings of the 2013 IEEE 25th International Conference on Tools with Artificial Intelligence (ICTAI), Washington DC, USA, 4–6 November, pp. 789–794. IEEE (2013)
Li, T., Zhao, D., Yi, J.: Adaptive dynamic programming for multi-intersections traffic signal intelligent control. In: Proceedings of the 2008 11th International IEEE Conference on Intelligent Transportation Systems (ITSC), Beijing, China, 12–15 October. pp. 286–291. IEEE (2008)
Cai, C., Wong, C.K., Heydecker, B.G.: Adaptive traffic signal control using approximate dynamic programming. Transport. Res. Part C: Emerg. Technol. 17(5), 456–474 (2009)
Yin, B., Dridi, M., El Moudni, A.: Adaptive traffic signal control for multi-intersection based on microscopic model. In: Proceedings of the 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), Vietri sul Mare, Italy, 9–11 November, pp. 49–55. IEEE (2015)
Yin, B., Dridi, M., El Moudni, A.: Approximate dynamic programming with recursive least-squares temporal difference learning for adaptive traffic signal control. In: Proceedings of the 2015 54th IEEE Conference on Decision and Control (CDC), Osaka, Japan, 15–18 December, pp. 3463–3468. IEEE (2015)
Vallati, M., Magazzeni, D., De Schutter, B., Chrpa, L., McCluskey, T.L.: Efficient macroscopic urban traffic models for reducing congestion: a pddl+ planning approach. In: Proceedings of the 2016 30th Conference on Artificial Intelligence (AAAI), Phoenix, Arizona USA, 12–17 February, pp. 3188–3194. AAAI (2016)
Kwatirayo, S., Almhana, J., Liu, Z., Siblini, J.: Optimizing road intersection traffic flow using stochastic and heuristic algorithms. In: Proceedings of the 2014 IEEE International Conference on Communications (ICC), Sydney, Australia, 10–14 June, pp. 586–591. IEEE (2014)
Renfrew, D., Yu, X.H.: Traffic signal optimization using ant colony algorithm. In: Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), Brisbane, Australia, 10–15 June, pp. 1–7. IEEE (2012)
De Oliveira, M.B., Neto, A.D.A.: Optimization of traffic lights timing based on multiple neural networks. In: Proceedings of the 2013 IEEE 25th International Conference on Tools with Artificial Intelligence (ICTAI), Washington DC, USA, 4–6 November, pp. 825–832. IEEE (2013)
Ezugwu, A.E., Frîncu, M., Adewumi, A.O., Buhari, S.M., Junaidu, S.B.: Neural network-based multi-agent approach for scheduling in distributed systems. Concurr. Comput. Prac. Exper. 29(1), e3887 (2017)
Cai, Y., Ji, R., Li, S.: Dynamic programming based optimized product quantization for approximate nearest neighbor search. Neurocomputing 217, 110–118 (2016)
Mannion, P., Duggan, J., Howley, E.: An experimental review of reinforcement learning algorithms for adaptive traffic signal control. In: McCluskey, T.L., Kotsialos, A., Müller, J.P., Klügl, F., Rana, O., Schumann, R. (eds.) Autonomic Road Transport Support Systems. AS, pp. 47–66. Springer, Cham (2016). doi:10.1007/978-3-319-25808-9_4
Bazzan, A.L.C.: Opportunities for multiagent systems and multiagent reinforcement learning in traffic control. Auton. Agents Multi-Agent Syst. 18(3), 342–375 (2009)
Xu, L.H., Xia, X.H., Luo, Q.: The study of reinforcement learning for traffic self-adaptive control under multiagent markov game environment. Math. Prob. Eng. 2013, 10p (2013). Article ID 962869
Wiering, M.: Multi-agent reinforcement learning for traffic light control. In: Proceedings of the 17th International Conference on Machine Learning (ICML), Stanford, USA, 29 June–2 July, JMLR.org, pp. 1151–1158 (2000)
Arel, I., Liu, C., Urbanik, T., Kohls, A.G.: Reinforcement learning-based multi-agent system for network traffic signal control. IET Intell. Transp. Syst. 4(2), 128–135 (2010)
Kuyer, L., Whiteson, S., Bakker, B., Vlassis, N.: Multiagent reinforcement learning for urban traffic control using coordination graphs. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008. LNCS, vol. 5211, pp. 656–671. Springer, Heidelberg (2008). doi:10.1007/978-3-540-87479-9_61
Prashanth, L.A., Bhatnagar, S.: Reinforcement learning with average cost for adaptive control of traffic lights at intersections. In: Proceedings of the 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), Washington, DC, USA, 5–7 October, pp. 1640–1645. IEEE (2011)
El-Tantawy, S., Abdulhai, B., Abdelgawad, H.: Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (marlin-atsc): methodology and large-scale application on downtown toronto. IEEE Trans. Intell. Transp. Syst. 14(3), 1140–1150 (2013)
Xie, X.F., Smith, S.F., Barlow, G.J.: Schedule-driven coordination for real-time traffic network control. In: Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), Atibaia, Brazil, 25–19 June. AAAI (2012)
Smith, S.F., Barlow, G.J., Xie, X.F., Rubinstein, Z.B.: Smart urban signal networks: Initial application of the surtrac adaptive traffic signal control system. In: Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), Rome, Italy, 10–14 June. AAAI (2013)
Jia, S.Y., Wang, G.H., Zhang, Y., Zhang, L.: Resolution and parameters estimations for multiple maneuvering targets. Sci. Chin. Inf. Sci. 57(8), 1–13 (2014)
Tong, S., Li, Y.: Robust adaptive fuzzy backstepping output feedback tracking control for nonlinear system with dynamic uncertainties. Sci. Chin. Inf. Sci. 53(2), 307–324 (2010)
Acknowledgements
This research is supported in part by the National Key Research and Development Program of China under grant No. 2016QY02D0202, NSFC under grants Nos. 61370233, 61422202, Foundation for the Author of National Excellent Doctoral Dissertation of PR China under grant No. 201345, and Research Fund of Guangdong Province under grant No. 2015B010131001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Chen, Y., Yao, J., He, C., Chen, H., Jin, H. (2017). Adaptive Traffic Signal Control with Network-Wide Coordination. In: Ibrahim, S., Choo, KK., Yan, Z., Pedrycz, W. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2017. Lecture Notes in Computer Science(), vol 10393. Springer, Cham. https://doi.org/10.1007/978-3-319-65482-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-65482-9_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65481-2
Online ISBN: 978-3-319-65482-9
eBook Packages: Computer ScienceComputer Science (R0)