Abstract
Traffic flow control plays a crucial role in the development of smart cities, as it directly impacts various key components of urban infrastructure. With the rapid developments in artificial intelligence and large amount of data, there has been growing interest in leveraging deep reinforcement learning (RL) techniques for traffic control. These methods have shown promise in improving intersection efficiency and reducing travel time. However, existing approaches often overlook the spatial and temporal correlations between intersections, leading to suboptimal performance and increased waiting times. A decentralized traffic management system that leverages a Graph-Structured Correlation time spatial attention (GSCTSA) network and Asynchronous Advantage Actor-Critic (GSCTSA -A3C)is proposed. The system utilizes traffic controllers installed at each traffic light to collect and process real-time traffic data from nearby sensors and cameras. By processing the data at the edge layer, closer to the intersections, instead of relying on cloud-based solutions, real-time decision-making becomes feasible. The proposed GSCTSA -A3C model combines LSTM, correlational attention, and Graph Attention Networks with the Asynchronous Advantage Actor-Critic (A3C) RL algorithm. It captures spatio-temporal correlations and optimizes traffic light timings to improve traffic flow and reduce congestion. The GSCTSA module extracts essential spatio-temporal correlations, while the A3C algorithm learns optimal traffic control policies. Experimental outcomes demonstrate the effectiveness of the GSCTSA -A3C approach in achieving efficient traffic management in IoT networks for smart cities.









Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
Not Applicable.
References
Abdoos M, Bazzan AL (2021) Hierarchical traffic signal optimization using reinforcement learning and traffic prediction with long-short term memory. Expert Syst Appl 171:114580
Agarwal P, Matta P, Sharma S (2021) Analysis based traffic flow control decision using IoT sensors. Mater Today: Proc 46:10707
Angayarkanni SA, Sivakumar R, Ramana Rao YV (2021) Hybrid Grey Wolf: Bald Eagle search optimized support vector regression for traffic flow forecasting. J Ambient Intell Humaniz Comput 12:1293–1304
Boukerche A, Zhong D, Sun P (2021) A novel reinforcement learning-based cooperative traffic signal system through max-pressure control. IEEE Trans Veh Technol 71(2):1187–1198
Celtek SA, Durdu A, Alı ME (2020) Real-time traffic signal control with swarm optimization methods. Measurement 166:108206
Chen S, Wang H, Meng Q (2021) An optimal dynamic lane reversal and traffic control strategy for autonomous vehicles. IEEE Trans Intell Transp Syst 23(4):3804–3815
Du W, Zhang Q, Chen Y, Ye Z (2021) An urban short-term traffic flow prediction model based on wavelet neural network with improved whale optimization algorithm. Sustainable Cities Soc 69:102858
Guidoni DL, Maia G, Souza FS, Villas LA, Loureiro AA (2020) Vehicular traffic management based on traffic engineering for vehicular ad hoc networks. IEEE Access 8:45167–45183
Hu T, Hu Z, Lu Z, Wen X (2023) Dynamic traffic signal control using mean field multi-agent reinforcement learning in large scale road‐networks. IET Intelligent Transport Systems
Jiang D (2020) The construction of smart city information system based on the internet of things and cloud computing. Comput Commun 150:158–166
Karimzadeh M, Aebi R, de Souza AM, Zhao Z, Braun T, Sargento S, Villas L (2021) Reinforcement learning-designed LSTM for trajectory and traffic flow prediction. In: 2021 IEEE wireless communications and networking conference (WCNC) (pp. 1–6). IEEE
Kumar N, Rahman SS, Dhakad N (2020) Fuzzy inference enabled deep reinforcement learning-based traffic light control for intelligent transportation system. IEEE Trans Intell Transp Syst 22(8):4919–4928
Liu Y, James JQ, Kang J, Niyato D, Zhang S (2020) Privacy-preserving traffic flow prediction: a federated learning approach. IEEE Internet Things J 7(8):7751–7763
Ma C, Dai G, Zhou J (2021) Short-term traffic flow prediction for urban road sections based on time series analysis and LSTM_BILSTM method. IEEE Trans Intell Transp Syst 23(6):5615–5624
Majumdar S, Subhani MM, Roullier B, Anjum A, Zhu R (2021) Congestion prediction for smart sustainable cities using IoT and machine learning approaches. Sustain Cities Soc 64:102500
Mushtaq A, Haq IU, Imtiaz MU, Khan A, Shafiq O (2021) Traffic flow management of autonomous vehicles using deep reinforcement learning and smart rerouting. IEEE Access 9:51005–51019
Narmadha S, Vijayakumar V (2021) Spatio-temporal vehicle traffic flow prediction using multivariate CNN and LSTM model. Mater Today: Proc
Nguyen TH, Jung JJ (2021) Swarm intelligence-based green optimization framework for sustainable transportation. Sustain Cities Soc 71:102947
Pan T, Guo R, Lam WH, Zhong R, Wang W, He B (2021) Integrated optimal control strategies for freeway traffic mixed with connected automated vehicles: a model-based reinforcement learning approach. Transp Res Part C: Emerg Technol 123:102987
Peng H, Du B, Liu M, Liu M, Ji S, Wang S, Zhang X, He L (2021) Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning. Inform Sci 578:401–416
Wang F, Zhu M, Wang M, Khosravi MR, Ni Q, Yu S, Qi L (2020) 6G-enabled short-term forecasting for large-scale traffic flow in massive IoT based on time-aware locality-sensitive hashing. IEEE Internet Things J 8(7):5321–5331
Wang M, Wu L, Li J, He L (2021) Traffic signal control with reinforcement learning based on region-aware cooperative strategy. IEEE Trans Intell Transp Syst 23(7):6774–6785
Yu X, Sun L, Yan Y, Liu G (2021) A short-term traffic flow prediction method based on spatial–temporal correlation using edge computing. Comput Electr Eng 93:107219
Zheng H, Lin F, Feng X, Chen Y (2020) A hybrid deep learning model with attention-based conv-LSTM networks for short-term traffic flow prediction. IEEE Trans Intell Transp Syst 22(11):6910–6920
Funding
There is no funding for this study.
Author information
Authors and Affiliations
Contributions
All the authors have participated in writing the manuscript and have revised the final version. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
Authors declares that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants and/or animals performed by any of the authors.
Informed consent
There is no informed consent for this study.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dhanvijay, M.M., Patil, S.C. Energy efficient deep reinforcement learning approach to control the traffic flow in iot networks for smart city. J Ambient Intell Human Comput 15, 3945–3961 (2024). https://doi.org/10.1007/s12652-024-04869-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-024-04869-w