Skip to main content

Advertisement

Log in

Energy efficient deep reinforcement learning approach to control the traffic flow in iot networks for smart city

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Traffic flow control plays a crucial role in the development of smart cities, as it directly impacts various key components of urban infrastructure. With the rapid developments in artificial intelligence and large amount of data, there has been growing interest in leveraging deep reinforcement learning (RL) techniques for traffic control. These methods have shown promise in improving intersection efficiency and reducing travel time. However, existing approaches often overlook the spatial and temporal correlations between intersections, leading to suboptimal performance and increased waiting times. A decentralized traffic management system that leverages a Graph-Structured Correlation time spatial attention (GSCTSA) network and Asynchronous Advantage Actor-Critic (GSCTSA -A3C)is proposed. The system utilizes traffic controllers installed at each traffic light to collect and process real-time traffic data from nearby sensors and cameras. By processing the data at the edge layer, closer to the intersections, instead of relying on cloud-based solutions, real-time decision-making becomes feasible. The proposed GSCTSA -A3C model combines LSTM, correlational attention, and Graph Attention Networks with the Asynchronous Advantage Actor-Critic (A3C) RL algorithm. It captures spatio-temporal correlations and optimizes traffic light timings to improve traffic flow and reduce congestion. The GSCTSA module extracts essential spatio-temporal correlations, while the A3C algorithm learns optimal traffic control policies. Experimental outcomes demonstrate the effectiveness of the GSCTSA -A3C approach in achieving efficient traffic management in IoT networks for smart cities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Not Applicable.

References

  • Abdoos M, Bazzan AL (2021) Hierarchical traffic signal optimization using reinforcement learning and traffic prediction with long-short term memory. Expert Syst Appl 171:114580

    Article  Google Scholar 

  • Agarwal P, Matta P, Sharma S (2021) Analysis based traffic flow control decision using IoT sensors. Mater Today: Proc 46:10707

    Google Scholar 

  • Angayarkanni SA, Sivakumar R, Ramana Rao YV (2021) Hybrid Grey Wolf: Bald Eagle search optimized support vector regression for traffic flow forecasting. J Ambient Intell Humaniz Comput 12:1293–1304

    Article  Google Scholar 

  • Boukerche A, Zhong D, Sun P (2021) A novel reinforcement learning-based cooperative traffic signal system through max-pressure control. IEEE Trans Veh Technol 71(2):1187–1198

    Article  Google Scholar 

  • Celtek SA, Durdu A, Alı ME (2020) Real-time traffic signal control with swarm optimization methods. Measurement 166:108206

    Article  Google Scholar 

  • Chen S, Wang H, Meng Q (2021) An optimal dynamic lane reversal and traffic control strategy for autonomous vehicles. IEEE Trans Intell Transp Syst 23(4):3804–3815

    Article  Google Scholar 

  • Du W, Zhang Q, Chen Y, Ye Z (2021) An urban short-term traffic flow prediction model based on wavelet neural network with improved whale optimization algorithm. Sustainable Cities Soc 69:102858

    Article  Google Scholar 

  • Guidoni DL, Maia G, Souza FS, Villas LA, Loureiro AA (2020) Vehicular traffic management based on traffic engineering for vehicular ad hoc networks. IEEE Access 8:45167–45183

    Article  Google Scholar 

  • Hu T, Hu Z, Lu Z, Wen X (2023) Dynamic traffic signal control using mean field multi-agent reinforcement learning in large scale road‐networks. IET Intelligent Transport Systems

  • Jiang D (2020) The construction of smart city information system based on the internet of things and cloud computing. Comput Commun 150:158–166

    Article  Google Scholar 

  • Karimzadeh M, Aebi R, de Souza AM, Zhao Z, Braun T, Sargento S, Villas L (2021) Reinforcement learning-designed LSTM for trajectory and traffic flow prediction. In: 2021 IEEE wireless communications and networking conference (WCNC) (pp. 1–6). IEEE

  • Kumar N, Rahman SS, Dhakad N (2020) Fuzzy inference enabled deep reinforcement learning-based traffic light control for intelligent transportation system. IEEE Trans Intell Transp Syst 22(8):4919–4928

    Article  Google Scholar 

  • Liu Y, James JQ, Kang J, Niyato D, Zhang S (2020) Privacy-preserving traffic flow prediction: a federated learning approach. IEEE Internet Things J 7(8):7751–7763

    Article  Google Scholar 

  • Ma C, Dai G, Zhou J (2021) Short-term traffic flow prediction for urban road sections based on time series analysis and LSTM_BILSTM method. IEEE Trans Intell Transp Syst 23(6):5615–5624

    Article  Google Scholar 

  • Majumdar S, Subhani MM, Roullier B, Anjum A, Zhu R (2021) Congestion prediction for smart sustainable cities using IoT and machine learning approaches. Sustain Cities Soc 64:102500

    Article  Google Scholar 

  • Mushtaq A, Haq IU, Imtiaz MU, Khan A, Shafiq O (2021) Traffic flow management of autonomous vehicles using deep reinforcement learning and smart rerouting. IEEE Access 9:51005–51019

    Article  Google Scholar 

  • Narmadha S, Vijayakumar V (2021) Spatio-temporal vehicle traffic flow prediction using multivariate CNN and LSTM model. Mater Today: Proc

  • Nguyen TH, Jung JJ (2021) Swarm intelligence-based green optimization framework for sustainable transportation. Sustain Cities Soc 71:102947

    Article  Google Scholar 

  • Pan T, Guo R, Lam WH, Zhong R, Wang W, He B (2021) Integrated optimal control strategies for freeway traffic mixed with connected automated vehicles: a model-based reinforcement learning approach. Transp Res Part C: Emerg Technol 123:102987

    Article  Google Scholar 

  • Peng H, Du B, Liu M, Liu M, Ji S, Wang S, Zhang X, He L (2021) Dynamic graph convolutional network for long-term traffic flow prediction with reinforcement learning. Inform Sci 578:401–416

    Article  MathSciNet  Google Scholar 

  • Wang F, Zhu M, Wang M, Khosravi MR, Ni Q, Yu S, Qi L (2020) 6G-enabled short-term forecasting for large-scale traffic flow in massive IoT based on time-aware locality-sensitive hashing. IEEE Internet Things J 8(7):5321–5331

    Article  Google Scholar 

  • Wang M, Wu L, Li J, He L (2021) Traffic signal control with reinforcement learning based on region-aware cooperative strategy. IEEE Trans Intell Transp Syst 23(7):6774–6785

    Article  Google Scholar 

  • Yu X, Sun L, Yan Y, Liu G (2021) A short-term traffic flow prediction method based on spatial–temporal correlation using edge computing. Comput Electr Eng 93:107219

    Article  Google Scholar 

  • Zheng H, Lin F, Feng X, Chen Y (2020) A hybrid deep learning model with attention-based conv-LSTM networks for short-term traffic flow prediction. IEEE Trans Intell Transp Syst 22(11):6910–6920

    Article  Google Scholar 

Download references

Funding

There is no funding for this study.

Author information

Authors and Affiliations

Authors

Contributions

All the authors have participated in writing the manuscript and have revised the final version. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mrinai M. Dhanvijay.

Ethics declarations

Conflict of interest

Authors declares that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants and/or animals performed by any of the authors.

Informed consent

There is no informed consent for this study.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dhanvijay, M.M., Patil, S.C. Energy efficient deep reinforcement learning approach to control the traffic flow in iot networks for smart city. J Ambient Intell Human Comput 15, 3945–3961 (2024). https://doi.org/10.1007/s12652-024-04869-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-024-04869-w

Keywords

Navigation