Skip to main content

Advertisement

Springer Nature Link
Log in

Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks

  • Published:
Peer-to-Peer Networking and Applications Aims and scope Submit manuscript

Abstract

Lack of a fully vehicular topology view and restricted vehicles' movement to streets with the time-varying traffic light conditions have caused drastic gaps in the traditional vehicular routing protocols. Using software-defined networks (SDN), this paper proposes HIFS, a Hierarchical Intersection-based routing strategy that incorporates Fuzzy SARSA reinforcement learning to fill these gaps. At the first level of our HIFS scheme, a utility-based intersections selection policy is presented using fuzzy logic that jointly considers delay estimation, curve distance, and predicted of moving vehicles towards intersections. Then, a fuzzy logic-based path selection policy is proposed to choose the paths with highest flexibility against the intermittent connectivity and increased traffic loads. Residual bandwidth, Euclidean distance, angular orientation, and congestion are considered inputs of the fuzzy logic system. Meanwhile, traffic light states and nodes' information are used to tune the output fuzzy membership functions via reinforcement learning algorithm. The efficiency of our scheme in controlling ambiguity and uncertainty of the vehicular environment is confirmed through simulations in various vehicle densities and different traffic lights duration. Simulation results of average gains obtained for both scenarios show that our HIFS scheme increases the packet delivery ratio on average by 48.75%, 54.63%, and 8.78%, increases the throughput by 48.66%, 53.79%, and 8.61%, reduces end-to-end delay by 33.35%, 46.14%, and 15.38%, reduces the path length by 25.25%, 36.47%, and 15.32%, and reduces normalized routing overhead by 37.09%, 49.79%, and 20.17%, compared to MISR, ITAR-FQ, and GLS methods, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Data availability

Not applicable.

References

  1. Al-Qurabat AKM, Mohammed ZA, Hussein ZJ (2021) Data traffic management based on compression and MDL techniques for smart agriculture in IoT. Wireless Pers Commun 120(3):2227–2258

    Google Scholar 

  2. Al-Qurabat AKM (2021) A lightweight huffman-based differential encoding lossless compression technique in IoT for smart agriculture. Int J Comput Digital Syst 11(1). https://doi.org/10.12785/ijcds/110109

  3. Saeedi IDI, Al-Qurabat AKM (2022) Perceptually important points-based data aggregation method for wireless sensor networks. Baghdad Sci J 19(4):0875–0875

    Google Scholar 

  4. Al-Qurabat AKM, Abdulzahra SA (2020) An overview of periodic wireless sensor networks to the internet of things. IOP Conf Ser: Mater Sci Eng 928(3):32055. https://doi.org/10.1088/1757-899X/928/3/032055

  5. Al-Qurabat AKM, Abdulzahra SA, Idrees AK (2022) Two-level energy-efficient data reduction strategies based on SAX-LZW and hierarchical clustering for minimizing the huge data conveyed on the internet of things networks. J Supercomput. 78:17844–17890. https://doi.org/10.1007/s11227-022-04548-7

  6. Al-Qurabat AKM, Idress AK, Makhoul A, Jooude CA (2022) A bi-level data lowering method to minimize transferring big data in the sensors of IoT applications. Karbala Int J Modern Sci 8(2):123–138

    Google Scholar 

  7. Jawad GAM, Al-Qurabat AKM, Idrees AK (2022) Maximizing the underwater wireless sensor networks’ lifespan using BTC and MNP5 compression techniques. Ann Telecommun 77:703–723. https://doi.org/10.1007/s12243-021-00903-6

  8. Al-Qurabat AKM, Salman HM, Finjan AAR (2022) Important extrema points extraction-based data aggregation approach for elongating the WSN lifetime. Int J Comput Appl Technol 68(4):357–368

    Google Scholar 

  9. Naderi M, Ghanbari M (2023) Adaptively prioritizing candidate forwarding set in opportunistic routing in VANETs. Ad Hoc Netw 140:03048. https://doi.org/10.1016/j.adhoc.2022.103048

  10. Cunha F, Villas L, Boukerche A, Maia G, Viana A, Mini RAF, Loureiro AAF (2016) Data communication in VANETs: Survey, applications and challenges. Ad Hoc Netw 44:90–103

    Google Scholar 

  11. Islam MMd, R-Khan MT, Saad MM, Kim D (2021) Software-defined vehicular network (SDVN): a survey on architecture and routing. J Syst Archit 114:101961

    Google Scholar 

  12. Zemouri S, Djahel S, Murphy J (2015) A fast, reliable and lightweight distributed dissemination protocol for safety messages in urban vehicular networks. Ad Hoc Netw 27:26–43

    Google Scholar 

  13. Qadri NN, Fleury M, Altaf M, Ghanbari M (2010) Multi-source video streaming in a wireless vehicular ad hoc network. IET Commun 4(11):1300–1311

    Google Scholar 

  14. Srivastava A, Prakash A, Tripathi R (2020) Location based routing protocols in VANET: Issues and existing solutions. Veh Commun 23:100231

    Google Scholar 

  15. Al-Heety OS, Zakaria O, Ismail M, Shakir MM, Alani S, Alsariera H (2020) A comprehensive survey: Benefits, services, recent works, challenges, security and use cases for SDN-VANET. IEEE Access 8:91028–91047

    Google Scholar 

  16. Naderi M, Zargari F, Sadatpour V, Ghanbari M (2017) A 3-parameter routing cost function for improving opportunistic routing performance in VANETs. Wireless Pers Commun 97(1):1–15

    Google Scholar 

  17. Darwish T, Abu Bakar K (2016) Traffic aware routing in vehicular ad hoc networks: Characteristics and challenges. Telecommun Syst 61(3):489–513

    Google Scholar 

  18. Darwish T, Abu Bakar K (2016) Lightweight intersection-based traffic aware routing in urban vehicular networks. Comput Commun 87:60–75

    Google Scholar 

  19. Abdel-Halim IT, Fahmy HMA (2018) Prediction-based protocols for vehicular ad hoc networks: Survey and taxonomy. Comput Netw 130:34–50

    Google Scholar 

  20. Jaballah WB, Conti M, Lal C (2019) Software-defined VANETs: Benefits, challenges, and future directions. Preprint at http://arxiv.org/abs/1904.04577

  21. Kadhim AJ, H-Seno SA (2019) Energy-efficient multicast routing protocol based on SDN and fog computing for vehicular networks. Ad Hoc Netw 85:68–81

    Google Scholar 

  22. Zhao L, Zhao W, Hawbani A, Al-Dubai AY, Min G, Zomaya AY, Gong C (2021) Novel online sequential learning-based adaptive routing for edge software-defined vehicular networks. IEEE Trans Wireless Commun 20(5):2991–3004

    Google Scholar 

  23. Oubbati OS, Atiquzzaman M, Lorenz P, Baz A, Alhakami H (2020) SEARCH: an SDN-enabled approach for vehicle path-planning. IEEE Trans Veh Technol 69(12):14523–14536

    Google Scholar 

  24. Noorani N, H-Seno SA (2020) SDN and fog computing-based switchable routing using path stability estimation for vehicular ad hoc networks. Peer Peer Netw Appl 13(3):948–964

    Google Scholar 

  25. Zhao L, Bi Zh, Lin M, Hawbani A, Shi J, Guan Y (2021) An intelligent fuzzy-based routing scheme for software-defined vehicular networks. Comput Netw 187:107837

    Google Scholar 

  26. Abbas MT, Muhammad A, Song WC (2020) SD-IoV: SDN enabled routing for internet of vehicles in road-aware approach. J Ambient Intell Humaniz Comput 11(3):1265–1280

    Google Scholar 

  27. Venkatramana DKN, Srikantaiah SB, Moodabidri J (2017) SCGRP: SDN-enabled connectivity-aware geographical routing protocol of VANETs for urban environment. IET Netw 6(5):102–111

    Google Scholar 

  28. Kong D, Zhang G (2020) Ant colony algorithm-based routing protocol in software defined vehicular networks. In: Proceedings of the 2020 the 4th International Conference on Innovation in Artificial Intelligence, ICIAI 2020, Association for Computing Machinery, New York, NY, USA, pp 200–204

  29. Gao Y, Zhang Z, Zhao D, Zhang Y, Luo T (2018) A hierarchical routing scheme with load balancing in software defined vehicular Ad hoc networks. IEEE Access 6:73774–73785

    Google Scholar 

  30. Chahal M, Harit S (2019) Network selection and data dissemination in heterogeneous software-defined vehicular network. Comput Netw 161:32–44

    Google Scholar 

  31. Balta M, Ozçelik İ (2020) A 3-stage fuzzy-decision tree model for traffic signal optimization in urban city via a SDN based VANET architecture. Futur Gener Comput Syst 104:142–158

    Google Scholar 

  32. Hagras H (2007) Type-2 FLCs: a new generation of fuzzy controllers. IEEE Comput Intell Mag 2(1):30–43

    Google Scholar 

  33. Mardani MR, Ghanbari M (2018) Robust resource allocation scheme under channel uncertainties for LTE-A systems. Wireless Netw 25(3):1313–1325

    Google Scholar 

  34. Ding Q, Sun B, Zhang X (2016) A traffic-light-aware routing protocol based on street connectivity for urban vehicular Ad hoc networks. IEEE Commun Lett 20(8):1635–1638

    Google Scholar 

  35. Chang J-J, Li Y-H, Liao W, Chang C (2012) Intersection-based routing for urban vehicular communications with traffic-light considerations. IEEE Wirel Commun 19(1):82–88

    Google Scholar 

  36. Xia Y, Qin X, Liu B, Zhang P (2018) A greedy traffic light and queue aware routing protocol for urban VANETs. China Commun 15(7):77–87

    Google Scholar 

  37. Zhou S, Li D, Tang Q, Fu Y, Guo C, Chen X (2021) Multiple intersection selection routing protocol based on road section connectivity probability for urban VANETs. Comput Commun 177:255–264

    Google Scholar 

  38. Cao Z, Silva BN, Diyan M, Li J, Han K (2020) Intersection routing based on fuzzy multi-factor decision for VANETs. Appl Sci 10(18):6613

    Google Scholar 

  39. Cao Z, Fan Z, Kim J (2021) Intersection-based routing with fuzzy multi-factor decision for VANETs. Appl Sci 11(16):7304

    Google Scholar 

  40. He T, He R, Yu C (2021) Intersection-based traffic-aware routing with Fuzzy Q-learning for urban VANETs. In: 2021 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), pp 511–515

  41. Debnath A, Basumatary H, Dhar M, Debbarma MK, Bhattacharyya BK (2021) Fuzzy logic-based VANET routing method to increase the QoS by considering the dynamic nature of vehicles. Computing 103(7):1391–1415

    Google Scholar 

  42. Naderi M, Zargari F, Ghanbari M (2019) Adaptive beacon broadcast in opportunistic routing for VANETs. Ad Hoc Netw 86:119–130

    Google Scholar 

  43. Alzamzami O, Mahgoub I (2020) Link utility aware geographic routing for urban VANETs using two-hop neighbor information. Ad Hoc Netw 106:102213

    Google Scholar 

  44. Luo L, Sheng L, Yu H, Sun G (2021) Intersection-based V2X routing via reinforcement learning in vehicular ad hoc networks. IEEE Trans Intell Transp Syst 23(6):5446–5459. https://doi.org/10.1109/TITS.2021.3053958

  45. Srivastava A, Prakash A, Tripathi R (2020) Fuzzy-based beaconless probabilistic broadcasting for information dissemination in urban VANET. Ad Hoc Netw 108:102285

    Google Scholar 

  46. Huynh T-T, Dinh-Duc A-V, Tran C-H (2016) Delay-constrained energy-efficient cluster-based multi-hop routing in wireless sensor networks. J Commun Netw 18(4):580–588

    Google Scholar 

  47. Tal I, Muntean GM (2017) Towards reasoning vehicles: a survey of fuzzy logic-based solutions in vehicular networks. ACM Comput Surv (CSUR) 50(6):1–37

    Google Scholar 

  48. Kim B-S, Ullah S, Kim KH, Roh B-S, Ham J-H, Kim K-I (2020) an enhanced geographical routing protocol based on multi criteria decision making method in mobile ad-hoc networks. Ad Hoc Netw 103:102157

    Google Scholar 

  49. Derhami V, J-Majd V, N-Ahmadabadi M (2008) Fuzzy Sarsa learning and the proof of existence of its stationary points. Asian J Control 10(5):535–549

    MathSciNet  Google Scholar 

  50. Wu T, Zhou P, Liu K, Yuan Y, Wang X, Huang H, Wu DO (2020) Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks. IEEE Trans Veh Technol 69(8):8243–8256

    Google Scholar 

  51. Rak J (2013) LLA: a new anypath routing scheme providing long path lifetime in VANETs. IEEE Commun Lett 18(2):281–284

    Google Scholar 

  52. Zemouri S, Djahel S, Murphy J (2018) An altruistic prediction-based congestion control for strict beaconing requirements in urban VANETs. IEEE Trans Syst Man Cybern Syst 49(12):2582–2597

    Google Scholar 

  53. NS-3 Homepage. [Online]. Available at https://www.nsnam.org. Accessed 25 Jul 2019

  54. Krajzewicz D, Hertkorn G, Rossel C, Wagner P (2002) SUMO (Simulation of Urban MObility), an open-source traffic simulation. In: Proc 4th MESM, Sharjah, UAE, pp 183–187

  55. Saleet H, Basir O, Langar R, Boutaba R (2009) Region-based location-service-management protocol for VANETs. IEEE Trans Veh Technol 59(2):917–931

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Mohammad Naderi & Khorshid Mahdaee

  2. Department of Computer Engineering, Pardis Branch, Islamic Azad University, Pardis, Iran

    Parisa Rahmani

Authors
  1. Mohammad Naderi
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Khorshid Mahdaee
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Parisa Rahmani
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors contributed equally to this manuscript.

Corresponding author

Correspondence to Mohammad Naderi.

Ethics declarations

Ethics approval

The paper is original, and any other publishing house is not considering it for publication. The paper reflects the author's own research and analysis in a truthful and complete manner. All sources used are properly disclosed (correct citation).

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Conflicts of interest

There is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naderi, M., Mahdaee, K. & Rahmani, P. Hierarchical traffic light-aware routing via fuzzy reinforcement learning in software-defined vehicular networks. Peer-to-Peer Netw. Appl. 16, 1174–1198 (2023). https://doi.org/10.1007/s12083-022-01424-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12083-022-01424-2

Keywords