Skip to main content
SpringerLink
Log in

Deep Reinforcement Learning for Load Balancing of Edge Servers in IoV

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

A Correction to this article was published on 30 May 2022

This article has been updated

Abstract

In recent years, the use of edge computing to solve the problem of limited resources in the IoV has attracted more and more attention. Vehicles can upload tasks to the edge servers within their radio range for computing cooperation and offloading. However, due to shared resources among edge servers, the uneven distribution of vehicles may lead to many problems such as uneven tasks distribution, unbalanced load, and low computing efficiency on different edge servers. On the other hand, most of the existing works are proposed to address the load unbalance issue of edge servers with the help of remote cloud or vehicle cloud, but still leading to resource wasting problems especially for edge servers with a light load. To address this issue, a task transfer scheme has been proposed among different edge servers in this paper. First, we designed a partitioned and hierarchical software-defined network architecture for IoV. After that, a load balancing model is proposed under this network architecture, in which the global controller can manage and schedule the local tasks from each edge server. Next, we proved that the task allocation problem for our load balancing model under the proposed network architecture is an NP-hard problem. To solve this problem, a Deep Q-Network (DQN) model is proposed to minimize the mean square deviation of loads among different edge servers. Numerical results show that our proposed load balancing model can significantly improve the resource utilization of edge servers as well as reduce the computational latency of vehicular tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Change history

References

  1. Ray PP (2016) A survey on internet of things architectures. J King Saud University - Comput Inf Sci 30:291–319

    Google Scholar 

  2. Chen C, Liu L, Wan S, Hui X, Pei Q (2021) Data dissemination for industry 4.0 applications in internet of vehicles based on short-term traffic prediction. ACM Trans Internet Technol (TOIT) 22:1–18

    Google Scholar 

  3. Gao H, Liu C, Yin Y, Xu Y, Li Y (2021) A hybrid approach to trust node assessment and management for vanets cooperative data communication: Historical interaction perspective. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2021.3129458

    Article  Google Scholar 

  4. Li X, Dang Y, Aazam M, Peng X, Chen T, Chen C (2020) Energy-efficient computation offloading in vehicular edge cloud computing. IEEE Access 8:37632–37644

    Article  Google Scholar 

  5. Sharma S, Kaushik BN (2019) A survey on internet of vehicles: applications, security issues and solutions. Veh Commun 2. https://doi.org/10.1016/j.vehcom.2019.100182

  6. Xu W, Zhou H, Cheng N, Lyu F, Shi W, Chen J, Shen X (2018) Internet of vehicles in big data era. IEEE/CAA J Automatica Sinica 5:19–35

    Article  Google Scholar 

  7. Cong W, Chen C, Qingqi P, Zhiyuan J, Shugong X (2021) An information centric in-network caching scheme for 5G-enabled internet of connected vehicles. IEEE Trans Mobile Comput. https://doi.org/10.1109/TMC.2021.3137219

  8. Wan S, Ding S, Chen C (2022) Edge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles. Pattern Recogn 121:108146

    Article  Google Scholar 

  9. Chen C, Zeng Y, Li H, Liu Y, Wan S (2022) A multi-hop task offloading decision model in MEC-enabled internet of vehicles. IEEE Internet Things J, 1–1. https://doi.org/10.1109/JIOT.2022.3143529

  10. Bozkaya E, Canberk B (2015) QoE-based flow management in software defined vehicular networks. In: 2015 IEEE Globecom workshops (GC Wkshps)

  11. Liu S, Yu J, Deng X, Wan S (2021) FedCPF: An efficient-communication federated learning approach for vehicular edge computing in 6G communication networks. IEEE Trans Intell Trans Syst

  12. Yin Y, Huang Q, Gao H, Xu Y (2020) Personalized APIs recommendation with cognitive knowledge mining for industrial systems. IEEE Trans Industr Inform PP(99):1–1

  13. Patel KD, Bhalodia TM (2019) An Efficient Dynamic Load Balancing Algorithm for Virtual Machine in Cloud Computing. In: 2019 International conference on intelligent computing and control systems (ICCS)

  14. Chien NK, Son NH, Loc HD (2016) Load balancing algorithm based on estimating finish time of services in cloud computing. In: 2016 18th International conference on advanced communication technology (ICACT)

  15. Tamilarasi P, Akila D (2021) Prediction based load balancing and VM migration in big data cloud environment. In: 2021 2nd International conference on computation, automation and knowledge management (ICCAKM)

  16. Maswood M, Rahman MR, Alharbi AG, Medhi D (2020) A novel strategy to achieve bandwidth cost reduction and load balancing in a cooperative three-layer fog-cloud computing environment. IEEE Access PP(99):1–1

    Google Scholar 

  17. Wang KY, Kao SJ, Kao MT (2018) An efficient load adjustment for balancing multiple controllers in reliable sdn systems. In: 2018 IEEE International conference on applied system innovation (ICASI)

  18. Li J, Lei B, Li N, Lv H (2020) A load balancing approach for distributed SDN architecture based on sharing data store. In: 2020 21st Asia-pacific network operations and management symposium (APNOMS)

  19. Li J, Yang L, Wang J, Yang S (2018) Research on SDN Load balancing based on ant colony optimization algorithm. In: IEEE Information technology and mechatronics engineering conference (ITOEC), pp 979–982. https://doi.org/10.1109/ITOEC.2018.8740366

  20. Patil S (2018) Load balancing approach for finding best path in SDN. In: 2018 International conference on inventive research in computing applications (ICIRCA)

  21. Nkosi MC, Lysko AA, Dlamini S (2018) Multi-path load balancing for SDN data plane. In: 2018 International conference on intelligent and innovative computing applications (ICONIC). IEEE, pp 1–6

  22. Liu L, Jiang Y, Shen G, Li Q, Lin D, Li L, Wang Y (2019) An SDN-based hybrid strategy for load balancing in data center networks. In: 2019 IEEE Symposium on computers and communications (ISCC). IEEE, pp 1–6

  23. Hu J, Chen C, Cai L, Khosravi MR, Pei Q, Wan S (2021) UAV-assisted vehicular edge computing for the 6G internet of vehicles: architecture, intelligence, and challenges. IEEE Commun Standards Magazine 5(2):12–18

    Article  Google Scholar 

  24. Yang L, Yao H, Wang J, Jiang C, Liu Y (2019) Multi-UAV enabled load-balance mobile edge computing for IoT networks. IEEE Internet Things J 7(8):6898–6908. https://doi.org/10.1109/JIOT.2020.2971645

    Article  Google Scholar 

  25. Li L, Chan S, Han G, Guizani M, Bandai M (2018) Performance modeling of representative load sharing schemes for clustered servers in multiaccess edge computing. IEEE Internet Things J PP (99):1–1

    Google Scholar 

  26. Chen C, Zhang Y, Wang Z, Wan S, Pei Q (2021) Distributed computation offloading method based on deep reinforcement learning in icv. Appl Soft Comput 103:107108

  27. Li G, Yao Y, Wu J, Liu X, Lin Q (2020) A new load balancing strategy by task allocation in edge computing based on intermediary nodes. EURASIP Journal on Wireless Communications and Networking 2020

  28. Babou C, Fall D, Kashihara S, Taenaka Y, Kadobayashi Y (2020) Hierarchical load balancing and clustering technique for home edge computing. IEEE Access PP:1–1

    Google Scholar 

  29. Tham CK, Chattopadhyay R (2017) A load balancing scheme for sensing and analytics on a mobile edge computing network. IEEE International Symposium on A World of Wireless, 1–9

  30. Ma X, Xu H, Gao H, Bian M (2021) Real-time multiple-workflow scheduling in cloud environment. IEEE Trans Netw Serv Manag 18:4002–4018

    Article  Google Scholar 

  31. Huang Y, Xu H, Gao H, Ma X, Hussain W (2021) SSUR: An approach to optimizing virtual machine allocation strategy based on user requirements for cloud data center. IEEE Trans Green Commun Netw 5:670–681

    Article  Google Scholar 

  32. Wei W, Yang R, Gu H, Zhao W, Chen C, Wan S (2021) Multi-objective optimization for resource allocation in vehicular cloud computing networks. IEEE Trans Intell Transp Syst

  33. Gao H, Zhang Y, Miao H, Barroso RJD, Yang X (2021) SDTIOA: Modeling the timed privacy requirements of iot service composition: a user interaction perspective for automatic transformation from BPEL to timed automata. Mobile Networks and Applications, 1–26

Download references

Author information

Authors and Affiliations

  1. The College of Information Science & Electronic Engineering, Zhejiang University, Hangzhou, Zhejiang, 310027, China

    Pu Li

  2. State Key Laboratory of Integrated Services Networks, Xidian University, Xi’an, 710071, China

    Wenxuan Xie, Ying Yuan & Chen Chen

  3. Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, 518110, China

    Shaohua Wan

  4. Department of Information Systems, King Abdulaziz University, Jeddah , 21589, Saudi Arabia

    Shaohua Wan

Authors
  1. Pu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenxuan Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaohua Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaohua Wan.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Key Research and Development Program of China(2019YFE0196600), the National Natural Science Foundation of China (62072360, 62172438, 62001357, 62072359, 62072355), the key research and development plan of Shaanxi province (2021ZDLGY02-09, 2019ZDLGY13-07, 2019ZDLGY13-04, 2020JQ-844), the Natural Science Foundation of Guangdong Province of China(2022A1515010988), the Xi’an Science and Technology Plan(20RGZN0005) and the Xi’an Key Laboratory of Mobile Edge Computing and Security (201805052-ZD3CG36).

Authors have found out that the affiliation of the first author Pu Li is not correct, which should be "The College of Information Science & Electronic Engineering, Zhejiang University, Hangzhou, Zhejiang China 310027

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, P., Xie, W., Yuan, Y. et al. Deep Reinforcement Learning for Load Balancing of Edge Servers in IoV. Mobile Netw Appl 27, 1461–1474 (2022). https://doi.org/10.1007/s11036-022-01972-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-022-01972-0

Keywords

Search

Navigation