Skip to main content

Advertisement

SpringerLink
Log in

Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

With the access devices that are densely deployed in multi-access edge computing environments, users frequently switch access devices when moving, which causes the imbalance of network load and the decline of service quality. To solve the problems above, a seamless handover scheme for wireless access points based on perception is proposed. First, a seamless handover model based on load perception is proposed to solve the unbalanced network load, in which a seamless handover algorithm for wireless access points is used to calculate the access point with the highest weight, and a software-defined network controller controls the switching process. A joint allocation method of communication and computing resources based on deep reinforcement learning is proposed to minimize the terminal energy consumption and the system delay. A resource allocation model is based on minimizing terminal energy consumption, and system delay is built. The optimal value of task offloading decision and resource allocation vector are calculated with deep reinforcement learning. Experimental results show that the proposed method can reduce the network load and the task execution cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Moazenzadeh R, Mohammadi B (2019) Assessment of bio-inspired metaheuristic optimisation algorithms for estimating soil temperature. Geoderma 353:152–171

    Article  Google Scholar 

  2. Donepudi S, Garg S, Agarwal K, et al. (2014) Dynamic multi-access wireless network virtualization

  3. Kim H, Feamster N (2013) Improving network management with software defined networking. IEEE Commun Mag 51(2):115–169

    Article  Google Scholar 

  4. Baktir AC, Ozgovde A, Ersoy C (2017) How can edge computing benefit from software-defined networking: a survey, use cases, and future directions. IEEE Commun Surv Tutor 19(4):2359–2391

    Article  Google Scholar 

  5. Li C, Song M, Zhang M, Luo Y (2020) Effective replica management for improving reliability and availability in edge-cloud computing environment. J Parallel Distrib Comput 143:107–128

    Article  Google Scholar 

  6. Li C, Song M, Yu C, Luo YL (2021) Mobility and marginal gain based content caching and placement for cooperative edge-cloud computing. Inf Sci 548(16):153–176

    Article  Google Scholar 

  7. Zhang K, Mao Y, Leng S et al (2017) Mobile-edge computing for vehicular networks: a promising network paradigm with predictive off-loading. IEEE Veh Technol Mag 12(2):36–44

    Article  Google Scholar 

  8. Huang A, Nikaein N, Stenbock T, et al. (2017) Low latency MEC framework for SDN-based LTE/LTE-A networks. In: 2017 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6

  9. Katov AN, Mihovska A, Prasad NR (2015) Hybrid SDN architecture for resource consolidation in MPLS networks. In: Wireless telecommunications symposium. IEEE

  10. Schiller E, Nikaein N, Kalogeiton E et al (2018) CDS-MEC: NFV/SDN-based application management for MEC in 5G systems. Comput Netw 135:96–107

    Article  Google Scholar 

  11. Peng H, Ye Q, Shen XS (2019) SDN-based resource management for autonomous vehicular networks: a multi-access edge computing approach. IEEE Wirel Commun 26(4):156–162

    Article  Google Scholar 

  12. Miladinovic I, Schefer-Wenzl S, Hirner H (2019) IoT architecture for smart cities leveraging machine learning and SDN. In: 2019 27th Telecommunications Forum. Washington: IEEE Computer Society Press, pp 1–4

  13. Xia W, Zhang J, Quek TQS et al (2020) Mobile edge cloud-based industrial internet of things: improving edge intelligence with hierarchical SDN controllers. IEEE Veh Technol Mag 15(1):36–45

    Article  Google Scholar 

  14. Wang J, Hu J, Min G et al (2019) Computation offloading in multi-access edge computing using a deep sequential model based on reinforcement learning. IEEE Commun Mag 57(5):64–69

    Article  Google Scholar 

  15. Tahaei H, Ko K, Seo W et al (2017) A QoE based trustable SDN framework for IoT devices in mobile edge computing. Springer, Singapore

    Google Scholar 

  16. Yazdinejad A, Parizi RM, Dehghantanha A et al (2020) An energy-efficient SDN controller architecture for IoT networks with blockchain-based security. IEEE Trans Serv Comput 13:625–638

    Article  Google Scholar 

  17. Bao W, Yuan D, Yang Z et al (2017) Follow me fog: toward seamless handover timing schemes in a fog computing environment. IEEE Commun Mag 55(11):72–78

    Article  Google Scholar 

  18. Yunoki K, Shinbo H (2018) Carry-on state service handover between edge hosts for latency strict applications in mobile networks. In: 2018 21st international symposium on wireless personal multimedia communications. Washington: IEEE Computer Society Press, pp 472–477

  19. Neto AJV, Silva FSD, Neto EDP et al (2020) A taxonomy of DDoS attack mitigation approaches featured by SDN technologies in IoT scenarios. Sensors 20(11):3078

    Article  Google Scholar 

  20. Bi Y, Han G, Lin C et al (2018) Mobility support for fog computing: an SDN approach. IEEE Commun Mag 56(5):53–59

    Article  Google Scholar 

  21. Zeljković E, Slamnik-Kriještorac N, Latré S et al (2019) ABRAHAM: machine learning backed proactive handover algorithm using SDN. IEEE Trans Netw Serv Manage 16(4):1522–1536

    Article  Google Scholar 

  22. Yin X, Wang L (2017) A fast handover scheme for SDN based vehicular network. In: International conference on mobile ad-hoc and sensor networks. Berlin: Springer, pp 293–302

  23. Mouawad N, Naja R, Tohme S (2019) Fast and seamless handover in software defined vehicular networks. In: 2019 eleventh international conference on ubiquitous and future networks. Washington: IEEE Computer Society Press, pp 484–489

  24. Zhang Y, Deng RH, Bertino E, et al. (2019) Robust and universal seamless handover authentication in 5G HetNets. In: IEEE transactions on dependable and secure computing, pp (99): 1–1

  25. Mohseni H, Eslamnour B (2019) Handover management for delay-sensitive IoT services on wireless software-defined network platforms. In: 2019 3rd international conference on internet of things and applications. Washington: IEEE Computer Society Press, pp 1–6

  26. Bi Y, Han G, Lin C et al (2019) Mobility management for intro/inter domain handover in software-defined networks. IEEE J Sel Areas Commun 37(8):1739–1754

    Article  Google Scholar 

  27. Zhong X, Wang X, Li L, et al. (2020) A cooperative learning framework for resource management in MEC: an ADMM perspective

  28. Lyu X, Tian H, Sengul C et al (2017) Multiuser joint task offloading and resource optimization in proximate clouds. IEEE Trans Veh Technol 66(4):1–1

    Article  Google Scholar 

  29. Tran TX, Pompili D (2018) Joint task offloading and resource allocation for multi-server mobile-edge computing networks. IEEE Trans Veh Technol 68(1):856–868

    Article  Google Scholar 

  30. Cheng K, Teng Y, Sun W, et al. (2018) Energy-efficient joint offloading and wireless resource allocation strategy in multi-MEC server systems. In: 2018 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6

  31. Liang C, He Y, Yu FR, et al. (2017) Energy-efficient resource allocation in software-defined mobile networks with mobile edge computing and caching. In: 2017 IEEE conference on computer communications workshops. Washington: IEEE Computer Society Press, pp 121–126

  32. Wang P, Yao C, Zheng Z et al (2018) Joint task assignment, transmission, and computing resource allocation in multilayer mobile edge computing systems. IEEE Internet Things J 6(2):2872–2884

    Article  Google Scholar 

  33. Qian LP, Shi B, Wu Y et al (2020) NOMA enabled mobile edge computing for internet of things via joint communication and computation resource allocations. IEEE Internet Things J 7(1):718–733

    Article  Google Scholar 

  34. Yang Z, Liu Y, Chen Y, et al. (2019) Deep reinforcement learning in cache-aided MEC networks. In: ICC 2019–2019 IEEE international conference on communications. Washington: IEEE Computer Society Press, pp 1–6

  35. Li C, Zhang Y, Zhiqiang H et al (2020) An effective scheduling strategy based on hypergraph partition in geographically distributed datacenters. Comput Networks 170:107096

    Article  Google Scholar 

  36. Li C, Bai J, Yi C et al (2020) Resource and replica management strategy for optimizing financial cost and user experience in edge cloud computing system. Inf Sci 516:33–55

    Article  MathSciNet  Google Scholar 

  37. Li C, Tang J, Ma T, Yang X, Luo Y (2020) A workflow job scheduling algorithm based on load balancing in distributed cloud. J Network Comput Appl 152:1518

    Article  Google Scholar 

  38. Wang S, Xu J, Zhang N et al (2018) A survey on service migration in mobile edge computing. IEEE Access 6:23511–23528

    Article  Google Scholar 

  39. Han Z, Lei T, Lu Z et al (2019) Artificial intelligence-based handover management for dense WLANs: a deep reinforcement learning approach. IEEE Access 7:31688–31701

    Article  Google Scholar 

  40. Banday Y, Rather GM, Begh GR (2019) SINR analysis and interference management of macrocell cellular networks in dense urban environments. Wirel Pers Commun 111:1–21

    Google Scholar 

  41. Guo S, Xiao B, Yang Y, et al. (2016) Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In: IEEE INFOCOM 2016-the 35th annual IEEE International conference on computer communications. Washington: IEEE Computer Society Press, pp 1–9

  42. Chen L, Qu H, Zhao J et al (2016) Efficient and robust deep learning with correntropy-induced loss function. Neural Comput Appl 27(4):1019–1031

    Article  Google Scholar 

  43. Liu W, Anguelov D, Erhan D, et al. (2016) Ssd: single shot multibox detector. European conference on computer vision. Berlin: Springer, pp 21–37

  44. Chen J, Chen S, Wang Q et al (2019) iRAF: a deep reinforcement learning approach for collaborative mobile edge computing IoT networks. IEEE Internet Things J 6(4):7011–7024

    Article  MathSciNet  Google Scholar 

  45. Dai H, Zeng X, Yu Z et al (2019) A scheduling algorithm for autonomous driving tasks on mobile edge computing servers. J Syst Architect 94:14–23

    Article  Google Scholar 

  46. Chen M, Hao Y (2018) Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J Sel Areas Commun 36(3):587–597

    Article  Google Scholar 

  47. Guo H, Zhang J, Liu J et al (2018) Energy-aware computation offloading and transmit power allocation in ultradense IoT networks. IEEE Internet Things J 6(3):4317–4329

    Article  Google Scholar 

  48. Rajule N, Ambudkar B, Dhande A (2013) Survey of vertical handover decision algorithms. Int J Innov Eng Technol 2(1):362–368

    Google Scholar 

  49. Larasati HT, Hakimi R, Juhana T (2017) Extended-LLF: a least loaded first (LLF)-based handover association control for software-defined wireless network. Int J Comput Eng Inf Technol 9(9):203

    Google Scholar 

  50. Goutam S, Unnikrishnan S (2019) QoS based vertical handover decision algorithm using fuzzy logic. In: 2019 international conference on nascent technologies in engineering. Washington: IEEE Computer Society Press, pp 1–7

  51. Kobayashi R, Adachi K (2019) Radio and computing resource allocation for minimizing total processing completion time in mobile edge computing. IEEE Access 7:141119–141132

    Article  Google Scholar 

  52. Nguyen PD, Ha VN, Le LB (2019) Computation offloading and resource allocation for backhaul limited cooperative MEC systems. In: 2019 IEEE 90th vehicular technology conference. Washington: IEEE Computer Society Press, pp 1–6

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation (NSF) under Grants (No. 61873341, 61771354), Key Research and Development Plan of Hubei Province (No. 2020BAB102), and Open Foundation of Industrial Software Engineering Technology Research and Development Center of Jiangsu Education Department (No. ZK19-04-04). Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views of the above agencies.

Author information

Authors and Affiliations

  1. Department of Computer Science, Wuhan University of Technology, Wuhan, 430063, People’s Republic of China

    Chunlin Li, Yong Zhang & Youlong Luo

  2. Industrial Software Engineering Technology Research and Development Center of Jiangsu Education Department, Nanjing Institute of Industry Technology, Nanjing, People’s Republic of China

    Chunlin Li

Authors
  1. Chunlin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Youlong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chunlin Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Zhang, Y. & Luo, Y. Deep reinforcement learning-based resource allocation and seamless handover in multi-access edge computing based on SDN. Knowl Inf Syst 63, 2479–2511 (2021). https://doi.org/10.1007/s10115-021-01590-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-021-01590-4

Keywords

Search

Navigation