Skip to main content
SpringerLink
Log in

Adaptive edge service deployment in burst load scenarios using deep reinforcement learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The development of edge computing provides a novel deployment strategy for delay-aware applications, in which applications initially deployed in central servers are shifted closer to end-users for higher-quality and lower-delay services. However, with the growth in the number of end-users and devices, edge services are increasingly susceptible to sudden load spikes. In burst load scenarios, deploying services and allocating resources to maintain service quality and load balancing of edge servers become challenging, particularly given the coupling of resource requirements between services. This paper addresses this challenge by modeling the load burst scenario as a Markov decision problem and proposing a deep reinforcement learning-based (DRL-based) approach. The proposed approach ranks services based on their migration status and request delay violations, and makes scaling and migration decisions for each service in turn, with the goal of maximizing the total request throughput while satisfying delay requirements and resource constraints. Simulation results show that the proposed approach outperforms other algorithms in terms of total throughput and delay violation rate.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Availability of data and materials

The data used in this paper are randomly generated, and the data generation setup is specified in Sect. 5.

References

  1. Mao Y, You C, Zhang J, Huang K, Letaief KB (2017) A survey on mobile edge computing: the communication perspective. IEEE Commun Surv Tutor 19(4):2322–2358

    Article  Google Scholar 

  2. Wang S, Zhang X, Zhang Y, Wang L, Yang J, Wang W (2017) A survey on mobile edge networks: convergence of computing, caching and communications. IEEE Access 5:6757–6779

    Article  Google Scholar 

  3. Casalicchio E, Iannucci S (2020) The state-of-the-art in container technologies: application, orchestration and security. Concurr Comput Pract Exp 32(17):5668

    Article  Google Scholar 

  4. Rastegar SH, Abbasfar A, Shah-Mansouri V (2020) Rule caching in SDN-enabled base stations supporting massive Iot devices with Bursty traffic. IEEE Internet Things J 7(9):8917–8931

    Article  Google Scholar 

  5. Deng S, Xiang Z, Taheri J, Khoshkholghi MA, Yin J, Zomaya AY, Dustdar S (2020) Optimal application deployment in resource constrained distributed edges. IEEE Trans Mob Comput 20(5):1907–1923

    Article  Google Scholar 

  6. Cao X, Tang G, Guo D, Li Y, Zhang W (2020) Edge federation: towards an integrated service provisioning model. IEEE/ACM Trans Netw 28(3):1116–1129

    Article  Google Scholar 

  7. Wang J, Zhao G, Xu H, Zhao Y, Yang X, Huang H (2022) Trust: real-time request updating with elastic resource provisioning in clouds. In: IEEE INFOCOM 2022—IEEE Conference on Computer Communications, pp 620–629. https://doi.org/10.1109/INFOCOM48880.2022.9796788

  8. Deng S, Zhang C, Li C, Yin J, Dustdar S, Zomaya AY (2021) Burst load evacuation based on dispatching and scheduling in distributed edge networks. IEEE Trans Parallel Distrib Syst 32(8):1918–1932

    Article  Google Scholar 

  9. Chen X, Tang S, Lu Z, Wu J, Duan Y, Huang S-C, Tang Q (2019) iDiSC: a new approach to IoT-data-intensive service components deployment in edge-cloud-hybrid system. IEEE Access 7:59172–59184

    Article  Google Scholar 

  10. Nguyen DT, Pham C, Nguyen KK, Cheriet M (2019) Placement and chaining for run-time IoT service deployment in edge-cloud. IEEE Trans Netw Serv Manag 17(1):459–472

    Article  Google Scholar 

  11. Fang J, Ma A (2020) IoT application modules placement and dynamic task processing in edge-cloud computing. IEEE Internet Things J 8(16):12771–12781

    Article  Google Scholar 

  12. Huang K-C, Lu Y-C, Tsai M-H, Wu Y-J, Chang H-Y (2016) Performance-efficient service deployment and scheduling methods for composite cloud services. In: Proceedings of the 9th International Conference on Utility and Cloud Computing, pp 240–244

  13. Maia AM, Ghamri-Doudane Y, Vieira D, de Castro MF (2021) An improved multi-objective genetic algorithm with heuristic initialization for service placement and load distribution in edge computing. Comput Netw 194:108146

    Article  Google Scholar 

  14. Li C, Jiang K, Luo Y (2022) Dynamic placement of multiple controllers based on SDN and allocation of computational resources based on heuristic ant colony algorithm. Knowl-Based Syst 241:108330

    Article  Google Scholar 

  15. Ma Y, Liang W, Huang M, Xu W, Guo S (2022) Virtual network function service provisioning in MEC via trading off the usages between computing and communication resources. IEEE Trans Cloud Comput 10(4):2949–2963. https://doi.org/10.1109/TCC.2020.3043313

    Article  Google Scholar 

  16. Shi D, Gao H, Wang L, Pan M, Han Z, Poor HV (2020) Mean field game guided deep reinforcement learning for task placement in cooperative multiaccess edge computing. IEEE Internet Things J 7(10):9330–9340

    Article  Google Scholar 

  17. Yang D, Yang T, Gao F, Shi P, Liang S (2020) The application of the edge-cloud computing system based on reinforcement learning in large-scale mask recognition. In: 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE). IEEE, pp 1756–1759

  18. Xue F, Hai Q, Dong T, Cui Z, Gong Y (2022) A deep reinforcement learning based hybrid algorithm for efficient resource scheduling in edge computing environment. Inf Sci 608:362–374

    Article  Google Scholar 

  19. Zhai Y, Bao T, Zhu L, Shen M, Du X, Guizani M (2020) Toward reinforcement-learning-based service deployment of 5g mobile edge computing with request-aware scheduling. IEEE Wirel Commun 27(1):84–91. https://doi.org/10.1109/MWC.001.1900298

    Article  Google Scholar 

  20. Chen L, Xu Y, Lu Z, Wu J, Gai K, Hung PCK, Qiu M (2021) IoT microservice deployment in edge-cloud hybrid environment using reinforcement learning. IEEE Internet Things J 8(16):12610–12622. https://doi.org/10.1109/JIOT.2020.3014970

    Article  Google Scholar 

  21. Zhou P, Wu G, Alzahrani B, Barnawi A, Alhindi A, Chen M (2021) Reinforcement learning for task placement in collaborative cloud- edge computing. In: 2021 IEEE Global Communications Conference (GLOBECOM), pp 1–6. https://doi.org/10.1109/GLOBECOM46510.2021.9685049

  22. Xiao Y, Zhang Q, Liu F, Wang J, Zhao M, Zhang Z, Zhang J (2019) Nfvdeep: adaptive online service function chain deployment with deep reinforcement learning. In: Proceedings of the International Symposium on Quality of Service, pp 1–10

  23. Chen Y, Deng S, Zhao H, He Q, Li Y, Gao H (2019) Data-intensive application deployment at edge: a deep reinforcement learning approach. In: 2019 IEEE International Conference on Web Services (ICWS). IEEE, pp 355–359

  24. Wang S, Guo Y, Zhang N, Yang P, Zhou A, Shen X (2021) Delay-aware microservice coordination in mobile edge computing: a reinforcement learning approach. IEEE Trans Mob Comput 20(3):939–951. https://doi.org/10.1109/TMC.2019.2957804

    Article  Google Scholar 

  25. Bonomi F, Milito R, Zhu J, Addepalli S (2012) Fog computing and its role in the internet of things. In: Proceedings of the 1st Edition of the MCC Workshop on Mobile Cloud Computing, pp 13–16

  26. Deng S, Huang L, Wu H, Tan W, Taheri J, Zomaya AY, Wu Z (2016) Toward mobile service computing: opportunities and challenges. IEEE Cloud Comput 3(4):32–41

    Article  Google Scholar 

  27. Alkhalaileh M, Calheiros RN, Nguyen QV, Javadi B (2020) Data-intensive application scheduling on mobile edge cloud computing. J Netw Comput Appl 167:102735

    Article  Google Scholar 

  28. Farhadi V, Mehmeti F, He T, La Porta TF, Khamfroush H, Wang S, Chan KS, Poularakis K (2021) Service placement and request scheduling for data-intensive applications in edge clouds. IEEE/ACM Trans Network 29(2):779–792

    Article  Google Scholar 

  29. Teodoro G, Sachetto R, Sertel O, Gurcan M.N, Meira W, Catalyurek U, Ferreira R (2009) Coordinating the use of GPU and CPU for improving performance of compute intensive applications. In: 2009 IEEE International Conference on Cluster Computing and Workshops. IEEE, pp 1–10

  30. Huang M, Liang W, Ma Y, Guo S (2019) Maximizing throughput of delay-sensitive NFV-enabled request admissions via virtualized network function placement. IEEE Trans Cloud Comput 9(4):1535–1548

    Article  Google Scholar 

  31. Liu Y, Yang B, Wu Y, Chen C, Guan X (2022) How to share: balancing layer and chain sharing in industrial microservice deployment. IEEE Trans Serv Comput

  32. Wang J, Zhao L, Liu J, Kato N (2019) Smart resource allocation for mobile edge computing: a deep reinforcement learning approach. IEEE Trans Emerg Top Comput 9(3):1529–1541

    Article  Google Scholar 

  33. Zhang J, Yu H, Fan G, Li Z (2022) Cold-start aware cloud-native service function chain caching in resource-constrained edge: a reinforcement learning approach. Comput Commun 195:334–345

    Article  Google Scholar 

  34. Deng S, Xiang Z, Zhao P, Taheri J, Gao H, Yin J, Zomaya AY (2020) Dynamical resource allocation in edge for trustable internet-of-things systems: a reinforcement learning method. IEEE Trans Industr Inf 16(9):6103–6113

    Article  Google Scholar 

  35. Zu J, Hu G, Peng D, Xie S, Gao W (2021) Fair scheduling and rate control for service function chain in NFV enabled data center. IEEE Trans Netw Serv Manag 18(3):2975–2986

    Article  Google Scholar 

  36. Darzanos G, Koutsopoulos I, Stamoulis GD (2019) Cloud federations: economics, games and benefits. IEEE/ACM Trans Network 27(5):2111–2124

    Article  Google Scholar 

  37. Long S, Long W, Li Z, Li K, Xia Y, Tang Z (2020) A game-based approach for cost-aware task assignment with QOS constraint in collaborative edge and cloud environments. IEEE Trans Parallel Distrib Syst 32(7):1629–1640

    Article  Google Scholar 

  38. Behravesh R, Harutyunyan D, Coronado E, Riggio R (2021) Time-sensitive mobile user association and SFC placement in MEC-enabled 5G networks. IEEE Trans Netw Serv Manag 18(3):3006–3020

    Article  Google Scholar 

  39. Vilaplana J, Solsona F, Teixidó I, Mateo J, Abella F, Rius J (2014) A queuing theory model for cloud computing. J Supercomput 69(1):492–507

    Article  Google Scholar 

  40. Konda V, Tsitsiklis J (1999) Actor-critic algorithms. In: Advances in neural information processing systems 12

  41. Yang S, Li F, Trajanovski S, Chen X, Wang Y, Fu X (2019) Delay-aware virtual network function placement and routing in edge clouds. IEEE Trans Mob Comput 20(2):445–459

    Article  Google Scholar 

  42. Sun J, Liu F, Wang H, Ahmed M, Li Y, Liu M (2021) Efficient VNF placement for Poisson arrived traffic. IEEE Trans Netw Serv Manag 18(4):4277–4293

    Article  Google Scholar 

  43. Lang S-D (1999) An extended banker’s algorithm for deadlock avoidance. IEEE Trans Softw Eng 25(3):428–432. https://doi.org/10.1109/32.798330

    Article  Google Scholar 

Download references

Funding

This work was partially supported by the Natural Science Foundation of Shanghai (No. 21ZR1416300), the Capacity Building Project of Local Universities Science and Technology Commission of Shanghai Municipality (No. 22010504100), the Research Programme of National Engineering Laboratory for Big Data Distribution and Exchange Technologies, and the Shanghai Municipal Special Fund for Promoting High Quality Development (No. 2021-GYHLW-01007).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

    Jin Xu, Huiqun Yu, Guisheng Fan, Jiayin Zhang & Zengpeng Li

  2. Shanghai Key Laboratory of Computer Software Evaluating and Testing, Shanghai, 201112, China

    Huiqun Yu

  3. Shanghai Data Exchange Corporation, Shanghai, 200436, China

    Qifeng Tang

Authors
  1. Jin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huiqun Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guisheng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiayin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zengpeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qifeng Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Jin Xu: Conceptualization, Methodology, Software, Validation, Formal Analysis, Investigation, Resources, Data curation, Writing (original draft), Visualization. Huiqun Yu: Supervision, Project Administration. Guisheng Fan: Supervision, Project Administration. Jiayin Zhang: Writing - review & editing. Zengpeng Li: Writing - review & editing. Qifeng Tang: Supervision, Project Administration.

Corresponding authors

Correspondence to Huiqun Yu or Guisheng Fan.

Ethics declarations

Ethical Approval

Not applicable.

Conflict of interest

The authors have no conflict of interest to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, J., Yu, H., Fan, G. et al. Adaptive edge service deployment in burst load scenarios using deep reinforcement learning. J Supercomput 80, 5446–5471 (2024). https://doi.org/10.1007/s11227-023-05656-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05656-8

Keywords

Search

Navigation