Abstract
The development of edge computing provides a novel deployment strategy for delay-aware applications, in which applications initially deployed in central servers are shifted closer to end-users for higher-quality and lower-delay services. However, with the growth in the number of end-users and devices, edge services are increasingly susceptible to sudden load spikes. In burst load scenarios, deploying services and allocating resources to maintain service quality and load balancing of edge servers become challenging, particularly given the coupling of resource requirements between services. This paper addresses this challenge by modeling the load burst scenario as a Markov decision problem and proposing a deep reinforcement learning-based (DRL-based) approach. The proposed approach ranks services based on their migration status and request delay violations, and makes scaling and migration decisions for each service in turn, with the goal of maximizing the total request throughput while satisfying delay requirements and resource constraints. Simulation results show that the proposed approach outperforms other algorithms in terms of total throughput and delay violation rate.








Similar content being viewed by others
Availability of data and materials
The data used in this paper are randomly generated, and the data generation setup is specified in Sect. 5.
References
Mao Y, You C, Zhang J, Huang K, Letaief KB (2017) A survey on mobile edge computing: the communication perspective. IEEE Commun Surv Tutor 19(4):2322–2358
Wang S, Zhang X, Zhang Y, Wang L, Yang J, Wang W (2017) A survey on mobile edge networks: convergence of computing, caching and communications. IEEE Access 5:6757–6779
Casalicchio E, Iannucci S (2020) The state-of-the-art in container technologies: application, orchestration and security. Concurr Comput Pract Exp 32(17):5668
Rastegar SH, Abbasfar A, Shah-Mansouri V (2020) Rule caching in SDN-enabled base stations supporting massive Iot devices with Bursty traffic. IEEE Internet Things J 7(9):8917–8931
Deng S, Xiang Z, Taheri J, Khoshkholghi MA, Yin J, Zomaya AY, Dustdar S (2020) Optimal application deployment in resource constrained distributed edges. IEEE Trans Mob Comput 20(5):1907–1923
Cao X, Tang G, Guo D, Li Y, Zhang W (2020) Edge federation: towards an integrated service provisioning model. IEEE/ACM Trans Netw 28(3):1116–1129
Wang J, Zhao G, Xu H, Zhao Y, Yang X, Huang H (2022) Trust: real-time request updating with elastic resource provisioning in clouds. In: IEEE INFOCOM 2022—IEEE Conference on Computer Communications, pp 620–629. https://doi.org/10.1109/INFOCOM48880.2022.9796788
Deng S, Zhang C, Li C, Yin J, Dustdar S, Zomaya AY (2021) Burst load evacuation based on dispatching and scheduling in distributed edge networks. IEEE Trans Parallel Distrib Syst 32(8):1918–1932
Chen X, Tang S, Lu Z, Wu J, Duan Y, Huang S-C, Tang Q (2019) iDiSC: a new approach to IoT-data-intensive service components deployment in edge-cloud-hybrid system. IEEE Access 7:59172–59184
Nguyen DT, Pham C, Nguyen KK, Cheriet M (2019) Placement and chaining for run-time IoT service deployment in edge-cloud. IEEE Trans Netw Serv Manag 17(1):459–472
Fang J, Ma A (2020) IoT application modules placement and dynamic task processing in edge-cloud computing. IEEE Internet Things J 8(16):12771–12781
Huang K-C, Lu Y-C, Tsai M-H, Wu Y-J, Chang H-Y (2016) Performance-efficient service deployment and scheduling methods for composite cloud services. In: Proceedings of the 9th International Conference on Utility and Cloud Computing, pp 240–244
Maia AM, Ghamri-Doudane Y, Vieira D, de Castro MF (2021) An improved multi-objective genetic algorithm with heuristic initialization for service placement and load distribution in edge computing. Comput Netw 194:108146
Li C, Jiang K, Luo Y (2022) Dynamic placement of multiple controllers based on SDN and allocation of computational resources based on heuristic ant colony algorithm. Knowl-Based Syst 241:108330
Ma Y, Liang W, Huang M, Xu W, Guo S (2022) Virtual network function service provisioning in MEC via trading off the usages between computing and communication resources. IEEE Trans Cloud Comput 10(4):2949–2963. https://doi.org/10.1109/TCC.2020.3043313
Shi D, Gao H, Wang L, Pan M, Han Z, Poor HV (2020) Mean field game guided deep reinforcement learning for task placement in cooperative multiaccess edge computing. IEEE Internet Things J 7(10):9330–9340
Yang D, Yang T, Gao F, Shi P, Liang S (2020) The application of the edge-cloud computing system based on reinforcement learning in large-scale mask recognition. In: 2020 5th International Conference on Mechanical, Control and Computer Engineering (ICMCCE). IEEE, pp 1756–1759
Xue F, Hai Q, Dong T, Cui Z, Gong Y (2022) A deep reinforcement learning based hybrid algorithm for efficient resource scheduling in edge computing environment. Inf Sci 608:362–374
Zhai Y, Bao T, Zhu L, Shen M, Du X, Guizani M (2020) Toward reinforcement-learning-based service deployment of 5g mobile edge computing with request-aware scheduling. IEEE Wirel Commun 27(1):84–91. https://doi.org/10.1109/MWC.001.1900298
Chen L, Xu Y, Lu Z, Wu J, Gai K, Hung PCK, Qiu M (2021) IoT microservice deployment in edge-cloud hybrid environment using reinforcement learning. IEEE Internet Things J 8(16):12610–12622. https://doi.org/10.1109/JIOT.2020.3014970
Zhou P, Wu G, Alzahrani B, Barnawi A, Alhindi A, Chen M (2021) Reinforcement learning for task placement in collaborative cloud- edge computing. In: 2021 IEEE Global Communications Conference (GLOBECOM), pp 1–6. https://doi.org/10.1109/GLOBECOM46510.2021.9685049
Xiao Y, Zhang Q, Liu F, Wang J, Zhao M, Zhang Z, Zhang J (2019) Nfvdeep: adaptive online service function chain deployment with deep reinforcement learning. In: Proceedings of the International Symposium on Quality of Service, pp 1–10
Chen Y, Deng S, Zhao H, He Q, Li Y, Gao H (2019) Data-intensive application deployment at edge: a deep reinforcement learning approach. In: 2019 IEEE International Conference on Web Services (ICWS). IEEE, pp 355–359
Wang S, Guo Y, Zhang N, Yang P, Zhou A, Shen X (2021) Delay-aware microservice coordination in mobile edge computing: a reinforcement learning approach. IEEE Trans Mob Comput 20(3):939–951. https://doi.org/10.1109/TMC.2019.2957804
Bonomi F, Milito R, Zhu J, Addepalli S (2012) Fog computing and its role in the internet of things. In: Proceedings of the 1st Edition of the MCC Workshop on Mobile Cloud Computing, pp 13–16
Deng S, Huang L, Wu H, Tan W, Taheri J, Zomaya AY, Wu Z (2016) Toward mobile service computing: opportunities and challenges. IEEE Cloud Comput 3(4):32–41
Alkhalaileh M, Calheiros RN, Nguyen QV, Javadi B (2020) Data-intensive application scheduling on mobile edge cloud computing. J Netw Comput Appl 167:102735
Farhadi V, Mehmeti F, He T, La Porta TF, Khamfroush H, Wang S, Chan KS, Poularakis K (2021) Service placement and request scheduling for data-intensive applications in edge clouds. IEEE/ACM Trans Network 29(2):779–792
Teodoro G, Sachetto R, Sertel O, Gurcan M.N, Meira W, Catalyurek U, Ferreira R (2009) Coordinating the use of GPU and CPU for improving performance of compute intensive applications. In: 2009 IEEE International Conference on Cluster Computing and Workshops. IEEE, pp 1–10
Huang M, Liang W, Ma Y, Guo S (2019) Maximizing throughput of delay-sensitive NFV-enabled request admissions via virtualized network function placement. IEEE Trans Cloud Comput 9(4):1535–1548
Liu Y, Yang B, Wu Y, Chen C, Guan X (2022) How to share: balancing layer and chain sharing in industrial microservice deployment. IEEE Trans Serv Comput
Wang J, Zhao L, Liu J, Kato N (2019) Smart resource allocation for mobile edge computing: a deep reinforcement learning approach. IEEE Trans Emerg Top Comput 9(3):1529–1541
Zhang J, Yu H, Fan G, Li Z (2022) Cold-start aware cloud-native service function chain caching in resource-constrained edge: a reinforcement learning approach. Comput Commun 195:334–345
Deng S, Xiang Z, Zhao P, Taheri J, Gao H, Yin J, Zomaya AY (2020) Dynamical resource allocation in edge for trustable internet-of-things systems: a reinforcement learning method. IEEE Trans Industr Inf 16(9):6103–6113
Zu J, Hu G, Peng D, Xie S, Gao W (2021) Fair scheduling and rate control for service function chain in NFV enabled data center. IEEE Trans Netw Serv Manag 18(3):2975–2986
Darzanos G, Koutsopoulos I, Stamoulis GD (2019) Cloud federations: economics, games and benefits. IEEE/ACM Trans Network 27(5):2111–2124
Long S, Long W, Li Z, Li K, Xia Y, Tang Z (2020) A game-based approach for cost-aware task assignment with QOS constraint in collaborative edge and cloud environments. IEEE Trans Parallel Distrib Syst 32(7):1629–1640
Behravesh R, Harutyunyan D, Coronado E, Riggio R (2021) Time-sensitive mobile user association and SFC placement in MEC-enabled 5G networks. IEEE Trans Netw Serv Manag 18(3):3006–3020
Vilaplana J, Solsona F, Teixidó I, Mateo J, Abella F, Rius J (2014) A queuing theory model for cloud computing. J Supercomput 69(1):492–507
Konda V, Tsitsiklis J (1999) Actor-critic algorithms. In: Advances in neural information processing systems 12
Yang S, Li F, Trajanovski S, Chen X, Wang Y, Fu X (2019) Delay-aware virtual network function placement and routing in edge clouds. IEEE Trans Mob Comput 20(2):445–459
Sun J, Liu F, Wang H, Ahmed M, Li Y, Liu M (2021) Efficient VNF placement for Poisson arrived traffic. IEEE Trans Netw Serv Manag 18(4):4277–4293
Lang S-D (1999) An extended banker’s algorithm for deadlock avoidance. IEEE Trans Softw Eng 25(3):428–432. https://doi.org/10.1109/32.798330
Funding
This work was partially supported by the Natural Science Foundation of Shanghai (No. 21ZR1416300), the Capacity Building Project of Local Universities Science and Technology Commission of Shanghai Municipality (No. 22010504100), the Research Programme of National Engineering Laboratory for Big Data Distribution and Exchange Technologies, and the Shanghai Municipal Special Fund for Promoting High Quality Development (No. 2021-GYHLW-01007).
Author information
Authors and Affiliations
Contributions
Jin Xu: Conceptualization, Methodology, Software, Validation, Formal Analysis, Investigation, Resources, Data curation, Writing (original draft), Visualization. Huiqun Yu: Supervision, Project Administration. Guisheng Fan: Supervision, Project Administration. Jiayin Zhang: Writing - review & editing. Zengpeng Li: Writing - review & editing. Qifeng Tang: Supervision, Project Administration.
Corresponding authors
Ethics declarations
Ethical Approval
Not applicable.
Conflict of interest
The authors have no conflict of interest to declare.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, J., Yu, H., Fan, G. et al. Adaptive edge service deployment in burst load scenarios using deep reinforcement learning. J Supercomput 80, 5446–5471 (2024). https://doi.org/10.1007/s11227-023-05656-8
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-023-05656-8