Skip to main content
Springer Nature Link
Log in

A Deep Reinforcement Learning Approach to Online Microservice Deployment in Mobile Edge Computing

  • Conference paper
  • First Online:
Service-Oriented Computing (ICSOC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14420))

Included in the following conference series:

  • 879 Accesses

Abstract

Mobile edge computing (MEC) is receiving growing attention. In MEC environments, application requests (i.e., a set of consecutive microservice requests) of users are first sent to nearby edge servers, which can significantly reduce the latency compared to sending requests to the cloud center. Therefore, it is vital to deploy suitable microservices on edge servers considering the resource and coverage limitations of edge servers and the movement of users. However, existing deployment approaches focus on offline scenarios, where a service vacuum may occur between two offline deployments due to the long deployment time. Online microservice deployment is thus becoming an urgent need to satisfy user requirements better. This paper proposes DDQN, a deep reinforcement learning approach to online microservice deployment. Specifically, DDQN leverages the Dueling DQN (Deep Q-Network) model to generate real-time microservice deployment plans. Experiments show that the proposed method can effectively improve the success rate of microservice deployment in online scenarios without losing timeliness.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bhandarkar, A.B., Jayaweera, S.K.: Optimal trajectory learning for UAV-mounted mobile base stations using RL and greedy algorithms. In: 17th International Conference on Wireless and Mobile Computing, Networking and Communications, WiMob 2021, Bologna, Italy, 11–13 October 2021, pp. 13–18. IEEE (2021)

    Google Scholar 

  2. Chen, F., Zhou, J., Xia, X., Jin, H., He, Q.: Optimal application deployment in mobile edge computing environment. In: 13th IEEE International Conference on Cloud Computing, CLOUD 2020, Virtual Event, 18–24 October 2020, pp. 184–192. IEEE (2020)

    Google Scholar 

  3. Chen, L.: IoT microservice deployment in edge-cloud hybrid environment using reinforcement learning. IEEE Internet Things J. 8(16), 12610–12622 (2021)

    Article  Google Scholar 

  4. Deng, J., Li, B., Wang, J., Zhao, Y.: Microservice pre-deployment based on mobility prediction and service composition in edge. In: 2021 IEEE International Conference on Web Services, ICWS 2021, Chicago, IL, USA, 5–10 September 2021, pp. 569–578. IEEE (2021)

    Google Scholar 

  5. Farhadi, V., et al.: Service placement and request scheduling for data-intensive applications in edge clouds. IEEE/ACM Trans. Netw. 29(2), 779–792 (2021)

    Article  Google Scholar 

  6. He, Q., et al.: A game-theoretical approach for user allocation in edge computing environment. IEEE Trans. Parallel Distrib. Syst. 31(3), 515–529 (2020)

    Article  Google Scholar 

  7. Lai, P., et al.: Optimal edge user allocation in edge computing with variable sized vector bin packing. In: Service-Oriented Computing - 16th International Conference, ICSOC, vol. 11236, pp. 230–245 (2018)

    Google Scholar 

  8. Li, B., He, Q., Cui, G., Xia, X., Yang, Y.: READ: robustness-oriented edge application deployment in edge computing environment. IEEE Trans. Serv. Comput. 15, 1746–1759 (2020)

    Article  Google Scholar 

  9. Luo, W., Liang, J., Wang, T.: Randomized and optimal algorithms for k-lifetime dominating set in wireless sensor networks. IEEE Access 10, 23774–23784 (2022)

    Article  Google Scholar 

  10. Lv, W., et al.: Microservice deployment in edge computing based on deep q learning. IEEE Trans. Parallel Distrib. Syst. 33(11), 2968–2978 (2022)

    Google Scholar 

  11. Ma, H., Zhou, Z., Chen, X.: Predictive service placement in mobile edge computing. In: 2019 IEEE/CIC International Conference on Communications in China (ICCC), pp. 792–797. IEEE (2019)

    Google Scholar 

  12. Mudam, R., Bhartia, S., Chattopadhyay, S., Bhattacharya, A.: Mobility-aware service placement for vehicular users in edge-cloud environment. In: Kafeza, E., Benatallah, B., Martinelli, F., Hacid, H., Bouguettaya, A., Motahari, H. (eds.) ICSOC 2020. LNCS, vol. 12571, pp. 248–265. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65310-1_19

    Chapter  Google Scholar 

  13. Rababah, O.: A survey of automated web service composition methods (2018)

    Google Scholar 

  14. Raponi, S., Caprolu, M., Pietro, R.D.: Intrusion detection at the network edge: Solutions, limitations, and future directions - slides. In: International Conference on Edge Computing (2019)

    Google Scholar 

  15. Tonini, F., Khorsandi, B.M., Amato, E., Raffaelli, C.: Scalable edge computing deployment for reliable service provisioning in vehicular networks. J. Sens. Actuator Netw. 8(4), 51 (2019)

    Article  Google Scholar 

  16. Wang, S., Guo, Y., Zhang, N., Yang, P., Zhou, A., Shen, X.: Delay-aware microservice coordination in mobile edge computing: a reinforcement learning approach. IEEE Trans. Mob. Comput. 20(3), 939–951 (2019)

    Article  Google Scholar 

  17. Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., Freitas, N.: Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1995–2003. PMLR (2016)

    Google Scholar 

  18. Xiang, Z., Deng, S., Taheri, J., Zomaya, A.: Dynamical service deployment and replacement in resource-constrained edges. Mob. Netw. Appl. 25(2), 674–689 (2020)

    Article  Google Scholar 

  19. Xiong, W., et al.: A self-adaptive approach to service deployment under mobile edge computing for autonomous driving. Eng. Appl. Artif. Intell. 81, 397–407 (2019)

    Article  Google Scholar 

  20. Zhao, X., Shi, Y., Chen, S.: MAESP: mobility aware edge service placement in mobile edge networks. Comput. Netw. 182, 107435 (2020)

    Article  Google Scholar 

  21. Zhao, Y., Li, B., Wang, J., Jiang, D., Li, D.: Integrating deep reinforcement learning with pointer networks for service request scheduling in edge computing. Knowl. Based Syst. 258, 109983 (2022)

    Article  Google Scholar 

  22. Zhou, J., Fan, J., Wang, J., Jia, J.: Dynamic service deployment for budget-constrained mobile edge computing. Concurr. Pract. Exp. 31(24), e5436.1–e5436.16 (2019)

    Google Scholar 

Download references

Acknowledgment

This work is supported by the National Natural Science Foundation of China (Nos. 62032016, 61832014, and 61972292), the Key Research and Development Program of Hubei Province (No. 2021BAA031), and the Foundation of Yunnan Key Lab of Service Computing (No. YNSC23102). Bing Li and Jian Wang are corresponding authors of the paper.

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Wuhan, People’s Republic of China

    Yuqi Zhao, Jian Wang & Bing Li

  2. Complex Network Research Center, Wuhan University, Wuhan, People’s Republic of China

    Bing Li

  3. Hubei Luojia Laboratory, Wuhan, People’s Republic of China

    Bing Li

  4. Yunnan Key Laboratory of Service Computing, Yunnan, People’s Republic of China

    Jian Wang

Authors
  1. Yuqi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jian Wang or Bing Li .

Editor information

Editors and Affiliations

  1. Sapienza University of Rome, Rome, Italy

    Flavia Monti

  2. Technical University of Munich, Garching, Germany

    Stefanie Rinderle-Ma

  3. University of Seville, Seville, Spain

    Antonio Ruiz Cortés

  4. Sun Yat-sen University, Guangzhou, China

    Zibin Zheng

  5. Sapienza University of Rome, Rome, Italy

    Massimo Mecella

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhao, Y., Wang, J., Li, B. (2023). A Deep Reinforcement Learning Approach to Online Microservice Deployment in Mobile Edge Computing. In: Monti, F., Rinderle-Ma, S., Ruiz Cortés, A., Zheng, Z., Mecella, M. (eds) Service-Oriented Computing. ICSOC 2023. Lecture Notes in Computer Science, vol 14420. Springer, Cham. https://doi.org/10.1007/978-3-031-48424-7_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-48424-7_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-48423-0

  • Online ISBN: 978-3-031-48424-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics