Skip to main content
Springer Nature Link
Log in

A DQN-Based Approach for Online Service Placement in Mobile Edge Computing

  • Conference paper
  • First Online:
Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom 2020)

Abstract

Due to the development of 5G networks, computation intensive applications on mobile devices have emerged, such as augmented reality and video stream analysis. Mobile edge computing is put forward as a new computing paradigm, to meet the low-latency requirements of applications, by moving services from the cloud to the network edge like base stations. Due to the limited storage space and computing capacity of an edge server, service placement is an important issue, determining which services are deployed at edge to serve corresponding tasks. The problem becomes particularly complicated, with considering the stochastic arrivals of tasks, the additional latency incurred by service migration, and the time spent for waiting in queues for processing at edge. Benefiting from reinforcement learning, we propose a deep Q network based approach, by formulating service placement as a Markov decision process. Real-time service placement strategies are output, to minimize the total latency of arrived tasks in a long term. Extensive simulation results demonstrate that our approach works effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aissioui, A., Ksentini, A., Gueroui, A.M., Taleb, T.: On enabling 5G automotive systems using follow me edge-cloud concept. IEEE Trans. Veh. Technol. 67(6), 5302–5316 (2018)

    Article  Google Scholar 

  2. Al-Shuwaili, A., Simeone, O.: Energy-efficient resource allocation for mobile edge computing-based augmented reality applications. IEEE Wirel. Commun. Lett. 6(3), 398–401 (2017)

    Article  Google Scholar 

  3. Ascigil, O., Phan, T.K., Tasiopoulos, A.G., Sourlas, V., Psaras, I., Pavlou, G.: On uncoordinated service placement in edge-clouds. In: 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom), pp. 41–48. IEEE (2017)

    Google Scholar 

  4. Farhadi, V., et al.: Service placement and request scheduling for data-intensive applications in edge clouds. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 1279–1287. IEEE (2019)

    Google Scholar 

  5. Gao, B., Zhou, Z., Liu, F., Xu, F.: Winning at the starting line: joint network selection and service placement for mobile edge computing. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 1459–1467. IEEE (2019)

    Google Scholar 

  6. He, T., Khamfroush, H., Wang, S., La Porta, T., Stein, S.: It’s hard to share: joint service placement and request scheduling in edge clouds with sharable and non-sharable resources. In: 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), pp. 365–375. IEEE (2018)

    Google Scholar 

  7. Hu, Y.C., Patel, M., Sabella, D., Sprecher, N., Young, V.: Mobile edge computing-a key technology towards 5G. ETSI White Paper 11(11), 1–16 (2015)

    Google Scholar 

  8. Ksentini, A., Taleb, T., Chen, M.: A Markov decision process-based service migration procedure for follow me cloud. In: 2014 IEEE International Conference on Communications (ICC), pp. 1350–1354. IEEE (2014)

    Google Scholar 

  9. Liu, J., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uWave: accelerometer-based personalized gesture recognition and its applications. Pervasive Mobile Comput. 5(6), 657–675 (2009)

    Article  Google Scholar 

  10. Mao, Y., You, C., Zhang, J., Huang, K., Letaief, K.B.: A survey on mobile edge computing: the communication perspective. IEEE Commun. Surv. Tutor. 19(4), 2322–2358 (2017)

    Article  Google Scholar 

  11. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

    Article  Google Scholar 

  12. Nadembega, A., Hafid, A.S., Brisebois, R.: Mobility prediction model-based service migration procedure for follow me cloud to support QoS and QoE. In: 2016 IEEE International Conference on Communications (ICC), pp. 1–6. IEEE (2016)

    Google Scholar 

  13. Ouyang, T., Li, R., Chen, X., Zhou, Z., Tang, X.: Adaptive user-managed service placement for mobile edge computing: an online learning approach. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 1468–1476. IEEE (2019)

    Google Scholar 

  14. Ouyang, T., Zhou, Z., Chen, X.: Follow me at the edge: mobility-aware dynamic service placement for mobile edge computing. IEEE J. Sel. Areas Commun. 36(10), 2333–2345 (2018)

    Article  Google Scholar 

  15. Poularakis, K., Llorca, J., Tulino, A.M., Taylor, I., Tassiulas, L.: Joint service placement and request routing in multi-cell mobile edge computing networks. In: IEEE INFOCOM 2019-IEEE Conference on Computer Communications, pp. 10–18. IEEE (2019)

    Google Scholar 

  16. Sutton, R., Barto, A.: Introduction to Reinforcement Learning. MIT Press, Cambridge (1998)

    Book  Google Scholar 

  17. Taleb, T., Ksentini, A., Frangoudis, P.: Follow-me cloud: when cloud services follow mobile users. IEEE Trans. Cloud Comput. PP, 1 (2016)

    Google Scholar 

  18. Wang, L., Jiao, L., He, T., Li, J., Mühlhäuser, M.: Service entity placement for social virtual reality applications in edge computing. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 468–476. IEEE (2018)

    Google Scholar 

  19. Wang, S., Dey, S.: Adaptive mobile cloud computing to enable rich mobile multimedia applications. IEEE Trans. Multimed. 15(4), 870–883 (2013)

    Article  Google Scholar 

  20. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992). https://doi.org/10.1007/BF00992698

    Article  MATH  Google Scholar 

  21. Xu, J., Chen, L., Zhou, P.: Joint service caching and task offloading for mobile edge computing in dense networks. In: IEEE INFOCOM 2018-IEEE Conference on Computer Communications, pp. 207–215. IEEE (2018)

    Google Scholar 

  22. Zeng, D., Gu, L., Pan, S., Cai, J., Guo, S.: Resource management at the network edge: a deep reinforcement learning approach. IEEE Netw. 33(3), 26–33 (2019)

    Article  Google Scholar 

Download references

Acknowledgement

This research is supported by NSFC (No. 61802245), the Shanghai Sailing Program (No. 18YF1408200), and STSCM (No. 19511121000). This work is also supported by the Open Project Program of Shanghai Key Laboratory of Data Science (No. 2020090600002).

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, China

    Xiaogan Jie, Tong Liu, Honghao Gao, Chenhong Cao, Peng Wang & Weiqin Tong

  2. Shanghai Key Laboratory of Data Science, Shanghai, China

    Tong Liu

  3. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai, China

    Tong Liu, Chenhong Cao & Weiqin Tong

  4. Shanghai Engineering Research Center of Intelligent Computing System, Shanghai, China

    Tong Liu & Weiqin Tong

Authors
  1. Xiaogan Jie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Honghao Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chenhong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Weiqin Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tong Liu .

Editor information

Editors and Affiliations

  1. Shanghai University, Shanghai, China

    Honghao Gao

  2. Xi’an Jiaotong-Liverpool University, Suzhou, China

    Xinheng Wang

  3. London South Bank University, London, UK

    Muddesar Iqbal

  4. Hangzhou Dianzi University, Hangzhou, China

    Yuyu Yin

  5. Zhejiang University, Hangzhou, China

    Jianwei Yin

  6. Fudan University, Shanghai, China

    Ning Gu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jie, X., Liu, T., Gao, H., Cao, C., Wang, P., Tong, W. (2021). A DQN-Based Approach for Online Service Placement in Mobile Edge Computing. In: Gao, H., Wang, X., Iqbal, M., Yin, Y., Yin, J., Gu, N. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 350. Springer, Cham. https://doi.org/10.1007/978-3-030-67540-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-67540-0_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-67539-4

  • Online ISBN: 978-3-030-67540-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics