Skip to main content

A Deep Reinforcement Learning Approach Towards Computation Offloading for Mobile Edge Computing

  • Conference paper
  • First Online:
Human Centered Computing (HCC 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11956))

Included in the following conference series:

  • 1476 Accesses

Abstract

In order to improve the quality of service for users and reduce the energy consumption of the cloud computing environment, Mobile Edge Computing (MEC) is a promising paradigm by providing computing resources which is close to the end device in physical distance. Nevertheless, the computation offloading policy to satisfy the requirements of the service provider and consumer at the same time within a MEC system still remains challenging. In this paper, we propose an offloading decision policy with three-level structure for MEC system different from the traditional two-level architecture to formulate the offloading decision optimization problem by minimizing the total cost of energy consumption and delay time. Because the traditional optimization methods could not solve this dynamic system problem efficiently, Reinforcement Learning (RL) has been used in complex control systems in recent years. We design a deep reinforcement learning (DRL) approach to minimize the total cost by applying deep Q-learning algorithm to address the issues of too large system state dimension. The simulation results show that the proposed algorithm has nearly optimal performance than traditional methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Varghese, B., Buyya, R.: Next generation cloud computing: new trends and research directions. Future Gener. Comput. Syst. 79, 849–861 (2018)

    Article  Google Scholar 

  2. Dave, E.: How the next evolution of the Internet is changing everything. The Internet of Things (2011)

    Google Scholar 

  3. Abbas, N., Zhang, Y., Taherkordi, A., Skeie, T.: Mobile edge computing: a survey. IEEE Internet Things J. 5(1), 450–465 (2017)

    Article  Google Scholar 

  4. Shi, W., Cao, J., Zhang, Q., Li, Y., Xu, L.: Edge computing: vision and challenges. IEEE Internet Things J. 3(5), 637–646 (2016)

    Article  Google Scholar 

  5. Kiumarsi, B., Vamvoudakis, K.G., Modares, H., Lewis, F.L.: Optimal and autonomous control using reinforcement learning: a survey. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2042–2062 (2018)

    Article  MathSciNet  Google Scholar 

  6. Guan, L., Ke, X., Song, M., Song, J.: A survey of research on mobile cloud computing. In: 2011 10th IEEE/ACIS International Conference on Computer and Information Science, pp. 387–392. IEEE (May 2011)

    Google Scholar 

  7. Fernando, N., Loke, S.W., Rahayu, W.: Mobile cloud computing: a survey. Future Gener. Comput. Syst. 29(1), 84–106 (2013)

    Article  Google Scholar 

  8. Chen, X.: Decentralized computation offloading game for mobile cloud computing. IEEE Trans. Parallel Distrib. Syst. 26(4), 974–983 (2014)

    Article  Google Scholar 

  9. Deng, S., Huang, L., Taheri, J., Zomaya, A.Y.: Computation offloading for service workflow in mobile cloud computing. IEEE Trans. Parallel Distrib. Syst. 26(12), 3317–3329 (2014)

    Article  Google Scholar 

  10. Guo, S., Xiao, B., Yang, Y., Yang, Y.: Energy-efficient dynamic offloading and resource scheduling in mobile cloud computing. In IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, pp. 1–9. IEEE (April 2016)

    Google Scholar 

  11. Chen, M., Hao, Y.: Task offloading for mobile edge computing in software defined ultra-dense network. IEEE J. Sel. Areas Commun. 36(3), 587–597 (2018)

    Article  MathSciNet  Google Scholar 

  12. Zhang, K., et al.: Energy-efficient offloading for mobile edge computing in 5G heterogeneous networks. IEEE Access 4, 5896–5907 (2016)

    Article  Google Scholar 

  13. Zhang, J., et al.: Energy-latency tradeoff for energy-aware offloading in mobile edge computing networks. IEEE Internet Things J. 5(4), 2633–2645 (2018)

    Article  MathSciNet  Google Scholar 

  14. Xu, Z., Wang, Y., Tang, J., et al.: A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs. In: 2017 IEEE International Conference on Communications (ICC), pp. 1–6 (2017)

    Google Scholar 

  15. Huang, L., Bi, S., Zhang, Y.J.A.: Deep reinforcement learning for online offload in wireless powered mobile-edge computing networks. arXiv preprint. arXiv:1808.01977 (2018)

  16. Mach, P., Becvar, Z.: Mobile edge computing: a survey on architecture and computation offloading. IEEE Commun. Surv. Tutor. 19(3), 1628–1656 (2017)

    Article  Google Scholar 

  17. Zhang, W., Wen, Y., Guan, K., Kilper, D., Luo, H., Wu, D.O.: Energy-optimal mobile cloud computing under stochastic wireless channel. IEEE Trans. Wirel. Commun. 12(9), 4569–4581 (2013)

    Article  Google Scholar 

  18. Miettinen, A.P., Nurminen, J.K.: Energy efficiency of mobile clients in cloud computing. HotCloud 10, 4 (2010)

    Google Scholar 

  19. Chen, X., Jiao, L., Li, W., Fu, X.: Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans. Netw. 24(5), 2795–2808 (2015)

    Article  Google Scholar 

  20. Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)

    MATH  Google Scholar 

  21. Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: Thirtieth AAAI Conference on Artificial Intelligence (March 2016)

    Google Scholar 

Download references

Acknowledgments

The paper is supported in part by the National Natural Science Foundation of China under Grant No. 61672022, and Key Disciplines of Computer Science and Technology of Shanghai Polytechnic University under Grant No. XXKZD1604.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Wenan Tan .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Q., Tan, W., Qin, X. (2019). A Deep Reinforcement Learning Approach Towards Computation Offloading for Mobile Edge Computing. In: Milošević, D., Tang, Y., Zu, Q. (eds) Human Centered Computing. HCC 2019. Lecture Notes in Computer Science(), vol 11956. Springer, Cham. https://doi.org/10.1007/978-3-030-37429-7_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37429-7_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37428-0

  • Online ISBN: 978-3-030-37429-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics