Skip to main content
Springer Nature Link
Log in

Research of Offloading Decision and Resource Scheduling in Edge Computing Based on Deep Reinforcement Learning

  • Conference paper
  • First Online:
Smart Grid and Internet of Things (SGIoT 2020)

Included in the following conference series:

  • 878 Accesses

Abstract

The increasing scale of the IOT poses challenges to the energy consumption, transmission bandwidth and processing delay of centralized cloud computing data centers. The cloud computing data centers is moving from the center of the network to edge nodes with lower latency, namely, edge computing. Meanwhile, it can meet the needs of users for real-time services. In the field of edge computing, offloading decision and resource scheduling are the hot-spot issues. As for offloading decision and resource scheduling problems of single-cell multi-user partial offloading, the system model is also firstly established from four aspects: network architecture, application type, local computing and offloading computing. Based on the system model, the optimization problem of resource scheduling is modeled, where the solution is hard to be found. Thus, the deep reinforcement learning method based on policy gradient is selected to establish the SPBDDPG algorithm that can solve the problem. Then, in order to solve the practical problems, the SPBDDPG algorithm is set up with the state and action for iteration, as well as the environment for generating new state and feedback reward value. Finally, an appropriate iteration step is written for the edge computing resource scheduling problem by combining with the original deep reinforcement learning algorithm. We also evaluate the proposed approaches by relevant experiments. The complexity and effectiveness of the results are validated.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Sardellitti, S., Scutari, G., Barbarossa, S.: Joint optimization of radio and computational resources for multicell mobile-edge computing. IEEE Trans. Signal Inf. Process. Over Netw. 1(2), 89–103 (2014)

    Article  MathSciNet  Google Scholar 

  2. Kumar, K., Lu, Y.H.: Cloud computing for mobile users: can offloading computation save energy. Computer 43(4), 51–56 (2010)

    Article  Google Scholar 

  3. Wu, H., Wang, Q., Katinka, W.: Tradeoff between performance improvement and energy saving in mobile cloud offloading systems. In: 2013 IEEE International Conference on Communications Workshops (ICC), pp. 728–732 (2013)

    Google Scholar 

  4. Barbarossa, S., Sardellitti, S., Lorenzo, P.D.: Joint allocation of computation and communication resources in multiuser mobile cloud computing. In: 2013 IEEE 14th Workshop on Signal Processing Advances in Wireless Communications (SPAWC), pp. 26–30 (2013)

    Google Scholar 

  5. Munoz, O., Pascual-Iserte, A., Vidal, J.: Optimization of radio and computational resources for energy efficiency in latency-constrained application offloading. IEEE Trans. Veh. Technol. 64(10), 4738–4755 (2015)

    Article  Google Scholar 

  6. Huang, D., Wang, P., Niyato, D.: A dynamic offloading algorithm for mobile computing. IEEE Trans. Wirel. Commun. 11(6), 1991–1995 (2012)

    Article  Google Scholar 

  7. Di Lorenzo, P., Barbarossa, S., Sardellitti, S.: Joint Optimization of Radio Resources and Code Partitioning in Mobile Edge Computing (2013). https://arxiv.org/abs/1307.3835.

  8. Yang, L., Cao, J., Cheng, H., et al.: Multi-user computation partitioning for latency sensitive mobile cloud applications. IEEE Trans. Comput. 64(8), 2253–2266 (2015)

    Article  MathSciNet  Google Scholar 

  9. Yang, L., Cao, J., Yuan, Y., et al.: A framework for partitioning and execution of data stream applications in mobile cloud computing. ACM SIGMETRICS Perform. Eval. Rev. 40(4), 23–32 (2013)

    Article  Google Scholar 

  10. Zhang, W., Wen, Y., Guan, K., et al.: Energy-optimal mobile cloud computing under stochastic wireless channel. IEEE Trans. Wirel. Commun. 12(9), 4569–4581 (2013)

    Article  Google Scholar 

  11. Chen, Z., Wang, X.: Decentralized computation offloading for multi-user mobile edge computing: a deep reinforcement learning approach (2018)

    Google Scholar 

  12. Zeng, D., Gu, L., Pan, S., et al.: resource management at the network edge: a deep reinforcement learning approach. IEEE Netw. 33(3), 26–33 (2019)

    Article  Google Scholar 

  13. Chen, X., Zhang, H., Wu, C., et al.: Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J. 6, 4005–4018 (2018)

    Article  Google Scholar 

  14. Mao, Y., You, C., Zhang, J., Huang, K., Letaief, K.B.: A survey on mobile edge computing: the communication perspective. IEEE Commun. Surv. Tuts. 19(4), 2322–2358 (2017)

    Article  Google Scholar 

  15. Mao, Y., Zhang, J., Song, S.H., Letaief, K.B.: Stochastic joint radio and computational resource management for multi-user mobile-edge computing systems. IEEE Trans. Wirel. Commun. 16(9), 5994–6009 (2017)

    Article  Google Scholar 

  16. Lyu, X., et al.: Optimal schedule of mobile edge computing for Internet of Things using partial information. IEEE J. Sel. Areas Commun. 35(11), 2606–2615 (2017)

    Article  Google Scholar 

  17. Liu, J., Mao, Y., Zhang, J., Letaief, K.B.: Delay-optimal computation task scheduling for mobile-edge computing systems. In: Proceedings of IEEE International Symposium Information Theory (ISIT), pp. 1451–1455 (2016)

    Google Scholar 

  18. Hong, S.-T., Kim, H.: QoE-aware computation offloading scheduling to capture energy-latency tradeoff in mobile clouds. In: Proceedings of 13th Annual IEEE International Conference Sensing, Communication Networking (SECON), pp. 1–9 (2016)

    Google Scholar 

  19. Li, J., Gao, H., Lv, T., Lu, Y.: Deep reinforcement learning based computation offloading and resource allocation for MEC. In: Proceedings of IEEE Wireless Communications, Networking Conference (WCNC), pp. 1–6 (2018)

    Google Scholar 

  20. FP7 European Project. Distributed Computing, Storage and Radio Resource Allocation Over Cooperative Femtocells (TROPIC) [EB/OL]. https://www.ict-tropic.eu.

Download references

Acknowledgement

The authors gratefully acknowledge the support and financial assistance provided by the National Natural Science Foundation under Grant No. 61772064 and 61701019, the National Key R&D Program of China under Grant No. 2018YFC0831900. The authors thank the anonymous reviewers who provided constructive feedback on earlier work of this paper.

Author information

Authors and Affiliations

  1. Department of Software Engineering, Key Laboratory of Communication and Information Systems, Beijing Municipal Commission of Education, Beijing Jiaotong University, Beijing, 100044, China

    Zhen-Jiang Zhang

  2. Department of Electronic and Information Engineering, Key Laboratory of Communication and Information Systems, Beijing Municipal Commission of Education, Beijing Jiaotong University, Beijing, 100044, China

    Tong Wu, Zhiyuan Li, Bo Shen & Jian Li

  3. School of Computer and Information Technology, Beijing Jiaotong University, Beijing Municipal Commission of Education, Beijing, 100044, China

    Naiyue Chen

Authors
  1. Zhen-Jiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhiyuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bo Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Naiyue Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jian Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhen-Jiang Zhang .

Editor information

Editors and Affiliations

  1. National Chiao Tung University, Hsinchu, Taiwan

    Yi-Bing Lin

  2. National Changhua University of Education, Changhua, Taiwan

    Der-Jiunn Deng

Rights and permissions

Reprints and permissions

Copyright information

© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, ZJ., Wu, T., Li, Z., Shen, B., Chen, N., Li, J. (2021). Research of Offloading Decision and Resource Scheduling in Edge Computing Based on Deep Reinforcement Learning. In: Lin, YB., Deng, DJ. (eds) Smart Grid and Internet of Things. SGIoT 2020. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 354. Springer, Cham. https://doi.org/10.1007/978-3-030-69514-9_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-69514-9_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-69513-2

  • Online ISBN: 978-3-030-69514-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics