Skip to main content
Springer Nature Link
Log in

Multi objective dynamic task scheduling optimization algorithm based on deep reinforcement learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In the stochastic and dynamic edge-cloud collaborative environment, the computing resources of the host are limited, and the resource requirements of computing tasks are random and changeable. Therefore, how to efficiently schedule dynamic tasks and improve system performance becomes challenging. The scheduling algorithm based on deep reinforcement learning optimizes the delay and energy consumption of the system by dynamically interacting with the environment, which can solve the problem of dynamic and changeable environment to a certain extent, but there are still problems such as poor model adaptability, low training efficiency, and unbalanced system load. In this paper, aiming at optimizing the average response time of task scheduling and the average energy consumption of the system, a multi-objective task scheduling model is designed, and a task scheduling policy optimization algorithm based on improved asynchronous advantage actor-critic (A3C) is proposed. The residual convolutional neural network (RCNN) improves the network structure of A3C, using asynchronous multi-threaded training methods to interact with the edge-cloud collaborative environment and capturing the random dynamic characteristics of resources required for computing tasks and heterogeneous edge-cloud hosts resource change characteristics to better adapt to random dynamic environments, and it can quickly updating network parameters, improving training speed, and make full use of host resources to solve system load imbalance problems. Simulation results show that the scheduling algorithm proposed in this paper can effectively reduce task response time and system energy consumption in an edge-cloud collaborative environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data presented in this study are available upon request from the corresponding.

Notes

  1. http://gwa.ewi.tudelft.nl/datasets/gwa-t-12-bitbrains.

References

  1. Shi W, Cao J, Zhang Q, Li Y, Xu L (2016) Edge computing: vision and challenges. IEEE Internet Things J 3(5):637–646

    Article  Google Scholar 

  2. Du M, Wang Y, Ye K, Xu C (2020) Algorithmics of cost-driven computation offloading in the edge-cloud environment. IEEE Trans Comput 69(10):1519–1532

    Article  MathSciNet  Google Scholar 

  3. Gill SS, Tuli S, Xu M, Singh I, Singh KV, Lindsay D et al (2019) Transformative effects of IoT, Blockchain and Artificial Intelligence on cloud computing: evolution, vision, trends and open challenges. Internet Things 8:100118

    Article  Google Scholar 

  4. Shyalika C, Silva T, Karunananda A (2020) Reinforcement learning in dynamic task scheduling: a review. SN Comput Sci 1(6):1–17

    Article  Google Scholar 

  5. Fox G, Glazier JA, Kadupitiya JCS, et al. (2019) Learning everywhere: Pervasive machine learning for effective high-performance computation. In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, pp 422–429

  6. Topcuoglu H, Hariri S, Wu MY (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  7. Beloglazov A, Buyya R (2012) Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in cloud data centers. Concurr Comput: Pract Exp 24(13):1397–1420

    Article  Google Scholar 

  8. Choudhari T, Moh M, Moh TS (2018) Prioritized task scheduling in fog computing. In: Proceedings of the ACMSE 2018 Conference, pp 1–8

  9. Yan H, Li Y, Zhu X, Zhang D, Wang J, Chen H, Bao W (2021) EASE: Energy-efficient task scheduling for edge computing under uncertain runtime and unstable communication conditions. Concurr Comput: Pract Exp 33(7):1–1

    Article  Google Scholar 

  10. Chen X, Jiao L, Li W, Fu X (2015) Efficient multi-user computation offloading for mobile-edge cloud computing. IEEE/ACM Trans Netw 24(5):2795–2808

    Article  Google Scholar 

  11. Sheng S, Chen P, Chen Z, Wu L, Yao Y (2021) Deep reinforcement learning-based task scheduling in iot edge computing. Sensors 21(5):1666

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  12. Tang Z, Jia W, Zhou X, Yang W, You Y (2020) Representation and reinforcement learning for task scheduling in edge computing. IEEE Trans Big Data 8(3):795–808

    Article  Google Scholar 

  13. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  ADS  CAS  PubMed  Google Scholar 

  14. Arulkumaran K, Deisenroth MP, Brundage M et al (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34(6):26–38

    Article  ADS  Google Scholar 

  15. Basu D, Wang X, Hong Y, Chen H, Bressan S (2019) Learn-as-you-go with megh: efficient live migration of virtual machines. IEEE Trans Parallel Distrib Syst 30(8):1786–1801

    Article  Google Scholar 

  16. Johnson VM, Rogers LL (2000) Accuracy of neural network approximators in simulation-optimization. J Water Resour Plan Manag 126(2):48–56

    Article  Google Scholar 

  17. Chai X (2020) Task scheduling based on swarm intelligence algorithms in high performance computing environment. J Ambient Intell Hum Comput. https://doi.org/10.1007/s12652-020-02255-w

    Article  Google Scholar 

  18. Liu S, Wang N (2020) Collaborative optimization scheduling of cloud service resources based on improved genetic algorithm. IEEE Access 8:150878–150890

    Article  Google Scholar 

  19. Aburukba RO, Landolsi T, Omer D (2021) A heuristic scheduling approach for fog-cloud computing environment with stationary IoT devices. J Netw Comput Appl 180:102994

    Article  Google Scholar 

  20. Li Z, Peng E (2021) Software-defined optimal computation task scheduling in vehicular edge networking. Sensors 21(3):955

    Article  ADS  PubMed  PubMed Central  Google Scholar 

  21. Talaat FM, Ali HA, Saraya MS, Saleh AI (2022) Effective scheduling algorithm for load balancing in fog environment using CNN and MPSO. Knowl Inf Syst 64(3):773–797

    Article  Google Scholar 

  22. Gao ZH, Hao WM, Han Z, Yang SY (2020) Q-learning-based task offloading and resources optimization for a collaborative computing system. IEEE Access 8:149011–149024

    Article  Google Scholar 

  23. Deng YQ, Chen ZG, Yao X, Hassan S, Ibrahim AMA (2019) Parallel offloading in green and sustainable mobile edge computing for delay-constrained IoT system. IEEE Trans Veh Technol 68(12):12202–12214

    Article  Google Scholar 

  24. Nguyen BM, Binh HTT, Anh TT, Son DB (2019) Evolutionary algorithms to optimize task scheduling problem for the iot based bag-of-tasks application in cloud-fog computing environment. Appl Sci-Basel 9(9):1730

    Article  Google Scholar 

  25. Huang PQ, Wang Y, Wang KZ, Liu ZZ (2020) A bilevel optimization approach for joint offloading decision and resource allocation in cooperative mobile edge computing. IEEE Trans Cybern 50(10):4228–4241

    Article  PubMed  Google Scholar 

  26. Chen L, Guo K, Fan G et al (2020) Resource constrained profit optimization method for task scheduling in edge cloud. IEEE Access 8:118638–118652

    Article  Google Scholar 

  27. Ranadheera S, Maghsudi S, Hossain E (2018) Computation offloading and activation of mobile edge computing servers: a minority game. IEEE Wirel Commun Lett 7(5):688–691

    Article  Google Scholar 

  28. Kim HJ, Lee DH, Lee JM, Lee KH, Lyu W, Choi SG (2008) The QoE evaluation method through the QoS-QoE correlation model. In: 2008 4th International Conference on Networked Computing and Advanced Information Management. IEEE, pp 719–725

  29. Tuli S, Poojara SR, Srirama SN et al (2021) COSCO: container orchestration using co-simulation and gradient based optimization for fog computing environments. IEEE Trans Parallel Distrib Syst 33(1):101–116

    Article  Google Scholar 

  30. Mnih V, Badia AP, Mirza M, et al. (2016) Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning. PMLR, pp 1928–1937

  31. Bogolubsky L, Dvurechenskii P, Gasnikov A, et al. (2016) Learning supervised pagerank with gradient-based and gradient-free optimization methods. Adv Neural Inf Process Syst 29

  32. Ding Y, Yang F, Feng J (2020) Intelligent cognitive anti-jamming algorithm based on long short-term memory network. In: IEEE 3rd International Conference on Electronics and Communication Engineering (ICECE). IEEE, pp 76–82

  33. Sun F, Kong X, Wu J et al (2022) DSM pricing method based on A3C and LSTM under cloud-edge environment. Appl Energy 315:118853

    Article  Google Scholar 

  34. Guo S, Zhang K, Gong B et al (2021) A delay-sensitive resource allocation algorithm for container cluster in edge computing environment. Comput Commun 170:144–150

    Article  Google Scholar 

  35. Auer G, Giannini V, Desset C et al (2011) How much energy is needed to run a wireless network. IEEE Wirel Commun 18(5):40–49

    Article  Google Scholar 

  36. Calheiros RN, Ranjan R, Beloglazov A, De Rose CA, Buyya R (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw: Pract Exp 41(1):23–50

    Google Scholar 

  37. Gupta H, Vahid Dastjerdi A, Ghosh SK, Buyya R (2017) iFogSim: a toolkit for modeling and simulation of resource management techniques in the Internet of Things, Edge and Fog computing environments. Softw Pract Exp 47(9):1275–1296

    Article  Google Scholar 

  38. Shen S, Van Beek V, Iosup A (2015) Statistical characterization of business-critical workloads hosted in cloud datacenters. In: 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, pp 465–474

Download references

Funding

This work is supported by Liaoning Province Applied Basic Research Program Project (Grant No. 2023JH2/101300195).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Dalian Maritime University, Dalian, 116026, China

    Yuqing Cheng, Zhiying Cao, Xiuguo Zhang & Dezhen Zhang

  2. School of Computer Science and Technology, Shandong Technology and Business University, Yantai, 264003, China

    Qilei Cao

Authors
  1. Yuqing Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhiying Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiuguo Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qilei Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Dezhen Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

All authors contributed equally to this work. All authors have read and agreed to this manuscript.

Corresponding authors

Correspondence to Zhiying Cao or Xiuguo Zhang.

Ethics declarations

Conflict of interest

The authors declare no competing interest with respect to this manuscript.

Ethical approval

No ethical approval was required for this research.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, Y., Cao, Z., Zhang, X. et al. Multi objective dynamic task scheduling optimization algorithm based on deep reinforcement learning. J Supercomput 80, 6917–6945 (2024). https://doi.org/10.1007/s11227-023-05714-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05714-1

Keywords