Skip to main content
SpringerLink
Log in

QL-HEFT: a novel machine learning scheduling scheme base on cloud computing environment

  • Advances in Parallel and Distributed Computing for Neural Computing
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Cloud computing is a computing model that fully utilizes the resources on the Internet to maximize the utilization of resources. Due to a large number of users and tasks, it is important to achieve efficient scheduling of tasks submitted by users. Task scheduling is one of the crucial and challenging non-deterministic polynomial-hard problems in cloud computing. In task scheduling, obtaining shorter makespan is an important objective and is related to the pros and cons of the algorithm. Machine learning algorithms represent a new method for solving this type of problem. In this paper, we propose a novel task scheduling algorithm called QL-HEFT that combines Q-learning with the heterogeneous earliest finish time (HEFT) algorithm to reduce the makespan. The algorithm uses the upward rank (ranku) value of HEFT as the immediate reward in the Q-learning framework. The agent can obtain better learning results to update the Q-table through the self-learning process. The QL-HEFT algorithm is divided into two major phases: a task sorting phase based on Q-learning for obtaining an optimal order and a processor allocation phase using the earliest finish time strategy. Experiments show that QL-HEFT achieves a shorter makespan compared to three other classical scheduling algorithms as well as good performances in terms of the average response time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Zimba A, Hongsong C (2016) Analyzing trust concerns in public clouds using finite state automata. In: 2016 2nd international conference on cloud computing and internet of things (CCIOT). IEEE, pp 25–29

  2. Li K, Mei J, Li K (2018) A fund-constrained investment scheme for profit maximization in cloud computing. IEEE Trans Serv Comput 11(6):893–907

    Article  Google Scholar 

  3. Lee J (2013) A view of cloud computing. Int J Netw Distrib Comput 1(1):2–8

    Article  Google Scholar 

  4. Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I (2010) A view of cloud computing. Commun ACM 53(4):50–58

    Article  Google Scholar 

  5. Liu C, Li K, Li K, Buyya R (2017) A new cloud service mechanism for profit optimizations of a cloud provider and its users. IEEE Trans Cloud Comput 7:1

    Google Scholar 

  6. Zhang L, Tong W, Lu S (2014) Task scheduling of cloud computing based on Improved CHC algorithm. In: 2014 international conference on audio, language and image processing (ICALIP). IEEE, pp 574–577

  7. Singh P, Walia NK (2016) A review: cloud computing using various task scheduling algorithms. Int J Comput Appl 142(7):30–32

    Google Scholar 

  8. Topcuoglu H, Hariri S, M-y Wu (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274

    Article  Google Scholar 

  9. Xu Y, Li K, Hu J, Li K (2014) A genetic algorithm for task scheduling on heterogeneous computing systems using multiple priority queues. Inf Sci 270:255–287

    Article  MathSciNet  MATH  Google Scholar 

  10. Mullainathan S, Spiess J (2017) Machine learning: an applied econometric approach. J Econ Perspect 31(2):87–106

    Article  Google Scholar 

  11. Chorowski JK, Bahdanau D, Serdyuk D, Cho K, Bengio Y (2015) Attention-based models for speech recognition. In: Advances in neural information processing systems, pp 577–585

  12. Xiong W, Droppo J, Huang X, Seide F, Seltzer M, Stolcke A, Yu D, Zweig G (2017) The microsoft 2016 conversational speech recognition system. In: IEEE international conference on acoustics, speech and signal processing, pp 5255–5259

  13. Liu Z, Luo P, Wang X, Tang X (2015) Deep learning face attributes in the wild. In: Proceedings of the IEEE international conference on computer vision, pp 3730–3738

  14. Duan M, Li K, Liao X, Li K (2018) A parallel multiclassification algorithm for big data using an extreme learning machine. IEEE Trans Neural Netw Learn Syst 29(6):2337–2351

    Article  MathSciNet  Google Scholar 

  15. Tong Z, Xiao Z, Li K, Li K (2014) Proactive scheduling in distributed computing—a reinforcement learning approach. J Parallel Distrib Comput 74(7):2662–2672

    Article  Google Scholar 

  16. Siar H, Nabavi SH, Shahaboddin S (2010) Static task scheduling in cooperative distributed systems based on soft computing techniques. Aust J Basic Appl Sci 4(6):1518–1526

    Google Scholar 

  17. Xiao Z, Tong Z, Li K, Li K (2017) Learning non-cooperative game for load balancing under self-interested distributed environment. Appl Soft Comput 52:376–386

    Article  Google Scholar 

  18. Xu C-Z, Rao J, Bu X (2012) URL: a unified reinforcement learning approach for autonomic cloud management. J Parallel Distrib Comput 72(2):95–105

    Article  Google Scholar 

  19. Xiao Z, Liang P, Tong Z, Li K, Khan SU, Li K (2017) Self-adaptation and mutual adaptation for distributed scheduling in benevolent clouds. Concurrency Comput Pract Exp 29(5):1–12

    Article  Google Scholar 

  20. Sutton RS, Barto AG (1998) Introduction to reinforcement learning. MIT press, Cambridge

    Book  MATH  Google Scholar 

  21. Kober J, Bagnell JA, Peters J (2013) Reinforcement learning in robotics: a survey. Int J Robot Res 32(11):1238–1274

    Article  Google Scholar 

  22. Roy A, Xu H, Pokutta S (2017) Reinforcement learning under model mismatch. In: Advances in neural information processing systems, pp 3043–3052

  23. Peng Z, Cui D, Zuo J, Li Q, Xu B, Lin W (2015) Random task scheduling scheme based on reinforcement learning in cloud computing. Cluster Comput 18(4):1595–1607

    Article  Google Scholar 

  24. Wei Z, Zhang Y, Xu X, Shi L, Feng L (2017) A task scheduling algorithm based on Q-learning and shared value function for WSNs. Comput Netw 126:141–149

    Article  Google Scholar 

  25. Agarwal M, Srivastava GMS (2016) A genetic algorithm inspired task scheduling in cloud computing. In: 2016 international conference on computing, communication and automation (ICCCA). IEEE, pp 364–367

  26. Tang Z, Zhang X, Li K, Li K (2018) An intermediate data placement algorithm for load balancing in Spark computing environment. Future Gener Comput Syst 78:287–301

    Article  Google Scholar 

  27. Mittal S, Katal A (2016) An optimized task scheduling algorithm in cloud computing. In: 2016 IEEE 6th international conference on advanced computing (IACC). IEEE, pp 197–202

  28. Liu J, Li K, Zhu D, Han J, Li K (2017) Minimizing cost of scheduling tasks on heterogeneous multicore embedded systems. ACM Trans Embed Comput Syst (TECS) 16(2):36

    Google Scholar 

  29. Xu L, Wang K, Ouyang Z, Qi X (2014) An improved binary PSO-based task scheduling algorithm in green cloud computing. In: 2014 9th international conference on communications and networking in China (CHINACOM). IEEE, pp 126–131

  30. Aladwani T (2017) Improving tasks scheduling performance in cloud computing environment by using analytic hierarchy process model. In: international conference on green informatics (ICGI). IEEE, pp 98–104

  31. Akbar MF, Munir EU, Rafique MM, Malik Z, Khan SU, Yang LT (2017) List-based task scheduling for cloud computing. In: IEEE international conference on internet of things, pp 652–659

  32. Li K (2016) Energy and time constrained task scheduling on multiprocessor computers with discrete speed levels. J Parallel Distrib Comput 95:15–28

    Article  Google Scholar 

  33. Arabnejad H, Barbosa JG (2014) List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans Parallel Distrib Syst 25(3):682–694

    Article  Google Scholar 

  34. Akbar MF, Munir EU, Rafique MM, Malik Z, Khan SU, Yang LT (2016) List-based task scheduling for cloud computing. In: 2016 IEEE international conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). IEEE, pp 652–659

  35. Wang G, Wang Y, Liu H, Guo H (2016) HSIP: a Novel Task Scheduling Algorithm for Heterogeneous Computing. Sci Program 2016:1–11

    Google Scholar 

  36. Sirisha D, Vijaya Kumari G (2016) Minimal start time heuristics for scheduling workflows in heterogeneous computing systems. In: International conference on distributed computing and internet technology. Springer, pp 199–212

  37. Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT Press, Cambridge

    MATH  Google Scholar 

  38. Cui D, Ke W, Peng Z, Zuo J (2015) Multiple DAGs workflow scheduling algorithm based on reinforcement learning in cloud computing. In: International symposium on intelligence computation and applications. Springer, pp 305–311

  39. Cui D, Peng Z, Lin W (2017) A reinforcement learning-based mixed job scheduler scheme for grid or IaaS cloud. IEEE Trans Cloud Comput 14(99):1

    Article  Google Scholar 

  40. Orhean AI, Pop F, Raicu I (2018) New scheduling approach using reinforcement learning for heterogeneous distributed systems. J Parallel Distrib Comput 117:292–302

    Article  Google Scholar 

  41. Moradi M (2016) A centralized reinforcement learning method for multiagent job scheduling in Grid. In: 2016 6th international conference on computer and knowledge engineering (ICCKE). IEEE, pp 171–176

  42. Li K, Li S, Xu Y, Xie Z (2014) A DAG task scheduling scheme on heterogeneous computing systems using invasive weed optimization algorithm. In: 2014 sixth international symposium on parallel architectures, algorithms and programming (PAAP). IEEE, pp 262–267

  43. Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

    Article  Google Scholar 

  44. Nie J, Haykin S (1999) A Q-learning-based dynamic channel assignment technique for mobile communication systems. IEEE Trans Veh Technol 48(5):1676–1687

    Article  Google Scholar 

  45. Calheiros RN, Ranjan R, Beloglazov A, De Rose CA, Buyya R (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23–50

    Article  Google Scholar 

  46. Chen W, Deelman E (2012) Workflowsim: a toolkit for simulating scientific workflows in distributed environments. In: 2012 IEEE 8th international conference on E-science (e-science). IEEE, pp 1–8

  47. Bharathi S, Chervenak A, Deelman E, Mehta G, Su M-H, Vahi K (2008) Characterization of scientific workflows. In: Third workshop on workflows in support of large-scale science, 2008. WORKS 2008. IEEE, pp 1–10

  48. Juve G, Chervenak A, Deelman E, Bharathi S, Mehta G, Vahi K (2013) Characterizing and profiling scientific workflows. Future Gener Comput Syst 29(3):682–692

    Article  Google Scholar 

  49. Kianpisheh S, Charkari NM, Kargahi M (2016) Ant colony based constrained workflow scheduling for heterogeneous computing systems. Cluster Comput 19(3):1053–1070

    Article  Google Scholar 

  50. Tong Z, Chen H, Deng X, Li K, Li K (2018) A novel task scheduling scheme in a cloud computing environment using hybrid biogeography-based optimization. Soft Comput 11:1–20

    Google Scholar 

  51. Arabnejad V, Bubendorfer K, Ng B (2017) Scheduling deadline constrained scientific workflows on dynamically provisioned cloud resources. Future Gener Comput Syst 75:348–364

    Article  Google Scholar 

  52. Brown DA, Brady PR, Dietz A, Cao J, Johnson B, McNabb J (2007) A case study on the use of workflow technologies for scientific analysis: gravitational wave data analysis. In: Workflows for e-Science. Springer, pp 39–59

  53. Deelman E, Callaghan S, Field E, Francoeur H, Graves R, Gupta N, Gupta V, Jordan TH, Kesselman C, Maechling P (2006) Managing large-scale workflow execution from resource provisioning to provenance tracking: The cybershake example. In: Second IEEE international conference on e-Science and grid computing, 2006. e-Science’06. IEEE, p 14

  54. Montage: An astronomical image engine. http://montage.ipac.caltech.edu. Accessed 10 Aug 2018

  55. USC Epigenome Center. http://epigenome.usc.edu. Accessed 10 Aug 2018

Download references

Acknowledgements

This research was partially funded by the Program of National Natural Science Foundation of China (Grant No. 61502165, 61602170), the National Outstanding Youth Science Program of National Natural Science Foundation of China (Grant No. 61625202) and the Research Foundation of Education Bureau of Hunan Province, China (Grant No. 17C0959).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Hunan Normal University, Changsha, 410012, Hunan, China

    Zhao Tong, Xiaomei Deng, Hongjian Chen, Jing Mei & Hong Liu

Authors
  1. Zhao Tong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaomei Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongjian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhao Tong.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tong, Z., Deng, X., Chen, H. et al. QL-HEFT: a novel machine learning scheduling scheme base on cloud computing environment. Neural Comput & Applic 32, 5553–5570 (2020). https://doi.org/10.1007/s00521-019-04118-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04118-8

Keywords

Search

Navigation