Skip to main content

Advertisement

SpringerLink
Log in

Workload-driven coordination between virtual machine allocation and task scheduling

  • Advances in Parallel and Distributed Computing for Neural Computing
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The current task scheduling is separated from the virtual machine (VM) allocation, which, to some extent, wastes resources or degrades application performance. The scheduling algorithm influences the demand of VMs in terms of service-level agreement, while the number of VMs determines the performance of task scheduling. Workload plays an indispensable role in both dynamic VM allocation and task scheduling. To address this problem, we coordinate task scheduling and VM allocation based on workload characteristics. Workload is empirically time-varying and stochastic. We demonstrate that the acquired workload data set has Markov property which can be modeled as a Markov chain. Then, three workload characteristic operators are extracted: persistence, recurrence and entropy, which quantify the relative stability, burstiness, and unpredictability of the workload, respectively. Experiments indicate that the persistence and recurrence of workloads has a direct bearing on the average response time and resource utilization of the system. A nonlinear model between the load characteristic operators and the number of VMs is established. In order to test the performance of the collaborative framework, we design a scheduling algorithm based on genetic algorithm (GA), which takes the estimated number of VMs as input and the task completion time as the optimization target. Simulation experiments have been performed on the CloudSim platform, testifying that the estimated average absolute VMs error is only 2.6%. The GA-based task scheduling algorithm could improve resource utilization and reduce task completion time compared with the first come first serve and greedy algorithm. The proposed coordination mechanism in this paper has proved able to find the optimal match and reduce the resource cost by utilizing the interaction between VM allocation and task scheduling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. Buyya R, Yeo CS, Venugopal S (2008) Market-oriented cloud computing: vision, hype, and reality for delivering it services as computing utilities. In: International Conferences on High Performance Computing and Communications, vol 11, no 4. IEEE, pp 5–13

  2. Crovella ME, Bestavros A (1996) Self-similarity in world wide web traffic: evidence and possible causes. In: ACM SIGMETRICS international conference on measurement and modeling of computer systems, pp 160–169

  3. Dave YP, Shelat AS, Patel DS, Jhaveri RH (2015) Various job scheduling algorithms in cloud computing: a survey. In: International conference on information communication and embedded systems, pp 1–5

  4. Di S, Kondo D, Cirne W (2012) Characterization and comparison of cloud versus grid workloads. In: IEEE international conference on CLUSTER computing, pp 230–238

  5. Dikaiakos MD, Katsaros D, Mehra P, Pallis G, Vakali A (2009) Cloud computing: distributed internet computing for it and scientific research. IEEE Internet Comput 13(5):10–13

    Article  Google Scholar 

  6. Duy TVT, Sato Y, Inoguchi Y (2010) Performance evaluation of a green scheduling algorithm for energy savings in cloud computing. In: IEEE international symposium on parallel & distributed processing workshops & Phd forum, pp 1–8

  7. Erramilli A, Narayan O, Willinger W (1996) Experimental queueing analysis with long-range dependent packet traffic. IEEE/ACM Trans Netw 4(2):209–223

    Article  Google Scholar 

  8. Fakhfakh F, Kacem HH, Kacem AH (2014) Workflow scheduling in cloud computing: a survey. In: Enterprise distributed object computing conference workshops and demonstrations, pp 372–378

  9. Fallah M, Ghobaei Arani M, Maeen M (2015) Nasla: novel auto scaling approach based on learning automata for web application in cloud computing environment. Int J Comput Appl 113(2):18–23

    Google Scholar 

  10. Ganapathi A, Chen Y, Fox A, Katz R, Patterson D (2010) Statistics-driven workload modeling for the cloud. In: IEEE international conference on data engineering workshops, pp 87–92

  11. Gong M, Jiao L, Zhang L (2010) Baldwinian learning in clonal selection algorithm for optimization. Inf Sci 180(8):1218–1236

    Article  Google Scholar 

  12. Hirofuchi T, Nakada H, Ogawa H, Itoh S, Sekiguchi S (2010) Eliminating datacenter idle power with dynamic and intelligent VM relocation. In: de Leon F. de Carvalho AP, Rodríguez-González S, De Paz Santana JF, Rodríguez JMC (eds) Distributed computing and artificial intelligence. Advances in intelligent and soft computing, vol 79. Springer, Berlin, Heidelberg

    Google Scholar 

  13. Jung JK, Kim NU, Jung SM, Chung TM (2013) Improved CloudSim for simulating QoS-based cloud services. Springer, Dordrecht

    Book  Google Scholar 

  14. Khan A, Yan X, Tao S, Anerousis N (2012)Workload characterization and prediction in the cloud: a multiple time series approach. In: Network operations and management symposium, pp 1287–1294

  15. Lei L, Zhong Z, Zheng K, Chen J (2013) Challenges on wireless heterogeneous networks for mobile cloud computing. IEEE Wirel Commun 20(3):34–44

    Article  Google Scholar 

  16. Mathew T, Sekaran KC, Jose J (2014) Study and analysis of various task scheduling algorithms in the cloud computing environment. In: International conference on advances in computing, communications and informatics, pp 658–664

  17. Matthews J, Garfinkel T, Hoff C, Wheeler J (2009) Virtual machine contracts for datacenter and cloud computing environments, pp 25–30

  18. Mazzucco M, Dyachuk D (2012) Optimizing cloud providers revenues via energy efficient server allocation. Sustain Comput Inform Syst 2(1):1–12

    Google Scholar 

  19. Mi N, Zhang Q, Riska A, Smirni E, Riedel E (2007) Performance impacts of auto-correlated flows in multi-tiered systems. Perform Eval 64(9–12):1082–1101

    Article  Google Scholar 

  20. Mishra AK, Hellerstein JL, Cirne W, Das CR (2010) Towards characterizing cloud backend workloads: insights from Google compute clusters. ACM Sigmetric Perform Eval Rev 37(4):34–41

    Article  Google Scholar 

  21. Nakada H, Hirofuchi T, Ogawa H, Itoh S (2009) Toward virtual machine packing optimization based on genetic algorithm. In: International work-conference on artificial neural networks, pp 651–654

  22. Patil S, Kulkarni RA, Patil SH, Balaji N (2015) Performance improvement in cloud computing through dynamic task scheduling algorithm. In: International conference on next generation computing technologies, pp 96–100

  23. Prevost JJ, Nagothu KM, Kelley B, Mo J (2011) Prediction of cloud data center networks loads using stochastic and neural models. In: International conference on system of systems engineering, pp 276–281

  24. Pumma S, Achalakul T, Li X (2013) Automatic vm allocation for scientific application. In: IEEE international conference on parallel and distributed systems, pp 828–833

  25. Reiss C, Tumanov A, Ganger GR, Katz RH (2012) Towards understanding heterogeneous clouds at scale: Google trace analysis

  26. Sallam A, Li K (2014) A multi-objective virtual machine migration policy in cloud systems. Comput J 57(2):195–204

    Article  Google Scholar 

  27. Skiena S (2008) The algorithm design manual, 2nd edn. Springer, Berlin, pp 351–358

    Book  MATH  Google Scholar 

  28. Tai J, Zhang J, Li J, Meleis W, Mi N (2011) Ara: adaptive resource allocation for cloud computing environments under bursty workloads. In: PERFORMANCE computing and communications conference, pp 1–8

  29. Tang Z, Mo Y, Li K, Li K (2014) Dynamic forecast scheduling algorithm for virtual machine placement in cloud computing environment. J Supercomput 70(3):1279–1296

    Article  Google Scholar 

  30. Tang Z, Qi L, Cheng Z, Li K, Khan SU, Li K (2016) An energy-efficient task scheduling algorithm in dvfs-enabled cloud environment. J Grid Comput 14(1):55–74

    Article  Google Scholar 

  31. Wang L, Von Laszewski G, Kunze M, Tao J (2010) Schedule distributed virtual machines in a service oriented environment. In: IEEE international conference on advanced information networking and applications, pp 230–236

  32. Xiao Z, Song W, Chen Q (2013) Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans Parallel Distrib Syst 24(6):1107–1117

    Article  Google Scholar 

  33. Yufen Z, Yalin Z (2012) Research on markov property and its test method. Value Engineering 31(2):312–313 (in chinese)

    Google Scholar 

Download references

Acknowledgements

This work is partially supported by Natural Science Foundation of China (Nos. 61872129 and 61802444) and Doctoral Scientific Research Foundation of Central South University of Forestry and Technology (No. 2016YJ047).

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Hunan University, Changsha, 410082, China

    Zheng Xiao, Bangyong Wang & Xing Li

  2. College of Computer and Information Engineering, Central South University of Forestry and Technology, Changsha, 410004, China

    Jiayi Du

Authors
  1. Zheng Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bangyong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiayi Du
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zheng Xiao or Jiayi Du.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, Z., Wang, B., Li, X. et al. Workload-driven coordination between virtual machine allocation and task scheduling. Neural Comput & Applic 32, 5535–5551 (2020). https://doi.org/10.1007/s00521-019-04022-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04022-1

Keywords

Search

Navigation