Skip to main content
SpringerLink
Log in

A hybrid algorithm for scheduling scientific workflows in IaaS cloud with deadline constraint

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Scientific workflows are used to process large amounts of data and perform complex analyses; thus, they require powerful computing resources to produce the desired results in an acceptable time and at reasonable costs. For this purpose, distributed resources such as cloud computing, with access to virtualized, infinite, and elastic resources are used to execute the workflows. For mapping tasks to computational resources, the problem must be modeled as a scheduling problem. The algorithm presented in this research is a hybrid algorithm based on a mathematical model called MHPSLP that performs the scheduling problem by breaking the problem into smaller subsets including scheduling bags of tasks, providing resources using an mixed integer linear mathematical (MILP) model. The benefit of this method against compared scheduling algorithms is reduction of executed task’s cost in a deadline constraint.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

In this paper, we used scientific workflows for our experimentations as presented in Sect. 5 in our paper, including Montage, Epigenomics, Sipht, and Ligo. These workflows with a different number of tasks collect as a zip file and put as a public and downloadable link in Git at the address given in https://github.com/Baran7292/paper. Although, we used the workflows with approximately 1000 tasks in our experiment. Also, I put other workflows like Inspiral and Cybershake in different sizes for further experimentation that could help other authors to access a wide range of workflows. If anybody needs some explanation about these datasets, contact me with the Email: malihe.hariri@alumni.um.ac.ir

References

  1. Rodriguez MA, Buyya R (2015) A responsive knapsack-based algorithm for resource provisioning and scheduling of scientific workflows in clouds. In: 2015 44th international conference on parallel processing (ICPP). IEEE, pp 839–848

  2. Thai L, Varghese B, Barker A (2018) A survey and taxonomy of resource optimisation for executing bag-of-task applications on public clouds. Futur Gener Comput Syst 82:1–11

    Article  Google Scholar 

  3. Rodriguez MA, Buyya R (2017) A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments. Concurr Comput Pract Exp 29(8):e4041

    Article  Google Scholar 

  4. Smanchat S, Viriyapant K (2015) Taxonomies of workflow scheduling problem and techniques in the cloud. Futur Gener Comput Syst 52:1–12

    Article  Google Scholar 

  5. Alkhanak EN, Lee SP, Khan SUR (2015) Cost-aware challenges for workflow scheduling approaches in cloud computing environments: taxonomy and opportunities. Futur Gener Comput Syst 50:3–21

    Article  Google Scholar 

  6. Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 71(9):3373–3418

    Article  Google Scholar 

  7. Abrishami S, Naghibzadeh M, Epema DH (2013) Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Futur Gener Comput Syst 29(1):158–169

    Article  Google Scholar 

  8. Calheiros RN, Buyya R (2014) Meeting deadlines of scientific workflows in public clouds with tasks replication. IEEE Trans Parallel Distrib Syst 25(7):1787–1796

    Article  Google Scholar 

  9. Alejandra RM, Buyya R (2014) Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. IEEE Trans Cloud Comput 2(2):222–235

    Article  Google Scholar 

  10. Malawski M, Figiela K, Bubak M, Deelman E, Nabrzyski J (2015) Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization. Sci Program 2015:14

    Google Scholar 

  11. Mao M, Humphrey M (2011) Auto-scaling to minimize cost and meet application deadlines in cloud workflows. In: International conference on high performance computing, networking, storage and analysis (SC). IEEE, pp 1–12

  12. Van den Bossche R, Vanmechelen K, Broeckhove J (2013) Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds. Futur Gener Comput Syst 29(4):973–985

    Article  Google Scholar 

  13. Kumar M, Sharma S (2017) Dynamic load balancing algorithm for balancing the workload among virtual machine in cloud computing. Procedia Comput Sci 115:322–329

    Article  Google Scholar 

  14. Byun E-K, Kee Y-S, Kim J-S, Maeng S (2011) Cost optimized provisioning of elastic resources for application workflows. Futur Gener Comput Syst 27(8):1011–1026

    Article  Google Scholar 

  15. Rodriguez MA, Buyya R (2017) Budget-driven scheduling of scientific workflows in IaaS clouds with fine-grained billing periods. ACM Trans Auton Adapt Syst 12(2):22

    Article  Google Scholar 

  16. Juve G, Chervenak A, Deelman E, Bharathi S, Mehta G, Vahi K (2013) Characterizing and profiling scientific workflows. Futur Gener Comput Syst 29(3):682–692

    Article  Google Scholar 

  17. Bharathi S et al (2008) Characterization of scientific workflows. In: 2008 third workshop on workflows in support of large-scale science. IEEE

  18. Berriman G, Laity A, Good J, Jacob J, Katz D, Deelman E, Singh G, Su M, Prince T (2006) Montage: the architecture and scientific applications of a national virtual observatory service for computing astronomical image mosaics. In: Proceedings of Earth Sciences Technology Conference

  19. Livny J, Teonadi H, Livny M, Waldor MK (2008) High-throughput, kingdom-wide prediction and annotation of bacterial non-coding RNAs. PLoS ONE 3(9):e3197

    Article  Google Scholar 

  20. USC Epigenome Center. http://epigenome.usc.edu. Accessed Oct 2015

  21. Abramovici A, Althouse WE, Drever RW, Gürsel Y, Kawamura S, Raab FJ, Shoemaker D, Sievers L, Spero RE, Thorne KS et al (1992) Ligo: the laser interferometer gravitational-wave observatory. Science 256(5055):325–333

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering, Ferdowsi University of Mashhad, Azadi Sqr, Mashhad, Razavi Khorasan Province, 9177948974, Iran

    Malihe Hariri, Mostafa Nouri-Baygi & Saeid Abrishami

Authors
  1. Malihe Hariri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mostafa Nouri-Baygi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saeid Abrishami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Malihe Hariri.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hariri, M., Nouri-Baygi, M. & Abrishami, S. A hybrid algorithm for scheduling scientific workflows in IaaS cloud with deadline constraint. J Supercomput 78, 16975–16996 (2022). https://doi.org/10.1007/s11227-022-04563-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04563-8

Keywords

Search

Navigation