Skip to main content
SpringerLink
Log in

Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Given that multi-cloud environments contain considerably diverse resources, scheduling workflows in these environments significantly reduces financial costs and overcomes the resource limitations imposed by commercial cloud providers. Accordingly, this study addressed the problem of scientific workflow scheduling in multi-cloud settings under deadline constraint to minimize associated financial costs. To this end, we proposed integer linear programming models that can be solved in a reasonable time by available solvers. In a mathematical model, the objective of a problem and real and physical constraints or restrictions are formulated using exact mathematical functions. Such formulation enabled us to comprehensively understand the system under evaluation, consider secondary preferences and post-optimality analysis and apply useful revisions to inappropriately selected input data. We analyzed the treatment of optimal cost under variations in deadline and workflow size. As part of the post-optimality analysis, sensitivity analysis and deadline revision were implemented. Results indicated that our proposed approach outperforms previously developed methods in terms of financial cost reduction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. https://aws.amazon.com/ec2.

  2. https://www.rightscale.com/lp/state-of-the-cloud.

  3. CloudHarmony Compute Unit.

  4. http://blog.cloudharmony.com/2010/09/benchmarking-of-ec2s-new-cluster.html.

  5. http://iperf.sourceforge.net/.

  6. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator.

  7. ILOG CPLEX. http://www.ilog.com/products/cplex.

References

  1. Abdi S, PourKarimi L, Ahmadi M, Zargari F (2018) Cost minimization for bag-of-tasks workflows in a federation of clouds. J Supercomput 74(6):2801–2822

    Article  Google Scholar 

  2. Toosi AN, Calheiros RN, Buyya R (2014) Interconnected cloud computing environments: challenges, taxonomy, and survey. ACM Comput Surv (CSUR) 47(1):7

    Article  Google Scholar 

  3. Rodriguez MA, Buyya R (2017) A taxonomy and survey on scheduling algorithms for scientific workflows in IaaS cloud computing environments. Concurr Comput Pract Exp 29(8):e4041. https://doi.org/10.1002/cpe.4041

    Article  Google Scholar 

  4. Wu F, Wu Q, Tan Y (2015) Workflow scheduling in cloud: a survey. J Supercomput 71(9):3373–3418

    Article  Google Scholar 

  5. Masdari M, ValiKardan S, Shahi Z, Azar SI (2016) Towards workflow scheduling in cloud computing: a comprehensive analysis. J Netw Comput Appl 66:64–82

    Article  Google Scholar 

  6. Abrishami S, Naghibzadeh M, Epema DH (2013) Deadline-constrained workflow scheduling algorithms for infrastructure as a service clouds. Future Gen Comput Syst 29(1):158–169

    Article  Google Scholar 

  7. Malawski M, Figiela K, Bubak M, Deelman E, Nabrzyski J (2015) Scheduling multilevel deadline-constrained scientific workflows on clouds based on cost optimization. Sci Program 2015:5

    Google Scholar 

  8. Deldari A, Naghibzadeh M, Abrishami S (2017) CCA: a deadline-constrained workflow scheduling algorithm for multicore resources on the cloud. J Supercomput 73(2):756–781

    Article  Google Scholar 

  9. Rodriguez MA, Buyya R (2014) Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds. IEEE Trans Cloud Comput 2(2):222–235

    Article  Google Scholar 

  10. Bilgaiyan S, Sagnika S, Das M (2014) Workflow scheduling in cloud computing environment using cat swarm optimization. In: IEEE International Advance Computing Conference (IACC), 2014, pp 680–685

  11. Zhou AC, He B, Liu C (2016) Monetary cost optimizations for hosting workflow-as-a-service in IaaS clouds. IEEE Trans Cloud Comput 4(1):34–48

    Article  Google Scholar 

  12. Zheng W, Sakellariou R (2013) Budget-deadline constrained workflow planning for admission control. J Grid Comput 11(4):633–651

    Article  Google Scholar 

  13. Zeng L, Veeravalli B, Zomaya AY (2015) An integrated task computation and data management scheduling strategy for workflow applications in cloud environments. J Netw Comput Appl 50:39–48

    Article  Google Scholar 

  14. Zeng L, Veeravalli B, Li X (2012) Scalestar: budget conscious scheduling precedence-constrained many-task workflow applications in cloud. In: IEEE 26th International Conference on Advanced Information Networking and Applications (AINA), 2012 pp 534–541

  15. Lin X, Wu CQ (2013) On scientific workflow scheduling in clouds under budget constraint. In: IEEE 42nd International Conference on Parallel Processing (ICPP), 2013, pp 90–99

  16. Arabnejad H, Barbosa JG (2014) A budget constrained scheduling algorithm for workflow applications. J Grid Comput 12(4):665–679

    Article  Google Scholar 

  17. Poola D, Garg SK, Buyya R, Yang Y, Ramamohanarao K (2014) Robust scheduling of scientific workflows with deadline and budget constraints in clouds. In: IEEE 28th International Conference on Advanced Information Networking and Applications (AINA), 2014, pp 858–865

  18. Durillo JJ, Prodan R, Barbosa JG (2015) Pareto tradeoff scheduling of workflows on federated commercial clouds. Simul Model Pract Theory 58:95–111

    Article  Google Scholar 

  19. Coutinho RDC, Drummond LM, Frota Y, de Oliveira D (2015) Optimizing virtual machine allocation for parallel scientific workflows in federated clouds. Future Gen Comput Syst 46:51–68

    Article  Google Scholar 

  20. Durillo JJ, Fard HM, Prodan R (2012) Moheft: a multi-objective list-based method for workflow scheduling. In: IEEE 4th International Conference on Cloud Computing Technology and Science (CloudCom), 2012 (pp 185–192)

  21. Fard HM, Prodan R, Barrionuevo JJD, Fahringer T (2012) A multi-objective approach for workflow scheduling in heterogeneous environments. In: 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), 2012, pp 300–309

  22. Yassa S, Chelouah R, Kadima H, Granado B (2013) Multi-objective approach for energy-aware workflow scheduling in cloud computing environments. Sci World J 2013:350934. https://doi.org/10.1155/2013/350934

    Article  MATH  Google Scholar 

  23. Bessai K, Youcef S, Oulamara A, Godart C, Nurcan S (2012) Bi-criteria workflow tasks allocation and scheduling in cloud computing environments. In: 2012 IEEE 5th International Conference on Cloud Computing (CLOUD), pp 638–645

  24. Zeng L, Veeravalli B, Li X (2015) SABA: a security-aware and budget-aware workflow scheduling strategy in clouds. J Parallel Distrib Comput 75:141–151

    Article  Google Scholar 

  25. Zhu Z, Zhang G, Li M, Liu X (2016) Evolutionary multi-objective workflow scheduling in cloud. IEEE Trans Parallel Distrib Syst 27(5):1344–1357

    Article  Google Scholar 

  26. Jena RK (2015) Multi objective task scheduling in cloud environment using nested PSO framework. Proced Comput Sci 57:1219–1227

    Article  Google Scholar 

  27. Casavant TL, Kuhl JG (1988) A taxonomy of scheduling in general-purpose distributed computing systems. IEEE Trans Softw Eng 14(2):141–154

    Article  Google Scholar 

  28. Malawski M, Juve G, Deelman E, Nabrzyski J (2015) Algorithms for cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. Future Gen Comput Syst 48:1–18

    Article  Google Scholar 

  29. Abdi S, PourKarimi L, Ahmadi M, Zargari F (2017) Cost minimization for deadline-constrained bag-of-tasks applications in federated hybrid clouds. Future Gen Comput Syst 71:113–128

    Article  Google Scholar 

  30. Genez TA, Bittencourt LF, Madeira ER (2012) Workflow scheduling for SaaS/PaaS cloud providers considering two SLA levels. In: Network Operations and Management Symposium (NOMS), 2012 IEEE, pp 906–912

  31. Genez TA, Bittencourt LF, Madeira ER (2013) Using time discretization to schedule scientific workflows in multiple cloud providers. In: IEEE Sixth International Conference on Cloud Computing (CLOUD), 2013, pp 123–130

  32. Lin B, Guo W, Xiong N, Chen G, Vasilakos AV, Zhang H (2016) A pretreatment workflow scheduling approach for big data applications in multicloud environments. IEEE Trans Netw Serv Manage 13(3):581–594

    Article  Google Scholar 

  33. Fard HM, Prodan R, Fahringer T (2013) A truthful dynamic workflow scheduling mechanism for commercial multicloud environments. IEEE Trans Parallel Distrib Syst 24(6):1203–1212

    Article  Google Scholar 

  34. Durillo JJ, Prodan R (2014) Workflow scheduling on federated clouds. In: European Conference on Parallel Processing. Springer, Cham, pp 318–329

  35. Heilig L, Lalla-Ruiz E, Vo S (2016) A cloud brokerage approach for solving the resource management problem in multi-cloud environments. Comput Ind Eng 95:16–26

    Article  Google Scholar 

  36. Duan R, Prodan R, Li X (2014) Multi-objective game theoretic schedulingof bag-of-tasks workflows on hybrid clouds. IEEE Trans Cloud Comput 2(1):29–42

    Article  Google Scholar 

  37. Malawski M, Figiela K, Nabrzyski J (2013) Cost minimization for computational applications on hybrid cloud infrastructures. Future Gen Comput Syst 29(7):1786–1794

    Article  Google Scholar 

  38. Oprescu AM, Kielmann T (2010) Bag-of-tasks scheduling under budget constraints. In: 2010 IEEE Second International Conference on Cloud Computing Technology and Science (CloudCom), pp 351–359

  39. Netto MA, Buyya R (2009) Offer-based scheduling of deadline-constrained bag-of-tasks applications for utility computing systems. In: IEEE International Symposium on Parallel & Distributed Processing, 2009. IPDPS 2009, pp 1–11

  40. Moschakis IA, Karatza HD (2015) Multi-criteria scheduling of Bag-of-Tasks applications on heterogeneous interlinked clouds with simulated annealing. J Syst Softw 101:1–14

    Article  Google Scholar 

  41. Juve G, Chervenak A, Deelman E, Bharathi S, Mehta G, Vahi K (2013) Characterizing and profiling scientific workflows. Future Gen Comput Syst 29(3):682–692

    Article  Google Scholar 

  42. Bharathi S, Chervenak A, Deelman E, Mehta G, Su MH, Vahi K (2008) Characterization of scientific workflows. In: Third Workshop on Workflows in Support of Large-Scale Science, 2008. WORKS 2008, pp 1–10

  43. Thimmapuram PR, Kim J, Botterud A, Nam Y (2010) Modeling and simulation of price elasticity of demand using an agent-based model. In: Innovative Smart Grid Technologies (ISGT), 2010, pp 1–8

  44. Lin B, Guo W, Chen G, Xiong N, Li R (2015) Cost-driven scheduling for deadline-constrained workflow on multi-clouds. In: IEEE International Parallel and Distributed Processing Symposium Workshop (IPDPSW), 2015, pp 1191–1198

  45. Ramakrishnan L, Plale B (2010) A multi-dimensional classification model for scientific workflow characteristics. In: Proceedings of the 1st International Workshop on Workflow Approaches to New Data-Centric Science. ACM, p 4

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Somayeh Mohammadi

  2. Department of Computer Engineering, Amir Kabir University of Technology, Tehran, Iran

    Hossein Pedram

  3. Department of Mathematics, Razi University, Kermanshah, Iran

    Latif PourKarimi

Authors
  1. Somayeh Mohammadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Pedram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Latif PourKarimi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Pedram.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohammadi, S., Pedram, H. & PourKarimi, L. Integer linear programming-based cost optimization for scheduling scientific workflows in multi-cloud environments. J Supercomput 74, 4717–4745 (2018). https://doi.org/10.1007/s11227-018-2465-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2465-8

Keywords

Search

Navigation