Skip to main content
SpringerLink
Log in

Cost minimization for bag-of-tasks workflows in a federation of clouds

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

We address the problem of resource allocation for bag-of-tasks (BoT) workflows in a federation of clouds and formulate it as an integer linear programming problem. The proposed model minimizes financial cost including fees for running VMs and fees for data transfer, and fulfills deadline and resource constraints in the clouds. We also formulate the problem of BoT scheduling in the hybrid clouds, and compare the financial cost in the federation of clouds with that in the hybrid clouds. Moreover, this paper discusses sensitivity analysis to investigate stability in the related allocation problem. Numerical results show that the resource allocation in the federation is considerably preferred to that in the hybrid clouds in terms of stability and cost-saving. In this paper, we also propose an approach named GRASP-FC for obtaining an approximate optimal solution of BoT scheduling in the federation. GRASP-FC is an extension of greedy randomized adaptive search procedure (GRASP), and it can be of great interest from the computational points of view.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Infrastructure as a service (IaaS).

  2. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-resource-limits.html.

  3. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator.

  4. CloudHarmony Compute Unit (CCU).

  5. http://blog.cloudharmony.com/2010/09/benchmarking-of-ec2s-new-cluster.html.

  6. Ec2 Compute Unit.

  7. Drug design.

  8. Southeastern coastal ocean observing and prediction.

  9. Panoramic survey telescope and rapid response system.

  10. Astronomy.

  11. Domain analysis of genome sized.

  12. http://iperf.sourceforge.net/.

  13. In practice, cost function attains a large enough number such as M.

  14. ILOG CPLEX. http://www.ilog.com/products/cplex.

  15. https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator.

References

  1. Abdi S, Pourkarimi L, Ahmadi M, Zargari F (2017) Cost minimization for deadline-constrained bag-of-tasks applications in federated hybrid clouds. Future Gen Comput Syst 71:113–128

    Article  Google Scholar 

  2. Altmann J, Kashef MM (2014) Cost model based service placement in federated hybrid clouds. Future Gen Comput Syst 41:79–90

    Article  Google Scholar 

  3. Coffman Edward G Jr, Garey MR, Johnson DS (1978) An application of bin-packing to multiprocessor scheduling. SIAM J Comput 7(1):1–17

    Article  MathSciNet  MATH  Google Scholar 

  4. Coutinho RDC, Drummond LMA, Frota Y, De Oliveira D (2015) Optimizing virtual machine allocation for parallel scientific workflows in federated clouds. Future Gen Comput Syst 46:51–68

    Article  Google Scholar 

  5. den Bossche R, Vanmechelen K, Broeckhove J (2010) Cost-optimal scheduling in hybrid iaas clouds for deadline constrained workloads. In: 2010 IEEE 3rd International Conference on Cloud Computing. IEEE, pp 228–235

  6. Durillo JJ, Prodan R, Barbosa JG (2015) Pareto tradeoff scheduling of workflows on federated commercial clouds. Simul Model Pract Theory 58:95–111

    Article  Google Scholar 

  7. Jaikar A, Noh S-Y (2014) Cost and performance effective data center selection system for scientific federated cloud. Peer-to-Peer Netw Appl 8(15):896–902

    Google Scholar 

  8. Juve G, Chervenak A, Deelman E, Bharathi S, Mehta G, Vahi K (2013) Characterizing and profiling scientific workflows. Future Gener Comput Syst 29(3):682–692

    Article  Google Scholar 

  9. Kapferer W, Domainko W, Schindler S, van Kampen E, Kimeswenger S, Mair M, Kronberger T, Breitschwerdt D (2005) Metal enrichment and energetics of galactic winds in galaxy clusters. Adv Space Res 36(4):682–684

    Article  Google Scholar 

  10. Malawski M, Figiela K, Nabrzyski J (2013) Cost minimization for computational applications on hybrid cloud infrastructures. Future Gener Comput Syst 29(7):1786–1794

    Article  Google Scholar 

  11. Minh TN, Nam T, Epema DHJ (2014) Parallel workload modeling with realistic characteristics. IEEE Trans Parallel Distrib Syst 25(8):2138–2148

    Article  Google Scholar 

  12. Moschakis I, Karatza HD (2015) Multi-criteria scheduling of Bag-of-Tasks applications on heterogeneous interlinked clouds with simulated annealing. J Syst Softw 101:1–14

    Article  Google Scholar 

  13. Prodan R, Li X, Duan R (2014) Multi-objective game theoretic scheduling of bag-of-tasks workflows on hybrid clouds. IEEE Trans Cloud Comput 2(1):29–42

    Article  Google Scholar 

  14. Ramakrishnan L, Plale B (2010) A multi-dimensional classification model for scientific workflow characteristics. In: Proceedings of the 1st International Workshop on Workflow Approaches to New Data-centric Science. ACM, p 4

  15. Feo TA, Resende MGC (1995) Greedy randomized adaptive search procedures. J Global Optim 6(2):109–133

    Article  MathSciNet  MATH  Google Scholar 

  16. Resende MGC, Ribeiro CC (2016) Optimization by GRASP: greedy randomized adaptive search procedures. Springer, New York

    Book  MATH  Google Scholar 

  17. Thimmapuram PR, Kim J, Botterud A, Nam Y (2010) Modeling and simulation of price elasticity of demand using an agent-based model. In: Innovative Smart Grid Technologies (ISGT). IEEE, pp 1–8

  18. Van den Bossche R, Vanmechelen K, Broeckhove J (2013) Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds. Future Gener Comput Syst 29(4):973–985

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Somayeh Abdi

  2. Department of Mathematics, Razi University, Kermanshah, Iran

    Latif PourKarimi

  3. Computer Engineering and Information Technology Department, Razi University, Kermanshah, Iran

    Mahmood Ahmadi

  4. Information Technology Faculty, Iran Telecom Research Center, Tehran, Iran

    Farzad Zargari

Authors
  1. Somayeh Abdi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Latif PourKarimi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahmood Ahmadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Farzad Zargari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mahmood Ahmadi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abdi, S., PourKarimi, L., Ahmadi, M. et al. Cost minimization for bag-of-tasks workflows in a federation of clouds. J Supercomput 74, 2801–2822 (2018). https://doi.org/10.1007/s11227-018-2322-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-018-2322-9

Keywords

Search

Navigation