Skip to main content
SpringerLink
Log in

Randomized approximation scheme for resource allocation in hybrid-cloud environment

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Using the virtually unlimited resource capacity of public cloud, dynamic scaling out of large-scale applications is facilitated. A critical question arises practically here is how to run such applications effectively in terms of both cost and performance. In this paper, we explore how resources in the hybrid-cloud environment should be used to run Bag-of-Tasks applications. Having introduced a simple yet effective objective function, our algorithm helps the user to make a better decision for realization of his/her goal. Then, we cope with the problem in two different cases of “known” and “unknown” running time of available tasks. A solution to approximate the optimal value of user’s objective function will be provided for each case. Specifically, a fully polynomial-time randomized approximation scheme based on a Monte Carlo sampling method will be presented in case of unknown running time. The experimental results confirm that our algorithm approximates the optimal solution with a little scheduling overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. \(L^p\)-norm (p-norm) of a vector z can be defined by (\(\forall p\ge 1 \in \mathfrak {R}\)): \(\Vert z\Vert _p=\left( |z_1|^p+|z_2|^p+\cdots +|z_n|^p\right) ^{\frac{1}{p}}\). The \(L^\infty \)-norm or Chebyshev distance is the limit of the \(L^p\)-norms when \(p \,\rightarrow \, \infty \) which has the same definition as: \(\ \Vert z\Vert _\infty =\max \left\{ |z_1|, |z_2|, \cdots , |z_n|\right\} \).

  2. Each point in Fig. 2 shows a particular value of \(r\) starting from 1 and being incremented by 1.

References

  1. Iosup A, Jan M, Sonmez OO, Epema DHJ (2007) The characteristics and performance of groups of jobs in grids. In: Proceedings of the International European Conference on Parallel Processing, LNCS, vol 4641. Springer, pp 382–393

  2. Minh TN, Wolters L, Epema DHJ (2010) A realistic integrated model of parallel system workloads. In: International Symposium on Cluster, Cloud the Grid, CCGRID’10. IEEE, pp 464–473

  3. HoseinyFarahabady MR, Lee YC, Zomaya AY (2012) Non-clairvoyant assignment of bag-of-tasks applications across multiple clouds. In: PDCAT’12. China

  4. HoseinyFarahabady MR, Lee YC, Liu X, Reisi Dehkordi H, Zomaya AY (2011) Approximation algorithm for scaling outlarge-scale bag-of-tasks applicationsacross multiple clouds. The University of Sydney, Tech. Rep. 684

  5. Dagum P, Karp R, Luby M, Ross S (1995) An optimal algorithm for monte carlo estimation. In: Foundations of Computer Science, pp 142–149

  6. Adams R, Fournier J (2003) Sobolev spaces. Pure and applied mathematics. Elsevier Science, Philadelphia

    Google Scholar 

  7. Cormen TH, Leiserson CE, Rivest RL, Stein C (2009) Introduction to algorithms, 3rd edn. The MIT Press, Cambridge

    MATH  Google Scholar 

  8. Chekuri C, Khanna S (2000) A ptas for the multiple knapsack problem. In: Proceedings of the eleventh annual ACM-SIAM symposium on Discrete algorithms., SODA ’00Society for Industrial and Applied Mathematics, Philadelphia, pp 213–222

  9. Kellerer H, Mansini R, Pferschy U, Speranza MG (2003) An efficient fully polynomial approximation scheme for the subset-sum problem. J Comput Syst Sci 66(2) 349–370. doi:10.1016/S0022-0000(03)00006-0

    Google Scholar 

  10. Lawler EL (1979) Fast approximation algorithms for knapsack problems. Math Oper Res 4(4):339–356

    Article  MATH  MathSciNet  Google Scholar 

  11. Arora S, Karger D, Karpinski M (1995) Polynomial time approximation schemes for dense instances of np-hard problems. In: Proceedings of the twenty-seventh annual ACM symposium on theory of computing. ACM, Las Vegas, Nevada, USA, pp 284–293. doi:10.1145/225058.225140

  12. Dsa G (2007) The tight bound of first fit decreasing bin-packing algorithm is ffd(i)=(11/9)opt(i)+6/9. In: Chen B, Paterson M, Zhang G (eds) ombinatorics, algorithms, probabilistic and experimental methodologies, Lecture Notes in Computer Science, vol 4614. Springer, Berlin Heidelberg, pp 1–11

    Google Scholar 

  13. Iosup A, Sonmez O, Anoep S, Epema D (2008) The performance of bags-of-tasks in large-scale distributed systems. In: International Symposium on High Performance Distributed Computing, HPDC 08, USA, pp 97–108

  14. Fujimoto N, Hagihara K (2003) Near-optimal dynamic task scheduling of independent coarse-grained tasks onto a computational grid. In: Proceedings of the International Conference on Parallel Processing, ICPP’03, Taiwan, pp 391–398

  15. Lee YC, Zomaya AY (2007) Practical scheduling of bag-of-tasks applications on grids with dynamic resilience. IEEE Trans Comput 56(6):815–825

    Article  MathSciNet  Google Scholar 

  16. Silberstein M, Sharov A, Geiger D, Schuster A (2009) Gridbot: execution of bags of tasks in multiple grids. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC’09. ACM

  17. Ibarra OH, Kim CE (1977) Heuristic algorithms for scheduling independent tasks on nonidentical processors. J ACM 24(2):280–289

    Article  MATH  MathSciNet  Google Scholar 

  18. Almeida VAF, Vasconcelos IMM, Árabe JNC, Menascé DA (1992) Using random task graphs to investigate the potential benefits of heterogeneity in parallel systems. In: Proceedings of the ACM/IEEE conference on Supercomputing, Supercomputing ’92. IEEE, USA, pp 683–691

  19. Menasce D, Almeida V (1990) Cost-performance analysis of heterogeneity in supercomputer architectures. In: Proceedings of the ACM/IEEE conference on Supercomputing, Supercomputing ’90. IEEE, USA pp 169–177

  20. Reistad B, Gifford DK (1994) Static dependent costs for estimating execution time. In: Proceedings of the Conference on LISP and functional programming. ACM, pp 65–78

  21. Yang J, Ahmad I, Ghafoor A (1993) Estimation of execution times on heterogeneous supercomputer architectures. In: Proceedings of International Conference on Parallel Processing, ICPP ’93. IEEE, USA, pp 219–226

  22. Iverson MA, Ozguner F, Follen GJ (1996) Run-time statistical estimation of task execution times for heterogeneous distributed computing. In: Proceedings of International Symposium on High Performance Distributed Computing, HPDC ’96. IEEE, USA, p 263

  23. de Assunccao MD, di Costanzo A, Buyya R (2009) Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. In: Proceedings of the International Symposium on High Performance, Distributed Computing, HPDC’09. pp 141–150

  24. Othman M, Abdullah M, Ibrahim H, Subramaniam S (2008) A2dlt: Divisible load balancing model for scheduling communication-intensive grid applications. In: Bubak M, Albada G, Dongarra J, Sloot P (eds) Computational Science ICCS 2008, Lecture Notes in Computer Science, vol. 5101. Springer, New York, pp 246–253

    Google Scholar 

  25. Othman, M., Abdullah, M., Ibrahim, H., Subramaniam, S.: New optimal load allocation for scheduling divisible data grid applications. In: Gabrielle A, Jaroslaw N, Edward S, GeertDick A, Jack D, Peter SMA (eds) Computational Science ICCS 2009, Lecture Notes in Computer Science, vol. 5544. Springer, New York, pp 165–174. doi:10.1007/978-3-642-01970-8_17

  26. Abdullah M, Othman M (2013) Cost-based multi-qos job scheduling using divisible load theory in cloud computing. Procedia Computer Science 18. 2013 International Conference on Computational Science. pp 928–935

  27. Iyer G, Veeravalli B, Krishnamoorthy S (2012) On handling large-scale polynomial multiplications in compute cloud environments using divisible load paradigm. Aerosp Electron Syst IEEE Trans 48(1):820–831

    Article  Google Scholar 

Download references

Acknowledgments

(1) Professor Albert Zomaya’s work is supported by the Australian Research Council Discovery Grant (DP1097110). (2) M. Reza HoseinyFarahabady’s work is partially supported by National ICT Australia (NICTA). NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program.

Author information

Authors and Affiliations

  1. Centre for Distributed and High Performance Computing, School of Information Technologies, The University of Sydney, Room: 409, Building J12, Sydney, NSW, 2006, Australia

    MohammadReza HoseinyFarahabady, Young Choon Lee & Albert Y. Zomaya

  2. National ICT Australia (NICTA), Australian Tech. Park, Sydney, NSW, 1430, Australia

    MohammadReza HoseinyFarahabady

Authors
  1. MohammadReza HoseinyFarahabady
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Young Choon Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Albert Y. Zomaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to MohammadReza HoseinyFarahabady.

Additional information

This paper was originally published in the Thirteenth International Conference on Parallel and Distributed Computing, Applications and Technologies, Beijing, China, Dec. 2012.

Rights and permissions

Reprints and permissions

About this article

Cite this article

HoseinyFarahabady, M., Lee, Y.C. & Zomaya, A.Y. Randomized approximation scheme for resource allocation in hybrid-cloud environment. J Supercomput 69, 576–592 (2014). https://doi.org/10.1007/s11227-014-1094-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-014-1094-0

Keywords

Search

Navigation