Skip to main content
Springer Nature Link
Log in

GA-based cloud resource estimation for agent-based execution of bag-of-tasks applications

  • Published:
Information Systems Frontiers Aims and scope Submit manuscript

Abstract

Executing bag-of-tasks applications in multiple Cloud environments while satisfying both consumers’ budgets and deadlines poses the following challenges: How many resources and how many hours should be allocated? What types of resources are required? How to coordinate the distributed execution of bag-of-tasks applications in resources composed from multiple Cloud providers?. This work proposes a genetic algorithm for estimating suboptimal sets of resources and an agent-based approach for executing bag-of-tasks applications simultaneously constrained by budgets and deadlines. Agents (endowed with distributed algorithms) compose resources and coordinate the execution of bag-of-tasks applications. Empirical results demonstrate that the genetic algorithm can autonomously estimate sets of resources to execute budget-constrained and deadline-constrained bag-of-tasks applications composed of more economical (but slower) resources in the presence of loose deadlines, and more powerful (but more expensive) resources in the presence of large budgets. Furthermore, agents can efficiently and successfully execute randomly generated bag-of-tasks applications in multi-Cloud environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Amazon EC2. (2011). http://aws.amazon.com/ec2/. Accessed 21 July 2011.

  • Amazon EC2 FAQs. (2011). http://aws.amazon.com/ec2/faqs/. Accessed 21 July 2011.

  • Amazon EC2 instance types. (2011). http://aws.amazon.com/ec2/instance-types/. Accessed 21 July 2011.

  • Amazon EC2 pricing. (2011). http://aws.amazon.com/ec2/pricing/. Accessed 21 July 2011.

  • Anglano, C., & Canonico, M. (2008). Scheduling algorithms for multiple Bag-of-Task applications on desktop Grids: a knowledge-free approach. In proceedings of the 22nd IEEE International Symposium on Parallel and Distributed Processing, 1–8.

  • Bellifemine, F., Poggi, A., & Rimassa, G. (1999). JADE - A FIPA-compliant agent framework. In proceedings of the 4th International Conference and Exhibition on the Practical Application of Intelligent Agents and Multi-Agents, 97–108.

  • Bonami, P., Kilinc, M., & Linderoth, J. (2009). Algorithms and Software for Convex Mixed Integer Nonlinear Programs. Technical Report #1664, Computer Sciences Department, University of Wisconsin-Madison.

  • Buyya, R., Murshed, M. M., Abramson, D., & Venugopal, S. (2005). Scheduling parameter sweep applications on global Grids: a deadline and budget constrained cost-time optimization algorithm. Software-Practice and Experience, 35(5), 491–512.

    Article  Google Scholar 

  • Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems, 25(6), 599–616.

    Article  Google Scholar 

  • Candeia, D., Araujo, R., Lopes, R., & Brasileiro, F. (2010). Investigating business-driven cloudburst schedulers for E-science Bag-of-Tasks applications. In proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 343–350.

  • Coello Coello, C. A. (2006). Evolutionary multi-objective optimization: A historical view of the field. IEEE Computational Intelligence Magazine, 1(1), 28–36.

    Article  Google Scholar 

  • Comin, M., Ferrari, C., & Guerra, C. (2004). Grid deployment of bioinformatics applications: a case study in protein similarity determination. Parallel Processing Letters, 14(2), 163–176.

    Article  Google Scholar 

  • Da Silva, F. A. B., Carvalho, S., Senger, H., Hruschka, E. R., & De Farias, C. R. G. (2004). Running data mining applications on the grid: A bag-of-tasks approach. In Lagana et al. (Eds.), Computational Science and Its Applications, LNCS 3044 (pp. 168–177). Heidelberg: Springer.

  • De Jong, K. A., & Spears, W. M. (1989). Using genetic algorithms to solve NP-complete problems. In proceedings of the 3rd International Conference on Genetic algorithms, 124–132.

  • Deb, K. (2001). Multi-objective optimization using evolutionary algorithms (1st ed.). Chichester: Wiley.

    Google Scholar 

  • Garey, M. R., & Johnson, D. S. (1990). Computers and intractability: A guide to the theory of NP-Completeness. New York: Freeman.

    Google Scholar 

  • Ghafoor, A., & Yang, J. (1993). A distributed heterogeneous supercomputing management system. Computer, 26(6), 78–86.

    Article  Google Scholar 

  • Grefenstette, J. J. (1986). Optimization of control parameters for genetic algorithms. IEEE Transactions on Systems, Man, and Cybernetics, 16(1), 122–128.

    Article  Google Scholar 

  • Gutierrez-Garcia, J. O., & Sim, K. M. (2010). Agent-based service composition in Cloud computing. In T.-H. Kim et al. (Eds.), GDC/CA 2010, CCIS 121 (pp. 1–10). Heidelberg: Springer.

    Google Scholar 

  • Jarvis, S. A., Spooner, D. P., Lim Choi Keung, H. N., Cao, J., Saini, S., & Nudd, G. R. (2006). Performance prediction and its use in parallel and distributed computing systems. Future Generation Computer Systems, 22(7), 745–754.

    Article  Google Scholar 

  • Jung, J. J. (2010). An evolutionary approach to query-sampling for heterogeneous systems. Expert Systems with Applications, 37(1), 226–232.

    Article  Google Scholar 

  • Kafil, M., & Ahmad, I. (1998). Optimal task assignment in heterogeneous distributed computing systems. IEEE Concurrency, 6(3), 42–51.

    Article  Google Scholar 

  • Konak, A., Coit, D., & Smith, A. (2006). Multi-objective optimization using genetic algorithms: a tutorial. Reliability Engineering and System Safety, 91(9), 992–1007.

    Article  Google Scholar 

  • Lee, Y. C., & Zomaya, A. Y. (2007). Practical scheduling of Bag-of-Tasks applications on Grids with dynamic resilience. IEEE Transactions on Computers, 56(6), 815–825.

    Article  Google Scholar 

  • Moulton, B. R. (1996). Bias in the consumer price index: what is the evidence? Journal of Economic Perspectives, American Economic Association, 10(4), 159–177.

    Article  Google Scholar 

  • Netto, M. A. S., & Buyya, R. (2009). Offer-based scheduling of deadline-constrained Bag-of-Tasks applications for utility computing systems. In proceedings of the 2009 IEEE International Symposium on Parallel and Distributed Processing, 1–11.

  • O’Brien, A., Newhouse, S., & Darlington, J. (2004). Mapping of scientific workflow within the e-protein project to distributed resources. In proceedings of the UK e-Science All Hands Meeting, 404–409.

  • Oliveira, M. C., Cirne, W., & De Azevedo-Marques, P. M. (2007). Towards applying content-based image retrieval in the clinical routine. Future Generation Computer Systems, 23(3), 466–474.

    Article  Google Scholar 

  • Oprescu, A. M., & Kielmann T. (2010). Bag-of-Tasks scheduling under budget constraints. In proceedings of the 2010 IEEE Second International Conference on Cloud Computing Technology and Science, 351–359.

  • Rotstan, N., & Meffert, K. (2011). JGAP: java genetic algorithms. http://jgap.sourceforge.net/. Accessed 21 July 2011.

  • Salehi, M. A., & Buyya, R. (2010). Adapting market-oriented scheduling policies for Cloud computing. In C.-H. Hsu et al. (Eds.), ICA3PP 2010, Part I, LNCS 6081 (pp. 351–362). Heidelberg: Springer.

    Google Scholar 

  • Silberstein, M., Sharov, A., Geiger, D., & Schuster, A. (2009). Gridbot: execution of bags of tasks in multiple grids. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, article 11, 12 pages.

  • Silva, J. N., Veiga, L., & Ferreira, P. (2008) Heuristic for resources allocation on utility computing infrastructures. In proceedings of the 6th International Workshop on Middleware for Grid Computing, Article 9, p. 6

  • Sim, K. M. (2009). Agent-based Cloud commerce. In proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management, 717–721.

  • Sim, K. M. (2010). Towards complex negotiation for cloud economy. In Chang et al. (Eds.), GPC 2010, LNCS 6104 (pp. 395–406). Heidelberg: Springer.

    Google Scholar 

  • Smith, R. G. (1980). The contract net protocol: high-level communication and control in a distributed problem solver. IEEE Transactions on Computers, 29(12), 1104–1113.

    Article  Google Scholar 

  • Smith, W., Foster, I. T., & Taylor, V. E. (1998). Predicting application run times using historical information. In D. G. Feitelson & L. Rudolph (Eds.), JSSPP’98, LNCS 1459 (pp. 122–142). Heidelberg: Springer.

    Google Scholar 

  • Sugavanam, P., Siegel, H. J., Maciejewski, A. A., Oltikar, M., Mehta, A., Pichel, R., et al. (2007). Robust static allocation of resources for independent tasks under makespan and dollar cost constraints. Journal of Parallel and Distributed Computing, 67(4), 400–416.

    Article  Google Scholar 

  • Sulistio, A., & Buyya, R. (2005). A time optimization algorithm for scheduling Bag-of-Task applications in auction-based proportional share systems. In proceedings of the 17th International Symposium on Computer Architecture on High Performance Computing, 235–242.

  • Wooldridge, M. (2009). An introduction to multiagent systems (2nd ed.). Chichester: Wiley.

    Google Scholar 

  • Yu, J., Buyya, R., & Khong Tham, C. (2005). Cost-based scheduling of scientific workflow application on utility Grids. In proceedings of the 1st International Conference on e-Science and Grid Computing, 140–147.

Download references

Acknowledgments

This work was supported by the Korea Research Foundation Grant funded by the Korean Government (MEST) (KRF-2009-220-D00092) and the DASAN International Faculty Fund (project code: 140316). The authors would like to thank the Guest Editors and the anonymous referees for their comments and suggestions.

Author information

Authors and Affiliations

  1. Department of Information and Communications, Gwangju Institute of Science and Technology, 261 Cheomdan-gwagiro, Buk-gu, Gwangju, 500-712, Republic of Korea

    J. Octavio Gutierrez-Garcia & Kwang Mong Sim

Authors
  1. J. Octavio Gutierrez-Garcia
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kwang Mong Sim
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Kwang Mong Sim.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gutierrez-Garcia, J.O., Sim, K.M. GA-based cloud resource estimation for agent-based execution of bag-of-tasks applications. Inf Syst Front 14, 925–951 (2012). https://doi.org/10.1007/s10796-011-9327-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10796-011-9327-8

Keywords