Skip to main content
Springer Nature Link
Log in

QoS-based Task Group Deployment on Grid by Learning the Performance Data

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Overhead of executing fine-grain tasks on computational grids led to task group or batch deployment in which a batch is resized according to the characteristics of the tasks, designated resource, and the interconnecting network. An economic grid demands an application to be processed within the given budget and deadline, referred to as the quality of service (QoS) requirements. In this paper, we increase the task success rate in an economic grid by optimally mapping the tasks to the resources prior to the batch deployment. The task-resource mapping (Advance QoS Planning) is decided based on QoS requirement and by mining the historical performance data of the application tasks using a genetic algorithm. The mapping is then used to assist in creating the task groups. Practical experiments are conducted to validate the proposed method and suggestions are given to implement our method in a cloud environment as well as to process real-time tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Jgap: java genetic algorithm package. http://jgap.sourceforge.net/. Accessed 30 March 2011

  2. Abramson, D., Buyya, R., Giddy, J.: A computational economy for grid computing and its implementation in the nimrod-g resource broker. Futur. Gener. Comput. Syst. 18(8), 1061–1074 (2002)

    Article  MATH  Google Scholar 

  3. Antani, S.: Batch processing with websphere compute grid: Delivering business value to the enterprise. Tech. rep. IBM. http://www.redbooks.ibm.com/abstracts/redp4566.html (2010)

  4. Baker, M., Buyya, R., Laforenza, D.: Grids and grid technologies for wide-area distribute computing. Softw. Pract. Exper. 32(15), 1437–1466 (2002)

    Article  MATH  Google Scholar 

  5. Barmouta, A., Buyya, R., Gridbank: A grid accounting services architecture (gasa) for distributed systems sharing and integration. In: Proceedings of the 17th International Symposium on Parallel and Distributed Processing, p. 245.1. IEEE Computer Society, Washington DC, USA (2003)

  6. Castillo, C., Rouskas, G.N., Harfoush, K.: On the design of online scheduling algorithms for advance reservations and qos in grids. In: International Symposium on Parallel and Distributed Processing, pp. 1–10. California, USA (2007)

  7. Castillo, C., Rouskas, G.N., Harfoush, K.: Online algorithms for advance resource reservations. J. Distrib. Parallel Comput. 71(7), 963–973 (2011)

    Article  Google Scholar 

  8. Elmroth, E., Tordsson, J.: Grid resource brokering algorithms enabling advance reservations and resource selection based on performance predictions. Futur. Gener. Comput. Syst. 24(6), 585–593 (2008)

    Article  Google Scholar 

  9. Feitelson, D.G.: Packing schemes for gang scheduling. In: Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, pp. 89–110. Springer , London (1996)

    Google Scholar 

  10. Feng, J., Wasson, G., Humphrey, M.: Resource usage policy expression and enforcement in grid computing. In: Proceedings of the 8th IEEE/ACM International Conference on Grid Computing. pp. 66–73. IEEE Computer Society, Washington, DC, USA (2007)

  11. Gao, Y., Rong, H., Huang, J.Z.: Adaptive grid job scheduling with genetic algorithms. Futur. Gener. Comput. Syst. 21(1), 151–161 (2005)

    Article  Google Scholar 

  12. Guttmacher, A.E., Collins, F.S.: Genomic medicine—a primer. The New England J MeD 347(19), 1512–1520 (2002)

    Article  Google Scholar 

  13. Huang, P., Peng, H., Lin, P., Li, X.: Static strategy and dynamic adjustment: an effective method for grid task scheduling. Futur. Gener. Comput. Syst. 25(8), 884–892 (2009)

    Article  Google Scholar 

  14. Hui, L., Yu, H., Xiaoming, L.: A lightweight execution framework for massive independent tasks. In: Workshop on Many-Task Computing on Grids and Supercomputers, pp. 1–9. IEEE (2008)

  15. Huu, T.T., Koslovski, G.P., Anhalt, F., Montagnat, J., Primet, P.V.B.: Joint elastic cloud and virtual network framework for application performance-cost optimization. J. Grid Comput. 9(1), 27–47 (2011)

    Article  Google Scholar 

  16. Jacob, B., Brown, M., Fukui, K., Trivedi, N.: Introduction to Grid Computing. IBM Publication (2005)

  17. James, H., Hawick, K., Coddington, P.: Scheduling independent tasks on metacomputing systems. In: Proceedings of Parallel and Distributed Computing Systems, pp. 156–162. Fort Lauderdale, US (1999)

  18. Li, H., Groep, D., Wolters, L.: Mining performance data for metascheduling decision support in the grid. Futur. Gener. Comput. Syst. 23, 92–99 (2007)

    Article  Google Scholar 

  19. Liu, D., Cao, Y.: Computational intelligence and security. In: Wang, Y., Cheung, Y.M., Liu, H. (eds.) CGA: Chaotic Genetic Algorithm for Fuzzy Job Scheduling in Grid Environment, CIS’06, chap., pp. 133–143. Springer, Berlin (2007)

  20. Maghraoui, K.E., Desell, T.J., Szymanski, B.K., Varela, C.A.: The internet operating system: Middleware for adaptive distributed computing. Int. J. High Perform. Comput. Appl. 20(4), 467–480 (2006)

    Article  Google Scholar 

  21. Mohr, E., Kranz, D.A., Halstead, R.H.J.: Lazy task creation: a technique for increasing the granularity of parallel programs. IEEE Trans. Parallel Distributed Syst. 2(3), 264-280 (1991)

    Article  Google Scholar 

  22. Moretti, C., Bui, H., Hollingsworth, K., Rich, B., Flynn, P., Thain, D.: All-pairs: an abstraction for data-intensive computing on campus grids. IEEE Trans. Parallel Distributed Syst. 21, 33–46 (2010)

    Article  Google Scholar 

  23. Muthuvelu, N., Chai, I., Chikkannan, E., Buyya, R.: On-line task granularity adaptation for dynamic grid applications. In: Proceedings of the 10th International Conference on Algorithms and Architectures for Parallel Processing, vol. 6081, pp. 266–277 (2010)

  24. Muthuvelu, N., Chai, I., Chikkannan, E., Buyya, R.: Batch resizing policies and techniques for fine-grain grid tasks: the nuts and bolts. J. Inf. Process. Syst. 7(2), 299–320 (2011)

    Article  Google Scholar 

  25. Prodan, R., Wieczorek, M.: Negotiation-based scheduling of scientific grid workflows through advance reservations. J. Grid Comput. 8(4), 493–510 (2010)

    Article  Google Scholar 

  26. Rahman, M., Ranjan, R., Buyya, R.: Cooperative and decentralized workflow scheduling in global grids. Futur. Gener. Comput. Syst. 26(5), 753–768 (2010)

    Article  Google Scholar 

  27. Ramrez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R., Schwiegelshohn, U., Quezada-Pina, A., Gonzalez-Garca, J.L., Hirales-Carbajal, A.: Job allocation strategies with user run time estimates for online scheduling in hierarchical grids. J. Grid Comput. 9(1), 95–116 (2011)

    Article  Google Scholar 

  28. Risch, M., Altmann, J.: Capacity planning in economic grid markets. In: Proceedings of the 4th International Conference on Advances in Grid and Pervasive Computing, (GPC)09, pp.1-12. Springer, Berlin (2009)

    Google Scholar 

  29. Sadasivam, G.S., Rajendran, V.V.: An efficient approach to task scheduling in computational grids. Int. J. Comput. Sci. Appl. 6(1), 53–69 (2009)

    Google Scholar 

  30. Siddiqui, M., Villazon, A., Fahringer, T.: Grid capacity planning with negotiation-based advance reservation for optimized qos. In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, pp. 103–118. ACM, New York (2006)

    Google Scholar 

  31. Smith, W., Foster, I., Taylor, V.: Predicting application run times with historical information. J. Parallel Distrib. Comput. 64, 1007–1016 (2004)

    Article  MATH  Google Scholar 

  32. Sodan, A.C., Kanavallil, A., Esbaugh, B.: Group-based optimizaton for parallel job scheduling with scojo-pect-o. In: Proceedings of the 22nd International Symposium on High Performance Computing Systems and Applications, pp. 102–109. IEEE Computer Society, Washington, DC, USA (2008)

  33. Takefusa, A., Nakada, H., Kudoh, T., Tanaka, Y.: An advance reservation-based co-allocation algorithm for distributed computers and network bandwidth on qos-guaranteed grids. In: Proceedings of the 15th International Conference on Job Scheduling Strategies for Parallel Processing, pp. 16–34. Springer, Berlin (2010)

    Google Scholar 

  34. Talby, D., Feitelson, D.G.: Improving and stabilizing parallel computer performance using adaptive backfilling. In: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium, p. 84.1. IEEE Computer Society, Washington, DC, USA (2005)

  35. Venugopal, S., Buyya, R., Lyle, W.: A grid service broker for scheduling e-science applications on global data grids. Concurrency and Computation: Practice and Experience (CCPE) 18, 685–699 (2006)

Download references

Author information

Authors and Affiliations

  1. Persiaran Multimedia, Multimedia University, 63100 Cyberjaya, Selangor, Malaysia

    Nithiapidary Muthuvelu, Ian Chai & Eswaran Chikkannan

  2. Cloud Computing and Distributed Systems (CLOUDS) Laboratory, Department of Computer Science and Software Engineering, The University of Melbourne, Melbourne, Australia

    Rajkumar Buyya

Authors
  1. Nithiapidary Muthuvelu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ian Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eswaran Chikkannan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rajkumar Buyya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nithiapidary Muthuvelu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Muthuvelu, N., Chai, I., Chikkannan, E. et al. QoS-based Task Group Deployment on Grid by Learning the Performance Data. J Grid Computing 12, 465–483 (2014). https://doi.org/10.1007/s10723-014-9308-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-014-9308-5

Keywords