Skip to main content
SpringerLink
Log in

Autonomic Clouds on the Grid

  • Published:
Journal of Grid Computing Aims and scope Submit manuscript

Abstract

Computational clouds constructed on top of existing Grid infrastructure have the capability to provide different entities with customized execution environments and private scheduling overlays. By designing these clouds to be autonomically self-provisioned and adaptable to changing user demands, user-transparent resource flexibility can be achieved without substantially affecting average job sojourn time. In addition, the overlay environment and physical Grid sites represent disjoint administrative and policy domains, permitting cloud systems to be deployed non-disruptively on an existing production Grid. Private overlay clouds administered by, and dedicated to the exclusive use of, individual Virtual Organizations are termed Virtual Organization Clusters. A prototype autonomic cloud adaptation mechanism for Virtual Organization Clusters demonstrates the feasibility of overlay scheduling in dynamically changing environments. Commodity Grid resources are autonomically leased in response to changing private scheduler loads, resulting in the creation of virtual private compute nodes. These nodes join a decentralized private overlay network system called IPOP (IP Over P2P), enabling the scheduling and execution of end user jobs in the private environment. Negligible overhead results from the addition of the overlay, although the use of virtualization technologies at the compute nodes adds modest service time overhead (under 10%) to computationally-bound Grid jobs. By leasing additional Grid resources, a substantial decrease (over 90%) in average job queuing time occurs, offsetting the service time overhead.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Murphy, M.A., Fenn, M., Goasguen, S.: Virtual organization clusters. In: 17th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP 2009), Weimar, Germany (2009)

  2. Foster, I., Kesselman, C., Tuecke, S.: The anatomy of the Grid: enabling scalable virtual organizations. Int. J. Supercomput. Appl. 15(3), 200–222 (2001)

    Article  Google Scholar 

  3. Murphy, M.A., Kagey, B., Fenn, M., Goasguen, S.: Dynamic provisioning of virtual organization clusters. In: 9th IEEE International Symposium on Cluster Computing and the Grid (CCGrid ’09), Shanghai, China (2009)

  4. Beaumont, O., Carter, L., Ferrante, J., Legrand, A., Marchal, L., Robert, Y.: Centralized versus distributed schedulers for bag-of-tasks applications. IEEE Trans. Parallel Distrib. Syst. 19(5), 698–709 (2008)

    Article  Google Scholar 

  5. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: Nineteenth ACM Symposium on Operating Systems Principles (2003)

  6. Fenn, M., Murphy, M.A., Goasguen, S.: A study of a KVM-based cluster for Grid computing. In: 47th ACM Southeast Conference (ACMSE ’09), Clemson, SC (2009)

  7. Tannenbaum, T., Wright, D., Miller, K., Livny, M.: Condor—a distributed job scheduler. In: Sterling, T. (ed.) Beowulf Cluster Computing with Linux. MIT, Cambridge (2001)

    Google Scholar 

  8. Xu, D., Ruth, P., Rhee, J., Kennell, R., Goasguen, S.: Autonomic adaptation of virtual distributed environments in a multi-domain infrastructure. In: 15th IEEE International Symposium on High Performance Distributed Computing (HPDC’06), Paris, France (2006)

  9. Ganguly, A., Agrawal, A., Boykin, P.O., Figueiredo, R.: IP over P2P: enabling self-configuring virtual IP networks for Grid computing. In: 20th International Parallel and Distributed Processing Symposium (IPDPS 2006) (2006)

  10. Figueiredo, R., Dinda, P.A., Fortes, J.: Resource virtualization renaissance. Computer 38(5), 28–31 (2005)

    Article  Google Scholar 

  11. Figueiredo, R.J., Dinda, P.A., Fortes, J.A.B.: A case for Grid computing on virtual machines. In: 23rd International Conference on Distributed Computing Systems (2003)

  12. Tsugawa, M., Fortes, J.A.B.: A virtual network (ViNe) architecture for Grid computing. In: 20th International Parallel and Distributed Processing Symposium (IPDPS 2006) (2006)

  13. Sundararaj, A.I., Dinda, P.A.: Towards virtual networks for virtual machine Grid computing. In: Third Virtual Machine Research and Technology Symposium, San Jose, CA (2004)

  14. Davoli, R.: VDE: virtual distributed ethernet. In: First International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities (Tridentcom 2005), Trento, Italy (2005)

  15. Adabala, S., Chadha, V., Chawla, P., Figueiredo, R., Fortes, J., Krsul, I., Matsunaga, A., Tsugawa, M., Zhang, J., Zhao, M., Zhu, L., Zhu, X.: From virtualized resources to virtual computing Grids: the In-VIGO system. Future Gener. Comput. Syst. 21(6), 896–909 (2005)

    Article  Google Scholar 

  16. Keahey, K., Foster, I., Freeman, T., Zhang, X., Galron, D.: Virtual workspaces in the Grid. In: 11th International Euro-Par Conference, Lisbon, Portugal (2005)

  17. Sotomayor, B., Keahey, K., Foster, I.: Combining batch execution and leasing using virtual machines. In: 17th International Symposium on High Performance Distributed Computing (HPDC 2008) (2008)

  18. Foster, I., Freeman, T., Keahey, K., Scheftner, D., Sotomayor, B., Zhang, X.: Virtual clusters for Grid communities. In: 6th IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2006), Singapore (2006)

  19. Irwin, D., Chase, J., Grit, L., Yumerefendi, A., Becker, D., Yocum, K.: Sharing network resources with brokered leases. In: USENIX Technical Conference, Boston, MA (2006)

  20. Chase, J.S., Irwin, D.E., Grit, L.E., Moore, J.D., Sprenkle, S.E.: Dynamic virtual clusters in a Grid site manager. In: HPDC ’03: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (2003)

  21. Grit, L., Irwin, D., Yumerefendi, A., Chase, J.: Virtual machine hosting for networked clusters: building the foundations for ‘autonomic’ orchestration. In: First International Workshop on Virtualization Technology in Distributed Computing (VTDC ’06), Tampa, FL (2006)

  22. Ramakrishnan, L., Grit, L., Iamnitchi, A., Irwin, D., Yumerefendi, A., Chase, J.: Toward a doctrine of containment: Grid hosting with adaptive resource control. In: 19th Annual Supercomputing Conference (SC ’06), Tampa, FL (2006)

  23. Ruth, P., McGachey, P., Xu, D.: VioCluster: virtualization for dynamic computational domains. In: IEEE International Conference on Cluster Computing, Boston, MA (2005)

  24. Ruth, P., Jiang, X., Xu, D., Goasguen, S.: Virtual distributed environments in a shared infrastructure. Computer 38(5), 63–69 (2005)

    Article  Google Scholar 

  25. Emeneker, W., Stanzione, D.: Dynamic virtual clustering. In: 2007 IEEE International Conference on Cluster Computing (2007)

  26. Carns, P.H., Ligon, W.B., Ross, R.B., Thakur, R.: PVFS: a parallel file system for Linux clusters. In: ALS’00: Proceedings of the 4th annual Linux Showcase and Conference (2000)

  27. Keahey, K., Freeman, T.: Contextualization: providing one-click virtual clusters. In: 4th IEEE International Conference on e-Science, Indianapolis, IN (2008)

  28. Boykin, P.O., Bridgewater, J.S.A., Kong, J.S., Lozev, K.M., Rezaei, B.A., Roychowdhury, V.P.: A symphony conducted by Brunet. Online (2007)

  29. Keahey, K., Doering, K., Foster, I.: From sandbox to playground: dynamic virtual environments in the Grid. In: 5th International Workshop on Grid Computing (Grid 2004), Pittsburgh, PA (2004)

  30. Pordes, R., Petravick, D., Kramer, B., Olson, D., Livny, M., Roy, A., Avery, P., Blackburn, K., Wenaus, T., Würthwein, F., Foster, I., Gardner, R., Wilde, M., Blatecky, A., McGee, J., Quick, R.: The Open Science Grid: Status and architecture. In: International Conference on Computing in High Energy and Nuclear Physics (CHEP ’07) (2007)

  31. Red Hat: Kernel-based Virtual Machine. http://www.linux-kvm.org

  32. Foster, I., Kesselman, C.: Globus: a metacomputing infrastructure toolkit. Int. J. Supercomput. Appl. 11(2), 115–128 (1997)

    Article  Google Scholar 

  33. Clemson Cyberinfrastructure Research Group: SimVOC. http://code.google.com/p/simvoc

  34. Sulistio, A., Yeo, C.S., Buyya, R.: A taxonomy of computer-based simulations and its mapping to parallel and distributed systems simulation tools. Int. J. Soft.: Practice and Experience 34(7), 653–673 (2004)

    Article  Google Scholar 

  35. EGEE: Grid observatory. http://www.grid-observatory.org

  36. Enabling Grids for E-sciencE: gLite. http://glite.web.cern.ch/glite/

  37. Sfiligoi, I., Quinn, G., Green, C., Thain, G.: Pilot job accounting and auditing in Open Science Grid. In: 9th IEEE/ACM International Conference on Grid Computing (Grid ’08) (2008)

Download references

Author information

Authors and Affiliations

  1. School of Computing, Clemson University, 120 McAdams Hall, Clemson, SC, 29634-0974, USA

    Michael A. Murphy, Linton Abraham, Michael Fenn & Sebastien Goasguen

Authors
  1. Michael A. Murphy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Linton Abraham
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Fenn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sebastien Goasguen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael A. Murphy.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Murphy, M.A., Abraham, L., Fenn, M. et al. Autonomic Clouds on the Grid. J Grid Computing 8, 1–18 (2010). https://doi.org/10.1007/s10723-009-9142-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10723-009-9142-3

Keywords

Search

Navigation