Skip to main content
SpringerLink
Log in

Condor-G: A Computation Management Agent for Multi-Institutional Grids

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In recent years, there has been a dramatic increase in the number of available computing and storage resources. Yet few tools exist that allow these resources to be exploited effectively in an aggregated form. We present the Condor-G system, which leverages software from Globus and Condor to enable users to harness multi-domain resources as if they all belong to one personal domain. We describe the structure of Condor-G and how it handles job management, resource selection, security, and fault tolerance. We also present results from application experiments with the Condor-G system. We assert that Condor-G can serve as a general-purpose interface to Grid resources, for use by both end users and higher-level program development tools.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D. Abramson, J. Giddy and L. Kotler, High performance parametric modeling with Nimrod/G: Killer application for the Global Grid~, in: IPDPS'2000 (IEEE Press, 2000).

  2. D. Abramson, R. Sosic, J. Giddy and B. Hall, Nimrod: A tool for performing parameterized simulations using distributed workstations, in: Proc. 4th IEEE Symp. on High Performance Distributed Computing (1995).

  3. C. Anglano et al. Integrating GRID tools to build a computing resource broker: Activities of DataGrid WP1, in: Computing in High Energy and Nuclear Physic (2001).

  4. K. Anstreicher, N. Brixius, J.-P. Goux and J. Linderoth, Solving large quadratic assignment problems on computational Grids, in: Mathematical Programming (2000). CONDOR-G 245

  5. K. Anstreicher, N. Brixius, J.-P. Goux and J. Linderoth, Solving large quadratic assignment problems on computational Grids, in: Mathematical Programming (to appear).

  6. J. Beiriger, W. Johnson, H. Bivens, S. Humphreys and R. Rhea, Constructing the ASCI Grid, in: Proc. 9th IEEE Symposium on High Performance Distributed Computing (IEEE Press, 2000).

  7. F. Berman, High-performance schedulers, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan Kaufmann, 1999) pp. 279-309.

  8. F. Berman, R.Wolski, S. Figueira, J. Schopf and G. Shao, Applicationlevel scheduling on distributed heterogeneous networks, in: Proc. Supercomputing '96 (1996).

  9. J. Bester, I. Foster, C. Kesselman, J. Tedesco and S. Tuecke, GASS: A data movement and access service for wide area computing systems, in: 6th Workshop on I/O in Parallel and Distributed Systems, 5 May 1999.

  10. H. Casanova, G. Obertelli, F. Berman and R. Wolski, The AppLeS parameter sweep template: User-level middleware for the Grid, in: Proc. SC'2000 (2000).

  11. K. Czajkowski, S. Fitzgerald, I. Foster and C. Kesselman, Grid information services for distributed resource sharing, in: Proc. 10th IEEE Int. Symp. on High-Performance Distributed Computing (2001) pp. 181-184.

  12. K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith and S. Tuecke, A resource management architecture for metacomputing systems, in: Proc. IPPS/SPDP '98 Workshop on Job Scheduling Strategies for Parallel Processing (1998).

  13. D.H.J. Epema, M. Livny, R.V. Dantzig, X. Evers and J. Pruyne, A worldwide flock of condors: Load sharing among workstation clusters, Future Generation Computer Systems 12 (1996).

  14. I. Foster and C. Kesselman, Globus: A toolkit-based Grid architecture, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan Kaufmann, 1999) pp. 259-278.

  15. I. Foster, C. Kesselman, G. Tsudik and S. Tuecke, A security architecture for computational Grids, in: ACM Conference on Computers and Security (1998) pp. 83-91.

  16. I. Foster, C. Kesselman and S. Tuecke, The anatomy of the Grid: Enabling scalable virtual organizations, Int. J. High Performance Computing Applications 15(3) (2001) 200–222, http://www.globus.org/ research/papers/anatomy.pdf.

    Google Scholar 

  17. J. Gray and A. Reuter, Two-phase commit: Making computations atomic, in: Transaction Processing: Concepts and Techniques (Morgan Kaufmann, 1993) pp. 562-573.

  18. R. Henderson and D. Tweten, Portable Batch System: External Reference Specification (1996).

  19. IBM, Using and Administering IBM LoadLeveler, Release 3.0, IBM CorporationSC23-3989 (1996).

  20. Institute S.C.R., DQS 3.1.3 User Guide, Florida State University, Tallahassee (1996).

  21. W.E. Johnston, D. Gannon and B. Nitzberg, Grids as production computing environments: The engineering aspects of NASA's information power Grid, in: Proc. 8th IEEE Symposium on High Performance Distributed Computing (IEEE Press, 1999).

  22. E. Korpela, D. Werthimer, D. Anderson, J. Cobb and M. Lebofsky, SETI@home: Massivel distributed computing for SETI, Computing in Science and Engineering 3(1) (2001).

  23. G. von Laszewski, I. Foster, J. Gawor and J. Lane, A Java commodity Grid toolkit, Concurrency: Practice and Experience 13 (2001) (to appear).

  24. M. Litzkow, M. Livny and M. Mutka, Condor - A hunter of idle workstations, in: Proc. 8th Int. Conf. on Distributed Computing Systems (1988) pp. 104-111.

  25. M. Litzkow, T. Tannenbaum, J. Basney and M. Livny, Checkpoint and migration of UNIX processes in the Condor distributed processing system, University of Wisconsin-Madison Computer Sciences, Technical Report 1346 (1997).

  26. M. Livny, High-throughput resource management, in: The Grid: Blueprint for a New Computing Infrastructure, eds. I. Foster and C. Kesselman (Morgan Kaufmann, 1999) pp. 311-337.

  27. NCSA Alliance, Grid-in-a-Box, http://www.ncsa.uiuc.edu/TechFocus/ Deployment/GiB.

  28. NCSA Alliance, GridGaussian, http://www.ncsa.uiuc.edu/Divisions/ ACES/APG/grid_gaussian.htm.

  29. J. Novotny, S. Tuecke and V. Welch, An online credential repository for the Grid: MyProxy, in: Proc. 10th IEEE Int. Symp. on High-Performance Distributed Computing (2001).

  30. M. Papakhian, Comparing job-management systems: The user's perspective, IEEE Computational Science & Engineering (April-June) (1998), http://pbs.mrj.com.

  31. R. Raman, M. Livny and M. Solomon, Resource management through multilateral matchmaking, in: Proc. of the 9th IEEE Symposium on High Performance Distributed Computing (HPDC9), Pittsburgh, Pennsylvania (August 2000) pp. 290-291.

  32. J. Steiner, B.C. Neuman and J. Schiller, Kerberos: An authentication system for open network systems, in: Proc. Usenix Conference (1988) pp. 191-202.

  33. R. Stevens, P. Woodward, T. DeFanti and C. Catlett, From the I-WAY to the national technology Grid, Communications of the ACM 40(11) (1997) 50–61.

    Google Scholar 

  34. S. Vazhkudai, S. Tuecke and I. Foster, Replica selection in the Globus data Grid, in: Proc. of the 1st IEEE/ACM Int. Conference on Cluster Computing and the Grid (CCGRID 2001) (IEEE Computer Society Press, 2001) pp. 106-113.

  35. S. Zhou, LSF: Load sharing in large-scale heterogeneous distributed systems, in: Proc. Workshop on Cluster Computing (1992).

  36. Condor Project, Globus ASCII Helper Protocol (GAHP), http://www.cs.wisc.edu/condor/gaph.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Wisconsin, Madison, WI, 53706, USA

    James Frey, Todd Tannenbaum & Miron Livny

  2. Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, 60439, USA

    Ian Foster & Steven Tuecke

Authors
  1. James Frey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Todd Tannenbaum
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Miron Livny
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ian Foster
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steven Tuecke
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Frey, J., Tannenbaum, T., Livny, M. et al. Condor-G: A Computation Management Agent for Multi-Institutional Grids. Cluster Computing 5, 237–246 (2002). https://doi.org/10.1023/A:1015617019423

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1015617019423

Search

Navigation