Abstract
The workload management task of the DataGrid project was mandated to define and implement a suitable architecture for distributed scheduling and resource management in a Grid environment. The result was the design and implementation of a Grid Workload Management System, a super-scheduler with the distinguishing property of being able to take data access requirements into account when scheduling jobs to the available Grid resources. Many novel issues in various fields were faced such as resource management, resource reservation and co-allocation, Grid accounting. In this paper, the architecture and the functionality provided by the DataGrid Workload Management System are presented.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
“Home page of the DataGrid project”, http://www.edg.org
D. Abramson, J. Giddy and R. Buyya, “An Economy Driven Resource Management Architecture for Global Computational Power Grids”, in Proc. of the 7th International International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2000), Las Vegas, USA, 2000.
R. Alfieri et al., “VOMS, an Authorization System for Virtual Organizations”, in Grid Computing, First European Across Grids Conference, 2004.
S. Andreozzi, M. Sgaravatto and C. Vistoli, “Sharing a Conceptual Model of Grid Resources and Services”, in Proceedings of the 2003 Computing in High Energy and Nuclear Physics Conference (CHEP03), La Jolla, CA, USA, March 2003.
C. Anglano et al., “Integrating Grid Tools to Build a Computing Resource Broker: Activities of DataGrid WP1”, in Proceedings of the 2001 Computing in High Energy and Nuclear Physics Conference (CHEP01), Beijing, China, September 2001.
G. Avellino et al., “The EU DataGrid Workload Management System: Towards the Second Major Release”, in Proceedings of the 2003 Computing in High Energy and Nuclear Physics Conference (CHEP03), La Jolla, CA, USA, March 2003.
G. Avellino et al., “The First Deployment of Workload Management Services on the EU DataGrid Testbed: Feedback on Design and Implementation”, in Proceedings of the 2003 Computing in High Energy and Nuclear Physics Conference (CHEP03), La Jolla, CA, USA, March 2003.
A. Barmouta and R. Buyya, “GridBank: A Grid Accounting Services Architecture (GASA) for Distributed System Sharing and Integration”, in Proceedings of the 17th Annual International Parallel & Distributed Processing Symposium (IPDPS 2003) Workshop on Internet Computing and E-Commerce.
G. Bronevetsky et al., “Automated Aplication-Level Checkpointing of MPI Programs”, in ACM Symposium on Principles and Practice of Parallel Programming, 2003.
A. Cooke et al., “Relational Grid Monitoring Architecture (R-GMA)”, Presented at UK e-Science All-Hands meeting, Nottingham, UK, 2003. https://edms.cern.ch/file/400756/1/rgma.pdf
K. Czajkowski, I. Foster, N. Karonis, C. Kesselman, S. Martin, W. Smith and S. Tuecke, “A Resource Management Architecture for Metacomputing Systems”, Lecture Notes in Computer Science, Vol. 1459, 1998.
H. Dail et al., “Scheduling in the Grid Application Development Software Project”, in [24], pp. 73–98.
E. Deelman, J. Blythe, Y. Gil and C. Kesselman, “Workflow Management in GriPhyn”, in [24], pp. 99–118.
D.F. Ferguson et al., “Economic Models for Allocating Resources in Computer Systems”, in Market-Based Control: A Paradigm for Distributed Resource Allocation. World Scientific: Hong Kong, 1996.
I. Foster and C. Kesselman, “The Globus Toolkit”, in I. Foster and C. Kesselman (eds), The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann: San Francisco, CA, 1999, Chap. 11, pp. 259–278.
I. Foster, C. Kesselman, G. Tsudik and S. Tuecke, “A Security Architecture for Computational Grids”, in Proc. 5th ACM Conference on Computer and Communications Security Conference, 1998.
J. Frey, T. Tannenbaum, I. Foster, M. Livny and S. Tuecke, “Condor-G: A Computation Management Agent for Multi-Institutional Grids”, in Proceedings of the Tenth IEEE Symposium on High Performance Distributed Computing (HPDC). San Francisco, California, 2001, pp. 7–9.
F. Giacomini, F. Prelz, M. Sgaravatto, I. Terekhov, G. Garzoglio and T. Tannenbaum, “Planning on the Grid: A Status Report”, Technical Report INFN-TC-02/26, INFN-GRID Project, 2002.
S. Jackson et al., “Charter of the Global Grid Forum Usage Records Working Group”, Technical Report, 2003.
K. Kurowski, J. Nabrzyski, A. Oleksiak and J. Weglarz, “Multicriteria Aspects of the Grid Resource Management”, in [24], pp. 271–294.
A. Křenek and Z. Salvet, “L&B API Reference”, DataGrid-01-TED-0139, 2003, http://lindir.ics.muni.cz/dg_public/
M. Litzkow, M. Livny and M. Mutka, “Condor – A Hunter of Idle Workstations”, in Proceedings of the 8th International Conference of Distributed Computing Systems, 1988.
M. Milka, G. Waligora and J. Weglarz, “A Metaheuristic Approach to Scheduling Workflow Jobs on a Grid”, in [24], pp. 295–320.
J. Nabryzski, J. Schopf and J. Weglarz (eds), Grid Resource Management. Kluwer Academic: Norwell, MA, 2003.
J. Novotny, S. Tuecke and V. Welch, “An Online Credential Repository for the Grid: MyProxy”, in Proceedings of the Tenth International Symposium on High Performance Distributed Computing (HPDC-10), 2001.
R. Piro, A. Guarise and A. Werbrouck, “An Economy-Based Accounting Infrastructure for the DataGrid”, in Proc. of the 4th International Workshop on Grid Computing (Grid2003), Phoenix, Arizona, USA, 2003.
R. Piro, A. Guarise and A. Werbrouck, “Simulation of Price-Sensitive Resource Brokering and the Hybrid Pricing Model with DGAS-Sim”, in Proc. of the 13th International Workshops on Enabling Technologies: Infrastructures for Collaborative Enterprises (WETICE 2004), Modena, Italy, 2004.
R. Raman, M. Livny and M. Solomon, “Matchmaking: Distributed Resource Management for High Throughput Computing”, in Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing (HPDC7), Chicago, IL, 1998.
R. Raman, M. Livny and M. Solomon, “Matchmaking: Distributed Resource Management for High Throughput Computing”, in Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing (HPDC7), Chicago, IL, 1998.
R. Raman, M. Livny and M. Solomon, “Resource Management through Multilateral Matchmaking”, in Proceedings of the Ninth IEEE Symposium on High Performance Distributed Computing (HPDC9), Pittsburgh, PA, 2000, pp. 290–291.
J.M. Schopf, “Ten Actions when SuperScheduling”, Technical Report GFD-I.4, Global Grid Forum, Scheduling Working Group, 2001.
D. Simmel et al., “Charter of the Global Grid Forum Grid Checkpoint Recovery Working Group”, Technical Report, 2004.
D. Thain and M. Livny, “Bypass: A Tool for Building Split Execution Systems”, in Proceedings of the Ninth IEEE Symposium on High Performance Distributed Computing (HPDC9), Pittsburgh, PA, 2000, pp. 79–85.
M.P. Thomas et al., “The GridPort Toolkit: a System for Building Grid Portals”, in Proc. of the Tenth IEEE International Symposium on High Performance Distributed Computing, 2001.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Avellino, G., Beco, S., Cantalupo, B. et al. The DataGrid Workload Management System: Challenges and Results. J Grid Computing 2, 353–367 (2004). https://doi.org/10.1007/s10723-005-0150-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-005-0150-7