Abstract
Scheduling constitutes an integral feature of Grid computing infrastructures, being also a key to realizing several of the Grid promises. In particular, scheduling can maximize the resources available to end users, accelerate the execution of jobs, while also supporting scalable and autonomic management of the resources comprising a Grid. Grid scheduling functionality hinges on middleware components called meta-schedulers, which undertake to automatically distribute jobs across the dispersed heterogeneous resources of a Grid. In this paper we present the design and implementation of a Grid meta-scheduler, which we call EMPEROR. EMPEROR provides a framework for implementing scheduling algorithms based on performance criteria. In implementing a particular instantiation of this framework, we have devised models for predicting host load and memory resources, and accordingly for estimating the running time of a task. These models hinge on time series analysis techniques and take into account results of the cluster computing literature. Apart from incorporating these models, EMPEROR provides fully fledged Grid scheduling functionality, which complies with OGSA standards as the later are reflected in the Globus toolkit. Specifically, EMPEROR interfaces to Globus middleware services (i.e., GSI, MDS, GRAM) towards discovering resources, implementing the scheduling algorithm and ultimately submitting jobs to local scheduling systems. By and large, EMPEROR is one of the few standards based meta-schedulers making use of dynamic scheduling information.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
I. Foster, “What is the Grid? A Three Point Checklist”, GRID Today (http:www.gridtoday.com), July 20, 2002.
I. Foster, C. Kesselman and S. Tuecke, “The Anatomy of the Grid: Enabling Scalable Virtual Organizations”, International Journal of Supercomputer Applications, Vol. 15, No. 3, 2001.
F. Berman, G. Fox and T. Hey (eds.), Grid Computing: Making The Global Infrastructure a Reality. ISBN: 0470853190, John Wiley & Sons, April 8, 2003.
K. Krauter, R. Buyya and M. Maheswaran, “A Taxonomy and Survey of Grid Resource Management Systems for Distributed Computing”, International Journal of Software: Practice and Experience (SPE), Wiley, USA, pp. 135–164, Vol. 32, No. 2, February 2002. ISSN: 0038-0644.
R. Buyya, D. Abramson, J. Giddy and H. Stockinger, “Economic Models for Resource Management and Scheduling in Grid Computing”, Special Issue on Grid Computing Environments, The Journal of Concurrency and Computation: Practice and Experience (CCPE), Wiley, USA, Vol. 14, No. 13–15, pp. 1507–1542. November–December 2002.
L. Ferreira et al. “Introduction to Grid Computing with Globus”, IBM redpaper, December 2002.
W. Lee, S. McGough, S. Newhouse and J. Darlington, “Load-balancing EU-DataGrid Resource Brokers”, in UK e-ScienceAll Hands Meeting, Nottingham, UK, pp. 137–144, September2003.
S. Vazhkudai and J.M. Schopf, “Using regression techniques to predict large data transfers”, The International Journal of High Performance Computing Applications (IJHPCA) special issue on Grid Computing: Infrastructure and Applications, Vol. 17, No. 3, August 2003.
L. Yang, J.M. Schopf and I. Foster, “Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments”, in The Proceedings of the ACM/IEEE Supercomputing 2003 Conference, p.31, November 2003.
T. Tannenbaum, D. Wright, K. Miller and M. Livny, “Condor – A Distributed Job Scheduler”, in Thomas Sterling (ed.), Beowulf Cluster Computing with Linux, The MIT, 2002, ISBN: 0-262-69274-0.
I. Foster, D. Gannon and H. Kishimoto (eds.), “The Open Grid Services Architecture”, Global Grid Forum Draft, draft-ggf-ogsa-ogsa-011, September 23, 2003.
The Globus toolkit, Globus Alliance, http://www.globus.org.
P.A. Dinda, “The Statistical Properties of Host Load”, Scientific Programming, Vol. 7, No. 3–4, pp. 211–229, 1999.
P. Dinda and D. O'Hallaron, “Host Load Prediction Using Linear Models”, Cluster Computing, Vol. 3, No. 4, 2000.
P. Dinda, “Online Prediction of the Running Time of Tasks”, Cluster Computing, Vol. 5, No. 3, 2002.
J.M. Schopf, “Ten Actions when Grid Scheduling”, in J. Nabrzyski, J.M. Schopf and J. Weglarz, (eds.), Grid Resource Management: State of the Art and Future Trends, Chapter 2, Kluwer September 2003, ISBN: 1402075758.
Z. Balaton, P. Kacsuk and N. Podhorszki, “Application Monitoring in the Grid with GRM and PROVE”, in the Proc. of the ICCS 2001, International Conference, San Francisco, CA, USA, May 28–30, 2001. Proceedings, Lecture Notes in Computer Science 2073 Springer 2001, pp. 253–262, ISBN 3-540-42232-3.
F. Vraalsen, R.A. Aydt, C.L. Mendes and D.A. Reed, “Performance Contracts: Predicting and Monitoring Grid Application Behavior”, in Proc. of the 2nd International Workshop on Grid Computing (GRID), Denver, Colorado, Vol. 2242, pp. 154–165, Nov. 2001.
J.L. Hennessy, D.A. Patterson and D. Goldberg, Computer Architecture: A Quantitative Approach, 3rd ed. Morgan Kaufmann Publishers, Inc., San Mateo, CA, 2002.
J. Soldatos, E Vayias and L Polymenakos, “Grid Donors Resources Utilization Analysis towards Optimal Job Scheduling”, in the Proc. of the DPSN '04, Workshop, held in the Scope of IFIP Networking, 2004, Athens, May 14th 2004.
A. Conny-Murray, “Grid Computing's Promises and Perils”, Network Magazine, February 2004.
P. Dinda, D. O'Hallaron, “An Extensible Toolkit for Resource Prediction In Distributed Systems”, in Technical Report CMU-CS-99-138, School of Computer Science, Carnegie Mellon University, July 1999.
P.J. Brockwell and R.A. Davis, Introduction to Time Series and Forecasting, 2nd ed., Springer, 2002. ISBN: 0-387-95351-5.
M. Maheswaran, S. Ali, H.J. Siegel, D. Hensgen and R.F. Freund, “Dynamic Matching and Scheduling of a Class of Independent Tasks onto Heterogeneous Computing Systems”, in the Proc. of the 8th Heterogeneous ComputingWorkshop, San Juan, Puerto Rico, 1999.
R.F. Freund, M. Gherrity, S. Ambrosius, M. Campbell, M. Halderman, D. Hensgen, E. Keith, T. Kidd, M. Kussow, J.D. Lima, F. Mirabile, L. Moore, B. Rust and H.J. Siegel, “Scheduling resources in multi-user, heterogeneous, computing environments with SmartNet”, in Seventh IEEE Heterogeneous Computing Workshop (HCW '98), Orlando, Florida, USA, pp. 184–199, March 1998.
W. Smith, I. Foster and V. Taylor, “Predicting Application Run Times Using Historical Information”, in Proc. IPPS/SPDP '98 Workshop on Job Scheduling Strategies for Parallel Processing, 1998.
J. Soldatos, L. Polymenakos and G. Kormentzas, ``Programmable Grids Framework enabling QoS in an OGSA context”, in the Proc. of the 1st International Workshop on “Active and Programmable Grids Architectures and Components in the scope of ICCS'04, Krakow, Poland, June 7–9, 2004.
H. Rajic et al., “Distributed Resource Management Application API Specification 1.0”, GFD-R-P.022 (Proposed Recommendation), Distributed Resource Management Application API (DRMAA) Working Group, June 2004.
P. Kacsuk, A. Goyeneche, T. Delaitre, T. Kiss, Z. Farkas and T. Boczko, “High-Level Grid Application Environment to Use Legacy Codes as OGSA Grid Services”, in the Proc. of the 5th International Workshop on Grid Computing (GRID 2004), 8 November 2004, Pittsburgh, PA, USA, IEEE Computer Society, pp. 428–435, 2004, ISBN 0-7695-2256-4.
The EU Data Grid project, http://eu-datagrid.web.cern.ch/eu-datagrid/.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Adzigogov, L., Soldatos, J. & Polymenakos, L. EMPEROR: An OGSA Grid Meta-Scheduler Based on Dynamic Resource Predictions. J Grid Computing 3, 19–37 (2005). https://doi.org/10.1007/s10723-005-9001-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10723-005-9001-9