Abstract
A grid consists of high-end computational, storage, and network resources that, while known a priori, are dynamic with respect to activity and availability. Efficient scheduling of requests to use grid resources must adapt to this dynamic environment while meeting administrative policies. This paper discusses the necessary requirements of such a scheduler and proposes a framework that can administrate grid policies and schedule complex and data intensive scientific applications. We present early experimental results for proposed a framework that effectively utilizes other grid infrastructure such as workflow management systems and execution systems. These results demonstrate that proposed a framework can effectively schedule work across a large number of distributed clusters that are owned by multiple units in a virtual organization.
This work was supported by a grand from Ministry of Commerce, Industry and Energy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Avery, P., Foster, I.: The GriPhyN Project: Towards Petascale Virtual-Data Grids. The 2000 NSF Information and Technology Research Program (2000)
Chervenak, A., et al.: Giggle: A Framework for Constructing Scalable Replica Location Services. In: Proceedings of SC2002 Conference (to appear, November 2002)
Deelman, E., Blythe, J., Gil, Y., Kesselman, C.: Pegasus: Planning for Execution in Grids. Technical Report GriPhyN-2002-20 (November 2002)
Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International J. Supercomputer Applications 15(3) (2001)
Foster, I., Voeckler, J., Wilde, M., Zhao, Y.C.: A Virtual Data System for Representing, Querying, and Automating Data Derivation. In: The 14th International Conference on Scientific and Statistical Database Management (SSDBM 2002) (2002)
Gerasoulis, A., Yang, T.: On the Granularity and Clustering of Directed Acyclic Task Graphs. IEEE Trans. Parallel and Distributed Systems 5(9), 951–967 (1994)
Ghafoor, A., Yang, J.: A Distributed Heterogeneous Supercomputing Management System. Computer 26(6), 78–86 (1993)
Kaddoura, M., Ranka, S.: Runtime Support for Parallelization of Data-Parallel Applications on Adaptive and Nonuniform Environments. Journal of Parallel and Distributed Computing, 163–168 (June 1997); Special Issue on Workstation Clusters and Network-based Computing
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. Technical Report, Department of Computer Science, University of Minnesota (1995)
Kwok, Y., Ahmad, I.: Static Scheduling Algorithms for Allocating Directed Task Graphs to Multiprocessors. ACM Computing Surveys 31(4) (December 1999)
Li, Y.A., Antonio, J.K., Siegel, H.J., Tan, M., Watson, D.W.: Determining the Execution Time Distribution for a Data Parallel Program in a Heterogeneous Computing Environment. Journal of Parallel and Distributed Computing 44(1), 35–52 (1997)
Ranka, S., Kaddoura, M., Wang, A., Fox, G.C.: Heterogeneous Computing on Scalable Heterogeneous Systems. In: Proceedings of Supercomputing 1993, pp. 763–764 (1993)
Sandholm, T., Gawor, J.: Globus Toolkit 3 Core – Agrid Service Container Framework, http://www-unix.globus.org/toolkit/documentation.html
Thain, D., Tannenbaum, T., Livny, M.: Condor and the Grid. In: Berman, F., Hey, A.J.G., Fox, G. (eds.) Grid Computing: Making The Global Infrastructure a Reality. John Wiley, Chichester (2003)
Yang, T., Gerasoulis, A.: DSC: Scheduling parallel tasks on an unbounded number of processors. IEEE Trans. Parallel and Distributed Systems 5(9), 951–967 (1994)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lee, Mh., In, Ju., Choi, Ei. (2006). A Scheduling Middleware for Data Intensive Applications on a Grid. In: Gabrys, B., Howlett, R.J., Jain, L.C. (eds) Knowledge-Based Intelligent Information and Engineering Systems. KES 2006. Lecture Notes in Computer Science(), vol 4253. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11893011_134
Download citation
DOI: https://doi.org/10.1007/11893011_134
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-46542-3
Online ISBN: 978-3-540-46544-7
eBook Packages: Computer ScienceComputer Science (R0)