Abstract
Data Grids seek to harness geographically distributed resources for large-scale data-intensive problems. The issues that need to be considered in the Data Grid research area include resource management for computation and data. Computation management comprises scheduling of jobs, load balancing, fault tolerance and response time; while data management includes replication and movement of data at selected sites. As jobs are data intensive, data management issues often become integral to the problems of scheduling and effective resource management in the Data Grids. Therefore, integration of data replication and scheduling strategies is important. Such an integrating solution is either non-existent or work in a centralized manner which is not scalable. The paper deals with the problem of integrating the scheduling and replication strategies in a distributed manner. As part of the solution, we have proposed a Distributed Replication and Scheduling Strategy (DistReSS) which aims at an iterative improvement of the performance based on coupling between scheduling and replication, which is achieved in distributed and hierarchical fashion. Results suggest that, in the context of our experiments, DistReSS performs comparable to the centralized approach when the parameters are tuned properly in addition to being more scalable to the centralized approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chervenak, I., Foster, C., Kesselman, C., Salisbury, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications 23, 187–200 (2001)
Beck, M., Moore, T.: The Internet2 distributed storage infrastructure project: An architecture for internet content channels. In: Comp. Net. and ISDN Systems (1998)
Foster, Kasselman, C.: The Grid 2: Blueprint for a new Computing Infrastructure. Morgan Kaufman, San Francisco (2004)
Foster, Kesselman, C.: The Globus Project: A Status Report. In: Proc. IPPS/SPDP 1998 Heterogeneous Computing Workshop, pp. 4–18 (1998)
Casanova, H., Obertelli, G., Berman, F., Wolski, R.: The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid. In: Proc. of SuperComputing (2000)
Banino, O., Beaumont, L., Carter, J., Ferrante, A.: Scheduling Strategies for Master-Slave tasking for Heterogeneous Processor Platforms. IEEE Trans. On Parallel and Distributed Systems 15(4) (April 2004)
Alhusaini, A.H., Prasanna, V.K., Raghavendra, C.S.: A Unified Resource Scheduling Framework for Heterogeneous Computing Environments. In: Eighth Heterogeneous Computing Workshop (1999)
Bell, W.H., Cameron, D.G., et al.: Simulation of Dynamic Grid Replication Strategies in OptorSim. In: Proceedings of the Third Int’l Workshop on Grid Computing (2002)
Ranganathan, K., Foster, I.: Identifying Dynamic Replication Strategies for a High Performance Data Grid. In: Proceedings of the Second Intl Work. on Grid Comp. (2001)
Bell, W.H., Cameron, D.G., Carvajal-Schiaffino, R., Millar, A.P., Stockinger, K., Zini, F.: Evaluation of an Economy-Based File Replication Strategy for a Data Grid. In: Proc. CCGrid (May 2003)
Cameron, D., Casey, J., Guy, L., Kunszt, P., Lemaitre, S., McCance, G., Stockinger, H., Stockinger, K., Andronico, G., Bell, W., Ben-Akiva, I., Bosio, D., Chytracek, R., Domenici, A., Donno, F., Hoschek, W., Laure, E., Lucio, L., Millar, P., Salconi, L., Segal, B., Silander, M.: Replica Management in the EU DataGrid Project. International J. of Grid Computing 2(4), 341–351 (2004)
Thain, J., Bent, A., Arpaci-Dusseau, R.: Gathering at the Well: Creating Communities for Grid I/O. In: Proc. of SuperComputing (2001)
Basney, J., Livny, M., Mazzanti, P.: Harnessing the Capacity of Computational Grids for High Energy Physics. In: CHEP 2000. Proceedings of the International Conference on Computing in High Energy and Nuclear Physics (2000)
Ranganathan, K., Foster, I.: Simulation Studies of Computation and Data Scheduling Algorithms for Data Grids. J. of Grid Computing 1(2), 53–62 (2003)
Chakrabarti, A., Dheepak, R.A.: Integration of Scheduling and Replication in Data Grids. In: Proc. IEEE HiPC ( December 2004)
Dheepak, R.A., Ali, S., Chakrabarti, A., Sengupta, S.: Study of Scheduling Strategies in Dynamic Data Grid Environment. In: Sen, A., Das, N., Das, S.K., Sinha, B.P. (eds.) IWDC 2004. LNCS, vol. 3326, Springer, Heidelberg (2004)
UCB/LBNL/VINT Network Simulator – ns (version 2), http://www.isi.edu/nsnam/ns
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chakrabarti, A., Sengupta, S. (2007). Scalable and Distributed Mechanisms for Integrated Scheduling and Replication in Data Grids. In: Rao, S., Chatterjee, M., Jayanti, P., Murthy, C.S.R., Saha, S.K. (eds) Distributed Computing and Networking. ICDCN 2008. Lecture Notes in Computer Science, vol 4904. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77444-0_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-77444-0_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77443-3
Online ISBN: 978-3-540-77444-0
eBook Packages: Computer ScienceComputer Science (R0)