Abstract
Many current international scientific projects are based on large scale applications that are both computationally complex and require the management of large amounts of distributed data. Grid computing is fast emerging as the solution to the problems posed by these applications. To evaluate the impact of resource optimisation algorithms, simulation of the Grid environment can be used to achieve important performance results before any algorithms are deployed on the Grid. In this paper, we study the effects of various job scheduling and data replication strategies and compare them in a variety of Grid scenarios using several performance metrics. We use the Grid simulator \textsf{OptorSim} , and base our simulations on a world-wide Grid testbed for data intensive high energy physics experiments.
Our results show that scheduling algorithms which take into account both the file access cost of jobs and the workload of computing resources are the most effective at optimising computing and storage resources as well as improving the job throughput. The results also show that, in most cases, the economy-based replication strategies which we have developed improve the Grid performance under changing network loads.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M.F. Arlitt and C.L. Williamson, “Web Server Workload Characterization: The Search for Invariants”, in ACM Sigmetrics International Conference on Measurements and Modeling of Computer Systems, Philadelphia, PA, USA, 1996.
P. Barford and M. Crovella, “Generating Representative Web Workloads for Network and Server Performance Evaluation”, in ACM Sigmetrics International Conference on Measurements and Modeling of Computer Systems, Madison, WI, USA, 1998.
W.H. Bell, D.G. Cameron, L. Capozza, P. Millar, K. Stockinger and F. Zini, “Design of a Replica Optimisation Framework”, Technical Report DataGrid-02-TED-021215, CERN, Geneva, Switzerland, 2002.
W.H. Bell, D.G. Cameron, L. Capozza, P. Millar, K. Stockinger and F. Zini, “OptorSim – a Grid Simulator for Studying Dynamic Data Replication Strategies”, International Journal of High Performance Computing Applications, Vol. 17, No. 4, 2003.
W.H. Bell, D.G. Cameron, R. Carvajal-Schiaffino, P. Millar, K. Stockinger and F. Zini, “Evaluation of an Economy-Based File Replication Strategy for a Data Grid”, in International Workshop on Agent Based Cluster and Grid Computing at CCGrid2003, Tokyo, Japan, 2003.
L. Breslau et al., “Web Caching and Zipf-like Distributions: Evidence and Implications”, in IEEE INFOCOM’99, New York, NY, USA, 1999.
R. Buyya and M. Murshed, “GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing”, The Journal of Concurrency and Computation: Practice and Experience, pp. 1–32, 2002.
L. Capozza, K. Stockinger and F. Zini, “Preliminary Evaluation of Revenue Prediction Functions for Economically-Effective File Replication”, DataGrid-02-TED-020724, CERN, Geneva, Switzerland, July 2002.
P. Crosby, “EDGSim”. http://www.hep.ucl.ac.uk/∼pac/EDGSim/
B.T. Huffman et al., “The CDF/DO UK GridPP Project”, CDF Internal Note 5858.
H. Lamehamedi, Z. Shentu, B. Szymanski and E. Deelman, “Simulation of Dynamic Data Replication Strategies in Data Grids”, in Proceedings of the 12th Heterogeneous Computing Workshop (HCW2003), Nice, France, 2003.
V. Lefebure and T.W. (eds), The Spring 2002 DAQ TDR Production, CMS Internal Note, 2005/000, Geneva, Switzerland.
Y.-T. Li, “GridNM”. http://www.hep.ucl.ac.uk/∼ytl/monitoring/gridnm/gridnm-client.html
C. Logg et al., “SLAC WAN Bandwidth Measuring Tests”. http://www.slac.stanford.edu/comp/net/bandwidth-tests/antonia/html/slac%_wan_bw_tests.html
M. Orlowska, S. Weerawarana, M. Papazoglou and J. Yang (eds.), “Service-Oriented Computing – ICSOC 2003 First International Conference”, in Lecture Notes in Computer Science, No. 2910. Springer-Verlag: Trento, Italy, 2003.
K. Ranganathan and I. Foster, “Identifying Dynamic Replication Strate gies for a High Performance Data Grid”, in Proceedings of the International Grid Computing Workshop, Denver, CO, USA, 2001.
K. Ranganathan and I. Foster, “Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications”, in International Symposium of High Performance Distributed Computing, Edinburgh, Scotland, 2002.
“SLAC WAN Bandwidth Measuring Tests at FNAL”. http://dmzmonO.deemz.net/∼wanbanmon/html/slac_wan_bw_tests.html#summary%.
“The European DataGrid Project”. http://www.edg.org
A. Tirumala et al., “Iperf.” http://dast.nlanr.net/Projects/Iperf/
“UK e-Science Grid Network Monitoring”. http://gridmon.ucs.ed.ac.uk/gridmon/
W. Vickrey, “Counterspeculation, Auctions, and Competitive Sealed Tenders”, The Journal of Finance, Vol. 16, No. 1, pp. 8–37, 1961.
WP2, “OptorSim, a Replica Optimiser Simulator”. http://cern.ch/edg-wp2/optimization/optorsim.html
WP7, “Network Services”. http://ccwp7.in2p3.fr/
G.K. Zipf, Selected Studies of the Principle of Relative Frequency in Language. Harvard University Press: Cambridge, MA, 1932.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Cameron, D.G., Millar, A.P., Nicholson, C. et al. Analysis of Scheduling and Replica Optimisation Strategies for Data Grids Using OptorSim. J Grid Computing 2, 57–69 (2004). https://doi.org/10.1007/s10723-004-6040-6
Issue Date:
DOI: https://doi.org/10.1007/s10723-004-6040-6