Abstract
Data intensive applications constitute a large and increasing share of Grid computing. However there are relatively few results on how to improve the efficiency of the basic data transfer mechanisms used to move large data set in and out of Grid nodes. In this paper we describe a simple and general technique to improve the throughput of data transfer protocols and we demonstrate it on the SDSC Storage Resource Broker (SRB), a remote storage access middleware for supercomputer applications. We achieve a maximum performance improvement of 43%/52% for remote reads/writes larger than 1MB with a few changes to the original SRB protocol. The protocol was restructured by introducing a notion of pipelining that enables the overlapping of the various stages of the data processing, such as network transfer and disk access. We present a detailed analysis of the pipelined SRB implementation and of the pipeline cost model we used to drive our design.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
C. Baru, R. Moore, A. Rajasekar, M. Wan, “The SDSC Storage Resource Broker,” Proc. CASCON’98 Conference, Nov.30-Dec.3, 1998, Toronto, Canada.
J. Bester, I. Foster, C. Kesselman, J. Tedesco, S. Tuecke, “GASS: A Data Movement and Access Service for Wide Area Computing Systems”, Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999.
“The SDSC Storage Resource Broker Homepage”: http://www.npaci.edu/DICE/SRB/
D.E. Culler, R. Karp, D.A. Patterson, A. Sahay, K.E. Schauser, E. Santos, R. Subramonian, and T. von Eicken, “LogP: towards a realistic model of parallel computation”, Procs. of the 4th SIGPLAN Symp. on Principles and Practices of Parallel Programming, ACM, May 1993.
R. Wang, A. Krishnamurthy, R. Martin, T. Anderson & D. E. Culler, “Modeling Communication Pipeline Latency”, Procs. Of ACM International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS’ 98), June 1998.
X. Shen, W. Liao and A. Choudhary. “Remote I/O Optimization and Evaluation for Tertiary Storage Systems through Storage Resource Broker” to appear in IASTED Applied Informatics, Innsbruck, Austria, Febuary, 2001
T. Kurc, M. Beynon, A. Sussman, J. Saltz, “ DataCutter and a Client Interface for the Storage Resource Broker with DataCutter services”.
Ian Foster and Carl Kesselman, The Grid, Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers Inc., 1999.
S. Pakin, M. Lauria, A. Chien “High Performance Messaging on Workstations: Illinois Fast Messages (FM) for Myrinet” Supercomputing’ 95.
J. Chase, D. Anderson, A. Gallatin, A. Lebeck, K. Yocum, “Network I/O with Trapeze”, 1999 Hot Interconnects Symposium, August 1999.
T. von Eicken,. D. Culler, S. Goldstein, K. E. Schauser, “Active Messages: A mechanism for Integrated, Communication and Computation:, Proceedings ASPLOS-V, May 1992, pp. 256–266.
C. Dubnicki, A. Bilas, Y. Chen, S. Damianakis, and Kai Li,“VMMC-2: Efficient Support for Reliable, Connection-Oriented Communication”, Hot Interconnects V, August 1997.
L. Prylli, B. Tourancheau,“BIP: A new protocol designed for high performance networking on Myrinet”, Workshop PC-NOW IPPS/SPDP98, 1998.
M. Litzkow, M. Livny, M. Mutka, “Condor-a hunter of idle workstations”, “Proc. of the 8th International Conference on Distributed Computing Systems”, April 1988, pp. 104–111.
A. Grimshaw, W. Wulf, J. French, A. Weaver and P. Reynolds Jr., “Legion: The next logical step toward a nationwide virtual computer”, Technical Report CS-94-21, Department of Computer Science, University of Virginia, 1994.
J. Morris, M. Satyanarayanan, M. Conner, J. Howard, D. Rosenthal and F. Smith,“, Andrew: A distributed personal computing environment”, Communications of the ACM, 1986, pp. 184–201.
I. Foster, C. Kesselman, “Globus: A Metacomputing Infrastructure Toolkit”, Intl J. Supercomputer Applications, 11(2):115–128, 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nallipogu, E., Özgüner, F., Lauria, M. (2002). Improving the Throughput of Remote Storage Access through Pipelining. In: Parashar, M. (eds) Grid Computing — GRID 2002. GRID 2002. Lecture Notes in Computer Science, vol 2536. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36133-2_28
Download citation
DOI: https://doi.org/10.1007/3-540-36133-2_28
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00133-1
Online ISBN: 978-3-540-36133-6
eBook Packages: Springer Book Archive