Abstract
The philosophy behind grid is to use idle resources to achieve a higher level of computational services (computation, storage, etc). Existing data grids solutions are based in new servers, specific APIs and protocols, however this approach is not a realistic solution for enterprises and universities, because this supposes the deployment of new data servers across the company. This paper describes a new approach to data access in computational grids. This approach is called GridExpand, a parallel I/O middleware that integrates heterogeneous data storage resources in grids. The proposed grid solution integrates available data network solutions (NFS, CIFS, WebDAV) and makes possible the access to a global grid file system. Our solution differs from others because it does not need the installation of new data servers with new protocols. Most of the data grid solutions use replication as the way to obtain high performance. Replication, however, introduce consistency problem for many collaborative applications, and sometimes requires the usage of lots of resources. To obtain high performance, we apply the parallel I/O techniques used in parallel file systems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baru, C., Moore, R., Rajasekar, A., Wan, M.: The SDSC Storage Resource Broker. In: Proceedings of the International Conference in High Energy and Nuclear Phisycs Teatro Antonianum, Padova, Italia (February 2002)
Beynon, M.D., Ferreira, R., Kurc, T., Sussman, A., Saltz, J.: DataCutter: Middleware for filtering very large scientific datasets of archival storage systems. In: Proceedings of the 2000 Mass Storage Systems Conference, College Park, MD, March 2000, pp. 119–133. IEEE Computer Society Press, Los Alamitos (2000)
Calderon, A., Garcia, F., Carretero, J., Perez, J.M., Fernandez, J.: An Implementation of MPIIO on Expand: A Parallel File System Based on NFS Servers. In: 9th PVM/MPI European Userś Group, Johannes Kepler University Linz, Austria, September 29-October 2, pp. 306–313 (2002)
Carns, P.H., Ligon III, W.B., Ross, R.B., Takhur, R.: PVFS: A Parallel File System for Linux Clusters. Tech. Rep. ANL/MCS-P804-0400 (2000)
Carretero, J., Perez, F., de Miguel, P., Garcia, F., Alonso, L.: Performance Increase Mechanisms for Parallel and Distributed File Systems. Parallel Computing: Special Issue on Parallel I/O Systems (3), 525–542 (1997)
Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific. Journal of Network and Computer Applications 23, 187–200 (2001)
Corbett, P., Johnson, S., Feitelson, D.: Overview of the Vesta Parallel File System. ACM Computer Architecture News 21(5), 7–15 (1993)
Foster, I., Kesselman, C. (eds.): The Grid: Blueprint for a New Computing Infraestructure. Morgan Kaufmann, San Francisco (1999)
Garcia, F., Calderon, A., Carretero, J., Perez, J.M., Fernandez, J.: The Design of the Expand Parallel File System. Accepted for publication in the International Journal of High Performance Computing Applications (2003)
Martinez, M.R., Roussopoulos, N.: MOCHA: A Self-extensible Database Middleware System for Distributed Data Sources. In: Proceedings of the ACMSIGMOD International Conference on Management of Data, Dallas, TX (May 2000)
Moyer, S.A., Sunderam, V.S.: PIOUS: A Scalable Parallel I/O System for Distributed Computing Environments. In: Proceedings of the Scalable High-Performance Computing Conference, pp. 71–78 (1994)
Nieuwejaar, N., Kotz, D.: The Galley Parallel File System. In: Proceedings of the 10th ACM International Conference on Supercomputing (May 1996)
Olfield, R., Kotz, D.: The armada parallel file system (1998), http://www.cs.dartmouth.edu/~dfk/armada/design.html
Gropp, W., Takhur, R., Lusk, E.: An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces. In: Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation, October 1996, pp. 180–187 (1996)
Tierney, B., Lee, J., Johnston, W., Crowley, B., Holding, M.: Holding. A Network-aware Distributed Storage Cache for Data-intensive Environments. In: Proceedings of the Eighth IEEE International Symposium on High Performance Distributed Computing, pp. 185–193. Redondo Beach, CA (August 1999)
Vazhkuda, S., Tuecke, S., Foster, I.: Replica Selection in the Globus Data Grid. In: Proceedings of the International workshop on Data Models and Databases on Clusters and the Grid (DataGrid2001). IEEE Computer Society Press, Los Alamitos (2001)
WebDav Resources. Web-based Distributed Authoring and Versioning (1998), http://www.webdav.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pérez Menor, J.M., García, F., Carretero, J., Calderón, A., Fernández, J., García, J.D. (2004). A Parallel I/O Middleware to Integrate Heterogeneous Storage Resources on Grids. In: Fernández Rivera, F., Bubak, M., Gómez Tato, A., Doallo, R. (eds) Grid Computing. AxGrids 2003. Lecture Notes in Computer Science, vol 2970. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24689-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-540-24689-3_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21048-1
Online ISBN: 978-3-540-24689-3
eBook Packages: Springer Book Archive