Abstract
This paper presents an implementation of the MPI-IO interface for GPFS inside ROMIO distribution. The experimental section presents a performance comparison among three collective I/O implementations: two-phase I/O, the default file system independent method of ROMIO, view-based I/O, a file system-independent method we developed in a previous work and a GPFS specific collective I/O implementation based on data-shipping. The results show that data shipping-based collective I/O performs better for writing, while view-based I/O for reading.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ligon, W., Ross, R.: An Overview of the Parallel Virtual File System. In: Proceedings of the Extreme Linux Workshop (June 1999)
Inc., C.F.S.: Lustre: A scalable, high-performance file system. Cluster File Systems Inc. white paper, version 1.0 (November 2002), http://www.lustre.org/docs/whitepaper.pdf
Schmuck, F., Haskin, R.: GPFS: A Shared-Disk File System for Large Computing Clusters. In: Proceedings of FAST (2002)
Message Passing Interface Forum: MPI: A Message-Passing Interface Standard (1995)
Corbett, P., Feitelson, D., Hsu, Y., Prost, J.P., Snir, M., Fineberg, S., Nitzberg, B., Traversat, B., Wong, P.: MPI-IO: A parallel file I/O interface for MPI. Technical Report NAS-95-002, NASA Ames Research Center, Moffet Field, CA (January 1995)
Prost, J.P., Treumann, R., Hedges, R., Jia, B., Koniges, A.: MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS. In: Supercomputing 2001: Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM), pp. 17–17. ACM Press, New York (2001)
Kotz, D.: Disk-directed I/O for MIMD Multiprocessors. In: Proc. of the First USENIX Symp. on Operating Systems Design and Implementation (1994)
del Rosario, J., Bordawekar, R., Choudhary, A.: Improved parallel I/O via a two-phase run-time access strategy. In: Proc. of IPPS Workshop on Input/Output in Parallel Computer Systems (1993)
Yu, W., Vetter, J., Canon, R.S., Jiang, S.: Exploiting Lustre File Joining for Effective Collective IO. In: CCGRID 2007: Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid, pp. 267–274. IEEE Computer Society Press, Washington (2007)
Thakur, R., Gropp, W., Lusk, E.: Optimizing Noncontiguous Accesses in MPI-IO. Parallel Computing 28(1), 83–105 (2002)
keng Liao, W., Coloma, K., Choudhary, A., Ward, L., Russel, E., Tideman, S.: Collective Caching: Application-Aware Client-Side File Caching. In: Proceedings of the 14th International Symposium on High Performance Distributed Computing (HPDC) (July 2005)
keng Liao, W., Coloma, K., Choudhary, A.N., Ward, L.: Cooperative Write-Behind Data Buffering for MPI I/O. In: PVM/MPI, pp. 102–109 (2005)
Isaila, F., Malpohl, G., Olaru, V., Szeder, G., Tichy, W.: Integrating Collective I/O and Cooperative Caching into the Clusterfile Parallel File System. In: Proceedings of ACM International Conference on Supercomputing (ICS), pp. 315–324. ACM Press, New York (2004)
Ching, A., Choudhary, A., Liao, W.K., Ross, R., Gropp, W.: Efficient Structured Data Access in Parallel File Systems. In: Proceedings of the IEEE International Conference on Cluster Computing (December 2003)
Thakur, R., Gropp, W., Lusk, E.: An abstract device interface for implementing portable paralllel-I/O interfaces
Blas, J.G., Florin Isaila, D.E.S., Carretero, J.: View-based collective i/o for mpi-io. In: CCGRID 2008: Proceedings of the 8th IEEE International Symposium on Cluster Computing and the Grid (2008)
Wong, P., der Wijngaart, R.: NAS Parallel Benchmarks I/O Version 2.4. Technical Report NAS-03-002, NASA Ames Research Center, Moffet Field, CA (January 2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Blas, F.J.G., Isailă, F., Carretero, J., Großmann, T. (2008). Implementation and Evaluation of an MPI-IO Interface for GPFS in ROMIO. In: Lastovetsky, A., Kechadi, T., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2008. Lecture Notes in Computer Science, vol 5205. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87475-1_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-87475-1_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87474-4
Online ISBN: 978-3-540-87475-1
eBook Packages: Computer ScienceComputer Science (R0)