Abstract
Parallel computation applications output intermediate data periodically, and typically the outputs are moved to a remote computer for visualization. A flexible intermediate library named Stampi realizes seamless MPI-I/O operations both inside a computer and among computers. MPI-I/O operations to a remote computer are realized by its MPI-I/O processes which are invoked on a remote computer. To realize data-intensive I/O operations, a Parallel Virtual File System (PVFS) was supported in the MPI-I/O mechanism. MPI-I/O operations to a PVFS file system on a remote computer are available with seamless interfaces of the Stampi library. Among many kinds of I/O functions, nonblocking MPI-I/O functions provide overlap of computation with I/O operations, and visible I/O times can be minimized with them. Due to its architectural constraints and slow network, visible I/O times of them became long with an increase in the number of user processes and message data size. To minimize the times, a circular buffer system has been implemented in the mechanism. With the help of the circular buffer, the visible I/O times have been minimized effectively.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Message Passing Interface Forum: MPI: A message-passing interface standard (1995)
Message Passing Interface Forum: MPI-2: Extensions to the Message-Passing Interface (1997)
Imamura, T., Tsujita, Y., Koide, H., Takemiya, H.: An architecture of stampi: MPI library on a cluster of parallel computers. In: Dongarra, J., Kacsuk, P., Podhorszki, N. (eds.) PVM/MPI 2000. LNCS, vol. 1908, pp. 200–207. Springer, Heidelberg (2000)
Tsujita, Y., Imamura, T., Takemiya, H., Yamagishi, N.: Stampi-I/O: A flexible parallel-I/O library for heterogeneous computing environment. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J., Volkert, J. (eds.) PVM/MPI 2002. LNCS, vol. 2474, pp. 288–295. Springer, Heidelberg (2002)
Carns, P.H., Ligon III, W.B., Ross, R.B., Thakur, R.: PVFS: A parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, USENIX Association, pp. 317–327 (2000)
PC Cluster Consortium, http://www.pccluster.org/
Matsuda, M., Kudoh, T., Ishikawa, Y.: Evaluation of MPI implementations on grid-connected clusters using an emulated WAN environment. In: Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2003), Tokyo, Japan, May 12-15, pp. 10–17. IEEE Computer Society, Los Alamitos (2003)
Gropp, W., Lusk, E., Doss, N., Skjellum, A.: A high-performance, portable implementation of the MPI Message-Passing Interface standard. Parallel Computing 22, 789–828 (1996)
Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pp. 23–32 (1999)
Thakur, R., Gropp, W., Lusk, E.: An abstract-device interface for implementing portable parallel-I/O interfaces. In: Proceedings of the Sixth Symposium on the Frontiers of Massively Parallel Computation, pp. 180–187 (1996)
Lee, J., Ma, X., Ross, R., Thakur, R., Winslett, M.: RFS: Efficient and flexible remote file access for MPI-IO. In: Proceedings of the 6th IEEE International Conference on Cluster Computing (CLUSTER 2004), pp. 71–81. IEEE Computer Society, Los Alamitos (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Tsujita, Y. (2005). Optimization of Nonblocking MPI-I/O to a Remote Parallel Virtual File System Using a Circular Buffer. In: Yang, L.T., Rana, O.F., Di Martino, B., Dongarra, J. (eds) High Performance Computing and Communications. HPCC 2005. Lecture Notes in Computer Science, vol 3726. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11557654_69
Download citation
DOI: https://doi.org/10.1007/11557654_69
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29031-5
Online ISBN: 978-3-540-32079-1
eBook Packages: Computer ScienceComputer Science (R0)