Abstract
The one-sided communication operations in MPI areintended to provide the convenience of directly accessing remote memory and the potential for higher performance than regular point-to-point communication. Our performance measurements with three MPI implementations (IBM MPI, Sun MPI, and LAM) indicate, however, that one-sided communication can perform much worse than point-to-point communication if the associated synchronization calls are not implemented efficiently. In this paper, we describe our efforts to minimize the overhead of synchronization in our implementation of one-sided communication in MPICH-2. We describe our optimizations for all three synchronization mechanisms defined in MPI: fence, post-start-complete-wait, and lock-unlock. Our performance results demonstrate that, for short messages, MPICH-2 performs six times faster than LAM for fence synchronization and 50% faster for post-start-complete-wait synchronization, and it performs more than twice as fast as Sun MPI for all three synchronization methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Asai, N., Kentemich, T., Lagier, P.: MPI-2 implementation on Fujitsu generic message passing kernel. In: Proceedings of SC99: High Performance Networking and Computing (November 1999)
Ashton, D., Gropp, W., Thakur, R., Toonen, B.: The CH3 design for a simple implementation of ADI-3 for MPICH-2 with a TCP-based implementation. Technical Report ANL/MCS-P1156-0504, Mathematics and Computer Science Division, Argonne National Laboratory (May 2004)
Booth, S., Mourão, E.: Single sided MPI implementations for SUN MPI. In: Proceedings of SC 2000: High Performance Networking and Computing (November 2000)
Gabriel, E., Fagg, G.E., Dongarra, J.J.: Evaluating the performance of MPI-2 dynamic communicators and one-sided communication. In: Dongarra, J., Laforenza, D., Orlando, S. (eds.) EuroPVM/MPI 2003. LNCS, vol. 2840, pp. 88–97. Springer, Heidelberg (2003)
Hill, J.M.D., McColl, B., Stefanescu, D.C., Goudreau, M.W., Lang, K., Rao, S.B., Suel, T., Tsantilas, T., Bisseling, R.H.: BSPlib: The BSP programming library. Parallel Computing 24(14), 1947–1980 (1998)
Cray Research Inc. Cray T3E C and C++ optimization guide (1994)
Jiang, W., Liu, J., Jin, H.-W., Panda, D.K., Gropp, W., Thakur, R.: High performance MPI-2 one-sided communication over InfiniBand. In: Proc. of 4th IEEE/ACM Int’l Symp. on Cluster Computing and the Grid (April 2004)
LAM/MPI Parallel Computing, http://www.lam-mpi.org
Message Passing Interface Forum. MPI-2: Extensions to the Message-Passing Interface (July 1997), http://www.mpi-forum.org/docs/docs.html
Mourão, E., Booth, S.: Single sided communications in multi-protocol MPI. In: Dongarra, J., Kacsuk, P., Podhorszki, N. (eds.) PVM/MPI 2000. LNCS, vol. 1908, pp. 176–183. Springer, Heidelberg (2000)
Mourão, F.E., Gabriel Silva, J.: Implementing MPI’s one-sided communications for WMPI. In: Margalef, T., Dongarra, J., Luque, E. (eds.) PVM/MPI 1999. LNCS, vol. 1697, pp. 231–238. Springer, Heidelberg (1999)
Nieplocha, J., Harrison, R.J., Littlefield, R.J.: Global Arrays: A non-uniform-memory-access programming model for high-performance computers. The Journal of Supercomputing 10(2), 169–189 (1996)
Träff, J.L., Ritzdorf, H., Hempel, R.: The implementation of MPI-2 one-sided communication for the NEC SX-5. In: Proceedings of SC 2000: High Performance Networking and Computing (November 2000)
Worringen, J., Gäer, A., Reker, F.: Exploiting transparent remote memory access for non-contiguous and one-sided-communication. In: Proceedings of the 2002 Workshop on Communication Architecture for Clusters (CAC) (April 2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Thakur, R., Gropp, W.D., Toonen, B. (2004). Minimizing Synchronization Overhead in the Implementation of MPI One-Sided Communication. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2004. Lecture Notes in Computer Science, vol 3241. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30218-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-540-30218-6_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23163-9
Online ISBN: 978-3-540-30218-6
eBook Packages: Springer Book Archive