Abstract
Recent studies have shown that processes in real applications can arrive at the collective calls at different times. This imbalanced process arrival pattern can significantly affect the performance of the collective operations. MPI_Alltoall() is a communication-intensive collective operation that is used in many parallel scientific applications. Its efficient implementation under different process arrival patterns is critical to the performance of applications that use them frequently. In this paper, we propose novel RDMA-based process arrival pattern aware MPI_Alltoall() algorithms over InfiniBand clusters. We extend the algorithms to be shared memory aware for small to medium size messages. The micro-benchmark and application results indicate that the proposed algorithms outperform the native implementation as well as their non-process arrival pattern aware counterparts when processes arrive at different times.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
MPI: A Message Passing Interface standard (1997)
Faraj, A., Patarasuk, P., Yuan, X.: A Study of Process Arrival Patterns for MPI Collective Operations. International Journal of Parallel Programming 36(6), 543–570 (2008)
Patarasuk, P., Yuan, X.: Efficient MPI_Bcast across Different Process Arrival Patterns. In: 22nd International Parallel and Distributed Processing Symposium, IPDPS (2008)
InfiniBand Architecture, http://www.infinibandta.org
MVAPICH, http://mvapich.cse.ohio-state.edu
Mellanox Technologies, http://www.mellanox.com
Bruck, J., Ho, C.-T., Kipnis, S., Upfal, E., Weathersby, D.: Efficient Algorithms for All-to-all Communications in Multiport Message-passing Systems. IEEE Transactions on Parallel and Distributed Systems 8(11), 1143–1156 (1997)
Thakur, R., Rabenseifner, R., Gropp, W.: Optimization of Collective Communication Operations in MPICH. International Journal of High Performance Computing Applications 19(1), 49–66 (2005)
Qian, Y., Afsahi, A.: Efficient Shared Memory and RDMA based Collectives on Multi-rail QsNetII SMP Clusters. Cluster Computing, Journal of Networks, Software Tools and Applications 11(4), 341–354 (2008)
OpenFabrics Alliance Homepage, http://www.openfabrics.org
NAS Benchmarks, version 2.4, http://www.nas.nasa.gov/Resources/Software/npb.html
Sur, S., Jin, H.-W., Panda, D.K.: Efficient and Scalable All-to-all Personalized Exchange for InfiniBand Clusters. In: 33rd International Conference on Parallel Processing (ICCP), pp. 275–282 (2004)
Buntinas, D., Mercier, G., Gropp, W.: Data Transfers Between Processes in an SMP System: Performance Study and Application to MPI. In: 35th International Conference on Parallel Processing (ICPP), pp. 487–496 (2006)
Tipparaju, V., Nieplocha, J., Panda, D.K.: Fast Collective Operations using Shared and Remote Memory Access Protocols on Clusters. In: 17th International Parallel and Distributed Processing Symposium, IPDPS (2003)
Qian, Y., Rashti, M.J., Afsahi, A.: Multi-connection and Multi-core Aware All-Gather on InfiniBand Clusters. In: 20th IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS), pp. 245–251 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Qian, Y., Afsahi, A. (2009). Process Arrival Pattern and Shared Memory Aware Alltoall on InfiniBand. In: Ropo, M., Westerholm, J., Dongarra, J. (eds) Recent Advances in Parallel Virtual Machine and Message Passing Interface. EuroPVM/MPI 2009. Lecture Notes in Computer Science, vol 5759. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03770-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-03770-2_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03769-6
Online ISBN: 978-3-642-03770-2
eBook Packages: Computer ScienceComputer Science (R0)