Abstract
OpenSHMEM is a library interface specification which has resulted from a unification effort among various vendors and users of SHMEM libraries. OpenSHMEM includes routines which aim to support a PGAS programming model, encompassing data management, one-sided communication, atomics, synchronization, collectives, and mutual exclusion. In the work described in this paper, we investigated the usage and performance of strided communication routines. Moreover, we propose and describe an implementation for new strided communication routines, shmem_iputmem and shmem_igetmem, which enable a more general means for expressing communications entailing data transfers for two-dimensional subarrays or for arrays of structures. We demonstrate the use of these routines on a halo exchange benchmark for which we achieved, on average, a 64.27 % improvement compared to the baseline implementation using non-strided communication routines and also 63.37 % improvement compared to the one using existing strided communication routines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cray - Message Passing Toolkit. http://docs.cray.com/books/004-3689-001/html-004-3689-001/zfixedsllc0bet.html
HPCTools PGAS-Microbenchmarks. https://github.com/uhhpctools/pgas-microbench
Mellanox ScalableSHMEM: Support the OpenSHMEM Parallel Programming Language over InfiniBand. http://www.mellanox.com/related-docs/prod_software/PB_ScalableSHMEM.pdf
OpenMPI: Open Source High Performance Computing. http://www.open-mpi.org/
SGI - Message Passing Toolkit. http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=0650&db=bks&srch=&fname=/SGI_Developer/MPT_MPI_PM/sgi_html/front.html
Unified MPI+PGAS Communication Runtime over OpenFabrics/Gen2 for Exascale Systems. http://mvapich.cse.ohio-state.edu/
Almasi, G., Padua, D.A.: Encyclopedia of Parallel Computing. Springer, cambridge (2011)
Bonachea, D.: GASNet Specification, V1.1. Technical report (2002)
Chapman, B., Curtis, T., Pophale, S., Poole, S., Kuehn, J., Koelbel, C., Smith, L.: Introducing OpenSHMEM: SHMEM for the PGAS community. In: Proceedings of the Fourth Conference on Partitioned Global Address Space Programming Model, PGAS 2010. ACM (2010)
Hammond, J.R.: Towards a matrix-oriented strided interface in OpenSHMEM. In: Proceedings of the 8th International Conference on Partitioned Global Address Space Programming Models, PGAS 2014 (2014)
Hoefler, T., Gottlieb, S.: Parallel zero-copy algorithms for fast fourier transform and conjugate gradient using MPI datatypes. In: Keller, R., Gabriel, E., Resch, M., Dongarra, J. (eds.) EuroMPI 2010. LNCS, vol. 6305, pp. 132–141. Springer, Heidelberg (2010)
Nieplocha, J., Carpenter, B.: ARMCI: A portable remote memory copy library for distributed array libraries and compiler run-time systems. In: Rolim, J.D.P. (ed.) IPPS-WS 1999 and SPDP-WS 1999. LNCS, vol. 1586. Springer, Heidelberg (1999)
Nieplocha, J., Tipparaju, V., Krishnan, M.: Optimizing strided remote memory access operations on the quadrics QsNetII network interconnect. In: Proceedings. Eighth International Conference on High-Performance Computing in Asia-Pacific Region, pp. 28–35, July 2005
Nieplocha, J., Tipparaju, V., Krishnan, M., Panda, D.K.: High performance remote memory access communication: the ARMCI approach. Int. J. High Perform. Comput. Appl. 20, 233–253 (2006)
Numrich, R.W., Reid, J.: Co-array fortran for parallel programming. SIGPLAN Fortran Forum 17(2), 1–13 (1998)
Pophale, S.S.: SRC: OpenSHMEM library development. In: Proceedings of the International Conference on Supercomputing, ICS 2011, pp. 374–374 (2011)
Santhanaraman, G., Wu, J., Panda, D.K.: Zero-Copy MPI derived datatype communication over InfiniBand. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J. (eds.) EuroPVM/MPI 2004. LNCS, vol. 3241, pp. 47–56. Springer, Heidelberg (2004)
Shamis, P., Venkata, M.G., Poole, S., Welch, A., Curtis, T.: Designing a high performance openshmem implementation using universal common communication substrate as a communication middleware. In: Poole, S., Hernandez, O., Shamis, P. (eds.) OpenSHMEM 2014. LNCS, vol. 8356, pp. 1–13. Springer, Heidelberg (2014)
ten Bruggencate, M., Roweth, D.: DMAPP: An API for One-Sided Programming Model on Baker Systems. Technical report, Cray Users Group (CUG) (2010)
Wu, J., Wyckoff, P., Panda, D.: High performance implementation of MPI derived datatype communication over InfiniBand. In: 2004 Proceedings of 18th International Parallel and Distributed Processing Symposium, April 2004
Acknowledgements
This work is supported through funding from Total. This research used resources of the Oak Ridge Leadership Computing Facility (OLCF) at the Oak Ridge National Laboratory, the Texas Advanced Computing Center (TACC) at the University of Texas at Austin and Total.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Namashivayam, N., Khaldi, D., Eachempati, D., Chapman, B. (2015). Extending the Strided Communication Interface in OpenSHMEM. In: Gorentla Venkata, M., Shamis, P., Imam, N., Lopez, M. (eds) OpenSHMEM and Related Technologies. Experiences, Implementations, and Technologies. OpenSHMEM 2014. Lecture Notes in Computer Science(), vol 9397. Springer, Cham. https://doi.org/10.1007/978-3-319-26428-8_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-26428-8_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26427-1
Online ISBN: 978-3-319-26428-8
eBook Packages: Computer ScienceComputer Science (R0)