Abstract
Concurrency and parallelism have long been viewed as important, but somewhat distinct concepts. While concurrency is extensively used to amortize latency (for example, in web- and database-servers, user interfaces, etc.), parallelism is traditionally used to enhance performance through execution on multiple functional units. Motivated by an evolving application mix and trends in hardware architecture, there has been a push toward integrating traditional programming models for concurrency and parallelism. Use of conventional threads APIs (POSIX, OpenMP) with messaging libraries (MPI), however, leads to significant programmability concerns, owing primarily to their disparate programming models. In this paper, we describe a novel API and associated runtime for concurrent programming, called MPI Threads (MPIT), which provides a portable and reliable abstraction of low-level threading facilities. We describe various design decisions in MPIT, their underlying motivation, and associated semantics. We provide performance measurements for our prototype implementation to quantify overheads associated with various operations. Finally, we discuss two real-world use cases: an asynchronous message queue and a parallel information retrieval system. We demonstrate that MPIT provides a versatile, low overhead programming model that can be leveraged to program large parallel ensembles.
Similar content being viewed by others
References
Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17:734–749
Alam SR, Kuehn JA, Barrett RF, Larkin JM, Fahey MR, Sankaran R, Worley PH (2007) Cray XT4: an early evaluation for petascale scientific simulation. In: Proc SC’07. ACM, New York, pp 39:1–39:12
Balaji P, Buntinas D, Goodell D, Gropp W, Thakur R (2010) Fine-grained multithreading support for hybrid threadd MPI programming. Int J High Perform Comput Appl 24:49–57
Ben-Ari M (2006) Principles of concurrent and distributed programming, 2nd edn. Addison-Wesley, Reading
Berka T, Hagenauer H, Vajteršic M (2011) Portable explicit threading and concurrent programming for MPI applications. In: Proc PPAM. Springer, Berlin, pp 36–47
Berka T, Vajteršic M (2011) Parallel retrieval of dense vectors in the vector space model. CAI 30(2)
Dagum L, Menon R (1998) OpenMP: an industry standard API for shared-memory programming. IEEE Comput Sci Eng 5(1):46–55
Deerwester SC, Dumais ST, Landauer TK, Furnas GW, Harshman RA (1990) Indexing by latent semantic analysis. JSIS 41(6):391–407
Duran A, Gonzàlez M, Corbalán J (2005) Automatic thread distribution for nested parallelism in OpenMP. In: Proc ICS’05. ACM, New York, pp 121–130
Goudy S (2005) A preliminary analysis of the MPI queue characteristics of several applications. In: Proc ICPP. IEEE Press, New York, pp 175–183. doi:10.1109/ICPP.2005.13
Graham RL, Bosilca G (2009) MPI forum: preview of the MPI 3 standard
Hoefler T, Kambadur P, Graham RL, Shipman G, Lumsdaine A (2007) A case for standard non-blocking collective operations. In: Proc Euro. PVM/MPI, France
Hybrid Programming Working Group (2011) MPI3: the behavior of MPI processes sharing memory, available at http://lists.mpi-forum.org/mpi3-hybridpm/2011/10/0414.php (last retrieved November 22, 2011)
Jolliffe IT (2002) Principal component analysis, 2nd edn. Springer, Berlin
Kobayashi M, Aono M, Takeuchi H, Samukawa H (2002) Matrix computations for information retrieval and major and outlier cluster detection. JCAM 149(1):119–129
Lusk E, Chan A (2008) Early experiments with the OpenMP/MPI hybrid programming model. In: Proc IWOMP. Springer, Berlin, pp 36–47
Message Passing Interface Forum (2009) MPI: a message-passing interface standard, version 2.2
OpenMP Architecture Review Board (2008) The OpenMP API specification for parallel programming
Orlando S, Perego R, Silvestri F (2004) Design of a parallel and distributed web search engine. ArXiv Computer Science e-prints
Saad Y (2011) Numerical methods for large eigenvalue problems, 2nd edn. SIAM, Philadelphia
Smith L, Bull M (2001) Development of mixed mode MPI/OpenMP applications. SPJ 9:83–98
Thakur R, Gropp W (2009) Test suite for evaluating performance of multithreaded MPI communication. Parallel Comput 35:608–617
The Austin Group, The Open Group, The IEEE (2009) ISO/IEC/IEEE 9945:2009 information technology—portable operating system interface (POSIX©) base specifications, issue 7
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Berka, T., Kollias, G., Hagenauer, H. et al. Concurrent programming constructs for parallel MPI applications. J Supercomput 63, 385–406 (2013). https://doi.org/10.1007/s11227-011-0739-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-011-0739-5