Abstract
Programmers productivity has always been overlooked as compared to the performance optimizations in high performance computing (HPC) community. In many parallel programming languages like MPI/MPI-IO, the performance optimizations are provided as various function options to the programmers. In order to write an efficient code, they are required to know the exact usage of the optimization functions, hence programmer productivity is limited. In this paper, we present RFSA, a Reduced Function Set Abstraction based on an existing parallel programming interface (MPI-IO) for I/O. The purpose of RFSA is to hide the performance optimization functions from the application developer, and relieve the application developer from deciding on a specific function. The proposed set of functions rely on a selection algorithm to decide among the most common optimizations provided by MPI-IO. We implement a selection algorithm for I/O functions like read, write, etc., and also merge a set of functions for data types and file views. By running different parallel I/O benchmarks on both medium-scale clusters and NERSC supercomputers, we show an improved programmer productivity (35.7% on average). This approach incurs an overhead of 2–5% for one particular optimization, and shows performance improvement of 17% when a combination of different optimizations is required by an application.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Cantonnet F, Yao Y, Zahran M, El-Ghazawi T (2004) Productivity analysis of the UPC language. In: 18th IEEE international parallel and distributed processing symposium (IPDPS’04)
Ching A, Feng W-C, Lin H, Ma X, Choudhary A (2006) Exploring I/O strategies for parallel sequence database search tools with S3aSim. In: 15th IEEE international symposium on high-performance distributed computing (HPDC’06), Paris, France, June 2006
García Blas J, Isaila F, Singh DE, Carretero J (2008) View-based collective i/o for mpi-io. In: CCGRID ’08: proceedings of the 2008 eighth IEEE international symposium on cluster computing and the grid, Washington, DC, USA, 2008. IEEE Computer Society, Los Alamitos, pp 409–416
Gropp WD (2004) MPI and High Productivity Programming. In: PVM/MPI, p 7
Gropp W, Thakur R, Lusk E (1999) Using MPI-2: advanced features of the message passing interface
Hochstein L, Carver J, Shull F, Asgari S, Basili V (2005) Parallel programmer productivity: a case study of novice parallel programmers. In: SC ’05: proceedings of the 2005 ACM/IEEE conference on supercomputing, Washington, DC, USA, 2005. IEEE Computer Society, Los Alamitos, p 35
Latham R, Ross R (2003) Noncontiguous MPI-IO performance on PVFS
Lustre Filesystem (2010) http://www.lustre.org/
MPI-2: Extensions to the message-passing interface. http://parallel.ru/docs/parallel/mpi2, July (1997)
National Energy Research Scientific Computing Center (2010) http://www.nersc.gov/
Parallel Virtual File System version 2. (2010) http://www.pvfs.org/
Patel I, Gilbert JR (2008) An empirical study of the performance and productivity of two parallel programming models. In: IPDPS, 2008, pp 1–7
Sebesta RW (2002) Concepts of programming languages, 5th edn. Addison-Wesley, Reading
Spetka S, Hadzimujic H, Peek S, Flynn C (2008) High productivity languages for parallel programming compared to mpi. In: HPCMP users group conference, 2008, pp 413–417
Thakur R, Gropp W, Lusk E (1999) Data sieving and collective I/O in ROMIO. In: FRONTIERS ’99: proceedings of the 7th symposium on the frontiers of massively parallel computation, Washington, DC, USA, 1999. IEEE Computer Society, Los Alamitos, p 182
Thakur R, Gropp W, Lusk E (1999) On implementing MPI-IO portably and with high performance. In: IOPADS ’99: proceedings of the sixth workshop on I/O in parallel and distributed systems, New York, NY, USA, 1999. ACM Press, New York, pp 23–32
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sehrish, S., Wang, J. Reduced Function Set Abstraction (RFSA) for MPI-IO. J Supercomput 59, 131–146 (2012). https://doi.org/10.1007/s11227-010-0429-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-010-0429-8