Skip to main content

Advertisement

Log in

Reduced Function Set Abstraction (RFSA) for MPI-IO

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Programmers productivity has always been overlooked as compared to the performance optimizations in high performance computing (HPC) community. In many parallel programming languages like MPI/MPI-IO, the performance optimizations are provided as various function options to the programmers. In order to write an efficient code, they are required to know the exact usage of the optimization functions, hence programmer productivity is limited. In this paper, we present RFSA, a Reduced Function Set Abstraction based on an existing parallel programming interface (MPI-IO) for I/O. The purpose of RFSA is to hide the performance optimization functions from the application developer, and relieve the application developer from deciding on a specific function. The proposed set of functions rely on a selection algorithm to decide among the most common optimizations provided by MPI-IO. We implement a selection algorithm for I/O functions like read, write, etc., and also merge a set of functions for data types and file views. By running different parallel I/O benchmarks on both medium-scale clusters and NERSC supercomputers, we show an improved programmer productivity (35.7% on average). This approach incurs an overhead of 2–5% for one particular optimization, and shows performance improvement of 17% when a combination of different optimizations is required by an application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Cantonnet F, Yao Y, Zahran M, El-Ghazawi T (2004) Productivity analysis of the UPC language. In: 18th IEEE international parallel and distributed processing symposium (IPDPS’04)

  2. Ching A, Feng W-C, Lin H, Ma X, Choudhary A (2006) Exploring I/O strategies for parallel sequence database search tools with S3aSim. In: 15th IEEE international symposium on high-performance distributed computing (HPDC’06), Paris, France, June 2006

  3. García Blas J, Isaila F, Singh DE, Carretero J (2008) View-based collective i/o for mpi-io. In: CCGRID ’08: proceedings of the 2008 eighth IEEE international symposium on cluster computing and the grid, Washington, DC, USA, 2008. IEEE Computer Society, Los Alamitos, pp 409–416

    Chapter  Google Scholar 

  4. Gropp WD (2004) MPI and High Productivity Programming. In: PVM/MPI, p 7

  5. Gropp W, Thakur R, Lusk E (1999) Using MPI-2: advanced features of the message passing interface

  6. Hochstein L, Carver J, Shull F, Asgari S, Basili V (2005) Parallel programmer productivity: a case study of novice parallel programmers. In: SC ’05: proceedings of the 2005 ACM/IEEE conference on supercomputing, Washington, DC, USA, 2005. IEEE Computer Society, Los Alamitos, p 35

    Google Scholar 

  7. Latham R, Ross R (2003) Noncontiguous MPI-IO performance on PVFS

  8. Lustre Filesystem (2010) http://www.lustre.org/

  9. MPI-2: Extensions to the message-passing interface. http://parallel.ru/docs/parallel/mpi2, July (1997)

  10. National Energy Research Scientific Computing Center (2010) http://www.nersc.gov/

  11. Parallel Virtual File System version 2. (2010) http://www.pvfs.org/

  12. Patel I, Gilbert JR (2008) An empirical study of the performance and productivity of two parallel programming models. In: IPDPS, 2008, pp 1–7

  13. Sebesta RW (2002) Concepts of programming languages, 5th edn. Addison-Wesley, Reading

    MATH  Google Scholar 

  14. Spetka S, Hadzimujic H, Peek S, Flynn C (2008) High productivity languages for parallel programming compared to mpi. In: HPCMP users group conference, 2008, pp 413–417

  15. Thakur R, Gropp W, Lusk E (1999) Data sieving and collective I/O in ROMIO. In: FRONTIERS ’99: proceedings of the 7th symposium on the frontiers of massively parallel computation, Washington, DC, USA, 1999. IEEE Computer Society, Los Alamitos, p 182

    Chapter  Google Scholar 

  16. Thakur R, Gropp W, Lusk E (1999) On implementing MPI-IO portably and with high performance. In: IOPADS ’99: proceedings of the sixth workshop on I/O in parallel and distributed systems, New York, NY, USA, 1999. ACM Press, New York, pp 23–32

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Saba Sehrish.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sehrish, S., Wang, J. Reduced Function Set Abstraction (RFSA) for MPI-IO. J Supercomput 59, 131–146 (2012). https://doi.org/10.1007/s11227-010-0429-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-010-0429-8

Keywords