Skip to main content
SpringerLink
Log in

Implementing a Parallel NetCDF Interface for Seamless Remote I/O Using Multi-dimensional Data

  • Conference paper
High Performance Computing for Computational Science - VECPAR 2008 (VECPAR 2008)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5336))

  • 1129 Accesses

Abstract

Parallel netCDF supports parallel I/O operations for a view of data as a collection of self-describing, portable, and array-oriented objects that can be accessed through a simple interface. Its parallel I/O operations are realized with the help of an MPI-I/O library. However, such the operations are not available in remote I/O operations. So, a remote I/O mechanism of a Stampi library was introduced in an MPI layer of the parallel netCDF to realize such the operations. This system was evaluated on two interconnected PC clusters, and sufficient performance was achieved with a huge amount of data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Rew, R.K., Davis, G.P.: The unidata netCDF: Software for scientific data access. In: Sixth International Conference on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, pp. 33–40. American Meteorology Society (February 1990)

    Google Scholar 

  2. Message Passing Interface Forum: MPI-2: Extensions to the Message-Passing Interface (July 1997)

    Google Scholar 

  3. Thakur, R., Gropp, W., Lusk, E.: On implementing MPI-IO portably and with high performance. In: Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pp. 23–32 (1999)

    Google Scholar 

  4. Li, J., Liao, W.K., Choudhary, A., Ross, R., Thakur, R., Gropp, W., Latham, R., Siegel, A., Gallagher, B., Zingale, M.: Parallel netCDF: A high-performance scientific I/O interface. In: SC 2003: Proceedings of the 2003 ACM/IEEE Conference on Supercomputing, p. 39. IEEE Computer Society, Los Alamitos (2003)

    Google Scholar 

  5. Tsujita, Y., Imamura, T., Takemiya, H., Yamagishi, N.: Stampi-I/O: A flexible parallel-I/O library for heterogeneous computing environment. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J., Volkert, J. (eds.) PVM/MPI 2002. LNCS, vol. 2474, pp. 288–295. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  6. Gropp, W., Lusk, E., Doss, N., Skjellum, A.: A high-performance, portable implementation of the MPI Message-Passing Interface standard. Parallel Computing 22(6), 789–828 (1996)

    Article  MATH  Google Scholar 

  7. PVFS2: http://www.pvfs.org/pvfs2/

  8. Rew, R., Davis, G., Emmerson, S., Davies, H., Hartnett, E.: NetCDF User’s Guide. Unidata Program Center (June 2006), http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/

  9. The National Center for Supercomputing Applications: http://hdf.ncsa.uiuc.edu/HDF5/

  10. Ross, R., Nurmi, D., Cheng, A., Zingale, M.: A case study in application I/O on Linux clusters. In: SC 2001: Proceedings of the 2001 ACM/IEEE Conference on Supercomputing, p. 11. ACM Press, New York (2001) (CDROM)

    Google Scholar 

  11. Thakur, R., Gropp, W., Lusk, E.: An abstract-device interface for implementing portable parallel-I/O interfaces. In: Proceedings of the Sixth Symposium on the Frontiers of Massively Parallel Computation, pp. 180–187 (1996)

    Google Scholar 

  12. Lee, J., Ma, X., Ross, R., Thakur, R., Winslett, M.: RFS: Efficient and flexible remote file access for MPI-IO. In: Proceedings of the 6th IEEE International Conference on Cluster Computing (CLUSTER 2004), pp. 71–81. IEEE Computer Society, Los Alamitos (2004)

    Google Scholar 

  13. Gabriel, E., Resch, M., Beisel, T., Keller, R.: Distributed computing in a heterogeneous computing environment. In: Alexandrov, V.N., Dongarra, J. (eds.) PVM/MPI 1998. LNCS, vol. 1497, pp. 180–187. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  14. GridMPI: http://www.gridmpi.org/

Download references

Author information

Authors and Affiliations

  1. Department of Electronic Engineering and Computer Science, School of Engineering, Kinki University, 1 Umenobe, Takaya, Higashi-Hiroshima, Hiroshima, 739-2116, Japan

    Yuichi Tsujita

Authors
  1. Yuichi Tsujita
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculdade de Engenharia da Universidade do Porto, Rua Dr. Roberto Frias s/n, 4200-465, Porto, Portugal

    José M. Laginha M. Palma

  2. University of Toulouse, INP (ENSEEIHT), IRIT, 2 rue Charles-Camichel, 31071, Toulouse CEDEX 7, France

    Patrick R. Amestoy

  3. University of Toulouse, INP (ENSEEIHT); IRIT, rue Charles-Camichel, 31071, Toulouse CEDEX 7, France

    Michel Daydé

  4. Federal University of Rio de Janeiro, P.O. Box 68511, 21941-972, Rio de Janeiro, RJ, Brazil

    Marta Mattoso

  5. Faculty of Engineering, University of Porto, Rua Dr. Roberto Frias, s/n, 4200-465, Porto, Portugal

    João Correia Lopes

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Tsujita, Y. (2008). Implementing a Parallel NetCDF Interface for Seamless Remote I/O Using Multi-dimensional Data. In: Palma, J.M.L.M., Amestoy, P.R., Daydé, M., Mattoso, M., Lopes, J.C. (eds) High Performance Computing for Computational Science - VECPAR 2008. VECPAR 2008. Lecture Notes in Computer Science, vol 5336. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-92859-1_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-92859-1_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-92858-4

  • Online ISBN: 978-3-540-92859-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation