Skip to main content
SpringerLink
Log in

High performance parallel I/O schemes for irregular applications on clusters of workstations

  • Workshop: High Performance Computing on Very Large Datasets
  • Conference paper
  • First Online:
Book cover High-Performance Computing and Networking (HPCN-Europe 1999)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1593))

Included in the following conference series:

  • 112 Accesses

Abstract

Due to the convergence of the fast microprocessors with low latency and high bandwidth communication networks, clusters of workstations are being used for high-performance computing. In this paper we present the design and implementation of a runtime system to support irregular applications on cluster of workstations, called “Collective I/O Clustering”. The system provides a friendly programming model for performing I/O in irregular applications on clusters of workstations, and is completely integrated with the underlying communication and I/O system. All the performance results were obtained on the IBM-SP machine, located at Argonne National Labs.

This work was supported in part by Sandia National Labs award AV-6193 under the ASCI program, and in part by NSF Young Investigator Award CCR-9357840 and NSF CCR-9509143.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Fern E. Bassow. Installing, managing, and using the IBM AIX Parallel I/O File System. IBM Document Number SH34-6065-00, February 1995. IBM Kingston, NY.

    Google Scholar 

  2. Rajesh Bordawekar, Juan Miguel del Rosario, and Alok Choudhary. Design and evaluation of primitives for parallel I/O. In Proceedings of Supercomputing '93, pages 452–461, 1993.

    Google Scholar 

  3. Alok Choudhary, Rajesh Bordawekar, Michael Harry, Rakesh Krishnaiyer, Ravi Ponnusamy, Tarvinder Singh, and Rajeev Thakur. PASSION: parallel and scalable software for input-output. Technical Report SCCS-636, ECE Dept. NPAC and CASE Center, Syracuse University, September 1994.

    Google Scholar 

  4. Peter F. Corbett, Sandra Johnson Baylor, and Dror G. Feitelson. Overview of the Vesta parallel file system. In IPPS '93 Workshop on Input/Output in Parallel Computer Systems, pages 1–16, 1993. Also published in Computer Architecture New 21(5), December 1993, pages 7–14.

    Google Scholar 

  5. Thomas H. Cormen and David Kotz. Integrating theory and practice in practice in parallel file systems. Technical Report PCS-TR93-188, Dept. of Math and Computer Science, Dartmouth College, March 1993. Revised 9/20/94.

    Google Scholar 

  6. Raja Das, Mustafa Uysal, Joel Saltz, and Yuan-Shin Hwang. Communication optimizations for irregular scientific computations on distributed memory architectures. Journal of Parallel and Distributed Computing, 22(3):462–479, September 1994.

    Article  Google Scholar 

  7. Juan Miguel del Rosario and Alok Choudhary. High performance I/O for parallel computers: Problems and prospects. IEEE Computer, 27(3):59–68, March 1994.

    Google Scholar 

  8. Intel Corporation. Paragon User's Guide, June 1994.

    Google Scholar 

  9. National Center for Supercomputing Applications. HDF reference manual version 4.1. Technical report, University of Illinois, 1997.

    Google Scholar 

  10. James T. Poole. Preliminary survey of I/O intensive applications. Technical Report CCSF-38, Scalable I/O Initiative, Caltech Concurrent Supercomputing Facilities, Caltech, 1994.

    Google Scholar 

  11. K. E. Seamons and M. Winslett. A data management approach for handling large compressed arrays in high performance computing. In Proceedings of the Fifth Symposium on the Frontiers of Massively Parallel Computation, pages 119–128, February 1995.

    Google Scholar 

  12. Shamik D. Sharma, Ravi Ponnusamy, Bongki Moon Yuan shin Hwang, Raja Das, and Joel Saltz. Run-time and compile-time support for adaptive irregular problems. In Supercomputing 1994. IEEE Press, November 1994.

    Google Scholar 

  13. R. Thakur, A. Choudhary, R. Bordawekar, S. More, and S. Kudatipidi. Passion: Optimized I/O for Parallel Systems. IEEE Computer, June 1996.

    Google Scholar 

  14. T. A. Welch. A technique for high performance data compression. IEEE Computer, 17(6):8–19, June 1984.

    Google Scholar 

Download references

Author information

Author notes

  1. Jesús Carretero

    Present address: Arquitectura y Tecnología de Sistemas Informáticos, Universidad Politécnica de, Madrid, Spain

  2. Alok Choudhary

    Present address: Electrical and Computer Engineering, Northwestern University, USA

Authors and Affiliations

  1. Dept. of Electrical Engineering and Computer Science, Syracuse University, USA

    Jaechun No, Jesús Carretero & Alok Choudhary

Authors
  1. Jaechun No
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jesús Carretero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alok Choudhary
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Peter Sloot Marian Bubak Alfons Hoekstra Bob Hertzberger

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag

About this paper

Cite this paper

No, J., Carretero, J., Choudhary, A. (1999). High performance parallel I/O schemes for irregular applications on clusters of workstations. In: Sloot, P., Bubak, M., Hoekstra, A., Hertzberger, B. (eds) High-Performance Computing and Networking. HPCN-Europe 1999. Lecture Notes in Computer Science, vol 1593. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100672

Download citation

  • DOI: https://doi.org/10.1007/BFb0100672

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-65821-4

  • Online ISBN: 978-3-540-48933-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation