Skip to main content
Springer Nature Link
Log in

Parallel I/O Support for HPF on Computational Grids

  • Conference paper
  • First Online:
High Performance Computing (ISHPC 2002)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2327))

Included in the following conference series:

  • 867 Accesses

Abstract

Recently several projects have started to implement large-scale high-performance computing on “computational grids” composed of heterogeneous and geographically distributed systems of computers, networks, and storage devices that collectively act as a single “virtual supercomuter”. One of the great challenges for this environment is to provide appropriate high-level programming models. High Performance Fortran (HPF) is a language of choice for development of data parallel components of Grid applications. Another challenge is to provide efficient access to data that is distributed across local and remote Grid resources. In this paper, constructs to specify parallel input and output (I/O) operations on multidimensional arrays on the Grid in the context of HPF are proposed. The paper also presents implementation concepts that are based on the HPF compiler VFC, the parallel I/O runtime system Panda, Internet, and Grid technologies. Preliminary experimental performance results are discussed in the context of a real application example.

The work described in this paper is being carried out as part of the research project “Aurora” supported by the Austrian Research Foundation, and was also supported by NASA under grant NAGW 4244 and by the US Department of Energy under grants B341494 to the Center for Simulation of Advanced Rockets and to the Center for Programming Models for Scalable Parallel Computing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bassow, F.:IBM AIX Parallel I/O File System: Installation, Administration, and Use. IBM, Doc. SH34-6065-00 (1995)

    Google Scholar 

  2. Benkner, S., Neuhold, Ch., Egger, M., Sanjari, K., Velkov, B.: VFC-The Vienna HPF+ Compiler. In: Proceedings of the International Conference on Compilers for Parallel Computers, Linkoping (1998)

    Google Scholar 

  3. Bester, J., Foster, I., Kesselman, C., Tedesco, J., Tuecke, S.: GASS: A Data Movement and Access Service for Wide Area Computing Systems. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems (1999)

    Google Scholar 

  4. Bordawekar, R.R., Choudhary, A.N.: Language and Compiler Support for Parallel I/O. In: Proceedings of the Working Conference on Programming Environments for Massively Parallel Distributed Systems, Switzerland (1994)

    Google Scholar 

  5. Brezany, P., Gerndt, M., Mehrotra, P., Zima, H.: Concurrent File Operations in a High Performance FORTRAN. In: Proceedings of Supercomputing’ 92 (1992) 230–237

    Google Scholar 

  6. Brezany, P., Czerwinski, P., Winslett, M.: A Generic Interface for Parallel Access to Large Data Sets from HPF Applications. Future Generation Computer Systems, 17 (2001) 977–985

    Article  MATH  Google Scholar 

  7. Brezany, P., Winslett, M.: Advanced Data Repository Support for Java Scientific Programming. In: HPCN Europe 1999, Lecture Notes in Computer Science, Vol. 1593, Springer-Verlag, Berlin Heidelberg New York (1999) 1127–1136

    Google Scholar 

  8. Carpenter, B., Fox, G.: HPJava: Data Parallel Extensions to Java. In: Proceedings of the ACM Workshop on Java for High-Performance Network Computing, Palo Alto (1998)

    Google Scholar 

  9. Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., Tuecke, S.:The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications (2001)

    Google Scholar 

  10. Choudhary, A. et. al.: PASSION: Parallel and Scalable Software for Input-Output. Technical Report SCCS-636, ECE Department, Syracuse University (1994)

    Google Scholar 

  11. Czajkowski, K., Fitzgerald, S., Foster, I., Kesselman, C.: Grid Information Services for Distributed Resource Sharing. In: Proceeding of the International Symposium on High Performance Distributed Computing (2001)

    Google Scholar 

  12. Denis, A., Perez, C., Priol, T.: Towards High Performance CORBA and MPI Middlewares for Grid Computing. In: Lee, C.A. (ed.) Grid Computing-GRID 2001, 2nd Int. Workshop, Denver, Lecture Notes in Computer Science, Vol. 2242, Springer Verlag, Berlin Heidelberg New York (2001)

    Google Scholar 

  13. Foster, I., Kesselman, C.: Globus: A Metacomputing Infrastructure Toolkit. International Journal on Supercomputer Applications, 2 (1997) 115–128

    Article  Google Scholar 

  14. Foster, I., Kesselman, C., Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. International Journal on Supercomp. Applications (2001)

    Google Scholar 

  15. Foster, I, Kohr, D., Krishnaiyer, R., Mogill, J.: Remote I/O: Fast Access to Distant Storage. In: Proceedings of the 5th Workshop on I/O in Parallel and Distributed Systems (1997) 14–25

    Google Scholar 

  16. Global Grid Forum. http://www.globalgridforum.org

  17. Globus Project. Globus Toolkit 2.0 Beta Release. http://www.globus.org/gt2/

  18. Grid Datafarm for Petascale Data Intensive Computing. http://datafarm.apgrid.org/

  19. Grimshaw, A., Wulf, W., French, J., Weaver, A., Reynolds, P.: Legion: The Next Step toward a Nationwide Virtual Computer. Technical Report CS-94-21, Department of Computer Science, University of Virginia (1994)

    Google Scholar 

  20. High Performance Fortran Forum: High Performance Fortran. Version 2.0 (1997)

    Google Scholar 

  21. HPC++. High-Performance C++. http://www.extreme.indiana.edu/hpc++/

  22. Huber, J., Elford, C.L., Reed, D.A., Chien, A.A., Bhune, S.S.: PPFS: A High-Performance Portable Parallel File System. In: Proceedings of the 9th ICS Conference, Barcelona (1995)

    Google Scholar 

  23. IETF: Rfc 2396. ftp://ftp.isi.edu/in-notes/rfc2396.txt

  24. Koppenhoefer, K., Gullerud, A., Ruggieri, C., Dodds, R. Jr.: WARP3D: Dynamic Nonlinear Analysis of Solids Using a Preconditioned Conjugate Gradient Software Architecture. Structural Research Series 596, University of Illinois (1994)

    Google Scholar 

  25. Laure, E.: Distributed High Performance Computing with OpusJava. In: Proceedings of the ParCo99 Conference, Delft (1999)

    Google Scholar 

  26. Lee, J.: Web-based Data Migration for High-performance Scientific Codes. MS thesis, Department of Computer Science, University of Illinois at Urbana-Champaign (1999)

    Google Scholar 

  27. Messina, P.: Distributed Supercomputing Applications. In: Foster, I. Kesselman, C. (eds.), The Grid. Blueprint for a New Computing Infrastructure. Morgan Kaufmann (1999)

    Google Scholar 

  28. Metcalf, M., Reid, C.: Fortran 90/95 Explained. Oxford University Press (1996)

    Google Scholar 

  29. Moyer, S.A., Sunderam, V.S.: PIOUS: A Scalable Parallel I/O System for Distributed Computing Environment. In: Proceedings of the Scalable High-Performance Computing Conference (1994)

    Google Scholar 

  30. Nieplocha, J., Foster, I.: Disk Resident Arrays: An Array-Oriented I/O Library for Out-Of-Core Computations. In: Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation, IEEE Computer Society Press, October (1996) 196–204

    Google Scholar 

  31. Nieuwejaar, M., Kotz, D.: The Galley Parallel File System. Parallel Computing, North-Holland (Elsevier Scientific) 23 (1997) 447–476

    Article  MATH  Google Scholar 

  32. Numrich, R.W., Reid, J.K.: Co-Array Fortran for Parallel Programming. ACM Fortran Forum (1998)

    Google Scholar 

  33. Oldfield, R.: Summary of Existing and Developing Data Grids. White paper for the Remote Data Access group of the Global Grid Forum 1, Amsterdam (2001)

    Google Scholar 

  34. Oldfield, R., Kotz, D.: Armada: A Parallel File System for Computational Grids. In: Proceedings of the International Symposium on Cluster Computing and the Grid, Brisbane, Australia (2001) 194–201

    Google Scholar 

  35. OpenMP Consortium: OpenMP Fortran API, version 1.0 (1997)

    Google Scholar 

  36. Seamons, K.E., Winslett, M.: Multidimensional Array I/O in Panda 1.0. Journal of Supercomputing 10 (1995) 191–211

    Google Scholar 

  37. Segal, B.: Datagrid-Data Management. Deliverable DataGrid-D2.2 (2001)

    Google Scholar 

  38. Snir, M.: Proposal for I/O. Posted to HPFF I/O Forum (1992)

    Google Scholar 

  39. Stockinger, H.: Database Replication in World-Wide Distributed Data Grids. PhD Thesis, Institute of Computer Science and Business Informatics, University of Vienna, Austria (2001)

    Google Scholar 

  40. Thakur, R., Gropp, W., Lusk, E.: On Implementing MPI-IO Portably and with High Performance. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, ACM Press, (1999) 23–32

    Google Scholar 

  41. Weissman, J.: Smart File Objects: A Remote File Access Paradigm. In: Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems (1999)

    Google Scholar 

  42. Wolski, R., Spring, N., Hayes, J.: The Network Weather Service: A Distributed Resource Performance Forecasting Service for Metacomputing. Journal of Future Generation Computing Systems 15 (1999) 757–768

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Software Science, University of Vienna, Liechtensteinstrasse 22, A-1090, Vienna, Austria

    Peter Brezany

  2. Department of Computer Science, University of Illinois, Urbana, IL, 61801, USA

    Jonghyun Lee & Marianne Winslett

Authors
  1. Peter Brezany
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonghyun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marianne Winslett
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Software Science, University of Vienna, Liechtensteinstr. 22, 1090, Vienna, Austria

    Hans P. Zima

  2. Department of Information and Computer Science, Nara Women’s University, Kitauoyanishimachi, Nara City, 630-8506, Japan

    Kazuki Joe

  3. Institute of Information Science and Electronics, University of Tsukuba, Tenno-dai 1-1-1, Tsukuba, Ibaraki, 305-8577, Japan

    Mitsuhisa Sato

  4. Internet Systems Research Laboratories, NEC Corporation, 4-1-1, Miyazaki, Miyamae, Kawasaki, Kanagawa, 216-8555, Japan

    Yoshiki Seo

  5. Kyoto University, Yoshidahonmachi, Sakyo-ku, Kyoto, 606-8501, Japan

    Masaaki Shimasaki

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Brezany, P., Lee, J., Winslett, M. (2002). Parallel I/O Support for HPF on Computational Grids. In: Zima, H.P., Joe, K., Sato, M., Seo, Y., Shimasaki, M. (eds) High Performance Computing. ISHPC 2002. Lecture Notes in Computer Science, vol 2327. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47847-7_50

Download citation

  • DOI: https://doi.org/10.1007/3-540-47847-7_50

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43674-4

  • Online ISBN: 978-3-540-47847-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation