Jacob Gotwals
Dennis Gannon
ABSTRACT
The design and implementation of portable, efficient, and expressive mechanisms for I/O on complex distributed data structures—such as found in adaptive parallel applications—is a challenging problem that we address in this paper.
We describe the design, programmer interface, implementation, and performance of pC++/streams, a library that provides an expressive mechanism for I/O on distributed arrays of variable-sized objects in pC++, an object-parallel language. pC++/streams is intended for developers of parallel programs requiring efficient high-level I/O abstractions for checkpointing, scientific visualization, and debugging.
pC++/streams is an implementation of d/streams, a language-independent abstraction for buffered I/O on distributed data structures. We describe the d/streams abstraction and present performance results on the Intel Paragon and SGI Challenge showing that d/streams can be implemented efficiently and portably.
- 1.Robert Bennett, Kelvin Bryant, Alan Sussman, Raja Das, and Joel Saltz. Jovian: A framework for optimizing parallel I/O. In Proceedings of the 1994 Scalable Parallel Libraries Conference, Oct 1994. Available on the WWW at http : llwww, cs .umd. edulprojectslhpslliolio, html.Google Scholar
- 2.Francois Bodin, Peter Beckman, Dennis Gannon, Srinivas Narayana, and Shelby Yang. Distributed pC++: Basic ideas for an object parallel language. Scientific Programming, 2(3), Fall 1993.Google Scholar
- 3.B.Stroustroup. The C++ programming language. Addison Wesley, 1986. Google ScholarDigital Library
- 4.Michael Carey, David Dewitt, et aI. Shoring up persistent applications. In Proceedings of A CM-SIGMOD Conference on the Management of Data 1994, May 1994. Available on the World Wide Web at http z //www, cs. wise. edu/p / shorelhtml / shore, home. html Google ScholarDigital Library
- 5.Alok Choudhary. Parallel I/O systems: Guest editor's introduction. Journal of Parallel and Distributed Computing, 17:1--3, 1993.Google ScholarCross Ref
- 6.Alok Choudhary et al. PASSION: Parallel And Scalable Software for i/O. Technical Report NPAC SCCS-636, Syracuse University, September 1994.Google Scholar
- 7.Peter Corbett and Dror Feitelson. Design and implementation of the vesta parallel file system. In Proceedings of Scalable High Performance Computing Conference, SHPCC94, May 1994.Google ScholarCross Ref
- 8.D.Gannon, P.Beckman, F.Bodin, J.Gotwals, S.Narayana, S.Srinivas, and B.Winnika. Sage++: An object oriented toolkit for program transformations. In Proceedings of Oonski 94, April 1994. Available on WWW at http : I lwww, extreme, indiana, edulsageldocs, htmlGoogle Scholar
- 9.D.Gannon, S.Yang, S.Srinivas, V.Menkov, and P.Bode. Object-oriented methods for parallel execution of astrophysics simulations. In Proceedings of Mardigras94, February 1994. Available from [email protected].Google Scholar
- 10.D.Loveman. High performance fortran. IEEE Parallel and Distributed Technology, 1:25--42, 1993. The HPF Forum archive for HPF related material is anonymous ftp from Google ScholarDigital Library
- 11.N. Galbreath, W. Gropp, and D. Levine. Application driven parallel I/O. In Proceedings of Supercomputing 93, November 1993. Also Argonne technical report MCS-P381-0893. Google ScholarDigital Library
- 12.L. Hernquist and J. P. Ostriker. A self-consistent field method for galactic dynamics. The Astrophysical Journal, 386:375--397, 1992.Google ScholarCross Ref
- 13.High Performance Fortran Forum. On the WWW at http z //www. erc. msstate, edu/hp ff/home, html.Google Scholar
- 14.Intel Supercomputing System Division. Paragon Users Guide. Chapter 5 and Chapter 8 available online from http ://www. ssd. intel, com/.Google Scholar
- 15.John Karpovich, Andrew Grimshaw, and James French. Extensible file systems (ELFS): An object-oriented approach to high performance file i/O. In Proceedings of OOPSLA 94, 9th conference on object-oriented programming systems, languages, and applications, October 1994. Available on the WWW at http # //uvacs. c#. virginia, edu. Google ScholarDigital Library
- 16.Thomas Kwan and Daniel Reed. Performance of the CM-5 scalable file system. In Proceedings of International Conference on Supercomputing 94, July 1994. Also accessible on the WWW at http, //bugle. cm.uiue, eau/. Google ScholarDigital Library
- 17.Lamb et al. The ObjectStore database system. Communications of the A CM, October 1991. Google ScholarDigital Library
- 18.MPI-IO: A parallel file I/O for MPI. Available on WWW at berg- //lovelace .nas. nasa. gov/MPI- IO/n#i- io. html.Google Scholar
- 19.Portable Parallel File System project. Available at http : //www-picasso. cs .uiuc. edu/Pro# ects IPPFS/.Google Scholar
- 20.Juan Rosario and Alok Choudhary. High-performance I/O for massively parallel computers: Problems and Prospects. IEEE Computer, pages 59--68, March 1994. Google ScholarDigital Library
- 21.Documents from the scalable I/O initiative. Available on the World Wide Web at http # I Iccs~. caltech, edu/SI01SZ0, html.Google Scholar
- 22.Kent Seamons and Marianne Winslett. An efficient abstract interface for multidimensional I/O. In Proceedings of Supercomputing 1994, November 1994. Accessible on the WWW at http : i lwww. computer, orglp3 Isc94home. html. Google ScholarDigital Library
- 23.Kent Seamons and Marianne Winslett. Physical schemas for large multidimensional arrays in scientific computing applications. In Proceedings of the 7th international Working Conference on Scientific and Statistical Database Management, September 1994. Accessible on the WWW at http : l/bunny, cs. uiuc. edu/CADR/wins lett/arrays, html. Google ScholarDigital Library
- 24.The parallel i/o archive. Available on the World Wide Web at http, / lwww. e#. dartmouth, edulpario, html.Google Scholar
- 25.Thinking Machines Corporation. CMMD Reference Manual. Chapter 12.Google Scholar
Index Terms
- pC++/streams: a library for I/O on complex distributed data sources
Recommendations
pC++/streams: a library for I/O on complex distributed data sources
The design and implementation of portable, efficient, and expressive mechanisms for I/O on complex distributed data structures—such as found in adaptive parallel applications—is a challenging problem that we address in this paper.
We describe the design, ...
Sketching distributed sliding-window data streams
While traditional data management systems focus on evaluating single, ad hoc queries over static data sets in a centralized setting, several emerging applications require (possibly, continuous) answers to queries on dynamic data that is widely ...
Comments