Skip to main content
SpringerLink
Log in

Virtual memory management in data parallel applications

  • Workshop: High Performance Computing on Very Large Datasets
  • Conference paper
  • First Online:
High-Performance Computing and Networking (HPCN-Europe 1999)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 1593))

Included in the following conference series:

Abstract

The PaLaDiN (PArallel LArge Data set In Network of workstations) project is concerned with parallel out-of-core application running on cluster of workstations or PCs. In such architectures, each node has a virtual memory manager and a first idea is to use this feature to run “parallel out-of-core” application as a parallel in-core one. The out-of-core part of the problem, i.e. the schedule of data fetch and data write-back, is relegated to the operating system.

In this paper we show that usual virtual memory manager is not well suited for parallel out-of-core application. Then, we propose an extension to modern operating system which allow to define application specific virtual memory manager. This extension is made up of one kernel module (MMUSSEL) and one library (MMUM) and run on Linux. We present a new pagination strategy for the LU decomposition program.

This work is partially supported by the “Pôle de modélisation de la région Picardie”.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Vadim Abrossimov, Marc Rozier, and March Shapiro. Generic virtual memory management for operating system kernels. In Proc. of th 12th ACM Symposium on Operating System Principles, December 1989.

    Google Scholar 

  2. Rajesh Bordawekar and Alok Choudhary. Communication strategies for out-of-core programs on distributed memory machines. In Proceedings of the 9th ACM International Conference on Supercomputing, pages 395–403, Barcelona, July 1995. ACM Press.

    Chapter  Google Scholar 

  3. J. Clinckemaillie, B. Elsner, G. Lonsdale, S. Meliciani, S. Vlachoutsis, F. de Bruyne, and M. Holzner. Performance issues of the parallel PAM-CRASH code. The International Journal of Supercomputer Applications and High Performance Computing, 11(1):3–11, Spring 1997.

    Google Scholar 

  4. Jean-François Collard and Gil Utard. Automatic data layout and code restructuring for out-of-core programs. In Proc. of the Workshop on Out-of-Core Computation and Adaptative Compilation (COCA'98), Cap-Hornu, Baie de Somme, France, September 1998. IEEE Yuforic and GDR ARP, LaRIA and PRiSM.

    Google Scholar 

  5. Olivier Cozette. Virtual memory managment in intensive computation. Master's thesis, LaRIA, Université de Picardie Jules Verne, Jully 1998.

    Google Scholar 

  6. G. Davis, L. Lau, R. Young, F. Duncalfe, and L. Brebber. Parallel run-length encoding (RLE) compression—reducing I/O in dynamic environmental simulations. The International Journal of High Performance Computing Applications, 12(4), Winter 1998. To appear in a Special Issue on I/O in Parallel Applications.

    Google Scholar 

  7. Keith Krueger, David Loftesness, Amin Vadhat, and Thomas Anderson. Tools for the development of applications-specific virtual memory management. Technical report, University of California, Berkeley, April 1993.

    Google Scholar 

  8. Dan C. Marinescu and Kuei Yu Wang. An analysis of the paging activity of parallel programs, part I: Correlation of the paging activity of individual node programs in the SPMD execution mode. Technical Report CSD-TR-94-042, Purdue University, June 1994.

    Google Scholar 

  9. Dan C. Marinescu and Kuei Yu Wang. Characterization of the Paging Activity of NAS Benchmark Programs on the Intel Paragon. Technical Report CSD-TR-95-015, Purdue University, March 1995.

    Google Scholar 

  10. Dan C. Marinescu and Kuei Yu Wang. Gang scheduling and demand paging. In Proc. of the Int. Conf. on High Performance Computing, pages 180–188, New Delhi, India, December 1995.

    Google Scholar 

  11. Todd C. Mowry, Angela K. Demke, and Orran Krieger. Automatic compiler-inserted I/O prefetching for out-of-core applications. In Proceedings of the 1996 Symposium on Operating Systems Design and Implementation, pages 3–17. USENIX Association, October 1996.

    Google Scholar 

  12. J.K. Ousterhout. Scheduling techniques for concurrent systems. In Proc. of the 3rd Int. Conf on Distributed Computing System, pages 22–30, October 1982

    Google Scholar 

  13. Yoonho Park, Ridgway Scott, and Stuart Sechrest. Virtual memory versus file interfaces for large, memory-intensive scientific applications. In Proceedings of Supercomputing '96. ACM Press and IEEE Computer Society Press, November 1996. Also available as UH Department of Computer Science Research Report UH-CH-96-7.

    Google Scholar 

  14. J.M. Del Rosario and A. Choudhary. High performance I/O for massively parallel computers: Problems and Prospects. IEEE Computer 27(3):59–68, 1994.

    Google Scholar 

  15. Rajeev Thakur, Ewing Lusk, and William Gropp. I/O characterization of a portable astrophysics application on the IBM SP and Intel Paragon. Technical Report MCS-P534-0895, Argonne National Laboratory, August 1995. Revised October 1995.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LaRIA Université de Picardie Jules Verne, 80000, Amiens, France

    Eddy Caron, Olivier Cozette, Dominique Lazure & Gil Utard

Authors
  1. Eddy Caron
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Olivier Cozette
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dominique Lazure
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gil Utard
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Peter Sloot Marian Bubak Alfons Hoekstra Bob Hertzberger

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag

About this paper

Cite this paper

Caron, E., Cozette, O., Lazure, D., Utard, G. (1999). Virtual memory management in data parallel applications. In: Sloot, P., Bubak, M., Hoekstra, A., Hertzberger, B. (eds) High-Performance Computing and Networking. HPCN-Europe 1999. Lecture Notes in Computer Science, vol 1593. Springer, Berlin, Heidelberg . https://doi.org/10.1007/BFb0100671

Download citation

  • DOI: https://doi.org/10.1007/BFb0100671

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-65821-4

  • Online ISBN: 978-3-540-48933-7

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation