Parallelization of irregular out-of-core applications for distributed-memory systems

Abstract

Large scale irregular applications involve data arrays and other data structures that are too large to fit in main memory and hence reside on disks; such applications are called out-of-core applications. This paper presents techniques for implementing this kind of applications. In particular we present a design for a runtime system to efficiently support parallel execution of irregular out-of-core codes on distributed-memory systems. Furthermore, we describe the appropriate program transformations required to reduce the I/O overheads for staging data as well as for communication while maintaining load balance. The proposed techniques can be used by a parallelizing compiler or by users writing programs in node + message passing style. We have done a preliminary implementation of the techniques presented here. We introduce experimental results from a template CFD code to demonstrate the efficacy of the presented techniques.

The work described in this paper is being carried out as part of the research project “Language, Compiler and Advanced Data Structure Support for Parallel I/O Operations” supported by the Austrian Research Foundation (FWF Grant P11006-MAT).