Implementing noncollective parallel I/O in cluster environments using Active Message communication

Abstract

A cost‐effective secondary storage architecture for parallel computers is to distribute storage across all processors, which then engage in either computation or I/O, depending on the demands of the moment. A difficulty associated with this architecture is that access to storage on another processor typically requires the cooperation of that processor, which can be hard to arrange if the processor is engaged in other computation. One partial solution to this problem is to require that remote I/O operations occur only via collective calls. In this paper, we describe an alternative approach based on the use of single‐sided communication operations such as Active Messages. We present an implementation of this basic approach called Distant I/O and present experimental results that quantify the low‐level performance of DIO mechanisms. This technique is exploited to support noncollective parallel shared file model for a large out‐of‐core scientific application with very high I/O bandwidth requirements. The achieved performance exceeds by a wide margin the performance of a well equipped PIOFS parallel filesystem on the IBM SP.