OPIOM: Off-Processor I/O with Myrinet

Abstract

As processors become more powerful and clusters larger, users will exploit this increased power to progressively run larger and larger problems. Today’s datasets in biology, physics or multimedia applications are huge and require high-performance storage sub-systems. As a result, the hot spot of cluster computing is gradually moving from high performance computing to high performance storage I/O. The solutions proposed by the parallel file-system community try to improve performance by working at the kernel level to enhance the regular I/O design or by using a dedicated Storage Area Network like Fiber Channel. We propose a new design to merge the communication network and the storage network at the best price. We have implemented it in OPIOM with the Myrinet interconnect: OPIOM moves data asynchronously from SCSI disks to the embedded memory of a Myrinet interface in order to send it to a remote node. This design presents attractive features: high performance and extremely low host overhead.

Introduction

The availability of powerful microprocessors and high-speed networks as commodity components is making clusters an appealing solution for cost-effective high performance computing. However, the bottleneck for users’ applications tends to shift from the computation and the communication sides to the I/O domain: the problem sizes are bigger and bigger and the time to load datasets into the cluster work pool and write the results to disks cannot be neglected any more.

The new generation of commodity components like storage controllers and high-speed networks can be efficiently used to break some architectural limitations inherited from the past, while keeping the price/performance ratio as low as possible. Our research effort improves a basic feature for the usage of parallel I/O on clusters by removing bottlenecks.

In Section 2, we present the motivation of this work, one current limitation of the parallel I/O design for clusters, and some related work that tries to improve it. We propose a new design in Section 3, describing our contribution and detailing the issues that occurred during the implementation. Then, Section 4 shows the results of some experimental benchmarks to highlight the benefit of our work for parallel I/O implementations. Finally, the conclusion in Section 5 summarizes our work and presents the short and medium term perspectives of our project.