Parallel adaptive computing on meta-systems including NOWs

Abstract

Load analysis of meta-systems including NOWs or COWs has shown that only a few percentage of the available power is used during long periods of time. Therefore, in order to exploit the idle time when executing a parallel application work load must be sent to a machine as soon as the latter becomes available. Furthermore, in order to keep respected the ownership of workstations work has to be stopped and resumed later as soon as the machine executing it is requisitioned by its owner. As a consequence, users need an adaptive system allowing to return events related to the goings and comings of workstations. On the other hand, it is necessary to provide them a parallel adaptive programming methodology that plans the handling of these events.

In this paper, we present the MARS (MARS: multi-user adaptive resource scheduler, developed at LIFL laboratory, Université de Lille I) system and its parallel adaptive programming methodology through the block-based Gauss–Jordan algorithm used in numerical analysis to invert large matrices. Moreover, we propose a work scheduling strategy and an application-oriented solution for the fault tolerance issue. Furthermore, we present some experimental results obtained on a DEC/ALPHA COW and a SUN/Sparc4 NOW. The results show that very high absolute efficiencies can be obtained if the size of the blocks is well chosen. We also present some experimentations related to the adaptability of the application in a meta-system including the DEC/ALPHA COW and the SUN/Sparc4 NOW. The results show that the management of the adaptability consumes just a short percentage of execution time.

Introduction

Over the past few years, the parallel computing has known the emergence of the networks (NOWs) and clusters (COWs) of workstations. Particularly, for applications with little or no time constraints NOWs/COWs compete seriously supercomputers like Intel Paragon, IBM SP2, etc. The main reasons of that are: first, NOWs are scalable; second, the performance/cost ratio of workstations constantly increase; third, the technologic evolution of the networks like Myrinet, ATM, GigaEthernet, etc., makes the latter more and more fast; and finally, the arrival of parallel programming environments like PVM [1] and MPI [2] has promoted the NOW/COW programming.

Obviously, NOWs/COWs do not make supercomputers coming to the end, but conversely both of them can be integrated in a platform called a meta-system. Meta-systems including a NOW/COW has some characteristics which have a great impact on the parallel programming methodology. The major of them are the following:

• Only a few percentage of the available power is used during long periods of time. Indeed, a load analysis presented in [3] on a meta-system of over 50 machines (DEC-ALPHA/OSF-1 processors, PCs/Linux and SunSparc4/(SunOS, Solaris)). The results show that the average observed idle time on a working day (24 h) is over 83%.

• Failures are frequent. According to an experimental study [4] done on a large meta-system and on a working day (24 h), one failure (hard and soft failures, a machine reboot for example) occurs every 2.78 h.

• The last characteristic is a psychologic factor. In fact, users do not want others running processes on their own workstations when they are using them.

Therefore, according to these statements programming on meta-systems including a NOW/COW requires to dispose of a programming environment allowing to automatically (adaptivity) inform the user about the occurrence of events related to the availability of machines (their goings and comings), and the fluctuation of their load and their failures. MARS¹ [3], Piranha [5] and CARMI [6] are examples of such environment. On the other hand, it is necessary to plan at application level (adaptability) a “reaction” for each event. Thus, the parallel programming methodology must be changed so that it takes into account the adaptive control of parallelism.

In this paper, we present the MARS [3] system and its parallel adaptive programming methodology through the block-based Gauss–Jordan algorithm [7], [8], [9] used in numerical analysis to invert large matrices. MARS with its programming methodology aims at:

• Exploiting the idle time of the machines, i.e., as soon as a machine becomes available it is owned for executing some work of the application. That operation is referred to as an application unfold.

• Keeping respected the machines' ownership, it means that when a machine is requisitioned by its owner the work, part of the application, is pulled out. That operation is referred to an application fold.

• Solving, in some way, the fault tolerance problem: if a failure of a given machine occurs only the work running on this latter is re-started.

The Gauss–Jordan algorithm is a direct method used in numerical analysis to invert large dense matrices. A classical parallel version of this application has already been developed in [8], [9]. In [8], the application is implemented on a MIMD machine which is constituted by 8 processor elements (PEs), 4 data migration controllers and 3 levels shared memory, interconnected by an OMEGA network. The method includes a real-time scheduler that schedules the tasks of the application according to a tasks' graph determined by hand. In [9], the proposed method is implemented with the Occam language on a network of 8 transputers connected as a cube. It is based on block row operations, i.e., a block row of the matrix to be inverted is sent to each transputer. During computation, each processor produces a block of intermediate results and broadcasts it to all the transputers.

Unlike these versions, our implementation is parallel adaptive. Its idea is the following: a process (server) maintains the two matrices A (to be inverted) and B (the result of the inversion) of the algorithm. According to the availability/lack of availability of machines it extracts blocks from these matrices and distributes or redistributes them among the machines. These blocks are used by workers created on available machines to compute new blocks which are returned to the server which uses them to update the matrices A and B. If a machine computing a block is requisitioned by its owner then the partial resulting block is folded; then, it is unfolded on an available machine which resumes it exactly at its interruption point. Moreover, if a machine fails its work is re-started on an available machine.

The remainder of this paper is organized as follows: Section 2 describes the MARS system and its parallel adaptive programming methodology. Section 3 presents the sequential version of the block-based Gauss–Jordan algorithm and Section 4 describes its parallelization. In Section 5, we present our proposed fault tolerant parallel adaptive version of the algorithm and its implementation with MARS. In Section 6, we present its performance evaluation. Finally, in Section 7 we give some conclusions and some investigations of our future work.