Support of automatic parallelization with concept comprehension

Abstract

Current approaches to parallelizing compilation perform a purely structural analysis of the sequential code. Conversely, a semantic analysis performing concept assignment for code sections, can support the recognition of the algorithms that the code implements. This can considerably help the parallelization process, by allowing the introduction of heuristics and an extensive pruning of the search space, and thus enabling the application of more aggressive code transformations. It can play an important role in overcoming the current limitations to Automatic Parallelization. In this paper we discuss the applicability of concept comprehension to the parallelization process, and we present a novel technique for automatic algorithmic recognition we have designed and implemented. We are currently developing a reverse engineering tool supporting the translation of sequential Fortran code into HPF, which is based on the recognition technique we have developed. Its working criteria are illustrated and discussed.

Introduction

An important approach to the programming of parallel architectures is based on the (semi-)automatic transformation of sequential code, possibly augmented with parallelization directives, into explicitly parallel code. This method is not only useful for supporting the reuse of “dusty decks”, but has also important advantages when developing an application from scratch:

• ease of design, development, verification and debugging of sequential code,

• existence of several environments for the support of sequential programming, and

• portability – the compiler performs the mapping from the sequential code to a code suited to a particular parallel target architecture, including machine-specific optimizations.

The main problem of this approach is the huge complexity of the search procedure for finding a close to optimal parallel version of a given sequential code, for a given target architecture.

If the target machine is a distributed memory architecture, the parallelization procedure can be thought as composed of the following conceptual phases.

• Selection of a parallel execution model (defined by a spawning, communication and synchronization topology among processes);

• selection of a data distribution across the processes defined in the execution model;

• selection of a work distribution across the processes, and consequent code decomposition and assignment to processes (possibly with replications);

• analysis and optimization of the communication needed by nonlocal accesses.

Each phase is implemented by a series of transformations applied to the code. Current approaches try to prune the search space associated with the selection of transformations by enforcing tight constraints on the alternatives allowed in each phase, which do not depend on the code to be parallelized, or on the target architecture, and by delegating to the user some decisions.

First, the execution model is generally selected a priori. It is referred in the literature as the Single Program Multiple Data (SPMD) paradigm (8, 22in its version for distributed memory architectural model).

Secondly, a default rule (such as owner computes rule [33]) determines the work distribution, in particular within parallel loops.

Within these constraints, the data and work distribution become tightly related problems, and a solution to one of them induces a solution to the other, provided that the computation is characterized by regular accesses to data [33].

Two orthogonal strategies arises from this scenario. One of them [1]aims to the automatic selection of sequences of unimodular transformations of loop nests, leading to loop iteration distributions which are optimal with regard to the locality of accesses.

The other strategy 34, 35, 21chooses the data distribution to drive the parallelization procedure, determining the work distribution and the required communication.

Unfortunately, even after the imposition of such tight constraints, the complexity of the problem remains untractable in the general case. For instance, within the approaches that attempt to determine automatically the data distribution 26, 19, 6, 1the main open questions are: (i) to find out the portions of the code (phases) where the data distribution has to remain unchanged; (ii) to find out suitable redistributions among phases; (iii) for each phase, to find out alignments among the array dimensions, such that the cost of communications due to alignment conflicts is minimal (alignment conflict resolution); (iv) for each phase, to find out the best distributions for array dimensions, such that data locality is maximized, thus minimizing the communication overhead.

Finally, we note that the use of a fixed parallel execution model, even a very flexible and general one like the SPMD model may prevent the selection of a parallelization strategy that is best suited to the characteristics of the code to be parallelized. For instance, in the context of irregular problems, execution models alternative to SPMD have been pointed out. Several authors 4, 7, 16, 17, 18have recently argued that parallel programs can be classified as belonging to a Parallel Programming Paradigm (also called template or skeleton), based on the way processes forming the parallel program are created, synchronize and communicate, abstracting from the details of their sequential part. According to this view, the SPMD model can be viewed as one paradigm; other paradigms may provide more flexible approaches to program parallelization. The idea is that a parallelizing compiler should be able to select the paradigm that is more suited to the characteristics of the algorithm, of the target architecture and of the run-time parameters, and derive the final parallel code, applying template based code transformations [9].

In this paper we present a novel approach to support parallelizing compilation: the integration of techniques for automatic recognition of algorithmic patterns within source code (concept comprehension). It can play an important role in overcoming the current limitations to Automatic Parallelization.

In Section 2we discuss the applicability of concept comprehension techniques to the parallelization process. A novel technique we have designed for automatic algorithmic recognition is then presented (Section 3) together with a prototype tool which implements it.

We are currently developing a reverse engineering tool supporting the translation of sequential Fortran code into HPF, Migrator, which is based on the recognition technique we have developed. Its design is presented in Section 4. In Section 5the features of the procedure for automated algorithmic recognition we have implemented are exemplified through a case study, which also shows an application to code restructuring for translation of Fortran into HPF.