Modeling overlapped operation between the control unit and processing elements in an SIMD machine

doi:10.1016/0743-7315(91)90004-S

Journal of Parallel and Distributed Computing

Volume 12, Issue 4, August 1991, Pages 329-342

https://doi.org/10.1016/0743-7315(91)90004-S Get rights and content

Abstract

A model for overlapped operation between the control unit (CU) and processing elements (PEs) in an SIMD machine is presented. The major requirements and structure of the CU for overlapped operation in SIMD mode are described and overlapped operation is formally defined. To use the computing power of both the CU and the PEs most effectively to execute a single program, a balanced work load between the CU and PEs is required. It is assumed that certain computations (e.g., the manipulation of loop index variables, PE-common array index calculations) can be migrated from the PEs to the CU and vice versa.This research demonstrates how to increase the effectiveness of an SIMD machine by allowing overlapped operation between the CU and PEs. The best overlapping can be achieved ideally by assigning an equal amount of work to be executed concurrently on the CU and PEs, resulting in a 2N speedup for an N-PE system. The goal of this research is to develop a model of overlapped operation in SIMD mode so that the actual maximum possible performance of the SIMD machine can be attained.

References (33)

M. Auguin et al.
Experience using a SIMD/SPMD multiprocessor architecture
Microprocess. Microprogramming
(Aug. 1987)
F. Darema et al.
A singleprogram-multiple-data computational model for EPEX/FORTRAN
Parallel Comput.
(Apr. 1988)
S.A. Fineberg et al.
Experimental analysis of a mixed-mode parallel architecture using bitonic sequence sorting
J. Parallel Distrib. Comput.
(Mar. 1991)
G.M. Amdahl
Validity of the single-processor approach to achieving large scale computing capabilities
G.H. Barnes et al.
The Illiac IV computer
IEEE Trans. Comput.
(Aug. 1968)
K.E. Batcher
STARAN parallel processor system hardware
K.E. Batcher
STARAN Series E
K.E. Batcher
Design of a massively parallel processor
IEEE Trans. Comput.
(Sept. 1980)
K.E. Batcher
Bit serial parallel processing systems
IEEE Trans. Comput.
(May 1982)
Berg, T.B., Kim, S.D., and Siegel, H.J., Limitations imposed on mixed-mode performance of optimized phases due to...

T.B. Berg et al.

Instruction execution trade-offs for SIMD vs. MIMD vs. mixed-mode parallelism

T. Blank

The MasPar MP-1 architecture

IEEE Compcon

(Feb. 1990)

E.C. Bronson et al.

Experimental application-driven architecture analysis of an SIMD/MIMD parallel processing system

IEEE Trans. Parallel Distrib. Systems

(Apr. 1990)

P. Christy

Software to support massively parallel computing on the MasPar MP-1

IEEE Compcon

(Feb. 1990)

P. Duclos et al.

Image processing on a SIMD/SPMD architecture: OPSILA

J.L. Gustafson

Reevaluating Amdahl's law

Comm. ACM

(May 1988)

Cited by (17)

Aspects of computational mode and data distribution for parallel range image segmentation
1999, Parallel Computing
Parallel processing methods are a means to achieve significant speedup of computationally expensive image understanding algorithms, such as those applied to range images. Practical implementations of these algorithms must deal with the problems of selecting an appropriate parallel architecture and mapping the algorithm onto that architecture. The parallel implementation approaches for range image segmentation that are presented here are applicable to many low-level image understanding algorithms in a variety of parallel architectures. An evaluation of initial data distribution is presented to determine whether a square subimage or a striped subimage distribution would result in the greatest overall reduction in execution time for the given range image segmentation problem. Novel implementations that consider each data distribution's treatment of edge pixels in window operations yield a trade-off between the number of data transfers versus the amount of computation. This trade-off is examined both analytically and experimentally. Additionally, using the same initial data distributions, a technique is introduced for changing the allocation of work to each of the processors to reduce the number of network settings by one half. This technique and the method for determining the better initial data distribution can be used with any machine and any window-based technique that requires a full window to perform image calculations. Comparisons of range image processing algorithms are performed using “pure” SIMD algorithms, “pure” MIMD algorithms, and mixed-mode implementations with both SIMD and MIMD elements. Each of these approaches are quantitatively analyzed and compared for implementing the different phases of a particular hybrid range segmentation algorithm. Results of this implementation study indicate that quantifiable reductions in execution time result from the proper choice of parallel mode for each portion of the segmentation process.
A Heterogeneous Mixed-Mode Execution Model for Massively Parallel Systems
1999, Journal of Parallel and Distributed Computing
In this paper, we consider a massively parallel system that is composed of heterogeneous processors, that is, processors with different processing power, and that combines the advantages of the SIMD and MIMD architectures. The heterogeneous mixed-mode (HeMM) execution model is composed of two main components, which operate in the well-known SIMD and MIMD paradigms. The main computing power comes from a component that is composed of a massive number of processors and operates in a data parallel manner. The other component is composed of a few (or even one) fast processors which operate in the MIMD paradigm. The operation of a small number of processors in an MIMD paradigm has been well demonstrated through actual systems. The processors in this component add flexibility to the execution of the parallel programs such that it adjusts to the changing parallelism of the program to enhance the performance. Based on this execution model we analyze the gains in performance that is obtainable by this new system. We show that substantial performance gains can be obtained by using the HeMM system.
A Block-based mode selection model for SIMD/SPMD parallel environments
1994, Journal of Parallel and Distributed Computing
One of the challenges for parallel compilers and compiler-related tools is, given a machine-independent parallel language, to generate executable code for a variety of computational models, and to identify those specific parallel modes for which a program is well-suited. One portion of this problem, developing a method for estimating the relative execution time of a data-parallel algorithm in an environment capable of the SIMD and SPMD (MIMD) modes of parallelism, is presented. Given a data-parallel program in a language whose syntax is mode-independent and empirical information about instruction execution time characteristics, the goal is to use static source-code analysis to determine an implementation that results in an optimal execution time for a mixed-mode machine capable of SIMD and SPMD parallelism. Statistical information about individual operation execution times and paths of execution through a parallel program is assumed. A secondary goal of this study is to indicate language, algorithm, and machine characteristics that must be researched to learn how to provide the information needed to obtain an optimal assignment of parallel modes to program segments.
Multiple quadratic forms: A case study in the design of data-parallel algorithms
1994, Journal of Parallel and Distributed Computing
Data-parallel implementations of the computationally intensive task of solving multiple quadratic forms (MQFs) have been examined. Coupled and uncoupled parallel methods are investigated, where coupling relates to the degree of interaction among the processors. Also, the impact of partitioning a large MQF problem into smaller non-interacting subtasks is studied. Trade-offs among the implementations for various data-size/machine-size ratios are categorized in terms of complex arithmetic operation counts, communication overhead, and memory storage requirements. Furthermore, the impact on performance of the mode of parallelism used is considered, specifically, SIMD versus MIMD versus SIMD/MIMD mixed-mode. From the complexity analyses, it is shown that none of the algorithms presented in this paper is best for all data-size/machine-size ratios. Thus, to achieve scalability (i.e., good performance as the number of processors available in a machine increases), instead of using a single algorithm, the approach discussed is to have a set of algorithms from which the most appropriate algorithm or combination of algorithms is selected based on the ratio calculated from the scaled machine size. The analytical results have been verified by experiments on the MasPar MP-1 (SIMD), nCUBE 2 (MIMD), and PASM (mixed-mode) prototype.
Predicting performance and selecting modes of parallelism: A case study using cyclic reduction on three parallel machines
1993, Journal of Parallel and Distributed Computing
Mapping cyclic reduction, a known approach for the parallel solution of tridiagonal and block tridiagonal systems of equations, onto the MasPar MP-1, nCUBE 2, and PASM parallel processing systems is discussed. Each of these represents a different mode of parallelism that can be used in the design of supercomputers. The MasPar MP-1 is a commercially available SIMD system with near-neighbor and multiplexed-multistage communications networks, the nCUBE 2 is a commercially available MIMD system with a partitionable hypercube communication network, and PASM is an experimental partitionable-SIMD/MIMD mixed-mode machine with a partitionable multistage cube network. Specific issues addressed here are SIMD/MIMD trade-offs, the effect on execution time of increasing the number of processors used, the impact of the interprocessor communications network on performance, the importance of predicting algorithm performance as a function of the mapping used, and the advantages of a partitionable system. Analytical results are validated by experimentation on all three machines.
Limitations imposed on mixed-mode performance of optimized phases due to temporal juxtaposition
1991, Journal of Parallel and Distributed Computing
Mixed-mode parallel processing systems are capable of executing in either SIMD (synchronous) or MIMD (asynchronous) mode of parallelism. The ability to switch between the two modes at instruction-level granularity with very little overhead allows the parallelism mode to vary for each portion of an algorithm. To fully exploit the capability of intermixing both SIMD and MIMD operations within a single program, one must determine the optimum mapping of an algorithm to the mixed-mode architecture. The phase optimization technique, where the programmer makes an implicit assumption that by combining the best version of each phase the optimal implementation of the entire program will be achieved, generally works in a serial computer environment. The application of this approach to the selection of a mode of parallelism for each phase is investigated by presenting a detailed study of a practical image-processing application, the Edge-Guided Thresholding algorithm, and its mapping to a mixed-mode parallel architecture. The six functional phases of the algorithm, as well as their temporal juxtaposition, are analyzed along with experimental performance measurements obtained from the PASM parallel processing prototype, a mixed-mode system. The results discussed here demonstrate a situation in which the advantages of a mixed-mode approach are limited.

View all citing articles on Scopus

^☆: This research was supported by the Naval Ocean Systems Center under the High Performance Computing Block, ONT, by the Office of Naval Research under Grant N00014-90-d-1937, and by the National Science Foundation under Grant CDA-9015696.

View full text

Regular articleModeling overlapped operation between the control unit and processing elements in an SIMD machine☆

Abstract

Microprocess. Microprogramming

Parallel Comput.

J. Parallel Distrib. Comput.

Validity of the single-processor approach to achieving large scale computing capabilities

The Illiac IV computer

IEEE Trans. Comput.

STARAN parallel processor system hardware

STARAN Series E

Design of a massively parallel processor

IEEE Trans. Comput.

Bit serial parallel processing systems

IEEE Trans. Comput.

Instruction execution trade-offs for SIMD vs. MIMD vs. mixed-mode parallelism

The MasPar MP-1 architecture

IEEE Compcon

Experimental application-driven architecture analysis of an SIMD/MIMD parallel processing system

IEEE Trans. Parallel Distrib. Systems

Software to support massively parallel computing on the MasPar MP-1

IEEE Compcon

Image processing on a SIMD/SPMD architecture: OPSILA

Reevaluating Amdahl's law

Comm. ACM

Regular article
Modeling overlapped operation between the control unit and processing elements in an SIMD machine☆