Abstract
Recent research in parallel numerical computation has tended to focus on the algorithmic level. Less attention has been given to the programming level where algorithm is matched, to some extent, to computer architecture. This two-part paper presents a three-level approach to parallel programming which distinguishes between mathematical algorithm, program and computer architecture. In part I, we motivate our approach by a case study using the Ada language. In part II, a mathematical concept of parallel algorithm is introduced in terms of partial orders. This serves as the basis of a theory of parallel computation which makes possible a precise semantics and a precise criterion of complexity of parallel programs. It also suggests some notation for specifying parallel numerical algorithms. To illustrate the ideas presented in part II, we concentrate here on parallel numerical computations which have vector spaces as their central data type and which are intended to be executed on a multi-processor system. The Ada language, with its task constructs, allows one to program computer algorithms to be executed on multi-processor systems, rather than on “vector (pipelined) architectures”. To provide a concrete example of the general problem of programming parallel numerical algorithms for multi-processor computers, we do a case study of how Ada can be used to program the solution of a system of linear equations on such computers. The case study includes an analysis of complexity which addresses the cost of data movement and process control/synchronization as well as the usual arithmetic complexity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Kai Wang,Supercomputer architectures, USC Computer Research Institute Report CRI 87-22, Mar. 25, 1987.
Larry Snyder and Z. Segall, Eds.:Proc. Workshop on Performance Efficient Parallel Programming, Seven Springs, Pa., Sept. 1986, CMU Reports 86-180, 181.
CRAY Fortran Manual.
G. Rodrigue, E. Giroux, and M. Pratt,Perspectives on large-scale scientific computation, Computer (IEEE), Oct. 1980, 65–80.
W. Gentleman,Some complexity results for matrix computation on parallel processors, JACM, Jan. 1978, 112–115.
T. Agerwala and B. Lint,Communication in parallel algorithms for boolean matrix multiplication, Proc. IEEE Conference on Parallel Processing, Aug. 1978, 146–153.
J. J. Dongarra et al.,LINPACK User's Guide, SIAM 1979.
C. Lawson, R. Hanson, D. Kincaid, and F. Krough,Basic linear algebra program for Fortran usage, ACM Trans. Math. Software, 1979.
Reference Manual for the Ada Programming Language, U.S. Dept. Defense, July 1982.
Proc. Conference on Programming Languages and Compilers for Parallel and Vector Machines, SIGPLAN Notices, March 1975.
C. W. Wetherell,LWG Fortran Manual, Draft, July 21, 1981.
George Paul,VECTRAN and the proposed vector/array extensions to ANSI Fortran, IBM Research Report RC 9223, Jan. 21, 1982.
ANSI X3J3 Committee Composite Array Proposal, Version 78, May 18, 1981.
E. K. Blum,Numerical Analysis and Computation: Theory and Practice, Addison-Wesley, Reading, Mass. 1972.
E. K. Blum,Ada-based architectures for parallel vector computation, TRW Report DSG-TR1, Mar. 31, 1983.
D. Heller,A survey of parallel algorithms in numerical linear algebra, SIAM Rev. 20, 1978, 740–777.
S. C. Chen and D. Kuck,Time and parallel processor bounds for linear recurrence systems, IEEE Trans. Comp., v. C-24, 1975, 701–717.
L. Csanky,Fast parallel matrix inversion algorithms, Proc. 16th Symp. on Foundations of Computer Science, Oct. 1975.
H. T. Kung,The structure of parallel algorithms, inAdvances in Computers, v. 19, 1980, Academic Press, 65–111.
H. S. Stone,An efficient parallel algorithm for the solution of a tridiagonal linear system of equations, JACM, v. 20, 1973, 27–38.
A. H. Sameh and D. Kuck,On stable parallel linear system solvers, JACM, 25, Jan. 1978, 81–91.
O. Wing and J. W. Huang,A computational model of parallel solution of linear equations, IEEE Trans. Comp., C-29, July 1980, 632–638.
R. E. Lord, J. S. Kowalik, and S. P. Kumar,Solving linear algebraic equations on an MIMD computer, JACM, 30, Jan. 1983, 103–117.
J. B. Smith,A pipelined shared resource MIMD computer, Int. Conf. on Parallel Processing, Detroit, Mich., Aug. 1978, 6–8.
A. H. Sameh,Numerical parallel algorithms, inHigh Speed Computer and Algorithm Organization, Ed. D. Kuck, D. Lawrie, Academic Press, 1977.
E. K. Blum,Programming parallel numerical algorithms in Ada, Proc. IFIP Conference on the Relationship Between Numerical Computation and Programming Language, North-Holland, 1982.
F. Belz, E. K. Blum, and D. Heimbigner,A multiprocessing implementation-oriented formal definition of Ada in SEMANOL, SIGPLAN Notices, Dec. 1980, Symp. on the Ada Language.
R. Milner,Calculi for Synchrony and Asynchrony, Theoretical Computer Science 25, 1983, 267–310.
Author information
Authors and Affiliations
Additional information
Dedicated to Peter Naur on the occasion of his 60th birthday
This research was partially supported by NSF Grants DCR-8406290 & CCR-8712192.