Abstract
It is widely believed that superscalar and superpipelined extensions of RISC style architecture will dominate future processor design, and that needs of parallel computing will have little effect on processor architecture. This belief ignores the issues of memory latency and synchronization, and fails to recognize the opportunity to support a general semantic model for parallel computing. Efforts to extend the shared-memory model using standard microprocessors have led to systems that implement no satisfactory model of computing, and present the programmer with a difficult interface on which to build parallel computing applications. A more satisfactory model for parallel computing may be obtained on the basis of functional programming concepts and the principles of modular software construction. We recommend that designs for computers be built on such a general semantic model of parallel computation. Multithreading concepts and dataflow principles can frame the architecture of these new machines.
Similar content being viewed by others
References
John L. Hennessy and Norman P. Jouppi, Computer technology and architecture: An evolving interaction,Comm. of the ACM,24(9):18–29 (September 1991).
Harold S. Stone and John Cocke, Computer architecture in the 1990s,Comm. of the ACM,24(9):30–38 (September 1991).
Arvind and Robert A. Iannucci, Two fundamental issues in multiprocessing,Parallel. Computing in Science and Engineering, Lecture Notes in Computer Science,295:61–88, Springer-Verlag (1987).
V. S. Sunderam, PVM: A framework for parallel distributed computing,ACM Trans. on Programming Languages and Systems,1:80–112 (1985).
A. Deshpande and M. Schultz,Efficient parallel programming with Linda,Proc. of Supercomputing'92, IEEE Computer Society and ACM SIGARCH, pp. 238–244 (1992).
W. Daniel Hillis and Guy L. Steele, Jr., Data parallel algorithms,Comm. of the ACM,29(12):18–29 (December 1986).
American National Standards Institute, FORTRAN 90: Draft of the international standard, Technical Report, The FORTRAN Technical Committe of ANSI X3J3 (June 1990).
Eugene Albert, Kathleen Knobe, Joan Lukas, and Guy L. Steele Jr., Compiling Fortran 8x array features for the Connection Machine computer system,Proc. of ACM SIGPLAN Symp. on Parallel Programming (July 1988).
J. Li and Marina Chen, Compiling communication-efficient programs for massively parallel machines,IEEE Trans. on Parallel and Distrib. Syst.,2(3):361–375 (July 1991).
A. Choudhary, G. Fox, S. Ranka, S. Hirandani, K. Kennedy, C. Koebel, and C.-W. Tseng, Compiling Fortran 77D and 90D for mimd distributed-memory machines,Proc. of the Fourth Symp. on the Frontiers of Massively Parallel Computation, ACM and IEEE, pp. 4–11 (1992).
Gary Sabot, A compiler for a massively paralled distributed memory MIMD computer.Proc. of the Fourth Symp. on the Frontiers of Massively Parallel Computation, ACM and IEEE, pp. 12–20, (1992).
D. J. Mavriplis, Raja Das, R. E. Vermeland, and Joel Saltz, Implementation of a parallel unstructured Euler solver on shared and distributed memory architectures,Proc. of Supercomputing'92, IEEE Computer Society and ACM SIGARCH, pp. 132–141 (1992).
S. Thakkar, P. Gifford, and G. Fielland, The Balance multiprocessor system,IEEE Micro (February 1988).
B. R. Rau and C. D. Glaeser, Some scheduling techniques and an easily schedulable horizontal architecture for high performance scientific computing,Proc. of the 14th Ann. Workshop on Microprogramming, pp. 183–198 (1981).
J. A. Fisher, Very long instruction work architectures and the ELI-512,Proc. of the Tenth Int'l. Symp. on Computer Architecture, ACM (1983).
J. B. Dennis, Evolution of “static” dataflow architecture, Jean-Luc Gaudiot and Lubomir Bic. (eds.),Advanced Topics in Dataflow Computing, Prentice-Hall, pp. 35–91 (1991).
J. B. Dennis, G. R. Gao, and K. W. Todd, Modeling the weather with a dataflow supercomputer,IEEE Trans. on Computer,C-33(7):592–603 (July 1984).
John Palmer and Guy L. Steele, Jr, Connection machine model CM-5 system overview,Proc. of the Fourth Symp. on the Frontiers of Massively Parallel Computation, ACM and IEEE, pp. 474–483 (1992).
R. S. Nikhil, G. M. Papadopoulos, and Arvind,*T: A multithreaded massively parallel architecture,Proc. of the 19th Int's Symp. on Computer Architecture, pp. 156–167 (1992).
Harry F. Jordan, HEP architecture: Programming and performance, Janusz S. Kowalik, (ed.),Parallel MIMD Computation: The HEP Supercomputer and its Applications, The MIT Press, pp. 1–40 (1985).
Robert Alverson, David Callahan, Daniel Cummings, Brian Koblenz, Allan Proterfield, and Burton Smith, The Tera computer system,Proc. of the Int'l Conf. on Supercomputing, ACM, pp. 1–6 (1990).
G. M. Papadopoulos and D. E. Culler, Monsoon: An explicit token-store architecture,Proc. of the 17th Ann. Int'l. Symp. of Comput. Architecture, ACM, pp. 82–91 (1990).
Gregory M. Papadopoulos and Kenneth R. Traub, Multithreading: A revisionist view of dataflow architectures,The 18th Ann. Int'l Symp. on Computer Architecture, ACM, pp. 342–351 (1991).
L. Lamport, How to make a multiprocessor that correctly executes multiprocess programs,IEEE Trans. on Comput.,C-28:690–691 (September 1979).
Philip Bitar, The weakest memory-access order.J. of Parallel and Distrib. Comput.,15:305–331 (1992).
James Archibald and Jean-Loup Baer, Cache coherence protocols: Evaluation using a multiprocessor simulation model,ACM Trans. on Comput. Syst.,4:273–298 (November 1986).
D. Lenoski,J. Laudon, T. Joe, K. Gharachorloo, A. Gupta, and J. Hennessy, The directorybased cache coherence protocol for the DASH multiprocessor,Proc. of the 17th Int'l Symp. on Comput. Architecture, ACM, pp. 148–159 (1990).
Erik Hagersten, Anders Landin, and Seif Haridi, DDM: A cache-only memory architecture,IEEE Computer,25(9):44–54 (September 1992).
J. B. Dennis and G. R. Gao, An efficient pipelined dataflow processor architecture,Proc. of the Supercomputing Conf., IEEE Computer Society and ACM SIGARCH, pp. 368–373 (1988).
A. Bensoussan, Charles T. Clingen, and Robert C. Daley, The Multics virtual memory,Proc. of the Second Symp. on Operating System Principles, ACM, pp. 30–42 (1969).
Robert C. Daley and Jack B. Dennis, Virtual memory, processes and sharing in Multics,Comm. of the ACM,11(5):306–312 (May 1968).
Peter Lucas and Kurt Walk, On the formal description of PL/1,Ann. Rev. in Automatic Programming,6:105–182, Pergamon Press (1965).
Dines Bjorner and O. N. Oest, (eds.),Towards a Formal Description of Ada, Lecture Notes in Computer Science, Vol. 98, Springler-Verlag, (1980).
L. L. Constantine and T. O. Barnett, eds.,Modular Programming: Proc. of a National Conf., Information and Systems Press, Cambridge, Massachusetts (1968).
David L. Parnas, On the criteria to be used in decomposing systems into modules,Comm. of ACM,15(12):1053–1058 (December 1972).
Jack B. Dennis, Programming generality, parallelism and computer architecture,Inf. Proc., North-Holland,68:484–492 (1969).
Jack B. Dennis, Modularity,Software Engineering: An Advanced Course, Lecture Notes in Computer Science,30:128–182, Springer-Verlag (1975).
Edsger W. Dijkstra, Co-operating sequential processes, F. Genuys, (ed.),Programming Languages, Academic Press, pp. 43–112 (1968).
Gilles Kahn, The semantics for a simple language for parallel processing,Infor. Processing: Proc. of IFIP Congress, pp. 471–475 (1974).
Suhas S. Patil, Closure properties of interconnections of determinate systems,Record of the Project MAC Conference on Concurrent Systems and Parallel Computation, ACM, pp. 107–116 (1970).
Adele Goldberg and D. Robson,Smalltalk-80: The Language and Its Implementation, Addison-Wesley (1980).
B. Liskov, R. Atkinson, T. Bloom, E. Moss, C. Schaffert, B. Scheifler, and A. Snyder, CLU Reference Manual, Technical Report MIT/LCS/TR-225, MIT Laboratory for Computer Science (October 1979).
Jim Waldo, (ed.),The Evolution of C++, MIT Press (1993).
Jack B. Dennis and Earl C. Van Horn, Programming semantics for multi-programmed computations,Comm. of ACM,9(2):143–155 (February) 1966).
Robert S. Fabry, Capability-based addressing,Comm. of the ACM,17(7):403–412 (July 1974).
Jack B. Dennis, First version of a data-flow procedure language,Proc. of the Colloque sur la Programmation, Lecture Notes in Computer Science,19:362–376, Springer-Verlag (1975).
John B. Johnston, The contour model of block structured processes,Proc. of a Symp. on Data Structures in Programming Languages, ACM, pp. 55–82 (1971).
Daniel M. Berry, Introduction to Oregano,Proc. of a Symp. on Data Structures in Programming Languages, ACM, pp. 171–190 (1971).
J. Arvind and Kim P. Gostelow, The U-Interpreter,IEEE Computer,15(2):42–49 (February 1982).
Jack B. Dennis, On the design and specification of a common base language,Proc. of the Symp. on Computers and Automata, Polytechnic Institute of Brooklyn, pp. 47–74 (1971).
J. B. Dennis, An operational semantics for a language with early completion data structures,Formalization of Programming Concepts. Lecture-Notes in Computer Science,107:260–267, Springer-Verlag (1981).
J. B. Dennis and K. S. Weng, An abstract implementation for concurrent computations with streams,Proc. Int'l. Conf. on Parallel Processing, IEEE Computer Society, pp. 35–45 (1979).
J. B. Dennis, J. E. Stoy, and B. Guharoy, VIM: An experimental multiuser system supporting functional programming,Proc. of the Int'l Workshop on High-Level Computer Architecture, IEEE Computer Society (1984).
K.-S. Weng, An abstract implementation for a generalized data flow language, Technical Report MIT/LCS-TR-228, Laboratory for Computer Science, MIT (1979).
David A. Turner, The semantic elegance of applicative languages,Proc. of ACM Conf. on Functional Programming Languages and Computer Architecture, ACM (1981).
R. S. Nikhil and Arvind, Id: A language with implicit parallelism, Computation Structures Group Memo 305, Laboratory for Computer Science, MIT (1990).
W. B. Ackerman and J. B. Dennis, VAL—A value-oriented algorithmic language, Technical Report 218, Laboratory for Computer Science, MIT (1979).
J. R. McGraw, S. Skedzielewski, S. Allan, D. Grit, R. Oldehoeft, J. R. W. Glauert, I. Dobes, and P. Hohensee, SISAL: Streams and iteration in a single assignment language: Language reference manual, Version 1.2, Technical Report TR M-146, Lawrence Livermore Laboratory, University of California (March 1985).
G. R. Gao, R. Yates, J. B. Dennis, and L. Mullin, An efficient monolithic array constructor,Proc. of the Third Workshop on Languages and Compilers for Parallel Computing, MIT Press (1990).
Paul S. Barth, Rishiyur S. Nikhil, and J. Arvind,M-structures: Extending a parallel, nonstrict, functional language with state,Proc. of the Fifth ACM Conf. on Functional Programming Languages and Computer Architecture, Lecture Notes in Computer Science,523:538–568, Springer-Verlag (1991).
Arvind and Dean N. Brock, Resource managers in functional programming,J. of Parallel and Distrib. Comput.,1:5–21 (1984).
C. A. R. Hoare, Monitors: an operating system structuring concept,Comm. of ACM,17(10):549–557 (October 1974).
S. K. Skedzielewski and M. L. Welcome, Dataflow graph optimization In IF1,Functional Programming Languages and Computer Architecture,201:17–34, Springer-Verlag (1985).
Richard Johnson and Keshav Pingali, Dependence-based program analysis,Proc. of the ACM SIGPLAN Conf. on Programming Language Design and Implementation, ACM, pp. 78–89 (1993).
Jack B. Dennis, The Paradigm Compiler: Mapping a functional language for the Connection Machine, Horst Simon, (ed.),Scientific Applications of the Connection Machine, World Scientific, Singapore, pp. 301–315 (1989).
Zena M. Ariola and Arvind, P-TAC: A parallel intermediate language,Proc. of the Fourth Int'l. Conf. on Functional Programming Languages and Computer Architecture, ACM, pp. 230–242 (1989).
Inmos Ltd,Transputer Reference Manual, Prentice-Hall (1988).
J. Hayes, T. Mudge, Q. Stout, S. Colley, and J. Palmer. Architecture of a hypercube supercomputer,Proc. of the Tenth Int'l Conf. on Parallel Processing, IEEE, pp. 653–660 (1986).
Michael J. Backerle, Overview of the START (*T) multithreaded computer,Proc. of the 1993 CompCon., pp. 148–156 (1993).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Dennis, J.B. Machines and models for parallel computing. Int J Parallel Prog 22, 47–77 (1994). https://doi.org/10.1007/BF02577792
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02577792