Skip to main content
Log in

Machines and models for parallel computing

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

It is widely believed that superscalar and superpipelined extensions of RISC style architecture will dominate future processor design, and that needs of parallel computing will have little effect on processor architecture. This belief ignores the issues of memory latency and synchronization, and fails to recognize the opportunity to support a general semantic model for parallel computing. Efforts to extend the shared-memory model using standard microprocessors have led to systems that implement no satisfactory model of computing, and present the programmer with a difficult interface on which to build parallel computing applications. A more satisfactory model for parallel computing may be obtained on the basis of functional programming concepts and the principles of modular software construction. We recommend that designs for computers be built on such a general semantic model of parallel computation. Multithreading concepts and dataflow principles can frame the architecture of these new machines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. John L. Hennessy and Norman P. Jouppi, Computer technology and architecture: An evolving interaction,Comm. of the ACM,24(9):18–29 (September 1991).

    Google Scholar 

  2. Harold S. Stone and John Cocke, Computer architecture in the 1990s,Comm. of the ACM,24(9):30–38 (September 1991).

    Google Scholar 

  3. Arvind and Robert A. Iannucci, Two fundamental issues in multiprocessing,Parallel. Computing in Science and Engineering, Lecture Notes in Computer Science,295:61–88, Springer-Verlag (1987).

  4. V. S. Sunderam, PVM: A framework for parallel distributed computing,ACM Trans. on Programming Languages and Systems,1:80–112 (1985).

    Google Scholar 

  5. A. Deshpande and M. Schultz,Efficient parallel programming with Linda,Proc. of Supercomputing'92, IEEE Computer Society and ACM SIGARCH, pp. 238–244 (1992).

  6. W. Daniel Hillis and Guy L. Steele, Jr., Data parallel algorithms,Comm. of the ACM,29(12):18–29 (December 1986).

    Article  Google Scholar 

  7. American National Standards Institute, FORTRAN 90: Draft of the international standard, Technical Report, The FORTRAN Technical Committe of ANSI X3J3 (June 1990).

  8. Eugene Albert, Kathleen Knobe, Joan Lukas, and Guy L. Steele Jr., Compiling Fortran 8x array features for the Connection Machine computer system,Proc. of ACM SIGPLAN Symp. on Parallel Programming (July 1988).

  9. J. Li and Marina Chen, Compiling communication-efficient programs for massively parallel machines,IEEE Trans. on Parallel and Distrib. Syst.,2(3):361–375 (July 1991).

    Article  Google Scholar 

  10. A. Choudhary, G. Fox, S. Ranka, S. Hirandani, K. Kennedy, C. Koebel, and C.-W. Tseng, Compiling Fortran 77D and 90D for mimd distributed-memory machines,Proc. of the Fourth Symp. on the Frontiers of Massively Parallel Computation, ACM and IEEE, pp. 4–11 (1992).

  11. Gary Sabot, A compiler for a massively paralled distributed memory MIMD computer.Proc. of the Fourth Symp. on the Frontiers of Massively Parallel Computation, ACM and IEEE, pp. 12–20, (1992).

  12. D. J. Mavriplis, Raja Das, R. E. Vermeland, and Joel Saltz, Implementation of a parallel unstructured Euler solver on shared and distributed memory architectures,Proc. of Supercomputing'92, IEEE Computer Society and ACM SIGARCH, pp. 132–141 (1992).

  13. S. Thakkar, P. Gifford, and G. Fielland, The Balance multiprocessor system,IEEE Micro (February 1988).

  14. B. R. Rau and C. D. Glaeser, Some scheduling techniques and an easily schedulable horizontal architecture for high performance scientific computing,Proc. of the 14th Ann. Workshop on Microprogramming, pp. 183–198 (1981).

  15. J. A. Fisher, Very long instruction work architectures and the ELI-512,Proc. of the Tenth Int'l. Symp. on Computer Architecture, ACM (1983).

  16. J. B. Dennis, Evolution of “static” dataflow architecture, Jean-Luc Gaudiot and Lubomir Bic. (eds.),Advanced Topics in Dataflow Computing, Prentice-Hall, pp. 35–91 (1991).

  17. J. B. Dennis, G. R. Gao, and K. W. Todd, Modeling the weather with a dataflow supercomputer,IEEE Trans. on Computer,C-33(7):592–603 (July 1984).

    Article  Google Scholar 

  18. John Palmer and Guy L. Steele, Jr, Connection machine model CM-5 system overview,Proc. of the Fourth Symp. on the Frontiers of Massively Parallel Computation, ACM and IEEE, pp. 474–483 (1992).

  19. R. S. Nikhil, G. M. Papadopoulos, and Arvind,*T: A multithreaded massively parallel architecture,Proc. of the 19th Int's Symp. on Computer Architecture, pp. 156–167 (1992).

  20. Harry F. Jordan, HEP architecture: Programming and performance, Janusz S. Kowalik, (ed.),Parallel MIMD Computation: The HEP Supercomputer and its Applications, The MIT Press, pp. 1–40 (1985).

  21. Robert Alverson, David Callahan, Daniel Cummings, Brian Koblenz, Allan Proterfield, and Burton Smith, The Tera computer system,Proc. of the Int'l Conf. on Supercomputing, ACM, pp. 1–6 (1990).

  22. G. M. Papadopoulos and D. E. Culler, Monsoon: An explicit token-store architecture,Proc. of the 17th Ann. Int'l. Symp. of Comput. Architecture, ACM, pp. 82–91 (1990).

  23. Gregory M. Papadopoulos and Kenneth R. Traub, Multithreading: A revisionist view of dataflow architectures,The 18th Ann. Int'l Symp. on Computer Architecture, ACM, pp. 342–351 (1991).

  24. L. Lamport, How to make a multiprocessor that correctly executes multiprocess programs,IEEE Trans. on Comput.,C-28:690–691 (September 1979).

    MATH  Google Scholar 

  25. Philip Bitar, The weakest memory-access order.J. of Parallel and Distrib. Comput.,15:305–331 (1992).

    Article  MATH  Google Scholar 

  26. James Archibald and Jean-Loup Baer, Cache coherence protocols: Evaluation using a multiprocessor simulation model,ACM Trans. on Comput. Syst.,4:273–298 (November 1986).

    Article  Google Scholar 

  27. D. Lenoski,J. Laudon, T. Joe, K. Gharachorloo, A. Gupta, and J. Hennessy, The directorybased cache coherence protocol for the DASH multiprocessor,Proc. of the 17th Int'l Symp. on Comput. Architecture, ACM, pp. 148–159 (1990).

  28. Erik Hagersten, Anders Landin, and Seif Haridi, DDM: A cache-only memory architecture,IEEE Computer,25(9):44–54 (September 1992).

    Google Scholar 

  29. J. B. Dennis and G. R. Gao, An efficient pipelined dataflow processor architecture,Proc. of the Supercomputing Conf., IEEE Computer Society and ACM SIGARCH, pp. 368–373 (1988).

  30. A. Bensoussan, Charles T. Clingen, and Robert C. Daley, The Multics virtual memory,Proc. of the Second Symp. on Operating System Principles, ACM, pp. 30–42 (1969).

  31. Robert C. Daley and Jack B. Dennis, Virtual memory, processes and sharing in Multics,Comm. of the ACM,11(5):306–312 (May 1968).

    Article  MATH  Google Scholar 

  32. Peter Lucas and Kurt Walk, On the formal description of PL/1,Ann. Rev. in Automatic Programming,6:105–182, Pergamon Press (1965).

    Google Scholar 

  33. Dines Bjorner and O. N. Oest, (eds.),Towards a Formal Description of Ada, Lecture Notes in Computer Science, Vol. 98, Springler-Verlag, (1980).

  34. L. L. Constantine and T. O. Barnett, eds.,Modular Programming: Proc. of a National Conf., Information and Systems Press, Cambridge, Massachusetts (1968).

    Google Scholar 

  35. David L. Parnas, On the criteria to be used in decomposing systems into modules,Comm. of ACM,15(12):1053–1058 (December 1972).

    Article  Google Scholar 

  36. Jack B. Dennis, Programming generality, parallelism and computer architecture,Inf. Proc., North-Holland,68:484–492 (1969).

    Google Scholar 

  37. Jack B. Dennis, Modularity,Software Engineering: An Advanced Course, Lecture Notes in Computer Science,30:128–182, Springer-Verlag (1975).

  38. Edsger W. Dijkstra, Co-operating sequential processes, F. Genuys, (ed.),Programming Languages, Academic Press, pp. 43–112 (1968).

  39. Gilles Kahn, The semantics for a simple language for parallel processing,Infor. Processing: Proc. of IFIP Congress, pp. 471–475 (1974).

  40. Suhas S. Patil, Closure properties of interconnections of determinate systems,Record of the Project MAC Conference on Concurrent Systems and Parallel Computation, ACM, pp. 107–116 (1970).

  41. Adele Goldberg and D. Robson,Smalltalk-80: The Language and Its Implementation, Addison-Wesley (1980).

  42. B. Liskov, R. Atkinson, T. Bloom, E. Moss, C. Schaffert, B. Scheifler, and A. Snyder, CLU Reference Manual, Technical Report MIT/LCS/TR-225, MIT Laboratory for Computer Science (October 1979).

  43. Jim Waldo, (ed.),The Evolution of C++, MIT Press (1993).

  44. Jack B. Dennis and Earl C. Van Horn, Programming semantics for multi-programmed computations,Comm. of ACM,9(2):143–155 (February) 1966).

    Article  MATH  Google Scholar 

  45. Robert S. Fabry, Capability-based addressing,Comm. of the ACM,17(7):403–412 (July 1974).

    Article  Google Scholar 

  46. Jack B. Dennis, First version of a data-flow procedure language,Proc. of the Colloque sur la Programmation, Lecture Notes in Computer Science,19:362–376, Springer-Verlag (1975).

  47. John B. Johnston, The contour model of block structured processes,Proc. of a Symp. on Data Structures in Programming Languages, ACM, pp. 55–82 (1971).

  48. Daniel M. Berry, Introduction to Oregano,Proc. of a Symp. on Data Structures in Programming Languages, ACM, pp. 171–190 (1971).

  49. J. Arvind and Kim P. Gostelow, The U-Interpreter,IEEE Computer,15(2):42–49 (February 1982).

    Google Scholar 

  50. Jack B. Dennis, On the design and specification of a common base language,Proc. of the Symp. on Computers and Automata, Polytechnic Institute of Brooklyn, pp. 47–74 (1971).

  51. J. B. Dennis, An operational semantics for a language with early completion data structures,Formalization of Programming Concepts. Lecture-Notes in Computer Science,107:260–267, Springer-Verlag (1981).

  52. J. B. Dennis and K. S. Weng, An abstract implementation for concurrent computations with streams,Proc. Int'l. Conf. on Parallel Processing, IEEE Computer Society, pp. 35–45 (1979).

  53. J. B. Dennis, J. E. Stoy, and B. Guharoy, VIM: An experimental multiuser system supporting functional programming,Proc. of the Int'l Workshop on High-Level Computer Architecture, IEEE Computer Society (1984).

  54. K.-S. Weng, An abstract implementation for a generalized data flow language, Technical Report MIT/LCS-TR-228, Laboratory for Computer Science, MIT (1979).

  55. David A. Turner, The semantic elegance of applicative languages,Proc. of ACM Conf. on Functional Programming Languages and Computer Architecture, ACM (1981).

  56. R. S. Nikhil and Arvind, Id: A language with implicit parallelism, Computation Structures Group Memo 305, Laboratory for Computer Science, MIT (1990).

  57. W. B. Ackerman and J. B. Dennis, VAL—A value-oriented algorithmic language, Technical Report 218, Laboratory for Computer Science, MIT (1979).

  58. J. R. McGraw, S. Skedzielewski, S. Allan, D. Grit, R. Oldehoeft, J. R. W. Glauert, I. Dobes, and P. Hohensee, SISAL: Streams and iteration in a single assignment language: Language reference manual, Version 1.2, Technical Report TR M-146, Lawrence Livermore Laboratory, University of California (March 1985).

  59. G. R. Gao, R. Yates, J. B. Dennis, and L. Mullin, An efficient monolithic array constructor,Proc. of the Third Workshop on Languages and Compilers for Parallel Computing, MIT Press (1990).

  60. Paul S. Barth, Rishiyur S. Nikhil, and J. Arvind,M-structures: Extending a parallel, nonstrict, functional language with state,Proc. of the Fifth ACM Conf. on Functional Programming Languages and Computer Architecture, Lecture Notes in Computer Science,523:538–568, Springer-Verlag (1991).

  61. Arvind and Dean N. Brock, Resource managers in functional programming,J. of Parallel and Distrib. Comput.,1:5–21 (1984).

    Article  Google Scholar 

  62. C. A. R. Hoare, Monitors: an operating system structuring concept,Comm. of ACM,17(10):549–557 (October 1974).

    Article  MATH  Google Scholar 

  63. S. K. Skedzielewski and M. L. Welcome, Dataflow graph optimization In IF1,Functional Programming Languages and Computer Architecture,201:17–34, Springer-Verlag (1985).

    Google Scholar 

  64. Richard Johnson and Keshav Pingali, Dependence-based program analysis,Proc. of the ACM SIGPLAN Conf. on Programming Language Design and Implementation, ACM, pp. 78–89 (1993).

  65. Jack B. Dennis, The Paradigm Compiler: Mapping a functional language for the Connection Machine, Horst Simon, (ed.),Scientific Applications of the Connection Machine, World Scientific, Singapore, pp. 301–315 (1989).

    Google Scholar 

  66. Zena M. Ariola and Arvind, P-TAC: A parallel intermediate language,Proc. of the Fourth Int'l. Conf. on Functional Programming Languages and Computer Architecture, ACM, pp. 230–242 (1989).

  67. Inmos Ltd,Transputer Reference Manual, Prentice-Hall (1988).

  68. J. Hayes, T. Mudge, Q. Stout, S. Colley, and J. Palmer. Architecture of a hypercube supercomputer,Proc. of the Tenth Int'l Conf. on Parallel Processing, IEEE, pp. 653–660 (1986).

  69. Michael J. Backerle, Overview of the START (*T) multithreaded computer,Proc. of the 1993 CompCon., pp. 148–156 (1993).

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dennis, J.B. Machines and models for parallel computing. Int J Parallel Prog 22, 47–77 (1994). https://doi.org/10.1007/BF02577792

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02577792

Key words

Navigation