Skip to main content
SpringerLink
Log in

Models for practical parallel computation

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

A major reason for the lack of practical use of parallel computers has been the absence of a suitable model of parallel computation. Many existing models are either theoretical or are tied to a particular architecture. A more general model must be architecture independent, must realistically reflect execution costs, and must reduce the cognitive overhead of managing massive parallelism. A growing number of models meeting some of these goals have been suggested. We discuss their properties and relative strengths and weaknesses. We conclude that data parallelism is a style with much to commend it, and discuss the Bird-Meertens formalism as a coherent approach to data parallel programming.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D. B. Skillicorn, Architecture-Independent Parallel Computation,IEEE Computer,23(12):38–51 (December 1990).

    Google Scholar 

  2. S. Peyton-Jones,Implementation of Functional Programming Languages, Prentice-Hall (1987).

  3. J. R. Glauert, J. R. Kennaway, and M. R. Sleep, DACTL: A Computational Model and Compiler Target Language Based on Graph Reduction, Technical Report SYS-C87-03, University of East Anglia (1987).

  4. M. Danelutto and M. Vanneschi, VLIW-in-the-large: A Model for Fine Grain Parallelism Exploitation on Distributed Memory Multiprocessors,Proc of the 23rd Annual Workshop on Microprogramming and Microarchitecture, pp. 7–16 (November 1990).

  5. S. L. Peyton-Jones, C. Clack, and N. Harris, GRIP—A Parallel Graph Reduction Machine, Technical Report, Department of Computer Science, University of London (1987).

  6. I. Watson, V. Woods, P. Watson, R. Banach, M. Greenberg, and J. Sargeant, Flagship: A Parallel Architecture for Declarative Programming,Proc. of the 15th Annual Int'l. Symp. on Computer Archit. pp. 124–130 (May 1988). Also appeared as University of Manchester, Department of Computer Science, Flagship Project, Report FS/MU/IW/025-87 (November 1987).

  7. P. Kelly,Functional Programming for Loosely-Coupled Multiprocessors, Pitman (1989).

  8. F. A. Rahbi and G. A. Manson, Experiments with a Transputer-Based Parallel Graph Reduction Machine,Concurrency Practice and Experience (to appear).

  9. J. A. Goguen and T. Winkler, Introducing OBJ3, Technical Report SRI-CSL-88-9, Computer Science Laboratory, SRI International (August 1988).

  10. J. Goguen, S. Leinwand, J. Meseguer, and T. Winkler, The Rewrite Rule Machine 1988, Oxford University Computing Laboratory, Programming Research Group, Technical Monograph PRG-76 (1989).

  11. K. M. Chandy and J. Misra,Parallel Program Design: A Foundation, Addison-Wesley (1988).

  12. R. J. R. Back, A Method for Refining Atomicity in Parallel Algorithms,PARLE '89, Parallel Architectures and Languages Europe, Springer Lecture Notes in Computer Science,366:199–216 (June 1989).

    Google Scholar 

  13. G. M. Papadopoulos, Implementation of a General-Purpose Dataflow Multiprocessor, Technical Report TR-432, MIT Laboratory for Computer Science (August 1988).

  14. T. Yuba, T. Shinada, Y. Yamaguchi, K. Hiraki and S. Sakai, Dataflow Computer Development in Japan,Int'l. Conf. on Supercomputing, ACM Press, Amsterdam, pp. 140–147 (June 1990). Appeared asComputer Architecture News, Vol. 18, No. 3.

    Google Scholar 

  15. S. Ahuja, N. Carriero, D. Gelernter, and V. Krishnswamy, Matching Languages and Hardware for Parallel Computation in the Linda Machine,IEEE Transactions on Computers,37(8):921–929 (August 1988).

    Google Scholar 

  16. Steven Ericsson Zenith, Programming with Ease,Centre de Recherche en Informatique, Ecole Nationale Supérieure des Mines de Paris (September 1991).

  17. G. Agha,Actors: A Model of Concurrent Computation in Distributed Systems, MIT Press (1986).

  18. F. Baude and G. Vidal-Naquet, Actors as a Parallel Programming Model,Proc. of 8th Symp. on Theoretical Aspects of Computer Science, Springer, Lecture Notes in Computer Science Vol. 480, 1991.

  19. W. J. Dally and D. S. Wills, Universal Mechanisms for Concurrency,PARLE '89, Parallel Architectures and Languages Europe, Springer-Verlag Lecture Notes in Computer Science,365:19–33 (June 1989).

    Google Scholar 

  20. D. S. Wills, Pi: A Parallel Architecture Interface for Multi-Model Execution Technical Report AI-TR-1245, MIT Artificial Intelligence Laboratory, (1990).

  21. M. O. Noakes and W. J. Dally, System Design of the J-machine,Proc. of the Sixth MIT Conf. on Adv. Res. in VLSI, MIT Press, pp. 179–194 (1990).

  22. A. Gottlieb, B. Lubachevsky, and L. Rudolph, Basic Techniques for the Efficient Coordination of Large Numbers of Cooperating Sequential Processes,ACM Transactions of Programming Languages and Systems, Vol. 5, No. 2 (April 1983).

  23. Peter A. Buhr and Richard A. Stroobosscher, The μSystem: Providing Light-Weight Concurrency on Shared-Memory Multiprocessor Computers Running UNIX,Software Practice and Experience,20(9):929–963 (September 1990).

    Google Scholar 

  24. Peter A. Buhr, Hamish I. Macdonald, and Richard A. Stroobosscher, μSystem Reference Manual, Version 4.3.3, Technical Report, Department of Computer Science, University of Waterloo, Ontario, Canada, (March 1991).

    Google Scholar 

  25. J. E. Faust and H. M. Levy, The Performance of an Object-Oriented Threads Package, N. Meyrowitz, (ed.),OOPSLA ECOOP90 Proceedings, SIGPLAN Notices,25(10):278–288. (October 1990).

    Google Scholar 

  26. G. Jones and M. Goldsmith,Programming in Occam2, Prentice-Hall (1988).

  27. C. A. R. Hoare,Communicating Sequential Processes, Prentice-Hall International Series in Computer Science (1985).

  28. H. E. Bal, J. G. Steiner, and A. S. Tanenbaum, Programming Languages for Distributed Computing Systems,Computing Surveys,21(3):261–322 (September 1989).

    Google Scholar 

  29. L. G. Valiant, General. Purpose Parallel Architectures, J. van Leeuwen, (ed.),Handbook of Theoretical Computer Science, Vol. A., Elsevier Science Publishers and MIT Press (1990).

  30. L. G. Valiant, A Bridging Model for Parallel Computation,Communications of the ACM,33(8):103–111 (August 1990).

    Google Scholar 

  31. W. F. McColl, Parallel Algorithms and Architectures, G. A. van Zee and J. G. G. van de Vorst, (eds.),Parallel Computing 88, Springer Lecture Notes in Computer Science,384:1–22 (June 1988).

    Google Scholar 

  32. G. Blelloch, Scans as Primitive Parallel Operations,Proc. of the Int'l. Conf. on Parallel Processing, pp. 355–362 (August 1987).

  33. R. E. Ladner and M. J. Fisher, Parallel prefix Computation,J. of the ACM,27(4):831–838 (1980).

    Google Scholar 

  34. A. G. Ranade, Fluent Parallel Computation. PhD Thesis, Yale University, (1989).

  35. P. de la Torre and C. P. Kruskal, Towards a Single Model of Efficient Computation in Real Parallel Machines,Parallel Architectures and Languages Europe PARLE91, Springer Lecture Notes in Computer Science (June 1991).

  36. T. Heywood and S. Ranka, A Practical Hierarchical Model of Parallel Computation: The Model, Technical Report SU-CIS-91-06, School of Computer and Information Science, Syracuse University (1991).

  37. T. Heywood and S. Ranka, A. Practical Hierarchical Model of Parallel Computation: Binary Tree and FFT Graph Algorithms, Technical Report SU-CIS-91-07, School of Computer and Information Science, Syracuse University (1991).

  38. C. P. Kruskal, L. Rudolph, and M. Snir, A Complexity Theory of Efficient Parallel Algorithms,Theoretical Computer Science,71:95–132 (1990).

    Google Scholar 

  39. M. Danelutto, A Massively Parallel Architecture Using VLIW for Fine Grain Parallelism Exploitation, PhD Thesis, Dipartimento di Informatica, Università di Pisa (March 1990).

  40. M. Danelutto and M. Vanneschi, VLIW-in-the-large: A Model for Fine Grain Parallelism Exploitation on Distributed Memory Multiprocessors,Proc. of the 23rd Annual Workshop on Microprogramming and Microarchitecture, pp. 7–16 (November 1990).

  41. D. R. Smith and M. R. Lowry, Algorithm Theories and Design Tactics,Mathematics of Program Construction, Springer-Verlag Lecture Notes in Computer Science,375:379–398 (June 1989).

    Google Scholar 

  42. M. Cole,Algorithmic Skeletrons Management of Parallel Computation, Research Monographs in Parallel and Distributed Computing, Pitman (1989).

  43. M. Danelutto, R. Di Meglio, S. Pelagatti, and M. Vanneschi, High-Level Language Constructs for Massively Parallel Computing, Technical Report, Hewlett Packard Pisa Science Center, HPL-PSC-90-19 (1990).

  44. M. Danelutto, S. Pelagatti, and M. Vanneschi, High-Level Languages for Easy Massively Parallel Computing, Technical Report, Hewlett Packard Pisa Science Center, HPL-PSC-91-16 (1991).

  45. F. Baiardi, M. Danelutto, M. Jazayeri, S. Pelagatti, and M. Vanneschi, Architectural Models and Design Methodologies for General-Purpose Highly-Parallel Computers,IEEE CompEuro 91-Advanced Computer Technology, Reliable Systems and Applications (May 1991).

  46. G. E. Blelloch and S. Chatterjee, VCODE: A Data-Parallel Intermediate Language,Proc. of the 3rd Symp. on the Frontiers of Massively Parallel Computation, pp. 471–480 (October 1990).

  47. G. E. Blelloch,Vector Models for Data-Parallel Computing, MIT Press, (1990).

  48. G. Sabot,The Paralation Model: Architecture-Independent Parallel Programming, MIT Press (1989).

  49. K. Knobe, J. D. Lukas, and G. L. Steele, Data Optimization: Allocation of Arrays to Reduce Communication on SIMD Machines,Journal of Parallel and Distributed Computing,8:102–118 (1990).

    Google Scholar 

  50. E. Albert, K. Knobe, J. Lukas, and G. L. Steele, Compiling Fortran 8× features for the Connection Machine Computer System, Technical Report CA-8801-0401, Compass (1988). Also appeared inSIGPLAN Notices, Volume 23 (September 1988).

  51. K. Knobe and V. K. Natarajan, Data Optimization: Minimizing Residual Interprocessor Data Motion on SIMD Machines, Technical Report CA-9006-2001, Compass (1990). Also appeared inThird Symp. on Frontiers of Massively Parallel Computation (October, 1990).

  52. K. Knobe, J. Lukas, and G. L. Steele, Massively Parallel Data Optimization, Technical Report CA-8810-1401, Compass (1988). Also appeared inSecond Symp. on Frontiers of Massively Parallel Computation (October, 1988).

  53. J. F. Prins, A Framework for Efficient, Execution of Array-Based Languages and SIMD Computers,Third Symp. on the Frontiers of Massively Parallel Computation, IEEE (1990).

  54. T. More, Notes on the Diagrams, Logic, and Operations of Array Theory, Technical Report G230-2137, IBM Cambridge Scientific Center (1981).

  55. T. More, On the Development of Array Theory, Technical Report, IBM Cambridge Scientific Center Technical Report (1986).

  56. L. M. R. Mullin, A Mathematics of Arrays, PhD Thesis, Syracuse University, (December 1988).

  57. R. S. Bird, An Introduction to the Theory of Lists, M. Broy, (ed.),Logic of Programming and Calculi of Discrete Design, Springer-Verlag, pp. 3–42 (1987).

  58. R. S. Bird, Algebraic Identities for Program Calculation,The Computer Journal,32(2):122–126 (February 1989).

    Google Scholar 

  59. G. Malcolm, Algebraic Data Types and Program Transformation, PhD Thesis, Rijksuniversiteit Groningen (September 1990).

  60. J. M. Spivey, A Categorical Approach to the Theory of Lists,Mathmatics of Program Construction, Springer-Verlag Lecture Notes in Computer Science,375:399–408 (June 1989).

    Google Scholar 

  61. T. Hagino, A Typed Lambda Calculus with Categorical Type Constructors, D. H. Pitt, A. Poigné, and D. E. Rydeheard, (eds.),Category Theory and Computer Science, Springer Lecture Notes in Computer Science,283:140–157 (1987).

    Google Scholar 

  62. R. S. Bird, A Calculus of Functions for Program Derivation, Oxford University Programming Research Group Monograph PRG-64 (1987).

  63. R. S. Bird, Lectures on Constructive Functional Programming, Oxford University Programming Research Group Monograph PRG-69 (1988).

  64. J. Dennis, Data Flow Supercomputers,Computer, pp. 48–56 (1980).

  65. J. R. Gurd, C. C. Kirkham, and I. Watson, The Manchester Prototype Dataflow Computer,CACM,28(1):34–52 (January 1985).

    Google Scholar 

  66. G. Hains, Multidimensional Arrays as Parallel Data Structures,Abstract Machine Models for Highly Parallel Computers, University of Leeds, United Kingdom, (March 1991).

    Google Scholar 

  67. G. Hains and L. M. R. Mullin,An Algebra of Multidimensional Arrays, preprint (1991).

Download references

Author information

Authors and Affiliations

  1. Department of Computing and Information Science, Queen's University, Kingston, Canada

    D. B. Skillicorn

Authors
  1. D. B. Skillicorn
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

This work was supported by the Natural Sciences and Engineering Research Council of Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Skillicorn, D.B. Models for practical parallel computation. Int J Parallel Prog 20, 133–158 (1991). https://doi.org/10.1007/BF01407840

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01407840

Key Words

Search

Navigation