Skip to main content
Springer Nature Link
Log in

Languages for Parallel Processing

  • Chapter
Handbook on Parallel and Distributed Processing

Part of the book series: International Handbooks on Information Systems ((INFOSYS))

  • 969 Accesses

Summary

This chapter is concerned with programming languages for parallel processing. We first review some basic principles and then use a series of four case studies to illustrate the practical application of these principles. These case studies involve representative systems based on new programming paradigms, explicitly parallel extensions to existing sequential languages, data parallelism, and messagepassing libraries, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ackerman, W. B., Data flow languages, Computer 15, 1982, 15–25.

    Article  Google Scholar 

  2. Agha, G., ACTORS: A Model of Concurrent Computation in Distributed Systems, MIT Press, Cambridge, Massachusetts, 1986.

    Google Scholar 

  3. Agrawal, G., Sussman, A., Saltz, J., Compiler and runtime support for structured and block structured applications, Proc. of Supercomputing, 1993, 578–587.

    Google Scholar 

  4. Albert, E., Lukas, J., Steele, G., Data parallel computers and the FORALL statement, Journal of Parallel and Distributed Computing 13, 1991, 185–192.

    Article  Google Scholar 

  5. America, P., POOL-T: A parallel object-oriented language, ObjectOriented Concurrent Programming, A. Yonezawa et al. (eds.), MIT Press, Cambridge, MA, 1988, 199–220.

    Google Scholar 

  6. Anderson, J., Lam, M., Global optimizations for parallelism and locality on scalable parallel machines, Proc. of the SIGPLAN ‘83 Conference on Program Language Design and Implementation, Albuquerque, New Mexico, 1993.

    Google Scholar 

  7. Armstrong, J., Virding, S., Programming telephony, Strand: New Concepts in Parallel Programming, Prentice-Hall, Englewood Cliffs, New Jersey, 1989.

    Google Scholar 

  8. Auerbach, J., Goldberg, A., Goldszmidt, G., Gopal, A., Kennedy, M., Rao, J., Russell, J., Concert/C: A language for distributed programming, Winter 194 USENIX Conference, Usenix Association, 1994.

    Google Scholar 

  9. Bal, H. E., Steiner, J. E., Tanenbaum, A. S., Programming languages for distributed computing systems, ACM Computing Surveys 21, 1989, 261–322.

    Article  Google Scholar 

  10. Bala, V., Kipnis, S., Process groups: A mechanism for the coordination of and communication among processes in the Venus collective communication library, Technical report, IBM T.J. Watson Research Center, 1992.

    Google Scholar 

  11. Bala, V., Kipnis, S., Rudolph, L., Snir, M., Designing efficient, scalable, and portable collective communication libraries, Technical report, IBM T.J.Watson Research Center, 1992.

    Google Scholar 

  12. Balay, S., Gropp, W., McInnes, L., Smith, B., Efficient management of parallelism in object oriented numerical software libraries, E. Arge, A. M. Bruaset, H. P. Langtangen (eds.), Modern Software Tools in Scientific Computing, Birkhauser Press, 1997, 163–202.

    Google Scholar 

  13. Banerjee, U., Dependence Analysis for Supercomputing, Kluwer Academic Publishers, Dordrecht, the Netherlands, 1988.

    Book  Google Scholar 

  14. Blelloch, G., Vector Models for Data-Parallel Computing, MIT Press, Cambridge, Massachusetts, 1990.

    Google Scholar 

  15. Bodin, F., Beckman, P., Gannon, D. B., Narayana, S., Yang, S., Distributed pC++: Basic ideas for an object parallel language, Proc. Supercomputing 1991, 273–282.

    Google Scholar 

  16. Boyle, J., Butler, R., Disz, T., Glickfeld, B., Lusk, E., Overbeek, R., Patterson, J., Stevens, R., Portable Programs for Parallel Processors, Holt, Rinehart and Winston, Inc., 1987.

    Google Scholar 

  17. Bozkus, Z., Choudhary, A., Fox, G., Haupt, T., Ranka, S., Fortran 90D/HPF compiler for distributed memory MIMD computers: Design, implementation, and performance results, Proc. Supercomputing IEEE Computer Society Press, 1993.

    Google Scholar 

  18. Butler, R., Butler, T., Foster, I., Karonis, N., Olson, R., Overbeek, R., Pfluger, N., Price, M., Tuecke, S., Aligning genetic sequences, Strand: New Concepts in Parallel Programming, Prentice-Hall, Englewood Cliffs, New Jersey, 1989.

    Google Scholar 

  19. Butler, R., Lusk, E., Monitors, message, and clusters: The p4 parallel programming system, Parallel Computing 20, 1994, 547–564.

    Google Scholar 

  20. Callahan, D., Kennedy, K., Compiling programs for distributed- memory multiprocessors, J. Supercomputing 2, 1988, 151–169.

    Article  Google Scholar 

  21. Cann, D., Feo, J., DeBoni, T., Sisal 1,2: High performance applicative computing, Proc. of the Symp. Parallel and Distributed Processing, IEEE Computer Society Press, 1990, 612–616.

    Google Scholar 

  22. Carriero, N., Gelernter, D., How to write parallel programs: A guide to the perplexed, Computing Surveys 21, 1989, 323–357.

    Article  Google Scholar 

  23. Carriero, N., Gelernter, D., Linda in context, Communications of the ACM 32, 1989, 444–458.

    Article  Google Scholar 

  24. Chandra, R., Gupta, A., Hennessy, J., COOL: An object-based language for parallel programming, Computer 27, 1994, 14–26.

    Article  Google Scholar 

  25. Chandy, K. M., Kesselman, C., CC++: A declarative concurrent object-oriented programming notation, Research Directions in Concurrent Object-Oriented Programming, MIT Press, Cambridge, Massachusetts, 1993.

    Google Scholar 

  26. Chandy, K. M., Misra, J., Parallel Program Design, Addison-Wesley, 1988.

    Google Scholar 

  27. Chandy, K. M., Taylor, S., An Introduction to Parallel Programming, Jones and Bartlett, 1991.

    Google Scholar 

  28. Chapman, B., Mehrotra, P., Zima, H., Programming in Vienna Fortran, Scientific Programming 1, 1992, 31–50.

    Google Scholar 

  29. Church, A., Rosser, J. B., Some properties of conversion, Trans. American Math. Soc., 39, 1936, 472–482.

    Google Scholar 

  30. Clark, K., Gregory, S., A relational language for parallel programming, Proc. of the 1981 ACM Conf. on Functional Programming Languages and Computer Architectures, 1981, 171–178.

    Google Scholar 

  31. Clocksin, W., Mellish, C., Programming in Prolog, Springer-Verlag, Berlin, 1981.

    Google Scholar 

  32. Cole, M., Algorithmic Skeletons: Structured Management of Parallel Computation, MIT Press, Cambridge, Massachusetts, 1989.

    Google Scholar 

  33. Dijkstra, E., Guarded commands, nondeterminacy and the formal derivation of programs, Communications of the ACM 18, 1975, 453–457.

    Article  Google Scholar 

  34. Dongarra, J., Pozo, R., Walker, D., ScaLAPACK++: An objectoriented linear algebra library for scalable systems, Proc. of the Scalable Parallel Libraries Conf., IEEE Computer Society Press, 1993, 216–223.

    Google Scholar 

  35. Edinburgh Parallel Computing Centre, CHIMP Concepts, University of Edinburgh, 1991.

    Google Scholar 

  36. Edinburgh Parallel Computing Centre, CHIMP Version 1.0 Interface, University of Edinburgh, 1992.

    Google Scholar 

  37. Foster, I., Designing and Building Parallel Programs, Addison Wesley, 1995.

    Google Scholar 

  38. Foster, I., Compositional parallel programming languages, ACM Transactions on Programming Languages and Systems 18, 1996, 454–476.

    Google Scholar 

  39. Foster, I., Avalani, B., Choudhary, R., Xu, M., A compilation system that integrates high performance Fortran and Fortran M, Proc. of the 199.4 Scalable High Performance Computing Conf., IEEE Computer Society Press, 1994.

    Google Scholar 

  40. Foster, I., Chandy, K. M., Fortran M: A language for modular parallel programming, Journal of Parallel and Distributed Computing 26, 1995, 24–35.

    Article  Google Scholar 

  41. Foster, I., Kesselman, C., eds., The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufmann, 1999.

    Google Scholar 

  42. Foster, I., Kesselman, C., Taylor, S., Concurrency: Simple concepts and powerful tools, Computer Journal 33, 1990, 501–550.

    Article  Google Scholar 

  43. Foster, I, Kohr, D. R. Jr., Krishnaiyer, R., Choudhary, A., A library-based approach to task parallelism in a data-parallel language, Journal of Parallel and Distributed Computing 45, 1998, 148158.

    Google Scholar 

  44. Foster, I., Olson, R., Tuecke, S., Productive parallel programming:.The PCN approach, Scientific Programming 1, 1992, 51–66.

    Google Scholar 

  45. Foster, I., Taylor, S., Strand: New Concepts in Parallel Programming, Prentice-Hall, Englewood Cliffs, New Jersey, 1989.

    Google Scholar 

  46. Fox, G., Hiranandani, S., Kennedy, K., Koelbel, C., Kremer, U., Tseng, C., Wu, M., Fortran D language specification, Technical report TR90–141, Dept. of Computer Science, Rice University, 1990.

    Google Scholar 

  47. Fox, G., Furmanski, W., High-performance commodity computing, I.

    Google Scholar 

  48. Foster, C. Kesselman (eds.), The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufmann, 1999, 237–255.

    Google Scholar 

  49. Gannon, G., Grimshaw, A., Object-based approaches, I. Foster, C. Kesselman (eds.), The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufmann, 1999, 205–236.

    Google Scholar 

  50. Gehani, N., Roome, W., The Concurrent C Programming Language, Silicon Press, 1988.

    Google Scholar 

  51. Geist, G. A., Heath, M. T., Peyton, B. W., Worley, P. H., A user’s guide to PICL: A portable instrumented communication library, Technical report TM-11616, Oak Ridge National Laboratory, 1990.

    Google Scholar 

  52. Gregory, S., Parallel Logic Programming in PARLOG, Addison- Wesley, 1987.

    Google Scholar 

  53. Grimshaw, A. S., An introduction to parallel object-oriented programming with Mentat, Technical report 91 07, University of Virginia, 1991.

    Google Scholar 

  54. Gropp, W., Huss-Lederman, S., Lumsdaine, A., Lusk, E., Nitzberg, B., Saphir, W., Snir, M., MPI—The Complete Reference: Volume 2, The MPI-2 Extensions, MIT Press, 1998.

    Google Scholar 

  55. Gropp, W., Lusk, E., Skjellum, A., Using MPI: Portable Parallel Programming with the Message Passing Interface, MIT Press, Cambridge, Massachusetts, 1995.

    Google Scholar 

  56. Halstead, R., MULTILISP: A language for concurrent symbolic computation, ACM Transactions on Programming Languages and Systems 7, 1985, 501–538.

    Google Scholar 

  57. Hatcher, P., Quinn, M., Data-Parallel Programming on MIMD Computers, MIT Press, Cambridge, Massachusetts, 1991.

    Google Scholar 

  58. Hatcher, P., Quinn, M., and others, Data-parallel programming on MIMD computers, IEEE Trans. Parallel and Distributed Syst. 2, 1991, 377–383.

    Google Scholar 

  59. Hempel, R., The ANL/GMD macros (PARMACS) in Fortran for portable parallel programming using the message passing programming model–users’ guide and reference manual, GMD, Germany, 1991.

    Google Scholar 

  60. Hempel, R., Hoppe,H.-C., Supalov, A., PARMACS 6.0 library interface specification, GMD, Germany, 1992.

    Google Scholar 

  61. Henderson, P., Functional Programming, Prentice-Hall, Englewood Cliffs, New Jersey, 1980.

    Google Scholar 

  62. High Performance Fortran Forum, High Performance Fortran language specification, version 1.0, Technical report CRPC-TR92225, Center for Research on Parallel Computation, Rice University, Houston, Texas, 1993.

    Google Scholar 

  63. Hilzer, R. C., Crowl, L. A., A Survey of Sequential and Parallel Implementation Techniques for Functional Programming Languages, Technical report 95–60–05, Department of Computer Science, Oregon State University, 1995.

    Google Scholar 

  64. Hiranandani, S., Kennedy, K., Tseng, C., Compiling Fortran D for MIMD distributed-memory machines, Communications of the ACM 35, 1992, 66–80.

    Article  Google Scholar 

  65. Hoare, C., Communicating sequential processes, Communications of the ACM 21, 1978, 666–677.

    Article  Google Scholar 

  66. Hudak, P., Conception, evolution, and application of functional programming languages, ACM Computing Surveys 21, 1989, 359–411.

    Article  Google Scholar 

  67. Jones, M., Hudak, P., Implicit and explicit parallel programming in Haskell, Research Report YALEU/DCS/RR-982, Yale University, 1993.

    Google Scholar 

  68. Kahn, G., MacQueen, D., Coroutines and networks of parallel processes, Information Processing 77:Proc. of the IFIP Congress, North-Holland, 1977, 993–998.

    Google Scholar 

  69. Karp, A., Babb, R., A comparison of twelve parallel Fortran dialects, IEEE Software 5, 1988, 52–67.

    Article  Google Scholar 

  70. Kelly, P., Functional Programming for Loosely-Coupled Multiprocessors, MIT Press, Cambridge, Massachusetts, 1989.

    Google Scholar 

  71. Kennedy, K., Compilers, languages, and libraries, I. Foster, C. Kesselman (eds.), The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufmann, 1999, 181–204.

    Google Scholar 

  72. Koelbel, C., Loveman, D., Schreiber, R., Steele, G., Zosel, M., The High Performance Fortran Handbook, MIT Press, Cambridge, Massachusetts, 1994.

    Google Scholar 

  73. Kowalski, R., Logic for Problem Solving, North-Holland, 1979.

    Google Scholar 

  74. Lemke, M., Quinlan, D., P++, a parallel C++ array class library for architecture-independent development of structured grid applications, Proc. of the Workshop on Languages, Compilers, and Runtime Environments for Distributed Memory Computers, ACM, 1992.

    Google Scholar 

  75. Loveman, D., High Performance Fortran, IEEE Parallel and Dis- tributed Technology 1, 1993, 25–42.

    Article  Google Scholar 

  76. Lucco, S., Sharp, O., Parallel programming with coordination structures, Proc. of the Eighteenth Annual ACM Symposium on the Principles of Programming Languages, Orlando, Florida, 1991.

    Google Scholar 

  77. McBryan, O., An overview of message passing environments, Parallel Computing 20, 1994, 417–444.

    Article  Google Scholar 

  78. McLennan, B., Functional Programming: Practice and Theory, Addison-Wesley, 1990.

    Google Scholar 

  79. Mehrotra, P., VanRosendale, J., Programming distributed memory architectures using Kali, Advances in Languages and Compilers for Parallel Computing, MIT Press, Cambridge, Massachusetts, 1991.

    Google Scholar 

  80. Message Passing Interface Forum, Document for a Standard Message-Passing Interface, University of Tennessee, Knoxville, Tenn., 1993.

    Google Scholar 

  81. Message Passing Interface Forum, MPI: A Message Passing Interface, Proc. of Supercomputing IEEE Computer Society Press, 1993, 878–883.

    Google Scholar 

  82. Mierowsky, C., Taylor, S., Shapiro, E., Levy, J., Safra, S., Design and implementation of Flat Concurrent Prolog, Technical report CS8509, Weizmann Institute of Science, Rehovot, Israel, 1985.

    Google Scholar 

  83. nCUBE Corporation, nCUBE 2 Programmers Guide, r2.0, 1990.

    Google Scholar 

  84. S.Pakin, M.Lauria, A.Chien, High performance messaging on workstations: Illinois Fast Messages (FM) for Myrinet, Proc. of Super-computing, 1995.

    Google Scholar 

  85. Parasoft Corporation, Express Version 1.0: A communication environment for parallel computers, Parasoft Corp., 1988.

    Google Scholar 

  86. Pierce, P., The NX/2 operating system, Proc. of the 3rd Conf. on Hypercube Concurrent Computers and Applications, ACM Press, 1988, 384–390.

    Google Scholar 

  87. Ringwood, G., PARLOG86 and the dining logicians, Communications of the ACM 31, 1988, 10–25.

    Article  Google Scholar 

  88. Rogers, A., Pingali, K., Process decomposition through locality of reference, Proc. of the SIGPLAN ‘89 Conference on Program Language Design and Implementation, ACM, 1989.

    Google Scholar 

  89. Rosing, M., Schnabel, R., Weaver, R., The DINO parallel programming language, Technical report CU-CS-501–90, Computer Science Department, University of Colorado at Boulder, Boulder, Colorado, 1990.

    Google Scholar 

  90. Saltz, J., Berryman, H., Wu, J., Multiprocessors and runtime compilation, Concurrency: Practice E4 Experience 3, 1991, 573–592.

    Article  Google Scholar 

  91. Seitz, C. L., The cosmic cube, Communications of the ACM28, 1985, 22–33.

    Google Scholar 

  92. Shapiro, E., Concurrent Prolog: Collected Papers, MIT Press, Cambridge, Massachusetts, 1987.

    Google Scholar 

  93. Shapiro, E., The family of concurrent logic programming languages, ACM Computing Surveys 21, 1989, 413–510.

    Article  Google Scholar 

  94. Skillicorn, D. B., Talia, D., Models and languages for parallel computation, Computing Surveys 30, 1998, 123–169.

    Article  Google Scholar 

  95. SKjellum, A., Smith, S., Doss, N., Leung, A., Morari, M., The design and evolution of Zipcode, Parallel Computing 20, 1994, 565–596.

    Article  Google Scholar 

  96. Subhlok, J., Stichnoth, J., O’Hallaron, D., Gross, T., Exploiting task and data parallelism on a multicomputer, Proc. of the 4th ACM SIG-PLAN Symposium on Principles and Practice of Parallel Programming, ACM, 1993.

    Google Scholar 

  97. Sunderam, V., PVM: A framework for parallel distributed computing, Concurrency: Practice E4 Experience 2, 1990, 315–339.

    Article  Google Scholar 

  98. Taylor, R., Wilson, P., Process-oriented language meets demands of distributed processing, Electronics, 1982.

    Google Scholar 

  99. Taylor, S., Parallel Logic Programming Techniques, Prentice-Hall, Englewood Cliffs, New Jersey, 1989.

    Google Scholar 

  100. Thinking Machines Corporation, The CM-2 Technical Summary, 1990.

    Google Scholar 

  101. Thinking Machines Corporation, CM Fortran Reference Manual, version 2. 1, 1993.

    Google Scholar 

  102. Trinder, P. W., Hammond, K., Loidl, H.-W., Peyton Jones, S. L.

    Google Scholar 

  103. Algorithm -F strategy = parallelism, Journal of Functional Programming 8, 1998, 23–60.

    Google Scholar 

  104. Trinder, P. W., Hammond, K., Mattson, J. S., Partridge, A. S., Peyton Jones, S. L., GUM: A portable implementation of Haskell, Proc. of Programming Language Design and Implementation, 1996.

    Google Scholar 

  105. Ueda, K., Guarded Horn clauses, Technical Raport TR-103, ICOT, 1985.

    Google Scholar 

  106. Watanabe, T., and others, Reflection in an object-oriented concurrent language SIGPLAN Notices 23, 1998, 306–315.

    Article  Google Scholar 

  107. Wilson, G. V., Lu, P., Parallel Programming Using C++, MIT Press, 1996.

    Google Scholar 

  108. Wolfe, M., Optimizing Supercompilers for Supercomputers, MIT Press, Cambridge, Massachusetts, 1989.

    Google Scholar 

  109. Xu, M., Turner, S., A multi-level time warp mechanism, Proc. Of the 1990 Summer Computer Simulation Conf., Society for Computer Simulation, 1990, 165–170.

    Google Scholar 

  110. Yokote, Y., and others, Concurrent programming in Concurrent Smalltalk, Object-Oriented Concurrent Programming, A. Yonezawa and others (eds.), MIT Press, Cambridge, MA, 1987, 129–158.

    Google Scholar 

  111. Yonezawa, A., and others, Object-Oriented Concurrent Programming, MIT Press, Cambridge, MA., 1987.

    Google Scholar 

  112. Zima, H., Bast, 11.-J., Gerndt, M., SUPERB: A tool for semiautomatic MIMD/SIMD parallelization, Parallel Computing 6, 1988, 1–18.

    Article  Google Scholar 

  113. Zima, H., Chapman, B, Supercompilers for Parallel and Vector Computers, Addison-Wesley, 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Argonne National Laboratory, The University of Chicago, USA

    Ian Foster

Authors
  1. Ian Foster
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Instytut Informatyki, Politechnika Poznanska, ul. Piotrowo 3a, 60-965, Poznań, Poland

    Jacek Błażewicz

  2. Institut für Informatik, Technische Universität Clausthal, Julius Albert Str. 4, D-78678, Clausthal-Zellerfeld, Germany

    Klaus Ecker

  3. Institut Fourier, LMC, 100 Rue des Mathematiques, BP 53X, F-38041, Grenoble Cedex 9, France

    Brigitte Plateau

  4. Institut National Polytechniques de Grenoble, 46, avenue Felix Viallet, F-38031, Grenoble Cedex, France

    Denis Trystram

Rights and permissions

Reprints and permissions

Copyright information

© 2000 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Foster, I. (2000). Languages for Parallel Processing. In: Błażewicz, J., Ecker, K., Plateau, B., Trystram, D. (eds) Handbook on Parallel and Distributed Processing. International Handbooks on Information Systems. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-04303-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-04303-5_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-08571-0

  • Online ISBN: 978-3-662-04303-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics