Skip to main content
SpringerLink
Log in

Automatic Data Layout for Distributed-Memory Machines in the D Programming Environment

  • Chapter
Book cover Automatic Parallelization

Abstract

Although distributed-memory message-passing parallel computers are among the most cost-effective high performance machines available, scientists find them extremely difficult to program. Most programmers feel uncomfortable working with a distributed-memoryprogramming model that requires explicit management of local name spaces. To address this problem, researchers have proposed using languages based on a global name space annotated with directives specifying how the data should be mapped onto a distributed memory machine. Using these annotations, a sophisticated compiler can automatically transform a code into a message-passing program suitable for execution on a distributed-memory machine. The Fortran77D and Fortran9OD languages support this programming style. Given a Fortran D program, the compiler uses data layout directives to automatically generate a single-program, multiple data (SPMD) node program for a given distributed-memory target machine.

This research was supported by the Center for Research on Parallel Computation (CRPC), a Science and Technology Center funded by NSF through Cooperative Agreement Number CCR-9120008. This work was also sponsored by DARPA under contract #DABT63-92-C-0038, and the IBM corporation. The content of this paper does not necessarily reflect the position or the policy of the U.S. Government and no official endorsement should be inferred.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. A.V.Aho,R.Sethi,and J.UIIman.Compilers:Principles,Techniques and Tools. Addison-Wesley, Reading,MA,second edition,1986

    Google Scholar 

  2. J. Anderson and M. Lam. Global optimizations for parallelism and locality on scalable parallel machines. In Proceedings of the SIGPLAN ‘83 Conference on Program Language Design and Implementation, Albuquerque, NM, June 1993.

    Google Scholar 

  3. V. Balasundaram, G. Fox, K. Kennedy, and U. Kremer. An interactive environment for data partitioning and distribution. In Proceedings of the 5th Distributed Memory Computing Conference, Charleston, SC, April 1990.

    Google Scholar 

  4. V. Balasundaram, G. Fox, K. Kennedy, and U. Kremer. A static performance estimator to guide data partitioning decisions. In Proceedings of the Third ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Williamsburg, VA, April 1991.

    Google Scholar 

  5. P. Briggs. Register Allocation via Graph Coloring. PhD thesis, Rice University, April 1992.

    Google Scholar 

  6. S. Carr. Memory-Hierarchy Management. PhD thesis, Rice University, September 1992.

    Google Scholar 

  7. B. Chapman, H. Herbeck, and H. Zima. Automatic support for data distribution. In Proceedings of the 6th Distributed Memory Computing Conference, Portland, OR, April 1991.

    Google Scholar 

  8. B. Chapman, P. Mehrotra, and H. Zima. Vienna Fortran - a Fortran language extension for distributed memory multiprocessors. In J. Saltz and P. Mehrotra, editors, Languages, Compilers, and Run-Time Environments for Distributed Memory Machines. North-Holland, Amsterdam, The Netherlands, 1992.

    Google Scholar 

  9. S. Chatterjee, J.R. Gilbert, R. Schreiber, and S-H. Teng. Automatic array alignment in data-parallel programs. In Proceedings of the Twentieth Annual ACM Symposium on the Principles of Programming Languages, Albuquerque, NM, January 1993.

    Google Scholar 

  10. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. The MIT Press, 1990.

    Google Scholar 

  11. G. Fox, S. Hiranandani, K. Kennedy, C. Koelbel, U. Kremer, C. Tseng, and M. Wu. Fortran D language specification. Technical Report TR90–141, Dept. of Computer Science, Rice University, December 1990.

    Google Scholar 

  12. M. Gupta.Automatic Data Partitioning on Distributed Memory Multicomputers. PhD thesis, University of Illinois at Urbana-Champaign, September 1992.

    Google Scholar 

  13. M. Gupta and P. Banerjee. Demonstration of automatic data partitioning techniques for parallelizing compilers on multicomputers. IEEE Transactions on Parallel and Distributed Systems, April 1992.

    Google Scholar 

  14. M. W. Hall, S. Hiranandani, K. Kennedy, and C. Tseng. Interprocedural compilation of Fortran D for MIMD distributed-memory machines. Technical Report TR91–169, Dept. of Computer Science, Rice University, November 1991.

    Google Scholar 

  15. High Performance Fortran Forum. High Performance Fortran language specification, version 1.0. Technical Report CRPC-TR92225, Center for Research on Parallel Computation, Rice University, Houston, TX, May 1993. To appear in Scientific Programming, vol. 2, no.

    Google Scholar 

  16. S. Hiranandani, K. Kennedy, C. Koelbel, U. Kremer, and C. Tseng. An overview of the Fortran D programming system. In Proceedings of the Fourth Workshop on Languages and Compilers for Parallel Computing, Santa Clara, CA, August 1991.

    Google Scholar 

  17. S. Hiranandani, K. Kennedy, and C. Tseng. Compiler optimizations for Fortran Don MIMD distributed-memory machines. In Proceedings of Supercomputing ‘81, Albuquerque, NM, November 1991.

    Google Scholar 

  18. S. Hiranandani, K. Kennedy, and C. Tseng. Evaluation of compiler optimizations for Fortran D on MIMD distributed-memory machines. In Proceedings of the 1992 ACM International Conference on Supercomputing, Washington, DC, July 1992.

    Google Scholar 

  19. [19]D. Hudak and S. Abraham. Compiler techniques for data partitioning of sequentially iterated parallel loops. In Proceedings of the 1990 ACM International Conference on Supercomputing, Amsterdam, The Netherlands, June 1990.

    Google Scholar 

  20. K. Ikudome, G. Fox, A. Kolawa, and J. Flower. An automatic and symbolic parallelization system for distributed memory parallel computers. In Proceedings of the 5th Distributed Memory Computing Conference, Charleston, SC, April 1990.

    Google Scholar 

  21. [21]K. Kennedy and U. Kremer. Initial framework for automatic data layout in Fortran D: A short update on a case study. Technical Report CRPC-TR93–324-S, Center for Research on Parallel Computation, Rice University, July 1993.

    Google Scholar 

  22. K. Knobe, J. Lukas, and G. Steele, Jr. Data optimization: Allocation of arrays to reduce communication on SIMD machines. Journal of Parallel and Distributed Computing, 8 (2): 102–118, February 1990.

    Article  Google Scholar 

  23. K. Knobe, J.D. Lukas, and W.J. Daily. Dynamic alignment on distributed memory systems. In Proceedings of the Third Workshop on Compilers for Parallel Computers, Vienna, Austria, July 1992.

    Google Scholar 

  24. K. Knobe and V. Natarajan. Data optimization: Minimizing residual interprocessor data motion on SIMD machines. In Frontiers90: The 3rd Symposium on the Frontiers of Massively Parallel Computation College Park, MD, October 1990.

    Google Scholar 

  25. C. Koelbel and P. Mehrotra. Compiling global name-space parallel loops for distributed execution. IEEE Transactions on Parallel and Distributed Systems, 2 (4): 440–451, October 1991.

    Google Scholar 

  26. U. Kremer. Automatic data layout for distributed-memory machines. Technical Report CRPC-TR93–299-S, Center for Research on Parallel Computation, Rice University, February 1993. (thesis proposal).

    Google Scholar 

  27. U. Kremer. NP-completeness of dynamic remapping. Technical Report CRPC-TR93–330-S, Center for Research on Parallel Computation, Rice University, August 1993. (also available as D Newsletter #8).

    Google Scholar 

  28. [28]J. Li and M. Chen. Index domain alignment: Minimizing cost of cross-referencing between distributed arrays. In Frontiers90: The 3rd Symposium on the Frontiers of Massively Parallel Computation, College Park, MD, October 1990.

    Google Scholar 

  29. [29]J. Li and M. Chen. Synthesis of explicit communication from shared-memory program references. Technical Report YALEU/DCS/TR-755, Dept. of Computer Science, Yale University, New Haven, CT, May 1990.

    Google Scholar 

  30. J. Li and M. Chen. The data alignment phase in compiling programs for distributed-memory machines. Journal of Parallel and Distributed Computing, 13 (4): 213–221, August 1991.

    Article  Google Scholar 

  31. J. Ramanujam. Compile-time Techniques for Parallel Execution of Loops on Distributed Memory Multiprocessors. PhD thesis, Department of Computer and Information Science, Ohio State University, Columbus, OH, 1990.

    Google Scholar 

  32. J. Ramanujam and P. Sadayappan. A methodology for parallelizing programs for multi-computers and complex memory multiprocessors. In Proceedings of Supercomputing ‘89, Reno, NV, November 1989.

    Google Scholar 

  33. L. Snyder and D. Socha. An algorithm producing balanced partitionings of data arrays. In Proceedings of the 5th Distributed Memory Computing Conference, Charleston, SC, April 1990.

    Google Scholar 

  34. A. Sussman. Model-Driven Mapping onto Distributed Memory Parallel Computers. PhD thesis, School of Computer Science, Carnegie Mellon University, September 1991.

    Google Scholar 

  35. R. E. Tarjan. Testing flow graph reducibility. Journal of Computer and System Sciences, 9: 355–365, 1974.

    Article  MathSciNet  MATH  Google Scholar 

  36. Thinking Machines Corporation, Cambridge, MA. CM Fortran Reference Manual, version 5.2–0.6 edition, September 1989.

    Google Scholar 

  37. C. Tseng. An Optimizing Fortran D Compiler for MIMD Distributed-Memory Machines. PhD thesis, Rice University, Houston, TX, January 1993. Rice COMP TR93–199.

    Google Scholar 

  38. S. Wholey. Automatic Data Mapping for Distributed-Memory Parallel Computers. PhD thesis, School of Computer Science, Carnegie Mellon University, May 1991.

    Google Scholar 

  39. S. Wholey. Automatic data mapping for distributed-memory parallel computers. In Proceedings of the 1992 ACM International Conference on Supercomputing, Washington, DC, July 1992.

    Google Scholar 

  40. M.E. Wolf. Improving Locality and Parallelism in Nested Loops. PhD thesis, Stanford University, August 1992.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Rice University, Houston, Texas, USA

    Ulrich Kremer, John Mellor-Crummey, Ken Kennedy & Alan Carle

Authors
  1. Ulrich Kremer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Mellor-Crummey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ken Kennedy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alan Carle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ulrich Kremer .

Editor information

Christoph W. Keßler

Rights and permissions

Reprints and permissions

Copyright information

© 1994 Friedr. Vieweg & Sohn Verlagsgesellschaft mbH, Braunschweig/Wiesbaden

About this chapter

Cite this chapter

Kremer, U., Mellor-Crummey, J., Kennedy, K., Carle, A. (1994). Automatic Data Layout for Distributed-Memory Machines in the D Programming Environment. In: Keßler, C.W. (eds) Automatic Parallelization. Vieweg+Teubner Verlag. https://doi.org/10.1007/978-3-322-87865-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-322-87865-6_8

  • Publisher Name: Vieweg+Teubner Verlag

  • Print ISBN: 978-3-528-05401-4

  • Online ISBN: 978-3-322-87865-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation