Skip to main content
SpringerLink
Log in

Communication-efficient parallel algorithms for distributed random-access machines

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

This paper introduces a model for parallel computation, called thedistributed randomaccess machine (DRAM), in which the communication requirements of parallel algorithms can be evaluated. A DRAM is an abstraction of a parallel computer in which memory accesses are implemented by routing messages through a communication network. A DRAM explicitly models the congestion of messages across cuts of the network.

We introduce the notion of aconservative algorithm as one whose communication requirements at each step can be bounded by the congestion of pointers of the input data structure across cuts of a DRAM. We give a simple lemma that shows how to “shortcut” pointers in a data structure so that remote processors can communicate without causing undue congestion. We giveO(lgn)-step, linear-processor, linear-space, conservative algorithms for a variety of problems onn-node trees, such as computing treewalk numberings, finding the separator of a tree, and evaluating all subexpressions in an expression tree. We giveO(lg2 n)-step, linear-processor, linear-space, conservative algorithms for problems on graphs of sizen, including finding a minimum-cost spanning forest, computing biconnected components, and constructing an Eulerian cycle. Most of these algorithms use as a subroutine a generalization of the prefix computation to trees. We show that any suchtreefix computation can be performed inO(lgn) steps using a conservative variant of Miller and Reif's tree-contraction technique.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. N. Bhatt and C. E. Leiserson, How to assemble tree machines, inAdvances in Computing Research, Vol. 2 (F. P. Preparata, ed.), JAI Press, Greenwich, CT, 1984, pp. 95–114.

    Google Scholar 

  2. R. P. Brent, The parallel evaluation of general arithmetic expressions,J. Assoc. Comput. Mach.,21 (2), 1974, 201–208.

    MATH  MathSciNet  Google Scholar 

  3. R. P. Brent and H. T. Kung, A regular layout for parallel adders,IEEE Trans. Comput.,31 (3), 1982, 260–264.

    Article  MATH  MathSciNet  Google Scholar 

  4. H. Chernoff, A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations,Ann. of Math. Stat.,23, 1952, 493–507.

    Article  MATH  MathSciNet  Google Scholar 

  5. R. Cole and U. Vishkin, Deterministic coin tossing and accelerating cascades: micro and macro techniques for designing parallel algorithms,Proceedings of the 18th Annual ACM Symposium on the Theory of Computing, May 1986, pp. 206–219.

  6. A. K. Dewdney, Computer recreations,Sci. Amer. 252 (6), 1985, 18–29.

    Article  Google Scholar 

  7. M. J. Fischer and R. E. Ladner, Parallel prefix computation,J. Assoc. Comput. Mach.,27 (4), 1980, 831–838.

    MATH  MathSciNet  Google Scholar 

  8. A. V. Goldberg and R. E. Tarjan, A new approach to the maximum flow problem,Proceedings of the 18th Annual ACM Symposium on the Theory of Computing, May 1986, pp. 136–146.

  9. A. V. Goldberg and R. E. Tarjan, Solving minimum-cost flow problems by successive approximation,Proceedings of the 19th Annual ACM Symposium on the Theory of Computing, May 1987, pp. 7–18.

  10. R. I. Greenberg and C. E. Leiserson, Randomized routing on fat-trees,Proceedings of the 26th Annual Symposium on Foundations of Computer Science, October 1985, pp. 241–249.

  11. W. D. Hillis and G. L. Steele, Jr., Data parallel algorithms,Comm. ACM,29 (12), 1986, 1170–1183.

    Article  Google Scholar 

  12. A. R. Karlin and E. Upfal, Parallel hashing-an efficient implementation of shared memory,Proceedings of the 18th Annual ACM Symposium on the Theory of Computing, May 1986, pp. 160–168.

  13. J. Kilian, Private communication, July 1986.

  14. D. E. Knuth,The Art of Computer Programming, Vol. 1, Second Edition, Addison-Wesley, Reading, MA, 1973.

    Google Scholar 

  15. H. T. Kung and C. E. Leiserson, Systolic arrays (for VLSI), inS1AM Sparse Matrix Proceedings (I. S. Duff and G. W. Stewart, ed.), SIAM, Philadelphia, PA, 1978, pp. 256–282.

    Google Scholar 

  16. F. T. Leighton and A. L. Rosenberg, Three-dimensional circuit layouts,SIAM J. Comput.,15 (3), 1986, 793–813.

    Article  MATH  MathSciNet  Google Scholar 

  17. C. E. Leiserson, Fat-trees: universal networks for hardware-efficient supercomputing,IEEE Trans. Comput.,34 (10), 1985, 892–901.

    Google Scholar 

  18. C. E. Leiserson,Area-Efficient VLSI Computation, MIT Press, Cambridge, MA, 1983.

    Google Scholar 

  19. C. E. Leiserson and B. M. Maggs, Communication-efficient parallel graph algorithms, Technical Memo MIT/ LCS/TM-318, MIT Laboratory for Computer Science, Cambridge, MA, December 1986.

    Google Scholar 

  20. R. J. Lipton and R. E. Tarjan, A planar separator theorem,SIAM J. Appl. Math.,36 (2), 1979, 177–189.

    Article  MATH  MathSciNet  Google Scholar 

  21. B. M. Maggs, Communication-Efficient Parallel Graph Algorithms, Master's thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, MA, May 1986.

    Google Scholar 

  22. G. Miller and J. Reif, Parallel tree contraction and its application,Proceedings of the 26th Annual Symposium on Foundations of Computer Science, October 1985, pp. 478–489.

  23. Yu. Ofman, On the algorithmic complexity of discrete functions,Soviet Phys. Dokl. 7 (7), 1963, 589–591.

    MathSciNet  Google Scholar 

  24. G. F. Pfister and V. A. Norton, “Hot spot” contention and combining in multistage interconnection networks,IEEE Trans. Comput.,34 (10), 1985, 943–948.

    Google Scholar 

  25. Y. Shiloach and U. Vishkin, AnO(logn) parallel connectivity algorithm,J. Algorithms,3, 1982, 57–67.

    Article  MATH  MathSciNet  Google Scholar 

  26. R. E. Tarjan,Data Structures and Network Algorithms, SIAM, Philadelphia, PA, 1983.

    Google Scholar 

  27. R. E. Tarjan and U. Vishkin, Finding biconnected components and computing tree functions in logarithmic parallel time,Proceedings of the 25th Annual Symposium on Foundations of Computer Science, October 1984, pp. 12–20.

  28. C. D. Thompson, A Complexity Theory for VLSI, Ph.D. thesis, Department of Computer Science, Carnegie-Mellon University, Pittsburgh, PA, 1980.

    Google Scholar 

  29. L. G. Valiant, A scheme for fast parallel communication,SIAM J. Comput.,11(2), 1982, 350–361.

    Article  MATH  MathSciNet  Google Scholar 

  30. J. C. Wyllie, The Complexity of Parallel Computations, Ph.D. thesis, Cornell University, Ithaca, NY, 1979.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Laboratory for Computer Science, Massachusetts Institute of Technology, 02139, Cambridge, MA, USA

    Charles E. Leiserson & Bruce M. Maggs

Authors
  1. Charles E. Leiserson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bruce M. Maggs
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by Jeffrey Scott Vitter.

This research was supported in part by the Defense Advanced Research Projects Agency under Contract N00014-80-C-0622 and by the Office of Naval Research under Contract N00014-86-K-0593. Charles Leiserson is supported in part by an NSF Presidential Young Investigator Award with matching funds provided by AT&T Bell Laboratories and Xerox Corporation. Bruce Maggs is supported in part by an NSF Fellowship.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Leiserson, C.E., Maggs, B.M. Communication-efficient parallel algorithms for distributed random-access machines. Algorithmica 3, 53–77 (1988). https://doi.org/10.1007/BF01762110

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01762110

Key words

Search

Navigation