Skip to main content
SpringerLink
Log in

Tight Lower Bounds for Multi-pass Stream Computation Via Pass Elimination

  • Conference paper
Book cover Automata, Languages and Programming (ICALP 2008)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5125))

Included in the following conference series:

Abstract

There is a natural relationship between lower bounds in the multi-pass stream model and lower bounds in multi-round communication. However, this connection is less understood than the connection between single-pass stream computation and one-way communication. In this paper, we consider data-stream problems for which reductions from natural multi-round communication problems do not yield tight bounds or do not apply. While lower bounds are known for some of these data-stream problems, many of these only apply to deterministic or comparison-based algorithms, whereas the lower bounds we present apply to any (possibly randomized) algorithms. Our results are particularly relevant to evaluating functions that are dependent on the ordering of the stream, such as the longest increasing subsequence and a variant of tree pointer jumping in which pointers are revealed according to a post-order traversal.

Our approach is based on establishing “pass-elimination” type results that are analogous to the round-elimination results of Miltersen et al. [23] and Sen [29]. We demonstrate our approach by proving tight bounds for a range of data-stream problems including finding the longest increasing sequences (a problem that has recently become very popular [22,16,30,15,12] and we resolve an open question of [30]), constructing convex hulls and fixed-dimensional linear programming (generalizing results of [8] to randomized algorithms), and the “greater-than” problem (improving results of [9]). These results will also clarify one of the main messages of our work: sometimes it is necessary to prove lower bounds directly for stream computation rather than proving a lower bound for a communication problem and then constructing a reduction to a data-stream problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Adler, M., Demaine, E.D., Harvey, N.J.A., Patrascu, M.: Lower bounds for asymmetric communication channels and distributed source coding. In: ACM-SIAM Symposium on Discrete Algorithms, pp. 251–260 (2006)

    Google Scholar 

  2. Alon, N., Matias, Y., Szegedy, M.: The space complexity of approximating the frequency moments. Journal of Computer and System Sciences 58(1), 137–147 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bar-Yossef, Z., Jayram, T.S., Kumar, R., Sivakumar, D.: An information statistics approach to data stream and communication complexity. In: IEEE Symposium on Foundations of Computer Science, pp. 209–218 (2002)

    Google Scholar 

  4. Beame, P., Jayram, T.S., Rudra, A.: Lower bounds for randomized read/write stream algorithms. In: ACM Symposium on Theory of Computing, pp. 689–698 (2007)

    Google Scholar 

  5. Chakrabarti, A.: Lower bounds for multi-player pointer jumping. In: IEEE Conference on Computational Complexity, pp. 33–45 (2007)

    Google Scholar 

  6. Chakrabarti, A., Khot, S., Sun, X.: Near-optimal lower bounds on the multi-party communication complexity of set disjointness. In: IEEE Conference on Computational Complexity, pp. 107–117 (2003)

    Google Scholar 

  7. Chakrabarti, A., Regev, O.: An optimal randomised cell probe lower bound for approximate nearest neighbour searching. In: IEEE Symposium on Foundations of Computer Science, pp. 473–482 (2004)

    Google Scholar 

  8. Chan, T.M., Chen, E.Y.: Multi-pass geometric algorithms. Discrete & Computational Geometry 37(1), 79–102 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chang, K.L., Kannan, R.: The space complexity of pass-efficient algorithms for clustering. In: ACM-SIAM Symposium on Discrete Algorithms, pp. 1157–1166 (2006)

    Google Scholar 

  10. Chu, M., Kannan, S., McGregor, A.: Checking and spot-checking of heaps. In: International Colloquium on Automata, Languages and Programming, pp. 728–739 (2007)

    Google Scholar 

  11. Demetrescu, C., Finocchi, I., Ribichini, A.: Trading off space for passes in graph streaming problems. In: ACM-SIAM Symposium on Discrete Algorithms, pp. 714–723 (2006)

    Google Scholar 

  12. Ergun, F., Jowhari, H.: On the distance to monotonicity and longest increasing subsequence of a data stream. In: ACM-SIAM Symposium on Discrete Algorithms (2008)

    Google Scholar 

  13. Feigenbaum, J., Kannan, S., McGregor, A., Suri, S., Zhang, J.: Graph distances in the streaming model: the value of space. In: ACM-SIAM Symposium on Discrete Algorithms, pp. 745–754 (2005)

    Google Scholar 

  14. Feigenbaum, J., Kannan, S., McGregor, A., Suri, S., Zhang, J.: On graph problems in a semi-streaming model. Theoretical Computer Science 348(2-3), 207–216 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  15. Gal, A., Gopalan, P.: Lower bounds on streaming algorithms for approximating the length of the longest increasing subsequence. In: IEEE Symposium on Foundations of Computer Science (2007)

    Google Scholar 

  16. Gopalan, P., Jayram, T., Krauthgamer, R., Kumar, R.: Estimating the sortedness of a data stream. In: ACM-SIAM Symposium on Discrete Algorithms (2007)

    Google Scholar 

  17. Guha, S., Koudas, N., Shim, K.: Approximation and streaming algorithms for histogram construction problems. ACM Trans. Database Syst. 31(1), 396–438 (2006)

    Article  Google Scholar 

  18. Guha, S., McGregor, A.: Lower bounds for quantile estimation in random-order and multi-pass streaming. In: International Colloquium on Automata, Languages and Programming, pp. 704–715 (2007)

    Google Scholar 

  19. Henzinger, M.R., Raghavan, P., Rajagopalan, S.: Computing on data streams. In: External memory algorithms, pp. 107–118 (1999)

    Google Scholar 

  20. Indyk, P.: Stable distributions, pseudorandom generators, embeddings, and data stream computation. J. ACM 53(3), 307–323 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  21. Klauck, H., Nayak, A., Ta-Shma, A., Zuckerman, D.: Interaction in quantum communication and the complexity of set disjointness. In: ACM Symposium on Theory of Computing, pp. 124–133 (2001)

    Google Scholar 

  22. Liben-Nowell, D., Vee, E., Zhu, A.: Finding longest increasing and common subsequences in streaming data. J. Comb. Optim. 11(2), 155–175 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  23. Miltersen, P.B., Nisan, N., Safra, S., Wigderson, A.: On data structures and asymmetric communication complexity. J. Comput. Syst. Sci. 57(1), 37–49 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  24. Munro, J.I., Paterson, M.: Selection and sorting with limited storage. Theor. Comput. Sci. 12, 315–323 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  25. Nisan, N., Wigderson, A.: Rounds in communication complexity revisited. SIAM J. Comput. 22(1), 211–219 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  26. Patrascu, M., Thorup, M.: Time-space trade-offs for predecessor search. In: ACM Symposium on Theory of Computing, pp. 232–240 (2006)

    Google Scholar 

  27. Patrascu, M., Thorup, M.: Randomization does not help searching predecessors. In: ACM-SIAM Symposium on Discrete Algorithms, pp. 555–564 (2007)

    Google Scholar 

  28. Razborov, A.A.: On the distributional complexity of disjointness. Theor. Comput. Sci. 106(2), 385–390 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  29. Sen, P.: Lower bounds for predecessor searching in the cell probe model. In: IEEE Conference on Computational Complexity, pp. 73–83 (2003)

    Google Scholar 

  30. Sun, X., Woodruff, D.: The communication and streaming complexity of computing the longest common and increasing subsequences. In: ACM-SIAM Symposium on Discrete Algorithms, pp. 336–345 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer and Information Science, University of Pennsylvania, USA

    Sudipto Guha

  2. Information Theory & Applications Center, University of California, San Diego, USA

    Andrew McGregor

Authors
  1. Sudipto Guha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew McGregor
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Science, Reykjavik University, Kringlan 1, 103, Reykjavík, Iceland

    Luca Aceto

  2. Department of Computer Science, IT-Parken, University of Aarhus, Åbogade 34, 8200, ÅrhusN, Denmark

    Ivan Damgård

  3. Department of Computer Science, University of Liverpool, Ashton Building, L69 3BX, Liverpool, UK

    Leslie Ann Goldberg

  4. School of Computer Science, Reykjavik University, Kringlan 1, 103, Reykjavik, Iceland

    Magnús M. Halldórsson

  5. Department of Computer Science, Reykjavik University, Kringlan 1, 103, Reykjavík, Iceland

    Anna Ingólfsdóttir

  6. Université de Bordeaux-1, LaBRI, 351, Cours de la Libération, 33405, Talence cedex, France

    Igor Walukiewicz

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Guha, S., McGregor, A. (2008). Tight Lower Bounds for Multi-pass Stream Computation Via Pass Elimination. In: Aceto, L., Damgård, I., Goldberg, L.A., Halldórsson, M.M., Ingólfsdóttir, A., Walukiewicz, I. (eds) Automata, Languages and Programming. ICALP 2008. Lecture Notes in Computer Science, vol 5125. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70575-8_62

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-70575-8_62

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-70574-1

  • Online ISBN: 978-3-540-70575-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation