Skip to main content

Advertisement

Log in

Succinct Summing over Sliding Windows

  • Published:
Algorithmica Aims and scope Submit manuscript

Abstract

This paper considers the problem of estimating the sum the last \(W\) elements of a stream of integers in \(\left\{ 0,1,\ldots , R \right\} \). Specifically, we study the memory requirements for computing a \( R W\varepsilon \)-additive approximation for the window’s sum. We derive a lower bound of \(W\log \left\lfloor {\frac{1}{2W\varepsilon } + 1}\right\rfloor \) bits when \(\varepsilon \le 1/2W\) and show a matching succinct algorithm that uses \((1+o(1)) \left( {W\log \left\lfloor {\frac{1}{2W\varepsilon } + 1}\right\rfloor }\right) \) bits. Next, we prove a \((1-o(1)) \varepsilon ^{-1} /2\) bits lower bound when \(\varepsilon =\omega \left( {W^{-1}}\right) \wedge \varepsilon =o(\log ^{-1}W)\) and provide a succinct algorithm that requires \((1+o(1)) \varepsilon ^{-1} /2\) bits. We show that when \(\varepsilon =\varOmega \left( {\log ^{-1}W}\right) \) any solution to the problem must consume at least \((1-o(1))\cdot \left( {{ \varepsilon ^{-1} /2}+\log W}\right) \) bits, while our algorithm needs \((1+o(1))\cdot \left( {{ \varepsilon ^{-1} /2}+2\log W}\right) \) bits. Finally, we show that our lower bounds generalize to randomized algorithms as well, while our algorithms are deterministic and can process elements and answer queries in O(1) worst-case time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Notes

  1. In this paper, the logarithms are of base 2 and the o(1) notation is for \(W\rightarrow \infty \).

  2. This does not follow immediately for \(\varepsilon =\varTheta (1)\). In this case, the correctness follows from Lemma 2.

References

  1. Albert, M.H., Golynski, A., Hamel, A.M., Lopez-Ortiz, A., Rao, S.S., Safari, M.A.: Longest increasing subsequences in sliding windows. Theor. Comput. Sci. 321(2), 405–414 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  2. Arasu, A., Manku, G.S.: Approximate counts and quantiles over sliding windows. In: Proceedings of the 23rd ACM Symposium on Principles of Database Systems (PODS) (2004)

  3. Babcock, B., Datar, M., Motwani, R., O’Callaghan, L.: Maintaining variance and k-medians over data stream windows. In: Neven, F., Beeri, C., Milo, T. (eds.) Proceedings of the Twenty- Second ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, June 9–12, 2003, San Diego, CA, USA. ACM, pp. 234–243 (2003)

  4. Ben Basat, R., Einziger, G., Friedman, R., Kassner, Y.: Heavy hitters in streams and sliding windows. In: Proceedings of IEEE INFOCOM (2016)

  5. Braverman, V., Gelles, R., Ostrovsky, R.: How to catch l2-heavy-hitters on sliding windows. Theor. Comput. Sci. 554, 82–94 (2014)

    Article  MATH  Google Scholar 

  6. Braverman, V., Ostrovsky, R.: Smooth histograms for sliding windows. In: 48th Annual IEEE Symposium on Foundations of Computer Science, 2007. FOCS’07, pp. 283–293

  7. Cohen, E., Strauss, M.J.: Maintaining time-decaying stream aggregates. J. Algorithms 59(1), 19–36 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  8. Cormode, G., Yi, K.: Tracking distributed aggregates over time-based sliding windows. In: Scientific and Statistical Database Management. pp. 416–430. Springer (2012)

  9. Crouch, M., Stubbs, D.S.: Improved streaming algorithms for weighted matching, via un-weighted matching. In: APPROX/RANDOM (2014)

  10. Crouch, M.S., McGregor, A., Stubbs, D.: Dynamic graphs in the sliding-window model. In: Algorithms-ESA 2013, pp. 337–348. Springer (2013)

  11. Datar, M., Gionis, A., Indyk, P., Motwani, R.: Maintaining stream statistics over sliding windows. SIAM J. Comput. 31(6), 1794–1813 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  12. Gibbons, P.B., Tirthapura, S.: Distributed streams algorithms for sliding windows. In: SPAA, pp. 63–72 (2002)

  13. Hung, R., Ting, H.: Finding heavy hitters over the sliding window of a weighted data stream. In: Laber, E., Bornstein, C., Nogueira, L., Faria, L. (eds.) LATIN 2008: Theoretical Informatics, LNCS, vol. 4957, pp. 699–710. Springer, Berlin (2008)

    Chapter  Google Scholar 

  14. Lee, L.K., Ting, H.F.: A simpler and more efficient deterministic scheme for finding frequent items over sliding windows. In: Proceedings of the Twenty-fifth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS) (2006)

  15. Lee, L., Ting, H.F.: Maintaining significant stream statistics over sliding windows. In: Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithms, SODA 2006, Miami, Florida, USA, January 22–26, 2006. ACM Press, pp. 724–732 (2006)

  16. Liu, Y., Chen, W., Guan, Y.: Near-optimal approximate membership query over time-decaying windows. In: INFOCOM, 2013 Proceedings IEEE, pp. 1447–1455 (2013)

  17. Mouratidis, K., Bakiras, S., Papadias, D.: Continuous monitoring of top-k queries over sliding windows. In: Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data. SIGMOD’06, pp. 635–646. ACM, New York, NY, USA (2006)

  18. Naor, M., Yogev, E.: Sliding bloom filters. In: Cai, L., Cheng, S.W., Lam, T.W. (eds.) Algorithms and Computation. Lecture Notes in Computer Science, vol. 8283, pp. 513–523. Springer, Berlin (2013)

    Google Scholar 

  19. Pripuzic, K., Zarko, I.P., Aberer, K.: Time- and space-efficient sliding window top-k query processing. ACM Trans. Database Syst. 40(1), 1:1–1:44 (2015)

    Article  MathSciNet  Google Scholar 

  20. Shen, H., Zhang, Y.: Improved approximate detection of duplicates for data streams over sliding windows. J. Comput. Sci. Technol. 23(6), 973–987 (2008)

    Article  Google Scholar 

  21. Shen, Z., Cheema, M., Lin, X., Zhang, W., Wang, H.: Efficiently monitoring top-k pairs over sliding windows. In: 2012 IEEE 28th International Conference on Data Engineering (ICDE), pp. 798–809

  22. Yao, A.C.: Probabilistic computations: toward a unified measure of complexity. In: 18th Annual Symposium on Foundations of Computer Science, 1977, pp. 222–227. IEEE

  23. Zhang, W., Zhang, Y., Cheema, M.A., Lin, X.: Counting distinct objects over sliding windows. In: Proceedings of the Twenty-First Australasian Conference on Database Technologies–Volume 104. ADC’10, pp. 75–84. Australian Computer Society, Inc., Darlinghurst, Australia (2010)

Download references

Acknowledgements

We thank Dror Rawitz for helpful comments. This work was partially funded by MOST Grant #3-10886 and the Technion-HPI research school.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Gil Einziger.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ben Basat, R., Einziger, G., Friedman, R. et al. Succinct Summing over Sliding Windows. Algorithmica 81, 2072–2091 (2019). https://doi.org/10.1007/s00453-018-0524-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00453-018-0524-4

Keywords