Abstract
Given a data stream of numerical data elements generated from multiple sources, we consider the problem of maintaining the sum of the elements for each data source over a sliding window of the data stream. The difficulties of the problem come from two parts. One is the number of data sources and the other is the number of elements in the sliding window. For massive data sources, we need a significant number of counters to maintain the sum for each data source, while for a large number of data elements in the sliding window, we need a huge space to keep all of them. We propose two methods, which shares the counters efficiently and merge the data elements systematically so that we are able to estimate the sums using a concise data structure. Two parameters, ε and δ, are needed to construct the data structure. ε controls the bounds of the estimate and δ represents the confidence level that the estimate is within the bounds. The estimates of both methods are proven to be bounded within a factor of ε at 1-δ probability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cormode, G., Muthukrishnan, S.: Improved Data Stream Summary: The CM sketch and its Applications. Journal of Algorithms (April 2005)
Cormode, G., Muthukrishnan, S., Korn, F., Srivastava, D.: Effective Computation of Biased Quantiles over Data Streams. In: Proceedings of the 21st International Conference on Data Engineering, pp. 20–31 (2005)
Datar, M., Gionis, A., Indyk, P., Motwani, R.: Maintaining Stream Statistics over Sliding Windows. SIAM Journal on Computing 31(6), 1794–1813 (2002)
Lin, C., Chiu, D., Wu, Y., Chen, A.: Mining Frequent Itemsets in Time-Sensitive Sliding Window over Data Streams. In: SIAM International Data Mining Conference (2005)
Lin, X., Xu, J., Lu, H., Yu, J.X.: Continuously Maintaining Quantile Summaries of the Most Recent N elements over a Data Stream. In: Proceedings of the 20th International Conference on Data Engineering, pp. 362–374 (2004)
Motwani, R., Raghavan, P.: Randomized Algorithms. Cambridge University Press, Cambridge (1995)
Qiao, L., Agrawal, D., El Abbadi, A.: Supporting Sliding Window Queries for Continuous Data Streams. In: Proceedings of 15th International Conference on Scientific and Statistical Database Management, pp. 85–94 (2003)
Zhu, Y., Shasha, D.: StatStream: Statistical monitoring of thousands of data streams in real time. In: Proceedings of the 28th International Conf. on Very Large Data Bases, pp. 358–369 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wu, TC., Chen, A.L.P. (2006). Maintaining Moving Sums over Data Streams. In: Li, X., Zaïane, O.R., Li, Z. (eds) Advanced Data Mining and Applications. ADMA 2006. Lecture Notes in Computer Science(), vol 4093. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11811305_117
Download citation
DOI: https://doi.org/10.1007/11811305_117
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-37025-3
Online ISBN: 978-3-540-37026-0
eBook Packages: Computer ScienceComputer Science (R0)