Article

Streaming in a connected world: querying and tracking distributed data streams

Authors:

Graham Cormode,

Minos GarofalakisAuthors Info & Claims

SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data

Pages 1178 - 1181

https://doi.org/10.1145/1247480.1247649

Published: 11 June 2007 Publication History

First page of PDF

Supplementary Material

Low Resolution (p1178-cormode_56k.mov)

Download
91.51 MB

High Resolution (p1178-cormode_768k.mov)

Download
545.79 MB

References

[1]

D. Abadi, Y. Ahmad, M. Balazinska, U. Cetintemel, M. Cherniack, J. Hwang, W. Lindner, A. Rasin, N. Tatbul, Y. Xing, and S. Zdonik. Distributed operation in the borealis stream processing engine. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2005.

Digital Library

[2]

N. Alon, P. Gibbons, Y. Matias, and M. Szegedy. Tracking join and self-join sizes in limited storage. In Proceedings of ACM Principles of Database Systems, pages 10--20, 1999.

Digital Library

[3]

N. Alon, Y. Matias, and M. Szegedy. The space complexity of approximating the frequency moments. In Proceedings of the ACM Symposium on Theory of Computing, pages 20--29, 1996. Journal version in Journal of Computer and System Sciences, 58:137--147, 1999.

Digital Library

[4]

B. Babcock and C. Olston. Distributed top-k monitoring. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2003.

Digital Library

[5]

S. Chandrasekaran, O. Cooper, A. Deshpande, M. J. Franklin, J. M. Hellerstein, W. Hong, S. Krishnamurthy, S. R. Madden, F. Reiss, and M. A. Shah. TelegraphCQ: continuous dataflow processing. In Proceedings of ACM SIGMOD International Conference on Management of Data, page 668, 2003.

Digital Library

[6]

D. Chu, A. Deshpande, J. M. Hellerstein, and W. Hong. Approximate data collection in sensor networks using probabilistic models. In IEEE International Conference on Data Engineering, 2006.

Digital Library

[7]

J. Considine, F. Li, G. Kollios, and J. Byers. Approximate aggregation techniques for sensor databases. In IEEE International Conference on Data Engineering, 2004.

Digital Library

[8]

G. Cormode and M. Garofalakis. Efficient strategies for continuous distributed tracking tasks. IEEE Data Engineering Bulletin, 28(1):33--39, March 2005.

[9]

G. Cormode and M. Garofalakis. Sketching streams through the net: Distributed approximate query tracking. In Proceedings of the International Conference on Very Large Data Bases, 2005.

Digital Library

[10]

G. Cormode, M. Garofalakis, S. Muthukrishnan, and R. Rastogi. Holistic aggregates in a networked world: Distributed tracking of approximate quantiles. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2005.

Digital Library

[11]

G. Cormode and S. Muthukrishnan. An improved data stream summary: The count-min sketch and its applications. Journal of Algorithms, 55(1):58--75, 2005.

Digital Library

[12]

G. Cormode and S. Muthukrishnan. Space efficient mining of multigraph streams. In Proceedings of ACM Principles of Database Systems, 2005.

Digital Library

[13]

G. Cormode, S. Muthukrishnan, and W. Zhuang. What's different: Distributed, continuous monitoring of duplicate resilient aggregates on data streams. In IEEE International Conference on Data Engineering, 2006.

Digital Library

[14]

C. Cranor, T. Johnson, O. Spatscheck, and V. Shkapenyuk. Gigascope: A stream database for network applications. In Proceedings of ACM SIGMOD International Conference on Management of Data, pages 647--651, 2003.

Digital Library

[15]

A. Das, S. Ganguly, M. Garofalakis, and R. Rastogi. Distributed set-expression cardinality estimation. In Proceedings of the International Conference on Very Large Data Bases, 2004.

Digital Library

[16]

A. Deshpande, C. Guestrin, S. R. Madden, J. M.Hellerstein, and W. Hong. Model-drive data acquisition in sensor networks. In Proceedings of the International Conference on Very Large Data Bases, 2004.

Digital Library

[17]

P. Flajolet and G. N. Martin. Probabilistic counting. In IEEE Conference on Foundations of Computer Science, pages 76--82, 1983. Journal version in Journal of Computer and System Sciences, 31:182--209, 1985.

Digital Library

[18]

M. Garofalakis. Special issue on in-network query processing. IEEE Data Engineering Bulletin, 28(1), March 2005.

[19]

M. Greenwald and S. Khanna. Power-conserving computation of order-statistics over sensor networks. In Proceedings of ACM Principles of Database Systems, pages 275--285, 2004.

Digital Library

[20]

C. Guestrin, P. Bodik, R. Thibaux, M. Paskin, and S. Madden. Distributed regression: an efficient framework for modeling sensor network data. In Information Processing in Sensor Networks, 2004.

Digital Library

[21]

M. Hadjieleftheriou, J. W. Byers, and G. Kollios. Robust sketching and aggregation of distributed data streams. Technical Report 2005-11, Boston University Computer Science Department, 2005.

[22]

A. Jain, J. Hellerstein, S. Ratnasamy, and D. Wetherall. A wakeup call for internet monitoring systems: The case for distributed triggers. In Proceedings of the 3rd Workshop on Hot Topics in Networks (Hotnets), 2004.

[23]

S. Jain, K. Fall, and P. Rabin. Routing in a delay tolerant network. In ACM SIGCOMM, 2005.

[24]

S. Kashyap, S. Deb, K. V. M. Naidu, R. Rastogi, and A. Srinivasan. Efficient gossip-based aggregate computation. In Proceedings of ACM Principles of Database Systems, 2006.

Digital Library

[25]

D. Kempe, A. Dobra, and J. Gehrke. Computing aggregates using gossip. In IEEE Conference on Foundations of Computer Science, 2003.

[26]

D. Kempe, J. Kleinberg, and A. Demers. Spatial gossip and resource location protocols. In Proceedings of the ACM Symposium on Theory of Computing, 2001.

Digital Library

[27]

R. Keralapura, G. Cormode, and J. Ramamirtham. Communication-efficient distributed monitoring of thresholded counts. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2006.

Digital Library

[28]

T. Loo, J. Hellerstein, I. Stoica, and R. Ramakrishnan. Declarative routing: Extensible routing with declarative queries. In ACM SIGCOMM, 2005.

Digital Library

[29]

S. Madden. Data management in sensor networks. In Proceedings of European Workshop on Sensor Networks, 2006.

Digital Library

[30]

S. Madden, M. Franklin, J. Hellerstein, and W. Hong. TAG: a Tiny AGgregation service for ad-hoc sensor networks. In Proceedings of Symposium on Operating System Design and Implementation, 2002.

[31]

S. Madden, M. Franklin, J. Hellerstein, and W. Hong. TinyDB: an acquisitional query processing system for sensor networks. ACM Transactions on Database Systems, 30(1):122--173, 2005.

Digital Library

[32]

A. Manjhi, S. Nath, and P. Gibbons. Tributaries and deltas: Efficient and robust aggregation in sensor network streams. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2005.

Digital Library

[33]

A. Manjhi, V. Shkapenyuk, K. Dhamdhere, and C. Olston. Finding (recently) frequent items in distributed data streams. In IEEE International Conference on Data Engineering, pages 767--778, 2005.

Digital Library

[34]

S. Nath, P. B. Gibbons, S. Seshan, and Z. R.Anderson. Synopsis diffusion for robust aggrgation in sensor networks. In ACM SenSys, 2004.

Digital Library

[35]

C. Olston, J. Jiang, and J. Widom. Adaptive filters for continuous queries over distributed data streams. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2003.

Digital Library

[36]

S. Rhea, G. Brighten, B. Karp, J. Kubiatowicz, S. Ratnasamy, S. Shenker, I. Stoica, and Y. Harlan. OpenDHT: A public DHT service and its uses. In ACM SIGCOMM, 2005.

Digital Library

[37]

J. Rissanen. Modeling by shortest data description. Automatica, 14:465--471, 1978.

Digital Library

[38]

I. Sharfman, A. Schuster, and D. Keren. A geometric approach to monitoring threshold functions over distribtuted data streams. In Proceedings of ACM SIGMOD International Conference on Management of Data, 2006.

Digital Library

[39]

S. Zdonik, M. Stonebraker, M. Cherniack, and U. Cetintemel. The aurora and medusa projects. Bulletin of the Technical Committee on Data Engineering, pages 3--10, Mar. 2003.

Cited By

Rakkesh SWeerasinghe ARanasinghe R(2023)Real-time Vehicular Pollution Detection Model using IoT and Distributed Streaming Techniques2023 3rd International Conference on Advanced Research in Computing (ICARC)10.1109/ICARC57651.2023.10145636(178-183)Online publication date: 23-Feb-2023
https://doi.org/10.1109/ICARC57651.2023.10145636
Gan EDing JTai KSharan VBailis P(2018)Moment-based quantile sketches for efficient high cardinality aggregation queriesProceedings of the VLDB Endowment10.14778/3236187.323621211:11(1647-1660)Online publication date: 1-Jul-2018
https://dl.acm.org/doi/10.14778/3236187.3236212
Palma WAkbarinia RPacitti EValduriez P(2018)DHTJoinDistributed and Parallel Databases10.1007/s10619-009-7054-726:2-3(291-317)Online publication date: 27-Dec-2018
https://dl.acm.org/doi/10.1007/s10619-009-7054-7
Show More Cited By

Index Terms

Streaming in a connected world: querying and tracking distributed data streams
1. Software and its engineering
  1. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles
        Organizing principles for web applications

Recommendations

Substream management in distributed streaming dataflows
DEBS '22: Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems

Most state-of-the-art SPEs use punctuations to divide a stream into bounded substreams of messages, such as epochs and windows. The punctuation approach is powerful but has limitations: it does not support cyclic dataflows, is poorly scalable in some ...
Survey: Streaming techniques and data aggregation in networks of tiny artefacts

In emerging pervasive scenarios, data is collected by sensing devices in streams that occur at several distributed points of observation. The size of the data typically far exceeds the storage and computational capabilities of the tiny devices that have ...
Streaming Linear Regression on Spark MLlib and MOA
ASONAM '15: Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015

In recent years, analyzing data streams has attracted considerable attention in different fields of computer science. In this paper, two different frameworks, namely MOA and Spark MLlib, are examined for linear regression on streaming data. The focus is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data

June 2007

1210 pages

ISBN:9781595936868

DOI:10.1145/1247480

General Chairs:
Lizhu Zhou
Tsinghua University, China
,
Tok Wang Ling
National University of Singapore, Singapore
,
Program Chair:
Beng Chin Ooi
National University of Singapore, Singapore

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGMOD/PODS07

Sponsor:

SIGMOD/PODS07: International Conference on Management of Data

June 11 - 14, 2007

Beijing, China

Acceptance Rates

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
66
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rakkesh SWeerasinghe ARanasinghe R(2023)Real-time Vehicular Pollution Detection Model using IoT and Distributed Streaming Techniques2023 3rd International Conference on Advanced Research in Computing (ICARC)10.1109/ICARC57651.2023.10145636(178-183)Online publication date: 23-Feb-2023
https://doi.org/10.1109/ICARC57651.2023.10145636
Gan EDing JTai KSharan VBailis P(2018)Moment-based quantile sketches for efficient high cardinality aggregation queriesProceedings of the VLDB Endowment10.14778/3236187.323621211:11(1647-1660)Online publication date: 1-Jul-2018
https://dl.acm.org/doi/10.14778/3236187.3236212
Palma WAkbarinia RPacitti EValduriez P(2018)DHTJoinDistributed and Parallel Databases10.1007/s10619-009-7054-726:2-3(291-317)Online publication date: 27-Dec-2018
https://dl.acm.org/doi/10.1007/s10619-009-7054-7
Giatrakos NDeligiannakis AGarofalakis MÖzcan FKoutrika GMadden S(2016)Scalable Approximate Query Tracking over Highly Distributed Data StreamsProceedings of the 2016 International Conference on Management of Data10.1145/2882903.2915225(1497-1512)Online publication date: 26-Jun-2016
https://dl.acm.org/doi/10.1145/2882903.2915225
Chong DShi H(2015)Big data analytics: a literature reviewJournal of Management Analytics10.1080/23270012.2015.10824492:3(175-201)Online publication date: 13-Oct-2015
https://doi.org/10.1080/23270012.2015.1082449
Giatrakos NDeligiannakis AGarofalakis MSharfman ISchuster A(2014)Distributed Geometric Query Monitoring Using Prediction ModelsACM Transactions on Database Systems10.1145/260213739:2(1-42)Online publication date: 26-May-2014
https://dl.acm.org/doi/10.1145/2602137
Jeon JLee KKim M(2014)Communication-efficient processing of multiple continuous aggregate queriesInformation Sciences10.1016/j.ins.2014.06.044284(1-17)Online publication date: Nov-2014
https://doi.org/10.1016/j.ins.2014.06.044
Garofalakis M(2014)Querying Distributed Data StreamsAdvances in Databases and Information Systems10.1007/978-3-319-10933-6_1(1-10)Online publication date: 2014
https://doi.org/10.1007/978-3-319-10933-6_1
Cormode G(2013)The continuous distributed monitoring modelACM SIGMOD Record10.1145/2481528.248153042:1(5-14)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.1145/2481528.2481530
Giatrakos NDeligiannakis AGarofalakis MSharfman ISchuster ACandan KChen YSnodgrass RGravano LFuxman A(2012)Prediction-based geometric monitoring over distributed data streamsProceedings of the 2012 ACM SIGMOD International Conference on Management of Data10.1145/2213836.2213867(265-276)Online publication date: 20-May-2012
https://dl.acm.org/doi/10.1145/2213836.2213867
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten