Article

Optimization of continuous queries with shared expensive filters

Authors:

Kamesh Munagala,

Utkarsh Srivastava,

Jennifer WidomAuthors Info & Claims

PODS '07: Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems

Pages 215 - 224

https://doi.org/10.1145/1265530.1265561

Published: 11 June 2007 Publication History

Abstract

We consider the problem of optimizing and executing multiple continuous queries, where each query is a conjunction of filters and each filter may occur in multiple queries. When filters are expensive, significant performance gains are achieved by sharing filter evaluations across queries. A shared execution strategy in our scenario can either be fixed, in which filters are evaluated in the same predetermined order for all input, or adaptive, in which the next filter to be evaluated is chosen at runtime based on the results of the filters evaluated so far. We show that as filter costs increase, the best adaptive strategy is superior to any fixed strategy, despite the overhead of adaptivity. We show that itis NP-hard to find the optimal adaptive strategy, even if we are willing to approximate within any factor smaller than m where m is the number of queries. We then present a greedy adaptive execution strategy and show that it approximates the best adaptive strategy to within a factor O(log²m log n) where n is the number of distinct filters. We also give a precomputation technique that can reduce the execution overhead of adaptive strategies.

References

[1]

A. Arasu and J. Widom. Resource sharing in continuous sliding-window aggregates. In Proc. of the 2004 Intl. Conf. on Very Large Data Bases, pages 336--347, 2004.

Digital Library

[2]

R. Avnur and J. Hellerstein. Eddies: Continuously adaptive query processing. In Proc. of the 2000 ACM SIGMOD Intl. Conf. on Management of Data, pages 261--272, 2000.

Digital Library

[3]

B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and issues in data stream systems. In Proc. of the 2002 ACM Symp. on Principles of Database Systems, pages 1--16, 2002.

Digital Library

[4]

S. Babu et al. Adaptive ordering of pipelined stream filters. In Proc. of the 2004 ACM SIGMOD Intl. Conf. on Management of Data, pages 407--418, 2004.

Digital Library

[5]

P. Bizarro, S. Babu, D. DeWitt, and J. Widom. Content-based routing: Different plans for different data. In Proc. of the 2005 Intl. Conf. on Very Large Data Bases, pages 757--768, 2005.

Digital Library

[6]

S. Chaudhuri and K. Shim. Optimization of queries with user-defined predicates. ACM Trans. on Database Systems, 24(2):177--228, 1999.

Digital Library

[7]

J. Chen, D. J. DeWitt, F. Tian, and Y. Wang. NiagaraCQ: A scalable continuous query system for internet databases. In Proc. of the 2000 ACM SIGMOD Intl. Conf. on Management of Data, pages 379--390, 2000.

Digital Library

[8]

N. Dalvi, S. Sanghai, P. Roy, and S. Sudarshan. Pipelining in multi-query optimization. In Proc. of the 2001 ACM Symp. on Principles of Database Systems, 2001.

Digital Library

[9]

B. Dean, M. Goemans, and J. Vondrák. Approximating the stochastic knapsack problem: The benefit of adaptivity. In Proc. of the 2004 Annual IEEE Symp. on Foundations of Computer Science, 2004.

Digital Library

[10]

A. Deshpande, C. Guestrin, S. Madden, J. M. Hellerstein, and W. Hong. Model-driven data acquisition in sensor networks. In Proc. of the 2004 Intl. Conf. on Very Large Data Bases, 2004.

Digital Library

[11]

O. Etzioni et al. Efficient information gathering on the internet. In Proc. of the 1996 Annual IEEE Symp. on Foundations of Computer Science, pages 234--243, 1996.

Digital Library

[12]

M. Garey and D. Johnson. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman & Co., 1979.

Digital Library

[13]

E. Hanson. The Interval Skip List: A data structure for finding all intervals that overlap a point. In WADS, pages 153--164, 1991.

[14]

E. Hanson. Rule condition testing and action execution in Ariel. In Proc. of the 1992 ACM SIGMOD Intl. Conf. on Management of Data, pages 49--58, 1992.

Digital Library

[15]

J. Hellerstein and M. Stonebraker. Predicate migration: Optimizing queries with expensive predicates. In Proc. of the 1993 ACM SIGMOD Intl. Conf. on Management of Data, pages 267--276, 1993.

Digital Library

[16]

S. Madden, M. Shah, J. Hellerstein, and V. Raman. Continuously adaptive continuous queries over streams. In Proc. of the 2002 ACM SIGMOD Intl. Conf. on Management of Data, pages 49--60, 2002.

Digital Library

[17]

R. Motwani and P. Raghavan. Randomized Algorithms. Cambridge University Press, 1995.

[18]

K. Munagala, U. Srivastava, and J. Widom. Optimization of continuous queries with shared expensive filters. Technical report, Stanford University, 2005. Available at http://dbpubs.stanford.edu/pub/2005-36.

[19]

Open source computer vision library. http://sourceforge.net/projects/ opencvlibrary.

[20]

M. Skutella and M. Uetz. Scheduling precedence-constrained jobs with stochastic processing times on parallel machines. In Proc. of the 2001 Annual ACM-SIAM Symp. on Discrete Algorithms, pages 589--590, 2001.

Digital Library

[21]

R. Strom et al. Gryphon: An information flow based approach to message brokering. In Intl. Symp. on Software Reliability Engineering, 1998.

[22]

V. Vazirani. Approximation Algorithms. Springer, 2001.

Digital Library

[23]

T. Yan and H. Garcia-Molina. The SIFT information dissemination system. ACM Trans. on Database Systems, 24(4):529--565, 1999.

Digital Library

Cited By

Li JLi CTian BLiu YSi C(2020)DAFIntelligent Data Analysis10.3233/IDA-19464024:6(1441-1453)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.3233/IDA-194640
Zaarour TCurry E(2019)Adaptive Filtering of Visual Content in Distributed Publish/Subscribe Systems2019 IEEE 18th International Symposium on Network Computing and Applications (NCA)10.1109/NCA.2019.8935050(1-5)Online publication date: Sep-2019
https://doi.org/10.1109/NCA.2019.8935050
Getahun FChbeir R(2018)Multi-Query Optimization on RSS FeedsJournal on Data Semantics10.1007/s13740-018-0085-37:1(47-64)Online publication date: 5-Feb-2018
https://doi.org/10.1007/s13740-018-0085-3
Show More Cited By

Index Terms

Optimization of continuous queries with shared expensive filters
1. Information systems
  1. Data management systems
    1. Database management system engines
      1. Database query processing
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query processing and optimization (theory)

Recommendations

Containment and Optimization of Object-Preserving Conjunctive Queries

In the optimization of queries in an object-oriented database (OODB) system, a natural first step is to use the typing constraints imposed by the schema to transform a query into an equivalent one that logically accesses a minimal set of objects. We ...
Equivalence and minimization of conjunctive queries under combined semantics
ICDT '12: Proceedings of the 15th International Conference on Database Theory

The problems of query containment, equivalence, and minimization are fundamental problems in the context of query processing and optimization. In their classic work [2] published in 1977, Chandra and Merlin solved the three problems for the language of ...
An Optimal Algorithm for Processing Distributed Star Queries

The problem of optimal query processing in distributed database systems was shown to be NP-hard. However, for a special type of queries called star queries, we have developed a polynomial optimal algorithm. Semijoin tactics are applied for query ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PODS '07: Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems

June 2007

328 pages

ISBN:9781595936851

DOI:10.1145/1265530

General Chair:
Phokion Kolaitis
IBM Almaden
,
Program Chair:
Leonid Libkin
University of Edinburgh

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGMOD/PODS07

Sponsor:

SIGMOD/PODS07: International Conference on Management of Data

June 11 - 13, 2007

Beijing, China

Acceptance Rates

PODS '07 Paper Acceptance Rate 28 of 187 submissions, 15%;

Overall Acceptance Rate 642 of 2,707 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
468
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li JLi CTian BLiu YSi C(2020)DAFIntelligent Data Analysis10.3233/IDA-19464024:6(1441-1453)Online publication date: 1-Jan-2020
https://dl.acm.org/doi/10.3233/IDA-194640
Zaarour TCurry E(2019)Adaptive Filtering of Visual Content in Distributed Publish/Subscribe Systems2019 IEEE 18th International Symposium on Network Computing and Applications (NCA)10.1109/NCA.2019.8935050(1-5)Online publication date: Sep-2019
https://doi.org/10.1109/NCA.2019.8935050
Getahun FChbeir R(2018)Multi-Query Optimization on RSS FeedsJournal on Data Semantics10.1007/s13740-018-0085-37:1(47-64)Online publication date: 5-Feb-2018
https://doi.org/10.1007/s13740-018-0085-3
Huang CLiang S(2017)AHS Model: Efficient Topological Operators for a Sensor Web Publish/Subscribe SystemISPRS International Journal of Geo-Information10.3390/ijgi60200546:2(54)Online publication date: 20-Feb-2017
https://doi.org/10.3390/ijgi6020054
Kang DEmmons JAbuzaid FBailis PZaharia M(2017)NoScopeProceedings of the VLDB Endowment10.14778/3137628.313766410:11(1586-1597)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.14778/3137628.3137664
Gupta ANagarajan VRavi R(2017)Approximation Algorithms for Optimal Decision Trees and Adaptive TSP ProblemsMathematics of Operations Research10.1287/moor.2016.083142:3(876-896)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1287/moor.2016.0831
Luan WYang YFermüller CBaras J(2017)Fast task-specific target detection via graph based constraints representation and checking2017 IEEE International Conference on Robotics and Automation (ICRA)10.1109/ICRA.2017.7989458(3984-3991)Online publication date: 29-May-2017
https://dl.acm.org/doi/10.1109/ICRA.2017.7989458
Im SNagarajan VZwaan R(2016)Minimum Latency Submodular CoverACM Transactions on Algorithms10.1145/298775113:1(1-28)Online publication date: 15-Nov-2016
https://dl.acm.org/doi/10.1145/2987751
Robinson ALippmeier BDuke DKameyama Y(2016)Icicle: write once, run onceProceedings of the 5th International Workshop on Functional High-Performance Computing10.1145/2975991.2975992(2-8)Online publication date: 8-Sep-2016
https://dl.acm.org/doi/10.1145/2975991.2975992
Xie QZhang XLi ZZhou X(2016)Optimizing Cost of Continuous Overlapping Queries over Data Streams by Filter AdaptionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2016.251654128:5(1258-1271)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1109/TKDE.2016.2516541
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten