Skip to main content
SpringerLink
Log in

Selectivity estimators for multidimensional range queries over real attributes

  • Published:
The VLDB Journal Aims and scope Submit manuscript

Abstract.

Estimating the selectivity of multidimensional range queries over real valued attributes has significant applications in data exploration and database query optimization. In this paper, we consider the following problem: given a table of d attributes whose domain is the real numbers and a query that specifies a range in each dimension, find a good approximation of the number of records in the table that satisfy the query. The simplest approach to tackle this problem is to assume that the attributes are independent. More accurate estimators try to capture the joint data distribution of the attributes. In databases, such estimators include the construction of multidimensional histograms, random sampling, or the wavelet transform. In statistics, kernel estimation techniques are being used. Many traditional approaches assume that attribute values come from discrete, finite domains, where different values have high frequencies. However, for many novel applications (as in temporal, spatial, and multimedia databases) attribute values come from the infinite domain of real numbers. Consequently, each value appears very infrequently, a characteristic that affects the behavior and effectiveness of the estimator. Moreover, real-life data exhibit attribute correlations that also affect the estimator. We present a new histogram technique that is designed to approximate the density of multidimensional datasets with real attributes. Our technique defines buckets of variable size and allows the buckets to overlap. The size of the cells is based on the local density of the data. The use of overlapping buckets allows a more compact approximation of the data distribution. We also show how to generalize kernel density estimators and how to apply them to the multidimensional query approximation problem. Finally, we compare the accuracy of the proposed techniques with existing techniques using real and synthetic datasets. The experimental results show that the proposed techniques behave more accurately in high dimensionalities than previous approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Aboulnaga A, Chaudhuri S (1999) Self-tuning histograms: building histograms without looking at data. In: Proceedings of the 1999 ACM SIGMOD international conference on management of data, Philadelphia, June 1999

  2. Acharya S, Poosala V, Ramaswamy S (1999) Selectivity estimation in spatial databases. In: Proceedings of the 1999 ACM SIGMOD international conference on management of data, Philadelphia, June 1999

  3. Blohsfeld B, Korus D, Seeger B (1999) A comparison of selectivity estimators for range queries on metric attributes. In: Proceedings of the 1999 ACM SIGMOD international conference on management of data, Philadelphia, June 1999

  4. Bruno N, Chaudhuri S, Gravano L (2001) STHoles: a multidimensional workload-aware histogram In: Proceedings of the 2001 ACM SIGMOD international conference on management of data, Santa Barbara, May 2001

  5. Chaudhuri S, Gravano L (1999) Evaluating top-K selection queries. In: Proceedings of the 25th international conference on very large data bases (VLDB-99), Edinburgh, September 1999

  6. Chaudhuri S, Motwani R, Narasayya VR (1998) Random sampling for histogram construction: how much is enough? In: Proceedings of the 1998 ACM SIGMOD international conference on management of data, Seattle, June 1998

  7. Cressie NQC (1993) Statistics for spatial data. Wiley, New York

  8. Diggle PJ A kernel method for smoothing point process data. Appl Stat 34:138-147

  9. Donjerkovic D, Ramakrishnan R (1999) Probabilistic optimization of top N queries. In: Proceedings of the 25th international conference on very large data bases (VLDB-99), Edinburgh, September 1999

  10. Gibbons PB, Matias Y (1998) New sampling-based summary statistics for improving approximate query answers. In: Proceedings of the 1998 ACM SIGMOD international conference on management of data, Seattle, June 1998

  11. Gibbons PB, Matias Y, Poosala V (1997) Fast incremental maintenance of approximate histograms. In: Proceedings of the 23rd international conference on very large data bases, Athens, Greece, August 1997

  12. Gunopulos D, Kollios G, Tsotras V, Domeniconi C (2000) Approximating multi-dimensional aggregate range queries over real attributes. In: Proceedings of the 2000 ACM SIMGOD international conference on management of data, Dallas, May 2000

  13. Haas PJ, Swami AN (1992) Sequential sampling procedures for query size estimation. In: Proceedings of the 1992 ACM SIGMOD international conference on management of data, San Diego, June 1992

  14. Hellerstein JM, Haas PJ, Wan H (1997) Online aggregation. In: Proceedings of the 1997 ACM SIGMOD international conference on management of data, Tucson, AZ, May 1997

  15. Imager Wavelet Library. www.cs.ubc.ca/nest/imager/contributions/bobl/wvlt/top.html

  16. Ioannidis Y, Poosala V (1999) Histogram-based approximation of set-valued query-answers. In: Proceedings of the 25th international conference on very large data bases (VLDB-99), Edinburgh, September 1999

  17. Jagadish HV, Koudas N, Muthukrishnan S, Poosala V, Sevcik KC, Suel T (1998) Optimal histograms with quality guarantees. In: Proceedings of the 24rd international conference on very large data bases, August 1998

  18. Khanna S, Muthukrishnan S, Patterson M (1998) On approximating rectangle tiling and packing. In: Proceedings of the 9th annual symposium on discrete algorithms (SODA), San Francisco, January 1998

  19. Konig A, Weikum G (1999) Combining histograms and parametric curve fitting for feedback-driven query result-size estimation. In: Proceedings of the 25th international conference on very large data bases (VLDB-99), Edinburgh, September 1999

  20. Korn F, Johnson T, Jagadish H (1999) Range selectivity estimation for continuous attributes. In: Proceedings of the 11th international conference on SSDBMs, Cleveland, OH, July 1999

  21. Lipton RJ, Naughton JF, Schneider D (1990) Practical selectivity estimation through adaptive sampling. In: Proceedings of the 1990 ACM SIGMOD international conference on management of data, Atlantic City, NJ, May 1990

  22. Lee J, Kim D, Chung C (1999) Multi-dimensional selectivity estimation using compressed histogram information. In: Proceedings of the 1999 ACM SIGMOD international conference on management of data, Philadelphia, June 1999

  23. Matias Y, Scott Vitter J, Wang M (1998) Wavelet-based histograms for selectivity estimation. In: Proceedings of the 1998 ACM SIGMOD international conference on management of data, Seattle, June 1998

  24. Matias Y, Scott Vitter J, Wang M (2000) Dynamic maintenance of wavelet-based histograms. In: Proceedings of the 26th international conference on very large data bases (VLDB 2000), Cairo, Egypt, September 2000

  25. Muralikrishna M, DeWitt DJ (1988) Equi-depth histograms for estimating selectivity factors for multi-dimensional queries. In: Proceedings of the 1988 ACM SIGMOD international conference on management of data, Chicago, June 1988

  26. Muthukrishnan S, Poosala V, Suel T (1999) On rectangular partitionings in two dimensions: algorithms, complexity, and applications. In: Proceedings of the ICDT 1999, Jerusalem, January 1999, pp 236-256

  27. Olken F, Rotem D (1990) Random sampling from database files: a survey. In: Proceedings of the 5th international conference on statistical and scientific database management, Charlotte, NC, July 1990

  28. Poosala V, Ganti V (1999) Fast approximate answers to aggregate queries on a data cube. In: Proceedings of the 11th international conference on scientific and statistical database management, Cleveland, OH, July 1999

  29. Poosala V, Ioannidis YE (1997) Selectivity estimation without the attribute value independence assumption. In: Proceedings of the 23rd international conference on very large data bases (VLDB 1997), Athens, Greece, August 1997

  30. Poosala V, Ioannidis YE, Haas PJ, Shekita EJ (1996) Improved histograms for selectivity estimation of range predicates. In: Proceedings of the 1996 ACM SIGMOD international conference on management of data, Montreal, May 1996

  31. Scott D (1992) Multivariate density estimation: theory, practice and visualization. Wiley, New York

    Google Scholar 

  32. Selinger PG, Astrahan MM, Chamberlin DD, Lorie RA, Price TG (1979) Access path selection in a relational database management system. In: Proceedings of the 1979 ACM SIGMOD international conference on management of data, Boston, June 1979

  33. Shanmugasundaram J, Fayyad U, Bradley P (1988) Compressed data cubes for OLAP aggregate query approximation on continuous dimensions. In: Proceedings of the 5th ACM SIGKDD international conference on knowledge discovery and data mining, San Diego, August 1988

  34. Silverman BW (1986) Density estimation for statistics and data analysis. Monographs on statistics and applied probability, Chapman & Hall, New York

  35. TPC benchmark D (decision support) (1995)

  36. Vitter JS, Wang M (1999) Approximate computation of multidimensional aggregates of sparse data using wavelets. In: Proceedings of the 1999 ACM SIGMOD international conference on management of data, Philadelphia, June 1999

  37. Vitter JS, Wang M, Iyer BR (1998) Data cube approximation and histograms via wavelets. In: Proceedings of the 1998 ACM CIKM international conference on information and knowledge management, Bethesda, MD, November 1998

  38. Wand MP, Jones MC (1995) Kernel smoothing. Monographs on statistics and applied probability, Chapman & Hall, New York

  39. Webber R, Schek HJ, Blott S (1998) A quantitative analysis and performance study for similarity search methods in high-dimensional spaces. In: Proceedings of the 24rd international conference on very large data bases, New York, August 1998

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Bourns College of Engineering, University of California, Riverside, CA 92521, Riverside, USA

    Dimitrios Gunopulos & Vassilis J. Tsotras

  2. Department of Computer Science, Boston University, MA 02215, Boston, USA

    George Kollios

  3. Department of Information and Software Engineering, George Mason University, VA 22030, Fairfax, USA

    Carlotta Domeniconi

Authors
  1. Dimitrios Gunopulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. George Kollios
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vassilis J. Tsotras
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carlotta Domeniconi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dimitrios Gunopulos.

Additional information

Received: 30 January 2001, Accepted: 9 June 2003, Published online: 4 March 2004

Edited by: Y. Ioannidis

Dimitrios Gunopulos: Supported by NSF ITR-0220148, NSF IIS-9907477 CAREER Award, NSF IIS-9984729, and NRDRP.

George Kollios: Supported by NSF IIS-0133825 CAREER Award.

Vassilis J. Tsotras: Supported by NSF IIS-9907477 and the US Dept. of Defense.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gunopulos, D., Kollios, G., Tsotras, V.J. et al. Selectivity estimators for multidimensional range queries over real attributes. The VLDB Journal 14, 137–154 (2005). https://doi.org/10.1007/s00778-003-0090-4

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00778-003-0090-4

Keywords

Search

Navigation