Abstract
Bitmap indices have gained wide acceptance in data warehouse applications and are an efficient access method for querying large amounts of read-only data. The main trend in bitmap index research focuses on typical business applications based on discrete attribute values. However, scientific data that is mostly characterised by non-discrete attributes cannot be queried efficiently by currently supported access methods.
In our previous work [13] we introduced a novel bitmap algorithm called GenericRangeEval for efficiently querying scientific data. We evaluated our approach based primarily on uniformly distributed and independent data. In this paper we analyse the behaviour of our bitmap index algorithm against various queries based on different data distributions.
We have implemented an improved version of one of the most cited bitmap compression algorithms called Byte Aligned Bitmap Compression and adapted it to our bitmap indices. To prove the efficiency of our access method, we carried out high-dimensional queries against real data taken from two different scientific applications, namely High Energy Physics and Astronomy. The results clearly show that depending on the underlying data distribution and the query access patterns, our proposed bitmap indices can significantly improve the response time of high-dimensional queries when compared to conventional access methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
G. Antoshenkov, Byte-Aligned Bitmap Compression, Technical Report, Oracle Corp., 1994.
S. Amer-Yahia, T. Johnson, Optimizing Queries On Compressed Bitmaps, Proceedings of 26th International Conference on Very Large Data Bases, Cairo, Egypt, Sept. 2000, Morgan Kaufmann.
S. Berchtold, C. Boehm, H.-P. Kriegel, The Pyramid-Tree: Breaking the Curse of D imensionality, SIGMOD 1998, Proceedings ACM SIGMOD International Conference on Management of Data, Seattle, Washington, USA, June 1998.
C. Chan, Y.E. Ioannidis, Bitmap Index Design and Evaluation, In Proceedings ACM SIGMOD International Conference on Management of Data, Seattle, Washington, USA, June 1998.
C. Chan, Y.E. Ioannidis, An Efficient Bitmap Encoding Scheme for Selection Queries, Proceedings ACM SIGMOD International Conference on Management of Data, Philadephia, Pennsylvania, USA, June 1999.
T. Johnson, Performance Measurements of Compressed Bitmap Indices, Proceedings of 25th International Conference on Very Large Data Bases, Edinburgh, Scotland, UK, September 1999, Morgan Kaufmann.
N. Koudas, Space Efficient Bitmap Indexing, International Conference on Information and Knowledge Management, McLean, VA, USA, November 2000.
P. O’Neil, D. Quass, Improved Query Performance with Variant Indexes, Proceedings ACM SIGMOD International Conference on Management of Data, Tucson, Arizona, USA, May 1997.
A. Shoshani, L.M. Bernardo, H. Nordberg, D. Rotem, A. Sim, Multidimensional Indexing and Query Coordination for Tertiary Storage Management, 11th International Conference on Scientific and Statistical Database Management, Cleveland, Ohio, USA, July 1999.
A. Szalay, P. Kunszt, A. Thakar, J. Gray, D. Slutz, Designing and Mining Multi-Terabyte Astronomy Archives: The Sloan Digital Sky Survey, Proceedings ACM SIGMOD International Conference on Management of Data, Philadephia, Pennsylvania, USA, June 1999.
K. Stockinger, D. Duellmann, W. Hoschek, E. Schikuta. Improving the Performance of High Energy Physics Analysis through Bitmap Indices. International Conference on Database and Expert Systems Applications, London-Greenwich, UK, Sept. 2000. Springer-Verlag.
K. Stockinger, Design and Implementation of Bitmap Indices for Scientific Data, International Database Engineering & Applications Symposium, Grenoble, France, July 2001, IEEE Computer Society Press.
K. Stockinger, Performance Analysis of Generic vs. Sliced Tags in HepODBMS, International Conference on Computing in High Energy and Nuclear Physics, Beijing, China, September, 2001.
K. Stockinger, Multi-Dimensional Bitmap Indices for Optimising Data Access within Object Oriented Databases at CERN, Ph.D. Thesis, University of Vienna, Austria, November 2001.
M. Wu, A.P. Buchmann, Encoded Bitmap Indexing for Data Warehouses, Proceedings of the Fourteenth International Conference on Data Engineering, Orlando, Florida, USA, February 1998.
M. Wu, Query Optimization for Selections Using Bitmaps, SIGMOD 1999, Proceedings ACM SIGMOD International Conference on Management of Data, Philadephia, Pennsylvania, USA, June 1999.
K. Wu, E. J. Otoo, A. Shoshani, A Performance Comparison of Bitmap Indexes, International Conference on Information and Knowledge Management, Atlanta, Georgia, USA, November, 2001.
K. Wu, P.S. Yu, Range-Based Bitmap Indexing for High-Cardinality Attributes with Skew, Technical Report, IBM Watson Research Center, May 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stockinger, K. (2002). Bitmap Indices for Speeding Up High-Dimensional Data Analysis. In: Hameurlain, A., Cicchetti, R., Traunmüller, R. (eds) Database and Expert Systems Applications. DEXA 2002. Lecture Notes in Computer Science, vol 2453. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-46146-9_87
Download citation
DOI: https://doi.org/10.1007/3-540-46146-9_87
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44126-7
Online ISBN: 978-3-540-46146-3
eBook Packages: Springer Book Archive