Abstract
Bitmap indices are popular multi-dimensional data structures for accessing read-mostly data such as data warehouse (DW) applications, decision support systems (DSS) and on-line analytical processing (OLAP). One of their main strengths is that they provide good performance characteristics for complex adhoc queries and an efficient combination of multiple index dimensions in one query. Considerable research work has been done in the area of finite (and low) attribute cardinalities. However, additional complexity is imposed on the design of bitmap indices for high cardinality or even non-discrete attributes, where different optimisation techniques than the ones proposed so far have to be applied.
In this paper we discuss the design and implementation of bitmap indices for High-Energy Physics (HEP) analysis, where the potential search space consists of hundreds of independent dimensions. A single HEP query typically covers 10 to 100 dimensions out of the whole search space. In this context we evaluated two different bitmap encoding techniques, namely equality encoding and range encoding. For both methods the number of bit slices (or bitmap vectors) per attribute is a central optimisation parameter. The paper presents some (first) results for choosing the optimal number of bit slices for multi-dimensional indices with attributes of different value distribution and query selectivity. We believe that this discussion is not only applicable to HEP but also to DW, DSS and OLAP type problems in general.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
S. Berchtold, C. Boehm, H.-P. Kriegel, The Pyramid-Tree: Breaking the Curse of Dimensionality, SIGMOD 1998, Proceedings ACM SIGMOD International Conference on Management of Data, June 1998, Seattle, Washington, USA
C. Chan, Y.E. Ioannidis, Bitmap Index Design and Evaluation, SIGMOD 1998, Proceedings ACM SIGMOD International Conference on Management of Data, June 1998, Seattle, Washington, USA
C. Chan, Y.E. Ioannidis, An Efficient Bitmap Encoding Scheme for Selection Queries, SIGMOD 1999, Proceedings ACM SIGMOD International Conference on Management of Data, June 1999, Philadephia, Pennsylvania, USA
M. Freestone, A General Solution of the n-dimensional B-tree Problem, SIGMOD Record (ACM Special Interest Group on Management of Data 24(2), June 1995
V. Gaeda, O. Guenther, Multidimensional Access Methods, Computing Surveys 30, September 1998
A. Guttman, R-trees: A Dynamic Index Structure for Spatial Searching, Proc. ACM SIGMOD, Int. Conf. on Management of Data, 1984
K. Holtman, P. v. d. Stok, I. Willers, Automatic Reclustering Objects in Very Large Databases for High Energy Physics, Proceedings of IDEAS 1998, Cardiff, UK
P. O’Neil, Informix and Indexing Support for Data Warehouses, Database and Programming Design, February 1997
P. O’Neil, D. Quass, Improved Query Performance with Variant Indexes, Proceedings ACM SIGMOD International Conference on Management of Data, May 1997, Tucson, Arizona, USA
A. Shoshani, L.M. Bernardo, H. Nordberg, D. Rotem, A. Sim, Multidimensional Indexing and Query Coordination for Tertiary Storage Management, 11th International Conference on Scientific and Statistical Database Management, Proceedings, Cleveland, Ohio, USA, July 1999
M. Wu, A.P. Buchmann, Encoded Bitmap Indexing for Data Warehouses, Proceedings of the Fourteenth International Conference on Data Engineering, February 1998, Orlando, Florida, USA
M. Wu, Query Optimization for Selections Using Bitmaps, SIGMOD 1999, Proceedings ACM SIGMOD International Conference on Management of Data, June 1999, Philadephia, Pennsylvania, USA
K. Wu, P.S. Yu, Range-Based Bitmap Indexing for High-Cardinality Attributes with Skew, Technical Report, IBM Watson Research Center, May 1996
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Stockinger, K., Duellmann, D., Hoschek, W., Schikuta, E. (2000). Improving the Performance of High-Energy Physics Analysis through Bitmap Indices. In: Ibrahim, M., Küng, J., Revell, N. (eds) Database and Expert Systems Applications. DEXA 2000. Lecture Notes in Computer Science, vol 1873. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44469-6_78
Download citation
DOI: https://doi.org/10.1007/3-540-44469-6_78
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67978-3
Online ISBN: 978-3-540-44469-5
eBook Packages: Springer Book Archive