Abstract
Classical multi-dimensional indexes are based on data space partitioning. The effectiveness declines because the number of indexing units grows exponentially as the number of dimensions increases. Then, unfortunately, using such index structures is less effective than linear scanning of all the data. The VA-file proposed a method of coordinate approximation, observing that nearest neighbor search becomes of linear complexity in high-dimensional spaces.
In this paper we propose C2VA(Clustered Compact VA) for dimensionality reduction. We investigate and find that real datasets are rarely uniformly distributed, which is the main assumption of VA-file. Instead of approximation on all dimensions, we figure out the condition of skipping less important dimensions. This avoids the problem of generating huge index file for a large, high dimensional dataset and hence saves a lot of I/O accesses when scanning. Moreover, we guarantee that C2VA preserves the precision of bounds as in VA-file, which maximizes the efficiency gain. The conviction is found in our experimental results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
An, J., Chen, H., Furuse, K., Ishikawa, M., Ohbo, N.:’ The Convex Polyhedra Technique: An Index Strucrture for High-dimensional Space’, Proc. of the 13th Australasian Database Conference, 2002. pp. 33–40.
An, J., Chen, H., Furuse, K., Ishikawa, M., Ohbo, N.:’ A Vector-wise Dimensionality Reduction for Indexing High Dimensional Data’, Proc. Pan-Yellow-Sea Inte’l Workshop on Infor. Technologies for Network Era, pp. 135–140. 2002.
Aggarwal, C., Yu, P.: Finding Generalized Projected Clusters in High Dimensional Spaces’ Proc. SIGMOD, pp. 70–81, 2000.
Berchtold, S., Bohm, C., Keim, D., Kriegel, H.-P.:’ A Cost Model For Nearest Neighbor Search in High-Dimensional Data Space’, ACM PODS Symposium on Principles of Database Systems, Tucson, Arizona, pp. 78–86, 1997.
Beckmann, N., Kriegel, H.-P., Schneider, R., Seeger, B.:’ The R*-tree:An efficient and Robust Access Method for Points and Rectangles’, Proc. SIGMOD, pp. 322–331, 1990.
Beyer, K. S., Goldstein, J., Ramakrishnan, R., Shaft U.:’ When Is “Nearest Neighbor” Meaningful’, Proc. 7th ICDT, pp. 217–235, 1999.
Fukunaga, K.:’ Statistical Pattern Recognition’, Cademic Press-October 1990.
Katayama, N., Satoh, S.:’ The SR-tree: An Index Structure for High-Dimensional Nearest Neighbor Queries.’, Proc. ACM SIGMOD Int. Conf. Management of Data, Tucson, Arizona, pp. 369–380, 1997.
Weber, R., Schek, H. J., Blott, S.:’A Quantitative Analysis and Performance Study for Similarity-Search Methods in high-Dimensional Spaces.’, Proc. of the VLDB conference, New York, pp. 194–205, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, H., An, J., Furuse, K., Ohbo, N. (2002). C2VA: Trim High Dimensional Indexes. In: Meng, X., Su, J., Wang, Y. (eds) Advances in Web-Age Information Management. WAIM 2002. Lecture Notes in Computer Science, vol 2419. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45703-8_28
Download citation
DOI: https://doi.org/10.1007/3-540-45703-8_28
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44045-1
Online ISBN: 978-3-540-45703-9
eBook Packages: Springer Book Archive