Abstract
A sketch is a lossy compression of high-dimensional data into compact bit strings such as locality sensitive hash. In general, k nearest neighbor search using sketch consists of the following two stages. The first stage narrows down the top K candidates, for some \(K \ge k\), using a priority measure of sketch as a filter. The second stage selects the k nearest objects from K candidates. In this paper, we discuss the search algorithms using fast filtering by sketch enumeration without using matching. Surprisingly, the search performance is rather improved by the proposed method when narrow sketches with smaller number of bits such as 16-bits than the conventional ones are used. Furthermore, we compare the search efficiency by sketches of various widths for several databases, which have different numbers of objects and dimensionalities. Then, we can observe that wider sketches are appropriate for larger databases, while narrower sketches are appropriate for higher dimension.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ciaccia, P., Patella, M., Zezula, P.: M-tree: an efficient access method for similarity search in metric spaces. In: Proceedings VLBD’97, pp. 426–435 (1997)
Dong, W., Charikar, M., Li, K.: Asymmetric distance estimation with sketches for similarity search in high-dimensional spaces. In: Proceedings ACM SIGIR’08, pp. 123–130 (2008)
Fukunaga, K.: Statistical Pattern Recognition, 2nd edn. Academic Press, Cambridge (1990)
Guttman, A.: R-trees: a dynamic index structure for spatial searching. In: Yormark, B. (ed.) Proceedings SIGMOD’84, pp. 47–57. ACM Press (1984)
Higuchi, N., Imamura, Y., Kuboyama, T., Hirata, K., Shinohara, T.: Nearest neighbor search using sketches as quantized images of dimension reduction. In: Proceedings ICPRAM 2018, pp. 356–363 (2018)
Higuchi, N., Imamura, Y., Kuboyama, T., Hirata, K., Shinohara, T.: Fast nearest neighbor search with narrow 16-bit sketch. In: Proceedings ICPRAM 2019, pp. 540–547 (2019)
Imamura, Y., Higuchi, N., Kuboyama, T., Hirata, K., Shinohara, T.: Pivot selection for dimension reduction using annealing by increasing resampling. In: Proceedings LWDA 2017, pp. 15–23 (2017)
Mic, V., Novak, D., Zezula, P.: Improving sketches for similarity search. In: Proceedings MEMICS’15, pp. 45–57 (2015)
Mic, V., Novak, D., Zezula, P.: Speeding up similarity search by sketches. In: Amsaleg, L., Houle, M.E., Schubert, E. (eds.) SISAP 2016. LNCS, vol. 9939, pp. 250–258. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46759-7_19
Müller, A., Shinohara, T.: Efficient similarity search by reducing I/O with compressed sketches. In: Proceedings SISAP’09, pp. 30–38 (2009)
Shinohara, T., Ishizaka, H.: On dimension reduction mappings for approximate retrieval of multi-dimensional data. In: Arikawa, S., Shinohara, A. (eds.) Progress in Discovery Science. LNCS, vol. 2281, pp. 224–231. Springer, Heidelberg (2002). https://doi.org/10.1007/3-540-45884-0_14
Wang, Z., Dong, W., Josephson, W., Lv, Q., Charikar, M., Li, K.: Sizing sketches: a rank-based analysis for similarity search. In: Proceedings ACM SIGMETRICS’07, pp. 157–168 (2007)
Yianilos, P.: Data structures and algorithms for nearest neighbor search in general metric spaces. In: Proceedings SODA 1993, pp. 311–321. ACM Press (1993)
Zezula, P., Amato, G., Dohnal, V., Batko, M.: Similarity Search: The Metric Space Approach. Advances in Database Systems. Springer, Heidelberg (2006). https://doi.org/10.1007/0-387-29151-2
Acknowledgments
This work was partially supported by JSPS KAKENHI Grant Numbers 16H02870, 17H00762, 16H01743, 17H01788, and 18K11443.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Higuchi, N., Imamura, Y., Kuboyama, T., Hirata, K., Shinohara, T. (2019). Fast Filtering for Nearest Neighbor Search by Sketch Enumeration Without Using Matching. In: Liu, J., Bailey, J. (eds) AI 2019: Advances in Artificial Intelligence. AI 2019. Lecture Notes in Computer Science(), vol 11919. Springer, Cham. https://doi.org/10.1007/978-3-030-35288-2_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-35288-2_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-35287-5
Online ISBN: 978-3-030-35288-2
eBook Packages: Computer ScienceComputer Science (R0)