Abstract
Vector quantization has been widely employed in nearest neighbor search because it can approximate the Euclidean distance of two vectors with the table look-up way that can be precomputed. Additive quantization (AQ) algorithm validated that low approximation error can be achieved by representing each input vector with a sum of dependent codewords, each of which is from its own codebook. However, the AQ algorithm relies on computational expensive beam search algorithm to encode each vector, which is prohibitive for the efficiency of the approximate nearest neighbor search. In this paper, we propose a fast AQ algorithm that significantly accelerates the encoding phase. We formulate the beam search algorithm as an optimization of codebook selection orders. According to the optimal order, we learn the codebooks with hierarchical construction, in which the search width can be set very small. Specifically, the codewords are firstly exchanged into proper codebooks by the indexed frequency in each step. Then the codebooks are updated successively to adapt the quantization residual of previous quantization level. In coding phase, the vectors are compressed with learned codebooks via the best order, where the search range is considerably reduced. The proposed method achieves almost the same performance as AQ, while the speed for the vector encoding phase can be accelerated dozens of times. The experiments are implemented on two benchmark datasets and the results verify our conclusion.
Similar content being viewed by others
References
Babenko A and Lempitsky V (2014) Additive Quantization for Extreme Vector Compression, in: 2014 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), IEEE
Barnes C, Rizvi S, Nasrabadi N (1996) Advances in residual vector quantization: a review. IEEE Trans Image Process 2:226–262
Boiman O, Shechtman E, and Irani M (2008) In defense of nearest-neighbor based image classification, in: 2008 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), IEEE
Brandt J (2010) “Transform Coding for Fast Approximate Nearest Neighbor Search in High Dimensions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1815–1822
Calonder M, Lepetit V, Strecha Christoph, and Fua Pascal (2010) BRIEF: Binary Robust Independent Elementary Features, In: European Conference on Computer Vision (ECCV), Springer
Chen Y, Guan T, Wang C (2010) Approximate nearest neighbor search by residual vector quantization. Sensors 10(12):11259–11273
Douze M, Jégou H, Singh H, Amsaleg L, and Schmid C (2009) Evaluation of GIST descriptors for web-scale image search, in: International Conference on Image and Video Retrival
Ge T, He K, Ke Q, and Sun J (2013) Optimized product quantization for approximate nearest neighbor search, in: 2013 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), IEEE
Gionis A, Indyk P, R (1999) Motwani, Similarity search in high dimensions via hashing, VLDB. pp:518–529
Gong Y, Lazebnik S (2013) Iterative quantization: a procrustean approach to learning binary codes, trans. Pattern analysis and machine. Intelligence 35(12):2916–2929
Gray R, Neuhoff D (1998) Quantization, IEEE Trans. Information. Theory 44:2325–2383
Guo Q, Zeng Z, Zhang S, Zhang G, Zhang Y (2016) Adaptive bit allocation product quantization. Neurocomputing 171:866–877
He K. the source code of optimization product quantization, https://research.microsoft.com/en-us/um/people/kahe/cvpr13
Jégou H, the source code of product quantization, https://gforge.inria.fr/frs/download.php/33241
Jégou H, Douze M, Schmid C (2011) Product quantization for nearest neighbor search, trans. Pattern analysis and machine. Intelligence 33(1)
Kulis B and Grauman K. Kernelized (2009) Locality-sensitive hashing for scalable image search, in: IEEE International Conference on Computer Vision, IEEE
Lloyd S (1982) Least squares quantization in PCM,” IEEE Trans. Information. Theory 28(2)
Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2)
Mallat S, Zhang Z (1993) Matching pursuit in a time frequency dictionary. IEEE Trans Signal Process 41(12)
Martinez J, Hoos H and Little J (2014) Stacked Quantizers for Compositional Vector Compression, in: arXiv preprint arXiv: 1411.2173
Obdrzalek S and Matas J (2005) Sub-linear indexing for large scale object recognition, in: 2005 British Machine Vision Conference (BMVC), BMVA press
Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3)
Pauleve L, Jegou H, Amsaleg L (2010) Locality sensitive hashing: a comparison of hash function types and querying mechanisms. Pattern Recogn Lett 31
Pearl J (1988) Probabilistic Reasoning in Intelligent Systems: Networksof Plausible Inference, Morgan Kaufmann, 3
Shapiro S (1987) Encyclopedia of Artificial Intelligence
Strecha C, Bronstein A, Bronstein M, Fua P (2010) Ldahash: improved matching with smaller descriptors, trans. Pattern analysis and machine. Intelligence 34
Torralba A, Fergus R, and Weiss Y (2008) Small codes and large image databases for recognition, in: 2008 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), IEEE
Wang J, Kumar S, and Chang S (2010) Semi-supervised hashing for scalable image retrieval, in: 2010 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), IEEE
Yu G and Yuan J (2014) Scalable forest hashing for fast similarity search, in: IEEE International Conference on Multimedia and Expo (ICME), IEEE
Zhang T, Du C, and Wang J (2014) Composite quantization for approximate nearest neighbor search, in: International Conference on Machine Learning (ICML)
Acknowledgments
This work was supported in part by the National Key Research and Development Program of China under grant No. 2016YFB1000903, and NSFC No. 61573268.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Li, J., Lan, X., Wang, J. et al. Fast additive quantization for vector compression in nearest neighbor search. Multimed Tools Appl 76, 23273–23289 (2017). https://doi.org/10.1007/s11042-016-4023-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-4023-9