Abstract
Hash-based methods can achieve a fast similarity search by representing high-dimensional data with compact binary codes. However, the spatial structure in row images was always lost in most previous methods. In this paper, a novel Locally Linear Spatial Pyramid Hash(LLSPH) algorithm is developed for the task of fast image retrieval. Unlike the conventional approach, the spatial extent of image features is exploited in our method. The spatial pyramid structure is used both to construct binary hash codes and to increase the discriminability of the description. To generate interpretable binary codes, the proposed LLSPH method captures the spatial characteristics of the original SPM and generates a low-dimensional sparse representation using multi-dictionaries Locality-constrained Linear Coding(MD_LLC). LLSPH then converts the low-dimensional data into Hamming space by the TF-IDF binarization rule. Our experimental results show that our LLSPH method can outperform several state-of-the-art hashing algorithms on the Caltech256 and ImageNet-500 datasets.
Similar content being viewed by others
Notes
References
Andoni A, Indyk P (2006) Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: 47th Annual IEEE Symposium on Foundations of Computer Science, 2006. FOCS’06. IEEE, pp 459–468
Cherian A, Morellas V, Papanikolopoulos N (2012) Robust sparse hashing. In: 2012 19th IEEE International Conference on Image Processing (ICIP). IEEE, pp 2417–2420
Chum O, Matas J (2012) Fast computation of min-hash signatures for image collections. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3077–3084
Chum O, Philbin J, Zisserman A, et al (2008) Near duplicate image detection: min-hash and tf-idf weighting. In: BMVC, vol 810, pp 812–815
Drineas P, Mahoney MW (2005) On the nystrm method for approximating a gram matrix for improved kernel-based learning. J Mach Learn Res 6:2153–2175
Gao L, Song J, Nie F, Yan Y, Sebe N, Shen HT (2015) Optimal graph learning with partial tags and multiple features for image and video annotation. In: CVPR
Gao L, Song J, Zou F, Zhang D, Shao J (2015) Scalable multimedia retrieval by deep learning hashing with relative similarity learning. In: ACM Multimedia, pp 903–906
Gionis A, Indyk P, Motwani R, et al. (1999) Similarity search in high dimensions via hashing. In: VLDB, vol 99, pp 518–529
Gong Y, Lazebnik S (2011) Iterative quantization: A procrustean approach to learning binary codes. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 817–824
Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset
Indyk P, Motwani R (1998) Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the thirtieth annual ACM symposium on Theory of computing. ACM, pp 604–613
Irie G, Li Z, Wu XM, Chang SF (2014) Locally linear hashing for extracting non-linear manifolds. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2123–2130
Jain P, Kulis B, Grauman K (2008) Fast image search for learned metrics. In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. Conference on IEEE. IEEE, pp 1–8
Kong W, Li WJ (2012) Isotropic hashing. In: Advances in Neural Information Processing Systems, pp 1646–1654
Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2. IEEE, pp 2169–2178
Mu Y, Shen J, Yan S (2010) Weakly-supervised hashing in kernel space. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3344–3351
Muja M, Lowe DG (2009) Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP 1:2
Nie F, Xu D, Tsang WH, Zhang C (2010) Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932
Norouzi M, Blei DM, Salakhutdinov RR (2012) Hamming distance metric learning. In: Advances in neural information processing systems, pp 1061–1069
Raginsky M, Lazebnik S (2009) Locality-sensitive binary codes from shift-invariant kernels. In: Advances in neural information processing systems, pp 1509–1517
Salakhutdinov R, Hinton G (2009) Semantic hashing. Int J Approx Reason 50(7):969–978
Sivic J, Zisserman A (2003) Video google: A text retrieval approach to object matching in videos. In: Proceedings. Ninth IEEE International Conference on Computer Vision, 2003. IEEE, pp 1470–1477
Song J, Gao L, Yan Y, Zhang D, Sebe N (2015) Supervised hashing with pseudo labels for scalable multimedia retrieval. In: ACM Multimedia, pp 827–830
Song J, Gao L, Zou F, Yan Y, Sebe N (2016) Deep and fast: Deep learning hashing with semi-supervised graph construction * Image & Vision Computing
Song J, Shen HT, Wang J, Huang Z (2016) A distance-computation-free search scheme for binary code databases. IEEE Trans Multimedia 18(3):1–1
Song J, Yang Y, Huang Z, Shen HT, Luo J (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimedia 15(8):1997–2008
Song J, Yang Y, Li X, Huang Z, Yang Y (2014) Robust hashing with local models for approximate similarity search. IEEE Transactions on Cybernetics 44 (7):1225–1236
Song J, Yang Y, Yang Y, Huang Z, Shen HT (2013) Inter-media hashing for large-scale retrieval from heterogeneous data sources. In: ACM SIGMOD International Conference on Management of Data, pp 785–796
Turpin A, Scholer F (2006) User performance versus precision measures for simple search tasks. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 11–18
Wang J, Kumar S, Chang SF (2010) Semi-supervised hashing for scalable image retrieval. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3424–3431
Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3360–3367
Wang J, Zhang T, Song J, Sebe N, Shen HT (2016) A survey on learning to hash
Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: Advances in neural information processing systems, pp 1753–1760
Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, pp 1794–1801
Zhang D, Wang J, Cai D, Lu J (2010) Self-taught hashing for fast similarity search. In: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 18–25
Zhou K, Liu Y, Song J, Yan L, Zou F, Shen F (2015) Deep self-taught hashing for image retrieval. In: ACM International Conference on Multimedia, pp 1215–1218
Zhu X, Huang Z, Cheng H, Cui J, Shen HT (2013) Sparse hashing for fast multimedia search. ACM Trans Inf Syst (TOIS) 31(2):9
Zou F, Chen Y, Song J, Zhou K, Yang Y, Sebe N (2015) Compact image fingerprint via multiple kernel hashing. IEEE Trans Multimedia 17(7):1–1
Acknowledgments
This research is partly supported by National Science Foundation of China under Grant 61272285, National High-Technology Program of China (863 Program, Grant No.2014AA012301), Program for Changjiang Scholars and Innovative Research Team in University (No.IRT13090), and Program of Shaanxi Province Innovative Research Team (No.2014KCT-17).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhao, W., Luo, H., Peng, J. et al. Locally linear spatial pyramid hash for large-scale image search. Multimed Tools Appl 77, 109–123 (2018). https://doi.org/10.1007/s11042-016-4221-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-4221-5