Skip to main content
SpringerLink
Log in

Locally linear spatial pyramid hash for large-scale image search

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hash-based methods can achieve a fast similarity search by representing high-dimensional data with compact binary codes. However, the spatial structure in row images was always lost in most previous methods. In this paper, a novel Locally Linear Spatial Pyramid Hash(LLSPH) algorithm is developed for the task of fast image retrieval. Unlike the conventional approach, the spatial extent of image features is exploited in our method. The spatial pyramid structure is used both to construct binary hash codes and to increase the discriminability of the description. To generate interpretable binary codes, the proposed LLSPH method captures the spatial characteristics of the original SPM and generates a low-dimensional sparse representation using multi-dictionaries Locality-constrained Linear Coding(MD_LLC). LLSPH then converts the low-dimensional data into Hamming space by the TF-IDF binarization rule. Our experimental results show that our LLSPH method can outperform several state-of-the-art hashing algorithms on the Caltech256 and ImageNet-500 datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://www.image-net.org

References

  1. Andoni A, Indyk P (2006) Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: 47th Annual IEEE Symposium on Foundations of Computer Science, 2006. FOCS’06. IEEE, pp 459–468

  2. Cherian A, Morellas V, Papanikolopoulos N (2012) Robust sparse hashing. In: 2012 19th IEEE International Conference on Image Processing (ICIP). IEEE, pp 2417–2420

  3. Chum O, Matas J (2012) Fast computation of min-hash signatures for image collections. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3077–3084

  4. Chum O, Philbin J, Zisserman A, et al (2008) Near duplicate image detection: min-hash and tf-idf weighting. In: BMVC, vol 810, pp 812–815

  5. Drineas P, Mahoney MW (2005) On the nystrm method for approximating a gram matrix for improved kernel-based learning. J Mach Learn Res 6:2153–2175

    MathSciNet  MATH  Google Scholar 

  6. Gao L, Song J, Nie F, Yan Y, Sebe N, Shen HT (2015) Optimal graph learning with partial tags and multiple features for image and video annotation. In: CVPR

  7. Gao L, Song J, Zou F, Zhang D, Shao J (2015) Scalable multimedia retrieval by deep learning hashing with relative similarity learning. In: ACM Multimedia, pp 903–906

  8. Gionis A, Indyk P, Motwani R, et al. (1999) Similarity search in high dimensions via hashing. In: VLDB, vol 99, pp 518–529

  9. Gong Y, Lazebnik S (2011) Iterative quantization: A procrustean approach to learning binary codes. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 817–824

  10. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset

  11. Indyk P, Motwani R (1998) Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the thirtieth annual ACM symposium on Theory of computing. ACM, pp 604–613

  12. Irie G, Li Z, Wu XM, Chang SF (2014) Locally linear hashing for extracting non-linear manifolds. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2123–2130

  13. Jain P, Kulis B, Grauman K (2008) Fast image search for learned metrics. In: Computer Vision and Pattern Recognition, 2008. CVPR 2008. Conference on IEEE. IEEE, pp 1–8

  14. Kong W, Li WJ (2012) Isotropic hashing. In: Advances in Neural Information Processing Systems, pp 1646–1654

  15. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 2. IEEE, pp 2169–2178

  16. Mu Y, Shen J, Yan S (2010) Weakly-supervised hashing in kernel space. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3344–3351

  17. Muja M, Lowe DG (2009) Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP 1:2

    Google Scholar 

  18. Nie F, Xu D, Tsang WH, Zhang C (2010) Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    Article  MathSciNet  MATH  Google Scholar 

  19. Norouzi M, Blei DM, Salakhutdinov RR (2012) Hamming distance metric learning. In: Advances in neural information processing systems, pp 1061–1069

  20. Raginsky M, Lazebnik S (2009) Locality-sensitive binary codes from shift-invariant kernels. In: Advances in neural information processing systems, pp 1509–1517

  21. Salakhutdinov R, Hinton G (2009) Semantic hashing. Int J Approx Reason 50(7):969–978

    Article  Google Scholar 

  22. Sivic J, Zisserman A (2003) Video google: A text retrieval approach to object matching in videos. In: Proceedings. Ninth IEEE International Conference on Computer Vision, 2003. IEEE, pp 1470–1477

  23. Song J, Gao L, Yan Y, Zhang D, Sebe N (2015) Supervised hashing with pseudo labels for scalable multimedia retrieval. In: ACM Multimedia, pp 827–830

  24. Song J, Gao L, Zou F, Yan Y, Sebe N (2016) Deep and fast: Deep learning hashing with semi-supervised graph construction * Image & Vision Computing

  25. Song J, Shen HT, Wang J, Huang Z (2016) A distance-computation-free search scheme for binary code databases. IEEE Trans Multimedia 18(3):1–1

    Article  Google Scholar 

  26. Song J, Yang Y, Huang Z, Shen HT, Luo J (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimedia 15(8):1997–2008

    Article  Google Scholar 

  27. Song J, Yang Y, Li X, Huang Z, Yang Y (2014) Robust hashing with local models for approximate similarity search. IEEE Transactions on Cybernetics 44 (7):1225–1236

    Article  Google Scholar 

  28. Song J, Yang Y, Yang Y, Huang Z, Shen HT (2013) Inter-media hashing for large-scale retrieval from heterogeneous data sources. In: ACM SIGMOD International Conference on Management of Data, pp 785–796

  29. Turpin A, Scholer F (2006) User performance versus precision measures for simple search tasks. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 11–18

  30. Wang J, Kumar S, Chang SF (2010) Semi-supervised hashing for scalable image retrieval. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3424–3431

  31. Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3360–3367

  32. Wang J, Zhang T, Song J, Sebe N, Shen HT (2016) A survey on learning to hash

  33. Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: Advances in neural information processing systems, pp 1753–1760

  34. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE, pp 1794–1801

  35. Zhang D, Wang J, Cai D, Lu J (2010) Self-taught hashing for fast similarity search. In: Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval. ACM, pp 18–25

  36. Zhou K, Liu Y, Song J, Yan L, Zou F, Shen F (2015) Deep self-taught hashing for image retrieval. In: ACM International Conference on Multimedia, pp 1215–1218

  37. Zhu X, Huang Z, Cheng H, Cui J, Shen HT (2013) Sparse hashing for fast multimedia search. ACM Trans Inf Syst (TOIS) 31(2):9

    Article  Google Scholar 

  38. Zou F, Chen Y, Song J, Zhou K, Yang Y, Sebe N (2015) Compact image fingerprint via multiple kernel hashing. IEEE Trans Multimedia 17(7):1–1

    Article  Google Scholar 

Download references

Acknowledgments

This research is partly supported by National Science Foundation of China under Grant 61272285, National High-Technology Program of China (863 Program, Grant No.2014AA012301), Program for Changjiang Scholars and Innovative Research Team in University (No.IRT13090), and Program of Shaanxi Province Innovative Research Team (No.2014KCT-17).

Author information

Authors and Affiliations

  1. School of Information and Technology, Northwest University of China, Xi’an, China

    Wanqing Zhao, Hangzai Luo & Jinye Peng

  2. Department of Computer Science, UNC-Charlotte, Charlotte, NC, 28223, USA

    Jianping Fan

Authors
  1. Wanqing Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hangzai Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jinye Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianping Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hangzai Luo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, W., Luo, H., Peng, J. et al. Locally linear spatial pyramid hash for large-scale image search. Multimed Tools Appl 77, 109–123 (2018). https://doi.org/10.1007/s11042-016-4221-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4221-5

Keywords

Search

Navigation