Skip to main content
Log in

Hierarchical deep hashing for image retrieval

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

We present a new method to generate efficient multi-level hashing codes for image retrieval based on the deep siamese convolutional neural network (DSCNN). Conventional deep hashing methods trade off the capability of capturing highly complex and nonlinear semantic information of images against very compact hash codes, usually leading to high retrieval efficiency but with deteriorated accuracy. We alleviate the restrictive compactness requirement of hash codes by extending them to a two-level hierarchical coding scheme, in which the first level aims to capture the high-level semantic information extracted by the deep network using a rich encoding strategy, while the subsequent level squeezes them to more global and compact codes. At running time, we adopt an attention-based mechanism to select some of its most essential bits specific to each query image for retrieval instead of using the full hash codes of the first level. The attention-based mechanism is based on the guides of hash codes generated by the second level, taking advantage of both local and global properties of deep features. Experimental results on various popular datasets demonstrate the advantages of the proposed method compared to several state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Smeulders A W M, Worring M, Santini S, Gupta A, Jain R. Contentbased image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12): 1349–1380

    Article  Google Scholar 

  2. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, 2012, 25(2): 2012

    Google Scholar 

  3. Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 580–587

    Google Scholar 

  4. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1337–1342

    Google Scholar 

  5. Zheng L, Yang Y, Tian Q. SIFT Meets CNN: a decade survey of instance retrieval. 2016, arXiv preprint arXiv:1608.01807

    Google Scholar 

  6. Babenko A, Slesarev A, Chigorin A, Lempitsky V. Neural codes for image retrieval. In: Proceedings of European Conference on Computer Vision. 2014, 584–599

    Google Scholar 

  7. Razavian A S, Azizpour H, Sullivan J, Carlsson S. CNN features offthe-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014, 512–519

    Google Scholar 

  8. Babenko A, Lempitsky V. Aggregating deep convolutional features for image retrieval. In: Proceedings of the IEEE Conference on Computer Vision. 2015, 1269–1277

    Google Scholar 

  9. Tolias G, Sicre R, Jégou H. Particular object retrieval with integral max-pooling of CNN activations. Computer Science, 2015

    Google Scholar 

  10. Ng Y H, Yang F, Davis L S. Exploiting local features from deep networks for image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 53–61

    Google Scholar 

  11. Zheng L, Zhao Y L, Wang S J, Wang J D, Tian Q. Good practice in CNN feature transfer. 2016, arXiv preprint arXiv:1604, 00133

    Google Scholar 

  12. Zheng L, Wang S J, Wang J D, Tian Q. Accurate image search with multi-scale contextual evidences. International Journal of Computer Vision, 2016(1): 1–13

    Article  MathSciNet  Google Scholar 

  13. Zhou B L, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 2921–2929

    Google Scholar 

  14. Andoni A, Indyk P. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. In: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science. 2006, 459–468

    Google Scholar 

  15. Liong V E, Lu JW, Wang G, Moulin P, Zhou J. Deep hashing for compact binary codes learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 2475–2483

    Google Scholar 

  16. Zhao F, Huang Y Z, Wang L, Tan T N. Deep semantic ranking based hashing for multi-label image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1556–1564

    Google Scholar 

  17. Chang S F. Supervised hashing with kernels. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2074–2081

    Google Scholar 

  18. Gong Y C, Pawlowski M, Yang F, Brandy L, Bourdev L, Fergus R. Web scale photo hash clustering on a single machine. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 19–27

    Google Scholar 

  19. Xia R K, Pan Y, Lai H J, Liu C, Yan S C. Supervised Hashing for Image Retrieval via Image Representation Learning. In: Proceedings of the 28th AAAI Conference on Artificial Intelligence. 2014

    Google Scholar 

  20. Li W J, Wang S, Kang W C. Feature learning based deep supervised hashing with pairwise labels. Computer Science, 2015

    Chapter  Google Scholar 

  21. Liu H M, Wang R P, Shan S G, Chen X L. Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 2064–2072

    Google Scholar 

  22. Paulin M, Douze M, Harchaoui Z, Mairal J, Perronin F, Schmid C. Local convolutional features with unsupervised training for image retrieval. In: Proceedings of the IEEE International Conference on Computer Vision. 2015, 91–99

    Google Scholar 

  23. Kalantidis Y, Mellina C, Osindero S. Cross-dimensional weighting for aggregated deep convolutional features. In: Proceedings of European Conference on Computer Vision. 2016, 685–701

    Google Scholar 

  24. Salvador A, Giroinieto X, Marques F, Satoh S I. Faster R-CNN features for instance search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2016, 9–16

    Google Scholar 

  25. Lin K, Yang H F, Hsiao J H, Chen C S. Deep learning of binary hash codes for fast image retrieval. In: Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2015, 27–35

    Google Scholar 

  26. Lai H J, Pan Y, Liu Y, Yan S C. Simultaneous feature learning and hash coding with deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 3270–3278

    Google Scholar 

  27. Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. In: Proceedings of European Conference on Computer Vision. 2013, 818–833

    Google Scholar 

  28. Mahendran A, Vedaldi A. Understanding deep image representations by inverting them. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 5188–5196

    Google Scholar 

  29. Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report. 2012

    Google Scholar 

  30. Jegou H, Douze M, Schmid C. Hamming embedding and weak geometric consistency for large scale image search. In: Proceedings of European conference on computer vision. 2008, 304–317

    Google Scholar 

  31. Philbin J, Chum O, Isard M, Sivic J, Zisserman A. Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE International Conference on Computer Vision. 2007, 1–8

    Google Scholar 

  32. Philbin J, Chum O, Isard M, Sivic J, Zisserman A. Lost in quantization: improving particular object retrieval in large scale image databases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2008, 1–8

    Google Scholar 

  33. Nister D, Stewenius H. Scalable recognition with a vocabulary tree. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2006, 2161–2168

    Google Scholar 

  34. Jia Y Q, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T. Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on Multimedia. 2014, 675–678

    Google Scholar 

  35. Gong Y C, Lazebnik S. Iterative quantization: A procrustean approach to learning binary codes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2011, 817–824

    Google Scholar 

  36. Weiss Y, Torralba A, Fergus R. Spectral hashing. In: Proceedings of the Neural Information Processing Systems Conference. 2008, 1753–1760

    Google Scholar 

  37. Heo J P, Lee Y, He J, Chang S F, Yoon S E. Spherical hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2012, 2957–2964

    Google Scholar 

  38. Jiang Q Y, Li WJ. Scalable graph hashing with feature transformation. In: Proceedings of the International Conference on Artificial Intelligence. 2015, 331–337

    Google Scholar 

  39. Lin G S, Shen C H, Shi Q F, van den Hengel A, Suter D. Fast supervised hashing with decision trees for high-dimensional data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 1971–1978

    Google Scholar 

  40. Shen FM, Shen C H, Liu W, Shen H T. Supervised discrete hashing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015, 37–45

    Google Scholar 

  41. Zhang P C, Zhang W, Li WJ, Guo MY. Supervised hashing with latent factor models. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2014, 173–182

    Google Scholar 

  42. Kang W C, Li W J, Zhou Z H. Column sampling based discrete supervised hashing. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. 2016

    Google Scholar 

  43. Zhang R M, Lin L, Zhang R, Zuo W M, Zhang L. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing, 2015, 24(12): 4766–4779

    Article  MathSciNet  Google Scholar 

  44. Arandjelovic R, Zisserman A. All about VLAD. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013, 1578–1585

    Google Scholar 

  45. Jégou H, Zisserman A. Triangulation embedding and democratic aggregation for image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014, 3310–3317

    Google Scholar 

  46. Razavian A S, Sullivan J, Carlsson S, Maki A. Visual instance retrieval with deep convolutional networks. 2014, arXiv preprint arXiv:1412.6574

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (Grant Nos. 61373060 and 61672280) and Qing Lan Project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoyang Tan.

Additional information

Ge Song received his BS degree in computer science and technology from Zhengzhou University, China in 2014. Now he is a PhD student in Nanjing University of Aeronautics and Astronautics, China. His research interests are in image retrieval, machine learning, pattern recognition, and computer vision.

Xiaoyang Tan received his BS and MS degrees in computer applications from Nanjing University of Aeronautics and Astronautics (NUAA), China in 1993 and 1996, respectively. Then he worked at NUAA in June 1996 as an assistant lecturer. He received a PhD degree from Department of Computer Science and Technology of Nanjing University, China in 2005. From September 2006 to October 2007, he worked as a postdoctoral researcher in the LEAR (Learning and Recognition in Vision) team at INRIA Rhone-Alpes in Grenoble, France. His research interests are in face recognition, machine learning, pattern recognition, and computer vision. In these fields, he has authored or coauthored over 40 scientific papers.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Song, G., Tan, X. Hierarchical deep hashing for image retrieval. Front. Comput. Sci. 11, 253–265 (2017). https://doi.org/10.1007/s11704-017-6537-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11704-017-6537-3

Keywords

Navigation