Skip to main content
SpringerLink
Log in

Deep hashing with top similarity preserving for image retrieval

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hashing has drawn more and more attention in image retrieval due to its high search speed and low storage cost. Traditional hashing methods project the high-dimensional hand-crafted visual features to compact binary codes by linear or non-linear hashing functions. Deep hashing methods, which integrate image representation learning and hash functions learning into a unified framework, have shown more superior performance. Most of existing supervised deep hashing methods mainly consider the semantic similarities among images by using pair-wise or triplet-wise constraints as supervision information. However, as a kind of crucial information, the rankings of the retrieval results, are neglected. Consequently, the produced hash codes may be suboptimal. In this paper, a new Deep Hashing with Top Similarity Preserving (DHTSP) method is proposed to optimize the quality of hash codes for image retrieval. Specifically, we utilize AlexNet to extract discriminative image representations directly from the raw image pixels and learn hash functions simultaneously. Then a top similarity preserving loss function is designed to preserve the similarity of returned images at the top of the ranking list. Experimental results on three benchmark datasets show that our proposed method outperforms most of state-of-the-art deep hashing methods and traditional hashing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. \(\frac {\partial \hat {{R}}({I^{q},I^{s}})}{\partial f_{h}(I^{q})}\), \(\frac {\partial \hat {{R}}({I^{q},I^{s}})}{\partial f_{h}(I^{s})}\), and \(\frac {\partial \hat {{R}}({I^{q},I^{s}})}{\partial f_{h}({I_{k}^{d}})}\) are easy to be computed. Due to space limitations, we do not give the specific expressions of above terms here.

References

  1. Cao Y, Xu J, Liu T Y, Li H, Huang Y, Hon HW (2006) Adapting ranking svm to document retrieval. In: ACM SIGIR

  2. Cao Y, Long M, Wang J, Zhu H, Wen Q (2016) Deep quantization network for efficient image retrieval. In: AAAI

  3. Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014) Return of the devil in the details: Delving deep into convolutional nets. arXiv:http://arXiv.org/abs/14053531

  4. Chua T S, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) Nus-wide: a real-world web image database from national university of singapore. In: ACM CIVR

  5. Erin Liong V, Lu J, Wang G, Moulin P, Zhou J (2015) Deep hashing for compact binary codes learning. In: IEEE CVPR

  6. Gionis A, Indyk P, Motwani R et al (1999) Similarity search in high dimensions via hashing. In: VLDB

  7. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE CVPR

  8. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: AISTATS

  9. Gong Y, Lazebnik S (2011) Iterative quantization: a procrustean approach to learning binary codes. In: IEEE CVPR

  10. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. arXiv preprint arXiv:http://arXiv.org/abs/160305027

  11. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: ACM Multimedia

  12. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Tech Report

  13. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: NIPS

  14. Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: IEEE ICCV

  15. Kulis B, Jain P, Grauman K (2009) Fast similarity search for learned metrics. IEEE Trans Pattern Anal Mach Intell 31(12):2143–2157

    Article  Google Scholar 

  16. Lai H, Pan Y, Liu Y, Yan S (2015) Simultaneous feature learning and hash coding with deep neural networks. In: IEEE CVPR

  17. LeCun Y, Cortes C, Burges CJ (1998) The mnist database of handwritten digits

  18. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521 (7553):436–444

    Article  Google Scholar 

  19. Li W J, Wang S, Kang WC (2016) Feature learning based deep supervised hashing with pairwise labels. In: IJCAI

  20. Lin G, Shen C, Shi Q, van den Hengel A, Suter D (2014) Fast supervised hashing with decision trees for high-dimensional data. In: IEEE CVPR

  21. Liu W, Wang J, Kumar S, Chang SF (2011) Hashing with graphs. In: ICML

  22. Liu W, Wang J, Ji R, Jiang Y G, Chang SF (2012) Supervised hashing with kernels. In: IEEE CVPR

  23. Liu W, Zha Z, Wang Y, Lu K, Tao D (2016) p-laplacian regularized sparse coding for human activity recognition. IEEE Trans Indus Electron 63(8):5120–5129

    Google Scholar 

  24. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE CVPR

  25. Lowe D G (1999) Object recognition from local scale-invariant features. In:IEEE ICCV

  26. Norouzi M, Fleet DJ, Salakhutdinov RR (2012) Hamming distance metric learning. In: NIPS, pp 1061–1069

  27. Norouzi M, Fleet D J, Salakhutdinov RR (2012) Hamming distance metric learning. In: NIPS

  28. Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. Int J Comput Vis 42(3):145–175

    Article  MATH  Google Scholar 

  29. Raginsky M, Lazebnik S (2009) Locality-sensitive binary codes from shift-invariant kernels. In: NIPS

  30. Shen F, Shen C, Liu W, Tao Shen H (2015) Supervised discrete hashing. In: IEEE CVPR

  31. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:http://arXiv.org/abs/14091556

  32. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE CVPR

  33. Taigman Y, Yang M, Ranzato M, Wolf L (2014) Deepface: closing the gap to human-level performance in face verification. In: IEEE CVPR

  34. Tao D, Tang X, Li X, Wu X (2006) Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval. IEEE Trans Pattern Anal Mach Intell 28(7):1088–1099

    Article  Google Scholar 

  35. Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: NIPS

  36. Xia R, Pan Y, Lai H, Liu C, Yan S (2014) Supervised hashing for image retrieval via image representation learning. In: AAAI

  37. Xie L, Shen J, Zhu L (2016) Online cross-modal hashing for web image retrieval. In: AAAI

  38. Xie L, Shen J, Han J, Zhu L, Shao L (2017) Dynamic multi-view hashing for online image retrieval. In: IJCAI

  39. Yang HF, Lin K, Chen CS (2015) Supervised learning of semantics-preserving hashing via deep neural networks for large-scale image search. arXiv:http://arXiv.org/abs/150700101

  40. Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Syst Man Cybern 1–11

  41. Zhang L, Zhang Y, Tang J, Gu X, Li J, Tian Q (2013) Topology preserving hashing for similarity search. In: ACM Multimedia

  42. Zhang R, Lin L, Zhang R, Zuo W, Zhang L (2015) Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans Image Process 24(12):4766–4779

    Article  MathSciNet  Google Scholar 

  43. Zhao F, Huang Y, Wang L, Tan T (2015) Deep semantic ranking based hashing for multi-label image retrieval. In: IEEE CVPR

  44. Zheng L, Wang S, Liu Z, Tian Q (2013) Lp-norm idf for large scale image search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1626–1633

  45. Zheng L, Wang S, Liu Z, Tian Q (2014) Packing and padding: coupled multi-index for accurate image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1939–1946

  46. Zheng L, Wang S, Tian Q (2014) Coupled binary embedding for large-scale image retrieval. IEEE Trans Image Process 23(8):3368–3380

    Article  MathSciNet  MATH  Google Scholar 

  47. Zheng L, Wang S, Zhou W, Tian Q (2014) Bayes merging of multiple vocabularies for scalable image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1955–1962

  48. Zheng L, Wang S, Tian L, He F, Liu Z, Tian Q (2015) Query-adaptive late fusion for image search and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1741–1750

  49. Zheng L, Wang S, Wang J, Tian Q (2016) Accurate image search with multi-scale contextual evidences. Int J Comput Vis 120(1):1–13

    Article  MathSciNet  Google Scholar 

  50. Zheng L, Yang Y, Tian Q (2017) Sift meets cnn: a decade survey of instance retrieval. IEEE Trans Pattern Anal Mach Intell

  51. Zhu H, Long M, Wang J, Cao Y (2016) Deep hashing network for efficient similarity retrieval. In: AAAI

  52. Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised visual hashing with semantic assistant for content-based image retrieval. IEEE Trans Knowl Data Eng 29 (2):472–486

    Article  Google Scholar 

  53. Zhu X, Zhang L, Huang Z (2014) A sparse embedding and least variance encoding approach to hashing. IEEE Trans Image Process 23(9):3737–3750

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Foundation for Innovative Research Groups of the National Natural Science Foundation of China (NSFC) under Grant 71421001, in part by the National Natural Science Foundation of China (NSFC) under Grant 61502073 and Grant 61429201, in part by the Open Projects Program of National Laboratory of Pattern Recognition under Grant 201407349, and in part to Dr. Qi Tian by ARO grants W911NF-15-1-0290 and Faculty Research Gift Awards by NEC Laboratories of America and Blippar.

Author information

Authors and Affiliations

  1. School of Information and Communication Engineering, Dalian University of Technology, Dalian, 116024, China

    Qiang Li, Haiyan Fu & Xiangwei Kong

  2. Department of Computer Science, University of Texas at San Antonio, San Antonio, TX, 78249-1604, USA

    Qi Tian

Authors
  1. Qiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiyan Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangwei Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qi Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangwei Kong.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Q., Fu, H., Kong, X. et al. Deep hashing with top similarity preserving for image retrieval. Multimed Tools Appl 77, 24121–24141 (2018). https://doi.org/10.1007/s11042-017-5596-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5596-7

Keywords

Search

Navigation