Skip to main content
SpringerLink
Log in

CapsNet-based supervised hashing

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the development of Internet technology, an increasing amount of data enters people’s daily life, which brings great challenges when users quickly search for interesting images. The existing exact nearest neighbor retrieval methods often fail to obtain results within an acceptable retrieval time, so researchers have begun to focus on approximate nearest neighbor retrieval. Recently, the hashing-based approximate nearest neighbor retrieval method has attracted increasing attention because of its small storage space and high retrieval efficiency. At present, one of the most advanced hashing methods is to use deep neural networks, especially convolutional neural networks (CNN), to obtain image hash codes to achieve fast image retrieval. However, CNN needs a large number of images during training, so it takes a lot of time to obtain training samples. In addition, CNN cannot handle ambiguity well, and a lot of information is lost in the pooling layer; furthermore, CNN cannot learn the hierarchical structure of the image. Aiming to address these problems while making full use of data classification information to guide hash learning in a supervised form to improve retrieval efficiency, we introduce the capsule network into the hash learning and propose CapsNet-based supervised hashing (CSH) to preserve the effective information of an image as much as possible. CSH adds a hashing layer equivalent to hash mapping between the capsule network and the decoder to perform hash learning. By optimizing the objective function defined for capsule network loss, reconstruction loss and hashing quantization loss, feature representations, hashing functions and classification results can be learned from the input data at the same time. To verify the effectiveness of the method, we performed experiments on multiple datasets. The experimental results show that this method is superior to the existing hashing-based image retrieval methods and achieves satisfactory results in image classification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014)

  2. Cheng S, Lai H, Wang L, Qin J (2019) A novel deep hashing method for fast image retrieval. Vis Comput 35(9):1255–1266

    Article  Google Scholar 

  3. Csurka G, Dance C, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision. Prague, vol 1. ECCV, pp 1–2

  4. Deng F, Pu S, Chen X, Shi Y, Yuan T, Pu S (2018) Hyperspectral image classification with capsule network using limited training samples. Sensors 18(9):3153

    Article  Google Scholar 

  5. Gionis A, Indyk P, Motwani R, et al. (1999) Similarity search in high dimensions via hashing. In: Vldb, vol 99, pp 518–529

  6. Gong Y, Lazebnik S, Gordo A, Perronnin F (2012) Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35 (12):2916–2929

    Article  Google Scholar 

  7. Gui J, Liu T, Sun Z, Tao D, Tan T (2018) Fast supervised discrete hashing. IEEE Trans Pattern Anal Mach Intell 40(2):490–496

    Article  Google Scholar 

  8. Jain H, Zepeda J, Pérez P., Gribonval R (2017) Subic: A supervised, structured binary code for image search. In: Proceedings of the IEEE international conference on computer vision, pp 833–842

  9. Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 3304–3311

  10. Ji J, Li J, Yan S, Zhang B, Tian Q (2012) Super-bit locality-sensitive hashing. In: Advances in neural information processing systems, pp 108–116

  11. Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images

  12. Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: 2009 IEEE 12th international conference on computer vision, IEEE, pp 2130–2137

  13. Lai H, Pan Y, Liu Y, Yan S (2015) Simultaneous feature learning and hash coding with deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3270–3278

  14. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  15. Li Q, Sun Z, He R, Tan T (2017) Deep supervised discrete hashing. In: Advances in neural information processing systems, pp 2482–2491

  16. Li WJ, Wang S, Kang WC (2015) Feature learning based deep supervised hashing with pairwise labels. arXiv:1511.03855

  17. Lin K, Yang HF, Hsiao JH, Chen CS (2015) Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 27–35

  18. Liu H, Wang R, Shan S, Chen X (2016) Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2064–2072

  19. Liu W, Wang J, Kumar S (2011) Chang. S.F., Hashing with graphs

    Google Scholar 

  20. Lu X, Song L, Xie R, Yang X, Zhang W (2017) Deep binary representation for efficient image retrieval. Adv Multimed 2017

  21. Norouzi M (2011) Fleet, D.J., Minimal loss hashing for compact binary codes

  22. Perronnin F, Liu Y, Sánchez J, Poirier H (2010) Large-scale image retrieval with compressed fisher vectors. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, IEEE, pp 3384–3391

  23. Rueckauer B, Lungu IA, Hu Y, Pfeiffer M, Liu SC (2017) Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front Neurosci 11: 682

    Article  Google Scholar 

  24. Sablayrolles A, Douze M, Usunier N, Jégou H (2017) How should we evaluate supervised hashing?. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp 1732–1736

  25. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Advances in neural information processing systems, pp 3856–3866

  26. Shen F, Shen C, Liu W, Tao Shen H (2015) Supervised discrete hashing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 37–45

  27. Shi C, Wang Y, Jia F, He K, Wang C, Xiao B (2017) Fisher vector for scene character recognition: A comprehensive evaluation. Pattern Recogn 72:1–14

    Article  Google Scholar 

  28. Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: A comprehensive study. In: Proceedings of the 22nd ACM international conference on multimedia, pp 157–166

  29. Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: Advances in neural information processing systems, pp 1753–1760

  30. Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemometr Intell Lab Syst 2(1-3):37–52

    Article  Google Scholar 

  31. Xia R, Pan Y, Lai H, Liu C, Yan S (2014) Supervised hashing for image retrieval via image representation learning. In: Twenty-eighth AAAI conference on artificial intelligence

  32. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist:, a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747

  33. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  34. Yao Y, Shi Y, Weng S, Guan B (2018) Deep learning for detection of object-based forgery in advanced video. Symmetry 10(1):3

    Article  Google Scholar 

  35. Ying W, Sang J, Yu J (2020) Locality-constrained discrete graph hashing. Neurocomputing 398:566–573

    Article  Google Scholar 

  36. Zhang R, Lin L, Zhang R, Zuo W, Zhang L (2015) Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans Image Process 24(12):4766–4779

    Article  MathSciNet  Google Scholar 

  37. Zhao F, Huang Y, Wang L, Tan T (2015) Deep semantic ranking based hashing for multi-label image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1556–1564

  38. Zheng Z, Zheng L, Yang Y (2017) A discriminatively learned cnn embedding for person reidentification. ACM Trans Multimedia Comput Commun Appl (TOMM) 14(1):1–20

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by Zhejiang NSF Grant No. LZ20F020001 and No. LY20F020009, China NSF Grants No. 61472194, No. 61572266, Ningbo NSF Grant No. 2019A610085 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. (Corresponding author: Jiangbo Qian.)

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, China

    Bolin Zhang, Jiangbo Qian, Xijiong Xie, Yu Xin & Yihong Dong

Authors
  1. Bolin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiangbo Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xijiong Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Xin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yihong Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiangbo Qian.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, B., Qian, J., Xie, X. et al. CapsNet-based supervised hashing. Appl Intell 51, 5912–5926 (2021). https://doi.org/10.1007/s10489-020-02180-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-02180-7

Keywords

Search

Navigation