Abstract
With the development of Internet technology, an increasing amount of data enters people’s daily life, which brings great challenges when users quickly search for interesting images. The existing exact nearest neighbor retrieval methods often fail to obtain results within an acceptable retrieval time, so researchers have begun to focus on approximate nearest neighbor retrieval. Recently, the hashing-based approximate nearest neighbor retrieval method has attracted increasing attention because of its small storage space and high retrieval efficiency. At present, one of the most advanced hashing methods is to use deep neural networks, especially convolutional neural networks (CNN), to obtain image hash codes to achieve fast image retrieval. However, CNN needs a large number of images during training, so it takes a lot of time to obtain training samples. In addition, CNN cannot handle ambiguity well, and a lot of information is lost in the pooling layer; furthermore, CNN cannot learn the hierarchical structure of the image. Aiming to address these problems while making full use of data classification information to guide hash learning in a supervised form to improve retrieval efficiency, we introduce the capsule network into the hash learning and propose CapsNet-based supervised hashing (CSH) to preserve the effective information of an image as much as possible. CSH adds a hashing layer equivalent to hash mapping between the capsule network and the decoder to perform hash learning. By optimizing the objective function defined for capsule network loss, reconstruction loss and hashing quantization loss, feature representations, hashing functions and classification results can be learned from the input data at the same time. To verify the effectiveness of the method, we performed experiments on multiple datasets. The experimental results show that this method is superior to the existing hashing-based image retrieval methods and achieves satisfactory results in image classification performance.
Similar content being viewed by others
References
Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014)
Cheng S, Lai H, Wang L, Qin J (2019) A novel deep hashing method for fast image retrieval. Vis Comput 35(9):1255–1266
Csurka G, Dance C, Fan L, Willamowski J, Bray C (2004) Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision. Prague, vol 1. ECCV, pp 1–2
Deng F, Pu S, Chen X, Shi Y, Yuan T, Pu S (2018) Hyperspectral image classification with capsule network using limited training samples. Sensors 18(9):3153
Gionis A, Indyk P, Motwani R, et al. (1999) Similarity search in high dimensions via hashing. In: Vldb, vol 99, pp 518–529
Gong Y, Lazebnik S, Gordo A, Perronnin F (2012) Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans Pattern Anal Mach Intell 35 (12):2916–2929
Gui J, Liu T, Sun Z, Tao D, Tan T (2018) Fast supervised discrete hashing. IEEE Trans Pattern Anal Mach Intell 40(2):490–496
Jain H, Zepeda J, Pérez P., Gribonval R (2017) Subic: A supervised, structured binary code for image search. In: Proceedings of the IEEE international conference on computer vision, pp 833–842
Jégou H, Douze M, Schmid C, Pérez P (2010) Aggregating local descriptors into a compact image representation. In: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, pp 3304–3311
Ji J, Li J, Yan S, Zhang B, Tian Q (2012) Super-bit locality-sensitive hashing. In: Advances in neural information processing systems, pp 108–116
Krizhevsky A, Hinton G et al (2009) Learning multiple layers of features from tiny images
Kulis B, Grauman K (2009) Kernelized locality-sensitive hashing for scalable image search. In: 2009 IEEE 12th international conference on computer vision, IEEE, pp 2130–2137
Lai H, Pan Y, Liu Y, Yan S (2015) Simultaneous feature learning and hash coding with deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3270–3278
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Li Q, Sun Z, He R, Tan T (2017) Deep supervised discrete hashing. In: Advances in neural information processing systems, pp 2482–2491
Li WJ, Wang S, Kang WC (2015) Feature learning based deep supervised hashing with pairwise labels. arXiv:1511.03855
Lin K, Yang HF, Hsiao JH, Chen CS (2015) Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 27–35
Liu H, Wang R, Shan S, Chen X (2016) Deep supervised hashing for fast image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2064–2072
Liu W, Wang J, Kumar S (2011) Chang. S.F., Hashing with graphs
Lu X, Song L, Xie R, Yang X, Zhang W (2017) Deep binary representation for efficient image retrieval. Adv Multimed 2017
Norouzi M (2011) Fleet, D.J., Minimal loss hashing for compact binary codes
Perronnin F, Liu Y, Sánchez J, Poirier H (2010) Large-scale image retrieval with compressed fisher vectors. In: 2010 IEEE Computer society conference on computer vision and pattern recognition, IEEE, pp 3384–3391
Rueckauer B, Lungu IA, Hu Y, Pfeiffer M, Liu SC (2017) Conversion of continuous-valued deep networks to efficient event-driven networks for image classification. Front Neurosci 11: 682
Sablayrolles A, Douze M, Usunier N, Jégou H (2017) How should we evaluate supervised hashing?. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp 1732–1736
Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. In: Advances in neural information processing systems, pp 3856–3866
Shen F, Shen C, Liu W, Tao Shen H (2015) Supervised discrete hashing. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 37–45
Shi C, Wang Y, Jia F, He K, Wang C, Xiao B (2017) Fisher vector for scene character recognition: A comprehensive evaluation. Pattern Recogn 72:1–14
Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: A comprehensive study. In: Proceedings of the 22nd ACM international conference on multimedia, pp 157–166
Weiss Y, Torralba A, Fergus R (2009) Spectral hashing. In: Advances in neural information processing systems, pp 1753–1760
Wold S, Esbensen K, Geladi P (1987) Principal component analysis. Chemometr Intell Lab Syst 2(1-3):37–52
Xia R, Pan Y, Lai H, Liu C, Yan S (2014) Supervised hashing for image retrieval via image representation learning. In: Twenty-eighth AAAI conference on artificial intelligence
Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist:, a novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747
Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403
Yao Y, Shi Y, Weng S, Guan B (2018) Deep learning for detection of object-based forgery in advanced video. Symmetry 10(1):3
Ying W, Sang J, Yu J (2020) Locality-constrained discrete graph hashing. Neurocomputing 398:566–573
Zhang R, Lin L, Zhang R, Zuo W, Zhang L (2015) Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans Image Process 24(12):4766–4779
Zhao F, Huang Y, Wang L, Tan T (2015) Deep semantic ranking based hashing for multi-label image retrieval. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1556–1564
Zheng Z, Zheng L, Yang Y (2017) A discriminatively learned cnn embedding for person reidentification. ACM Trans Multimedia Comput Commun Appl (TOMM) 14(1):1–20
Acknowledgements
This work was supported in part by Zhejiang NSF Grant No. LZ20F020001 and No. LY20F020009, China NSF Grants No. 61472194, No. 61572266, Ningbo NSF Grant No. 2019A610085 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. (Corresponding author: Jiangbo Qian.)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, B., Qian, J., Xie, X. et al. CapsNet-based supervised hashing. Appl Intell 51, 5912–5926 (2021). https://doi.org/10.1007/s10489-020-02180-7
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-02180-7