Abstract
Perceiving multiple objects within an image without the labels’ supervision is the challenge of multi-label image hashing tasks. Existing unsupervised hashing approaches do reconstruction or contrastive learning for the representation of the object of interest but ignore the other objects in the image. We propose to use pseudo labels to provide candidate objects, making the image match the possible objects’ features by the co-occurrence correlations between labels. As a result, we explore the co-occurrence correlations based on empirical models and design a data augmentation strategy in a self-supervised learning framework to learn label-level embeddings. We also build the image visual correlations and design a dual overlapping group sum-pooling (OGSP) component to fuse label-level and visual-level embeddings into image representations, alleviating noise from empirical models. Extensive experiments on public multi-label image datasets using pseudo labels demonstrate that our self-supervised label-visual correlation hashing framework outperforms state-of-the-art label-free hashing algorithms for retrieval. GitHub address: https://github.com/lzHZWZ/SS-LVH.git.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cao, Y., Long, M., Liu, B., Wang, J.: Deep cauchy hashing for hamming space retrieval. In: CVPR, pp. 1229–1237 (2018)
Cao, Y., Long, M., Wang, J., Liu, S.: Deep visual-semantic quantization for efficient image retrieval. In: CVPR, pp. 916–925. IEEE Computer Society (2017)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. In: ICML, vol. 119, pp. 1597–1607. PMLR (2020)
Chen, Z., Wei, X., Wang, P., Guo, Y.: Multi-label image recognition with graph convolutional networks. In: CVPR, pp. 5177–5186 (2019)
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) NAACL-HLT, pp. 4171–4186. Association for Computational Linguistics (2019)
Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Gattupalli, V., Zhuo, Y., Li, B.: Weakly supervised deep image hashing through tag embeddings. In: CVPR, pp. 10375–10384 (2019)
Grill, J., et al.: Bootstrap your own latent - a new approach to self-supervised learning (2020)
Huang, C., Yang, S., Pan, Y., Lai, H.: Object-location-aware hashing for multi-label image retrieval via automatic mask learning. IEEE Trans. Image Process. 27(9), 4490–4502 (2018)
Huiskes, M.J., Lew, M.S.: The MIR flickr retrieval evaluation. In: SIGMM, pp. 39–43 (2008)
Jin, L., Li, Z., Pan, Y., Tang, J.: Weakly-supervised image hashing through masked visual-semantic graph-based reasoning. In: MM, pp. 916–924 (2020)
Lai, H., Yan, P., Shu, X., Wei, Y., Yan, S.: Instance-aware hashing for multi-label image retrieval. IEEE Trans. Image Process. 25(6), 2469–2479 (2016)
Lanchantin, J., Wang, T., Ordonez, V., Qi, Y.: General multi-label image classification with transformers, pp. 16478–16488 (2021)
Li, Y., van Gemert, J.: Deep unsupervised image hashing by maximizing bit entropy. In: EAAI, pp. 2002–2010 (2021)
Lin, T.-S., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Liu, Y., et al.: Deep self-taught hashing for image retrieval. IEEE Trans. Cybern. 49(6), 2229–2241 (2019)
Luo, X., et al.: A statistical approach to mining semantic similarity for deep unsupervised hashing. In: MM, pp. 4306–4314. ACM (2021)
Qiu, Z., Su, Q., Ou, Z., Yu, J., Chen, C.: Unsupervised hashing with contrastive information bottleneck. In: IJCAI, pp. 959–965. ijcai.org (2021)
Shen, Y., et al.: Auto-encoding twin-bottleneck hashing. In: CVPR, pp. 2815–2824 (2020)
Song, J., He, T., Gao, L., Xu, X., Shen, H.T.: Deep region hashing for efficient large-scale instance search from images. CoRR abs/1701.07901 (2017)
Wang, D., Cui, P., Zhu, W.: Structural deep network embedding. In: SIGKDD, pp. 1225–1234 (2016)
Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., Xu, W.: CNN-RNN: a unified framework for multi-label image classification. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, 27–30 June 2016, pp. 2285–2294. IEEE Computer Society (2016)
Wang, Y., Song, J., Zhou, K., Liu, Y.: Unsupervised deep hashing with node representation for image retrieval. Pattern Recogn. 112, 107785 (2021)
Wu, Z., Shen, C., van den Hengel, A.: Wider or deeper: revisiting the resnet model for visual recognition. Pattern Recogn. 90, 119–133 (2019)
Xie, Y., Liu, Y., Wang, Y., Gao, L., Wang, P., Zhou, K.: Label-attended hashing for multi-label image retrieval, pp. 955–962 (2020)
Yang, E., Liu, T., Deng, C., Liu, W., Tao, D.: DistillHash: unsupervised deep hashing by distilling data pairs. In: CVPR, pp. 2946–2955 (2019)
Zhang, W., Wu, D., Zhou, Y., Li, B., Wang, W., Meng, D.: Deep unsupervised hybrid-similarity hadamard hashing. In: MM, pp. 3274–3282 (2020)
Acknowledgments
This work is supported by the National Natural Science Foundation of China No. 61902135 and No. 62172180, and the Joint Founds of ShanDong Natural Science Funds (Grant No. ZR2019LZH003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y., Xie, Y., Song, J., Wei, R., Zhou, K. (2023). Self-supervised Label-Visual Correlation Hashing for Multi-label Image Retrieval. In: Li, B., Yue, L., Tao, C., Han, X., Calvanese, D., Amagasa, T. (eds) Web and Big Data. APWeb-WAIM 2022. Lecture Notes in Computer Science, vol 13422. Springer, Cham. https://doi.org/10.1007/978-3-031-25198-6_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-25198-6_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25197-9
Online ISBN: 978-3-031-25198-6
eBook Packages: Computer ScienceComputer Science (R0)