Skip to main content
SpringerLink
Log in

Anchor-Based Self-Ensembling for Semi-Supervised Deep Pairwise Hashing

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Deep hashing has attracted considerable attention to tackle large-scale retrieval tasks, because of automatic and powerful feature extraction of convolutional neural networks and the gain of hashing in computation and storage costs. Most current supervised deep hashing methods only utilize the semantic information of labeled data without exploiting unlabeled data. However, data annotation is expensive and thus only scarce labeled data are available, which are difficult to represent the true distribution of all data. In this paper, we propose a novel semi-supervised deep pairwise hashing method to leverage both labeled and unlabeled data to learn hash functions. Our method utilizes the transduction of anchors to preserve the pairwise similarity relationship among both labeled and unlabeled samples. Additionally, to explore the semantic similarity information hidden in unlabeled data, it adopts self-ensembling to create strong ensemble targets for latent binary vectors of training samples and form a consensus predicting similarity relationship to multiple anchors. Unlike previous pairwise based hashing methods without maintaining the relevance among similar neighbors, we further explain and exhibit the capability of our method on preserving their relevance through calculating their similarities to anchors. Finally, extensive experiments on benchmark databases demonstrate the superior performance of the proposed method over recent state-of-the-art hashing methods on multiple retrieval tasks. The source codes of the proposed method are available on: https://github.com/xsshi2015/Semi-supervised-Deep-Pairwise-Hashing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Broder, A. Z., Charikar, M., Frieze, A. M., & Mitzenmacher, M. (2000). Min-wise independent permutations. Journal of Computer and System Sciences, 60(3), 630–659.

    MathSciNet  MATH  Google Scholar 

  • Cao, Y., Long, M., Liu, B., & Wang, J. (2018). Deep cauchy hashing for hamming space retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition(CVPR), (pp. 229–1237).

  • Cao, Z., Long, M., Wang, J., & Yu, P.S. (2017). Hashnet: Deep learning to hash by continuation. In Proceedings of the IEEE international conference on computer vision (ICCV).

  • Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., & Zheng, Y. (2009). Nus-wide: A real-world web image database from national university of singapore. In Proceedings of the ACM international conference on image and video retrieval (ACM-CIVR), ACM, (p. 48).

  • Datar, M., Immorlica, N., Indyk, P., & Mirrokni, V.S. (2004). Locality-sensitive hashing scheme based on p-stable distributions. In Proceedings of the twentieth annual symposium on computational geometry (SoCG), ACM, (pp. 253–262).

  • Datta, R., Joshi, D., Li, J., & Wang, J. Z. (2008). Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys, 40(2), 5.

    Google Scholar 

  • Delalleau, O., Bengio, Y., & Le Roux, N. (2005). Efficient non-parametric function induction in semi-supervised learning. In AISTATS, vol 27, (p. 100).

  • Gong, Y., Lazebnik, S., Gordo, A., & Perronnin, F. (2013). Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(12), 2916–2929.

    Google Scholar 

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp 770–778).

  • Hu, H., Wang, K., Lv, C., Wu, J., & Yang, Z. (2019). Semi-supervised metric learning-based anchor graph hashing for large-scale image retrieval. IEEE Transactions on Image Processing, 28(2), 739–754.

    MathSciNet  MATH  Google Scholar 

  • Indyk, P., & Motwani, R. (1998). Approximate nearest neighbors: Towards removing the curse of dimensionality. In Proceedings of the thirtieth annual ACM symposium on theory of computing (STOC), ACM, (pp 604–613).

  • Järvelin, K., & Kekäläinen, J. (2000). Ir evaluation methods for retrieving highly relevant documents. In Proceedings of international ACM SIGIR conference on research and development in information retrieval (SIGIR), ACM, (pp 41–48).

  • Jiang, Q.Y., & Li, W.J. (2018). Asymmetric deep supervised hashing. In Proceedings of the AAAI conference on artificial intelligence (AAAI).

  • Kang, W.C., Li, W.J., & Zhou, Z.H. (2016). Column sampling based discrete supervised hashing. In Proceedings of the AAAI conference on artificial intelligence (AAAI), (pp 1230–1236).

  • Kingma, D.P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

  • Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images. Technical report, Citeseer.

  • Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. In Proceedings of the advances in neural information processing systems (NIPS), (pp. 1097–1105).

  • Kulis, B., & Darrell, T. (2009). Learning to hash with binary reconstructive embeddings. In Proceedings of the advances in neural information processing systems (NIPS), (pp. 1042–1050).

  • Kulis, B., Jain, P., & Grauman, K. (2009). Fast similarity search for learned metrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(12), 2143–2157.

    Google Scholar 

  • Lai, H., Pan, Y., Liu, Y., & Yan, S. (2015). Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp. 3270–3278).

  • Laine, S., & Aila, T. (2016). Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242.

  • LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Google Scholar 

  • Li, W.J., Wang, S., & Kang, W.C. (2015). Feature learning based deep supervised hashing with pairwise labels. arXiv preprint arXiv:1511.03855.

  • Li, X., Lin, G., Shen, C., Hengel, Avd, & Dick, A. (2013). Learning hash functions using column generation. arXiv preprint arXiv:1303.0339.

  • Lin, K., Yang, H.F., Hsiao, J.H., & Chen, C.S. (2015). Deep learning of binary hash codes for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (CVPR workshops), (pp. 27–35).

  • Lin TY, Maire M, Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C.L. (2014). Microsoft coco: Common objects in context. In Proceedings of the European conference on computer vision (ECCV), Springer, (pp. 740–755).

  • Liu, H., Wang, R., Shan, S., & Chen, X. (2016). Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp. 2064–2072).

  • Liu, L., Shao, L., Shen, F., & Yu, M. (2017). Discretely coding semantic rank orders for supervised image hashing. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp. 1425–1434).

  • Liu, W., He, J., & Chang, S.F. (2010). Large graph construction for scalable semi-supervised learning. In Proceedings of the international conference on machine learning (ICML), (pp. 679–686).

  • Liu, W., Wang, J., Ji, R., Jiang, Y.G., & Chang, S.F. (2012). Supervised hashing with kernels. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), IEEE, (pp. 2074–2081).

  • Liu, W., Mu, C., Kumar, S., & Chang, S.F. (2014). Discrete graph hashing. In Proceedings of the advances in neural information processing systems (NIPS), (pp. 3419–3427).

  • Maaten, Lvd, & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9, 2579–2605.

    MATH  Google Scholar 

  • Miyato, T., Maeda, Si, Ishii, S., & Koyama, M. (2018). Virtual adversarial training: A regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8), 1979–1993.

    Google Scholar 

  • Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42(3), 145–175.

    MATH  Google Scholar 

  • Qiu, Z., Pan, Y., Yao, T., & Mei, T. (2017). Deep semantic hashing with generative adversarial networks. In Proceedings of the international ACM SIGIR conference on research and development in information retrieval (SIGIR), ACM, (pp. 225–234).

  • Raginsky, M., & Lazebnik, S. (2009). Locality-sensitive binary codes from shift-invariant kernels. In Proceedings of the advances in neural information processing systems (NIPS), (pp. 1509–1517).

  • Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., et al. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211–252.

    MathSciNet  Google Scholar 

  • Sablayrolles, A., Douze, M., Usunier, N., & Jégou, H. (2017). How should we evaluate supervised hashing? In Proceedings of the IEEE conference on acoustics, speech and signal processing (ICASSP), IEEE, (pp. 1732–1736).

  • Sapkota, M., Shi, X., Xing, F., & Yang, L. (2018). Deep convolutional hashing for low dimensional binary embedding of histopathological images. IEEE Journal of Biomedical and Health Informatics, 23(2), 805–816.

  • Shen, F., Liu, W., Zhang, S., Yang, Y., & Tao, Shen H. (2015a) Learning binary codes for maximum inner product search. In Proceedings of the IEEE international conference on computer vision (ICCV), (pp. 4148–4156).

  • Shen, F., Shen, C., Liu, W., & Tao, Shen H. (2015b). Supervised discrete hashing. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp. 37–45).

  • Shen, F., Gao, X., Liu, L., Yang, Y., & Shen, H.T. (2017). Deep asymmetric pairwise hashing. In Proceedings of the ACM on multimedia conference (ACMMM), ACM, (pp. 1522–1530).

  • Shi, X., Xing, F., Cai, J., Zhang, Z., Xie, Y., & Yang, L. (2016). Kernel-based supervised discrete hashing for image retrieval. In Proceedings of the European conference on computer vision (ECCV), Springer, (pp. 419–433).

  • Shi, X., Xing, F., Xu, K., Sapkota, M., & Yang, L. (2017a). Asymmetric discrete graph hashing. In Proceedings of the AAAI conference on artificial intelligence (AAAI), (pp. 2541–2547).

  • Shi, X., Xing, F., Xu, K., Xie, Y., Su, H., & Yang, L. (2017b). Supervised graph hashing for histopathology image retrieval and classification. Medical image analysis, 42, 117–128.

    Google Scholar 

  • Shi, X., Sapkota, M., Xing, F., Liu, F., Cui, L., & Yang, L. (2018). Pairwise based deep ranking hashing for histopathology image classification and retrieval. Pattern Recognition, 81, 14–22.

    Google Scholar 

  • Song, J., Zhang, H., Li, X., Gao, L., Wang, M., & Hong, R. (2018). Self-supervised video hashing with hierarchical binary auto-encoder. IEEE Transactions on Image Processing, 27(7), 3210–3221.

    MathSciNet  MATH  Google Scholar 

  • Tarvainen, A., & Valpola, H. (2017). Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the advances in neural information processing systems (NIPS), (pp. 1195–1204).

  • Torralba, A., Fergus, R., & Freeman, W. T. (2008). 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11), 1958–1970.

    Google Scholar 

  • Wang, G., Hu, Q., Cheng, J., & Hou, Z. (2018). Semi-supervised generative adversarial hashing for image retrieval. In Proceedings of the European conference on computer vision (ECCV), (pp. 469–485).

  • Wang, J., Kumar, S., & Chang, S. F. (2012). Semi-supervised hashing for large-scale search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 2393–2406.

    Google Scholar 

  • Wang, J., Zhang, T., Sebe, N., Shen, H. T., et al. (2017). A survey on learning to hash. IEEE transactions on pattern analysis and machine intelligence, 40(4), 769–790.

    Google Scholar 

  • Weiss, Y., Torralba, A., & Fergus, R. (2009). Spectral hashing. In Proceedings of the advances in neural information processing systems (NIPS), (pp. 1753–1760).

  • Xia, R., Pan, Y., Lai, H., Liu, C., & Yan, S. (2014). Supervised hashing for image retrieval via image representation learning. In Proceedings of the AAAI conference on artificial intelligence (AAAI), vol 1, (p 2).

  • Xu, K., Liu, S., Zhao, P., Chen, P.Y., Zhang, H., Erdogmus, D., Yanzhi, W., & Xue, L. (2018). Structured adversarial attack: Towards general implementation and better interpretability. In Proceedings of international Conference on Learning Representations (ICLR).

  • Xu, K., Chen, H., Liu, S., Chen, P.Y., Weng, T.W., Hong, M., & Xue, L. (2019). Topology attack and defense for graph neural networks: An optimization perspective. In Proceedings of international joint conference on artificial intelligence (IJCAI).

  • Yan, X., Zhang, L., & Li, W.J. (2017). Semi-supervised deep hashing with a bipartite graph. In Proceedings of the international joint conference on artificial intelligence (IJCAI), AAAI Press, (pp. 3238–3244).

  • Zhang, J., & Peng, Y. (2017). Ssdh: semi-supervised deep hashing for large scale image retrieval. IEEE Transactions on Circuits and Systems for Video Technology, 29(1), 212–225.

    MathSciNet  Google Scholar 

  • Zhang, J., Peng, Y., & Yuan, M. (2018). Sch-gan: Semi-supervised cross-modal hashing by generative adversarial network. arXiv preprint arXiv:1802.02488.

  • Zhang, R., Lin, L., Zhang, R., Zuo, W., & Zhang, L. (2015). Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Transactions on Image Processing, 24(12), 4766–4779.

    MathSciNet  MATH  Google Scholar 

  • Zhao, F., Huang, Y., Wang, L., & Tan, T. (2015). Deep semantic ranking based hashing for multi-label image retrieval. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp. 1556–1564).

  • Zhu, H., Long, M., Wang, J., & Cao, Y. (2016). Deep hashing network for efficient similarity retrieval. In Proceedings of the AAAI conference on artificial intelligence (AAAI), (pp. 2415–2421).

  • Zhuang, B., Lin, G., Shen, C., & Reid, I. (2016). Fast training of triplet-based deep binary embedding networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), (pp. 5955–5964).

Download references

Acknowledgements

The work is partially supported by the Natural Science Foundation of China (NSFC) (No. 61772296), Shenzhen fundamental research fund (No. JCYJ20170412170438636).

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, University of Florida, Gainesville, USA

    Xiaoshuang Shi, Yun Liang & Lin Yang

  2. Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, People’s Republic of China

    Zhenhua Guo

  3. Department of Biostatistics and Informatics, University of Colorado Denver, Denver, USA

    Fuyong Xing

Authors
  1. Xiaoshuang Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenhua Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fuyong Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yun Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoshuang Shi.

Additional information

Communicated by Li Liu, Matti Pietikäinen, Jie Qin, Jie Chen, Wanli Ouyang, Luc Van Gool.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 81 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, X., Guo, Z., Xing, F. et al. Anchor-Based Self-Ensembling for Semi-Supervised Deep Pairwise Hashing. Int J Comput Vis 128, 2307–2324 (2020). https://doi.org/10.1007/s11263-020-01299-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-020-01299-x

Keywords

Search

Navigation