Skip to main content
SpringerLink
Log in

Deep Hashing with Active Pairwise Supervision

  • Conference paper
  • First Online:
Computer Vision – ECCV 2020 (ECCV 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12364))

Included in the following conference series:

Abstract

In this paper, we propose a Deep Hashing method with Active Pairwise Supervision (DH-APS). Conventional methods with passive pairwise supervision obtain labeled data for training and require large amount of annotations to reach their full potential, which are not feasible in realistic retrieval tasks. On the contrary, we actively select a small quantity of informative samples for annotation to provide effective pairwise supervision so that discriminative hash codes can be obtained with limited annotation budget. Specifically, we generalize the structural risk minimization principle and obtain three criteria for the pairwise supervision acquisition: uncertainty, representativeness and diversity. Accordingly, samples involved in the following training pairs should be labeled: pairs with most uncertain similarity, pairs that minimize the discrepancy between labeled and unlabeled data, and pairs which are most different from the annotated data, so that the discriminality and generalization ability of the learned hash codes are significantly strengthened. Moreover, our DH-APS can also be employed as a plug-and-play module for semi-supervised hashing methods to further enhance the performance. Experiments demonstrate that the presented DH-APS achieves the accuracy of supervised hashing methods with only \(30\%\) labeled training samples and improves the semi-supervised binary codes by a sizable margin.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Babenko, A., Lempitsky, V.: Aggregating local deep features for image retrieval. In: ICCV, pp. 1269–1277 (2015)

    Google Scholar 

  2. Balcan, M.-F., Broder, A., Zhang, T.: Margin based active learning. In: Bshouty, N.H., Gentile, C. (eds.) COLT 2007. LNCS (LNAI), vol. 4539, pp. 35–50. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-72927-3_5

    Chapter  MATH  Google Scholar 

  3. Bartlett, P.L., Mendelson, S.: Rademacher and Gaussian complexities: risk bounds and structural results. JMLR 3(Nov), 463–482 (2002)

    MathSciNet  MATH  Google Scholar 

  4. Beluch, W.H., Genewein, T., Nürnberger, A., Köhler, J.M.: The power of ensembles for active learning in image classification. In: CVPR, pp. 9368–9377 (2018)

    Google Scholar 

  5. Borgwardt, K.M., Gretton, A., Rasch, M.J., Kriegel, H.P., Schölkopf, B., Smola, A.J.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), 49–57 (2006)

    Article  Google Scholar 

  6. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J., et al.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends® Mach. Learn. 3(1), 1–122 (2011)

    MATH  Google Scholar 

  7. Chattopadhyay, R., Wang, Z., Fan, W., Davidson, I., Panchanathan, S., Ye, J.: Batch mode active sampling based on marginal probability distribution matching. TKDD 7(3), 13 (2013)

    Article  Google Scholar 

  8. Chua, T.S., Tang, J., Hong, R., Li, H., Luo, Z., Zheng, Y.: NUS-WIDE: a real-world web image database from national university of Singapore. In: Proceedings of the ACM International Conference on Image and Video Retrieval, p. 48 (2009)

    Google Scholar 

  9. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)

    Google Scholar 

  10. Duan, Y., Lu, J., Wang, Z., Feng, J., Zhou, J.: Learning deep binary descriptor with multi-quantization. In: CVPR, pp. 1183–1192 (2017)

    Google Scholar 

  11. Duan, Y., Wang, Z., Lu, J., Lin, X., Zhou, J.: GraphBit: bitwise interaction mining via deep reinforcement learning. In: CVPR, pp. 8270–8279 (2018)

    Google Scholar 

  12. Erin Liong, V., Lu, J., Wang, G., Moulin, P., Zhou, J.: Deep hashing for compact binary codes learning. In: CVPR, pp. 2475–2483 (2015)

    Google Scholar 

  13. Freytag, A., Rodner, E., Denzler, J.: Selecting influential examples: active learning with expected model output changes. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 562–577. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_37

    Chapter  Google Scholar 

  14. Gal, Y., Islam, R., Ghahramani, Z.: Deep Bayesian active learning with image data. In: ICML, pp. 1183–1192 (2017)

    Google Scholar 

  15. Ghasedi Dizaji, K., Zheng, F., Sadoughi, N., Yang, Y., Deng, C., Huang, H.: Unsupervised deep generative adversarial hashing network. In: CVPR, pp. 3664–3673 (2018)

    Google Scholar 

  16. Goodfellow, I., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)

    Google Scholar 

  17. Gordo, A., Almazán, J., Revaud, J., Larlus, D.: Deep image retrieval: learning global representations for image search. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 241–257. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_15

    Chapter  Google Scholar 

  18. Gretton, A., Borgwardt, K.M., Rasch, M.J., Schölkopf, B., Smola, A.: A kernel two-sample test. JMLR 13(Mar), 723–773 (2012)

    Google Scholar 

  19. Hasan, M., Roy-Chowdhury, A.K.: Context aware active learning of activity recognition models. In: ICCV, pp. 4543–4551 (2015)

    Google Scholar 

  20. Jegou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. TPAMI 33(1), 117–128 (2010)

    Article  Google Scholar 

  21. Johnson, J., et al.: Image retrieval using scene graphs. In: CVPR, pp. 3668–3678 (2015)

    Google Scholar 

  22. Joshi, A.J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: CVPR, pp. 2372–2379 (2009)

    Google Scholar 

  23. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Technical report (2009)

    Google Scholar 

  24. Lai, H., Pan, Y., Liu, Y., Yan, S.: Simultaneous feature learning and hash coding with deep neural networks. In: CVPR, pp. 3270–3278 (2015)

    Google Scholar 

  25. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  26. Li, W.J., Wang, S., Kang, W.C.: Feature learning based deep supervised hashing with pairwise labels. In: IJCAI, pp. 1711–1717 (2016)

    Google Scholar 

  27. Li, X., Guo, Y.: Multi-level adaptive active learning for scene classification. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 234–249. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10584-0_16

    Chapter  Google Scholar 

  28. Liu, B., Ferrari, V.: Active learning for human pose estimation. In: ICCV, pp. 4363–4372 (2017)

    Google Scholar 

  29. Liu, H., Wang, R., Shan, S., Chen, X.: Deep supervised hashing for fast image retrieval. In: CVPR, pp. 2064–2072 (2016)

    Google Scholar 

  30. Luo, W., Schwing, A., Urtasun, R.: Latent structured active learning. In: NIPS, pp. 728–736 (2013)

    Google Scholar 

  31. Mac Aodha, O., Campbell, N.D., Kautz, J., Brostow, G.J.: Hierarchical subquery evaluation for active learning on a graph. In: CVPR, pp. 564–571 (2014)

    Google Scholar 

  32. Melville, P., Mooney, R.J.: Diverse ensembles for active learning. In: ICML, p. 74 (2004)

    Google Scholar 

  33. Nguyen, H.T., Smeulders, A.: Active learning using pre-clustering. In: ICML, p. 79 (2004)

    Google Scholar 

  34. Paul, S., Bappy, J.H., Roy-Chowdhury, A.K.: Non-uniform subset selection for active learning in structured data. In: CVPR, pp. 6846–6855 (2017)

    Google Scholar 

  35. Pidhorskyi, S., Jones, Q., Motiian, S., Adjeroh, D., Doretto, G.: Deep supervised hashing with spherical embedding. In: Jawahar, C.V., Li, H., Mori, G., Schindler, K. (eds.) ACCV 2018. LNCS, vol. 11364, pp. 417–434. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20870-7_26

    Chapter  Google Scholar 

  36. Qin, D., Gammeter, S., Bossard, L., Quack, T., Van Gool, L.: Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors. In: CVPR, pp. 777–784 (2011)

    Google Scholar 

  37. Rényi, A., et al.: On measures of entropy and information. In: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics (1961)

    Google Scholar 

  38. Sener, O., Savarese, S.: Active learning for convolutional neural networks: a core-set approach. arXiv preprint arXiv:1708.00489 (2017)

  39. Settles, B., Craven, M.: An analysis of active learning strategies for sequence labeling tasks. In: EMNLP, pp. 1070–1079 (2008)

    Google Scholar 

  40. Settles, B., Craven, M., Ray, S.: Multiple-instance active learning. In: NIPS, pp. 1289–1296 (2008)

    Google Scholar 

  41. Shen, F., Shen, C., Liu, W., Tao Shen, H.: Supervised discrete hashing. In: CVPR, pp. 37–45 (2015)

    Google Scholar 

  42. Shen, F., Xu, Y., Liu, L., Yang, Y., Huang, Z., Shen, H.T.: Unsupervised deep hashing with similarity-adaptive and discrete optimization. TPAMI 40(12), 3034–3044 (2018)

    Article  Google Scholar 

  43. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  44. Vasisht, D., Damianou, A., Varma, M., Kapoor, A.: Active learning for sparse Bayesian multilabel classification. In: KDD, pp. 472–481 (2014)

    Google Scholar 

  45. Vijayanarasimhan, S., Grauman, K.: Large-scale live active learning: training object detectors with crawled data and crowds. IJCV 108(1–2), 97–114 (2014). https://doi.org/10.1007/s11263-014-0721-9

    Article  MathSciNet  Google Scholar 

  46. Wang, G., Hu, Q., Cheng, J., Hou, Z.: Semi-supervised generative adversarial hashing for image retrieval. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11219, pp. 491–507. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01267-0_29

    Chapter  Google Scholar 

  47. Wang, J., Kumar, S., Chang, S.F.: Semi-supervised hashing for large-scale search. TPAMI 34(12), 2393–2406 (2012)

    Article  Google Scholar 

  48. Wang, Q., Si, L., Zhang, Z., Zhang, N.: Active hashing with joint data example and tag selection. In: SIGIR, pp. 405–414 (2014)

    Google Scholar 

  49. Wang, X., Yang, M., Cour, T., Zhu, S., Yu, K., Han, T.X.: Contextual weighting for vocabulary tree based image retrieval. In: ICCV, pp. 209–216 (2011)

    Google Scholar 

  50. Yang, H.F., Lin, K., Chen, C.S.: Supervised learning of semantics-preserving hash via deep convolutional neural networks. TPAMI 40(2), 437–451 (2017)

    Article  Google Scholar 

  51. Zhang, J., Peng, Y.: SSDH: semi-supervised deep hashing for large scale image retrieval. TCSVT 29(1), 212–225 (2017)

    MathSciNet  Google Scholar 

  52. Zhang, S., Li, J., Zhang, B.: Pairwise teacher-student network for semi-supervised hashing. In: CVPR, pp. 0–0 (2019)

    Google Scholar 

  53. Zhao, F., Huang, Y., Wang, L., Tan, T.: Deep semantic ranking based hashing for multi-label image retrieval. In: CVPR, pp. 1556–1564 (2015)

    Google Scholar 

  54. Zhen, Y., Yeung, D.Y.: Active hashing and its application to image and text retrieval. Data Min. Knowl. Disc. 26(2), 255–274 (2013). https://doi.org/10.1007/s10618-012-0249-y

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgement

This work was supported in part by the National Key Research and Development Program of China under Grant 2017YFA0700802, in part by the National Natural Science Foundation of China under Grant 61822603, Grant U1813218, Grant U1713214, and Grant 61672306, in part by Beijing Natural Science Foundation under Grant No. L172051, in part by Beijing Academy of Artificial Intelligence (BAAI), in part by a grant from the Institute for Guo Qiang, Tsinghua University, in part by the Shenzhen Fundamental Research Fund (Subject Arrangement) under Grant JCYJ20170412170602564, and in part by Tsinghua University Initiative Scientific Research Program.

Author information

Authors and Affiliations

  1. Department of Automation, Tsinghua University, Beijing, China

    Ziwei Wang, Quan Zheng, Jiwen Lu & Jie Zhou

  2. State Key Lab of Intelligent Technologies and Systems, Beijing, China

    Ziwei Wang, Quan Zheng, Jiwen Lu & Jie Zhou

  3. Beijing National Research Center for Information Science and Technology, Beijing, China

    Ziwei Wang, Quan Zheng, Jiwen Lu & Jie Zhou

  4. Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China

    Jie Zhou

Authors
  1. Ziwei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quan Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiwen Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiwen Lu .

Editor information

Editors and Affiliations

  1. University of Oxford, Oxford, UK

    Andrea Vedaldi

  2. Graz University of Technology, Graz, Austria

    Horst Bischof

  3. University of Freiburg, Freiburg im Breisgau, Germany

    Thomas Brox

  4. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Jan-Michael Frahm

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 226 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, Z., Zheng, Q., Lu, J., Zhou, J. (2020). Deep Hashing with Active Pairwise Supervision. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12364. Springer, Cham. https://doi.org/10.1007/978-3-030-58529-7_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58529-7_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58528-0

  • Online ISBN: 978-3-030-58529-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation