Skip to main content
SpringerLink
Log in

When Active Learning Meets Implicit Semantic Data Augmentation

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13685))

Included in the following conference series:

Abstract

Active learning (AL) is a label-efficient technique for training deep models when only a limited labeled set is available and the manual annotation is expensive. Implicit semantic data augmentation (ISDA) effectively extends the limited amount of labeled samples and increases the diversity of labeled sets without introducing a noticeable extra computational cost. The scarcity of labeled instances and the huge annotation cost of unlabelled samples encourage us to ponder on the combination of AL and ISDA. A nature direction is a pipelined integration, which selects the unlabeled samples via acquisition function in AL for labeling and generates virtual samples by changing the selected samples to semantic transformation directions within ISDA. However, this pipelined combination would not guarantee the diversity of virtual samples. This paper proposes diversity-aware semantic transformation active learning, or DAST-AL framework, that looks ahead the effect of ISDA in the selection of unlabeled samples. Specifically, DAST-AL exploits expected partial model change maximization (EPMCM) to consider selected samples’ potential contribution of the diversity to the labeled set by leveraging the semantic transformation within ISDA when selecting the unlabeled samples. After that, DAST-AL can confidently and efficiently augment the labeled set by implicitly generating more diverse samples. The empirical results on both image classification and semantic segmentation tasks show that the proposed DAST-AL can slightly outperform the state-of-the-art AL approaches. Under the same condition, the proposed method takes less than 3 min for the first cycle of active labeling while the existing agreement discrepancy selection incurs more than 40 min.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abraham, I., Murphey, T.D.: Active learning of dynamics for data-driven control using Koopman operators. IEEE Trans. Rob. 35(5), 1071–1083 (2019)

    Article  Google Scholar 

  2. Beluch, W.H., Genewein, T., Nürnberger, A., Köhler, J.M.: The power of ensembles for active learning in image classification. In: CVPR, pp. 9368–9377 (2018)

    Google Scholar 

  3. Bottou, L.: Stochastic gradient descent tricks. In: Neural Networks: Tricks of the Trade, pp. 421–436 (2012)

    Google Scholar 

  4. Cai, W., Zhang, M., Zhang, Y.: Batch mode active learning for regression with expected model change. IEEE Trans. Neural Netw. Learn. Syst. 28(7), 1668–1681 (2016)

    Article  MathSciNet  Google Scholar 

  5. Cai, W., Zhang, Y., Zhou, J.: Maximizing expected model change for active learning in regression. In: IEEE International Conference on Data Mining, pp. 51–60 (2013)

    Google Scholar 

  6. Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In: ECCV, pp. 132–149 (2018)

    Google Scholar 

  7. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR, pp. 3213–3223 (2016)

    Google Scholar 

  8. Curtiss, J.H.: A note on the theory of moment generating functions. Ann. Math. Stat. 13(4), 430–433 (1942)

    Article  MathSciNet  Google Scholar 

  9. Dasgupta, S., Hsu, D.: Hierarchical sampling for active learning. In: International Conference on Machine Learning, pp. 208–215 (2008)

    Google Scholar 

  10. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR, pp. 248–255 (2009)

    Google Scholar 

  11. Doersch, C., Gupta, A., Efros, A.A.: Unsupervised visual representation learning by context prediction. In: CVPR, pp. 1422–1430 (2015)

    Google Scholar 

  12. Ebrahimi, S., Rohrbach, A., Darrell, T.: Gradient-free policy architecture search and adaptation. In: Conference on Robot Learning, pp. 505–514 (2017)

    Google Scholar 

  13. Fu, M., Yuan, T., Wan, F., Xu, S., Ye, Q.: Agreement-discrepancy-selection: active learning with progressive distribution alignment. In: AAAI, pp. 7466–7473 (2021)

    Google Scholar 

  14. Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059 (2016)

    Google Scholar 

  15. Gal, Y., Islam, R., Ghahramani, Z.: Deep bayesian active learning with image data. In: International Conference on Machine Learning, pp. 1183–1192 (2017)

    Google Scholar 

  16. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)

    Google Scholar 

  17. Jiang, B., Zhang, Z., Lin, D., Tang, J., Luo, B.: Semi-supervised learning with graph learning-convolutional networks. In: CVPR, pp. 11313–11320 (2019)

    Google Scholar 

  18. Joshi, A.J., Porikli, F., Papanikolopoulos, N.: Multi-class active learning for image classification. In: CVPR, pp. 2372–2379 (2009)

    Google Scholar 

  19. Kim, K., Park, D., Kim, K.I., Chun, S.Y.: Task-aware variational adversarial active learning. In: CVPR, pp. 8166–8175 (2021)

    Google Scholar 

  20. Kingma, D.P., Mohamed, S., Rezende, D.J., Welling, M.: Semi-supervised learning with deep generative models. In: NeurIPS, pp. 3581–3589 (2014)

    Google Scholar 

  21. Kuo, W., Häne, C., Yuh, E., Mukherjee, P., Malik, J.: Cost-sensitive active learning for intracranial hemorrhage detection. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 715–723. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00931-1_82

    Chapter  Google Scholar 

  22. Lewis, D.D., Gale, W.A.: A sequential algorithm for training text classifiers, pp. 3–12 (1994)

    Google Scholar 

  23. Li, J., Chen, Z., Chen, J., Lin, Q.: Diversity-sensitive generative adversarial network for terrain mapping under limited human intervention. IEEE Trans. Cybern. 51, 6029–6040 (2020)

    Article  Google Scholar 

  24. Mahajan, D., et al.: Exploring the limits of weakly supervised pretraining. In: ECCV, pp. 181–196 (2018)

    Google Scholar 

  25. Making, M.O.D.: Synthesis lectures on artificial intelligence and machine learning (2012)

    Google Scholar 

  26. Mayer, C., Timofte, R.: Adversarial sampling for active learning. In: IEEE Winter Conference on Applications of Computer Vision, pp. 3071–3079 (2020)

    Google Scholar 

  27. Noroozi, M., Pirsiavash, H., Favaro, P.: Representation learning by learning to count. In: ICCV, pp. 5898–5906 (2017)

    Google Scholar 

  28. Peyre, J., Sivic, J., Laptev, I., Schmid, C.: Weakly-supervised learning of visual relations. In: CVPR, pp. 5179–5188 (2017)

    Google Scholar 

  29. Saito, S., Yang, J., Ma, Q., Black, M.J.: Scanimate: weakly supervised learning of skinned clothed avatar networks. In: CVPR, pp. 2886–2897 (2021)

    Google Scholar 

  30. Sener, O., Savarese, S.: Active learning for convolutional neural networks: a core-set approach. In: ICLR (2018)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)

    Google Scholar 

  32. Sinha, S., Ebrahimi, S., Darrell, T.: Variational adversarial active learning. In: ICCV, pp. 5972–5981 (2019)

    Google Scholar 

  33. Sohn, K., Lee, H., Yan, X.: Learning structured output representation using deep conditional generative models. NeurIPS 28, 3483–3491 (2015)

    Google Scholar 

  34. Wang, D., Zhang, Y., Zhang, K., Wang, L.: Focalmix: semi-supervised learning for 3D medical image detection. In: CVPR, pp. 3951–3960 (2020)

    Google Scholar 

  35. Wang, K., Zhang, D., Li, Y., Zhang, R., Lin, L.: Cost-effective active learning for deep image classification. IEEE TCSVT 27(12), 2591–2600 (2016)

    Google Scholar 

  36. Wang, Y., Huang, G., Song, S., Pan, X., Xia, Y., Wu, C.: Regularizing deep networks with semantic data augmentation. IEEE TPAMI 44, 3733–3748 (2021)

    Google Scholar 

  37. Yoo, D., Kweon, I.S.: Learning loss for active learning. In: CVPR, pp. 93–102 (2019)

    Google Scholar 

  38. Yu, F., Koltun, V., Funkhouser, T.: Dilated residual networks. In: CVPR, pp. 472–480 (2017)

    Google Scholar 

  39. Yuan, T., et al.: Multiple instance active learning for object detection. In: CVPR, pp. 5330–5339 (2021)

    Google Scholar 

  40. Zhai, X., Oliver, A., Kolesnikov, A., Beyer, L.: S4l: self-supervised semi-supervised learning. In: ICCV, pp. 1476–1485 (2019)

    Google Scholar 

  41. Zhang, B., Li, L., Yang, S., Wang, S., Zha, Z.J., Huang, Q.: State-relabeling adversarial active learning. In: CVPR, pp. 8756–8765 (2020)

    Google Scholar 

  42. Zhu, J.J., Bento, J.: Generative adversarial active learning. arXiv preprint (2017)

    Google Scholar 

  43. Zhuang, C., Zhai, A.L., Yamins, D.: Local aggregation for unsupervised learning of visual embeddings. In: ICCV, pp. 6002–6012 (2019)

    Google Scholar 

  44. Zhukov, D., Alayrac, J.B., Cinbis, R.G., Fouhey, D., Laptev, I., Sivic, J.: Cross-task weakly supervised learning from instructional videos. In: CVPR, pp. 3537–3545 (2019)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Nature Science Foundation of China under Grants U2013201, 62073225, 62072315, 61836005 and 62006157, the Natural Science Foundation of Guangdong Province-Outstanding Youth Program under Grant 2019B151502018, the Guangdong “Pearl River Talent Recruitment Program” under Grant 2019ZT08X603, the Guangdong"Pearl River Talent Plan" under Grant 2019JC01X235, and the Shenzhen Science and Technology Innovation Commission R2020A045.

Author information

Authors and Affiliations

  1. Shenzhen University, Shenzhen, 518060, China

    Zhuangzhuang Chen, Jin Zhang, Pan Wang, Jie Chen & Jianqiang Li

Authors
  1. Zhuangzhuang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianqiang Li .

Editor information

Editors and Affiliations

  1. Tel Aviv University, Tel Aviv, Israel

    Shai Avidan

  2. University College London, London, UK

    Gabriel Brostow

  3. Google AI, Accra, Ghana

    Moustapha Cissé

  4. University of Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Facebook (United States), Menlo Park, CA, USA

    Tal Hassner

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chen, Z., Zhang, J., Wang, P., Chen, J., Li, J. (2022). When Active Learning Meets Implicit Semantic Data Augmentation. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13685. Springer, Cham. https://doi.org/10.1007/978-3-031-19806-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19806-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19805-2

  • Online ISBN: 978-3-031-19806-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation