Skip to main content
SpringerLink
Log in

LCMV: Lightweight Classification Module for Video Domain Adaptation

  • Conference paper
  • First Online:
Image Analysis and Processing – ICIAP 2023 (ICIAP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14234))

Included in the following conference series:

  • 594 Accesses

Abstract

Video action recognition models exhibit high performance on in-distribution data but struggle with distribution shifts in test data. To mitigate this issue, Unsupervised Domain Adaptation (UDA) methods have been proposed, consisting in training on labeled data from a source domain and incorporating unlabeled test data from a target domain to reduce the domain gap. This requires simultaneous access to data from both domains, which may not be practical in real-world scenarios due to privacy issues. A more practical approach called Source-Free Domain Adaptation (SFDA) has been recently proposed, which consists in adapting a well-trained source model using only unlabeled target data. However, existing SFDA methods are computationally intensive and designed for specific architectures. In this paper, we propose an approach called Lightweight Classification Module for Video Domain Adaptation (LCMV). LCMV is based on a backpropagation-free prototypical algorithm, which efficiently adapts a source model using unlabeled target data only. Results on two popular datasets, HMDB-UCF\(_{full}\) and EPIC-Kitchens-55, show significant improvements of LCMV compared to the previous state-of-the-art SFDA methods, and competitive results when compared to state-of-the-art UDA methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Carreira, J., Zisserman, A.: Quo vadis, action recognition? a new model and the kinetics dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6299–6308 (2017)

    Google Scholar 

  2. Chen, C.F.R., et al.: Deep analysis of cnn-based spatio-temporal representations for action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6165–6175 (2021)

    Google Scholar 

  3. Chen, M.H., Kira, Z., AlRegib, G., Yoo, J., Chen, R., Zheng, J.: Temporal attentive alignment for large-scale video domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6321–6330 (2019)

    Google Scholar 

  4. Chen, P., Ma, A.J.: Source-free temporal attentive domain adaptation for video action recognition. In: Proceedings of the 2022 International Conference on Multimedia Retrieval, pp. 489–497 (2022)

    Google Scholar 

  5. Chen, W.Y., Liu, Y.C., Kira, Z., Wang, Y.C.F., Huang, J.B.: A closer look at few-shot classification. In: International Conference on Learning Representations (2019)

    Google Scholar 

  6. Choi, J., Sharma, G., Schulter, S., Huang, J.-B.: Shuffle and attend: video domain adaptation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 678–695. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_40

    Chapter  Google Scholar 

  7. Damen, D., et al.: Scaling egocentric vision: The epic-kitchens dataset. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 720–736 (2018)

    Google Scholar 

  8. Dasgupta, A., Jawahar, C., Alahari, K.: Overcoming label noise for source-free unsupervised video domain adaptation. In: ICVGIP 2022-Indian Conference on Computer Vision, Graphics and Image Processing, pp. 1–9. ACM (2022)

    Google Scholar 

  9. Fan, Q., Chen, C.F.R., Kuehne, H., Pistoia, M., Cox, D.: More is less: Learning efficient video representations by big-little network and depthwise temporal aggregation. Adv. Neural. Inf. Process. Syst. 32, 2261–2270 (2019)

    Google Scholar 

  10. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. The J. Mach. Learn. Res. 17(1), 2030–2096 (2016)

    MathSciNet  MATH  Google Scholar 

  11. Huang, Y., Yang, X., Zhang, J., Xu, C.: Relative alignment network for source-free multimodal video domain adaptation. In: Proceedings of the 30th ACM International Conference on Multimedia, pp. 1652–1660 (2022)

    Google Scholar 

  12. Iwasawa, Y., Matsuo, Y.: Test-time classifier adjustment module for model-agnostic domain generalization. Adv. Neural. Inf. Process. Syst. 34, 2427–2440 (2021)

    Google Scholar 

  13. Kim, D., et al.: Learning cross-modal contrastive features for video domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13618–13627 (2021)

    Google Scholar 

  14. Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., Serre, T.: Hmdb: a large video database for human motion recognition. In: 2011 International Conference on Computer Vision, pp. 2556–2563. IEEE (2011)

    Google Scholar 

  15. Liang, J., Hu, D., Feng, J.: Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In: International Conference on Machine Learning, pp. 6028–6039. PMLR (2020)

    Google Scholar 

  16. Lin, J., Gan, C., Han, S.: TSM: Temporal shift module for efficient video understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7083–7093 (2019)

    Google Scholar 

  17. Long, M., Cao, Y., Wang, J., Jordan, M.: Learning transferable features with deep adaptation networks. In: International Conference on Machine Learning, pp. 97–105. PMLR (2015)

    Google Scholar 

  18. Munro, J., Damen, D.: Multi-modal domain adaptation for fine-grained action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 122–132 (2020)

    Google Scholar 

  19. Sahoo, A., Shah, R., Panda, R., Saenko, K., Das, A.: Contrast and mix: temporal contrastive video domain adaptation with background mixing. Adv. Neural. Inf. Process. Syst. 34, 23386–23400 (2021)

    Google Scholar 

  20. Singh, S., Arora, C., Jawahar, C.: First person action recognition using deep learned descriptors. In: Proceedings of the IEEE Conference On Computer Vision and Pattern Recognition, pp. 2620–2628 (2016)

    Google Scholar 

  21. Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  22. Song, X., et al.: Spatio-temporal contrastive domain adaptation for action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9787–9795 (2021)

    Google Scholar 

  23. Soomro, K., Zamir, A.R., Shah, M.: Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012)

  24. Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: CVPR 2011, pp. 1521–1528. IEEE (2011)

    Google Scholar 

  25. Wang, L.: Temporal segment networks: towards good practices for deep action recognition. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 20–36. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_2

    Chapter  Google Scholar 

  26. Wei, P., et al.: Unsupervised video domain adaptation: A disentanglement perspective. arXiv preprint arXiv:2208.07365 (2022)

  27. Wu, C.Y., Feichtenhofer, C., Fan, H., He, K., Krahenbuhl, P., Girshick, R.: Long-term feature banks for detailed video understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 284–293 (2019)

    Google Scholar 

  28. Xu, Y., Yang, J., Cao, H., Wu, K., Wu, M., Chen, Z.: Source-free video domain adaptation by learning temporal consistency for action recognition. In: Computer Vision-ECCV 2022: 17th European Conference. pp. 147–164. Springer (2022). https://doi.org/10.1007/978-3-031-19830-4_9

  29. Yang, L., Huang, Y., Sugano, Y., Sato, Y.: Interact before align: Leveraging cross-modal knowledge for domain adaptive action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14722–14732 (2022)

    Google Scholar 

  30. Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime TV-L1 optical flow. In: Hamprecht, F.A., Schnörr, C., Jähne, B. (eds.) DAGM 2007. LNCS, vol. 4713, pp. 214–223. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74936-3_22

    Chapter  Google Scholar 

  31. Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017)

  32. Zhou, B., Andonian, A., Oliva, A., Torralba, A.: Temporal relational reasoning in videos. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 803–818 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Politecnico di Torino, Torino, Italy

    Julian Neubert, Mirco Planamente, Chiara Plizzari & Barbara Caputo

  2. Istituto Italiano di Tecnologia, Genoa, Italy

    Mirco Planamente

  3. CINI Consortium, Torino, Italy

    Mirco Planamente & Barbara Caputo

Authors
  1. Julian Neubert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mirco Planamente
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chiara Plizzari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Barbara Caputo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Julian Neubert .

Editor information

Editors and Affiliations

  1. University of Udine, Udine, Italy

    Gian Luca Foresti

  2. University of Udine, Udine, Italy

    Andrea Fusiello

  3. University of York, York, UK

    Edwin Hancock

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Neubert, J., Planamente, M., Plizzari, C., Caputo, B. (2023). LCMV: Lightweight Classification Module for Video Domain Adaptation. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing – ICIAP 2023. ICIAP 2023. Lecture Notes in Computer Science, vol 14234. Springer, Cham. https://doi.org/10.1007/978-3-031-43153-1_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43153-1_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43152-4

  • Online ISBN: 978-3-031-43153-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation