Abstract
Domain Adaptation (DA) has been a crucial topic for action recognition, as the test set and training set are not always subject to the identical distribution, which will lead to significant performance degradation. Existing researches focus on DA methods based on the entire videos, ignoring the different contributions of different samples and regions. In this paper, we propose an uncertainty-aware domain adaptation method for action recognition from a new perspective. The aleatoric uncertainty is firstly used in the classifier to improve the performance by alleviating the impact of noisy labels. Then the aleatoric uncertainty calculated with Bayesian Neural Network is embedded in the discriminator to help the network focus on the spatial areas and temporal clips with lower uncertainty during training. The spatial-temporal attention map is generated to enhance the features with the guidance of backward passing. Extensive experiments are conducted on both small-scale and large-scale datasets, and the results indicate that the proposed method achieves competitive performance with fewer computational workloads. The code is available at: https://github.com/ChickAndMushroom/Uncertainty-aware-DA-for-AR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Song, X., et al.: Spatio-temporal contrastive domain adaptation for action recognition. In: CVPR (2021)
Choi, J., Sharma, G., Schulter, S., Huang, J.-B.: Shuffle and attend: video domain adaptation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 678–695. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_40
Pan, B., et al.: Adversarial cross-domain action recognition with co-attention. In: AAAI (2020)
Sultani, I.W.: Human action recognition across datasets by foreground-weighted histogram decomposition. Phys. Lett. B 690(2), 764–771 (2014)
Xu, T., et al.: Dual many-to-one-encoder-based transfer learning for cross-dataset human action recognition. Image Vis. Comput. 55(PT.2), 127–137 (2016)
Jamal, A., Namboodiri, V.P., Deodhare, D., Venkatesh, K.S.: Deep domain adaptation in action space. In: BMVC (2018)
Zhang, X.Y., et al.: Learning transferable self-attentive representations for action recognition in untrimmed videos with weak supervision. In: AAAI (2019)
Girdhar, R., Ramanan, D., Gupta, A., et al.: ActionVLAD: learning spatio-temporal aggregation for action classification. In: CVPR (2017)
Liang, J., He, R., Sun, Z., et al.: Exploring uncertainty in pseudo-label guided unsupervised domain adaptation. Pattern Recognit. 96, 1069(96) (2019)
Wen, J., Zheng, N., Yuan, J., et al.: Bayesian uncertainty matching for unsupervised domain adaptation. In: IJCAI (2019)
Han, L., Zou, Y., Gao, R., et al.: Unsupervised domain adaptation via calibrating uncertainties. In: CVPRW (2019)
Manders, J., Laarhoven, T.V., Marchiori, E.: Simple Domain Adaptation with Class Prediction Uncertainty Alignment. In: CORR, Arxiv/abs/1804.04448 (2018)
Kurmi, V.K., Kumar, S., Namboodiri, V.P.: Attending to discriminative certainty for domain adaptation. In: CVPR (2020)
Gal, Y.: Uncertainty in deep learning. PhD thesis, University of Cambridge (2016). 2, 3
Pei, Z., Cao, Z., Long, M., Wang, J.: Multi-adversarial domain adaptation. In: AAAI (2018)
Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 59:1–59:35 (2016)
Chen, M.H., et al.: Temporal attentive alignment for large-scale video domain adaptation. In: ICCV (2019)
Peng, X., Usman, B., Saito, K., Kaushik, N., Hoffman, J., Saenko, K.: Syn2real: A new benchmark for synthetic-to-real visual domain adaptation. arXiv preprint arXiv:1806.09755 (2018)
Long, M., et al.: Deep transfer learning with joint adaptation networks. In: ICML (2017)
Li, Y., et al.: Adaptive batch normalization for practical domain adaptation. Pattern Recognit. 80, 109–117 (2018)
Saito, K., Watanabe, K., Ushiku, Y., et al.: Maximum classifier discrepancy for unsupervised domain adaptation. In: CVPR (2018)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9(Nov), 2579–2605 (2008)
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: NeuRIPS (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhu, X., Wu, Y., Yang, Z., Liu, P. (2021). Uncertainty-Aware Domain Adaptation for Action Recognition. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13110. Springer, Cham. https://doi.org/10.1007/978-3-030-92238-2_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-92238-2_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92237-5
Online ISBN: 978-3-030-92238-2
eBook Packages: Computer ScienceComputer Science (R0)