Skip to main content
SpringerLink
Log in

Causal view mechanism for adversarial domain adaptation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Studies show that the challenge for adversarial domain adaptation is learning domain-invariant representations and alleviating the domain gap. However, the construction of domain-invariant representations suppresses the class-level structure information, and the pursuit of class-level structure information distorts the constructed domain-invariant representations. Till present, it still has difficulty to capture the domain-invariant representations while preserving the class-level structure information in the adversarial training process explicitly. In this paper, we propose a Causal view Mechanism for adversarial Domain Adaptation (CMDA). Firstly, a causal effect model of adversarial DA is proposed and reveals the influence of potential confounders in the adversarial training process. Then, CMDA is proposed to disentangle the domain-specific representations into multiple underlying factors and filter out irrelevant confounding characteristics. Specifically, CMDA capture the desired domain-invariant representations by subtracting the domain-level and class-level confounding characteristics. CMDA not only could preserve the class-level structure information to reduce classification error, but also improve transferability simultaneously. Finally, experiments carried out on Office-31, Office-Home, and VisDA-2017 datasets show that our CMDA method presents strong competition among some recent domain adaptation methods, and the average accuracies achieve 71.3%, 89.3% and 76.1% respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Notes

  1. Code available at https://drive.google.com/drive/folders/1uJKHN5JfsVsM0dLIGBgx9P-DoIK2wllw?usp=sharing.

References

  1. Ben-David S, Blitzer J, Crammer K, Pereira F (2007) Analysis of representations for domain adaptation. Adv Neural Inf Process Syst 19:137

    Google Scholar 

  2. Chen M, Zhao S, Liu H, Cai D (2020) Adversarial-learned loss for domain adaptation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp. 3521–3528

  3. Cui S, Wang S, Zhuo J, Li L, Huang Q, Tian Q (2020) Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), pp. 3941–3950

  4. Cui S, Wang S, Zhuo J, Su C, Huang Q, Tian Q (2020) Gradually vanishing bridge for adversarial domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12455–12464

  5. Deng Z, Luo Y, Zhu J (2019) Cluster alignment with a teacher for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9944–9953

  6. Ding Y, Fan H, Xu M, Yang Y (2020) Adaptive exploration for unsupervised person re-identification. ACM Trans Multimed Comput Commun Appl (TOMM) 16(1):1–19

    Article  Google Scholar 

  7. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In International conference on machine learning (PMLR), pp. 1180–1189

  8. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17(1):2096–2030

    MathSciNet  MATH  Google Scholar 

  9. Ghifary M, Kleijn WB, Zhang M, Balduzzi D, Li W (2016) Deep reconstruction-classification networks for unsupervised domain adaptation. European conference on computer vision (ICCV). Springer, Cham, pp 597–613

    Google Scholar 

  10. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst (NIPS) 27:1–9

  11. He Z, Yang B, Chen C, Mu Q, Li Z (2020) CLDA: an adversarial unsupervised domain adaptation method with classifier-level adaptation. Multimed Tools Appl 79(45):33973–33991

    Article  Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778

  13. Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Darrell T (2018) Cycada: Cycle-consistent adversarial domain adaptation. In: International conference on machine learning (PMLR), pp. 1989–1998

  14. Jing M, Zhao J, Li J, Zhu L, Yang Y, Shen HT (2020) Adaptive component embedding for domain adaptation. IEEE Trans Cybern 51(7):3390–3403

  15. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  16. Li J, Chen E, Ding Z, Zhu L, Lu K, Huang Z (2019) Cycle-consistent conditional adversarial transfer networks. In Proceedings of the 27th ACM International Conference on Multimedia, pp. 747–755

  17. Li J, Chen E, Ding Z, Zhu L, Lu K, Shen HT (2020) Maximum density divergence for domain adaptation. IEEE Trans Pattern Anal Mach Intell 43(11):3918–3930

  18. Li G, Kang G, Liu W, Wei Y, Yang Y (2020) Content-consistent matching for domain adaptive semantic segmentation. In European conference on computer vision Springer, Cham. pp. 440–456

  19. Li J, Li Z, Lü S (2021) Feature concatenation for adversarial domain adaptation. Expert Syst Appl 169:114490

    Article  Google Scholar 

  20. Li J, Lu K, Huang Z, Zhu L, Shen HT (2018) Heterogeneous domain adaptation through progressive alignment. IEEE Trans Neural Netw Learn Syst 30(5):1381–1391

    Article  MathSciNet  Google Scholar 

  21. Li J, Lu K, Huang Z, Zhu L, Shen HT (2018) Transfer independently together: A generalized framework for domain adaptation. IEEE Trans Cybern 49(6):2144–2155

    Article  Google Scholar 

  22. Li S, Song S, Huang G, Ding Z, Wu C (2018) Domain invariant and class discriminative feature learning for visual domain adaptation. IEEE Trans Image Process 27(9):4260–4273

    Article  MathSciNet  Google Scholar 

  23. Liu MY, Breuel T, Kautz J (2017) Unsupervised image-to-image translation networks. In: Advances in neural information processing systems (NIPS) 30:700–708

  24. Liu H, Guo F, Xia D (2021) Domain adaptation with structural knowledge transfer learning for person re-identification. Multimed Tools Appl 80(19):29321–29337

  25. Liu H, Long M, Wang J, Jordan M (2019) Transferable adversarial training: A general approach to adapting deep classifiers. In International Conference on Machine Learning, (PMLR), pp. 4013–4022

  26. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In International conference on machine learning (PMLR), pp. 97–105

  27. Long M, Cao Z, Wang J, Jordan MI (2017) Conditional adversarial domain adaptation. arXiv preprint arXiv:1705.10667

  28. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In International conference on machine learning (PMLR), pp. 2208–2217

  29. Long M, Zhu H, Wang J, Jordan MI (2016) Unsupervised domain adaptation with residual transfer networks. Adv Neural Inf Process Syst 2016:136–144

    Google Scholar 

  30. Niu Y, Tang K, Zhang H, Lu Z, Hua XS, Wen JR (2021) Counterfactual vqa: A cause-effect look at language bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12700–12710

  31. Pearl J (2000) Models, reasoning and inference. Cambridge University Press, Cambridge, UK, p 19

    MATH  Google Scholar 

  32. Pearl Judea, Glymour Madelyn, Jewell Nicholas P (2016) Causal inference in statistics: A primer. John Wiley & Sons

    MATH  Google Scholar 

  33. Peng X, Usman B, Kaushik N, Hoffman J, Wang D, Saenko K (2017) Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924

  34. Pinheiro PO (2018) Unsupervised domain adaptation with similarity learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 8004–8013

  35. Qi J, Niu Y, Huang J, Zhang H (2020) Two causal principles for improving visual dialog. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10860–10869

  36. Rubin DB (2019) Essential concepts of causal inference: a remarkable history and an intriguing future. Biostat Epidemiol 3(1):140–155

    Article  Google Scholar 

  37. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. European conference on computer vision (ICCV). Springer, Berlin, Heidelberg, pp 213–226

    Google Scholar 

  38. Sankaranarayanan S, Balaji Y, Castillo CD, Chellappa R (2018) Generate to adapt: Aligning domains using generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8503–8512

  39. Sugiyama M, Krauledat M, Müller KR (2007) Covariate shift adaptation by importance weighted cross validation. J Mach Learn Res (JMLR) 8(5):985–1005

  40. Sur C (2019) Survey of deep learning and architectures for visual captioning transitioning between media and natural languages. Multimed Tools Appl 78(22):32187–32237

    Article  Google Scholar 

  41. Tang K, Huang J, Zhang H (2020) Long-tailed classification by keeping the good and removing the bad momentum causal effect. In: Conference on neural information processing systems (NIPS)

  42. Tang K, Niu Y, Huang J, Shi J, Zhang H (2020) Unbiased scene graph generation from biased training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3716–3725

  43. Tang K, Tao M, Zhang H (2021) Adversarial Visual Robustness by Causal Intervention. arXiv:2106.09534

  44. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7167–7176

  45. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 5018–5027

  46. Wang T, Huang J, Zhang H, Sun Q (2020) Visual commonsense r-cnn. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10760–10770

  47. Wei G, Lan C, Zeng W, Chen Z (2021) MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16643–16653

  48. Xie Y, Du Z, Li J, Jing M, Chen E, Lu K (2020) Joint metric and feature representation learning for unsupervised domain adaptation. Knowl-Based Syst 192:105222

    Article  Google Scholar 

  49. Yang X, Zhang H, Cai J (2021) Deconfounded image captioning: A causal retrospect. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2021.3121705

  50. Yang Y, Zhuang Y, Pan Y (2021) Multiple knowledge representation for big data artificial intelligence: framework, applications, and case studies. Front Inf Technol Electron Eng 22(12):1551–1558

    Article  Google Scholar 

  51. Yue Z, Sun Q, Hua XS, Zhang H (2021) Transporting causal mechanisms for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 8599–8608

  52. Zhang Q, Zhang J, Liu W, Tao D (2019) Category anchor-guided unsupervised domain adaptation for semantic segmentation. Adv Neural Inf Process Syst(NIPS) 32:1–11

  53. Zou Y, Yu Z, Kumar BVK, Wang J (2018) Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference on computer vision (ECCV), pp. 289–305

Download references

Acknowledgements

This work is supported by the Innovation Capacity Construction Project of Jilin Province Development and Reform Commission(2021FGWCXNLJSSZ10,2019C053-3), the National Key Research and Development Program of China (No. 2020YFA0714103) and the Fundamental Research Funds for the Central Universities, JLU.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Zihao Fu, Shengsheng Wang, Xin Zhao, Sifan Long & Bilin Wang

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Zihao Fu, Shengsheng Wang, Xin Zhao, Sifan Long & Bilin Wang

Authors
  1. Zihao Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengsheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sifan Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bilin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengsheng Wang.

Ethics declarations

Competing interest

The author declares that there exists no conflict of interest or personal relationship with other people and organizations regarding the publication of this article.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fu, Z., Wang, S., Zhao, X. et al. Causal view mechanism for adversarial domain adaptation. Multimed Tools Appl 82, 47347–47366 (2023). https://doi.org/10.1007/s11042-023-15683-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15683-5

Keywords

Search

Navigation