Skip to main content
SpringerLink
Log in

Domain generalization based on domain-specific adversarial learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep learning models often suffer from degraded performance when the distributions of the training and testing data differ (i.e., domain shift). Domain generalization (DG) techniques can help improve the generalization performance for unseen target domains by using multiple source domains. The recently developed domain generalization methods focus on extracting domain-invariant features from all source domains. However, some task-relevant discriminative information can be removed during this process. In addition, the various source domains are treated equally ignoring the negative impacts of distant source domains. Both problems can lead to unsatisfactory performance. This paper proposed a domain-specific adversarial neural network (DSANN) based on adversarial learning to learn effective feature representations and reduce the influence of distantsource domains. The DSANN introduces a reference distribution that is adaptively generated during training. Additionally, domain-invariant features are extracted through a domain-specific adversarial learning process , in which each source domain distribution is aligned only with the reference distribution instead of all the other source domains. Moreover, the DSANN also aligns the outputs of multiple classifiers and adopts the weighted average of the predictions; thus, the employed label classifiers can become more robust to unknown domain shifts. Experiments conducted on popular benchmark datasets demonstrate that our proposed method can achieve remarkable generalization performance and has better classification accuracy than the existing DG algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The datasets used in this work, PACS and Office-Home, can be accessible through https://docs.activeloop.ai/datasets/pacs-dataset and https://docs.activeloop.ai/datasets/office-home-data-set, respectively.

Code Availability

Our Pytorch implementation of DSANN can be accessible through https://github.com/zipingwang929/DSANN

References

  1. Ayodele K, Ikezogwo W, Komolafe M, Ogunbona P (2020) Supervised domain generalization for integration of disparate scalp eeg datasets for automatic epileptic seizure detection. Computers in Biology and Medicine 120:103757

    Article  Google Scholar 

  2. Carlucci FM, D’Innocente A, Bucci S, Caputo B, Tommasi T (2019) Domain generalization by solving jigsaw puzzles. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2229–2238

  3. Carlucci FM, Porzi L, Caputo B, Ricci E, Buló SR (2021) Multidial: Domain alignment layers for (multisource) unsupervised domain adaptation. IEEE Transactions on pattern analysis and machine intelligence 43(12):4441–4452

    Article  Google Scholar 

  4. Chattopadhyay P, Balaji Y, Hoffman J (2020) Learning to balance specificity and invariance for in and out of domain generalization. In: European conference on computer vision, Springer, pp 301–318

  5. Chen K, Zhuang D, Chang JM (2022) Discriminative adversarial domain generalization with meta-learning based cross-domain validation. Neurocomputing 467:418–426

    Article  Google Scholar 

  6. D’Innocente A, Caputo B (2018) Domain generalization with domain-specific aggregation modules. In: German Conference on Pattern Recognition, Springer, pp 187–198

  7. Donahue J, Jia Y, Vinyals O, Hoffman J, Darrell T (2014) Decaf: A deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, PMLR, pp 647–655

  8. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning, PMLR, pp 1180–1189

  9. Ge Z, Song Z, Li X, Zhang L (2022) Meta conditional variational auto-encoder for domain generalization. Computer Vision and Image Understanding 222:103503

    Article  Google Scholar 

  10. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inform Process Syst 27

  11. Huang Z, Wang H, Xing EP, Huang D (2020) Self-challenging improves cross-domain generalization. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, Proceedings, Part II 16, Springer, pp 124–140. Accessed 23–28 Aug 2020

  12. Jin X, Lan C, Zeng W, Chen Z, Zhang L (2020) Style normalization and restitution for generalizable person re-identification. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 3140–3149

  13. Jin X, Lan C, Zeng W, Chen Z (2022) Style normalization and restitution for domain generalization and adaptation. IEEE Transactions on Multimedia 24:3636–3651

    Article  Google Scholar 

  14. Kang G, Lu J, Yi Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4893–4902

  15. Krueger D, Caballero E, Jacobsen JH, Zhang A, Binas J, Zhang D, Priol RL, Courville A (2021) Out-of-distribution generalization via risk extrapolation (rex). In: International conference on machine learning

  16. Li D, Yang Y, Song YZ, Hospedales TM (2017) Deeper, broader and artier domain generalization. In: Proceedings of the IEEE international conference on computer vision, pp 5542–5550

  17. Li H, Pan SJ, Wang S, Kot AC (2018) Domain generalization with adversarial feature learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5400–5409

  18. Li P, Li D, Li W, Gong S, Fu Y, Hospedales TM (2021) A simple feature augmentation for domain generalization. In: 2021 IEEE/CVF International conference on computer vision (ICCV), pp 8866–8875

  19. Li X, Zhang W, Ma H, Luo Z, Li X (2020) Domain generalization in rotating machinery fault diagnostics using deep neural networks. Neurocomputing 403:409–420

    Article  Google Scholar 

  20. Li X, Zhang Z, Gao L, Wen L (2022) A new semi-supervised fault diagnosis method via deep coral and transfer component analysis. IEEE Transactions on emerging topics in computational intelligence 6(3):690–699

    Article  Google Scholar 

  21. Li Y, Gong M, Tian X, Liu T, Tao D (2018) Deep domain generalization via conditional invariant adversarial networks. In: Proceedings of the European conference on computer vision (ECCV), pp 624–639

  22. Li Y, Yang Y, Zhou W, Hospedales TM (2019) Feature-critic networks for heterogeneous domain generalization. In: International conference on machine learning, PMLR, pp 3915–3924

  23. Li Y, Hu W, Li H, Dong H, Zhang B, Tian Q (2020) Aligning discriminative and representative features: An unsupervised domain adaptation method for building damage assessment. IEEE Transactions on image processing 29:6110–6122

  24. Liu ZG, Ning LB, Zhang ZW (2024) A new progressive multisource domain adaptation network with weighted decision fusion. IEEE Transactions on neural networks and learning systems 35(1):1062–1072

    Article  Google Scholar 

  25. Matsuura T, Harada T (2020) Domain generalization using a mixture of multiple latent domains. Proceedings of the AAAI conference on artificial intelligence 34:11749–11756

    Article  Google Scholar 

  26. Nam H, Lee H, Park J, Yoon W, Yoo D (2021) Reducing domain gap by reducing style bias. In: 2021 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 8686–8695

  27. Parascandolo G, Neitz A, Orvieto A, Gresele L, Schlkopf B (2021) Learning explanations that are hard to vary. In: International conference on learning representations

  28. Rahman MM, Fookes C, Baktashmotlagh M, Sridharan S (2020) Correlation-aware adversarial domain adaptation and generalization. Pattern Recog 100:107124

    Article  Google Scholar 

  29. Ren CX, Ge P, Yang P, Yan S (2021) Learning target-domain-specific classifier for partial domain adaptation. IEEE Transactions on neural networks and learning systems 32(5):1989–2001

    Article  MathSciNet  Google Scholar 

  30. Sicilia A, Zhao X, Hwang SJ (2023) Domain adversarial neural networks for domain generalization: when it works and how to improve. Mach Learn 112:26856–2721

  31. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5018–5027

  32. Xu R, Chen Z, Zuo W, Yan J, Liang L (2018) Deep cocktail network: Multi-source unsupervised domain adaptation with category shift. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3964–3973

  33. Zhang W, Li X, Ma H, Luo Z, Li X (2021) Universal domain adaptation in fault diagnostics with hybrid weighted deep adversarial learning. IEEE Transactions on industrial informatics 17(12):7957–7967

    Article  Google Scholar 

  34. Zhao S, Gong M, Liu T, Fu H, Tao D (2020) Domain generalization via entropy regularization. Adv Neural Inform Process Syst 33:16096–16107

    Google Scholar 

  35. Zhao S, Yu Z, Marbach TG, Wang G, Yin A, Zhou Y, Liu X (2023) Mdgad: Meta domain generalization for distribution drift in anomaly detection. Neurocomputing 550:126483

  36. Zheng H, Yang Y, Yin J, Li Y, Wang R, Xu M (2021) Deep domain generalization combining a priori diagnosis knowledge toward cross-domain fault diagnosis of rolling bearing. IEEE Transactions on instrumentation and measurement 70:1–11

    Article  Google Scholar 

  37. Zheng T, Chen Z, Ding S, Cai C, Luo J (2024) Adv-4-adv: Thwarting changing adversarial perturbations via adversarial domain adaptation. Neurocomputing 569:127114

    Article  Google Scholar 

  38. Zhu Y, Zhuang F, Wang D (2019) Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. Proceedings of the AAAI conference on artificial intelligence 33:5989–5996

    Article  Google Scholar 

Download references

Funding

This work was partially supported by National Natural Science Foundation of China (72271034).

Author information

Authors and Affiliations

  1. School of Economics and Management, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Ziping Wang & Xiaohang Zhang

  2. Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Ziping Wang & Xiaohang Zhang

  3. School of Modern Posts, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Zhengren Li & Fei Chen

Authors
  1. Ziping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengren Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaohang Zhang.

Ethics declarations

Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Zhang, X., Li, Z. et al. Domain generalization based on domain-specific adversarial learning. Appl Intell 54, 4878–4889 (2024). https://doi.org/10.1007/s10489-024-05423-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05423-z

Keywords

Search

Navigation