skip to main content
research-article

Unbiased Semantic Representation Learning Based on Causal Disentanglement for Domain Generalization

Published: 12 June 2024 Publication History

Abstract

Domain generalization primarily mitigates domain shift among multiple source domains, generalizing the trained model to an unseen target domain. However, the spurious correlation usually caused by context prior (e.g., background) makes it challenging to get rid of the domain shift. Therefore, it is critical to model the intrinsic causal mechanism. The existing domain generalization methods only attend to disentangle the semantic and context-related features by modeling the causation between input and labels, which totally ignores the unidentifiable but important confounders. In this article, a Causal Disentangled Intervention Model (CDIM) is proposed for the first time, to the best of our knowledge, to construct confounders via causal intervention. Specifically, a generative model is employed to disentangle the semantic and context-related features. The contextual information of each domain from generative model can be considered as a confounder layer, and the center of all context-related features is utilized for fine-grained hierarchical modeling of confounders. Then the semantic and confounding features from each layer are combined to train an unbiased classifier, which exhibits both transferability and robustness across an unknown distribution domain. CDIM is evaluated on three widely recognized benchmark datasets, namely, Digit-DG, PACS, and NICO, through extensive ablation studies. The experimental results clearly demonstrate that the proposed model achieves state-of-the-art performance.

References

[1]
Kartik Ahuja, Divyat Mahajan, Yixin Wang, and Yoshua Bengio. 2023. Interventional causal representation learning. In International Conference on Machine Learning. PMLR, 372–407.
[2]
Kartik Ahuja, Karthikeyan Shanmugam, Kush Varshney, and Amit Dhurandhar. 2020. Invariant risk minimization games. In International Conference on Machine Learning. PMLR, 145–155.
[3]
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization. arXiv:1907.02893. Retrieved from https://arxiv.org/abs/1907.02893
[4]
Haoyue Bai, Rui Sun, Lanqing Hong, Fengwei Zhou, Nanyang Ye, Han-Jia Ye, S.-H. Gary Chan, and Zhenguo Li. 2021. Decaug: Out-of-distribution generalization via decomposed feature representation and semantic augmentation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 6705–6713.
[5]
Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. 2018. Metareg: Towards domain generalization using meta-regularization. In Advances in Neural Information Processing Systems, Vol. 31 (2018).
[6]
Fabio M. Carlucci, Antonio D’Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. 2019. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2229–2238.
[7]
Prithvijit Chattopadhyay, Yogesh Balaji, and Judy Hoffman. 2020. Learning to balance specificity and invariance for in and out of domain generalization. In Proceedings of the 16th European Conference on Computer Vision (ECCV’20), Part IX 16. Springer, 301–318.
[8]
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems, Vol. 29.
[9]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 248–255.
[10]
Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. 2019. Domain generalization via model-agnostic learning of semantic features. In Advances in Neural Information Processing Systems, Vol. 32 (2019).
[11]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning. PMLR, 1126–1135.
[12]
Christina M. Funke, Paul Vicol, Kuan-Chieh Wang, Matthias Kümmerer, Richard Zemel, and Matthias Bethge. 2022. Disentanglement and generalization under correlation shifts. In Conference on Lifelong Learning Agents, Vol. 199. PMLR, 116–141.
[13]
Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning, Vol. 37. PMLR, 1180–1189.
[14]
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 1 (2016), 2096–2030.
[15]
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. 2020. Shortcut learning in deep neural networks. Nat. Mach. Intell. 2, 11 (2020), 665–673.
[16]
Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. 2005. Measuring statistical dependence with Hilbert-Schmidt norms. In Proceedings of the 16th International Conference on Algorithmic Learning Theory (ALT’05), Vol. 3734. Springer, 63–77.
[17]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[18]
Yue He, Zheyan Shen, and Peng Cui. 2021. Towards non-iid image classification: A dataset and baselines. Pattern Recogn. 110 (2021), 107383.
[19]
Bingyu Hu, Jiawei Liu, Kecheng Zheng, and Zheng-Jun Zha. 2024. Unleashing knowledge potential of source hypothesis for source-free domain adaptation. IEEE Trans. Multimedia 26 (2024), 5422–5434. DOI:
[20]
Maximilian Ilse, Jakub M. Tomczak, and Patrick Forré. 2021. Selecting data augmentation for simulating interventions. In International Conference on Machine Learning, Vol. 139. PMLR, 4555–4562.
[21]
Maximilian Ilse, Jakub M. Tomczak, Christos Louizos, and Max Welling. 2020. Diva: Domain invariant variational autoencoders. In Medical Imaging with Deep Learning, Vol. 121. PMLR, 322–348.
[22]
Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational bayes. Retrieved from https://arxiv.org/abs/1312.6114
[23]
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2017. Deeper, broader and artier domain generalization. In Proceedings of the IEEE International Conference on Computer Vision. 5542–5550.
[24]
Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2018. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 32 (2018).
[25]
Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, and Timothy M. Hospedales. 2019. Episodic training for domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1446–1455.
[26]
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C. Kot. 2018. Domain generalization with adversarial feature learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5400–5409.
[27]
Jinfeng Li, Weifeng Liu, Yicong Zhou, Jun Yu, Dapeng Tao, and Changsheng Xu. 2022. Domain-invariant graph for adaptive semi-supervised domain adaptation. ACM Trans. Multimedia Comput. Commun. Appl. 18, 3, Article 72 (Mar.2022), 18 pages. DOI:
[28]
Pan Li, Da Li, Wei Li, Shaogang Gong, Yanwei Fu, and Timothy M. Hospedales. 2022. A simple feature augmentation for domain generalization. In Proceedings of the International Conference on Computer Vision. IEEE, 8866–8875.
[29]
Shiqi Lin, Zhizheng Zhang, Zhipeng Huang, Yan Lu, Cuiling Lan, Peng Chu, Quanzeng You, Jiang Wang, Zicheng Liu, Amey Parulkar, et al. 2023. Deep frequency filtering for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11797–11807.
[30]
Phillip Lippe, Sara Magliacane, Sindy Löwe, Yuki M. Asano, Taco Cohen, and Efstratios Gavves. 2022. Intervention design for causal representation learning. In UAI 2022 Workshop on Causal Representation Learning.
[31]
Chang Liu, Xinwei Sun, Jindong Wang, Haoyue Tang, Tao Li, Tao Qin, Wei Chen, and Tie-Yan Liu. 2021. Learning causal semantic representation for out-of-distribution prediction. In Advances in Neural Information Processing Systems 34 (2021), 6155–6170.
[32]
Jiawei Liu, Zhipeng Huang, Liang Li, Kecheng Zheng, and Zheng-Jun Zha. 2022. Debiased batch normalization via gaussian process for generalizable person re-identification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 1729–1737.
[33]
Yajing Liu, Zhiwei Xiong, Ya Li, Yuning Lu, Xinmei Tian, and Zheng-Jun Zha. 2023. Category-stitch learning for union domain generalization. ACM Trans. Multimedia Comput. Commun. Appl. 19, 1, Article 25 (Jan.2023), 19 pages. DOI:
[34]
Yen-Cheng Liu, Yu-Ying Yeh, Tzu-Chien Fu, Sheng-De Wang, Wei-Chen Chiu, and Yu-Chiang Frank Wang. 2018. Detach and adapt: Learning cross-domain disentangled deep representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8867–8876.
[35]
Hoang-Chau Luong and Minh-Triet Tran. 2023. Applying adaptive sharpness-aware minimization to improve out-of-distribution generalization. In Proceedings of the 12th International Symposium on Information and Communication Technology. 450–455.
[36]
Jianxin Ma, Peng Cui, Kun Kuang, Xin Wang, and Wenwu Zhu. 2019. Disentangled graph convolutional networks. In International Conference on Machine Learning, Vol. 97. PMLR, 4212–4221.
[37]
Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, and Gianfranco Doretto. 2017. Unified deep supervised domain adaptation and generalization. In Proceedings of the IEEE International Conference on Computer Vision. 5715–5725.
[38]
Ziwei Niu, Junkun Yuan, Xu Ma, Yingying Xu, Jing Liu, Yen-Wei Chen, Ruofeng Tong, and Lanfen Lin. 2023. Knowledge distillation-based domain-invariant representation learning for domain generalization. IEEE Trans. Multimedia (2023), 1–11. DOI:
[39]
Giambattista Parascandolo, Niki Kilbertus, Mateo Rojas-Carulla, and Bernhard Schölkopf. 2018. Learning independent causal mechanisms. In International Conference on Machine Learning, Vol. 80. PMLR, 4036–4044.
[40]
Judea Pearl. 1995. Causal diagrams for empirical research. Biometrika 82, 4 (1995), 669–688.
[41]
Judea Pearl. 2009. Causal inference in statistics: An overview. Stat. Surv. 3 (2009), 96–146.
[42]
Xingchao Peng, Zijun Huang, Ximeng Sun, and Kate Saenko. 2019. Domain agnostic learning with disentangled representations. In International Conference on Machine Learning, Vol. 97. PMLR, 5102–5112.
[43]
Jonas Peters, Peter Bühlmann, and Nicolai Meinshausen. 2016. Causal inference by using invariant prediction: Identification and confidence intervals. J. Roy. Stat. Soc.: Ser. B (Stat. Methodol.) 78, 5 (2016), 947–1012.
[44]
Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. 2017. Elements of Causal Inference: Foundations and Learning Algorithms. The MIT Press.
[45]
Mohammad Pezeshki, Oumar Kaba, Yoshua Bengio, Aaron C. Courville, Doina Precup, and Guillaume Lajoie. 2021. Gradient starvation: A learning proclivity in neural networks. In Advances in Neural Information Processing Systems 34 (2021), 1256–1272.
[46]
Xin Qin, Jindong Wang, Yiqiang Chen, Wang Lu, and Xinlong Jiang. 2022. Domain generalization for activity recognition via adaptive feature fusion. ACM Trans. Intell. Syst. Technol. 14, 1, Article 9 (Nov.2022), 21 pages. DOI:
[47]
Bernhard Schölkopf, Dominik Janzing, Jonas Peters, Eleni Sgouritsa, Kun Zhang, and Joris Mooij. 2012. On causal and anticausal learning. arXiv:1206.6471. Retrieved from https://arxiv.org/abs/1206.6471
[48]
Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. 2021. Toward causal representation learning. Proc. IEEE 109, 5 (2021), 612–634.
[49]
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision. 618–626.
[50]
Seonguk Seo, Yumin Suh, Dongwan Kim, Geeho Kim, Jongwoo Han, and Bohyung Han. 2020. Learning to optimize domain specific normalization for domain generalization. In Proceedings of the 16th European Conference on Computer Vision (ECCV’20), Part XXII 16. Springer, 68–83.
[51]
Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita Sarawagi. 2018. Generalizing across domains via cross-gradient training. arXiv:1804.10745. Retrieved from https://arxiv.org/abs/1804.10745
[52]
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. 2017. Domain randomization for transferring deep neural networks from simulation to the real world. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’17). IEEE, 23–30.
[53]
Yue Wang, Lei Qi, Yinghuan Shi, and Yang Gao. 2022. Feature-based style randomization for domain generalization. IEEE Trans. Circ. Syst. Vid. Technol. 32, 8 (2022), 5495–5509.
[54]
Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A discriminative feature learning approach for deep face recognition. In Proceedings of the 14th European Conference on Computer Vision (ECCV’16), Part VII 14. Springer, 499–515.
[55]
Lei Wu, Hefei Ling, Yuxuan Shi, and Baiyan Zhang. 2022. Instance correlation graph for unsupervised domain adaptation. ACM Trans. Multimedia Comput. Commun. Appl. 18, 1s, Article 33 (Jan.2022), 23 pages. DOI:
[56]
Yifan Xu, Kekai Sheng, Weiming Dong, Baoyuan Wu, Changsheng Xu, and Bao-Gang Hu. 2022. Towards corruption-agnostic robust domain adaptation. ACM Trans. Multimedia Comput. Commun. Appl. 18, 4, Article 99 (Mar.2022), 16 pages. DOI:
[57]
Mengyue Yang, Furui Liu, Zhitang Chen, Xinwei Shen, Jianye Hao, and Jun Wang. 2021. CausalVAE: Disentangled representation learning via neural structural causal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9593–9602.
[58]
Shuai Yang, Xianjie Guo, Kui Yu, Xiaoling Huang, Tingting Jiang, Jin He, and Lichuan Gu. 2023. Causal feature selection in the presence of sample selection bias. ACM Trans. Intell. Syst. Technol. 14, 5, Article 78 (Aug.2023), 18 pages. DOI:
[59]
Xiangyu Yue, Yang Zhang, Sicheng Zhao, Alberto Sangiovanni-Vincentelli, Kurt Keutzer, and Boqing Gong. 2019. Domain randomization and pyramid consistency: Simulation-to-real generalization without accessing target domain data. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 2100–2110.
[60]
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. 2020. Learning to generate novel domains for domain generalization. In European Conference on Computer Vision, Vol. 12361. Springer, 561–578.
[61]
Kaiyang Zhou, Yongxin Yang, Yu Qiao, and Tao Xiang. 2021. Domain generalization with mixstyle. arXiv:2104.02008. Retrieved from https://arxiv.org/abs/2104.02008

Index Terms

  1. Unbiased Semantic Representation Learning Based on Causal Disentanglement for Domain Generalization

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 20, Issue 8
    August 2024
    726 pages
    EISSN:1551-6865
    DOI:10.1145/3618074
    • Editor:
    • Abdulmotaleb El Saddik
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 June 2024
    Online AM: 24 April 2024
    Accepted: 03 April 2024
    Revised: 26 March 2024
    Received: 31 August 2023
    Published in TOMM Volume 20, Issue 8

    Check for updates

    Author Tags

    1. Transfer learning
    2. domain generalization
    3. disentangled representation
    4. causal intervention
    5. semantic representation

    Qualifiers

    • Research-article

    Funding Sources

    • National Natural Science Foundation of China
    • Key Research and Development Project of Zhejiang Province
    • Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 330
      Total Downloads
    • Downloads (Last 12 months)330
    • Downloads (Last 6 weeks)14
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media