research-article

Disentangled Representation Learning with Causality for Unsupervised Domain Adaptation

Authors:

Xingyi ZhangAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 2918 - 2926

https://doi.org/10.1145/3581783.3611725

Published: 27 October 2023 Publication History

Abstract

Most efforts in unsupervised domain adaptation (UDA) focus on learning the domain-invariant representations between the two domains. However, such representations may still confuse two patterns due to the domain gap. Considering that semantic information is useful for the final task and domain information always indicates the discrepancy between two domains, to address this issue, we propose to decouple the representations of semantic features from domain features to reduce domain bias. Different from traditional methods, we adopt a simple but effective module with only one domain discriminator to decouple the representations, offering two benefits. Firstly, it eliminates the need for labeled sample pairs, making it more suitable for UDA. Secondly, without adversarial learning, our model can achieve a more stable training phase. Moreover, to further enhance the task-specific features, we employ a causal mechanism to separate semantic features related to causal factors from the overall feature representations. Specially, we utilize a dual-classifier strategy, where each classifier is fed with the entire features and the semantic features, respectively. By minimizing the discrepancy between the outputs of the two classifiers, the causal influence of the semantic features is accentuated. Experiments on several public datasets demonstrate the proposed model can outperform the state-of-the-art methods. Our code is available at: https://github.com/qzxRtY37/DRLC https://github.com/qzxRtY37/DRLC.

References

[1]

Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning (2010), 151--175.

[2]

Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2006. Analysis of representations for domain adaptation. NeurIPS, Vol. 19 (2006).

[3]

Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. NeurIPS, Vol. 29 (2016).

[4]

Ruichu Cai, Zijian Li, Pengfei Wei, Jie Qiao, Kun Zhang, and Zhifeng Hao. 2019. Learning disentangled semantic representation for domain adaptation. In IJCAI.

[5]

Tianyu Chang, Xun Yang, Tianzhu Zhang, and Meng Wang. 2023. Domain Generalized Stereo Matching via Hierarchical Visual Transformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 9559--9568.

[6]

Lin Chen, Huaian Chen, Zhixiang Wei, Xin Jin, Xiao Tan, Yi Jin, and Enhong Chen. 2022. Reusing the task-specific classifier as a discriminator: Discriminator-free adversarial domain adaptation. In CVPR. 7181--7190.

[7]

Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. 2019. Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation. In ICML, Vol. 97. 1081--1090.

[8]

Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, and Qi Tian. 2020. Gradually vanishing bridge for adversarial domain adaptation. In CVPR. 12455--12464.

[9]

Zhekai Du, Jingjing Li, Hongzu Su, Lei Zhu, and Ke Lu. 2021. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In CVPR. 3937--3946.

[10]

Geoffrey French, Michal Mackiewicz, and Mark Fisher. 2017. Self-ensembling for visual domain adaptation. arXiv preprint arXiv:1706.05208 (2017).

[11]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In ICML. 1180--1189.

[12]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.

[13]

Qibin Hou, Daquan Zhou, and Jiashi Feng. 2021. Coordinate attention for efficient mobile network design. In CVPR. 13713--13722.

[14]

Junguang Jiang, Ximei Wang, Mingsheng Long, and Jianmin Wang. 2020. Resource efficient domain adaptation. In ACM MM. 2220--2228.

[15]

ZQ John Lu. 2010. The elements of statistical learning: data mining, inference, and prediction.

[16]

Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013).

[17]

Jogendra Nath Kundu, Suvaansh Bhambri, Akshay R Kulkarni, Hiran Sarkar, Varun Jampani, et al. 2022. Subsidiary prototype alignment for universal domain adaptation. NeurIPS, Vol. 35 (2022), 29649--29662.

[18]

Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht. 2019. Sliced wasserstein discrepancy for unsupervised domain adaptation. In CVPR. 10285--10295.

[19]

Shuang Li, Mixue Xie, Kaixiong Gong, Chi Harold Liu, Yulin Wang, and Wei Li. 2021. Transferable semantic augmentation for domain adaptation. In CVPR. 11516--11525.

[20]

Yicong Li, Xiang Wang, Junbin Xiao, Wei Ji, and Tat-Seng Chua. 2022. Invariant grounding for video question answering. In CVPR. 2928--2937.

[21]

Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. 2019. Transferable adversarial training: A general approach to adapting deep classifiers. In ICML. 4013--4022.

[22]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In ICML. PMLR, 97--105.

[23]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. NeurIPS, Vol. 31 (2018).

[24]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017. Deep transfer learning with joint adaptation networks. In ICML. PMLR, 2208--2217.

[25]

You-Wei Luo, Chuan-Xian Ren, Pengfei Ge, Ke-Kun Huang, and Yu-Feng Yu. 2020. Unsupervised domain adaptation via discriminative manifold embedding and alignment. In AAAI, Vol. 34. 5029--5036.

[26]

Fangrui Lv, Jian Liang, Shuang Li, Bin Zang, Chi Harold Liu, Ziteng Wang, and Di Liu. 2022. Causality inspired representation learning for domain generalization. In CVPR. 8046--8056.

[27]

Lei Meng, Long Chen, Xun Yang, Dacheng Tao, Hanwang Zhang, Chunyan Miao, and Tat-Seng Chua. 2019. Learning using privileged information for food recognition. In Proceedings of the 27th ACM international conference on multimedia. 557--565.

Digital Library

[28]

Raha Moraffah, Kai Shu, Adrienne Raglin, and Huan Liu. 2019. Deep causal representation learning for unsupervised domain adaptation. arXiv preprint arXiv:1910.12417 (2019).

[29]

Jaemin Na, Dongyoon Han, Hyung Jin Chang, and Wonjun Hwang. 2022. Contrastive vicinal space for unsupervised domain adaptation. In ECCV. Springer, 92--110.

[30]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. NeurIPS, Vol. 32 (2019).

[31]

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. 2019a. Moment matching for multi-source domain adaptation. In ICCV. 1406--1415.

[32]

Xingchao Peng, Zijun Huang, Ximeng Sun, and Kate Saenko. 2019b. Domain agnostic learning with disentangled representations. In ICML. 5102--5112.

[33]

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting Visual Category Models to New Domains. In ECCV, Kostas Daniilidis, Petros Maragos, and Nikos Paragios (Eds.). 213--226.

[34]

Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR. 3723--3732.

[35]

Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. 2021. Toward causal representation learning. Proc. IEEE, Vol. 109, 5 (2021), 612--634.

[36]

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In ICCV. 618--626.

[37]

Tao Sun, Cheng Lu, and Haibin Ling. 2022. Prior Knowledge Guided Unsupervised Domain Adaptation. In ECCV. Springer, 639--655.

[38]

Yi Tan, Yanbin Hao, Xiangnan He, Yinwei Wei, and Xun Yang. 2021. Selective dependency aggregation for action classification. In Proceedings of the 29th ACM International Conference on Multimedia. 592--601.

Digital Library

[39]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In CVPR. 7167--7176.

[40]

Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).

[41]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. JMLR (2008).

[42]

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. In ICCV. 5018--5027.

[43]

Haotian Wang, Wenjing Yang, Ji Wang, Ruxin Wang, Long Lan, and Mingyang Geng. 2020. Pairwise similarity regularization for adversarial domain adaptation. In ACM MM. 2409--2418.

[44]

Shanshan Wang and Lei Zhang. 2020. Self-adaptive re-weighted adversarial domain adaptation. In IJCAI. 3181--3187.

[45]

Shanshan Wang, Lei Zhang, Pichao Wang, MengZhu Wang, and Xingyi Zhang. 2023. BP-triplet net for unsupervised domain adaptation: A Bayesian perspective. Pattern Recognition, Vol. 133 (2023), 108993.

Digital Library

[46]

Guoqiang Wei, Cuiling Lan, Wenjun Zeng, and Zhibo Chen. 2021. Metaalign: Coordinating domain alignment and classification for unsupervised domain adaptation. In CVPR. 16643--16653.

[47]

Yuan Wu, Diana Inkpen, and Ahmed El-Roby. 2020. Dual mixup regularized learning for adversarial domain adaptation. In ECCV. 540--555.

[48]

Haifeng Xia and Zhengming Ding. 2020. Structure preserving generative cross-domain learning. In CVPR. 4364--4373.

[49]

Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. 2019. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In ICCV. 1426--1435.

[50]

Xun Yang, Jianfeng Dong, Yixin Cao, Xun Wang, Meng Wang, and Tat-Seng Chua. 2020. Tree-augmented cross-modal encoding for complex-query video retrieval. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval. 1339--1348.

Digital Library

[51]

Xun Yang, Fuli Feng, Wei Ji, Meng Wang, and Tat-Seng Chua. 2021. Deconfounded video moment retrieval with causal intervention. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1--10.

Digital Library

[52]

Xun Yang, Shanshan Wang, Jian Dong, Jianfeng Dong, Meng Wang, and Tat-Seng Chua. 2022. Video moment retrieval with cross-modal neural architecture search. IEEE Transactions on Image Processing, Vol. 31 (2022), 1204--1216.

[53]

Zhongqi Yue, Qianru Sun, Xian-Sheng Hua, and Hanwang Zhang. 2021. Transporting causal mechanisms for unsupervised domain adaptation. In ICCV. 8599--8608.

[54]

Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. 2019a. Bridging theory and algorithm for domain adaptation. In ICML. 7404--7413.

[55]

Yabin Zhang, Hui Tang, Kui Jia, and Mingkui Tan. 2019b. Domain-symmetric networks for adversarial domain adaptation. In CVPR. 5031--5040.

[56]

Zhizheng Zhang, Cuiling Lan, Wenjun Zeng, Quanzeng You, Zicheng Liu, Kecheng Zheng, and Zhibo Chen. 2021. Disentanglement-based Cross-Domain Feature Augmentation for Effective Unsupervised Domain Adaptive Person Re-identification. arXiv preprint arXiv:2103.13917 (2021).

[57]

Kecheng Zheng, Jiawei Liu, Wei Wu, Liang Li, and Zheng-jun Zha. 2021. Calibrated Feature Decomposition for Generalizable Person Re-Identification. arXiv preprint arXiv:2111.13945 (2021).

Cited By

Li HLiu YZhang YLi JYu Z(2025)Breaking the Paired Sample Barrier in Person Re-Identification: Leveraging Unpaired Samples for Domain GeneralizationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.354304020(2357-2371)Online publication date: 2025
https://doi.org/10.1109/TIFS.2025.3543040
Wu JFang Y(2025)Dual-view global and local category-attentive domain alignment for unsupervised conditional adversarial domain adaptationNeural Networks10.1016/j.neunet.2025.107129185(107129)Online publication date: May-2025
https://doi.org/10.1016/j.neunet.2025.107129
Xu JSong P(2024)Active Exploration of Modality Complementarity for MultimodalSentiment AnalysisProceedings of the 2024 2nd Asia Conference on Computer Vision, Image Processing and Pattern Recognition10.1145/3663976.3663986(1-7)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3663976.3663986
Show More Cited By

Index Terms

Disentangled Representation Learning with Causality for Unsupervised Domain Adaptation
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Multi-task learning
        Transfer learning

Recommendations

Cross-domain feature enhancement for unsupervised domain adaptation
Abstract
Till the present, the domain adaptation has been widely researched by transferring the knowledge from a labeled source domain to an unlabeled target domain. Adversarial adaptation methods have achieved great success, learning domain-invariant ...
Learning intra-domain style-invariant representation for unsupervised domain adaptation of semantic segmentation
Highlights
- A novel conception of learning intra-domain style-invariant representation for unsupervised domain adaptation of semantic segmentation.
Abstract
In this paper, we aim to tackle the problem of unsupervised domain adaptation (UDA) of semantic segmentation and improve the UDA performance with a novel conception of learning intra-domain style-invariant representation. Previous UDA ...
Unsupervised Domain Adaptation for Small Bowel Segmentation Using Disentangled Representation
Medical Image Computing and Computer Assisted Intervention – MICCAI 2021
Abstract
We present a novel unsupervised domain adaptation method for small bowel segmentation based on feature disentanglement. To make the domain adaptation more controllable, we disentangle intensity and non-intensity features within a unique two-stream ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Joint Funds of the National Natural Science Foundation of China
National Natural Science Fund of China

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
525
Total Downloads

Downloads (Last 12 months)321
Downloads (Last 6 weeks)21

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li HLiu YZhang YLi JYu Z(2025)Breaking the Paired Sample Barrier in Person Re-Identification: Leveraging Unpaired Samples for Domain GeneralizationIEEE Transactions on Information Forensics and Security10.1109/TIFS.2025.354304020(2357-2371)Online publication date: 2025
https://doi.org/10.1109/TIFS.2025.3543040
Wu JFang Y(2025)Dual-view global and local category-attentive domain alignment for unsupervised conditional adversarial domain adaptationNeural Networks10.1016/j.neunet.2025.107129185(107129)Online publication date: May-2025
https://doi.org/10.1016/j.neunet.2025.107129
Xu JSong P(2024)Active Exploration of Modality Complementarity for MultimodalSentiment AnalysisProceedings of the 2024 2nd Asia Conference on Computer Vision, Image Processing and Pattern Recognition10.1145/3663976.3663986(1-7)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3663976.3663986
Gao JMa XXu C(2024)Learning Transferable Conceptual Prototypes for Interpretable Unsupervised Domain AdaptationIEEE Transactions on Image Processing10.1109/TIP.2024.345962633(5284-5297)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3459626
Wang SHu MZhang X(2024)Self-Training with Contrastive Learning for Adversarial Domain Adaptation2024 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN60899.2024.10651413(1-8)Online publication date: 30-Jun-2024
https://doi.org/10.1109/IJCNN60899.2024.10651413
Li ZWan BMu CZhao RQiu SYan C(2024)AD-Aligning: Emulating Human-Like Generalization for Cognitive Domain Adaptation in Deep Learning2024 5th International Conference on Electronic Communication and Artificial Intelligence (ICECAI)10.1109/ICECAI62591.2024.10675013(794-798)Online publication date: 31-May-2024
https://doi.org/10.1109/ICECAI62591.2024.10675013
Wang SXiao QWang KYang XZhang X(2024)Mutual-weighted feature disentanglement for unsupervised domain adaptationMultimedia Systems10.1007/s00530-024-01477-830:6Online publication date: 7-Oct-2024
https://doi.org/10.1007/s00530-024-01477-8
Basak HYin Z(2024)Forget More to Learn More: Domain-Specific Feature Unlearning for Semi-supervised and Unsupervised Domain AdaptationComputer Vision – ECCV 202410.1007/978-3-031-72920-1_8(130-148)Online publication date: 1-Oct-2024
https://doi.org/10.1007/978-3-031-72920-1_8

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten