research-article

Collaborative Learning of Diverse Experts for Source-free Universal Domain Adaptation

Authors:

Andy J. MaAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 2054 - 2065

https://doi.org/10.1145/3581783.3612211

Published: 27 October 2023 Publication History

Abstract

Source-free universal domain adaptation (SFUniDA) is a challenging yet practical problem that adapts the source model to the target domain in the presence of distribution and category shifts without accessing source domain data. Most existing methods are developed based on a single-expert target model for both known- and unknown-class data training, such that the known- and unknown-class data in the target domain may not be separated well from each other. To address this issue, we propose a novel Cobllaborative Learning of Diverse Experts (CoDE) method for SFUniDA. In our method, unknown-class compatible source model training is designed to reserve space for the potential target unknown-class data. Two diverse experts are learned to better recognize the target known- and unknown-class data respectively by the specialized entropy discrimination. We improve the transferability of both experts by collaboratively correcting the possible misclassification errors with consistency and diversity learning. The final prediction with high confidence is obtained by gating the diverse experts based on soft neighbor density. Extensive experiments on four publicly available benchmarks demonstrate the superiority of our method compared to the state of the art.

References

[1]

Silvia Bucci, Mohammad Reza Loghmani, and Tatiana Tommasi. 2020. On the Effectiveness of Image Rotation for Open Set Domain Adaptation. In European Conference on Computer Vision. 422--438.

[2]

Zhangjie Cao, Kaichao You, Mingsheng Long, Jianmin Wang, and Qiang Yang. 2019. Learning to Transfer Examples for Partial Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 2985--2994.

[3]

Wanxing Chang, Ye Shi, Hoang Duong Tuan, and Jingya Wang. 2022. Unified Optimal Transport Framework for Universal Domain Adaptation. In Advances in Neural Information Processing Systems.

[4]

Lin Chen, Huaian Chen, ZhixiangWei, Xin Jin, Xiao Tan, Yi Jin, and Enhong Chen. 2022. Reusing the Task-specific Classifier as a Discriminator: Discriminator-free Adversarial Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 7171--7180.

[5]

Liang Chen, Qianjin Du, Yihang Lou, Jianzhong He, Tao Bai, and Minghua Deng. 2022. Mutual Nearest Neighbor Contrast and Hybrid Prototype Self-Training for Universal Domain Adaptation. In the Proceedings of the AAAI Conference on Artificial Intelligence. 6248--6257.

[6]

Liang Chen, Yihang Lou, Jianzhong He, Tao Bai, and Minghua Deng. 2022. Evidential Neighborhood Contrastive Learning for Universal Domain Adaptation. In the Proceedings of the AAAI Conference on Artificial Intelligence. 6258--6267.

[7]

Liang Chen, Yihang Lou, Jianzhong He, Tao Bai, and Minghua Deng. 2022. Geometric Anchor Correspondence Mining with Uncertainty Modeling for Universal Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 16113--16122.

[8]

Zenggui Chen and Zhouhui Lian. 2022. Semi-supervised Semantic Segmentation via Prototypical Contrastive Learning. In ACM International Conference on Multimedia. 6696--6705.

Digital Library

[9]

Lechao Cheng, Chaowei Fang, Dingwen Zhang, Guanbin Li, and Gang Huang. 2022. Compound Batch Normalization for Long-tailed Image Classification. In ACM International Conference on Multimedia. 1925--1934.

[10]

Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. StableMoE: Stable Routing Strategy for Mixture of Experts. In Proceedings of the Association for Computational Linguistics. 7085--7095.

[11]

Bo Fu, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. 2020. Learning to detect open classes for universal domain adaptation. In European Conference on Computer Vision. 567--583.

Digital Library

[12]

Yuan Gao, Peipeng Chen, Yue Gao, Jinpeng Wang, Youngsun Pan, and Andy J. Ma. 2022. Hierarchical feature disentangling network for universal domain adaptation. Pattern Recognition 127 (2022), 108616.

Digital Library

[13]

Jiang Guo, Darsh J. Shah, and Regina Barzilay. 2018. Multi-Source Domain Adaptation with Mixture of Experts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Brussels. 4694--4703.

[14]

Hussein Hazimeh, Zhe Zhao, Aakanksha Chowdhery, Maheswaran Sathiamoorthy, Yihua Chen, Rahul Mazumder, Lichan Hong, and Ed H. Chi. 2021. DSelect-k: Differentiable Selection in the Mixture of Experts with Applications to Multi-Task Learning. In Advances in Neural Information Processing Systems. 29335--29347.

[15]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770--778.

[16]

Yunzhong Hou and Liang Zheng. 2021. Visualizing adapted knowledge in domain transfer. In IEEE Conference on Computer Vision and Pattern Recognition. 13824--13833.

[17]

Sungsu Hur, Inkyu Shin, Kwanyong Park, Sanghyun Woo, and In So Kweon. 2023. Learning Classifiers of Prototypes and Reciprocal Points for Universal Domain Adaptation. In IEEE Winter Conference on Applications of Computer Vision,. 531--540.

[18]

Taotao Jing, Hongfu Liu, and Zhengming Ding. 2021. Towards Novel Target Discovery Through Open-Set Domain Adaptation. In IEEE International Conference on Computer Vision. 9302--9311.

[19]

Jogendra Nath Kundu, Suvaansh Bhambri, Akshay Kulkarni, Hiran Sarkar, Varun Jampani, and R. Venkatesh Babu. 2022. Subsidiary Prototype Alignment for Universal Domain Adaptation. In Advances in Neural Information Processing Systems.

[20]

Jogendra Nath Kundu, Suvaansh Bhambri, Akshay R. Kulkarni, Hiran Sarkar, Varun Jampani, and R. Venkatesh Babu. 2022. Concurrent Subsidiary Supervision for Unsupervised Source-Free Domain Adaptation. In European Conference on Computer Vision. 177--194.

[21]

Jogendra Nath Kundu, Akshay R. Kulkarni, Suvaansh Bhambri, Deepesh Mehta, Shreyas Anand Kulkarni, Varun Jampani, and Venkatesh Babu Radhakrishnan. 2022. Balancing Discriminability and Transferability for Source-Free Domain Adaptation. In International Conference on Machine Learning. 11710--11728.

[22]

Jogendra Nath Kundu, Naveen Venkat, Rahul M. V., and R. Venkatesh Babu. 2020. Universal Source-Free Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 4543--4552.

[23]

Guangrui Li, Guoliang Kang, Yi Zhu, Yunchao Wei, and Yi Yang. 2021. Domain Consensus Clustering for Universal Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 9757--9766.

[24]

Rui Li, Qianfen Jiao,Wenming Cao, Hau-SanWong, and SiWu. 2020. Model adaptation: Unsupervised domain adaptation without source data. In IEEE Conference on Computer Vision and Pattern Recognition. 9638--9647.

[25]

Xinhao Li, Jingjing Li, Zhekai Du, Lei Zhu, andWen Li. 2022. Interpretable Open-Set Domain Adaptation via Angular Margin Separation. In European Conference on Computer Vision. 1--18.

Digital Library

[26]

Jian Liang, Dapeng Hu, and Jiashi Feng. 2020. Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation. In International Conference on Machine Learning. 6028--6039.

[27]

Jian Liang, Dapeng Hu, Jiashi Feng, and Ran He. 2021. UMAD: Universal Model Adaptation under Domain and Category Shift. https://arxiv.org/abs/2112.08553 (2021).

[28]

Kun-Yu Lin, Jiaming Zhou, Yukun Qiu, and Wei-Shi Zheng. 2022. Adversarial Partial Domain Adaptation by Cycle Inconsistency. In European Conference on Computer Vision. 530--548.

Digital Library

[29]

Hong Liu, Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Qiang Yang. 2019. Separate to Adapt: Open Set Domain Adaptation via Progressive Separation. In IEEE Conference on Computer Vision and Pattern Recognition. 2927--2936.

[30]

Muhammad Jehanzeb Mirza, Jakub Micorek, Horst Possegger, and Horst Bischof. 2022. The Norm Must Go On: Dynamic Unsupervised Domain Adaptation by Normalization. In IEEE Conference on Computer Vision and Pattern Recognition. 14745--14755.

[31]

Gaurav Kumar Nayak, Konda Reddy Mopuri, Saksham Jain, and Anirban Chakraborty. 2022. Mining Data Impressions From Deep Models as Substitute for the Unavailable Training Data. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2022), 8465--8481.

[32]

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. 2019. Moment Matching for Multi-Source Domain Adaptation. In IEEE International Conference on Computer Vision. 1406--1415.

[33]

Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. 2017. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924 (2017).

[34]

Zhen Qiu, Yifan Zhang, Hongbin Lin, Shuaicheng Niu, Yanxia Liu, Qing Du, and Mingkui Tan. 2021. Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence,. 2921--2927.

[35]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015. Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115, 3 (2015), 211--252.

Digital Library

[36]

Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In European Conference on Computer Vision. 213--226.

[37]

Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko. 2020. Universal Domain Adaptation through Self Supervision. In Advances in Neural Information Processing Systems. 16282--16292.

[38]

Kuniaki Saito, Donghyun Kim, Piotr Teterwak, Stan Sclaroff, Trevor Darrell, and Kate Saenko. 2021. Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density. In IEEE International Conference on Computer Vision. 9164--9173.

[39]

Kuniaki Saito and Kate Saenko. 2021. OVANet: One-vs-All Network for Universal Domain Adaptation. In IEEE International Conference on Computer Vision. 9000--9009.

[40]

Kuniaki Saito, Shohei Yamamoto, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Open set domain adaptation by backpropagation. In European Conference on Computer Vision. 153--168.

Digital Library

[41]

Tao Sun, Cheng Lu, Tianshuo Zhang, and Haibin Ling. 2022. Safe Self-Refinement for Transformer-based Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 7181--7190.

[42]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).

[43]

Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 5018--5027.

[44]

Zhenyu Wu, Lin Wang, Wei Wang, Tengfei Shi, Chenglizhao Chen, Aimin Hao, and Shuo Li. 2022. Synthetic Data Supervised Salient Object Detection. In ACM International Conference on Multimedia. 5557--5565.

[45]

Baoyao Yang, Andy Jinhua Ma, and Pong C. Yuen. 2022. Revealing Task-Relevant Model Memorization for Source-Protected Unsupervised Domain Adaptation. IEEE Transactions on Information Forensics and Security 17 (2022), 716--731.

Digital Library

[46]

Baoyao Yang, Hao-Wei Yeh, Tatsuya Harada, and Pong C. Yuen. 2021. Model-Induced Generalization Error Bound for Information-Theoretic Representation Learning in Source-Data-Free Unsupervised Domain Adaptation. IEEE Transactions on Image Processing 31 (2021), 419--432.

[47]

Shiqi Yang, Joost van de Weijer, Luis Herranz, Shangling Jui, et al. 2021. Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation. Advances in Neural Information Processing Systems (2021), 29393--29405.

[48]

Shiqi Yang, YaxingWang, KaiWang, Shangling Jui, and Joost van deWeijer. 2022. One Ring to Bring Them All: Towards Open-Set Recognition under Domain Shift. https://arxiv.org/abs/2206.03600v2 (2022).

[49]

Kaichao You, Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. 2019. Universal Domain Adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. 2720--2729.

[50]

Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. 2022. Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognition. In Advances in Neural Information Processing Systems. 34077--34090.

[51]

Lihua Zhou, Mao Ye, Xiatian Zhu, Shuaifeng Li, and Yiguang Liu. 2022. Class Discriminative Adversarial Learning for Unsupervised Domain Adaptation. In ACM International Conference on Multimedia. 4318--4326.

[52]

Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Jianfeng Gao, and Tuo Zhao. 2022. Taming Sparsely Activated Transformer with Stochastic Experts. In International Conference on Learning Representations.

Cited By

Lu YZhang MMa AXie XLai J(2024)Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00614(6420-6429)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00614
Liu XZhou Y(2024)COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain AdaptationComputer Vision – ACCV 202410.1007/978-981-96-0963-5_20(337-353)Online publication date: 8-Dec-2024
https://doi.org/10.1007/978-981-96-0963-5_20

Index Terms

Collaborative Learning of Diverse Experts for Source-free Universal Domain Adaptation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition

Recommendations

COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain Adaptation
Computer Vision – ACCV 2024
Abstract
Universal domain adaptation (UniDA) aims to address domain and category shifts across data sources. Recently, due to more stringent data restrictions, researchers have introduced source-free UniDA (SF-UniDA). SF-UniDA methods eliminate the need ...
Universal multi-Source domain adaptation for image classification
Highlights
- The hypothesis of universal domain adaptation is violated in the multi-source setting.
Abstract
Unsupervised domain adaptation (DA) enables intelligent models to learn transferable knowledge from a labeled source domain and adapt to a similar but unlabeled target domain. Studies showed that knowledge could be transferred from one ...
Domain consensual contrastive learning for few-shot universal domain adaptation
Abstract
Traditional unsupervised domain adaptation (UDA) aims to transfer the learned knowledge from a fully labeled source domain to another unlabeled target domain on the same label set. The strong assumptions of full annotations on the source domain ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Science and Technology Program of Guangzhou

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
288
Total Downloads

Downloads (Last 12 months)100
Downloads (Last 6 weeks)11

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lu YZhang MMa AXie XLai J(2024)Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00614(6420-6429)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00614
Liu XZhou Y(2024)COCA: Classifier-Oriented Calibration via Textual Prototype for Source-Free Universal Domain AdaptationComputer Vision – ACCV 202410.1007/978-981-96-0963-5_20(337-353)Online publication date: 8-Dec-2024
https://doi.org/10.1007/978-981-96-0963-5_20

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten