research-article

Mixture-of-Experts Learner for Single Long-Tailed Domain Generalization

Authors:

Zhibin WangAuthors Info & Claims

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

Pages 290 - 299

https://doi.org/10.1145/3581783.3611871

Published: 27 October 2023 Publication History

Abstract

Domain generalization (DG) refers to the task of training a model on multiple source domains and test it on a different target domain with different distribution. In this paper, we address a more challenging and realistic scenario known as Single Long-Tailed Domain Generalization, where only one source domain is available and the minority class in this domain has an abundance of instances in other domains. To tackle this task, we propose a novel approach called Mixture-of-Experts Learner for Single Long-Tailed Domain Generalization (MoEL), which comprises two key strategies. The first strategy is a simple yet effective data augmentation technique that leverages saliency maps to identify important regions on the original images and preserves these regions during augmentation. The second strategy is a new skill-diverse expert learning approach that trains multiple experts from a single long-tailed source domain and leverages mutual learning to aggregate their learned knowledge for the unknown target domain. We evaluate our method on various benchmark datasets, including Digits-DG, CIFAR-10-C, PACS, and DomainNet, and demonstrate its superior performance compared to previous single domain generalization methods. Additionally, the ablation study is also conducted to illustrate the inner workings of our approach.

References

[1]

Mahsa Baktashmotlagh, Mehrtash Tafazzoli Harandi, Brian C. Lovell, and Mathieu Salzmann. 2013. Unsupervised Domain Adaptation by Domain Invariant Projection. In ICCV.

[2]

Fabio Maria Carlucci, Antonio D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. 2019. Domain Generalization by Solving Jigsaw Puzzles. In CVPR.

[3]

Gabriela Csurka. 2017. A Comprehensive Survey on Domain Adaptation for Visual Applications. In Domain Adaptation in Computer Vision Applications.

[4]

Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. 2020. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 702--703.

[5]

Ilke Cugu, Massimiliano Mancini, Yanbei Chen, and Zeynep Akata. 2022. Attention Consistency on Visual Corruptions for Single-Source Domain Generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4165--4174.

[6]

John Denker, W Gardner, Hans Graf, Donnie Henderson, R Howard, W Hubbard, Lawrence D Jackel, Henry Baird, and Isabelle Guyon. 1988. Neural network recognizer for hand-written zip code digits. Advances in neural information processing systems, Vol. 1 (1988).

[7]

Terrance DeVries and Graham W Taylor. 2017. Improved regularization of convolutional neural networks with cutout. arXiv preprint arXiv:1708.04552 (2017).

[8]

Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. 2019. Domain Generalization via Model-Agnostic Learning of Semantic Features. In NeurIPS.

[9]

Ying-Jun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees G. M. Snoek, and Ling Shao. 2020. Learning to Learn with Variational Information Bottleneck for Domain Generalization. In ECCV.

[10]

Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. PMLR, 1180--1189.

[11]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor S. Lempitsky. 2016. Domain-Adversarial Training of Neural Networks. J. Mach. Learn. Res. (2016).

[12]

Muhammad Ghifary, W. Bastiaan Kleijn, Mengjie Zhang, and David Balduzzi. 2015. Domain Generalization for Object Recognition with Multi-task Autoencoders. In ICCV.

[13]

Chengyue Gong, Dilin Wang, Meng Li, Vikas Chandra, and Qiang Liu. 2021. Keepaugment: A simple information-preserving data augmentation approach. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 1055--1064.

[14]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[15]

Dan Hendrycks and Thomas Dietterich. 2019. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261 (2019).

[16]

Dan Hendrycks, Norman Mu, Ekin D Cubuk, Barret Zoph, Justin Gilmer, and Balaji Lakshminarayanan. 2019. Augmix: A simple data processing method to improve robustness and uncertainty. arXiv preprint arXiv:1912.02781 (2019).

[17]

Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. 2020. Self-challenging improves cross-domain generalization. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II 16. Springer, 124--140.

Digital Library

[18]

Hal Daumé III, Abhishek Kumar, and Avishek Saha. 2010. Co-regularization Based Semi-supervised Domain Adaptation. In NeurIPS.

[19]

Juwon Kang, Sohyun Lee, Namyup Kim, and Suha Kwak. 2022. Style neophile: Constantly seeking novel styles for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7130--7140.

[20]

Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE, Vol. 86, 11 (1998), 2278--2324.

[21]

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2017a. Deeper, Broader and Artier Domain Generalization. In ICCV.

[22]

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. 2018. Learning to Generalize: Meta-Learning for Domain Generalization. In AAAI.

[23]

Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. 2017b. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision. 5542--5550.

[24]

Jing Li, Qiu-Feng Wang, Kaizhu Huang, Xi Yang, Rui Zhang, and John Y Goulermas. 2023. Towards Better Long-Tailed Oracle Character Recognition with Adversarial Data Augmentation. Pattern Recognition (2023), 109534.

[25]

Wei-Wei Lin, Man-Wai Mak, Longxin Li, and Jen-Tzung Chien. 2018. Reducing domain mismatch by maximum mean discrepancy based autoencoders. In Odyssey. 162--167.

[26]

Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan. 2015. Learning Transferable Features with Deep Adaptation Networks. In ICML.

Digital Library

[27]

Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I. Jordan. 2018. Conditional Adversarial Domain Adaptation. In NeurIPS.

[28]

Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan. 2017. Deep Transfer Learning with Joint Adaptation Networks. In ICML.

Digital Library

[29]

Yadan Luo, Zijian Wang, Zi Huang, and Mahsa Baktashmotlagh. 2020. Progressive Graph Learning for Open-Set Domain Adaptation. In ICML.

[30]

Omae Manabu, Takehiko Fujioka, Naohisa Hashimoto, and Hiroshi Shimizu. 2006. The application of RTK-GPS and steer-by-wire technology to the automatic driving of vehicles and an evaluation of driver behavior. IATSS research, Vol. 30, 2 (2006), 29--38.

[31]

Saeid Motiian, Quinn Jones, Seyed Mehdi Iranmanesh, and Gianfranco Doretto. 2017a. Few-Shot Adversarial Domain Adaptation. In NeurIPS.

[32]

Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, and Gianfranco Doretto. 2017b. Unified Deep Supervised Domain Adaptation and Generalization. In ICCV.

[33]

Krikamol Muandet, David Balduzzi, and Bernhard Schö lkopf. 2013. Domain Generalization via Invariant Feature Representation. In ICML.

[34]

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. 2011. Reading digits in natural images with unsupervised feature learning. (2011).

[35]

Sinno Jialin Pan, Ivor W Tsang, James T Kwok, and Qiang Yang. 2010. Domain adaptation via transfer component analysis. IEEE transactions on neural networks, Vol. 22, 2 (2010), 199--210.

[36]

Sinno Jialin Pan and Qiang Yang. 2010. A Survey on Transfer Learning. IEEE Trans. Knowl. Data Eng. (2010).

Digital Library

[37]

Changhwa Park, Junho Yim, and Eunji Jun. 2023. Mutual Learning for Long-Tailed Recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2675--2684.

[38]

Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. 2019. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF international conference on computer vision. 1406--1415.

[39]

Foster Provost and Ron Kohavi. 1998. On applied research in machine learning. MACHINE LEARNING-BOSTON-, Vol. 30 (1998), 127--132.

Digital Library

[40]

Fengchun Qiao, Long Zhao, and Xi Peng. 2020a. Learning to Learn Single Domain Generalization. In CVPR.

[41]

Fengchun Qiao, Long Zhao, and Xi Peng. 2020b. Learning to learn single domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12556--12565.

[42]

Jiawei Ren, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, et al. 2020. Balanced meta-softmax for long-tailed visual recognition. Advances in neural information processing systems, Vol. 33 (2020), 4175--4186.

[43]

Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. 2019. Semi-Supervised Domain Adaptation via Minimax Entropy. In ICCV.

[44]

Swami Sankaranarayanan and Yogesh Balaji. 2023. Meta learning for domain generalization. In Meta-Learning with Medical Imaging and Health Informatics Applications. Elsevier, 75--86.

[45]

Swami Sankaranarayanan, Yogesh Balaji, Arpit Jain, Ser Nam Lim, and Rama Chellappa. 2018. Learning from synthetic data: Addressing domain shift for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3752--3761.

[46]

Shiv Shankar, Vihari Piratla, Soumen Chakrabarti, Siddhartha Chaudhuri, Preethi Jyothi, and Sunita Sarawagi. 2018. Generalizing Across Domains via Cross-Gradient Training. In ICLR.

[47]

Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034 (2013).

[48]

Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7167--7176.

[49]

Riccardo Volpi, Hongseok Namkoong, Ozan Sener, John C. Duchi, Vittorio Murino, and Silvio Savarese. 2018. Generalizing to Unseen Domains via Adversarial Data Augmentation. In NeurIPS.

[50]

Chaoqun Wan, Xu Shen, Yonggang Zhang, Zhiheng Yin, Xinmei Tian, Feng Gao, Jianqiang Huang, and Xian-Sheng Hua. 2022. Meta convolutional neural networks for single domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4682--4691.

[51]

Haohan Wang, Zexue He, Zachary C. Lipton, and Eric P. Xing. 2019. Learning Robust Representations by Projecting Superficial Statistics Out. In ICLR.

[52]

Mengzhu Wang, Shanshan Wang, Wei Wang, Li Shen, Xiang Zhang, Long Lan, and Zhigang Luo. 2023. Reducing bi-level feature redundancy for unsupervised domain adaptation. Pattern Recognit., Vol. 137 (2023), 109319.

Digital Library

[53]

Mengzhu Wang, Jianlong Yuan, Qi Qian, Zhibin Wang, and Hao Li. 2022a. Semantic Data Augmentation based Distance Metric Learning for Domain Generalization. In Proceedings of the 30th ACM International Conference on Multimedia. 3214--3223.

Digital Library

[54]

Mengzhu Wang, Jianlong Yuan, Qi Qian, Zhibin Wang, and Hao Li. 2022b. Semantic Data Augmentation based Distance Metric Learning for Domain Generalization. In ACM International Conference on Multimedia. 3214--3223.

Digital Library

[55]

Zijian Wang, Yadan Luo, Zi Huang, and Mahsa Baktashmotlagh. 2020. Prototype-Matching Graph Network for Heterogeneous Domain Adaptation. In MM.

[56]

Zijian Wang, Yadan Luo, Ruihong Qiu, Zi Huang, and Mahsa Baktashmotlagh. 2021. Learning to diversify for single domain generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 834--843.

[57]

Qinwei Xu, Ruipeng Zhang, Ya Zhang, Yanfeng Wang, and Qi Tian. 2021. A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14383--14392.

[58]

Xiang Xu, Xiong Zhou, Ragav Venkatesan, Gurumurthy Swaminathan, and Orchid Majumder. 2019. d-sne: Domain adaptation using stochastic neighborhood embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2497--2506.

[59]

Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, and Marc Niethammer. 2020. Robust and generalizable visual representation learning via random convolutions. arXiv preprint arXiv:2007.13003 (2020).

[60]

Yuzhe Yang, Hao Wang, and Dina Katabi. 2022. On multi-domain long-tailed recognition, imbalanced domain generalization and beyond. In Computer Vision--ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XX. Springer, 57--75.

[61]

Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. 2019. Cutmix: Regularization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision. 6023--6032.

[62]

Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. 2017. mixup: Beyond empirical risk minimization. arXiv preprint arXiv:1710.09412 (2017).

[63]

Yifan Zhang, Bryan Hooi, Lanqing Hong, and Jiashi Feng. 2022. Self-supervised aggregation of diverse experts for test-agnostic long-tailed recognition. Advances in Neural Information Processing Systems, Vol. 35 (2022), 34077--34090.

[64]

Yuchen Zhang, Tianle Liu, Mingsheng Long, and Michael Jordan. 2019. Bridging theory and algorithm for domain adaptation. In International conference on machine learning. PMLR, 7404--7413.

[65]

Ying Zhang, Tao Xiang, Timothy M Hospedales, and Huchuan Lu. 2018. Deep mutual learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4320--4328.

[66]

Long Zhao, Ting Liu, Xi Peng, and Dimitris Metaxas. 2020a. Maximum-entropy adversarial data augmentation for improved generalization and robustness. Advances in Neural Information Processing Systems, Vol. 33 (2020), 14435--14447.

[67]

Long Zhao, Ting Liu, Xi Peng, and Dimitris N. Metaxas. 2020b. Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness. In NeurIPS.

[68]

Kaiyang Zhou, Yongxin Yang, Timothy M. Hospedales, and Tao Xiang. 2020. Learning to Generate Novel Domains for Domain Generalization. In ECCV.

Cited By

Wang MWang SYang XYuan JZhang W(2024)Equity in Unsupervised Domain Adaptation by Nuclear Norm MaximizationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.334644434:7(5533-5545)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3346444
Xu HShi CFan WChen Z(2024)Improving diversity and discriminability based implicit contrastive learning for unsupervised domain adaptationApplied Intelligence10.1007/s10489-024-05351-y54:20(10007-10017)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10489-024-05351-y

Index Terms

Mixture-of-Experts Learner for Single Long-Tailed Domain Generalization
1. Networks
  1. Network architectures

Recommendations

Source domain prior-assisted segment anything model for single domain generalization in medical image segmentation
Abstract
Deep learning-based medical image segmentation models often suffer from performance degradation across domains due to domain discrepancies arising from data collected by various healthcare centers. Recent advancements, particularly the Segment ...
Highlights
- Improved Segment Anything Model (SAM) generalization for single domain generalization.
- Addressing medical image domain generalization challenges.
- Source domain prior-assisted medical image segmentation.
- Memory bank mechanism ...
Category-Stitch Learning for Union Domain Generalization
Domain generalization aims at generalizing the network trained on multiple domains to unknown but related domains. Under the assumption that different domains share the same classes, previous works can build relationships across domains. However, in ...
Domain generalization based on domain-specific adversarial learning
Abstract
Deep learning models often suffer from degraded performance when the distributions of the training and testing data differ (i.e., domain shift). Domain generalization (DG) techniques can help improve the generalization performance for unseen ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '23: Proceedings of the 31st ACM International Conference on Multimedia

October 2023

9913 pages

ISBN:9798400701085

DOI:10.1145/3581783

General Chairs:
Abdulmotaleb El Saddik
University of Ottawa, Canada & MBZUAI, UAE
,
Tao Mei
HiDream.ai, China
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chairs:
Marco Bertini
University of Florence, Italy
,
Diana Patricia Tobon Vallejo
Unversidad de Medellin, Colombia
,
Pradeep K. Atrey
University at Albany, State University of New York, USA
,
M. Shamim Hossain
M. Shamim Hossain (King Saud University, KSA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '23

Sponsor:

SIGMM

MM '23: The 31st ACM International Conference on Multimedia

October 29 - November 3, 2023

Ottawa ON, Canada

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
366
Total Downloads

Downloads (Last 12 months)242
Downloads (Last 6 weeks)23

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang MWang SYang XYuan JZhang W(2024)Equity in Unsupervised Domain Adaptation by Nuclear Norm MaximizationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.334644434:7(5533-5545)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3346444
Xu HShi CFan WChen Z(2024)Improving diversity and discriminability based implicit contrastive learning for unsupervised domain adaptationApplied Intelligence10.1007/s10489-024-05351-y54:20(10007-10017)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10489-024-05351-y

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten