Mengzhu Wang
ABSTRACT
A classifier trained on one dataset rarely works on other datasets obtained under different conditions because of domain shifting. Such a problem is usually solved by domain adaptation methods. In this paper, we propose a novel unsupervised domain adaptation (UDA) method based on Interchangeable Batch Normalization (InterBN) to fuse different channels in deep neural networks for adversarial domain adaptation.Specifically, we first observe that the channels with small batch normalization scaling factor have less influence on the whole domain adaption, followed by a theoretical proof that the scaling factors for some channels will definitely come close to zero when imposing a sparsity regularization. Then, we replace the channels that have smaller scaling factors in the source domain with the mean of the channels which have larger scaling factors in the target domain or vice versa. Such a simple but effective channel fusion scheme can drastically increase the domain adaption ability.Extensive experimental results show that our InterBN significantly outperforms the current adversarial domain adaptation methods by a large margin on four visual benchmarks. In particular, InterBN achieves a remarkable improvement of 7.7% over the conditional adversarial adaptation networks (CDAN) on VisDA-2017 benchmark.
- Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016).Google Scholar
- Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning (2010), 151--175. Google ScholarDigital Library
- Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han. 2019. Domain-specific batch normalization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7354--7362.Google ScholarCross Ref
- Xinyang Chen, Sinan Wang, Mingsheng Long, and Jianmin Wang. 2019. Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. In International conference on machine learning. 1081--1090.Google Scholar
- Tai-Te Chu, Chia-Chun Chang, An-Zi Yen, Hen-Hsen Huang, and Hsin-Hsi Chen. 2020. Multimodal Retrieval through Relations between Subjects and Objects in Lifelog Images. In Proceedings of the Third Annual Workshop on Lifelog Search Challenge. 51--55. Google ScholarDigital Library
- Zhijie Deng, Yucen Luo, and Jun Zhu. 2019. Cluster alignment with a teacher for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 9944--9953.Google ScholarCross Ref
- Thomas Forgione, Axel Carlier, Géraldine Morin, Wei Tsang Ooi, Vincent Charvillat, and Praveen Kumar Yadav. 2018. An implementation of a dash client for browsing networked virtual environment. In Proceedings of the 26th ACM international conference on Multimedia. 1263--1264. Google ScholarDigital Library
- Yaroslav Ganin and Victor Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In International conference on machine learning. 1180--1189. Google ScholarDigital Library
- Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Francc ois Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. The journal of machine learning research (2016), 2096--2030. Google ScholarDigital Library
- Yixiao Ge, Dapeng Chen, and Hongsheng Li. 2020. Mutual mean-teaching: Pseudo label refinery for unsupervised domain adaptation on person re-identification. International Conference on Learning Representations (2020).Google Scholar
- Arthur Gretton, Karsten M Borgwardt, Malte J Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. The Journal of Machine Learning Research (2012), 723--773. Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google ScholarCross Ref
- Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell. 2018. Cycada: Cycle-consistent adversarial domain adaptation. In International conference on machine learning. 1989--1998.Google Scholar
- Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. 448--456. Google ScholarDigital Library
- Minsoo Kang and Bohyung Han. 2020. Operation-Aware Soft Channel Pruning using Differentiable Masks. In International Conference on Machine Learning. 5122--5131.Google Scholar
- Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE (1998), 2278--2324.Google Scholar
- Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, and Zi Huang. 2019 a. Cycle-consistent conditional adversarial transfer networks. In Proceedings of the 27th ACM International Conference on Multimedia. 747--755. Google ScholarDigital Library
- Jingjing Li, Erpeng Chen, Zhengming Ding, Lei Zhu, Ke Lu, and Heng Tao Shen. 2020. Maximum density divergence for domain adaptation. IEEE transactions on pattern analysis and machine intelligence (2020).Google ScholarCross Ref
- Shuang Li, Chi Harold Liu, Binhui Xie, Limin Su, Zhengming Ding, and Gao Huang. 2019 b. Joint adversarial domain adaptation. In Proceedings of the 27th ACM International Conference on Multimedia. 729--737. Google ScholarDigital Library
- Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou. 2016. Revisiting batch normalization for practical domain adaptation. International Conference on Learning Representations (2016).Google Scholar
- Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. 2017. Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE International Conference on Computer Vision. 2736--2744.Google ScholarCross Ref
- Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. 2015. Learning transferable features with deep adaptation networks. In International conference on machine learning. 97--105. Google ScholarDigital Library
- Mingsheng Long, Yue Cao, Jianmin Wang, and Philip S Yu. 2016a. Composite correlation quantization for efficient multimodal retrieval. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 579--588. Google ScholarDigital Library
- Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2017a. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems (2017). Google ScholarDigital Library
- Mingsheng Long, Jianmin Wang, Guiguang Ding, Jiaguang Sun, and Philip S Yu. 2014. Transfer joint matching for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1410--1417. Google ScholarDigital Library
- Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2016b. Unsupervised domain adaptation with residual transfer networks. International Conference on Neural Information Processing Systems (2016), 136--144. Google ScholarDigital Library
- Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan. 2017b. Deep transfer learning with joint adaptation networks. In International conference on machine learning. 2208--2217. Google ScholarDigital Library
- Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo. 2017. Autodial: Automatic domain alignment layers. In Proceedings of the IEEE International Conference on Computer Vision. 5067--5075.Google ScholarCross Ref
- Grégoire Montavon, Wojciech Samek, and Klaus-Robert Müller. 2018. Methods for interpreting and understanding deep neural networks. Digital Signal Processing (2018), 1--15.Google Scholar
- Hideki Nakasone, Mats Remberger, Lu Tian, Petter Brodin, Bita Sahaf, Fang Wu, Jonas Mattsson, Robert Lowsky, Robert Negrin, David B Miklos, et al. 2015. Risks and benefits of sex-mismatched hematopoietic cell transplantation differ according to conditioning strategy. Haematologica (2015), 1477.Google Scholar
- Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko. 2017. Visda: The visual domain adaptation challenge. arXiv preprint arXiv:1710.06924 (2017).Google Scholar
- Pedro O Pinheiro. 2018. Unsupervised domain adaptation with similarity learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8004--8013.Google ScholarCross Ref
- Murray Rosenblatt. 1956. A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences of the United States of America (1956), 43.Google ScholarCross Ref
- Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. 2010. Adapting visual category models to new domains. In European conference on computer vision. 213--226. Google ScholarDigital Library
- Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Maximum classifier discrepancy for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3723--3732.Google ScholarCross Ref
- Mark Schmidt, Glenn Fung, and Rmer Rosales. 2007. Fast optimization methods for l1 regularization: A comparative study and two new approaches. In European Conference on Machine Learning. 286--297. Google ScholarDigital Library
- Zhiqiang Tang, Yunhe Gao, Yi Zhu, Zhi Zhang, Mu Li, and Dimitris Metaxas. 2021. SelfNorm and CrossNorm for Out-of-Distribution Robustness. arXiv preprint arXiv:2102.02811 (2021).Google Scholar
- Marco Toldo, Umberto Michieli, and Pietro Zanuttigh. 2021. Unsupervised Domain Adaptation in Semantic Segmentation via Orthogonal and Clustered Embeddings. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1358--1368.Google ScholarCross Ref
- Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7167--7176.Google ScholarCross Ref
- Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014).Google Scholar
- Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016).Google Scholar
- Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research (2008).Google Scholar
- Vedran Vukotić, Christian Raymond, and Guillaume Gravier. 2016. Multimodal and crossmodal representation learning from textual and visual features with bidirectional deep neural networks for video hyperlinking. In Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion. 37--44. Google ScholarDigital Library
- Haotian Wang, Wenjing Yang, Ji Wang, Ruxin Wang, Long Lan, and Mingyang Geng. 2020 b. Pairwise Similarity Regularization for Adversarial Domain Adaptation. In Proceedings of the 28th ACM International Conference on Multimedia. 2409--2418. Google ScholarDigital Library
- Ximei Wang, Ying Jin, Mingsheng Long, Jianmin Wang, and Michael Jordan. 2019. Transferable normalization: Towards improving transferability of deep neural networks. (2019). Google ScholarDigital Library
- Yikai Wang, Wenbing Huang, Fuchun Sun, Tingyang Xu, Yu Rong, and Junzhou Huang. 2020 a. Deep multimodal fusion by channel exchanging. arXiv preprint arXiv:2011.05005 (2020).Google Scholar
- Zhonghao Wang, Mo Yu, Yunchao Wei, Rogerio Feris, Jinjun Xiong, Wen-mei Hwu, Thomas S Huang, and Honghui Shi. 2020 c. Differential treatment for stuff and things: A simple unsupervised domain adaptation method for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12635--12644.Google ScholarCross Ref
- Yuxin Wu and Kaiming He. 2018. Group normalization. In Proceedings of the European conference on computer vision (ECCV). 3--19.Google ScholarDigital Library
- Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin. 2019. Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 1426--1435.Google ScholarCross Ref
- Xun Yang, Jianfeng Dong, Yixin Cao, Xun Wang, Meng Wang, and Tat-Seng Chua. 2020. Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval. In SIGIR. 1339--1348. Google ScholarDigital Library
- Xun Yang, Fuli Feng, Wei Ji, Meng Wang, and Tat-Seng Chua. 2021. Deconfounded Video Moment Retrieval with Causal Intervention. In SIGIR. Google ScholarDigital Library
- Xun Yang, Xiangnan He, Xiang Wang, Yunshan Ma, Fuli Feng, Meng Wang, and Tat-Seng Chua. 2019. Interpretable fashion matching with rich attributes. In SIGIR. 775--784. Google ScholarDigital Library
- Xun Yang, Meng Wang, Richang Hong, Qi Tian, and Yong Rui. 2017. Enhancing person re-identification in a self-trained subspace. TOMM (2017), 1--23. Google ScholarDigital Library
- Xun Yang, Peicheng Zhou, and Meng Wang. 2018. Person reidentification via structural deep metric learning. IEEE transactions on neural networks and learning systems, Vol. 30, 10 (2018), 2987--2998.Google Scholar
- Werner Zellinger, Thomas Grubinger, Edwin Lughofer, Thomas Natschl"ager, and Susanne Saminger-Platz. 2017. Central moment discrepancy (cmd) for domain-invariant representation learning. International Conference on Learning Representations (2017).Google Scholar
- Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. 2018. Collaborative and adversarial network for unsupervised domain adaptation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3801--3809.Google ScholarCross Ref
- Yongchun Zhu, Fuzhen Zhuang, Jindong Wang, Guolin Ke, Jingwu Chen, Jiang Bian, Hui Xiong, and Qing He. 2020. Deep subdomain adaptation network for image classification. IEEE transactions on neural networks and learning systems (2020).Google ScholarCross Ref
- Yang Zou, Zhiding Yu, BVK Kumar, and Jinsong Wang. 2018. Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In Proceedings of the European conference on computer vision (ECCV). 289--305.Google ScholarDigital Library
Index Terms
- InterBN: Channel Fusion for Adversarial Unsupervised Domain Adaptation
Recommendations
CLDA: an adversarial unsupervised domain adaptation method with classifier-level adaptation
AbstractDomain adaptation is an active and important research field in transfer learning. Unsupervised domain adaptation, which is better in line with real-world scenarios than supervised and semi-supervised domain adaptation, has attracted much attention ...
Semi-supervised adversarial discriminative domain adaptation
AbstractDomain adaptation is a potential method to train a powerful deep neural network across various datasets. More precisely, domain adaptation methods train the model on training data and test that model on a completely separate dataset. The ...
Unsupervised domain adaptation with adversarial distribution adaptation network
AbstractAdversarial domain adaptation is a powerful approach to transfer the knowledge of the label-rich source domain to the label-scarce target domain by mitigating domain shifts across distributions. Existing domain adaptation methods align either the ...
Comments