Skip to main content
SpringerLink
Log in

Domain-specific feature elimination: multi-source domain adaptation for image classification

  • Research Article
  • Published:
Frontiers of Computer Science Aims and scope Submit manuscript

Abstract

Multi-source domain adaptation utilizes multiple source domains to learn the knowledge and transfers it to an unlabeled target domain. To address the problem, most of the existing methods aim to minimize the domain shift by auxiliary distribution alignment objectives, which reduces the effect of domain-specific features. However, without explicitly modeling the domain-specific features, it is not easy to guarantee that the domain-invariant representation extracted from input domains contains domain-specific information as few as possible. In this work, we present a different perspective on MSDA, which employs the idea of feature elimination to reduce the influence of domain-specific features. We design two different ways to extract domain-specific features and total features and construct the domain-invariant representations by eliminating the domain-specific features from total features. The experimental results on different domain adaptation datasets demonstrate the effectiveness of our method and the generalization ability of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016, 770–778

  2. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, Berg A C. SSD: single shot MultiBox detector. In: Proceedings of the 14th European Conference on Computer Vision. 2016, 21–37

  3. He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 2980–2988

  4. Fang T, Lu N, Niu G, Sugiyama M. Rethinking importance weighting for deep learning under distribution shift. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 11996–12007

  5. Sun Y P, Zhang M L. Compositional metric learning for multi-label classification. Frontiers of Computer Science, 2021, 15(5): 155320

    Article  Google Scholar 

  6. Li Y F, Liang D M. Safe semi-supervised learning: a brief introduction. Frontiers of Computer Science, 2019, 13(4): 669

    Article  Google Scholar 

  7. Fan J, Wang Y, Guan H, Song C, Zhang Z. Toward few-shot domain adaptation with perturbation-invariant representation and transferable prototypes. Frontiers of Computer Science, 2022, 16(3): 163347

    Article  Google Scholar 

  8. Gao Z, Guo L, Guan W, Liu A A, Ren T, Chen S. A pairwise attentive adversarial spatiotemporal network for cross-domain few-shot action recognition-R2. IEEE Transactions on Image Processing, 2021, 30: 767

    Article  Google Scholar 

  9. Quan R, Wu Y, Yu X, Yang Y. Progressive transfer learning for face anti-spoofing. IEEE Transactions on Image Processing, 2021, 30: 3946

    Article  Google Scholar 

  10. Yang Y, Zhuang Y, Pan Y. Multiple knowledge representation for big data artificial intelligence: framework, applications, and case studies. Frontiers of Information Technology & Electronic Engineering, 2021, 22(12): 1551

    Article  Google Scholar 

  11. Zhao S, Li B, Reed C, Xu P, Keutzer K. Multi-source domain adaptation in the deep learning era: a systematic survey. 2020, arXiv preprint arXiv: 2002.12169

  12. Mansour Y, Mohri M, Rostamizadeh A. Multiple source adaptation and the rényi divergence. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 2009, 367–374

  13. Cui S, Jin X, Wang S, He Y, Huang Q. Heuristic domain adaptation. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 7571–7583

  14. Long M, Cao Y, Wang J, Jordan M I. Learning transferable features with deep adaptation networks. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 97–105

  15. Sun B, Feng J, Saenko K. Correlation alignment for unsupervised domain adaptation. In: Csurka G, ed. Domain Adaptation in Computer Vision Applications. Cham: Springer, 2017, 153–171

    Chapter  Google Scholar 

  16. Li S, Xie B, Wu J, Zhao Y, Liu C H, Ding Z. Simultaneous semantic alignment network for heterogeneous domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020, 3866–3874

  17. Jing T, Xia H, Ding Z. Adaptively-accumulated knowledge transfer for partial domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020, 1606–1614

  18. Kang G, Jiang L, Yang Y, Hauptmann A G. Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 4888–4897

  19. Hoffman J, Tzeng E, Park T, Zhu J Y, Isola P, Saenko K, Efros A, Darrell T. CyCADA: cycle-consistent adversarial domain adaptation. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1989–1998

  20. Sankaranarayanan S, Balaji Y, Castillo C D, Chellappa R. Generate to adapt: aligning domains using generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 8503–8512

  21. Luo Y, Huang Z, Wang Z, Zhang Z, Baktashmotlagh M. Adversarial bipartite graph learning for video domain adaptation. In: Proceedings of the 28th ACM International Conference on Multimedia. 2020, 19–27

  22. Hu H, Yang Y, Yin Y, Wu J. Metric learning for domain adversarial network. Frontiers of Computer Science, 2022, 16(5): 165341

    Article  Google Scholar 

  23. Wang Y, Wang C, Xue H, Chen S. Self-corrected unsupervised domain adaptation. Frontiers of Computer Science, 2022, 16(5): 165323

    Article  Google Scholar 

  24. Tian Q, Sun H, Peng S, Ma T. Self-adaptive label filtering learning for unsupervised domain adaptation. Frontiers of Computer Science, 2023, 17(1): 171308

    Article  Google Scholar 

  25. Shao H. Query by diverse committee in transfer active learning. Frontiers of Computer Science, 2019, 13(2): 280

    Article  Google Scholar 

  26. Hoffman J, Mohri M, Zhang N. Algorithms and theory for multiple-source adaptation. In: Proceedings of the 32nd International Conference on Information Processing Systems. 2018, 8256–8266

  27. Zhao H, Zhang S, Wu G, Costeira J P, Moura J M F, Gordon G J. Adversarial multiple source domain adaptation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems. 2018, 8568–8579

  28. Xu R, Chen Z, Zuo W, Yan J, Lin L. Deep cocktail network: multi-source unsupervised domain adaptation with category shift. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 3964–3973

  29. Zhao S, Wang G, Zhang S, Gu Y, Li Y, Song Z, Xu P, Hu R, Chai H, Keutzer K. Multi-source distilling domain adaptation. In: Proceedings of the 34th the AAAI Conference on Artificial Intelligence. 2020, 12975–12983

  30. Wang H, Xu M, Ni B, Zhang W. Learning to combine: knowledge aggregation for multi-source domain adaptation. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 727–744

  31. Cui S, Wang S, Zhuo J, Su C, Huang Q, Tian Q. Gradually vanishing bridge for adversarial domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020, 12452–12461

  32. Saito K, Watanabe K, Ushiku Y, Harada T. Maximum classifier discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2018, 3723–3732

  33. Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B. Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019, 1406–1415

  34. Zhu Y, Zhuang F, Wang D. Aligning domain-specific distribution and classifier for cross-domain classification from multiple sources. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 2019, 5989–5996

  35. Yang L, Balaji Y, Lim S N, Shrivastava A. Curriculum manager for source selection in multi-source domain adaptation. In: Proceedings of the 16th European Conference on Computer Vision. 2020, 608–624

  36. Venkat N, Kundu J N, Singh D K, Revanur A, Babu R V. Your classifier can secretly suffice multi-source domain adaptation. In: Proceedings of the 34th International Conference on Neural Information Processing Systems. 2020, 390

  37. Sun B, Saenko K. Deep CORAL: correlation alignment for deep domain adaptation. In: Proceedings of the European Conference on Computer Vision. 2016, 443–450

  38. Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 1180–1189

  39. Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 2962–2971

  40. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S. Deep hashing network for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017, 5385–5394

  41. Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. In: Proceedings of the 11th European Conference on Computer Vision. 2010, 213–226

  42. van der Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9(86): 2579

    MATH  Google Scholar 

  43. Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 618–626

  44. Salzmann M, Ek C H, Urtasun R, Darrell T. Factorized orthogonal latent spaces. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. 2010, 701–708

  45. Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D. Domain separation networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 343–351

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (NSFC) (Grant Nos. 61876130 and 61932009).

Author information

Authors and Affiliations

  1. College of Intelligence and Computing, Tianjin University, Tianjin, 300350, China

    Kunhong Wu, Fan Jia & Yahong Han

  2. Tianjin Key Lab of Machine Learning, Tianjin University, Tianjin, 300350, China

    Yahong Han

Authors
  1. Kunhong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yahong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yahong Han.

Additional information

Kunhong Wu received the BS degree from Tianjin University, China in 2020. He is currently pursuing the MS degree with the College of Intelligence and Computing, Tianjin University, China. His research interests include computer vision and domain adaptation.

Fan Jia received the BS degree from Tianjin University, China in 2019. He also received the MS degree with the College of Intelligence and Computing, Tianjin University, China. He is interested in computer vision and adversarial machine learning.

Yahong Han received the PhD degree from Zhejiang University, China in 2012. He is currently a professor with the College of Intelligence and Computing, Tianjin University, China. From Nov. 2014 to Nov. 2015, he visited Prof. Bin Yu’s group at UC Berkeley, USA as a Visiting Scholar. His current research interests include multimedia analysis, computer vision, and machine learning.

Electronic supplementary material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, K., Jia, F. & Han, Y. Domain-specific feature elimination: multi-source domain adaptation for image classification. Front. Comput. Sci. 17, 174705 (2023). https://doi.org/10.1007/s11704-022-2146-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11704-022-2146-x

Keywords

Search

Navigation