Skip to main content
SpringerLink
Log in

Unsupervised double weighted domain adaptation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Domain adaptation can effectively transfer knowledge between domains with different distributions. Most existing methods use distribution alignment to mitigate the domain shift. But they typically align the marginal and conditional distributions with equal weights. This neglects the relative importance of different distribution alignments. In this paper, we propose a double weighted domain adaptation (DWDA) method, which employs new distribution alignment weighting and sample reweighting strategies. Specifically, the distribution alignment weighting strategy explores the relative importance of marginal and conditional distribution alignments, based on the maximum mean discrepancy; the sample reweighting strategy weights the source and target samples separately based on k-means clustering. The two strategies reinforce each other in the iterative optimization procedure, thus improving the overall performance. In addition, our method also considers the geometry structure preservation. The closed-form solution of the objective function is presented, and the computational complexity and convergence analysis are given. Experimental results demonstrate that DWDA outperforms state-of-the-art domain adaptation methods on several public datasets, and it also has good performance on an imbalanced dataset. Besides, DWDA is robust to a wide range of parameters. Moreover, the convergence curves show that DWDA generally converges rapidly within five iterations. We also evaluate the components of DWDA and show that each component is necessary. Finally, we compare the computational time with the methods of the recent three years to demonstrate the efficiency of DWDA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Aljundi R, Emonet R, Muselet D, Sebban M (2015) Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 56–63. https://doi.org/10.1109/CVPR.2015.7298600

  2. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MathSciNet  MATH  Google Scholar 

  3. Bruzzone L (2010) Marconcini M Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787. https://doi.org/10.1109/TPAMI.2009.57

    Article  Google Scholar 

  4. Cai D, He X, Han J (2007) Spectral regression: a unified approach for sparse subspace learning. In: 7th IEEE international conference on data mining, pp 73–82. https://doi.org/10.1109/ICDM.2007.89

  5. Cao Y, Long M, Wang J (2018) Unsupervised domain adaptation with distribution matching machines. In: AAAI conference on artificial intelligence, pp 2795–2802. http://ise.thss.tsinghua.edu.cn/~mlong/doc/distribution-matching-machines-aaai18.pdf

  6. Chung FRK (1997) Spectral graph theory. CBMS regional conference series in mathematics, vol 92. https://doi.org/10.1090/cbms/092

  7. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: a deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp 988–996. https://doi.org/10.1097/00003643-201406001-00333

  8. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: IEEE international conference on computer vision, pp 2960–2967. https://doi.org/10.1109/ICCV.2013.368

  9. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res (JMLR) 17:189–209. https://doi.org/10.1007/978-3-319-58347-1_10

    Article  MathSciNet  MATH  Google Scholar 

  10. Gedik E, Hung H (2016) Speaking status detection from body movements using transductive parameter transfer. In: ACM international joint conference on pervasive and ubiquitous, pp 69–72. https://doi.org/10.1145/2968219.2971444

  11. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 2066–2073. https://doi.org/10.1109/CVPR.2012.6247911

  12. Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: an unsupervised approach. In: International conference on computer vision, pp 999–1006. https://doi.org/10.1109/ICCV.2011.6126344

  13. He X, Yan S, Hu Y, Niyogi P, Zhang HJ (2005) Face recognition using laplacianfaces. IEEE Trans Pattern Anal Mach Intell 27(3):328–340. https://doi.org/10.1109/TPAMI.2005.55

    Article  Google Scholar 

  14. Hou CA, Tsai YHH, Yeh YR, Wang YCF (2016) Unsupervised domain adaptation with label and structural consistency. IEEE Trans Image Process 25:5552–5562. https://doi.org/10.1109/TIP.2016.2609820

    Article  MathSciNet  MATH  Google Scholar 

  15. Hou CA, Yeh YR, Wang YCF (2015) An unsupervised domain adaptation approach for cross-domain visual classification. In: 12th IEEE international conference on advanced video and signal based surveillance, pp 1–6. https://doi.org/10.1109/AVSS.2015.7301758

  16. Hsu TMH, Chen WY, Hou CA, Tsai YHH, Yeh YR, Wang YCF (2015) Unsupervised domain adaptation with imbalanced cross-domain data. In: IEEE international conference on computer vision, pp 4121–4129. https://doi.org/10.1109/ICCV.2015.469

  17. Kouw WM, Loog M (2018) An introduction to domain adaptation and transfer learning. https://doi.org/10.13140/RG.2.2.33906.56004

  18. Li J, Lu K, Huang Z, Zhu L, Shen HT (2019) Transfer independently together: a generalized framework for domain adaptation. IEEE Trans Cybern 49:2144–2155. https://doi.org/10.1109/TCYB.2018.2820174

    Article  Google Scholar 

  19. Li J, Wu Y, Lu K (2017) Structured domain adaptation. IEEE Trans Circuits Syst Video Technol 27:1700–1713. https://doi.org/10.1109/TCSVT.2016.2539541

    Article  Google Scholar 

  20. Li L, Zhan Z (2019) Semi-supervised domain adaptation by covariance matching. IEEE Trans Pattern Anal Mach Intell 41(11):2724–2739. https://doi.org/10.1109/TPAMI.2018.2866846

    Article  Google Scholar 

  21. Liu F, Zhang G, Lu J (2020) Heterogeneous domain adaptation: an unsupervised approach. IEEE Trans Neural Netw Learn Syst (TNNLS). https://doi.org/10.1109/TNNLS.2020.2973293

    Article  Google Scholar 

  22. Long M, Cao Y, Cao Z, Wang J, Jordan MI (2019) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell 41(12):3071–3085. https://doi.org/10.1109/TPAMI.2018.2868685

    Article  Google Scholar 

  23. Long M, Cao Y, Wang J, Jordan MI (2015) Learning transferable features with deep adaptation networks. In: The 32nd international conference on machine learning, pp 97–105. http://arxiv.org/abs/1502.02791

  24. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. In: Advances in neural information processing systems, pp 1640–1650. https://doi.org/10.1007/978-3-030-01237-3_9

  25. Long M, Wang J, Ding G, Pan SJ, Yu PS (2014) Adaptation regularization: a general framework for transfer learning. IEEE Trans Pattern Anal Mach Intell 26:1076–1089. https://doi.org/10.1109/TKDE.2013.111

    Article  Google Scholar 

  26. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: IEEE international conference on computer vision, pp 2200–2207. https://doi.org/10.1109/ICCV.2013.274

  27. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 1410–1417. https://doi.org/10.1109/CVPR.2014.183

  28. Long M, Wang J, Sun J, Yu PS (2015) Domain invariant transfer kernel learning. IEEE Trans Knowl Data Eng 27:1519–1532. https://doi.org/10.1109/TKDE.2014.2373376

    Article  Google Scholar 

  29. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210. https://doi.org/10.1109/TNN.2010.2091281

    Article  Google Scholar 

  30. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359. https://doi.org/10.1109/TKDE.2009.191

    Article  Google Scholar 

  31. Saenko K, Kulis B, Fritz M, Darrell T (2010) Transferring visual cateogry models to new domains. In: European conference on computer vision, pp 213–226

  32. Schökopf B, Platt J, Hofmann T (2007) A kernel method for the two-sample-problem. https://doi.org/10.7551/mitpress/7503.003.0069

  33. Shrivastava A, Shekhar S, Patel VM (2014) Unsupervised domain adaptation using parallel transport on grassmann manifold. In: IEEE winter conference on applications of computer vision, pp 277–284. https://doi.org/10.1109/WACV.2014.6836088

  34. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: AAAI conference on artificial intelligence, pp 2058–2065. http://arxiv.org/abs/1511.05547

  35. Tsai YHH, Yeh YR, Wang YCF (2016) Learning cross-domain landmarks for heterogeneous domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 5081–5090. https://doi.org/10.1109/CVPR.2016.549

  36. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2962–2971. https://doi.org/10.1109/CVPR.2017.316

  37. van Opbroek A, Ikram MA, Vernooij MW, de Bruijne M (2015) Transfer learning improves supervised image segmentation across imaging protocols. IEEE Trans Med Imaging 34:1018–1030. https://doi.org/10.1109/TMI.2014.2366792

    Article  Google Scholar 

  38. Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: IEEE international conference on data mining, pp 1129–1134. https://doi.org/10.1109/ICDM.2017.150

  39. Wang J, Chen Y, Yu H, Huang M, Yang Q (2019) Easy transfer learning by exploiting intra-domain structures. In: IEEE international conference on multimedia and expo, pp 1210–1215. https://doi.org/10.1109/ICME.2019.00211

  40. Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: ACM multimedia, pp 402–410. https://doi.org/10.1145/3240508.3240512

  41. Wang R, Utiyama M, Liu L, Chen K, Sumita E (2017) Instance weighting for neural machine translation domain adaptation. In: Conference on empirical methods in natural language processing, pp 1482–1488. https://doi.org/10.18653/v1/D17-1155

  42. Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29:40–51. https://doi.org/10.1109/TPAMI.2007.250598

    Article  Google Scholar 

  43. You K, Long M, Cao Z, Wang J, Jordan MI (2019) Universal domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 2720–2729. http://ise.thss.tsinghua.edu.cn/~mlong/doc/universal-domain-adaptation-cvpr19.pdf

  44. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 5150–5158. https://doi.org/10.1109/CVPR.2017.547

  45. Zhao L, Pan SJ, Yang Q (2017) A unified framework of active transfer learning for cross-system recommendation. Artif Intell 245:38–55. https://doi.org/10.1016/j.artint.2016.12.004

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

We sincerely thank the anonymous reviewers for their careful work and thoughtful suggestions, which have greatly improved this article. This study was funded by the National Natural Science Foundation of China (Grant No. 61802056), the National Natural Science Foundation of China (Grant No. 61300049), the Natural Science Research Foundation of Jilin Province of China (Grant No. 20180101043JC), the Natural Science Research Foundation of Jilin Province of China (Grant No. 20180101053JC), the Industrial Technology Research and Development Project of the Development and Reform Commission of Jilin Province (Grant No. 2019C053-9).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Jingyao Li, Zhanshan Li & Shuai Lü

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering (Jilin University), Ministry of Education, Changchun, 130012, China

    Jingyao Li, Zhanshan Li & Shuai Lü

Authors
  1. Jingyao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhanshan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuai Lü
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shuai Lü.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Li, Z. & Lü, S. Unsupervised double weighted domain adaptation. Neural Comput & Applic 33, 3545–3566 (2021). https://doi.org/10.1007/s00521-020-05228-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05228-4

Keywords

Search

Navigation