Skip to main content

Advertisement

Springer Nature Link
Log in

Dynamic bias alignment and discrimination enhancement for unsupervised domain adaptation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Unsupervised domain adaptation (UDA) aims to explore the knowledge of labeled source domain to help training the model of unlabeled target domain. By now, while most existing UDA approaches typically learn domain-invariant representations by directly matching the distributions across the domains, they pay less attention on respecting the cross-domain similarity and discrimination exploration. To address these issues, this article designs a kind of UDA with dynamic bias alignment and discrimination enhancement (UDA-DBADE). Specifically, in UDA-DBADE we define a dynamic balance factor by the ratio of the normalized cross-domain discrepancy to the discrimination, which decreases gradually in the process of UDA-DBADE. Afterward, we construct domain alignment with adversarial learning as well as distinguishable representations through advancing the discrepancy of multiple classifiers, and dynamically balance them with the defined dynamic factor. In this way, a larger weight is originally assigned on the domain alignment and then gradually on the discrimination enhancement in the learning process of UDA-DBADE. In addition, we further construct a bias matrix to characterize the discrimination alignment between the source and target domain samples. Compared to current state-of-the-art methods, UDA-DBADE achieves an average accuracy of 88.8% and 89.8% on Office-31 dataset and ImageCLEF-DA dataset, respectively. Finally, extensive experiments demonstrate that UDA-DBADE has an excellent performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

All data generated or analyzed during this study are included in this published article [and its supplementary information files].

Notes

  1. http://yann.lecun.com/exdb/mnist/.

  2. http://ai.bu.edu/adaptation.html.

  3. https://www.imageclef.org/2014/adaptation.

  4. https://www.hemanthdv.org/officeHomeDataset.html.

References

  1. Ding Y, Feng J, Chong Y, Pan S, Sun X (2021) Adaptive sampling toward a dynamic graph convolutional network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–7

    Google Scholar 

  2. Xu H, Yang M, Deng L, Qian Y, Wang C (2021) Neutral cross-entropy loss based unsupervised domain adaptation for semantic segmentation. IEEE Trans Image Process 30:4516–4525

    Article  MathSciNet  Google Scholar 

  3. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. The Journal of Machine Learning Research 17(1):2096–2030

    MathSciNet  Google Scholar 

  4. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7167–7176

  5. Long M, Cao Z, Wang J, Jordan MI (2017) Conditional adversarial domain adaptation. arXiv preprint arXiv:1705.10667

  6. Tian Q, Sun H, Ma C, Cao M, Chu Y, Chen S (2021) Heterogeneous domain adaptation with structure and classification space alignment. IEEE Trans Cybernet 52(10):10328–10338

    Article  Google Scholar 

  7. Geng B, Tao D, Xu C (2011) Daml: domain adaptation metric learning. Proc IEEE Trans Image Process 20(10):2980–2989

    Article  MathSciNet  Google Scholar 

  8. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: international conference on machine learning, pp 97–105

  9. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: proceedings of the AAAI conference on artificial intelligence, vol. 30

  10. Tian Q, Sun H, Peng S, Ma T (2023) Self-adaptive label filtering learning for unsupervised domain adaptation. Front Comput Sci 17(1):1–3

    Article  Google Scholar 

  11. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: international conference on machine learning, pp 1180–1189

  12. Tian Q, Zhu Y, Sun H, Chen S, Yin H (2022) Unsupervised domain adaptation through dynamically aligning both the feature and label spaces. IEEE Trans Circuits Syst Video Technol 32(12):8562–8573

    Article  Google Scholar 

  13. Saito K, Watanabe K, Ushiku Y, Harada T (2018) Maximum classifier discrepancy for unsupervised domain adaptation. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 3723–3732

  14. Lee CY, Batra T, Baig MH, Ulbricht D (2019) Sliced wasserstein discrepancy for unsupervised domain adaptation. In: proceedings of the IEEE conference on computer vision and pattern recognition, pp 10285–10295

  15. Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B (2019) Moment matching for multi-source domain adaptation. In: proceedings of the IEEE international conference on computer vision, pp 1406–1415

  16. Zellinger W, Grubinger T, Lughofer E, Natschl T, Saminger-Platz S (2017) Central moment discrepancy (cmd) for domain-invariant representation learning. arXiv preprint arXiv:1702.08811

  17. Peng X, Saenko K (2018) Synthetic to real adaptation with generative correlation alignment networks. In: proceedings of the IEEE winter conference on applications of computer vision, pp 1982–1991

  18. Sun B, Saenko K (2016) Deep coral: correlation alignment for deep domain adaptation. In: European conference on computer vision, pp 443–450

  19. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680

    Google Scholar 

  20. Xiao N, Zhang L (2021) Dynamic weighted learning for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 15242–15251

  21. Wei G, Lan C, Zeng W, Chen Z (2021) Metaalign: coordinating domain alignment and classification for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16643–16653

  22. Huang J, Xiao N, Zhang L (2022) Balancing transferability and discriminability for unsupervised domain adaptation. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2022.3201623

    Article  Google Scholar 

  23. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3722–3731

  24. Sener O, Song HO, Saxena A, Savarese S (2016) Learning transferrable representations for unsupervised domain adaptation. In: Advances in neural information processing systems, pp 2110–2118

  25. Bousmalis K, Trigeorgis G, Silberman N, Krishnan D, Erhan D (2016) Domain separation networks. Adv Neural Inf Process Syst 29:343–351

    Google Scholar 

  26. Zhang M, Wang H, He P, Malik A, Liu H (2022) Exposing unseen gan-generated image using unsupervised domain adaptation. Knowl-Based Syst 257:109905

    Article  Google Scholar 

  27. Zhao D, Wang Z, Li H, Xiang J (2022) Gan-based privacy-preserving unsupervised domain adaptation. In: 2022 IEEE 22nd international conference on software quality, reliability and security (QRS), pp 117–126

  28. Kalina B, Lee J (2023) Improving unsupervised domain adaptation with auxiliary classifier gans. In Proceedings of the 2023 international conference on research in adaptive and convergent systems, pp 1–6

  29. Kang G, Jiang L, Yang Y, Hauptmann AG (2019) Contrastive adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4893–4902 (2019)

  30. Xie S, Zheng Z, Chen L, Chen C (2018) Learning semantic representations for unsupervised domain adaptation. In: International conference on machine learning, pp 5423–5432

  31. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. In: Thirty-second AAAI conference on artificial intelligence

  32. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: International conference on machine learning, pp 2208–2217

  33. Yan H, Ding Y, Li P, Wang Q, Xu Y, Zuo W (2017) Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2272–2281

  34. Shen J, Qu Y, Zhang W, Yu Y (2018) Wasserstein distance guided representation learning for domain adaptation. In: Thirty-second AAAI conference on artificial intelligence

  35. Chen Q, Liu Y, Wang Z, Wassell I, Chetty K (2018) Re-weighted adversarial adaptation network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7976–7985

  36. Zhang Y, Tang H, Jia K, Tan M (2019) Domain-symmetric networks for adversarial domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5031–5040

  37. Shen G, Yu Y, Tang Z-R, Chen H, Zhou Z (2022) Hqa-trans: an end-to-end high-quality-awareness image translation framework for unsupervised cross-domain pedestrian detection. IET Comput Vision 16(3):218–229

    Article  Google Scholar 

  38. Liu H, Long M, Wang J, Jordan M (2019) Transferable adversarial training: a general approach to adapting deep classifiers. In: International conference on machine learning, pp 4013–4022

  39. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

  40. Oh Song H, Xiang Y, Jegelka S, Savarese S (2016) Deep metric learning via lifted structured feature embedding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4004–4012

  41. Sohn K (2016) Improved deep metric learning with multi-class n-pair loss objective. In: Advances in neural information processing systems, pp 1857–1865

  42. Wang X, Han X, Huang W, Dong D, Scott MR (2019) Multi-similarity loss with general pair weighting for deep metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5022–5030

  43. Aziere N, Todorovic S (2019) Ensemble deep manifold similarity learning using hard proxies. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7299–7307

  44. Kim S, Kim D, Cho M, Kwak S (2020) Proxy anchor loss for deep metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3238–3247

  45. Qian Q, Shang L, Sun B, Hu J, Li H, Jin R (2019) Softtriple loss: deep metric learning without triplet sampling. In: Proceedings of the IEEE international conference on computer vision, pp 6450–6458

  46. Movshovitz-Attias Y, Toshev A, Leung TK, Ioffe S, Singh S (2017) No fuss distance metric learning using proxies. In: Proceedings of the IEEE international conference on computer vision, pp 360–368

  47. Tang Z, Jiao Q, Zhong J, Wu S, Wong HS (2022) Source-free unsupervised cross-domain pedestrian detection via pseudo label mining and screening. In: 2022 IEEE international conference on multimedia and expo (ICME), pp 1–6. IEEE

  48. Liang J, Hu D, Feng J (2021) Domain adaptation with auxiliary target domain-oriented classifier. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 16632–16642

  49. Oord AVD, Li Y, Vinyals O (2018) Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748

  50. Mnih A, Kavukcuoglu K (2013) Learning word embeddings efficiently with noise-contrastive estimation. In: Advances in neural information processing systems, pp 2265–2273

  51. Wang S, Zhang L (2020) Self-adaptive re-weighted adversarial domain adaptation. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, pp 3181–3187

  52. Wang S, Zhang L, Wang P, Wang M, Zhang X (2023) Bp-triplet net for unsupervised domain adaptation: a bayesian perspective. Pattern Recognit. 133:108993

    Article  Google Scholar 

  53. Dorfer M, Kelz R, Widmer G (2015) Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707

  54. Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(11):2579–2605

    Google Scholar 

  55. LeCun Y, Bottou Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  56. Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning

  57. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision, pp 213–226

  58. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: International conference on machine learning, pp 2208–2217

  59. Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Efros A, Darrell T (2018) Cycada: cycle-consistent adversarial domain adaptation. In: International conference on machine learning, pp 1989–1998

  60. Deng Z, Luo Y, Zhu J (2019) Cluster alignment with a teacher for unsupervised domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 9944–9953

  61. Pinheiro PO (2018) Unsupervised domain adaptation with similarity learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8004–8013

  62. Pan Y, Yao T, Li Y, Wang Y, Ngo CW, Mei T (2019) Transferrable prototypical networks for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2239–2247

  63. Ye S, Wu K, Zhou M, Yang Y, Tan SH, Xu K, Song J, Bao C, Ma K (2020) Light-weight calibrator: a separable component for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 13736–13745

  64. Li M, Zhai YM, Luo YW, Ge PF, Ren CX (2020) Enhanced transport distance for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 13936–13944

  65. Du Z, Li J, Su H, Zhu L, Lu K (2021) Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3937–3946

  66. Li S, Song S-J, Wu C (2018) Layer-wise domain correction for unsupervised domain adaptation. Front Inf Technol Electron Eng 19(1):91–103

    Article  Google Scholar 

  67. Xu R, Li G, Yang J, Lin L (2019) Larger norm more transferable: an adaptive feature norm approach for unsupervised domain adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 1426–1435

  68. Wang H, Tian J, Li S, Zhao H, Wu F, Li X (2022) Structure-conditioned adversarial learning for unsupervised domain adaptation. Neurocomputing 497:216–226

    Article  Google Scholar 

  69. Hu L, Kan M, Shan S, Chen X (2020) Unsupervised domain adaptation with hierarchical gradient synchronization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4043–4052

  70. Liu M-Y, Tuzel O (2016) Coupled generative adversarial networks. Adv Neural Inf Process Syst 29:469–477

    Google Scholar 

  71. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248–255

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 62176128, the Natural Science Foundation of Jiangsu Province under Grant BK20231143, the Open Projects Program of State Key Laboratory for Novel Software Technology of Nanjing University under Grant KFKT2022B06, the Fundamental Research Funds for the Central Universities No. NJ2022028, the Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) fund, as well as the Qing Lan Project of Jiangsu Province.

Author information

Authors and Affiliations

  1. School of Software, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Qing Tian, Hong Yang & Yao Cheng

  2. Wuxi Institute of Technology, Nanjing University of Information Science and Technology, Wuxi, 214000, China

    Qing Tian

  3. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Qing Tian

Authors
  1. Qing Tian
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Hong Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yao Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qing Tian.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tian, Q., Yang, H. & Cheng, Y. Dynamic bias alignment and discrimination enhancement for unsupervised domain adaptation. Neural Comput & Applic 36, 7763–7777 (2024). https://doi.org/10.1007/s00521-024-09507-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-09507-2

Keywords