Skip to main content
Springer Nature Link
Log in

Dara: distribution-aware representation alignment for semi-supervised domain adaptation in image classification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Semi-supervised domain adaptation (SSDA) aims to adapt a model trained on an annotated source domain to a related, but different, target domain with limited labeled and abundant unlabeled data. However, current self-training approaches often produce incorrect pseudo-labels due to the lack of calibration. To alleviate this issue, we propose a simple yet effective approach, termed Distribution-Aware Representation Alignment (DARA), to address the SSDA problem. Our approach introduces a distribution calibration strategy that reduces spurious pseudo-labels and enhances pseudo-label quality by normalizing the current probability distribution with the holistic class mean. Based on this calibration, we apply probability-level and feature-level representation alignments to reduce domain discrepancy. The probability-level alignment merges the source domain’s ground-truth labels with corrected pseudo-labels from the target domain to supervise image mixtures. The feature-level alignment identifies matching features from both domains based on shared label predictions, enforcing consistent labels and alignment. By building the representation alignment upon distribution calibration, our approach can effectively reduce confirmation bias and domain shift, thus improving the generalization from the source domain to the target domain. Comprehensive experiments on standard SSDA benchmarks (i.e., Office-31, Office-Home, and DomainNet) demonstrate the superiority of DARA and the effectiveness of its components.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Meng M, Lan M, Yu J, Wu J, Tao D (2019) Constrained discriminative projection learning for image classification. IEEE Trans Image Process 29:186–198

    Article  MathSciNet  MATH  Google Scholar 

  2. Minaee S, Boykov Y, Porikli F, Plaza A, Kehtarnavaz N, Terzopoulos D (2021) Image segmentation using deep learning: a survey. IEEE Trans Pattern Analy Mach Intell 44(7):3523–3542

    Google Scholar 

  3. Sun P, Zhang R, Jiang Y, Kong T, Xu C, Zhan W, Tomizuka M, Li L, Yuan Z, Wang C, et al (2021) “Sparse r-cnn: end-to-end object detection with learnable proposals.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14454–14463

  4. Fang Y, Yap P-T, Lin W, Zhu H, Liu M (2024) “Source-free unsupervised domain adaptation: a survey.” Neural Netw, p 106230

  5. Yang J, Liu J, Xu N, Huang J (2023)‘Tvt: transferable vision transformer for unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 520–530

  6. Zhu J, Bai H, Wang L (2023) “Patch-mix transformer for unsupervised domain adaptation: a game perspective.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3561–3571

  7. Meng M, Wu Z, Liang T, Yu J, Wu J (2022) Exploring fine-grained cluster structure knowledge for unsupervised domain adaptation. IEEE Trans Circuits Syst Video Technol 32(8):5481–5494

    Article  MATH  Google Scholar 

  8. Yang Y, Zhang T, Li G, Kim T, Wang G (2022) An unsupervised domain adaptation model based on dual-module adversarial training. Neurocomputing 475:102–111

    Article  MATH  Google Scholar 

  9. Zhang C, Zhao Q, Wu H (2022) Deep domain adaptation via joint transfer networks. Neurocomputing 489:441–448

    Article  MATH  Google Scholar 

  10. Ge P, Ren C-X, Xu X-L, Yan H (2023) Unsupervised domain adaptation via deep conditional adaptation network. Pattern Recogn 134:109088

    Article  Google Scholar 

  11. Rahman MM, Panda R, Alam MAU (2023) “Semi-supervised domain adaptation with auto-encoder via simultaneous learning.” In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 402–411

  12. Yu H, Sano A (2023) Semi-supervised learning for wearable-based momentary stress detection in the wild. Proceed ACM Interact, Mobile, Wearable Ubiquit Technol 7(2):1–23

    Article  MATH  Google Scholar 

  13. Wang Y, Yin J, Li W, Frossard P, Yang R, Shen J (2023) Ssda3d: semi-supervised domain adaptation for 3d object detection from point cloud. Proceed AAAI Conf Artif Intell 37:2707–2715

    MATH  Google Scholar 

  14. Saito K, Kim D, Sclaroff S, Darrell T, Saenko K (2019) “Semi-supervised domain adaptation via minimax entropy.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 8050–8058

  15. Zhang Y, Deng B, Jia K, Zhang L (2020) “Label propagation with augmented anchors: A simple semi-supervised learning baseline for unsupervised domain adaptation.” In: European Conference on Computer Vision, pp. 781–797, Springer

  16. Kim T, Kim C (2020) “Attract, perturb, and explore: learning a feature alignment network for semi-supervised domain adaptation.” In: European Conference on Computer Vision, pp 591–607, Springer

  17. Jiang P, Wu A, Han Y, Shao Y, Qi M, Li B (2020) “Bidirectional adversarial training for semi-supervised domain adaptation.” In: IJCAI, pp. 934–940

  18. Qin C, Wang L, Ma Q, Yin Y, Wang H, Fu Y (2021) “Contradictory structure learning for semi-supervised domain adaptation.” In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pp 576–584, SIAM

  19. Li J, Li G, Shi Y, Yu Y (2021) “Cross-domain adaptive clustering for semi-supervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2505–2514

  20. Li K, Liu C, Zhao H, Zhang Y, Fu Y (2021) “Ecacl: a holistic framework for semi-supervised domain adaptation.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 8578–8587

  21. Singh A (2021) Clda: contrastive learning for semi-supervised domain adaptation. Adv Neur Inform Process Syst 34:5089–5101

    MATH  Google Scholar 

  22. Pérez-Carrasco M, Protopapas P, Cabrera-Vives G (2022) “\(\text{Con}^{2}\)da: Simplifying semi-supervised domain adaptation by learning consistent and contrastive feature representations,” arXiv preprint arXiv:2204.01558

  23. Li J, Xiong C, Hoi SC (2021) “Comatch: semi-supervised learning with contrastive graph regularization.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 9475–9484

  24. Saenko K, Kulis B, Fritz M, Darrell T (2010) “Adapting visual category models to new domains.” In: European Conference on Computer Vision, pp 213–226, Springer

  25. Venkateswara H, Eusebio J, Chakraborty S, Panchanathan S (2017) “Deep hashing network for unsupervised domain adaptation.” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5018–5027

  26. Peng X, Bai Q, Xia X, Huang Z, Saenko K, Wang B (2019) “Moment matching for multi-source domain adaptation.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1406–1415

  27. Wu H, Ma Z, Guo Z, Wu Y, Zhang J, Zhou G, Long J (2024) “Online privacy-preserving eeg classification by source-free transfer learning.” IEEE Transactions on Neural Systems and Rehabilitation Engineering

  28. Wu H, Xie Q, Yu Z, Zhang J, Liu S, Long J (2024) Unsupervised heterogeneous domain adaptation for eeg classification. J Neur Eng 21(4):046018

    Article  Google Scholar 

  29. Chen S, Zheng L, Wu H (2023) Riemannian representation learning for multi-source domain adaptation. Pattern Recogn 137:109271

    Article  MATH  Google Scholar 

  30. Wu H, Long J, Li N, Yu D, Ng MK (2022) Adversarial auto-encoder domain adaptation for cold-start recommendation with positive and negative hypergraphs. ACM Trans Inform Syst 41(2):1–25

    Google Scholar 

  31. Du Z, Li X, Li F, Lu K, Zhu L, Li J (2024) “Domain-agnostic mutual prompting for unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 23375–23384

  32. Du Y, Zhou D, Xie Y, Lei Y, Shi J (2023) Prototype-guided feature learning for unsupervised domain adaptation. Pattern Recogn 135:109154

    Article  MATH  Google Scholar 

  33. Zhang Y, Wang Z, He W (2023) “Class relationship embedded learning for source-free unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7619–7629

  34. Ganin Y, Lempitsky V (2015) “Unsupervised domain adaptation by backpropagation.” In: International Conference on Machine Learning, pp 1180–1189, PMLR

  35. Chen C, Chen Z, Jiang B, Jin X (2019) Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation. Proceed AAAI Conf Artif intell 33:3296–3303

    MATH  Google Scholar 

  36. Wu H, Ng MK (2022) Multiple graphs and low-rank embedding for multi-source heterogeneous domain adaptation. ACM Trans Knowl Discov Data (TKDD) 16(4):1–25

    Article  MATH  Google Scholar 

  37. Yan Y, Wu H, Ye Y, Bi C, Lu M, Liu D, Wu Q, Ng MK (2021) Transferable feature selection for unsupervised domain adaptation. IEEE Trans Knowl Data Eng 34(11):5536–5551

    Article  MATH  Google Scholar 

  38. Zhang C, Hu C, Xie J, Wu H, Zhang J (2024) Wcal: weighted and center-aware adaptation learning for partial domain adaptation. Eng Appl Artif Intell 130:107740

    Article  MATH  Google Scholar 

  39. Litrico M, Del Bue A, Morerio P (2023) “Guiding pseudo-labels with uncertainty estimation for source-free unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7640–7650

  40. Saito K, Watanabe K, Ushiku Y, Harada T (2018) “Maximum classifier discrepancy for unsupervised domain adaptation.” In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3723–3732

  41. Deng Z, Luo Y, Zhu J (2019) “Cluster alignment with a teacher for unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 9944–9953

  42. Zheng X, Pan T, Luo Y, Wang L (2023) “Look at the neighbor: distortion-aware unsupervised domain adaptation for panoramic semantic segmentation.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 18687–18698

  43. Yue Z, Sun Q, Zhang H (2023) Make the u in uda matter: invariant consistency learning for unsupervised domain adaptation. Adv Neur Inform Process Syst 36:26991–27004

    Google Scholar 

  44. Choi J, Jeong M, Kim T, Kim C (2019) “Pseudo-labeling curriculum for unsupervised domain adaptation,” arXiv preprint arXiv:1908.00262

  45. Wang Q, Breckon T (2020) Unsupervised domain adaptation via structured prediction based selective pseudo-labeling. Proceed AAAI Conf Artif Intell 34:6243–6250

    MATH  Google Scholar 

  46. Morerio P, Volpi R, Ragonesi R, Murino V (2020) “Generative pseudo-label refinement for unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp 3130–3139

  47. Zhang Y, Davison BD (2021) “Efficient pre-trained features and recurrent pseudo-labeling in unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2719–2728

  48. Chen C, Xie W, Huang W, Rong Y, Ding X, Huang Y, Xu T, Huang J (2019) “Progressive feature alignment for unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 627–636

  49. Liang J, He R, Sun Z, Tan T (2019) Exploring uncertainty in pseudo-label guided unsupervised domain adaptation. Pattern Recogn 96:106996

    Article  MATH  Google Scholar 

  50. Li J, Li G, Yu Y (2023) “Adaptive betweenness clustering for semi-supervised domain adaptation.” IEEE Trans Image Process

  51. Li J, Li G, Yu Y (2024) Inter-domain mixup for semi-supervised domain adaptation. Pattern Recogn 146:110023

    Article  MATH  Google Scholar 

  52. Chen T, Guo Y, Hao S, Hong R (2023) “Semi-supervised domain adaptation for major depressive disorder detection.” IEEE Trans Multimedia

  53. Jain SK, Das S (2023) “Marrs: Modern backbones assisted co-training for rapid and robust semi-supervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4580–4589

  54. Yang L, Wang Y, Gao M, Shrivastava A, Weinberger KQ, Chao W-L, Lim S-N (2021) “Deep co-training with task decomposition for semi-supervised domain adaptation.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 8906–8916

  55. Wang S, Chen Z, Du S, Lin Z (2021) Learning deep sparse regularizers with applications to multi-view clustering and semi-supervised classification. IEEE Trans Pattern Analy Mach Intell 44(9):5042–5055

    MATH  Google Scholar 

  56. Sajjadi M, Javanmardi M, Tasdizen T (2016) Regularization with stochastic transformations and perturbations for deep semi-supervised learning. Adv Neur Inform Process Syst 29

  57. Gui Q, Zhou H, Guo N, Niu B (2023) A survey of class-imbalanced semi-supervised learning. Mach Learn 1–30

  58. Laine S, Aila T (2016) “Temporal ensembling for semi-supervised learning,” arXiv preprint arXiv:1610.02242

  59. Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neur Inform Process Syst 30

  60. Sohn K, Berthelot D, Carlini N, Zhang Z, Zhang H, Raffel CA, Cubuk ED, Kurakin A, Li C-L (2020) Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv Neur Inform Process Syst 33:596–608

    Google Scholar 

  61. Zheng M, You S, Huang L, Wang F, Qian C, Xu C (2022) “Simmatch: Semi-supervised learning with similarity matching.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 14471–14481

  62. Miao J, Chen C, Liu F, Wei H, Heng P-A (2023) “Caussl: causality-inspired semi-supervised learning for medical image segmentation.” In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 21426–21437

  63. Huang S, Wang K, Liu H, Chen J, Li Y (2023) “Contrastive semi-supervised learning for underwater image restoration via reliable bank.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 18145–18155

  64. Sun Y, Shi Z, Li Y (2024) A graph-theoretic framework for understanding open-world semi-supervised learning. Adv Neur Inform Process Syst 36

  65. Li B, Wang Y, Zhang S, Li D, Keutzer K, Darrell T, Zhao H (2021) “Learning invariant representations and risks for semi-supervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1104–1113

  66. Na J, Jung H, Chang HJ, Hwang W (2021) “Fixbi: Bridging domain spaces for unsupervised domain adaptation.” In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1094–1103

  67. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17:2096

    MathSciNet  MATH  Google Scholar 

  68. Saito K, Ushiku Y, Harada T, Saenko K (2017) “Adversarial dropout regularization,” arXiv preprint arXiv:1711.01575

  69. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. Adv Neur Inform Process Syst 31

  70. Grandvalet Y, Bengio Y (2004) Semi-supervised learning by entropy minimization. Adv Neur Inform Process Syst 17

  71. Bhalerao MM, Singh A, Biswas S (2022) “Pred &guide: labeled target class prediction for guiding semi-supervised domain adaptation,” arXiv preprint arXiv:2211.11975

  72. Li D, Hospedales T (2020) “Online meta-learning for multi-source and semi-supervised domain adaptation.” In: European Conference on Computer Vision, pp 382–403, Springer

  73. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comp Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  74. Zheng Z, Wu H, Lv L, Ye H, Zhang C, Yu G (2023) Iccl: independent and correlative correspondence learning for few-shot image classification. Knowl-Based Syst 266:110412

    Article  Google Scholar 

  75. Zheng Z, Feng X, Yu H, Li X, Gao M (2023) Unsupervised few-shot image classification via one-vs-all contrastive learning. Appl Intell 53(7):7833–7847

    Article  Google Scholar 

  76. Zheng Z, Feng X, Yu H, Li X, Gao M (2023) Bdla: bi-directional local alignment for few-shot learning. Appl Intell 53(1):769–785

    Article  MATH  Google Scholar 

  77. Zheng Z, Feng X, Yu H, Gao M (2022) Cooperative density-aware representation learning for few-shot visual recognition. Neurocomputing 471:208–218

    Article  MATH  Google Scholar 

  78. Cubuk ED, Zoph B, Shlens J, Le QV (2020) “Randaugment: practical automated data augmentation with a reduced search space.” In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp 702–703

  79. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inform Process Syst 32

Download references

Acknowledgements

This work was supported in part by Natural Science Foundation of China (Grant 12071104), Natural Science Foundation of Zhejiang Province (Grant LD19A010002), Scientific Research Fund of Zhejiang Provincial Education Department (Grant Y202456908), Research Project on the Communist Youth League in Hangzhou Schools (Grant hzxx24069), Fundamental Research Funds for the Provincial Universities of Zhejiang (Grant 230056), Natural Science Foundation of China (Grant 62261002), Jiangxi Double Thousand Plan (Grant jxsq2019201061), and Science and Technology Program of Jiangxi Province (Grants 20192BCB23019, and 20202BBE53024).

Author information

Authors and Affiliations

  1. College of Information Engineering, Hangzhou Vocational & Technical College, Hangzhou, China

    Heng Wu

  2. College of Education Science, Ludong University, Yantai, Shandong, China

    Heng Wu

  3. College of Sciences, China Jiliang University, Hangzhou, Zhejiang, China

    Zijun Zheng

  4. College of Information Engineering, China Jiliang University, Hangzhou, Zhejiang, China

    Laishui Lv

  5. School of Technology, Beijing Forestry University, Beijing, 100083, China

    Changchun Zhang

  6. ICOSI Laboratory, Department of Computer Science and Mathematics, University of Abbes Laghrour, Khenchela, Algeria

    Dalal Bardou

  7. School of Mathematics and Computer Science, Gannan Normal University, Ganzhou, China

    Shanzhou Niu

  8. Department of Mathematics, Hangzhou Dianzi University, Hangzhou, China

    Gaohang Yu

Authors
  1. Heng Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zijun Zheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Laishui Lv
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Changchun Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Dalal Bardou
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Shanzhou Niu
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Gaohang Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Zijun Zheng or Laishui Lv.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, H., Zheng, Z., Lv, L. et al. Dara: distribution-aware representation alignment for semi-supervised domain adaptation in image classification. J Supercomput 81, 376 (2025). https://doi.org/10.1007/s11227-024-06886-0

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11227-024-06886-0

Keywords