Skip to main content
SpringerLink
Log in

Enhancing unsupervised domain adaptation by discriminative relevance regularization

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Unsupervised domain adaptation (UDA) serves to transfer specific knowledge from massive labeled source domain data to unlabeled target domain data via mitigating domain shift. In this paper, we propose a discriminative relevance regularization term (DRR) to enhance the performance of UDA by reducing the domain shift from the aspect of semantic relevance across domains. In particular, DRR is formulated as the min–max rank problem which seeks a projection matrix to minimize the rank of intra-class projected features and maximize the rank of the means of inter-class projected features simultaneously. To test the potential effectiveness of DRR, we design a relevance regularized distribution adaptation method (RRDA) and relevance regularized adaptation networks (RRAN) for image classification, and a relevance regularized self-supervised learning method (RRSL) for semantic segmentation by incorporation of DRR. The corresponding optimization algorithms are proposed to solve them. Experiments of cross-domain image classification show that both RRDA and RRAN outperform several state-of-the-art compared methods. Moreover, experiments of domain-adaptation semantic segmentation on two synthetic-to-real segmentation datasets demonstrate the capacity of RRSL. Such results imply the efficacy of DRR on both image classification and semantic segmentation tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Bousmalis K, Silberman N, Dohan D, Erhan D, Krishnan D (2017) Unsupervised pixel-level domain adaptation with generative adversarial networks. In: IEEE conference on computer vision and pattern recognition. pp 95–104

  2. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  3. Chen M, Xue H, Cai D (2019) Domain adaptation for semantic segmentation with maximum squares loss. In: IEEE international conference on computer vision. pp 2090–2099

  4. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In: IEEE conference on computer vision and pattern recognition. pp 3213–3223

  5. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: A deep convolutional activation feature for generic visual recognition. In: International conference on machine learning. pp 647–655

  6. Fazel M (2002) Matrix rank minimization with applications. Ph.D. thesis, Stanford University

  7. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Humn Genet 7(2):179–188

    Google Scholar 

  8. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning. pp 1180–1189

  9. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17(1):2030–2096

    MathSciNet  MATH  Google Scholar 

  10. Ghifary M, Balduzzi D, Kleijn WB, Zhang M (2015) Scatter component analysis: a unified framework for domain adaptation and domain generalization. IEEE Trans Pattern Anal Mach Intell 39(7):1414–1430

    Article  Google Scholar 

  11. Glorot X, Bordes A, Bengio Y (2011) Domain adaptation for large-scale sentiment classification: a deep learning approach. In: International conference on machine learning. pp 513–520

  12. Gong B, Grauman K, Sha F (2013) Connecting the dots with landmarks: discriminatively learning domain-invariant features for unsupervised domain adaptation. In: International conference on machine learning. pp 222–230

  13. Gong B, Shi Y, Sha F, Kristen G (2012) Geodesic flow kernel for unsupervised domain adaptation. In: IEEE conference on computer vision and pattern recognition. pp 2066–2073

  14. Gong M, Zhang K, Liu T, Tao D, Glymour C, Schölkopf B (2016) Domain adaptation with conditional transferable components. In: International conference on machine learning. pp 2839–2848

  15. Grave E, Obozinski G, Bach F (2011) Trace lasso: a trace norm regularization for correlated designs. In: International conference on neural information processing systems. pp 2187–2195

  16. Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13:723–773

    MathSciNet  MATH  Google Scholar 

  17. Gretton A, Smola A, Huang J, Schmittfull M, Borgwardt K, Schölkopf B (2009) Covariate shift by kernel mean matching. In: Dataset shift in machine learning. pp 131–157

  18. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset. California Institute of Technology

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition. pp 770–778

  20. Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Efros AA, Darrell T (2017) Cycada: Cycle-consistent adversarial domain adaptation. arXiv preprint arXiv:1711.03213

  21. Hoffman J, Wang D, Yu F, Darrell T (2016) Fcns in the wild: pixel-level adversarial and constraint-based adaptation. arXiv preprint arXiv:1612.02649

  22. Hsieh Y, Tao S, Tsai YH, Yeh Y, Wang YF (2016) Recognizing heterogeneous cross-domain data via generalized joint distribution adaptation. In: IEEE international conference on multimedia and expo. pp 1–6

  23. Hsu TH, Chen W, Hou C, Tsai YH, Yeh Y, Wang YF (2015) Unsupervised domain adaptation with imbalanced cross-domain data. In: IEEE international conference on computer vision. pp 4121–4129

  24. Huang J, Smola AJ, Gretton A, Borgwardt KM, Schölkopf B (2006) Correcting sample selection bias by unlabeled data. In: Advances in neural information processing systems. pp 601–608

  25. Isola P, Zhu JY, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: IEEE conference on computer vision and pattern recognition. pp 1125–1134

  26. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp 1097–1105

  27. Li Y, Yuan L, Vasconcelos N (2019) Bidirectional learning for domain adaptation of semantic segmentation. In: IEEE conference on computer vision and pattern recognition. pp 6936–6945

  28. Liu T, Tao D (2015) Classification with noisy labels by importance reweighting. IEEE Trans Pattern Anal Mach Intell 38(3):447–461

    Article  Google Scholar 

  29. Liu T, Yang Q, Tao D (2017) Understanding how feature structure transfers in transfer learning. In: International joint conference on artificial intelligence. pp 2365–2371

  30. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition. pp 3431–3440

  31. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning. pp 97–105

  32. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer feature learning with joint distribution adaptation. In: IEEE international conference on computer vision. pp 2200–2207

  33. Long M, Zhu H, Wang J, Jordan MI (2016) Unsupervised domain adaptation with residual transfer networks. In: Advances in neural information processing systems. pp 136–144

  34. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: International conference on machine learning. pp 2208–2217

  35. Luo Y, Liu T, Tao D, Xu C (2014) Decomposition-based transfer distance metric learning for image classification. IEEE Trans Image Process 23(9):3789–3801

    Article  MathSciNet  Google Scholar 

  36. Luo Y, Wen Y, Liu T, Tao D (2018) Transferring knowledge fragments for learning distance metric from a heterogeneous domain. IEEE Trans Pattern Anal Mach Intell 41(4):1013–1026

    Article  Google Scholar 

  37. Luo Y, Zheng L, Guan T, Yu J, Yang Y (2019) Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation. In: IEEE conference on computer vision and pattern recognition. pp 2507–2516

  38. Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE conference on computer vision and pattern recognition. pp 1717–1724

  39. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  40. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  41. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. In: AAAI conference on artificial intelligence

  42. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434

  43. Richter SR, Vineet V, Roth S, Koltun V (2016) Playing for data: ground truth from computer games. In: European conference on computer vision. Springer, pp 102–118

  44. Ros G, Sellart L, Materzynska J, Vazquez D, Lopez AM (2016) The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes. In: IEEE conference on computer vision and pattern recognition. pp 3234–3243

  45. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision. pp 213–226

  46. Shao L, Zhu F, Li X (2015) Transfer learning for visual categorization: a survey. IEEE Trans Neural Netw Learn Syst 26(5):1019–1034

    Article  MathSciNet  Google Scholar 

  47. Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942

    Article  Google Scholar 

  48. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  49. Sun B, Feng J, Saenko K (2015) Return of frustratingly easy domain adaptation. In: AAAI conference on artificial intelligence. pp 2058–2065

  50. Tsai YH, Hung WC, Schulter S, Sohn K, Yang MH, Chandraker M (2018) Learning to adapt structured output space for semantic segmentation. In: IEEE conference on computer vision and pattern recognition. pp 7472–7481

  51. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: IEEE conference on computer vision and pattern recognition. pp 2962–2971

  52. Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: maximizing for domain invariance. arXiv preprint arXiv:1412.3474

  53. Vu TH, Jain H, Bucher M, Cord M, Pérez P (2019) Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In: IEEE conference on computer vision and pattern recognition. pp 2517–2526

  54. Wang K, He R, Wang W, Wang L, Tan T (2013) Learning coupled feature spaces for cross-modal matching. In: IEEE international conference on computer vision. pp 2088–2095

  55. Wang M, Deng W (2018) Deep visual domain adaptation: a survey. Neurocomputing 312:135–153

    Article  Google Scholar 

  56. Wu X, Song L, He R, Tan T (2018) Coupled deep learning for heterogeneous face recognition. In: AAAI conference on artificial intelligence

  57. Wu Z, Han X, Lin YL, Gokhan Uzunbas M, Goldstein T, Nam Lim S, Davis LS (2018) Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation. In: European conference on computer vision (ECCV). pp 518–534

  58. Xie J, Kiefel M, Sun MT, Geiger A (2016) Semantic instance annotation of street scenes by 3d to 2d label transfer. In: IEEE Conference on computer vision and pattern recognition. pp 3688–3697

  59. Yang Y, Hospedales TM (2017) Trace norm regularised deep multi-task learning. In: International conference on learning representations workshop

  60. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems. pp 3320–3328

  61. Yu X, Liu T, Gong M, Zhang K, Batmanghelich K, Tao D (2017) Transfer learning with label noise. arXiv preprint arXiv:1707.09724

  62. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: IEEE conference on computer vision and pattern recognition. pp. 5150–5158

  63. Zhang Y, David P, Gong B (2017) Curriculum domain adaptation for semantic segmentation of urban scenes. In: Proceedings of the IEEE international conference on computer vision. pp 2020–2030

  64. Zhang Y, Yeung DY (2010) Transfer metric learning by learning task relationships. In: ACM international conference on knowledge discovery and data mining. pp 1199–1208

  65. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: IEEE conference on computer vision and pattern recognition. pp 2881–2890

  66. Zou Y, Yu Z, Liu X, Kumar B, Wang J (2019) Confidence regularized self-training. arXiv preprint arXiv:1908.09822

  67. Zou Y, Yu Z, Vijaya Kumar B, Wang J (2018) Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In: European conference on computer vision (ECCV). pp 289–305

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China [61806213, 61702134, 61906210].

Author information

Authors and Affiliations

  1. Science and Technology on Parallel and Distributed Laboratory, National University of Defense Technology, Changsha, 410073, China

    Wenju Zhang & Zhigang Luo

  2. Institute for Quantum Information and State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha, 410073, China

    Xiang Zhang & Long Lan

  3. College of Computer, National University of Defense Technology, Changsha, 410073, China

    Wenju Zhang, Xiang Zhang, Long Lan & Zhigang Luo

Authors
  1. Wenju Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Long Lan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhigang Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiang Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Zhang, X., Lan, L. et al. Enhancing unsupervised domain adaptation by discriminative relevance regularization. Knowl Inf Syst 62, 3641–3664 (2020). https://doi.org/10.1007/s10115-020-01466-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-020-01466-z

Keywords

Search

Navigation