Skip to main content

Advertisement

Springer Nature Link
Log in

Robust and high-order correlation alignment for unsupervised domain adaptation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

How to measure the domain discrepancy is of significant importance in the field of unsupervised domain adaptation. Among them, Correlation Alignment (CORAL), aligning second-order statistics of source and target domains, has become one of the most widely used discrepancy-based methods. However, the performance of CORAL is limited by: (1) aligning covariance with usual Euclidean metric is suboptimal, and (2) second-order statistics have limited expression for the non-Gaussian distribution. To address these limitations, we propose a Robust Correlation Alignment as well as a High-order Correlation Alignment method. The Robust Correlation Alignment exploits the geometric structure of covariance with matrix square-root normalization. To circumvent unstable and time-consuming properties of the Singular Value Decomposition, we employ the variant of Newton iteration to compute the matrix square-root. Besides, we also propose a High-order Correlation Alignment method, which exploits the third-order statistics for domain alignment. We show that the High-order CORAL can be generalized to Maximum Mean Discrepancy, CORAL and arbitrary-order statistics. Specifically, we propose group matching to reduce space complexity and improve the feasibility in real-world application. Extensive experiments on standard benchmark datasets demonstrate that our proposed methods outperform previous methods by a large margin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Arsigny V, Fillard P, Pennec X, Ayache N (2007) Geometric means in a novel vector space structure on symmetric positive-definite matrices. SIAM J Matrix Anal Appl 29(1):328–347

    Article  MathSciNet  Google Scholar 

  2. Ben-David S, Blitzer J, Crammer K, Pereira F (2007) Analysis of representations for domain adaptation. In: Advances in neural information processing systems, pp 137–144

  3. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Machine Intell 35(8):1798–1828

    Article  Google Scholar 

  4. Ben-Israel A (1966) A note on an iterative method for generalized inversion of matrices. Math Comput 20(95):439–440

    Article  Google Scholar 

  5. Bini DA, Iannazzo B (2013) Computing the Karcher mean of symmetric positive definite matrices. Linear Algebra Appl 438(4):1700–1710

    Article  MathSciNet  Google Scholar 

  6. Chen C, Chen Z, Jiang B, Jin X (2019) Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation. Proc AAAI Conf Artif Intell 33:3296–3303

    Google Scholar 

  7. Chen Z, Chen C, Cheng Z, Fang K, Jin X (2019) Selective transfer with reinforced transfer network for partial domain adaptation. arXiv preprint arXiv:1905.10756

  8. Chen Z, Chen C, Jin X, Liu Y, Cheng Z (2019) Deep joint two-stream Wasserstein auto-encoder and selective attention alignment for unsupervised domain adaptation. Neural Comput Appl, pp 1–14

  9. Chen C, Fu Z, Chen Z, Jin S, Cheng Z, Jin X, Hua XS (2019) Homm: higher-order moment matching for unsupervised domain adaptation. arXiv preprint arXiv:1912.11976

  10. Chen C, Jiang B, Jin X (2018) Parameter transfer extreme learning machine based on projective model. In: 2018 International joint conference on neural networks (IJCNN), pp 1–8. IEEE

  11. Cherian A, Sra S, Banerjee A, Papanikolopoulos N (2012) Jensen-Bregman logdet divergence with application to efficient similarity search for covariance matrices. IEEE Trans Pattern Anal Mach Intell 35(9):2161–2174

    Article  Google Scholar 

  12. Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask

  13. Daume H III, Marcu D (2006) Domain adaptation for statistical classifiers. J Artif Intell Res 26:101–126

    Article  MathSciNet  Google Scholar 

  14. De Lathauwer L, Castaing J, Cardoso JF (2007) Fourth-order cumulant-based blind identification of underdetermined mixtures. IEEE Trans Signal Process 55(6):2965–2973

    Article  MathSciNet  Google Scholar 

  15. Denman ED, Beavers AN Jr (1976) The matrix sign function and computations in systems. Applied mathematics and Computation 2(1):63–94

    Article  MathSciNet  Google Scholar 

  16. Dryden IL, Koloydenko A, Zhou D et al (2009) Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. Ann Appl Stat 3(3):1102–1123

    Article  MathSciNet  Google Scholar 

  17. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967

  18. Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V (2016) Domain-adversarial training of neural networks. J Mach Learn Res 17(1):2030–2096

    MathSciNet  MATH  Google Scholar 

  19. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: International conference on machine learning, pp 1180–1189

  20. Gatys LA, Ecker AS, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2414–2423

  21. Gheisari M, Baghshah MS (2015) Unsupervised domain adaptation via representation learning and adaptive classifier learning. Neurocomputing 165:300–311

    Article  Google Scholar 

  22. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on computer vision and pattern recognition, pp 2066–2073. IEEE

  23. Gou M, Camps O, Sznaier M (2017) mom: Mean of moments feature for person re-identification. In: Proceedings of the IEEE international conference on computer vision workshops, pp 1294–1303

  24. Gretton A, Borgwardt K, Rasch M, Schölkopf B, Smola AJ (2007) A kernel method for the two-sample-problem. In: Advances in neural information processing systems, pp 513–520

  25. Gu Q, Li Z, Han J (2011) Joint feature selection and subspace learning. In: Twenty-second international joint conference on artificial intelligence

  26. Higham NJ (1986) Newton’s method for the matrix square root. Math Comput 46(174):537–549

    MathSciNet  MATH  Google Scholar 

  27. Higham NJ (1997) Stable iterations for the matrix square root. Numer Algorithms 15(2):227–242

    Article  MathSciNet  Google Scholar 

  28. Hoffman J, Rodner E, Donahue J, Kulis B, Saenko K (2014) Asymmetric and category invariant feature transformations for domain adaptation. Int J Comput Vis 109:28–41

    Article  MathSciNet  Google Scholar 

  29. Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Efros A, Darrell T (2018) Cycada: cycle-consistent adversarial domain adaptation. In: International conference on machine learning, pp 1994–2003

  30. Ionescu C, Vantzos O, Sminchisescu C (2015) Matrix backpropagation for deep networks with structured layers. In: Proceedings of the IEEE international conference on computer vision, pp 2965–2973

  31. Jakubowski J, Kwiatos K, Chwaleba A, Osowski S (2002) Higher order statistics and neural network for tremor recognition. IEEE Trans Biomed Eng 49(2):152–159

    Article  Google Scholar 

  32. Jhuo IH, Liu D, Lee D, Chang SF (2012) Robust visual domain adaptation with low-rank reconstruction. In: 2012 IEEE conference on computer vision and pattern recognition, pp 2168–2175. IEEE

  33. Jia Y, Darrell T (2011) Heavy-tailed distances for gradient based image descriptors. In: Advances in neural information processing systems, pp 397–405

  34. Jiang B, Chen C, Jin X (2018) Unsupervised domain adaptation with target reconstruction and label confusion in the common subspace. In: Neural computing and applications, pp 1–14

  35. Jiang J, Zhai C (2007) Instance weighting for domain adaptation in nlp. In: Proceedings of the 45th annual meeting of the association of computational linguistics, pp 264–271

  36. Kriegl A, Michor PW (2003) Differentiable perturbation of unbounded operators. Math Ann 327(1):191–201

    Article  MathSciNet  Google Scholar 

  37. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  38. Kulis B, Sustik MA, Dhillon IS (2009) Low-rank kernel learning with Bregman matrix divergences. J Mach Learn Res 10(Feb):341–376

    MathSciNet  MATH  Google Scholar 

  39. Ledoit O, Wolf M (2004) A well-conditioned estimator for large-dimensional covariance matrices. J Multivar Anal 88(2):365–411

    Article  MathSciNet  Google Scholar 

  40. Lin TY, Maji S (2017) Improved bilinear pooling with cnns. arXiv preprint arXiv:1707.06772

  41. Li Y, Swersky K, Zemel R (2015) Generative moment matching networks. In: International conference on machine learning, pp 1718–1727

  42. Li Y, Wang N, Liu J, Hou X (2017) Demystifying neural style transfer. In: Proceedings of the 26th international joint conference on artificial intelligence. AAAI Press, pp 2230–2236

  43. Li P, Xie J, Wang Q, Zuo W (2017) Is second-order information helpful for large-scale visual recognition? In: Proceedings of the IEEE international conference on computer vision, pp 2070–2078

  44. Long M, Cao Y, Cao Z, Wang J, Jordan MI (2018) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell

  45. Long M, Cao Y, Wang J, Jordan M (2015) Learning transferable features with deep adaptation networks. In: International conference on machine learning, pp 97–105

  46. Long M, Ding G, Wang J, Sun J, Guo Y, Yu PS (2013) Transfer sparse coding for robust image representation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 407–414

  47. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417

  48. Long M, Zhu H, Wang J, Jordan MI (2017) Deep transfer learning with joint adaptation networks. In: International conference on machine learning, pp 2208–2217

  49. Mansour A, Jutten C (1995) Fourth-order criteria for blind sources separation. IEEE Trans Signal Process 43(8):2022–2025

    Article  Google Scholar 

  50. Masaeli M, Dy JG, Fung GM (2010) From transformation-based dimensionality reduction to feature selection. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 751–758

  51. Morerio P, Cavazza J, Murino V (2017) Minimal-entropy correlation alignment for unsupervised deep domain adaptation. arXiv preprint arXiv:1711.10288

  52. Morerio P, Murino V (2017) Correlation alignment by Riemannian metric for domain adaptation. arXiv preprint arXiv:1705.08180

  53. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  54. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  55. Pauwels E, Lasserre JB (2016) Sorting out typicality with the inverse moment matrix sos polynomial. In: Advances in neural information processing systems, pp 190–198

  56. Perronnin F, Sénchez J, Xerox YL (2010) Large-scale image categorization with explicit data embedding. In: 2010 IEEE computer society conference on computer vision and pattern recognition, pp 2297–2304. IEEE

  57. Quang MH, San Biagio M, Murino V (2014) Log-Hilbert-Schmidt metric between positive definite operators on Hilbert spaces. In: Advances in neural information processing systems, pp 388–396

  58. Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning. MIT Press, Cambridge

    Google Scholar 

  59. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767

  60. Stein C (1986) Lectures on the theory of estimation of many parameters. J Sov Math 34(1):1373–1403

    Article  Google Scholar 

  61. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: AAAI, vol 6, p 8

  62. Sun B, Saenko K (2016) Deep coral: correlation alignment for deep domain adaptation. In: European conference on computer vision. Springer, Berlin, pp 443–450

  63. Torralba A, Efros AA, et al (2011) Unbiased look at dataset bias. In: CVPR, vol 1, p 7. Citeseer

  64. Tuzel O, Porikli F, Meer P (2008) Pedestrian detection via classification on Riemannian manifolds. IEEE Trans Pattern Anal Mach Intell 30(10):1713–1727

    Article  Google Scholar 

  65. Tzeng E, Hoffman J, Saenko K, Darrell T (2017) Adversarial discriminative domain adaptation. In: Computer vision and pattern recognition (CVPR), vol 1, p 4

  66. Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: maximizing for domain invariance. arXiv preprint arXiv:1412.3474

  67. Wang R, Guo H, Davis LS, Dai Q (2012) Covariance discriminative learning: a natural and efficient approach to image set classification. In: 2012 IEEE conference on computer vision and pattern recognition, pp 2496–2503. IEEE

  68. Wang Q, Li P, Zhang L (2017) G2denet: global Gaussian distribution embedding network and its application to visual recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2730–2739

  69. Wang H, Nie F, Huang H, Ding C (2011) Dyadic transfer learning for cross-domain image classification. In: 2011 International conference on computer vision, pp 551–556. IEEE

  70. Wang Q, Xie J, Zuo W, Zhang L, Li P (2020) Deep cnns meet global covariance pooling: better representation and generalization. IEEE Trans Pattern Anal Machine Intell

  71. Xu J, Ye P, Li Q, Du H, Liu Y, Doermann D (2016) Blind image quality assessment based on high order statistics aggregation. IEEE Trans Image Process 25(9):4444–4457

    Article  MathSciNet  Google Scholar 

  72. Yang Z, Yu W, Liang P, Guo H, Xia L, Zhang F, Ma Y, Ma J (2019) Deep transfer learning for military object recognition under small training set condition. Neural Comput Appl 31(10):6469–6478

    Article  Google Scholar 

  73. Yang E, Lozano A, Ravikumar P (2014) Elementary estimators for sparse covariance matrices and other structured moments. In: International conference on machine learning, pp 397–405

  74. Yang J, Yan R, Hauptmann AG (2007) Adapting svm classifiers to data with shifted distributions. In: Seventh IEEE international conference on data mining workshops (ICDMW 2007), pp 69–76. IEEE

  75. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: International conference on neural information processing systems

  76. Zadrozny B (2004) Learning and evaluating classifiers under sample selection bias. In: Proceedings of the twenty-first international conference on machine learning, p 114. ACM

  77. Zellinger W, Grubinger T, Lughofer E, Natschläger T, Saminger-Platz S (2017) Central moment discrepancy (cmd) for domain-invariant representation learning. arXiv preprint arXiv:1702.08811

  78. Zhuang F, Qi Z, Duan K, Xi D, Zhu Y, Zhu H, Xiong H, He Q (2019) A comprehensive survey on transfer learning. arXiv: abs/1911.02685

Download references

Acknowledgements

This work was supported by the opening foundation of the State Key Laboratory (No. 2014KF06), and the National Science and Technology Major Project (No. 2013ZX03005013).

Author information

Authors and Affiliations

  1. Institution of Information Science and Electrical Engineering, Zhejiang University, Hangzhou, 310037, Zhejiang, China

    Zhaowei Cheng, Chao Chen, Zhihong Chen, Ke Fang & Xinyu Jin

Authors
  1. Zhaowei Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Chao Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhihong Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ke Fang
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Xinyu Jin
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xinyu Jin.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, Z., Chen, C., Chen, Z. et al. Robust and high-order correlation alignment for unsupervised domain adaptation. Neural Comput & Applic 33, 6891–6903 (2021). https://doi.org/10.1007/s00521-020-05465-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05465-7

Keywords