Skip to main content
SpringerLink
Log in

Lie group manifold analysis: an unsupervised domain adaptation approach for image classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Domain adaptation aims to minimize the mismatch between the source domain in which models are trained and the target domain to which those models are applied. Most existing works focus on instance reweighting, feature representation, and classifier learning independently, which are ineffective when the domain discrepancy is substantially large. In this study, we propose a new unified hybrid approach that takes advantage of Lie group theory, weighted distribution alignment, and manifold alignment, which are referred to as Lie Group Manifold Analysis (LGMA). LGMA mainly finds a one-parameter sub-group decided by the Lie algebra elements of the intrinsic mean of all samples, and this one-parameter sub-group is a geodesic on the original Lie group. Moreover, the Lie group samples are projected onto the geodesics to maximize the separability of the projected samples for realizing discrimination in the nonlinear Lie group manifold space. As far as we know, LGMA is the first attempt to perform Lie algebra transformation to project the original features in the Lie group space onto Lie algebra manifold space for domain adaptation. Comprehensive experiments validate that our approach considerably outperforms competitive methods on real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Kim S, Jeong M, Ko BC (2020) Energy efficient pupil tracking based on rule distillation of cascade regression forest. Sensors 20(18):5141

    Article  Google Scholar 

  2. Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3(1):1–40

    Article  Google Scholar 

  3. Pan J, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  4. Xing FZ, Pallucchini F, Cambria E (2019) Cognitive-inspired domain adaptation of sentiment lexicons. Inf Process Manag 56(3):554–564

    Article  Google Scholar 

  5. Zhao C, Wang S, Li D (2020) Multi-source domain adaptation with joint learning for cross-domain sentiment classification. Knowl-Based Syst 191:105254

    Article  Google Scholar 

  6. Long M, Wang J, Ding G, Pan SJ, Yu PS (2014) Adaptation regularization: A general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089

    Article  Google Scholar 

  7. Chong Y, Peng C, Zhang C, Wang Y, Feng W, Pan S (2021) Learning domain invariant and specific representation for cross-domain person re-identification. Appl Intell:1–14

  8. Lee C-Y, Batra T, Baig MH, Ulbricht D (2019) Sliced wasserstein discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 10285–10295

  9. Khan MAAH, Roy N, Misra A (2018) Scaling human activity recognition via deep learning-based domain adaptation. In: 2018 IEEE International Conference on Pervasive Computing and Communications (PerCom). IEEE, pp 1–9

  10. Ziser Y, Reichart R (2018) Pivot based language modeling for improved neural domain adaptation. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp 1241–1251

  11. Quanz B, Huan J, Mishra M (2012) Knowledge transfer with low-quality data: A feature extraction issue. IEEE Trans Knowl Data Eng 24(10):1789–1802

    Article  Google Scholar 

  12. Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942

    Article  Google Scholar 

  13. Wei P, Ke Y, Goh CK (2016) Deep nonlinear feature coding for unsupervised domain adaptation. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp 2189–2195

  14. Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: A deep convolutional activation feature for generic visual recognition. Int Conf Mach Learn:647–655

  15. Bendavid S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79(1):151–175

    Article  MathSciNet  Google Scholar 

  16. Ma Z, Yang Y, Nie F, Sebe N, Yan S, Hauptmann AG (2014) Harnessing lab knowledge for real-world action recognition. Int J Comput Vis 109(1):60–73

    Article  Google Scholar 

  17. Li S, Song S, Huang G, Ding Z, Wu C (2018) Domain invariant and class discriminative feature learning for visual domain adaptation. IEEE Trans Image Process PP(99):1–1

    Article  MathSciNet  Google Scholar 

  18. Huang J, Zhou Z (2019) Transfer metric learning for unsupervised domain adaptation. IET Image Process 13(5):804–810

    Article  Google Scholar 

  19. Li J, Jing M, Lu K, Zhu L, Shen HT (2019) Locality preserving joint transfer for domain adaptation. IEEE Trans Image Process 28(12):6103–6115

    Article  MathSciNet  Google Scholar 

  20. Li J, Lu K, Huang Z, Zhu L, Shen HT (2018) Transfer independently together: A generalized framework for domain adaptation. IEEE Trans Cybern 49(6):2144–2155

    Article  Google Scholar 

  21. Qin C, Wang L, Zhang Y, Fu Y (2019) Generatively inferential co-training for unsupervised domain adaptation. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). IEEE, pp 1055–1064

  22. Ahmadvand M, Tahmoresnezhad J (2021) Metric transfer learning via geometric knowledge embedding. Appl Intell 51:921–934

    Article  Google Scholar 

  23. Hoffman J, Rodner E, Donahue J, Darrell T, Saenko K (2013) Efficient learning of domain-invariant image representations. International Conference on Learning Representations

  24. Zhu F, Shao L (2014) Weakly-supervised cross-domain dictionary learning for visual recognition. Int J Comput Vis 109(1):42–59

    Article  Google Scholar 

  25. Gholenji E, Tahmoresnezhad J (2020) Joint discriminative subspace and distribution adaptation for unsupervised domain adaptation. Appl Intell 50(7):2050–2066

    Article  Google Scholar 

  26. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7(2):179–188

    Article  Google Scholar 

  27. Baktashmotlagh M, Harandi MT, Lovell BC, Salzmann M (2014) Domain adaptation on the statistical manifold. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2481–2488

  28. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  29. Ben-David S, Blitzer J, Crammer K, Pereira F (2006) Analysis of representations for domain adaptation. In: International Conference on Neural Information Processing Systems

  30. Fernando B, Habrard A, Sebban M, Tuytelaars T (2014) Unsupervised visual domain adaptation using subspace alignment. In: IEEE International Conference on Computer Vision, pp 2960–2967

  31. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2066– 2073

  32. Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: An unsupervised approach. In: 2011 international conference on computer vision. IEEE, pp 999– 1006

  33. Cui Z, Chang H, Shan S, Chen X (2014) Generalized unsupervised manifold alignment. In: Advances in Neural Information Processing Systems, pp 2429–2437

  34. Shao M, Castillo C, Gu Z, Fu Y (2012) Low-rank transfer subspace learning. In: 2012 IEEE 12th International Conference on Data Mining. IEEE, pp 1104–1109

  35. Ben-David S, Blitzer J, Crammer K, Pereira F (2010) Manifold alignment via corresponding projections. In: British Machine Vision Conference, pp 1–11

  36. Cao Y, Long M, Wang J (2018) Unsupervised domain adaptation with distribution matching machines.. In: AAAI, pp 2795– 2802

  37. Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10 (5):988–999

    Article  Google Scholar 

  38. Tuzel O, Porikli F, Meer P (2008) Learning on lie groups for invariant detection and tracking. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 1– 8

  39. Li F, Zhang L, Zhang Z (2018) Lie group machine learning. Walter de Gruyter GmbH & Co KG

  40. Ghifary M, Balduzzi D, Kleijn WB, Zhang M (2017) Scatter component analysis: A unified framework for domain adaptation and domain generalization. IEEE Trans Pattern Anal Mach Intell 39(7):1414–1430

    Article  Google Scholar 

  41. Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1859–1867

  42. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer feature learning with joint distribution adaptation. In: IEEE International Conference on Computer Vision, pp 2200–2207

  43. Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM international conference on Multimedia, pp 402–410

  44. Georgi H, Jagannathan K (1983) Lie algebras in particle physics. Phys Today 36(12):62–62

    Article  Google Scholar 

  45. Hall BC (2015) Lie groups, lie algebras, and representations. Springer Berl 659(5):xiv,351

    MATH  Google Scholar 

  46. Quanz B, Huan J (2009) Large margin transductive transfer learning. In: Acm Conference on Information & Knowledge Management

  47. Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: IEEE International Conference on Data Mining

  48. Belkin M, Niyogi P, Sindhwani V, Bartlett P (2006) Manifold regularization: a geometric framework for learning from examples. J Mach Learn Res 7(1):2399–2434

    MathSciNet  MATH  Google Scholar 

  49. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European Conference on Computer Vision, pp 213–226

  50. Fang C, Xu Y, Rockmore DN (2014) Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In: IEEE International Conference on Computer Vision, pp 1657–1664

  51. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  52. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: Computer Vision and Pattern Recognition, pp 1410– 1417

Download references

Acknowledgments

This work was supported in part by the Key-Area Research and Development Program for Guangdong Province (2019B010136001) and the National Key Research and Development Plan under Grant 2017YFB0801801, in part by the National Natural Science Foundation of China (NSFC) under Grant 61672186 and Grant 61872110.

Author information

Authors and Affiliations

  1. School of Cyberspace Science, Harbin Institute of Technology, Harbin, 150001, China

    Hongwei Yang, Hui He, Weizhe Zhang, Yawen Bai & Tao Li

  2. Cyberspace Security Research Center, Pengcheng Laboratory, Shenzhen, 518055, China

    Weizhe Zhang

Authors
  1. Hongwei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weizhe Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yawen Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongwei Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., He, H., Zhang, W. et al. Lie group manifold analysis: an unsupervised domain adaptation approach for image classification. Appl Intell 52, 4074–4088 (2022). https://doi.org/10.1007/s10489-021-02564-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02564-3

Keywords

Search

Navigation