Skip to main content
SpringerLink
Log in

Cross- and multiple-domains visual transfer learning via iterative Fischer linear discriminant analysis

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

The standard machine learning tasks often assume that the training (source domain) and test (target domain) data follow the same distribution and feature space. However, many real-world applications suffer from the limited number of training labeled data and benefit from the related available labeled datasets to train the model. In this way, since there is the distribution difference across the source and target domains (i.e., domain shift problem), the learned classifier on the training set might perform poorly on the test set. To address the shift problem, domain adaptation provides variety of solutions to learn robust classifiers to deal with distribution mismatch across the source and target domains. In this paper, we put forward a novel domain adaptation approach, referred to as cross- and multiple-domains visual transfer learning via iterative Fischer linear discriminant analysis (CIDA) to tackle shift problem across domains. CIDA transfers the source and target domains into a shared low-dimensional Fischer linear discriminant analysis (FLDA)-based subspace in an unsupervised manner. CIDA benefits joint FLDA and domain adaptation criterions to reduce the distribution mismatch across the training and test sets. Moreover, CIDA employs an adaptive classifier to build a robust model against data drift across different domains. Also, CIDA generates the intermediate pseudotarget labels to utilize the target data in training process. In this way, CIDA refines the pseudolabels using an iterative manner to converge the model. Our extensive experiments illustrate that CIDA significantly outperforms the baseline machine learning and other state-of-the-art transfer learning methods on nine visual benchmark datasets under different difficulties.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Mahya A, Jafar T (2021) Metric transfer learning via geometric knowledge embedding. Appl Intell 51(2):921–934

    Article  Google Scholar 

  2. Karimpour M, Saray SN, Tahmoresnezhad J, Pourmahmood AM (2020) Multi-source domain adaptation for image classification. Mach Vis Appl 31(6):1–19

    Article  Google Scholar 

  3. John B, Mark D, Fernando P et al (2007) Biographies, bollywood, boom-boxes and blenders: domain adaptation for sentiment classification. ACL 7:440–447

    Google Scholar 

  4. Sanodiya RK, Paul D, Yao L, Mathew J, Juhi A (2020) A feature selection approach to visual domain adaptation in classification. In: International conference on neural information processing, Springer, pp 77–89

  5. Wang F, Ding Y, Liang H, Wen J (2021) Discriminative and selective pseudo-labeling for domain adaptation. In: International conference on multimedia modeling, Springer, pp 365–377

  6. Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proceedings of the 2006 conference on empirical methods in natural language processing, Association for Computational Linguistics, pp 120–128

  7. Chen M, Weinberger KQ, Blitzer J (2011) Co-training for domain adaptation. In: Advances in neural information processing systems, pp 2456–2464

  8. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE conference on Computer vision and pattern recognition (CVPR), IEEE, pp 2066–2073

  9. Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: an unsupervised approach. In: 2011 IEEE international conference on computer vision (ICCV), IEEE, pp 999–1006

  10. Tahmoresnezhad J, Hashemi S (2016) Transductive transfer learning via maximum margin criterion. Sci Iran 23(3):1239–1250

    Google Scholar 

  11. Bergamo A, Torresani L (2010) Exploiting weakly-labeled web images to improve object classification: a domain adaptation approach. In: Advances in neural information processing systems, pp 181–189

  12. Kumar A, Saha A, Daume H (2010) Co-regularization based semi-supervised domain adaptation. In: Advances in neural information processing systems, pp 478–486

  13. Kate S, Brian K, Mario F, Trevor D (2010) Adapting visual category models to new domains. Comput Vis ECCV 2010:213–226

    Google Scholar 

  14. Ben-David S, Blitzer J, Crammer K, Pereira F (2007) Analysis of representations for domain adaptation. In: Advances in neural information processing systems, pp 137–144

  15. Ciprian C, Alex A (2006) Adaptation of maximum entropy capitalizer: little data can help a lot. Comput Speech Lang 20(4):382–399

    Article  Google Scholar 

  16. Hal Daume III and Daniel Marcu (2006) Domain adaptation for statistical classifiers. J Artif Intell Res 26:101–126

    Article  MathSciNet  Google Scholar 

  17. Jafar T, Sattar H (2017) Exploiting kernel-based feature weighting and instance clustering to transfer knowledge across domains. Turk J Electr Eng Comput Sci 25(1):292–307

    Google Scholar 

  18. Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation with multiple sources. In: Advances in neural information processing systems, pp 1041–1048

  19. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79(1):151–175

    Article  MathSciNet  Google Scholar 

  20. Crammer K, Earns M, Wortman J (2008) Learning from multiple sources. J Mach Learn Res, 9(Aug):1757–1774

  21. Tahmoresnezhad J, Hashemi S (2015) Common feature extraction in multi-source domains for transfer learning. In: 2015 7th conference on information and knowledge technology (IKT), IEEE, pp 1–5

  22. Fan W, Davidson I, Zadrozny B, Yu PS (2005) An improved categorization of classifier’s sensitivity on sample selection bias. In: Fifth IEEE international conference on data mining, IEEE

  23. Gong B, Grauman K, Sha F (2013) Connecting the dots with landmarks: discriminatively learning domain-invariant features for unsupervised domain adaptation. In: International conference on machine learning, pp 222–230

  24. Jing J, ChengXiang Z (2007) Instance weighting for domain adaptation in nlp. ACL 7:264–271

    Article  Google Scholar 

  25. Sugiyama M, Nakajima S, Kashima H, Buenau PV, Kawanabe M (2008) Direct importance estimation with model selection and its application to covariate shift adaptation. In: Advances in neural information processing systems, pp 1433–1440

  26. Samaneh R, Jafar T, Vahid S (2021) A transductive transfer learning approach for image classification. Int J Mach Learn Cybernet 12(3):747–762

    Article  Google Scholar 

  27. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417

  28. Satpal S, Sarawagi S (2007) Domain adaptation of conditional probability models via feature subsetting. In: PKDD, vol 4702, pp 224–235. Springer

  29. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  30. Si S, Dacheng T, Bo G (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942

    Article  Google Scholar 

  31. Tahmoresnezhad J, Hashemi S (2015) A generalized kernel-based random k-samplesets method for transfer learning. Iran J Sci Technol Trans Electr Eng 39:193–207

    Google Scholar 

  32. Duan L, Tsang IW, Xu D, Maybank SJ (2009) Domain transfer svm for video concept detection. In: CVPR 2009. IEEE conference on computer vision and pattern recognition

  33. Lorenzo B, Mattia M (2010) Domain adaptation problems: a dasvm classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32(5):770–787

    Article  Google Scholar 

  34. Long M, Wang J, Ding G, Pan SJ, Philip SY (2014) Adaptation regularization: A general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089

    Article  Google Scholar 

  35. Shiva Noori Saray and Jafar Tahmoresnezhad (2021) Joint distinct subspace learning and unsupervised transfer classification for visual domain adaptation. SIViP 15(2):279–287

    Article  Google Scholar 

  36. Marzieh Gheisari and Mahdieh Soleymani Baghshah (2015) Unsupervised domain adaptation via representation learning and adaptive classifier learning. Neurocomputing 165:300–311

    Article  Google Scholar 

  37. Elahe G, Jafar T (2020) Joint discriminative subspace and distribution adaptation for unsupervised domain adaptation. Appl Intell 50(7):2050–2066

    Article  Google Scholar 

  38. Vural Elif (2018) Generalization bounds for domain adaptation via domain transformations. In: 2018 IEEE 28th international workshop on machine learning for signal processing (MLSP), IEEE, pp 1–6

  39. Ghifary M, Balduzzi D, Kleijn WB, Zhang M (2017) Scatter component analysis: a unified framework for domain adaptation and domain generalization. IEEE Trans Pattern Anal Mach Intell 1:1–1

    Google Scholar 

  40. Kouw WM, Van Der Maaten LJP, Krijthe JH, Loog M (2016) Feature-level domain adaptation. J Mach Learn Res 17(1):5943–5974

    MathSciNet  MATH  Google Scholar 

  41. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207

  42. Jafar T, Sattar H (2017) Visual domain adaptation via transfer feature learning. Knowl Inf Syst 50(2):585–605

    Article  Google Scholar 

  43. Dinh CV, Duin RPW, Piqueras-Salazar I, Loog M (2013) Fidos: a generalized fisher based feature extraction method for domain shift. Pattern Recogn 46(9):2510–2518

    Article  Google Scholar 

  44. Wenting Tu, Shiliang Sun (2012) Cross-domain representation-learning framework with combination of class-separate and domain-merge objectives. In Proceedings of the 1st International Workshop on Cross Domain Knowledge Discovery in Web and Social Network Mining, pages 18–25. ACM

  45. Wang Z, Song Y, Zhang C (2008) Transferred dimensionality reduction. Mach Learn Knowl Discov Databases, pp 550–565

  46. Borgwardt KM, Gretton A, Rasch MJ, Kriegel H-P, Schölkopf B, Smola AJ (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14):e49–e57

    Article  Google Scholar 

  47. Yao Y, Doretto G (2010) Boosting for transfer learning with multiple sources. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp 1855–1862. IEEE

  48. Yoshida Y, Hirao T, Iwata T, Nagata M, Matsumoto Y (2011) Transfer learning for multiple-domain sentiment analysis-identifying domain dependent/independent word polarity. In: AAAI

  49. Moreno O, Shapira B, Rokach L, Shani G (2012) Talmud: transfer learning for multiple domains. In: Proceedings of the 21st ACM international conference on Information and knowledge management, ACM, pp 425–434

  50. Li S, Zong C (2008) Multi-domain adaptation for sentiment classification: Using multiple classifier combining methods. In: International conference on natural language processing and knowledge engineering, 2008. NLP-KE’08, IEEE, pp 1–8

  51. Rita C, Qian S, Wei F, Ian D, Sethuraman P, Jieping Y (2012) Multisource domain adaptation and its application to early detection of fatigue. ACM Trans Knowl Discov Data (TKDD) 6(4):18

    Google Scholar 

  52. Zhang Y, Cao B, Yeung D-Y (2010) Multi-domain collaborative filtering. In: Proceedings of the twenty-sixth conference on uncertainty in artificial intelligence, UAI’10, Arlington, Virginia, United States, AUAI Press, pp 725–732

  53. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res, 7(Nov):2399–2434

  54. Ulrike VL (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  55. Schölkopf B, Herbrich R, Smola A (2001) A generalized representer theorem. In: Computational learning theory, Springer, pp 416–426

  56. Long M, Wang J, Sun J, Yu Philip S (2015) Domain invariant transfer kernel learning. IEEE Trans Knowl Data Eng 27(6):1519–1532

    Article  Google Scholar 

  57. Griffin G, Holub A, Perona P 2007) Caltech-256 object category dataset

  58. Herbert B, Tinne T, Luc VG (2006) Surf: speeded up robust features. Comput Vis ECCV 2006:404–417

    Google Scholar 

  59. Ming S, Dmitry K, Yun F (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vision 109(1–2):74–93

    MathSciNet  MATH  Google Scholar 

  60. Yong X, Xiaozhao F, Jian W, Xuelong L, David Z (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of IT and Computer Engineering, Urmia University of Technology, Urmia, Iran

    Mehri Mardani & Jafar Tahmoresnezhad

Authors
  1. Mehri Mardani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Tahmoresnezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jafar Tahmoresnezhad.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mardani, M., Tahmoresnezhad, J. Cross- and multiple-domains visual transfer learning via iterative Fischer linear discriminant analysis. Knowl Inf Syst 63, 2157–2188 (2021). https://doi.org/10.1007/s10115-021-01586-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-021-01586-0

Keywords

Search

Navigation