Skip to main content

Advertisement

SpringerLink
Log in

Particle swarm optimization based parameter selection technique for unsupervised discriminant analysis in transfer learning framework

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The purpose of transfer learning is to utilize the knowledge gained from the existing (source) domain to enhance the performance on a distinct but related (target) domain. Existing works on transfer learning are not capable of optimizing different quality measures (components) such as minimizing the marginal distribution, minimizing the conditional distribution, maximizing the target domain variance, modeling the manifold by utilizing the common geometric properties in the source as well as the target domain at the same time. Moreover, existing transfer learning methods use conventional approaches to determine the appropriate values of their parameters, which is very hectic and time-consuming. Therefore, in order to overcome the drawbacks of existing approaches, we propose a Particle Swarm Optimization based Parameter Selection Approach for Unsupervised Discriminant Analysis (UDATL-PSO) in transfer learning framework. In UDATL-PSO, all the quality measures are considered at the same time, as well as the PSO approach has been used to select the best values of their parameters. Extensive experiments on various transfer learning tasks show that the proposed method has a significant influence on state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Chen M, Xu Z, Weinberger KQ, Sha F (2012) Marginalized denoising autoencoders for domain adaptation. in: International conference on machine learning, ICML’12. Omnipress, USA, pp 1627–1634

  2. Ionescu B, Lupu M, Rohm M, Gînsca AL, Müller H (2018) Datasets column: diversity and credibility for social images and image retrieval. ACM SIGMultimedia Rec 9(3):7

    Article  Google Scholar 

  3. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  4. Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: 2007. ICCV 2007. IEEE 11th International Conference on Computer Vision. IEEE, pp 1–7

  5. Shao L, Zhu F, Li X (2015) Transfer learning for visual categorization: a survey. IEEE Trans Neural Netw Learn Syst 26(5):1019–1034

    Article  MathSciNet  Google Scholar 

  6. Ding Z, Fu Y (2017) Robust transfer metric learning for image classification. IEEE Trans Image Process 26(2):660–670

    Article  MathSciNet  Google Scholar 

  7. Yang L, Jin R (2006) Distance metric learning: a comprehensive survey. Michigan State Universiy 2(2):4

    Google Scholar 

  8. Huang K-K, Dai D-Q, Ren C-X (2017) Regularized coplanar discriminant analysis for dimensionality reduction. Pattern Recogn 62:87–98

    Article  Google Scholar 

  9. Zhang J, Li W, Ogunbona P Joint geometrical and statistical alignment for visual domain adaptation, arXiv:1705.05498

  10. Chen L, Zhang H, Xiao J, Liu W, Chang S-F (2018) Zero-shot visual recognition using semantics-preserving adversarial embedding network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol 2

  11. Long M, Wang J, Ding G, Sun J, Yu PS (201) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200– 2207

  12. Long M, Wang J, Ding G, Sun J, Yu PS (2014) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417

  13. Dai Wenyuan YQ, Guirong X, et al. (2007) Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning. Corvallis, USA, pp 193–200

  14. Sanodiya RK, Mathew J (2019) A framework for semi-supervised metric transfer learning on manifolds. Knowl-Based Syst 176:1–14

    Article  Google Scholar 

  15. Sanodiya RK, Mathew J, Saha S, Thalakottur MD (2019) A new transfer learning algorithm in semi-supervised setting. IEEE Access 7:42956–42967

    Article  Google Scholar 

  16. Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171

    Article  Google Scholar 

  17. Luo L, Chen L, Hu S, Lu Y, Wang X Discriminative and geometry aware unsupervised domain adaptation (2017). arXiv:1712.10042

  18. Jhuo I-H, Liu D, Lee D, Chang S-F (2012) Robust visual domain adaptation with low-rank reconstruction. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp 2168–2175

  19. Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  20. Nguyen BH, Xue B, Andreae P (2018) A particle swarm optimization based feature selection approach to transfer learning in classification. In: Proceedings of the Genetic and Evolutionary Computation Conference. ACM, pp 37–44

  21. Kennedy J (2010) Particle swarm optimization. Encyclopedia of Machine Learning:760–766

  22. Nguyen BH, Xue B, Andreae P (2017) A novel binary particle swarm optimization algorithm and its applications on knapsack and feature selection problems. In: Intelligent and Evolutionary Systems. Springer, pp 319–332

  23. Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: 2018 ACM Multimedia Conference on Multimedia Conference ACM, pp 402–410

  24. Baktashmotlagh M, Harandi M, Salzmann M (2016) Distribution-matching embedding for visual domain adaptation. J Mach Learn Res 17(1):3760–3789

    MathSciNet  MATH  Google Scholar 

  25. Satpal S, Sarawagi S (2007) Domain adaptation of conditional probability models via feature subsetting. In: European Conference on Principles of Data Mining and Knowledge Discovery. Springer, pp 224–235

  26. Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942

    Article  Google Scholar 

  27. Hofmann T, Schölkopf B, Smola A (2008) Kernel methods in machine learning. The annals of statistics:1171–1220

  28. Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(Mar):723–773

    MathSciNet  MATH  Google Scholar 

  29. Motiian S, Piccirilli M, Adjeroh DA, Doretto G (2017) Unified deep supervised domain adaptation and generalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 5715–5725

  30. Wang H, Wang W, Zhang C, Xu F (2014) Cross-domain metric learning based on information theory. In: Twenty-eighth AAAI conference on artificial intelligence

  31. Xu Y, Fang X, Wu J, Li X, Zhang D (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863

    Article  MathSciNet  Google Scholar 

  32. Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1-2):74–93

    Article  MathSciNet  Google Scholar 

  33. Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Thirtieth AAAI conference on artificial intelligence

  34. Shu L, Latecki LJ Transductive domain adaptation with affinity learning. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, ACM, 2015, pp 1903–1906

  35. Herath S, Harandi M, Porikli F (2017) Learning an invariant hilbert space for domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3845–3854

  36. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: 2012 IEEE Conference on Computer vision and pattern recognition (CVPR). IEEE, pp 2066–2073

  37. Zhou W, Kamata S-I (2013) L1-norm based linear discriminant analysis: an application to face recognition. IEICE Trans Inf Syst 96(3):550–558

    Article  Google Scholar 

  38. Yamamoto S, Hosoya Y, Tsumura N, Ogawa-Ochiai K (2012) Principal component analysis for dental shade color. Dent Mater 28(7):736–742

    Article  Google Scholar 

  39. Wan M, Lai Z, Jin Z (2011) Feature extraction using two-dimensional local graph embedding based on maximum margin criterion. Appl Math Comput 217(23):9659–9668

    MathSciNet  MATH  Google Scholar 

  40. Mahadevan S, Mishra B, Ghosh S A unified framework for domain adaptation using metric learning on manifolds, arXiv:1804.10834

  41. Chen J, Liu Y (2011) Locally linear embedding: a survey. Artif Intell Rev 36(1):29–48

    Article  Google Scholar 

  42. He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160

  43. Shi L, Zhang L, Zhao L, Zhang L, Li P, Wu D (2016) Adaptive laplacian eigenmap-based dimension reduction for ocean target discrimination. IEEE Geosci Remote Sens Lett 13(7):902–906

    Article  Google Scholar 

  44. He X, Cai D, Yan S, Zhang H-J (2005) Neighborhood preserving embedding. In: Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, vol 2, pp 1208–1213

  45. Wang S, Lu J, Gu X, Du H, Yang J (2016) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recogn 57:179–189

    Article  Google Scholar 

  46. Nie F, Xiang S, Jia Y, Zhang C (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recogn 42(11):2615–2627

    Article  Google Scholar 

  47. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967

  48. Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE International Conference on Data Mining (ICDM), pp 1129–1134. https://doi.org/10.1109/ICDM.2017.150

  49. Cai D, He X, Han J (2008) Training linear discriminant analysis in linear time. In: 2008 IEEE 24th International Conference on Data Engineering. IEEE, pp 209–217

  50. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 815–823

Download references

Acknowledgments

Dr. Sriparna Saha gratefully acknowledges support from SERB Women in Excellence Award 2018 of Science and Engineering Research Board (SERB) of Department of Science & Technology, Govt. of India for conducting this research.

Author information

Authors and Affiliations

  1. Department of Computer Science, Engineering, Indian Institute of Technology Patna, Patna, India

    Rakesh Kumar Sanodiya, Jimson Mathew & Sriparna Saha

  2. IIEST Shibpur, West Bengal, India

    Piyush Tripathi

Authors
  1. Rakesh Kumar Sanodiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jimson Mathew
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sriparna Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Piyush Tripathi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rakesh Kumar Sanodiya.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanodiya, R.K., Mathew, J., Saha, S. et al. Particle swarm optimization based parameter selection technique for unsupervised discriminant analysis in transfer learning framework. Appl Intell 50, 3071–3089 (2020). https://doi.org/10.1007/s10489-020-01710-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01710-7

Keywords

Search

Navigation