Abstract
Transfer learning (TL) method has captured an attractive presence because it facilitates the learning ability in the target domain by acquiring knowledge from well-established source domains. To gain strong knowledge from the source domain, it is important to narrow down the distribution difference between the source and the target domains. For this purpose, it is necessary to consider the objectives such as preserving the discriminative information, preserving the original similarity of the source and the target domain data, maximizing the variance of the target domain, and preserving marginal and conditional distribution at the same time. Furthermore, some existing TL methods use only original feature data, so there is a threat of degenerated feature transformation. To overcome all these limitations, in this paper, a novel feature selection-based transfer learning approach using particle swarm optimization (PSO) for unsupervised transfer learning (FSUTL-PSO) is implemented. In FSUTL-PSO, we incorporate all such objectives into one fitness function and select common good features from the source and target domains based on the fitness function for eliminating the threat of degenerated features. Extensive experiments have been done on all possible tasks of Office+Caltech and PIE Face datasets and our proposed method FSUTL-PSO has shown significant improvement over the existing transfer or non-transfer learning methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amid E, Gionis A, Ukkonen A (2015) A kernel-learning approach to semi-supervised clustering with relative distance comparisons. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, New York, pp 219–234
Baktashmotlagh M, Harandi M, Salzmann M (2016) Distribution-matching embedding for visual domain adaptation. J Mach Learn Res 17(1):3760–3789
Chen J, Liu Y (2011) Locally linear embedding: a survey. Artif Intell Rev 36(1):29–48
Ding Z, Fu Y (2016) Robust transfer metric learning for image classification. IEEE Trans Image Process 26(2):660–670
Ding Z, Fu Y (2017) Robust transfer metric learning for image classification. IEEE Trans Image Process 26(2):660–670
Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: MHS’95 Proceedings of the sixth international symposium on micro machine and human science. IEEE, pp 39–43
Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision, pp 2960–2967
Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2066–2073
Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(Mar):723–773
He X, Niyogi P (2004) Locality preserving projections. In: Advances in neural information processing systems, pp 153–160
Herath S, Harandi M, Porikli F (2017) Learning an invariant hilbert space for domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3845–3854
Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat 1171–1220
Huang K-K, Dai D-Q, Ren C-X (2017) Regularized coplanar discriminant analysis for dimensionality reduction. Pattern Recognit 62:87–98
Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554
Ionescu B, Lupu M, Rohm M, Gînsca AL, Müller H (2018) Datasets column: diversity and credibility for social images and image retrieval. ACM SIGMultimedia Records 9(3):7
Kennedy J (1997) The particle swarm: social adaptation of knowledge. In: Proceedings of 1997 IEEE international conference on evolutionary computation (ICEC’97). IEEE, pp 303–308
Kennedy J, Eberhart R (1995) Particle swarm optimization (pso). In: Proceedings of IEEE international conference on neural networks, Perth, Australia, pp 1942–1948
Li H, Jiang T, Zhang K (2004) Efficient and robust feature extraction by maximum margin criterion. In: Advances in neural information processing systems, pp 97–104
Long M, Wang J, Ding G, Pan SJ, Philip SY (2014a) Adaptation regularization: a general framework for transfer learning. IEEE Trans Knowl Data Eng 26(5):1076–1089
Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207
Long M, Wang J, Ding G, Sun J, Yu PS (2014b) Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1410–1417
Luo L, Chen L, Hu S, Lu Y, Wang X (2017) Discriminative and geometry aware unsupervised domain adaptation. arXiv preprint arXiv:1712.10042
Nguyen BH, Xue B, Andreae P (2018) A particle swarm optimization based feature selection approach to transfer learning in classification. In: Proceedings of the genetic and evolutionary computation conference, pp 37–44
Nie F, Xiang S, Jia Y, Zhang C (2009) Semi-supervised orthogonal discriminant analysis via label propagation. Pattern Recognit 42(11):2615–2627
Pan SJ, Tsang IW, Kwok JT, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210
Patel VM, Gopalan R, Li R, Chellappa R (2015) Visual domain adaptation: A survey of recent advances. IEEE Signal Processing Mag 32(3):53–69
Sanodiya RK, Mathew J, Saha S, Thalakottur MD (2019) A new transfer learning algorithm in semi-supervised setting. IEEE Access 7:42956–42967
Satpal, S. and Sarawagi, S. (2007). Domain adaptation of conditional probability models via feature subsetting. In: European conference on principles of data mining and knowledge discovery. Springer, New York, pp 224–235
Shao M, Kit D, Fu Y (2014) Generalized transfer subspace learning through low-rank constraint. Int J Comput Vis 109(1–2):74–93
Shi L, Zhang L, Zhao L, Zhang L, Li P, Wu D (2016) Adaptive Laplacian eigenmap-based dimension reduction for ocean target discrimination. IEEE Geosci Remote Sens Lett 13(7):902–906
Shu L, Latecki LJ (2015) Transductive domain adaptation with affinity learning. In: Proceedings of the 24th ACM international on conference on information and knowledge management. ACM, pp 1903–1906
Si S, Tao D, Geng B (2010) Bregman divergence-based regularization for transfer subspace learning. IEEE Trans Knowl Data Eng 22(7):929–942
Sun B, Feng J, Saenko K (2016) Return of frustratingly easy domain adaptation. In: Thirtieth AAAI conference on artificial intelligence
Sun B, Saenko K (2015) Subspace distribution alignment for unsupervised domain adaptation. BMVC 4:24–1
Uzair M, Mian A (2017) Blind domain adaptation with augmented extreme learning machine features. IEEE Trans Cybern 47(3):651–660
Wang H, Wang W, Zhang C, Xu F (2014) Cross-domain metric learning based on information theory. In: Twenty-Eighth AAAI conference on artificial intelligence
Wang J, Chen Y, Hao S, Feng W, Shen Z (2017) Balanced distribution adaptation for transfer learning. In: 2017 IEEE international conference on data mining (ICDM), pp 1129–1134
Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: 2018 ACM multimedia conference on multimedia conference. ACM, pp 402–410
Wang S, Lu J, Gu X, Du H, Yang J (2016) Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognit 57:179–189
Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10(Feb):207–244
Xu Y, Fang X, Wu J, Li X, Zhang D (2016) Discriminative transfer subspace learning via low-rank and sparse representation. IEEE Trans Image Process 25(2):850–863
Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171
Yan K, Kou L, Zhang D (2018) Learning domain-invariant subspace using domain features and independence maximization. IEEE Trans Cybern 48(1):288–299
Zhang J, Li W, Ogunbona P (2017a) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1859–1867
Zhang J, Li W, Ogunbona P (2017b) Transfer learning for cross-dataset recognition: a survey. arXiv preprint arXiv:1705.04396
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study
Additional information
Communicated by V. Loia.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Sanodiya, R.K., Tiwari, M., Mathew, J. et al. A particle swarm optimization-based feature selection for unsupervised transfer learning. Soft Comput 24, 18713–18731 (2020). https://doi.org/10.1007/s00500-020-05105-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-020-05105-1