Skip to main content
SpringerLink
Log in

Metric transfer learning via geometric knowledge embedding

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The usefulness of metric learning in image classification has been proven and has attracted increasing attention in recent research. In conventional metric learning, it is assumed that the source and target instances are distributed identically, however, real-world problems may not have such an assumption. Therefore, for better classifying, we need abundant labeled images, which are inaccessible due to the high cost of labeling. In this way, the knowledge transfer could be utilized. In this paper, we present a metric transfer learning approach entitled as “Metric Transfer Learning via Geometric Knowledge Embedding (MTL-GKE)” to actuate metric learning in transfer learning. Specifically, we learn two projection matrices for each domain to project the source and target domains to a new feature space. In the new shared sub-space, Mahalanobis distance metric is learned to maximize inter-class and minimize intra-class distances in target domain, while a novel instance reweighting scheme based on the graph optimization is applied, simultaneously, to employ the weights of source samples for distribution matching. The results of different experiments on several datasets on object and handwriting recognition tasks indicate the effectiveness of the proposed MTL-GKE compared to other state-of-the-arts methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. L’heureux A, Grolinger K, Elyamany HF, Capretz MAM (2017) Machine learning with big data: Challenges and approaches. IEEE Access 5:7776–7797

    Article  Google Scholar 

  2. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22 (10):1345–1359

    Article  Google Scholar 

  3. Weiss K, Khoshgoftaar TM, Wang D (2016) A survey of transfer learning. J Big Data 3 (1):9

    Article  Google Scholar 

  4. Xu Y, Pan SJ, Xiong H, Wu Q, Luo R, Min H, Song H (2017) A unified framework for metric transfer learning. IEEE Trans Knowl Data Eng 29(6):1158–1171

    Article  Google Scholar 

  5. Chattopadhyay R, Sun Q, Fan W, Davidson I, Panchanathan S, Ye J (2012) Multisource domain adaptation and its application to early detection of fatigue. ACM Trans Knowl Discov Data (TKDD) 6(4):18

    Google Scholar 

  6. Huang J, Gretton A, Borgwardt K, Schölkopf B, Smola AJ (2007) Correcting sample selection bias by unlabeled data. In: Advances in neural information processing systems, pp 601–608

  7. Jiang J, Zhai C (2007) Instance weighting for domain adaptation in nlp. In: Proceedings of the 45th annual meeting of the association of computational linguistics, pp 264–271

  8. Yi Y, Doretto G (2010) Boosting for transfer learning with multiple sources. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE 2010, pp 1855–1862

  9. Margolis A (2011) A literature review of domain adaptation with unlabeled data. Technical Report, pp 1–42

  10. Pan SJ, Tsang IW, Kwok JT, Yang Q (2010) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  11. Yu Z, Yeung D-Y (2010) Transfer metric learning by learning task relationships. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. . ACM, pp 1199–1208

  12. Hu J, Lu J, Tan Y-P (2015) Deep transfer metric learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 325–333

  13. Nguyen B, De Baets B (2019) Kernel-based distance metric learning for supervised k-means clustering. IEEE transactions on neural networks and learning systems

  14. Tahmoresnezhad J, Hashemi S (2016) An efficient yet effective random partitioning and feature weighting approach for transfer learning. Int J Pattern Recogn Artif Intell 30(02):1651003

  15. Tahmoresnezhad J, Hashemi S (2015) Common feature extraction in multi-source domains for transfer learning. In: 2015 7th Conference on Information and Knowledge Technology (IKT). IEEE, pp 1–5

  16. Hal Daumé III (2009) Frustratingly easy domain adaptation. arXiv:0907.1815

  17. Duan L, Tsang IW, Xu D (2012) Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell 34(3):465– 479

    Article  Google Scholar 

  18. Long M, Wang J, Ding G, Sun J, Yu PS (2013) Transfer feature learning with joint distribution adaptation. In: Proceedings of the IEEE international conference on computer vision, pp 2200–2207

  19. Cao B, Ni X, Sun J-T, Wang G, Yang Q (2011) Distance metric learning under covariate shift. In: Twenty-Second International Joint Conference on Artificial Intelligence

  20. Shimodaira H (2000) Improving predictive inference under covariate shift by weighting the log-likelihood function. J Stat Plann Inference 90(2):227–244

    Article  MathSciNet  MATH  Google Scholar 

  21. Tahmoresnezhad J, Hashemi S (2017) Visual domain adaptation via transfer feature learning. Knowl Inf Syst 50(2):585– 605

    Article  Google Scholar 

  22. Sugiyama M, Nakajima S, Kashima H, Buenau PV, Kawanabe M (2008) Direct importance estimation with model selection and its application to covariate shift adaptation. In: Advances in neural information processing systems, pp 1433–1440

  23. Li J, Lu K, Zi H, Zhu L, Shen HT (2018) Transfer independently together: a generalized framework for domain adaptation. IEEE Trans Cybern 49(6):2144–2155

    Article  Google Scholar 

  24. De Maesschalck R, Jouan-Rimbaud D, Massart DL (2000) The mahalanobis distance. Chem Intel Labor Syst 50(1):1–18

    Article  Google Scholar 

  25. Soleimani A, Araabi BN, Fouladi K (2016) Deep multitask metric learning for offline signature verification. Pattern Recogn Lett 80:84–90

    Article  Google Scholar 

  26. Chen Y, Wang N, Zhang Z (2018) Darkrank: Accelerating deep metric learning via cross sample similarities transfer. In: Thirty-Second AAAI Conference on Artificial Intelligence

  27. Ding Z, Fu Y (2016) Robust transfer metric learning for image classification. IEEE Trans Image Process 26(2):660–670

    Article  MathSciNet  MATH  Google Scholar 

  28. Yu J, Wang M, Tao D (2012) Semisupervised multiview distance metric learning for cartoon synthesis. IEEE Trans Image Process 21(11):4636–4648

    Article  MathSciNet  MATH  Google Scholar 

  29. Zhu Y, Chen Y, Lu Z, Pan SJ, Xue G-R, Yu Y, Yang Q (2011) Heterogeneous transfer learning for image classification. In: Twenty-Fifth AAAI Conference on Artificial Intelligence

  30. Singh AP, Gordon GJ (2008) Relational learning via collective matrix factorization. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 650–658

  31. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition. IEEE 2012, pp 2066–2073

  32. Wang W, Wang H, Zhang C, Xu F (2015) Transfer feature representation via multiple kernel learning. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. AAAI Press, pp 3073–3079

  33. Smola A, Gretton A, Song L, Schölkopf B (2007) A hilbert space embedding for distributions. In International Conference on Algorithmic Learning Theory. Springer, pp 13–31

  34. Xu P, Deng Z, Wang J, Zhang Q, Choi K-S, Wang S (2019) Transfer representation learning with tsk fuzzy system. IEEE Transactions on Fuzzy Systems

  35. Wang L-X (1999) Analysis and design of hierarchical fuzzy systems. IEEE Trans Fuzzy Syst 7 (5):617–624

    Article  Google Scholar 

  36. Long M, Cao Y, Cao Z, Wang J, Jordan MI (2018) Transferable representation learning with deep adaptation networks. IEEE Trans Pattern Anal Mach Intell 41(12):3071–3085

    Article  Google Scholar 

  37. Rossiello G, Gliozzo A, Glass M (2019) Learning to transfer relational representations through analogy. In: proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 10015–10016

  38. Gong B, Grauman K, Sha F (2013) Connecting the dots with landmarks Discriminatively learning domain-invariant features for unsupervised domain adaptation. In: International Conference on Machine Learning, pp 222–230

  39. Aljundi R, Emonet R, Muselet D, Sebban M (2015) Landmarks-based kernelized subspace alignment for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 56–63

  40. Wang J, Lu H, Plataniotis KN, Lu J (2009) Gaussian kernel optimization for pattern classification. Pattern Recogn 42(7):1237–1247

    Article  MATH  Google Scholar 

  41. Hershey JR, Olsen PA (2007) Approximating the kullback leibler divergence between gaussian mixture models. In: IEEE International Conference on Acoustics, Speech and Signal processing-ICASSP’07, vol 4. IEEE 2007, pp IV–317

  42. Gretton A, Borgwardt KM, Rasch M, Schölkopf B, Smola AJ (2007) A kernel approach to comparing distributions. In: Proceedings of the National Conference on Artificial Intelligence, vol 22, pp 1637. AAAI Press; MIT Press, Menlo Park

  43. Holland SM (2008) Principal components analysis (pca). Department of Geology, University of Georgia, Athens, pp 30602–2501

  44. Yan S, Xu D, Zhang B, Zhang H-J, Yang Q, Lin S (2006) Graph embedding and extensions: A general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  45. Guattery S, Miller GL (2000) Graph embeddings and laplacian eigenvalues. SIAM J Matrix Anal Appl 21(3):703–723

    Article  MathSciNet  MATH  Google Scholar 

  46. Tsai Y-HH, Yeh Y-R, Wang Y-CF (2016) Learning cross-domain landmarks for heterogeneous domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5081–5090

  47. Liao X, Ya X, Lawrence Carin. (2005) Logistic regression with an auxiliary data source. In: Proceedings of the 22nd international conference on Machine learning. ACM, pp 505–512

  48. Weinberger KQ, Tesauro G (2007) Metric learning for kernel regression. In: Artificial Intelligence and Statistics, pp 612–619

  49. Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset. California Institute of Technology

  50. Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  51. LeCun Y, Bottou L, Bengio Y, Haffner P et al (1998) Gradient-based learning applied to document recognitio. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  52. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  53. Wang H, Wang W, Zhang C, Xu F (2014) Cross-domain metric learning based on information theory. In: Twenty-Eighth AAAI Conference on Artificial Intelligence

  54. Davis JV, Kulis B, Jain P, Sra S, Dhillon IS (2007) Information-theoretic metric learning. In: Proceedings of the 24th international conference on Machine learnin. ACM, pp 209–216

  55. Lu K, Kou Y, Zhang D (2017) Learning domain-invariant subspace using domain features and independence maximization. IEEE Trans Cybern 48(1):288–299

    Google Scholar 

  56. Sanodiya RKx, Mathew J (2019) A framework for semi-supervised metric transfer learning on manifolds. Knowl-Based Syst 176:1–14

    Article  Google Scholar 

  57. Sanodiya RK, Mathew J, Saha S, Thalakottur MD (2019) A new transfer learning algorithm in semi-supervised setting. IEEE Access 7:42956–42967

    Article  Google Scholar 

  58. Mahadevan S, Mishra B, Ghosh S (2018) A unified framework for domain adaptation using metric learning on manifolds. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, pp 843–860

  59. Chen M, Xu Z, Weinberger K, Sha F (2012) Marginalized denoising autoencoders for domain adaptation. arXiv:1206.468

  60. Hoffman J, Rodner E, Donahue J, Darrell T, Saenko K (2013) Efficient learning of domain-invariant image representation. arXiv:1301.3224

Download references

Author information

Authors and Affiliations

  1. Faculty of IT, Computer Engineering, Urmia University of Technology, Urmia, Iran

    Mahya Ahmadvand & Jafar Tahmoresnezhad

Authors
  1. Mahya Ahmadvand
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Tahmoresnezhad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jafar Tahmoresnezhad.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmadvand, M., Tahmoresnezhad, J. Metric transfer learning via geometric knowledge embedding. Appl Intell 51, 921–934 (2021). https://doi.org/10.1007/s10489-020-01853-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01853-7

Keywords

Search

Navigation