Skip to main content
SpringerLink
Log in

A linear unsupervised transfer learning by preservation of cluster-and-neighborhood data organization

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

The paper has proposed a linear unsupervised transfer learning (LUTL). Therefore, a cost function has been introduced. In the cost function of the proposed LUTL, the aim is to minimize the difference between the distribution of the transformed source domain (SD) data and the distribution of the target domain (TD) data. In the proposed cost function, it is also targeted to preserve the local structures of the untransformed SD data. Three mechanisms have been proposed for the preservation of local structures in the untransformed SD data: (1) minimization of the distances between the data pairs that are similar to each other in the untransformed SD data, (2) preservation of the clusters emerged in the untransformed SD data and finally (3) their combination. The optimization problem has emerged as a nonlinear one. Two techniques have been introduced to obtain an approximation of the optimal weight matrix. Each technique guarantees to reach a local optimum, but no one guarantees to reach the global solution. While the first method is an iterative one, the second is a relaxed version of the optimization problem. The paper shows also experimentally that the proposed techniques overshadow the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Absil PA, Mahony R, Sepulchre R (2008) Optimization algorithms on matrix manifolds. Princeton University Press, Princeton

    Book  MATH  Google Scholar 

  2. Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed Aug 2013

  3. Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359

    Article  Google Scholar 

  4. Boyd S, Vandenberghe L (2006) Convex optimization. Cambridge University Press, New York

    MATH  Google Scholar 

  5. Caruana R (1997) Multitask learning. Mach Learn 28(1):41–75

    Article  MathSciNet  Google Scholar 

  6. Chen Q, Xue B, Zhang M (2015) Generalisation and domain adaptation in GP with gradient descent for symbolic regression. In: CEC 2015, pp 1137–1144

  7. Chopra S, Balakrishnan S, Gopalan R (2013) Dlid: deep learning for domain adaptation by interpolating between domains. In: ICML workshop on challenges in representation learning

  8. Dai W, Yang Q, Xue GR, Yu Y (2007) Boosting for transfer learning. In: International conference on machine learning, pp 193–200

  9. Duan L, Xu D, Tsang IWH (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23:504–518

    Article  Google Scholar 

  10. Feng L, Ong Y, Lim M, Tsang I (2014) Memetic search with inter-domain learning: a realization between CVRP and CARP. IEEE Trans Evolut Comput. https://doi.org/10.1109/TEVC.2014.2362558

    Google Scholar 

  11. Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: International conference in computer vision, pp 2960–2967

  12. Ghifary M, Kleijn WB, Zhang M (2014) Domain adaptive neural networks for object recognition. In: 13th Pacific Rim international conference on artificial intelligence, pp 898–904

  13. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 2066–2073

  14. Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: an unsupervised approach. In: International conference in computer vision, pp 999–1006

  15. Guo K, Wu S, Xu Y (2017) Face recognition using both visible light image and near-infrared image and a deep network. CAAI Trans Intell Technol 2(1):39–47

    Article  Google Scholar 

  16. Hoffmann H (2007) Kernel PCA for novelty detection. Pattern Recognit 40:863–874

    Article  MATH  Google Scholar 

  17. Jiang W, Zavesky E, Chang SF, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: International conference on image processing, pp 161–164

  18. Jhou I, Liu D, Lee DT, Chang S (2012) Robust visual domain adaptation with low-rank reconstruction. In: CVPR, pp 2168–2175

  19. Li Z, Liu J, Tang J, Lu H (2015) Robust structured subspace learning for data representation. IEEE Trans Pattern Anal Mach Intell 37(10):2085–2098

    Article  Google Scholar 

  20. Li Z, Liu J, Yang Y, Zhou X, Lu H (2014) Clustering-guided sparse structural learning for unsupervised feature selection. IEEE Trans Knowl Data Eng 26(9):2138–2150

    Article  Google Scholar 

  21. Liu H, Hu L, Ma L (2017) Online RGB-D person re-identification based on metric model update. CAAI Trans Intell Technol 2(1):48–55

    Article  Google Scholar 

  22. Löfberg J (2004) YALMIP: a toolbox for modeling and optimization in MATLAB. In: Proceedings of the CACSD conference, Taiwan Taipei

  23. Long M, Wang J (2015) Learning transferable features with deep adaptation networks. CoRR, arXiv:1502.02791

  24. Long M, Wang J, Ding G, Sun J, Yu P (2013) Transfer feature learning with joint distribution adaptation. In: IEEE International conference on computer vision (ICCV), pp 2200–2207

  25. Mencia EL (2010) Multilabel classification in parallel tasks. In: Working notes of the second international workshop on learning from multi-label data, Haifa, Israel, 2010, pp 20–36

  26. Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE conference on computer vision and pattern recognition, conference version of the paper. https://hal.inria.fr/hal-00911179. Accessed Aug 2013

  27. Pan SJ, Tsang I, Kwok J, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22(2):199–210

    Article  Google Scholar 

  28. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

    Article  Google Scholar 

  29. Pezeshki A, Scharf LL, Chong EK (2010) The geometry of linearly and quadratically constrained optimization problems for signal processing and communications. J Frankl Inst 347:818–835

    Article  MathSciNet  MATH  Google Scholar 

  30. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: Computer vision ECCV 2010, series lecture notes in computer science, vol 6314. Springer, Berlin, pp 213–226

  31. Song XP, Huang C, Townshend JR (2017) Improving global land cover characterization through data fusion. Geo-spatial Inf Sci 20(2):141–150

    Article  Google Scholar 

  32. Shi X, Fan W, Ren J (2008) Actively transfer domain knowledge. In: European conference on machine learning, pp 342–357

  33. Sugiyama M, Nakajima S, Kashima H, von Bünau P, Kawanabe M (2007) Direct importance estimation with model selection and its application to covariate shift adaptation. In: Proceedings of neural information processing systems, pp 1962–1965

  34. Sun B, Feng J, Saenko K (2017) Correlation alignment for unsupervised domain adaptation. In: Csurka G (ed) Domain adaptation in computer vision applications. Advances in computer vision and pattern recognition. Springer, Cham

    Google Scholar 

  35. Tagare HD (2011) Notes on optimization on Stiefel manifolds, Tech. Rep., Department of Diagnostic Radiology, Department of Biomedical Engineering, Yale University

  36. Tan F, Li L, Zhang Z, Guo Y (2016) A multi-attribute probabilistic matrix factorization model for personalized recommendation. Pattern Anal Appl 19(3):857–866

    Article  MathSciNet  Google Scholar 

  37. Tsoumakas G, Katakis I, Vlahavas I (2008) Effective and efficient multilabel classification in domains with large number of labels. In Working Notes of the ECML PKDD’08 workshop on mining multidimensional data, Antwerp, Belgium, 2008

  38. Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: maximizing for domain invariance. CoRR, arXiv:1412.3474

  39. Wan C, Pan R, Li J (2011) Bi-weighting domain adaptation for cross-language text classification. In: 22th International joint conference on artificial intelligence, pp 1535–1540

  40. Wei B, Pal C (2010) Cross lingual adaptation: an experiment on sentiment classifications. In: Proceedings of the ACL 2010 conference short papers. Association for Computational Linguistics, pp 258–262

  41. Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Program 142:397–434

    Article  MathSciNet  MATH  Google Scholar 

  42. Xia R, Zong C, Hu X, Cambria E (2013) Feature ensemble plus sample selection: domain adaptation for sentiment classification. IEEE Intell Syst 28(3):10–18

    Article  Google Scholar 

  43. Xue Y, Liao X, Carin L, Krishnapuram B (2007) Multi-task learning for classification with Dirichlet process priors. J Mach Learn Res 8:35–63

    MathSciNet  MATH  Google Scholar 

  44. Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive SVMs. In: International conference on multimedia, pp 188–197

  45. Yang M, Zhang L, Feng X, Zhang D (2011) Fisher discrimination dictionary learning for sparse representation. In: ICCV, pp 543–550

  46. Yang X, Xie L, Han J, Wang Z (2017) Cognitive-affective regulation process for micro-expressions based on Gaussian cloud distribution. CAAI Trans Intell Technol 2(1):56–61

    Article  Google Scholar 

  47. Zhang Y, Yeung D-Y (2010) A convex formulation for learning task relationships in multi-task learning. In: Proceedings of the 26th conference on uncertainty in artificial intelligence, pp 733–742

  48. Zhang Y, Yeung D-Y (2014) A regularization approach to learning task relationships in multitask learning. ACM Trans Knowl Discov Data 8(3):12

    Article  Google Scholar 

  49. Zhao B, Gao L, Liao W, Zhang B (2017) A new kernel method for hyperspectral image feature extraction. Geo-spatial Inf Sci 20(4):309–318

    Article  Google Scholar 

Download references

Acknowledgements

The paper has been extracted from Amin Pirbonyeh’s PhD thesis. He is undersupervision of Hamid Parvin and Vahideh Rezaie. Amin Pirbonyeh’s consultants have been Samad Nejatian and Mehdi Mehrabi.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Shiraz Branch, Islamic Azad University, Shiraz, Iran

    Amin Pirbonyeh & Mehdi Mehrabi

  2. Department of Mathematics, Yasooj Branch, Islamic Azad University, Yasooj, Iran

    Vahideh Rezaie

  3. Young Researchers and Elite Club, Yasooj Branch, Islamic Azad University, Yasooj, Iran

    Vahideh Rezaie & Samad Nejatian

  4. Department of Computer Engineering, Nourabad Mamasani Branch, Islamic Azad University, Nourabad Mamasani, Iran

    Hamid Parvin

  5. Young Researchers and Elite Club, Nourabad Mamasani Branch, Islamic Azad University, Nourabad Mamasani, Iran

    Hamid Parvin

  6. Department of Electrical Engineering, Yasooj Branch, Islamic Azad University, Yasooj, Iran

    Samad Nejatian

Authors
  1. Amin Pirbonyeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vahideh Rezaie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hamid Parvin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samad Nejatian
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mehdi Mehrabi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vahideh Rezaie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pirbonyeh, A., Rezaie, V., Parvin, H. et al. A linear unsupervised transfer learning by preservation of cluster-and-neighborhood data organization. Pattern Anal Applic 22, 1149–1160 (2019). https://doi.org/10.1007/s10044-018-0753-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-018-0753-9

Keywords

Search

Navigation