Skip to main content
Log in

Self-taught support vector machines

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

In this paper, a new approach to self-taught learning is proposed. Classification in target task with limited labeled target data gets improved thanks to enormous unlabeled source data. The target and source data can be drawn from different distributions. In the previous approaches, covariate shift assumption is considered in which the marginal distributions p(x) change over domains and the conditional distributions p(y|x) remain the same. In our approach, we propose a new objective function which simultaneously learns a common space ℑ(.) where the conditional distributions over domains p(ℑ(x)|y) remain the same and learns robust SVM classifiers for target task using both source and target data in the new representation. Hence, in the proposed objective function, the hidden label of the source data is also incorporated. We applied the proposed approach on Caltech-256 and MSRC + LMO datasets and compared the performance of our algorithm to the available competing methods. Our method has a superior performance to the successful existing algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://www.cse.wustl.edu/~mchen/code/coda.tar, http://www.cse.ust.hk/TL/code/TCA-AAAI2012-version2.0.zip.

References

  1. Ablavsky VH, Becker CJ, Fua P (2012) Transfer learning by sharing support vectors: No. EPFL-REPORT-181360

  2. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79:151–175

    Article  MathSciNet  Google Scholar 

  3. Bickel S, Bruckner M, Scheffer T (2007) Discriminative learning for differing training and test distributions. In: International conference on machine learning, pp 81–88

  4. Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Scholkopf B, Smola AJ (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22:e49–e57

    Article  Google Scholar 

  5. Bosch A, Zisserman A, Munoz X (2007) Representing shape with a spatial pyramid kernel. In: ACM international conference on image and video retrieval

  6. Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787

    Article  Google Scholar 

  7. Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:27:21–27:27

    Article  Google Scholar 

  8. Chen M, Weinberger KQ, Blitzer J (2011) Co-training for domain adaptation. In: Advances in neural information processing systems (NIPS), pp 2456–2464

  9. Chen Y, Guoping W, Shihai D (2003) Learning with progressive transductive support vector machine. Pattern Recogn Lett 24:1845–1855

    Article  Google Scholar 

  10. Daume III H (2009) Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815

  11. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  12. Duan L, Tsang IW, Xu D, Maybank SJ (2009) Domain transfer svm for video concept detection. In: IEEE conference on computer vision and pattern recognition, pp 1375–1381

  13. Gammerman A, Vovk V, Vapnik V (1998) Learning by transduction. In: Proceedings of uncertainty in artificial intelligence, pp 148–156

  14. Germain P, Habrard A, Laviolette F, Morvant E (2013) A PAC-Bayesian approach for domain adaptation with specialization to linear classifiers. In: International conference on machine learning, pp 738–746

  15. Germain P, Habrard A, Laviolette F, Morvant E (2016) A new PAC-Bayesian perspective on domain adaptation. In: International conference on machine learning

  16. Gong M, Zhang K, Liu T, Tao D, Glymour C, Schölkopf B (2016) Domain adaptation with conditional transferable components. In: International conference on machine learning, pp 2839–2848

  17. Grant M, Boyd S (2012) CVX users’ guide

  18. Griffin G, Holub A, Perona P (2007) Caltech 256 object category dataset. Technical Report UCB/CSD-04-1366: California Institute of Technology

  19. Laviolette F, Marchand M, Roy JF (2011) From PAC-Bayes bounds to quadratic programs for majority votes. In: International conference on machine learning

  20. Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Computer vision and pattern recognition, pp 2169–2678

  21. Li L, Jin X, Long M (2012) Topic correlation analysis for cross-domain text classification. In: AAAI conference on artificial intelligence

  22. Li S, Li K, Fu Y (2017) Self-taught low-rank coding for visual learning. IEEE Trans Neural Netw Learn Syst 29:645–656

    Article  MathSciNet  Google Scholar 

  23. Liu C, Yuen J, Torralba A (2009) Nonparametric scene parsing: label transfer via dense scene alignment. In: Conference on computer vision and pattern recognition

  24. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  25. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60:91–110

    Article  Google Scholar 

  26. Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer learning using computational intelligence: a survey. Knowl Based Syst 80:14–23

    Article  Google Scholar 

  27. Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation: learning bounds and algorithms. arXiv preprint arXiv:0902.3430

  28. Margolis A (2011) A literature review of domain adaptation with unlabeled data. Washington University, St. Louis, pp 1–42

    Google Scholar 

  29. Morvant E (2015) Domain adaptation of weighted majority votes via perturbed variation-based self-labeling. Pattern Recogn Lett 51:37–43

    Article  Google Scholar 

  30. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. IEEE Trans Pattern Anal Mach Intell 24:971–987

    Article  MATH  Google Scholar 

  31. Orabona F, Castellini C, Caputo B, Fiorilla E, Sandini G (2009) Model adaptation with least-squares SVM for hand prosthetics. In: IEEE international conference on robotics and automation

  32. Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  33. Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning. The MIT Press, Cambridge

    Google Scholar 

  34. Raina R, Battle A, Lee H, Packer B, Ng AY (2007) Self-taught learning: transfer learning from unlabeled data. In: International conference on machine learning, pp 759–766

  35. Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521

    MathSciNet  MATH  Google Scholar 

  36. Shimodaira H (2000) Improving predictive inference under covariate shift by weighting the log-likelihood function. J Stat Plann Inference 90:227–244

    Article  MathSciNet  MATH  Google Scholar 

  37. Shotton J, Johnson M, Cipolla R (2008) Semantic texton forests for image categorization and segmentation. In: Conference on computer vision and pattern recognition, pp 1–8

  38. Sugiyama M, Kawanabe M (2012) Machine learning in non-stationary environments: Introduction to covariate shift adaptation. MIT Press, Cambridge

    Book  Google Scholar 

  39. Tommasi T, Orabona F, Caputo B (2010) Safety in numbers: learning categories from few examples with multi model knowledge transfer. In: Conference on computer vision and pattern recognition (CVPR), pp 3081–3088

  40. Tuzel O, Porikli F, Meer P (2007) Human detection via classification on riemannian manifold. In: Conference on computer vision and pattern recognition

  41. Wang H, Nie F, Huang H (2013) Robust and discriminative self-taught learning. In: International conference on machine learning, pp 298–306

  42. Weijer JVD, Schmid C (2006) Coloring local feature extraction. In: European conference on computer vision, pp 334–348

Download references

Acknowledgement

This research was in part supported by a grant from the Institute for Research in Fundamental Sciences (IPM) (Grant number CS1395-4-68).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Parvin Razzaghi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (RAR 1984 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Razzaghi, P. Self-taught support vector machines. Knowl Inf Syst 59, 685–709 (2019). https://doi.org/10.1007/s10115-018-1218-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-018-1218-6

Keywords

Navigation