Abstract
In this paper, a new approach to self-taught learning is proposed. Classification in target task with limited labeled target data gets improved thanks to enormous unlabeled source data. The target and source data can be drawn from different distributions. In the previous approaches, covariate shift assumption is considered in which the marginal distributions p(x) change over domains and the conditional distributions p(y|x) remain the same. In our approach, we propose a new objective function which simultaneously learns a common space ℑ(.) where the conditional distributions over domains p(ℑ(x)|y) remain the same and learns robust SVM classifiers for target task using both source and target data in the new representation. Hence, in the proposed objective function, the hidden label of the source data is also incorporated. We applied the proposed approach on Caltech-256 and MSRC + LMO datasets and compared the performance of our algorithm to the available competing methods. Our method has a superior performance to the successful existing algorithms.










Similar content being viewed by others
References
Ablavsky VH, Becker CJ, Fua P (2012) Transfer learning by sharing support vectors: No. EPFL-REPORT-181360
Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79:151–175
Bickel S, Bruckner M, Scheffer T (2007) Discriminative learning for differing training and test distributions. In: International conference on machine learning, pp 81–88
Borgwardt KM, Gretton A, Rasch MJ, Kriegel HP, Scholkopf B, Smola AJ (2006) Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22:e49–e57
Bosch A, Zisserman A, Munoz X (2007) Representing shape with a spatial pyramid kernel. In: ACM international conference on image and video retrieval
Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification technique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787
Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:27:21–27:27
Chen M, Weinberger KQ, Blitzer J (2011) Co-training for domain adaptation. In: Advances in neural information processing systems (NIPS), pp 2456–2464
Chen Y, Guoping W, Shihai D (2003) Learning with progressive transductive support vector machine. Pattern Recogn Lett 24:1845–1855
Daume III H (2009) Frustratingly easy domain adaptation. arXiv preprint arXiv:0907.1815
Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Duan L, Tsang IW, Xu D, Maybank SJ (2009) Domain transfer svm for video concept detection. In: IEEE conference on computer vision and pattern recognition, pp 1375–1381
Gammerman A, Vovk V, Vapnik V (1998) Learning by transduction. In: Proceedings of uncertainty in artificial intelligence, pp 148–156
Germain P, Habrard A, Laviolette F, Morvant E (2013) A PAC-Bayesian approach for domain adaptation with specialization to linear classifiers. In: International conference on machine learning, pp 738–746
Germain P, Habrard A, Laviolette F, Morvant E (2016) A new PAC-Bayesian perspective on domain adaptation. In: International conference on machine learning
Gong M, Zhang K, Liu T, Tao D, Glymour C, Schölkopf B (2016) Domain adaptation with conditional transferable components. In: International conference on machine learning, pp 2839–2848
Grant M, Boyd S (2012) CVX users’ guide
Griffin G, Holub A, Perona P (2007) Caltech 256 object category dataset. Technical Report UCB/CSD-04-1366: California Institute of Technology
Laviolette F, Marchand M, Roy JF (2011) From PAC-Bayes bounds to quadratic programs for majority votes. In: International conference on machine learning
Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: Computer vision and pattern recognition, pp 2169–2678
Li L, Jin X, Long M (2012) Topic correlation analysis for cross-domain text classification. In: AAAI conference on artificial intelligence
Li S, Li K, Fu Y (2017) Self-taught low-rank coding for visual learning. IEEE Trans Neural Netw Learn Syst 29:645–656
Liu C, Yuen J, Torralba A (2009) Nonparametric scene parsing: label transfer via dense scene alignment. In: Conference on computer vision and pattern recognition
Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110
Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60:91–110
Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer learning using computational intelligence: a survey. Knowl Based Syst 80:14–23
Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation: learning bounds and algorithms. arXiv preprint arXiv:0902.3430
Margolis A (2011) A literature review of domain adaptation with unlabeled data. Washington University, St. Louis, pp 1–42
Morvant E (2015) Domain adaptation of weighted majority votes via perturbed variation-based self-labeling. Pattern Recogn Lett 51:37–43
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. IEEE Trans Pattern Anal Mach Intell 24:971–987
Orabona F, Castellini C, Caputo B, Fiorilla E, Sandini G (2009) Model adaptation with least-squares SVM for hand prosthetics. In: IEEE international conference on robotics and automation
Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359
Quionero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2009) Dataset shift in machine learning. The MIT Press, Cambridge
Raina R, Battle A, Lee H, Packer B, Ng AY (2007) Self-taught learning: transfer learning from unlabeled data. In: International conference on machine learning, pp 759–766
Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) SimpleMKL. J Mach Learn Res 9:2491–2521
Shimodaira H (2000) Improving predictive inference under covariate shift by weighting the log-likelihood function. J Stat Plann Inference 90:227–244
Shotton J, Johnson M, Cipolla R (2008) Semantic texton forests for image categorization and segmentation. In: Conference on computer vision and pattern recognition, pp 1–8
Sugiyama M, Kawanabe M (2012) Machine learning in non-stationary environments: Introduction to covariate shift adaptation. MIT Press, Cambridge
Tommasi T, Orabona F, Caputo B (2010) Safety in numbers: learning categories from few examples with multi model knowledge transfer. In: Conference on computer vision and pattern recognition (CVPR), pp 3081–3088
Tuzel O, Porikli F, Meer P (2007) Human detection via classification on riemannian manifold. In: Conference on computer vision and pattern recognition
Wang H, Nie F, Huang H (2013) Robust and discriminative self-taught learning. In: International conference on machine learning, pp 298–306
Weijer JVD, Schmid C (2006) Coloring local feature extraction. In: European conference on computer vision, pp 334–348
Acknowledgement
This research was in part supported by a grant from the Institute for Research in Fundamental Sciences (IPM) (Grant number CS1395-4-68).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Razzaghi, P. Self-taught support vector machines. Knowl Inf Syst 59, 685–709 (2019). https://doi.org/10.1007/s10115-018-1218-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-018-1218-6