Abstract
Supervised learning models assume that the training and test data are drawn from the same underlying distribution. These algorithms become ineffective when such assumptions are violated. Therefore, domain adaptation methods were proposed to handle data from different, yet correlated distributions. They have been applied in many fields such as computer vision, sentiment analysis, natural language processing and so on. Often in these cases, source domains provide large numbers of labeled data for learning, while target domain may have few labeled data available. In this paper, we explore a new supervised domain adaption method with twin support vector machines called aggregate domain adaptation twin support vector machines, which obtain the classifiers trained on the source and target training data. Then we evaluate the performance of our proposed methods on brain-computer interfaces and sentiment analysis datasets. The experimental results show the effectiveness and the reliability of our proposed methods.
Similar content being viewed by others
References
Pan S, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359
Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92
Klinkenberg R, Joachims T (2000) Detecting concept drift with support vector machines. In: Proceedings of the international conference on machine learning, pp 487–494
Wu P, Dietterich T (2004) Improving SVM accuracy by training on auxiliary data sources. In: Proceedings of the international conference on machine learning, pp 871–878
Liao X, Xue Y, Carin L (2005) Logistic regression with an auxiliary data source. In: Proceedings of the 21st international conference on machine learning, pp 505–512
Daumé III H, Marcu D (2006) Domain adaptation for statistical classifiers. J Artif Intell Res 26:101–126
Blitzer J, Dredze M, Pereira F (2007) Boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 440–447
Ling X, Dai W, Xue G, Yang Q, Yu Y (2008) Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 488–496
Pan S, Tsang I, Kwok J, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210
Satpal S, Sarawagi S (2007) Domain adaptation of conditional probability models via feature subsetting. In: Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases, pp 224–235
Seah C, Tsang I, Ong Y, Lee K (2010) Predictive distribution matching SVM for multi-domain learning. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases, pp 231–247
Pan S, Kwok J, Yang Q (2008) Transfer learning via dimensionality reduction. In: Proceedings of the association for the advancement of artificial intelligence, pp 677–682
Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O (2009) Cross domain distribution adaptation via kernel mapping. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 1027–1036
Shawe-Taylor J, Sun S (2011) A review of optimization methodologies in support vector machines. Neurocomputing 74:3609–3618
Li J, Xu C, Yang W, Sun C, Tao D (2015) Discriminative multi-view interactive image re-ranking. IEEE Trans Image Process 26:3113–3127
Xu C, Tao D, Xu C (2015) Large-margin multi-label causal feature learning. In: Proceedings of the AAAI conference on artificial intelligence, pp 1924–1930
Xu C, Tao D, Li Y, Xu C (2013) Large-margin multi-view Gaussian process for image classification. In: Proceedings of the international conference on internet multimedia computing and service, pp 7–12
Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proceedings of the conference on empiriacal methods in natural language processing, pp 120–128
Daum\(\acute{e}\) III H (2007) Frustratingly easy domain adaptation. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 256–263
Yang J, Yan R, Hauptmann A (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the international conference on multimedia, pp 188–197
Jiang W, Zavesky E, Chang S, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: Proceedings of the international conference on image processing, pp 161–164
Storkey A, Sugiyama M (2006) Mixture regression for covariate shift. In: Proceedings of the conference on advances in neural information processing systems, pp 1337–1344
Huang J, Sola A, Gretton A, Borgwardt K, Schölkopf B (2006) Correcting sample selection bias by unlabeled data. Adv Neural Inf Proces Syst 19:601–608
Crammer K, Kearns M, Wortman J (2008) Learning from multiple sources. J Mach Learn Res 9:1757–1774
Luo P, Zhuang F, Xiong H, Xiong Y, He Q (2008) Transfer learning from multiple source domains via consensus regularization. In: Proceedings of the ACM conference on information and knowledge management, pp 103–112
Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation: learning bounds and algorithms. In: Proceedings of the 22nd annual conference on learning theory, pp 34–47
Schweikert G, Widmer C, Schölkopf B, Rätsch G (2008) An empirical analysis of domain adaptation algorithm for genomic sequence analysis. Adv Neural Inf Process Syst 21:1433–1440
Blitzer J, Crammer K, Kulesza A, Pereira F, Wortman J (2007) Learning bounds for domain adaptation. In: Proceedings of conference on advances in neural information processing systems, pp 129–136
Duan L, Tsang I, Xu D, Maybank S (2009) Domain transfer SVM for video concept detection. In: Proceedings of the international conference computer vision and pattern recognition, pp 1375–1381
Duan L, Tsang I, Xu D (2012) Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell 34:465–479
Duan L, Xu D, Tsang I (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23:504–518
Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification techique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787
Tao J, Chung F, Wang S (2012) On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recognit 45:3962–3984
Venkateswara H, Lade P, Ye J, Panchanathan S (2015) Coupled support vector machines for supervised domain adaptation. In: Proceedings of the ACM international conference on multimedia, pp 1295–1298
Gong M, Zhang K, Liu T, Tao D, Glymour C, Schlkopf B (2016) Domain adaptation with conditional transferable components. In: Proceedings of the 33rd international conference on machine learning, pp 2839–2848
Yaroslav G, Lempitsky V (2014) Unsupervised domain adaptation by backpropagation, arXiv preprint arXiv:1409.7495
Mangasarian O, Wild E (2006) MultisurFace proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74
Jayadeva R, Khemchandani S, Chandra (2007) Twin support vector machines for pattern classification. IEEE Trans on Pattern Anal Mach Intell 74:905–910
Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22:962–968
Ding S, Zhao Y, Qi B, Huang H (2014) An overview on twin support vector machines. Artif Intell Rev 42:245–252
Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543
Xu Y, Pan X, Zhou Z, Yang Z, Zhang Y (2015) Structural least square twin support vector machine for classification. Appl Intell 42:1–10
Xie X, Sun S (2015) Multitask centroid twin support vector machines. Neurocomputing 149:1085–1091
Xie X, Sun S (2015) Multi-view twin support vector machines. Intell Data Anal 19:701–712
Xie X, Sun S (2014) Multi-view Laplacian twin support vector machines. Appl Intell 41:1059–1068
Chen W, Shao Y, Li C, Deng N (2016) MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recognit 52:61–74
Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53
Peng X (2010) An efficient twin support vector machine for regression. Neural Netw 23:365–372
Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17:649–664
Zhang K, Schlkopf B, Muandet K, Wang Z (2013) Domain adaptation under target and conditional shift. J Volcanol Geotherm Res 173:185–195
Liu T, Yang Q, Tao D (2017) Understanding how feature structure transfers in transfer learning. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 2365–2371
Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24:56–59
Yu J, Zhang B, Kuang Z, Lin D, Fan J (2017) Image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans Inf Forensics Secur 12:1005–1016
Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 12:1–11
Yu J, Rui Y, Tao D (2014) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23:2019–2032
Sun S, Zhou J (2014) A review of adaptive feature extraction and classification methods for EEG-based brain-computer interfaces. In: Proceedings of the international joint conference on neural networks, pp 1746–1753
Sun S, Luo C, Chen J (2016) A review of natural language processing techniques for opinion mining systems. Inf Fusion 36:10–25
Acknowledgements
This work is supported by Ningbo University talent project 421703670 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. It is also supported by the Zhejiang Provincial Department of Education under Projects 801700472 and National Natural Science Foundation of China under Projects 61572266 and 61472194.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Xie, X., Sun, S., Chen, H. et al. Domain Adaptation with Twin Support Vector Machines. Neural Process Lett 48, 1213–1226 (2018). https://doi.org/10.1007/s11063-017-9775-3
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-017-9775-3