Skip to main content
Log in

Domain Adaptation with Twin Support Vector Machines

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Supervised learning models assume that the training and test data are drawn from the same underlying distribution. These algorithms become ineffective when such assumptions are violated. Therefore, domain adaptation methods were proposed to handle data from different, yet correlated distributions. They have been applied in many fields such as computer vision, sentiment analysis, natural language processing and so on. Often in these cases, source domains provide large numbers of labeled data for learning, while target domain may have few labeled data available. In this paper, we explore a new supervised domain adaption method with twin support vector machines called aggregate domain adaptation twin support vector machines, which obtain the classifiers trained on the source and target training data. Then we evaluate the performance of our proposed methods on brain-computer interfaces and sentiment analysis datasets. The experimental results show the effectiveness and the reliability of our proposed methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Pan S, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359

    Article  Google Scholar 

  2. Sun S, Shi H, Wu Y (2015) A survey of multi-source domain adaptation. Inf Fusion 24:84–92

    Article  Google Scholar 

  3. Klinkenberg R, Joachims T (2000) Detecting concept drift with support vector machines. In: Proceedings of the international conference on machine learning, pp 487–494

  4. Wu P, Dietterich T (2004) Improving SVM accuracy by training on auxiliary data sources. In: Proceedings of the international conference on machine learning, pp 871–878

  5. Liao X, Xue Y, Carin L (2005) Logistic regression with an auxiliary data source. In: Proceedings of the 21st international conference on machine learning, pp 505–512

  6. Daumé III H, Marcu D (2006) Domain adaptation for statistical classifiers. J Artif Intell Res 26:101–126

    Article  MathSciNet  Google Scholar 

  7. Blitzer J, Dredze M, Pereira F (2007) Boom-boxes and blenders: domain adaptation for sentiment classification. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 440–447

  8. Ling X, Dai W, Xue G, Yang Q, Yu Y (2008) Spectral domain transfer learning. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining, pp 488–496

  9. Pan S, Tsang I, Kwok J, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210

    Article  Google Scholar 

  10. Satpal S, Sarawagi S (2007) Domain adaptation of conditional probability models via feature subsetting. In: Proceedings of the 11th European conference on principles and practice of knowledge discovery in databases, pp 224–235

  11. Seah C, Tsang I, Ong Y, Lee K (2010) Predictive distribution matching SVM for multi-domain learning. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases, pp 231–247

    Chapter  Google Scholar 

  12. Pan S, Kwok J, Yang Q (2008) Transfer learning via dimensionality reduction. In: Proceedings of the association for the advancement of artificial intelligence, pp 677–682

  13. Zhong E, Fan W, Peng J, Zhang K, Ren J, Turaga D, Verscheure O (2009) Cross domain distribution adaptation via kernel mapping. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 1027–1036

  14. Shawe-Taylor J, Sun S (2011) A review of optimization methodologies in support vector machines. Neurocomputing 74:3609–3618

    Article  Google Scholar 

  15. Li J, Xu C, Yang W, Sun C, Tao D (2015) Discriminative multi-view interactive image re-ranking. IEEE Trans Image Process 26:3113–3127

    Article  MathSciNet  Google Scholar 

  16. Xu C, Tao D, Xu C (2015) Large-margin multi-label causal feature learning. In: Proceedings of the AAAI conference on artificial intelligence, pp 1924–1930

  17. Xu C, Tao D, Li Y, Xu C (2013) Large-margin multi-view Gaussian process for image classification. In: Proceedings of the international conference on internet multimedia computing and service, pp 7–12

  18. Blitzer J, McDonald R, Pereira F (2006) Domain adaptation with structural correspondence learning. In: Proceedings of the conference on empiriacal methods in natural language processing, pp 120–128

  19. Daum\(\acute{e}\) III H (2007) Frustratingly easy domain adaptation. In: Proceedings of the 45th annual meeting of the assocation for computational linguistics, pp 256–263

  20. Yang J, Yan R, Hauptmann A (2007) Cross-domain video concept detection using adaptive svms. In: Proceedings of the international conference on multimedia, pp 188–197

  21. Jiang W, Zavesky E, Chang S, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: Proceedings of the international conference on image processing, pp 161–164

  22. Storkey A, Sugiyama M (2006) Mixture regression for covariate shift. In: Proceedings of the conference on advances in neural information processing systems, pp 1337–1344

  23. Huang J, Sola A, Gretton A, Borgwardt K, Schölkopf B (2006) Correcting sample selection bias by unlabeled data. Adv Neural Inf Proces Syst 19:601–608

    Google Scholar 

  24. Crammer K, Kearns M, Wortman J (2008) Learning from multiple sources. J Mach Learn Res 9:1757–1774

    MathSciNet  MATH  Google Scholar 

  25. Luo P, Zhuang F, Xiong H, Xiong Y, He Q (2008) Transfer learning from multiple source domains via consensus regularization. In: Proceedings of the ACM conference on information and knowledge management, pp 103–112

  26. Mansour Y, Mohri M, Rostamizadeh A (2009) Domain adaptation: learning bounds and algorithms. In: Proceedings of the 22nd annual conference on learning theory, pp 34–47

  27. Schweikert G, Widmer C, Schölkopf B, Rätsch G (2008) An empirical analysis of domain adaptation algorithm for genomic sequence analysis. Adv Neural Inf Process Syst 21:1433–1440

    Google Scholar 

  28. Blitzer J, Crammer K, Kulesza A, Pereira F, Wortman J (2007) Learning bounds for domain adaptation. In: Proceedings of conference on advances in neural information processing systems, pp 129–136

  29. Duan L, Tsang I, Xu D, Maybank S (2009) Domain transfer SVM for video concept detection. In: Proceedings of the international conference computer vision and pattern recognition, pp 1375–1381

  30. Duan L, Tsang I, Xu D (2012) Domain transfer multiple kernel learning. IEEE Trans Pattern Anal Mach Intell 34:465–479

    Article  Google Scholar 

  31. Duan L, Xu D, Tsang I (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23:504–518

    Article  Google Scholar 

  32. Bruzzone L, Marconcini M (2010) Domain adaptation problems: a DASVM classification techique and a circular validation strategy. IEEE Trans Pattern Anal Mach Intell 32:770–787

    Article  Google Scholar 

  33. Tao J, Chung F, Wang S (2012) On minimum distribution discrepancy support vector machine for domain adaptation. Pattern Recognit 45:3962–3984

    Article  Google Scholar 

  34. Venkateswara H, Lade P, Ye J, Panchanathan S (2015) Coupled support vector machines for supervised domain adaptation. In: Proceedings of the ACM international conference on multimedia, pp 1295–1298

  35. Gong M, Zhang K, Liu T, Tao D, Glymour C, Schlkopf B (2016) Domain adaptation with conditional transferable components. In: Proceedings of the 33rd international conference on machine learning, pp 2839–2848

  36. Yaroslav G, Lempitsky V (2014) Unsupervised domain adaptation by backpropagation, arXiv preprint arXiv:1409.7495

  37. Mangasarian O, Wild E (2006) MultisurFace proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28:69–74

    Article  Google Scholar 

  38. Jayadeva R, Khemchandani S, Chandra (2007) Twin support vector machines for pattern classification. IEEE Trans on Pattern Anal Mach Intell 74:905–910

    Article  Google Scholar 

  39. Shao Y, Zhang C, Wang X, Deng N (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22:962–968

    Article  Google Scholar 

  40. Ding S, Zhao Y, Qi B, Huang H (2014) An overview on twin support vector machines. Artif Intell Rev 42:245–252

    Article  Google Scholar 

  41. Kumar M, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36:7535–7543

    Article  Google Scholar 

  42. Xu Y, Pan X, Zhou Z, Yang Z, Zhang Y (2015) Structural least square twin support vector machine for classification. Appl Intell 42:1–10

    Article  Google Scholar 

  43. Xie X, Sun S (2015) Multitask centroid twin support vector machines. Neurocomputing 149:1085–1091

    Article  Google Scholar 

  44. Xie X, Sun S (2015) Multi-view twin support vector machines. Intell Data Anal 19:701–712

    Article  Google Scholar 

  45. Xie X, Sun S (2014) Multi-view Laplacian twin support vector machines. Appl Intell 41:1059–1068

    Article  Google Scholar 

  46. Chen W, Shao Y, Li C, Deng N (2016) MLTSVM: a novel twin support vector machine to multi-label learning. Pattern Recognit 52:61–74

    Article  Google Scholar 

  47. Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53

    Article  Google Scholar 

  48. Peng X (2010) An efficient twin support vector machine for regression. Neural Netw 23:365–372

    Article  Google Scholar 

  49. Xie J, Hone K, Xie W, Gao X, Shi Y, Liu X (2013) Extending twin support vector machine classifier for multi-category classification problems. Intell Data Anal 17:649–664

    Google Scholar 

  50. Zhang K, Schlkopf B, Muandet K, Wang Z (2013) Domain adaptation under target and conditional shift. J Volcanol Geotherm Res 173:185–195

    Google Scholar 

  51. Liu T, Yang Q, Tao D (2017) Understanding how feature structure transfers in transfer learning. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 2365–2371

  52. Hong C, Yu J, Wan J, Tao D, Wang M (2015) Multimodal deep autoencoder for human pose recovery. IEEE Trans Image Process 24:56–59

    Article  MathSciNet  Google Scholar 

  53. Yu J, Zhang B, Kuang Z, Lin D, Fan J (2017) Image privacy protection by identifying sensitive objects via deep multi-task learning. IEEE Trans Inf Forensics Secur 12:1005–1016

    Article  Google Scholar 

  54. Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern 12:1–11

    Google Scholar 

  55. Yu J, Rui Y, Tao D (2014) Click prediction for web image reranking using multimodal sparse coding. IEEE Trans Image Process 23:2019–2032

    Article  MathSciNet  Google Scholar 

  56. Sun S, Zhou J (2014) A review of adaptive feature extraction and classification methods for EEG-based brain-computer interfaces. In: Proceedings of the international joint conference on neural networks, pp 1746–1753

  57. Sun S, Luo C, Chen J (2016) A review of natural language processing techniques for opinion mining systems. Inf Fusion 36:10–25

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Ningbo University talent project 421703670 as well as programs sponsored by K.C. Wong Magna Fund in Ningbo University. It is also supported by the Zhejiang Provincial Department of Education under Projects 801700472 and National Natural Science Foundation of China under Projects 61572266 and 61472194.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xijiong Xie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, X., Sun, S., Chen, H. et al. Domain Adaptation with Twin Support Vector Machines. Neural Process Lett 48, 1213–1226 (2018). https://doi.org/10.1007/s11063-017-9775-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-017-9775-3

Keywords

Navigation