Abstract
Semi-supervised support vector machine is a popular method in the research area of machine learning. Considering a large amount of unlabeled data points in real-life world, the semi-supervised support machine has the ability of good generalization for dealing with nonlinear classification problems. In this paper, a proximal quadratic surface support vector machine model is proposed for semi-supervised binary classification. The main advantage of our new model is that the proximal quadratic surfaces are constructed directly for nonlinear classification instead of using the kernel function, which avoids the tasks of choosing kernels and tuning their parameters. We reformulate this proposed model as an unconstrained mixed-integer quadratic programming problem. Semi-definite relaxation is then adopted, and a primal alternating direction method is further proposed for fast computation. We test the proposed method on some artificial and public benchmark data sets. Preliminary results indicate that our method outperforms some well-known methods for semi-supervised classification in terms of the efficiency and classifying accuracy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Astorino A, Fuduli A (2007) Nonsmooth optimization techniques for semisupervised classification. IEEE Trans Pattern Anal 29:2135–2142
Astorino A, Fuduli A (2015a) Semisupervised spherical separation. Appl Math Model 39:6351–6358
Astorino A, Fuduli A (2015b) Support vector machine polyhedral separability in semisupervised learning. J Optim Theory Appl 164:1039–1050
Bai Y, Yan X (2016) Conic relaxation for semi-supervised support vector machines. J Optim Theory Appl 169:299–313
Bai Y, Chen Y, Niu B (2012) SDP relaxation for semi-supervised support vector machine. Pac J Optim 8(1):3–14
Bai Y, Niu B, Chen Y (2013) New SDP models for protein homology detection with semi-supervised SVM. Optimization 62(4):561–572
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434
Bennett K, Demiriz A (1999) Semi-supervised support vector machines. In: Kearns MJ, Solla SA, Cohn DA (eds) Advances in neural information processing systems 11. MIT Press, Cambridge, pp 368–374
Cao L, Tay FEH (2003) Support vector machine with adaptive parameters in financial time series forecasting. IEEE Trans Neural Netw Learn Syst 14(6):1506–1518
Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, pp 57–64
Chapelle O, Schölkopf B, Zien A (2006) Semi-supervised learning. MIT Press, Cambridge
Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233
Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712
Dagher I (2008) Quadratic kernel-free non-linear support vector machine. J Glob Optim 4(1):15–30
De Bie T, Cristianini N (2004) Convex methods for transduction. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems 16. MIT Press, Cambridge, pp 73–80
Deng N, Tian Y, Zhang C (2012) Support vector machines-optimization based theory, algorithms and extensions. CRC Press, Boca Raton
Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of 7th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 77–86
Grant M, Boyd S, Ye YY (2016) CVX: Matlab software for disciplined convex programming (version 2.1). http://cvxr.com/cvx/
He B, Xu M, Yuan X (2011) Solving large-scale least squares semidefinite programming by alternating direction methods. SIAM J Matrix Anal Appl 32(1):136–152
Jayadeva RK, Suresh C (2007) Fuzzy multi-category proximal support vector classification via generalized eigenvalues. Soft Comput 11(7):679–685
Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of 16th international conference on machine learning, pp 200–209
Luo J (2014) Quadratic surface support vector machines with applications. Ph.D. thesis, North Carolina State University
Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12:1149–1184
Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 130–136
Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53
Sindhwani V, Keerthi SS, Chapelle O (2006) Deterministic annealing for semi-supervised kernel machines. In: Proceedings of 23rd international conference on machine learning, pp 841–848
Sun J, Zhang S (2010) A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur J Oper Res 207(3):1210–1220
Sun L, Jing L, Xia XD (2006) A new proximal support vector machine for semi-supervised classification. In: Wang J, Yi Z, Zurada JM, Lu B, Yin H (eds) Advances in neural networks-ISNN 2006, Lecture notes in computer science, vol 3971. Springer, Heidelburg, pp 1076–1082
Sun D, Toh KC, Yang L (2015) A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J Optim 25(2):882–915
Tian Y, Luo J (2017) A new branch-and-bound approach to semi-supervised support vector machine. Soft Comput 21(1):245–254
Valizadegan H, Jin R (2006) Generalized maximum margin clustering and unsupervised kernel learning. In: Schölkopf B, Platt J, Hofmann T (eds) Advances in neural information processing systems 19. MIT Press, Cambridge, pp 1417–1424
Wen Z, Goldfarb D, Yin W (2010) Alternating direction augmented Lagrangian methods for semidefinite programming. Math Prog Comp 2(3–4):203–230
Xu L, Schuurmans D (2005) Unsupervised and semi-supervised multi-class support vector machines. In: Proceedings of 20th National conference on artificial intelligence, pp 904–910
Xu Z, Jin R, Zhu J, King I, Lyu M (2008) Efficient convex relaxation for transductive support vector machine. In: Platt JC, Koller D, Singer Y, Roweis S (eds) Advances in neural information processing systems 20. MIT Press, Cambridge, pp 1641–1648
Yan X, Bai Y, Fang SC, Luo J (2016) A kernel-free quadratic surface support vector machine for semi-supervised learning. J Oper Res Soc 67(7):1001–1011
Yang J, Deng J, Li S, Hao Y (2017) Improved traffic detection with support vector machine based on restricted boltzmann machine. Soft Comput. doi:10.1007/s00500-015-1994-9
Zhao B, Wang F, Zhang C (2008) CutS3VM: a fast semi-supervised SVM algorithm. In: Proceedings of 4th ACM SIGKDD international conference on knowledge discovery and data mining, pp 830–838
Acknowledgements
This research was partially supported by the National Natural Science Foundation of China (Grant Number 11371242) and the US Army Research Office (Grant Number W911NF-15-1-0223).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All the authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Yan, X., Bai, Y., Fang, SC. et al. A proximal quadratic surface support vector machine for semi-supervised binary classification. Soft Comput 22, 6905–6919 (2018). https://doi.org/10.1007/s00500-017-2751-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-017-2751-z