Abstract
Since labeling all the samples by the user is time-consuming and fastidious, we often obtain a large amount of unlabeled examples and only a small number of labeled examples in classification. In this context, the classification is called semi-supervised learning. In this paper, we propose a novel semi-supervised learning methodology, named Laplacian mixed-norm proximal support vector machine Lap-MNPSVM for short. In the optimization problem of Lap-MNPSVM, the information from the unlabeled examples is used in a form of Laplace regularization, and \(l_p\) norm (\(0\,<\,p\,<\,1\)) regularizer is introduced to standard proximal support vector machine to control sparsity and the feature selection. To solve the nonconvex optimization problem in Lap-MNPSVM, an efficient algorithm is proposed by solving a series systems of linear equations, and the lower bounds of the solution are established, which are extremely helpful for feature selection. Experiments carried out on synthetic datasets and the real-world datasets show the feasibility and effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
\(\Vert x\Vert _p (0 < p < 1)\) is a quasi-norm, which satisfies the norm axioms except the triangle inequality.
Sparsity is here defined as the number of nonzero components in the normal vector \(w\). This means that more zero components in \(w\), more sparse the hyperplane.
References
Chapelle O, Schölkopf B, Zien A (2010) Semi-supervised learning. MIT Press, MA
Zhu X, Ghahramani Z, Lafferty Z (2003) Semi-supervised learning using Gaussian fields and harmonic functions. In: Proceedings of the 20th international conference on machine learning (ICML), pp 912–919
Zhou Z-H, Li M (2010) Semi-supervised learning by disagreement. Knowl Inf Syst 24:415–439
Wang Y, Chen S, Zhou Z (2012) New semi-supervised classification method based on modified cluster assumption. IEEE Trans Neural Netw Learn Syst 23:689–702
Culp M, Michailidis G (2008) Graph-based semisupervised learning. IEEE Trans Pattern Anal Mach Intell 30:174–179
Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Morgan and Claypool Press, New York, p 25
Yang ZX (2013) Nonparallel hyperplanes proximal classifiers based on manifold regularization for labeled and unlabeled examples. Int J Pattern Recognit Artif Intell 27(5):1350015
Ren X, Wang Y, Wang J, Zhang XS (2012) A unified computational model for revealing and predicting subtle subtypes of cancers. BMC Bioinform. doi:10.1186/1471-2105-13-70
Ren XW, wang Y, Zhang XS, Qin J (2013) iPcc: a novel feature extraction method for accurate disease class discovery and prediction. Nucleic Acids Res 41(4):e143
Zheng X, Wu LY, Zhou XB, Stephen TC (2010) Wong Semi-supervised drug-protein interaction prediction from heterogeneous biological spaces. BMC Syst Biol. doi:10.1186/1752-0509-4-S2-S6
Belkin M, Niyogi P (2003) Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput 15:1373–1396
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434
Zhang S, Lei Y, Wu Y (2011) Semi-supervised locally discriminant projection for classification and recognition. Knowl Based Syst 24:341–346
Xue H, Chen S, Yang Q (2009) Discriminatively regularized least-squares classification. Pattern Recogn 42:93–104
Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of international conference of knowledge discovery and data mining, pp 77–86
Wu F, Wang W, Yang Y, Zhuang Y, Nie F (2010) Classification by semisupervised discriminative regularization. Neurocomputing 73:1641–1651
Chen XJ, Xu FM, Ye YY (2009) Lower bound theory of nonzero entries in solutions of \(l_2\)–\(l_p\) minimization. http://www.polyu.edu.hk/ama/staff/xjchen/cxyfinal
Bruckstein AM, Donoho DL, Elad M (2009) From sparse sulutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51:34–81
Fan J, Li R (2001) Varible selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360
Xu Z, Zhang H, Wang Y, Chang X (2009) \(L_{\frac{1}{2}}\) regularizer. Sci in China Ser F Inf Sci 52:1–9
Chen WJ, Tian YJ (2010) \(L_p\)-norm proximal support vector machine and its applications. Procedia Comput Sci ICCS 1(1):2417–2423
Tian YJ, Yu J, Chen WJ (2010) \(l_p\)-norm support vector machine with CCCP. In: Proceedings of the 7th FSKD, pp 1560–1564
Tan JY, Zhang CH, Deng NY (2010) Cancer related gene identification via \(p\)-norm support vector machine. In: The 4th international conference on computational systems biology, pp 101–108
Tan J-Y, Zhang Z-Q, Zhen Ling, Zhang C-H, Deng N-Y (2013) Adaptive feature selection via a new version of support vector machine. Neural Comput Appl 23:937–945
Zhang C-H, Shao Y-H, Tan J-Y, Deng N-Y (2013) A mixed-norm linear support vector machine. Neural Comput Appl 23:2159–2166
Saad Y (2003) Iterative methods for sparse linear systems. SIAM Press, Philadelphia
Joachims T (2002) Learning to classify text using support vector machines: methods, theory and algorithms. Springer, New York
Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semisupervised classification. Neural Netw 35:46–53
Acknowledgments
This paper was supported by National Natural Science Foundation of China (No. 11301535, No. 11371365).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Zhang, Z., Zhen, L., Deng, N. et al. Manifold proximal support vector machine with mixed-norm for semi-supervised classification. Neural Comput & Applic 26, 399–407 (2015). https://doi.org/10.1007/s00521-014-1728-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-014-1728-4