Abstract
Deep learning is a rapidly growing field that can effectively extract latent features from data and use them to make predictions based on the learned features, but most models just sum the loss of each sample without considering the relationship between samples. On the other hand, the traditional Laplacian Support Vector Machine (LapSVM) can effectively utilize samples and the relationship between samples by constructing a Laplacian graph, and performs well on semi-supervised data. In this paper, we combine LapSVM and deep learning and propose Deep Laplacian Support Vector Machine. Our approach is to first use a Deep Neural Network to extract the latent features from the image, then based on the extracted feature information and a small amount of original label information, we use LapSVM for classification, build a loss function, and finally iteratively update the two parts together. We evaluate our method on several benchmark datasets and demonstrate that it outperforms other semi-supervised learning methods.







Similar content being viewed by others
Data Availability
The data analyzed or used in the course of this study are available in publicly available machine learning repositories, at http://www.cs.toronto.edu/~kriz/cifar.html and https://archive.ics.uci.edu/ml/datasets.php. The code is available at https://github.com/chyahdr/Semi-Supervised-Learning-with-Deep-Laplacian-Support-Vector-Machine.
References
Learning S-S (2006) Semi-supervised learning. CSZ2006. html
Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. In: Synthesis lectures on artificial intelligence and machine learning
Ouali Y, Hudelot C, Tami M (2020) An overview of deep semi-supervised learning. arXiv preprint arXiv:2006.05278
Gomez-Chova L, Camps-Valls G, Munoz-Mari J, Calpe J (2008) Semisupervised image classification with laplacian support vector machines. IEEE Geosci Remote Sens Lett 5(3):336–340. https://doi.org/10.1109/LGRS.2008.916070
Bennett K, Demiriz A (1998) Semi-supervised support vector machines. Adv Neural Inf Process Syst 11
Wang Y, Chen L, Zhou J, Li T, Yu Y (2023) Pairwise constraints-based semi-supervised fuzzy clustering with multi-manifold regularization. Inf Sci 638:118994
Shao W, Li X, Yao Y, Chen J, Zhao D (2023) Semi-supervised local manifold regularization model based on dual representation for industrial soft sensor development. Chemom Intell Lab Syst 242:104937
Feng W, Wang Z, Cao X, Cai B, Guo W, Ding W (2024) Discriminative sparse subspace learning with manifold regularization. Expert Syst Appl 249:123831
Yuan W, Zhang C, Song W, Yang S (2023) Two applications of manifold regularization in deep learning architectures. In: Journal of Physics: Conference Series 2547:012004.IOP Publishing
Xie Q, Luong M-T, Hovy E, Le QV (2020) Self-training with noisy student improves imagenet classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10687–10698
Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907
Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inf Process Syst 30
Kingma DP, Mohamed S, Jimenez Rezende D, Welling M (2014) Semi-supervised learning with deep generative models. Adv Neural Inf Process Syst 27
Laine S, Aila T (2016) Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242
Zhang Y, Zhang H, Deng B, Li S, Jia K, Zhang L (2021) Semi-supervised models are strong unsupervised domain adaptation learners. arXiv preprint arXiv:2106.00417
Lee D-H et al (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, vol 3. Atlanta, p 896
Rizve MN, Duarte K, Rawat YS, Shah M (2021) In defense of pseudo-labeling: an uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv preprint arXiv:2101.06329
Girsch M (2023) Bayesian estimation using deep learning-based feature extraction and pseudo-labels. PhD thesis, Wien
Xu W, Sun H, Deng C, Tan Y (2017) Variational autoencoder for semi-supervised text classification. In: Proceedings of the AAAI conference on artificial intelligence, p 31
Ghasedi Dizaji K, Wang X, Huang H (2018) Semi-supervised generative adversarial network for gene expression inference. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1435–1444
Springenberg JT (2015) Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.06390
Ding X, Wang Y, Xu Z, Welch WJ, Wang ZJ (2020) Ccgan: continuous conditional generative adversarial networks for image generation. In: International conference on learning representations
Li D, Liu Y, Song L (2022) Adaptive weighted losses with distribution approximation for efficient consistency-based semi-supervised learning. IEEE Trans Circuits Syst Video Technol 32(11):7832–7842
Li D, Zhu X, Song L (2023) Mutual match for semi-supervised online evolutive learning. Appl Intell 53(3):3336–3350
Ding Shifei Q, Hongyan T (2011) An overview on theory and algorithm of support vector machines. J Univ Electron Sci Technol China 40(1):2
Sánchez AVD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20
El-Naqa I, Yang Y, Wernick MN, Galatsanos NP, Nishikawa RM (2002) A support vector machine approach for detection of microcalcifications. IEEE 21:1552–1563
Maldonado S, Weber R, Famili F (2014) Feature selection for high-dimensional class-imbalanced data sets using support vector machines. Inf Sci 286:228–246
Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12(3)
Qi Z, Tian Y, Shi Y, Yu X (2013) Cost-sensitive support vector machine for semi-supervised learning. Procedia Comput Sci 18:1684–1689
Tan J, Zhen L, Deng N, Zhang Z (2014) Laplacian p-norm proximal support vector machine for semi-supervised classification. Neurocomputing 144:151–158
Yang L, Yang S, Jin P, Zhang R (2013) Semi-supervised hyperspectral image classification using spatio-spectral Laplacian support vector machine. IEEE Geosci Remote Sens Lett 11(3):651–655
Qi Z, Tian Y, Shi Y (2014) Successive overrelaxation for Laplacian support vector machine. IEEE Trans Neural Netw Learn Syst 26(4):674–683
Karimi Z, Ghidary SS (2017) Semi-supervised classification in stratified spaces by considering non-interior points using Laplacian behavior. Neurocomputing 239:223–231
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(11)
Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(86):2579–2605
Duan K-B, Keerthi SS (2005) Which is the best multiclass svm method? an empirical study. In: International workshop on multiple classifier systems. Springer, pp 278–285
Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53
Khemchandani R, Chandra S et al (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910
Chen W-J, Shao Y-H, Deng N-Y, Feng Z-L (2014) Laplacian least squares twin support vector machine for semi-supervised classification. Neurocomputing 145:465–476
Chen W-J, Shao Y-H, Xu D-K, Fu Y-F (2014) Manifold proximal support vector machine for semi-supervised classification. Appl Intell 40:623–638
Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543
Acknowledgements
This work is supported by National Natural Science Foundation of China (No.61906101). It is also supported by the Ningbo Municipal Natural Science Foundation of China (No. 2023J115).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no Conflict of interest to declare that are content of this article. The authors have no financial or proprietary interests in any material discussed in this article.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, H., Xie, X. & Li, D. Semi-supervised learning with Deep Laplacian Support Vector Machine. Pattern Anal Applic 28, 14 (2025). https://doi.org/10.1007/s10044-024-01395-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10044-024-01395-5