Skip to main content
Log in

Semi-supervised learning with Deep Laplacian Support Vector Machine

  • Original Article
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Deep learning is a rapidly growing field that can effectively extract latent features from data and use them to make predictions based on the learned features, but most models just sum the loss of each sample without considering the relationship between samples. On the other hand, the traditional Laplacian Support Vector Machine (LapSVM) can effectively utilize samples and the relationship between samples by constructing a Laplacian graph, and performs well on semi-supervised data. In this paper, we combine LapSVM and deep learning and propose Deep Laplacian Support Vector Machine. Our approach is to first use a Deep Neural Network to extract the latent features from the image, then based on the extracted feature information and a small amount of original label information, we use LapSVM for classification, build a loss function, and finally iteratively update the two parts together. We evaluate our method on several benchmark datasets and demonstrate that it outperforms other semi-supervised learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

The data analyzed or used in the course of this study are available in publicly available machine learning repositories, at http://www.cs.toronto.edu/~kriz/cifar.html and https://archive.ics.uci.edu/ml/datasets.php. The code is available at https://github.com/chyahdr/Semi-Supervised-Learning-with-Deep-Laplacian-Support-Vector-Machine.

References

  1. Learning S-S (2006) Semi-supervised learning. CSZ2006. html

  2. Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. In: Synthesis lectures on artificial intelligence and machine learning

  3. Ouali Y, Hudelot C, Tami M (2020) An overview of deep semi-supervised learning. arXiv preprint arXiv:2006.05278

  4. Gomez-Chova L, Camps-Valls G, Munoz-Mari J, Calpe J (2008) Semisupervised image classification with laplacian support vector machines. IEEE Geosci Remote Sens Lett 5(3):336–340. https://doi.org/10.1109/LGRS.2008.916070

    Article  MATH  Google Scholar 

  5. Bennett K, Demiriz A (1998) Semi-supervised support vector machines. Adv Neural Inf Process Syst 11

  6. Wang Y, Chen L, Zhou J, Li T, Yu Y (2023) Pairwise constraints-based semi-supervised fuzzy clustering with multi-manifold regularization. Inf Sci 638:118994

    Article  Google Scholar 

  7. Shao W, Li X, Yao Y, Chen J, Zhao D (2023) Semi-supervised local manifold regularization model based on dual representation for industrial soft sensor development. Chemom Intell Lab Syst 242:104937

    Article  MATH  Google Scholar 

  8. Feng W, Wang Z, Cao X, Cai B, Guo W, Ding W (2024) Discriminative sparse subspace learning with manifold regularization. Expert Syst Appl 249:123831

    Article  Google Scholar 

  9. Yuan W, Zhang C, Song W, Yang S (2023) Two applications of manifold regularization in deep learning architectures. In: Journal of Physics: Conference Series 2547:012004.IOP Publishing

  10. Xie Q, Luong M-T, Hovy E, Le QV (2020) Self-training with noisy student improves imagenet classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10687–10698

  11. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  12. Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inf Process Syst 30

  13. Kingma DP, Mohamed S, Jimenez Rezende D, Welling M (2014) Semi-supervised learning with deep generative models. Adv Neural Inf Process Syst 27

  14. Laine S, Aila T (2016) Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242

  15. Zhang Y, Zhang H, Deng B, Li S, Jia K, Zhang L (2021) Semi-supervised models are strong unsupervised domain adaptation learners. arXiv preprint arXiv:2106.00417

  16. Lee D-H et al (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML, vol 3. Atlanta, p 896

  17. Rizve MN, Duarte K, Rawat YS, Shah M (2021) In defense of pseudo-labeling: an uncertainty-aware pseudo-label selection framework for semi-supervised learning. arXiv preprint arXiv:2101.06329

  18. Girsch M (2023) Bayesian estimation using deep learning-based feature extraction and pseudo-labels. PhD thesis, Wien

  19. Xu W, Sun H, Deng C, Tan Y (2017) Variational autoencoder for semi-supervised text classification. In: Proceedings of the AAAI conference on artificial intelligence, p 31

  20. Ghasedi Dizaji K, Wang X, Huang H (2018) Semi-supervised generative adversarial network for gene expression inference. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1435–1444

  21. Springenberg JT (2015) Unsupervised and semi-supervised learning with categorical generative adversarial networks. arXiv preprint arXiv:1511.06390

  22. Ding X, Wang Y, Xu Z, Welch WJ, Wang ZJ (2020) Ccgan: continuous conditional generative adversarial networks for image generation. In: International conference on learning representations

  23. Li D, Liu Y, Song L (2022) Adaptive weighted losses with distribution approximation for efficient consistency-based semi-supervised learning. IEEE Trans Circuits Syst Video Technol 32(11):7832–7842

    Article  MATH  Google Scholar 

  24. Li D, Zhu X, Song L (2023) Mutual match for semi-supervised online evolutive learning. Appl Intell 53(3):3336–3350

    Article  MATH  Google Scholar 

  25. Ding Shifei Q, Hongyan T (2011) An overview on theory and algorithm of support vector machines. J Univ Electron Sci Technol China 40(1):2

    MATH  Google Scholar 

  26. Sánchez AVD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20

    Article  MATH  Google Scholar 

  27. El-Naqa I, Yang Y, Wernick MN, Galatsanos NP, Nishikawa RM (2002) A support vector machine approach for detection of microcalcifications. IEEE 21:1552–1563

    Google Scholar 

  28. Maldonado S, Weber R, Famili F (2014) Feature selection for high-dimensional class-imbalanced data sets using support vector machines. Inf Sci 286:228–246

    Article  MATH  Google Scholar 

  29. Melacci S, Belkin M (2011) Laplacian support vector machines trained in the primal. J Mach Learn Res 12(3)

  30. Qi Z, Tian Y, Shi Y, Yu X (2013) Cost-sensitive support vector machine for semi-supervised learning. Procedia Comput Sci 18:1684–1689

    Article  MATH  Google Scholar 

  31. Tan J, Zhen L, Deng N, Zhang Z (2014) Laplacian p-norm proximal support vector machine for semi-supervised classification. Neurocomputing 144:151–158

    Article  MATH  Google Scholar 

  32. Yang L, Yang S, Jin P, Zhang R (2013) Semi-supervised hyperspectral image classification using spatio-spectral Laplacian support vector machine. IEEE Geosci Remote Sens Lett 11(3):651–655

    Article  MATH  Google Scholar 

  33. Qi Z, Tian Y, Shi Y (2014) Successive overrelaxation for Laplacian support vector machine. IEEE Trans Neural Netw Learn Syst 26(4):674–683

    Article  MathSciNet  MATH  Google Scholar 

  34. Karimi Z, Ghidary SS (2017) Semi-supervised classification in stratified spaces by considering non-interior points using Laplacian behavior. Neurocomputing 239:223–231

    Article  MATH  Google Scholar 

  35. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(11)

  36. Maaten L, Hinton G (2008) Visualizing data using t-sne. J Mach Learn Res 9(86):2579–2605

    MATH  Google Scholar 

  37. Duan K-B, Keerthi SS (2005) Which is the best multiclass svm method? an empirical study. In: International workshop on multiple classifier systems. Springer, pp 278–285

  38. Qi Z, Tian Y, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53

    Article  MATH  Google Scholar 

  39. Khemchandani R, Chandra S et al (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  40. Chen W-J, Shao Y-H, Deng N-Y, Feng Z-L (2014) Laplacian least squares twin support vector machine for semi-supervised classification. Neurocomputing 145:465–476

    Article  MATH  Google Scholar 

  41. Chen W-J, Shao Y-H, Xu D-K, Fu Y-F (2014) Manifold proximal support vector machine for semi-supervised classification. Appl Intell 40:623–638

    Article  MATH  Google Scholar 

  42. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Article  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No.61906101). It is also supported by the Ningbo Municipal Natural Science Foundation of China (No. 2023J115).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xijiong Xie.

Ethics declarations

Conflict of interest

The authors have no Conflict of interest to declare that are content of this article. The authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Xie, X. & Li, D. Semi-supervised learning with Deep Laplacian Support Vector Machine. Pattern Anal Applic 28, 14 (2025). https://doi.org/10.1007/s10044-024-01395-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10044-024-01395-5

Keywords

Navigation