Skip to main content

Advertisement

Springer Nature Link
Log in

Robust nonparallel support vector machine with privileged information for pattern recognition

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In the field of pattern recognition, collected data always include some additional information which are usually termed as privileged information. The privileged information is latent information belonging to the training samples, which can be easily ignored. The privileged information can help to build a better classifier for classification. In this paper, we try to construct a robust nonparallel support vector machine (NPSVM) model under the privileged information learning (LUPI) setting, termed as R-NPSVM+. On the one hand, we introduce the privileged information into NPSVM so as to build a model for classification. In the training process, both the privileged information and “usual” samples are used to train the model, which can enhance the accuracy. On the other hand, due to the ε-insensitive loss and hinge loss, NPSVM is sensitive to noise or outliers. Hence, we use two robust loss functions in R-NPSVM+ model, which can further ensure the robustness of the model. In addition, we use the Lagrange multiplier method and the dual coordinate descent (DCD) algorithm to optimize the proposed objective function, respectively. Lastly, to evaluate the performance of R-NPSVM+, we conduct a series of experiments. Experimental results confirm that compared with other classical SVM-type algorithms, our R-NPSVM+ can produce a better performance, especially when the samples are corrupted by noise and outliers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Vapnik V (2006) Estimation of dependences based on empirical data. Springer, NY, p 505

    Book  MATH  Google Scholar 

  2. Zhou ZH (2021) Support vector machine. Machine learning. Springer, Singapore, pp 129–153

    Chapter  Google Scholar 

  3. Cai F (2011) Advanced learning approaches based on SVM+ methodology, University of Minnesota

  4. Wang H, Peng M, Hines JW et al (2019) Upadhyayam A hybrid fault diagnosis methodology with support vector machine and improved particle swarm optimization for nuclear power plants. ISA Trans 95:358–371

    Article  Google Scholar 

  5. Richhariya B, Gupta D (2019) Facial expression recognition using iterative universum twin support vector machine. Appl Soft Comput 76:53–67

    Article  Google Scholar 

  6. Tian Y, Chen C, Chen X et al (2020) Research on real-time analysis technology of urban land use based on support vector machine. Pattern Recogn Lett 133:320–326

    Article  Google Scholar 

  7. Feyereisl J, Aickelin U (2012) Privileged information for data clustering. Inf Sci 194:4–23

    Article  Google Scholar 

  8. Wang S, Chen S, Chen T et al (2018) Learning with privileged information for multi-label classification. Pattern Recogn 81:60–70

    Article  Google Scholar 

  9. Liang L, Cai F, Cherkassky V (2009) Predictive learning with structured (grouped) data. Neural Netw 22(5–6):766–773

    Article  MATH  Google Scholar 

  10. Guo Y, Xiao H, Kan Y et al (2018) Learning using privileged information for HRRP-based radar target recognition. IET Signal Proc 12(2):188–197

    Article  Google Scholar 

  11. Smolyakov D, Sviridenko N, Burikov E et al (2018) Anomaly pattern recognition with privileged information for sensor fault detection, IAPR Workshop on artificial neural networks in pattern recognition. Springer, Cham, p 320–332

  12. Yan Y, Nie F, Li W et al (2016) Image classification by cross-media active learning with privileged information. IEEE Trans Multimedia 18(12):2494–2502

    Article  Google Scholar 

  13. Vapnik V, Vashist A (2009) A new learning paradigm: learning using privileged information. Neural Netw 22(5–6):544–557

    Article  MATH  Google Scholar 

  14. Li X, Du B, Xu C et al (2018) R-SVM+: robust learning with privileged information. IJCA I:2411–2417

    Google Scholar 

  15. Shu Y, Li Q, Xu C, et al (2021) V-SVR+: support vector regression with variational privileged information, IEEE Transactions on Multimedia, 2021

  16. Li W, D. Dai D, Tan M et al (2016) Fast algorithms for linear and kernel svm+, Proceedings of the IEEE Conference on computer vision and pattern recognition, p 2258–2266

  17. Jayadeva R, Khemchandani S (2007) Chandra, Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Article  MATH  Google Scholar 

  18. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Article  Google Scholar 

  19. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Article  Google Scholar 

  20. Qi Z, Tian Y, Shi Y (2014) A new classification model using privileged information and its application. Neurocomputing 129:146–152

    Article  Google Scholar 

  21. Che Z, Liu B, Xiao Y et al (2021) Twin support vector machines with privileged information. Inf Sci 573:141–153

    Article  MathSciNet  Google Scholar 

  22. Li Y, Sun H, Yan W et al (2021) R-CTSVM+: Robust capped L1-norm twin support vector machine with privileged information. Inf Sci 574:12–32

    Article  Google Scholar 

  23. Tian Y, Qi Z, Ju X, Shi Y, Liu X (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079

    Article  Google Scholar 

  24. Hsieh C A dual coordinate descent method for large-scale linear SVM, In Proceedings of International Conference on Machine Learning. ACM, 2016.

  25. Liu L, Chu M, Gong R, Zhang L (2020) An improved nonparallel support vector machine. IEEE Trans Neural Netw Learn Syst 32(1):5129–5143

    MathSciNet  Google Scholar 

  26. The Math Works (MATLAB 2016b), Inc. [Online]. Available: http://www.mathworks.com.

  27. Dua D, Taniskidou EK, UCI machine learning repository, [Online]. Available: http://archive.ics.uci.edu/ml/.

  28. Rosario SF, Thangadurai K (2015) RELIEF: feature selection approach. Int J Innov Res Dev 4(11):218–24

    Google Scholar 

  29. Li J, Cheng K, Wang S, Morstatter F, Trevino RP, Tang J, Liu H (2017) Feature selection: a data perspective. ACM Comput Surv (CSUR) 50(6):1–45

    Article  Google Scholar 

  30. Xue B, Zhang M, Browne WN, Yao X (2015) A survey on evolutionary computation approaches to feature selection. IEEE Trans Evol Comput 20(4):606–626

    Article  Google Scholar 

  31. Dokeroglu T, Deniz A, Kiziloz HE (2022) A comprehensive survey on recent metaheuristics for feature selection. Neurocomputing. https://doi.org/10.1016/j.neucom.2022.04.083

    Article  Google Scholar 

  32. Tomasi C (2012) Histograms of oriented gradients, Comput Vis Sampl, 1–6

  33. Pietikäinen M (2010) Local binary patterns. Scholarpedia 5(3):9775

    Article  Google Scholar 

  34. Peterson LE (2009) K-nearest neighbor. Scholarpedia 4(2):1883

    Article  Google Scholar 

  35. Chen WJ, Shao YH, Li CN et al (2020) NPrSVM: nonparallel sparse projection support vector machine with efficient algorithm. Appl Soft Comput 90:106142

    Article  Google Scholar 

  36. Demšar J (2006) Statistical comparisons of classifiers over multiple datasets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  37. Woolson RF (2007) Wilcoxon signed-rank test. Wiley encyclopaedia of clinical trials. Wiley, Hoboken, NJ, USA, pp 1–3

    Google Scholar 

  38. LeCun Y (1998) The MNIST database of handwritten digits. [Online]. Available: http://yann.lecun.com/exdb/mnist/

  39. Hu H, Liu Y, Liu M et al (2016) Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181:86–95

    Article  Google Scholar 

  40. Chu M, Gong R, Gao S et al (2017) Steel surface defects recognition based on multi-type statistical features and enhanced twin support vector machine. Chemom Intell Lab Syst 171:140–150

    Article  Google Scholar 

  41. Li CN, Huang YF, Wu HJ et al (2016) Multiple recursive projection twin support vector machine for multi-class classification. Int J Mach Learn Cybern 7(5):729–740

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grants (61673199), Natural Science Foundation of Liaoning Province of China (2022-MS-353), Basic Scientific Research Project of Education Department of Liaoning Province (2020LNZD06 and LJKMZ20220640), University of Science and Technology Liaoning Talent Project Grants (601011507–20), University of Science and Technology Liaoning Team Building Grants (601013360–17) and Graduate Science and Technology Innovation Program of University of Science and Technology Liaoning (LKDYC201909).

Funding

National Natural Science Foundation of China, 61673199, Ping Li.

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, University of Science and Technology Liaoning, Anshan, 114051, Liaoning, China

    Liming Liu, Ping Li, Maoxiang Chu & Shuming Liu

Authors
  1. Liming Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ping Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Maoxiang Chu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Shuming Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Ping Li or Maoxiang Chu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, L., Li, P., Chu, M. et al. Robust nonparallel support vector machine with privileged information for pattern recognition. Int. J. Mach. Learn. & Cyber. 14, 1465–1482 (2023). https://doi.org/10.1007/s13042-022-01709-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01709-1

Keywords