Skip to main content
Springer Nature Link
Log in

Identification of uncertainty and decision boundary for SVM classification training using belief function

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The existence of noisy samples increases the inefficiency of SVM training. In SVM training, the classification hyperplane is determined by the support vectors, therefore, the other samples do not affect the SVM classifier. This paper presents a novel method that does not use all samples for SVM training. The basic idea is a novel method in which, at the first step, using the belief function theory and fuzzy rough set theory, the boundary samples are identified. And at the second step, the boundary samples uncertainty such as noisy samples are identified and discarded. Finally, at the last step, using the obtained boundary samples, the training of the SVM classifier is done. To show the performance of the proposed method, BFFR-BS (Belief Function and Fuzzy Rough Set-Boundary Samples), several experiments have been conducted on various real-world data sets from UCI repository. Experimental results reveal that the proposed method is superior to the state-of-the-art competing methods regarding accuracy, precision, time, and AUC metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Vapnik V (1995) The nature of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Article  MATH  Google Scholar 

  2. Yang L, Xu Z (2017) Feature extraction by PCA and diagnosis of breast tumors using SVM with DE-based parameter tuning. Int J Mach Learn & Cyber:1–11

  3. Mao WT, Xu JC, Wang C et al (2014) A fast and robust model selection algorithm for multi-input multi-output support vector machine. Neurocomputing 130:10–19

    Article  Google Scholar 

  4. Santhanama V, Morariua VI, Harwooda D, Davisa LS (2016) A non-parametric approach to extending generic binary classifiers for multi-classification. Pattern Recogn 58:149–158

    Article  Google Scholar 

  5. Vanir V (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Article  Google Scholar 

  6. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  7. Xue Y, Zhang L, Wang B, Zhang Z, Li F (2018) Nonlinear feature selection using Gaussian kernel SVM-RFE for fault diagnosis. Appl Intell:1–26

  8. Moghaddam VH, Hamidzadeh J (2016) New Hermite orthogonal polynomial kernel and combined kernels in support vector machine classifier. Pattern Recogn 60:921–935

    Article  Google Scholar 

  9. Hamidzadeh J, Moradi M (2018) Improved one-class classification using filled function. Appl Intell:1–17

  10. Hamidzadeh J, Sadeghi R, Namaei N (2017) Weighted support vector data description based on chaotic bat algorithm. Appl Soft Comput 60:540–551

    Article  Google Scholar 

  11. Hamidzadeh J, Namaei N (2018) Belief-based chaotic algorithm for support vector data description. Soft Comput:1–26

  12. Hsu HT, Lee PL, Shyu KK (2017) Improvement of classification accuracy in a phase-tagged steady-state visual evoked potential-based brain–computer Interface using adaptive neuron-fuzzy classifier. International Journal of Fuzzy Systems 19:542–552

    Article  Google Scholar 

  13. Onan A (2015) A fuzzy-rough nearest neighbor classifier combined with consistency-based subset evaluation and instance selection for automated diagnosis of breast cancer. Expert Syst Appl 42:6844–6852

    Article  Google Scholar 

  14. Zhou Q, Chao F, Lin CM (2018) A functional-link-based fuzzy brain emotional learning network for breast tumor classification and chaotic system synchronization. International Journal of Fuzzy Systems 20:349–365

    Article  MathSciNet  Google Scholar 

  15. Yue X, Chen Y, Miao D, Qian J (2017) Tri-partition neighborhood covering reduction for robust classification. Int J Approx Reason 83:371–384

    Article  MathSciNet  MATH  Google Scholar 

  16. Chen Y, Xue Y, Ma Y, Xu F (2017) Measures of uncertainty for neighborhood rough sets. Knowl-Based Syst 120:1–10

    Article  Google Scholar 

  17. Kar S, Majumder DD (2016) An investigative study on early diagnosis of breast Cancer using a new approach of mathematical shape theory and neuro-fuzzy classification system. International Journal of Fuzzy Systems 18:349–366

    Article  MathSciNet  Google Scholar 

  18. Du SQ, Wei W, May D, Younan NH (2010) Noise-adjusted principal component analysis for buried radioactive target detection and classification. IEEE Trans Nucl Sci 57:349–366

    Google Scholar 

  19. Han D, Liu W, Dezert J, Yang Y (2016) A novel approach to pre-extracting support vectors based on the theory of belief functions. Knowl-Based Syst 110:210–223

    Article  Google Scholar 

  20. Han DQ, Han CZ, Yang Y (2009) Approach for pre-extracting support vectors based on K-NN. Control Decis 24(4):494–498

    MATH  Google Scholar 

  21. Zhou C, Lu X, Huang M (2016) Dempster–Shafer theory-based robust least squares support vector machine for stochastic modelling. Neurocomputing 182:145–153

    Article  Google Scholar 

  22. Yang X, Song Q, Cao A (2005) Weighted support vector machine for data classification. IEEE International Joint Conference on Neural Networks 2:859–864

    Google Scholar 

  23. Jayadeva R, Khemchandani S, Chandra HZ (2004) Fast and robust learning through fuzzy linear proximal support vector machines. Neurocomputing 61:401–411

    Article  Google Scholar 

  24. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13:464–471

    Article  Google Scholar 

  25. Lu X, Liu W, Zhou C, Huang M (2017) Probabilistic weighted support vector machine for robust modeling with application to hydraulic actuator. IEEE Trans Industrial Informatics 13(4):1723–1733

    Article  Google Scholar 

  26. Chau AL, Li X, Yu W (2013) Convex and concave hulls for classification with support vector machine. Neurocomputing 122:198–209

    Article  Google Scholar 

  27. Xiaa S y, Xiong Z y, Luo Y g, Dong L m (2015) A method to improve support vector machine based on distance to hyperplane. Optik - International Journal for Light and Electron Optics 126:2405–2410

    Article  Google Scholar 

  28. Triguero I, Peralta D, Bacardit J, García S, Herrera F (2015) MRPR: a MapReduce solution for prototype reduction in big data classification. Neurocomputing 150 (331–345

    Article  Google Scholar 

  29. Hamidzadeh J, Monsefi R, Yazdi HS (2015) IRAHC: Instance reduction algorithm using hyperrectangle clustering. Pattern Recogn 48:1878–1889

    Article  MATH  Google Scholar 

  30. Shafer G (1976) A mathematical theory of evidence. Princeton University Press, Princeton

    MATH  Google Scholar 

  31. Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17:191–209

    Article  MATH  Google Scholar 

  32. Xu P, Davoine F, Zha H, Denœux T (2016) Evidential calibration of binary SVM classifiers. Int J Approx Reason 72:55–70

    Article  MathSciNet  MATH  Google Scholar 

  33. Liu Z g, Pan Q, Dezert J, Mercier G (2014) Credal classification rule for uncertain data based on belief functions. Pattern Recognit 47:2532–2541

    Article  Google Scholar 

  34. Djelloul M, Sari Z, Latreche K (2018) Uncertain fault diagnosis problem using neuro-fuzzy approach and probabilistic model for manufacturing systems. Appl Intell:1–18

  35. Reineking T, Denœux T (2016) Active classification using belief functions and information gain maximization. Int J Approx Reason 72:43–54

    Article  MathSciNet  MATH  Google Scholar 

  36. Liu ZG, Pan Q, Mercier G, Dezert J (2015) A new incomplete pattern classification method based on evidential reasoning. IEEE Transactions on Cybernetics 45:635–646

    Article  Google Scholar 

  37. Zhu F, Ye N, Yu W, Xu S, Li G (2014) Boundary detection and sample reduction for one-class support vector machines. Neurocomputing 123:166–173

    Article  Google Scholar 

  38. Wang L, Sui M, Li Q, Xiao H (2012) A New Method of Sample Reduction for Support Vector Classification, 2012 IEEE Asia-Pacific Services Computing Conference 301–304

  39. Xia S, Xiong Z, Luo Y, Dong L, Xing C (2015) Relative density based support vector machine. Neurocomputing 149 (1424–1432

    Article  Google Scholar 

  40. Wang S, Li Z, Liu C, Zhang X, Zhang H (2014) Training data reduction to speed up SVM training. Appl Intell 41:405–420

    Article  Google Scholar 

  41. Han DQ, Dezert J, Duan ZS (2016) Evaluation of probability transformations of belief functions for decision making. IEEE Trans. Syst. Man Cybern. 46(1):93–108

    Article  Google Scholar 

  42. Liu ZG, Pan Q, Dezert J (2013) Evidential classifier for imprecise data based on belief functions. Knowl-Based Syst 52:246–257

    Article  Google Scholar 

  43. Liu ZG, Pan Q, Dezert J, Mercier G (2015) Credal c-means clustering method based on belief functions. Knowl-Based Syst 74:119–132

    Article  Google Scholar 

  44. Jousselme AL, Liu CS, Grenier D (2006) Measuring ambiguity in the evidence theory. IEEE Trans Syst Man Cybern 36(5):890–903

    Article  Google Scholar 

  45. Yager RR (2007) Entropy and specificity in a mathematical theory of evidence. Int J General Syst 9(4):249–260

    Article  MathSciNet  MATH  Google Scholar 

  46. Smets P, Kennes R (1994) The transferable belief model. Artif Intell 66(2):191–234

    Article  MathSciNet  MATH  Google Scholar 

  47. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  48. Pawlak Z (1982) Rough sets. Int J Comput Inf Sci 11:341–356

    Article  MATH  Google Scholar 

  49. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27

    Google Scholar 

  50. M. Lichman, UCI machine learning repository, 2013 http://archive.ics.uci.edu/ml

    Google Scholar 

  51. Musicant JDR (1998) Ndc:normally distributed clustered datasets

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering and Information Technology, Sadjad University of Technology, Mashhad, Iran

    Javad Hamidzadeh

  2. Department of Computer Engineering, Salman Institute of Higher Education, Mashhad, Iran

    Somaye Moslemnejad

Authors
  1. Javad Hamidzadeh
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Somaye Moslemnejad
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Javad Hamidzadeh.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hamidzadeh, J., Moslemnejad, S. Identification of uncertainty and decision boundary for SVM classification training using belief function. Appl Intell 49, 2030–2045 (2019). https://doi.org/10.1007/s10489-018-1374-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1374-0

Keywords