Skip to main content
Springer Nature Link
Log in

Entropy-Based Fuzzy Least Squares Twin Support Vector Machine for Pattern Classification

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Least squares twin support vector machine (LSTSVM) is a new machine learning method, as opposed to solving two quadratic programming problems in twin support vector machine (TWSVM), which generates two nonparallel hyperplanes by solving a pair of linear system of equations. However, LSTSVM obtains the resultant classifier by giving same importance to all training samples which may be important for classification performance. In this paper, by considering the fuzzy membership value for each sample, we propose an entropy-based fuzzy least squares twin support vector machine where fuzzy membership values are assigned based on the entropy values of all training samples. The proposed method not only retains the superior characteristics of LSTSVM which is simple and fast algorithm, but also implements the structural risk minimization principle to overcome the possible over- fitting problem. Experiments are performed on several synthetic as well as benchmark datasets and the experimental results illustrate the effectiveness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Cortes C, Vapnik VN (1995) Support-vector networks. Mach Learn 20(3):273–297

    Google Scholar 

  2. Osuna E, Freund R, Girosi F (1997) Training support vector machines: an application to face detection. In: Proceedings of computer vision and pattern recognition, pp 130–136

  3. Noble WS (2004) Kernel methods in computational biology. In: Schölkopf B, Tsuda K, Vert JP (eds) Support vector machine applications in computational biology. MIT, Cambridge, pp 71–92

    Google Scholar 

  4. Isa D, Lee LH, Kallimani VP, Rajkumar R (2008) Text document preprocessing with the Bayes formula for classification using the support vector machine. IEEE Trans Knowl Data Eng 20(9):1264–1272

    Google Scholar 

  5. Khan L, Awad M, Thuraisingham B (2007) A new intrusion detection system using support vector machines and hierarchical clustering. VLDB J 16(4):507–521

    Google Scholar 

  6. Schmidt M, Gish H (1996) Speaker identification via support vector classifiers and hierarchical clustering. In: Proceedings of IEEE international conference on acoustics, speech, and signal processing, pp 105–108

  7. Zhang J, Liu Y (2004) Cervical cancer detection using SVM-based feature screening. In: Proceedings of international conference on medical image computing and computer-assisted intervention, pp 873–880

  8. Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings KDD-2001, knowledge discovery and data mining, pp 77–86

  9. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues. IEEE Trans Pattern Anal Mach Intell 28(1):69–74

    Google Scholar 

  10. Jayadeva Khemchandai R, Chandra S (2007) Twin support vector machine classification for pattern classification. IEEE Trans Pattern Anal Mach Intell 29(5):905–910

    Google Scholar 

  11. Kumar MA, Gopal M (2009) Least squares twin support vector machines for pattern classification. Expert Syst Appl 36(4):7535–7543

    Google Scholar 

  12. Shao YH, Zhang CH, Wang XB, Deng NY (2011) Improvements on twin support vector machines. IEEE Trans Neural Netw 22(6):962–968

    Google Scholar 

  13. Peng XJ (2011) TPMSVM: a novel twin parametric-margin support vector machine for pattern recognition. Pattern Recognit 44(10):2678–2692

    Google Scholar 

  14. Qi ZQ, Tian YJ, Shi Y (2013) Robust twin support vector machine for pattern classification. Pattern Recognit 46(1):305–316

    Google Scholar 

  15. Tian YJ, Qi ZQ, Ju XC, Shi Y, Liu XH (2014) Nonparallel support vector machines for pattern classification. IEEE Trans Cybern 44(7):1067–1079

    Google Scholar 

  16. Rastogi R, Saigal P, Chandra S (2018) Angle-based twin parametric-margin support vector machine for pattern classification. Knowl Based Syst 139:64–77

    Google Scholar 

  17. Mehrkanoon S, Huang XL, Suykens JAK (2014) Non-parallel support vector classifiers with different loss functions. Neurocomputing 143:294–301

    Google Scholar 

  18. Chen SG, Wu XJ, Zhang RF (2016) A novel twin support vector machine for binary classification problems. Neural Process Lett 44(3):795–811

    Google Scholar 

  19. Ding SF, An YX, Zhang XK, Wu FL, Xue Y (2017) Wavelet twin support vector machine based on glowworm swarm optimization. Neurocomputing 225:157–163

    Google Scholar 

  20. Xu YT, Yang ZJ, Pan XL (2017) A novel twin support vector machine with pinball loss. IEEE Trans Neural Netw Learn Syst 28(2):359–370

    Google Scholar 

  21. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471

    Google Scholar 

  22. Batuwita R, Palade V (2010) FSVM-CIL: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571

    Google Scholar 

  23. Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831

    Google Scholar 

  24. Tao Q, Wang J (2004) A new fuzzy support vector machine based on the weighted margin. Neural Process Lett 20(3):139–150

    Google Scholar 

  25. An WJ, Liang MG (2013) Fuzzy support vector machine based on within-class scatter for classification problems with outliers or noises. Neurocomputing 110(7):101–110

    Google Scholar 

  26. Li K, Ma HY (2013) A fuzzy twin support vector machine algorithm. Int J Appl Innov Eng Manag 2(3):459–465

    Google Scholar 

  27. Chen SG, Wu XJ (2017) A new fuzzy twin support vector machine for pattern classification. Int J Mach Learn Cybern 9(9):1553–1564

    Google Scholar 

  28. Shen H, Li F, Wu ZG, Park JH, Sreeram V (2018) Fuzzy-model-based nonfragile control for nonlinear singularly perturbed systems with semi-Markov jump parameters. IEEE Trans Fuzzy Syst 26(6):3428–3439

    Google Scholar 

  29. Shen H, Men YZ, Wu ZG, Cao JD, Lu GP (2019) Network-based quantized control for fuzzy singularly perturbed semi-Markov jump systems and its application. IEEE Trans Circuits Syst I Regul Pap 66(3):1130–1140

    Google Scholar 

  30. Shen H, Li F, Yan HC, Karimi HR, Lam HK (2018) Finite-time event-triggered H∞ control for T-S fuzzy Markov jump systems. IEEE Trans Fuzzy Syst 26(5):3122–3135

    Google Scholar 

  31. Fan Q, Wang Z, Li DD, Gao DQ, Zha HY (2017) Entropy-based fuzzy support vector machine for imbalance datasets. Knowl Based Syst 115:87–99

    Google Scholar 

  32. Gupta D, Richhariya B, Borah P (2018) A fuzzy twin support vector machine based on information entropy for class imbalance learning. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3551-9

    Google Scholar 

  33. Suykens JAK, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Google Scholar 

  34. Shannon C (2001) A mathematical theory of communication. ACM SIGMOBILE Mobile Comput Commun 5(1):3–55

    Google Scholar 

  35. Benkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7(11):2399–2434

    Google Scholar 

  36. Chen SG, Wu XJ, Xu J (2016) Locality preserving projection twin support vector machine and its application in classification. J Algorithms Comput Technol 10(2):65–72

    Google Scholar 

  37. Ripley BD (2008) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    Google Scholar 

  38. Muphy PM, Aha DW (1992) UCI repository of machine learning databases. University of California, Irvine. http://www.ics.uci.edu/~mlearn. Accessed 10 Jan 2018

  39. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    Google Scholar 

  40. Iman RL, Davenport JM (1980) Approximations of the critical region of the Friedman statistic. Commun Stat Theory Methods 9(6):571–595

    Google Scholar 

  41. The USPS Database. http://www.cs.nyu.edu/roweis/data.html. Accessed 15 Nov 2018

  42. Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-20). Technical report CUCS-005096, February

  43. Martinez AM, Benavente R (1998) The AR face database. CVC technical report #24, June

  44. Musicant DR (1998) NDC: Normally distributed clustered datasets. Computer Science Department, University of Wisconsin, Madison, USA. http://research.cs.wisc.edu/dmi/svm/ndc/. Accessed 16 Feb 2018

  45. Golub GH, Van Loan CF (2012) Matrix computations, vol 3. JHU Press, Baltimore

    Google Scholar 

  46. Nasiri JA, Charkari NM, Jalili S (2015) Least squares twin multi-class classification support vector machine. Pattern Recognit 48(3):984–992

    Google Scholar 

  47. Chen SG, Wu XJ (2017) Multiple birth least squares support vector machine for multi-class classification. Int J Mach Learn Cybern 8(6):1731–1742

    Google Scholar 

  48. Tang JJ, Li DW, Tian YJ, Liu DL (2018) Multi-view learning based on nonparallel support vector machine. Knowl Based Syst 158:94–108

    Google Scholar 

  49. Houthuys L, Langone R, Suykens JAK (2018) Multi-view least squares support vector machines classification. Neurocomputing 282:78–88

    Google Scholar 

  50. Qi ZQ, Tian YJ, Shi Y (2012) Laplacian twin support vector machine for semi-supervised classification. Neural Netw 35:46–53

    Google Scholar 

  51. Chen WJ, Shao YH, Deng NY, Feng ZL (2014) Laplacian least squares twin support vector machine for semi-supervised classification. Neurocomputing 145:465–476

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China (Grant No. 61702012), the University Outstanding Young Talent Support Project of Anhui Province of China (Grant No. gxyq2017026), the University Natural Science Research Project of Anhui Province of China (Grant Nos. KJ2016A431, KJ2017A361 and KJ2017A368) and the Program for Innovative Research Team in Anqing Normal University.

Author information

Authors and Affiliations

  1. School of Mathematics and Computational Science, Anqing Normal University, Anqing, 246133, Anhui, People’s Republic of China

    Sugen Chen, Fenglin Chen & Bingbing Liu

  2. Key Laboratory of Modeling, Simulation and Control of Complex Ecosystem in Dabie Mountains of Anhui Higher Education Institutes, Anqing Normal University, Anqing, 246133, China

    Sugen Chen

  3. School of Science, Jiangnan University, Wuxi, 214122, Jiangsu, People’s Republic of China

    Junfeng Cao

Authors
  1. Sugen Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Junfeng Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Fenglin Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Bingbing Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Sugen Chen.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, S., Cao, J., Chen, F. et al. Entropy-Based Fuzzy Least Squares Twin Support Vector Machine for Pattern Classification. Neural Process Lett 51, 41–66 (2020). https://doi.org/10.1007/s11063-019-10078-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-019-10078-w

Keywords