Skip to main content

Advertisement

SpringerLink
Log in

Support vector-based fuzzy classifier with adaptive kernel

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Fuzzy neural network (FNN) and support vector machine (SVM) are popular techniques for pattern classification. FNN has a good local representation power and human-like reasoning benefit. But the learning algorithms used in most FNN classifiers only focus on minimizing empirical risk. SVM has excellent generalization performance due to focusing on simultaneously minimizing both empirical and expected risks. But this performance is dependent on choosing an appropriate kernel function. To overcome these challenges some efforts have been made to combine SVM and FNN; however, the methods proposed so far are faced with some major problems such as significantly increasing number of fuzzy rules. In this paper a support vector-based fuzzy classifier with adaptive kernel (SVFC-AK) is proposed. SVFC-AK implements the mentioned combining idea and yet not have disadvantages of the previous methods. This is realized by leveraging an underlying relation between FNN and kernel SVM. More precisely, SVFC-AK is a TS-type FNN for which the parameters are tuned through a new adaptive fuzzy kernel SVM. Also, the proposed fuzzy kernel, which is calculated automatically, is inspired by the fuzzy rule-based representation of FNN. Moreover, the fuzzy rules of SVFC-AK are generated using a modified clustering method based on Gaussian mixture model, which determines the number of required rules automatically. The experimental results illustrate that the classification performance of SVFC-AK is competitive or even better than some related methods, while the number of its fuzzy rules or hyper-parameters is substantially smaller.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Aeberhard S, Coomans D, De Vel O (1992) The classification performance of rda. Department of Computer Science and Department of Mathematics and Statistics, James Cook University of North Queensland, Technical Report, pp 92–01

  2. Aiolli F, Donini M (2015) Easymkl: a scalable multiple kernel learning algorithm. Neurocomputing 169:215–224

    Article  Google Scholar 

  3. Argyriou A, Micchelli CA, Pontil M (2005) Learning convex combinations of continuously parameterized basic kernels. In: Auer P, Meir R (eds) Learning theory. Springer, Berlin, pp 338–352

    Chapter  MATH  Google Scholar 

  4. Batuwita R, Palade V (2010) Fsvm-cil: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571

    Article  Google Scholar 

  5. Berthold MR, Diamond J (1998) Constructive training of probabilistic neural networks. Neurocomputing 19(1):167–183

    Article  Google Scholar 

  6. Blake CL, Merz CJ (1998) Uci repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html

  7. Chai Y, Jia L, Zhang Z (2009) Mamdani model based adaptive neural fuzzy inference system and its application. Int J Comput Intell 5(1):22–29

    Google Scholar 

  8. Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27. http://www.csie.nte.edu.tw/~cjlin/libsvm/

  9. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159

    Article  MATH  Google Scholar 

  10. Chen Y, Wang JZ (2003) Support vector learning for fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 11(6):716–728

    Article  Google Scholar 

  11. Cheng W, Juang C (2013) A fuzzy model with online incremental svm and margin-selective gradient descent learning for classification problems. IEEE Trans Fuzzy Syst 21(99):324–337

    Google Scholar 

  12. Chiang JH, Hao PY (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12

    Article  Google Scholar 

  13. Cortes C, Mohri M, Rostamizadeh A (2010) Two-stage learning kernel algorithms. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 239–246

  14. Cristianini N, Kandola J, Elisseeff A, Shawe-Taylor J (2006) On kernel target alignment. In: Holmes DE, Jain LC (eds) Innovations in machine learning. Springer, Berlin, pp 205–256

    Chapter  Google Scholar 

  15. Cristianini N, Shawe-Taylor J (2000) An Introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  16. Diaz JG (2016) Supervised machine learning with kernel embeddings of fuzzy sets and probability measures. Ph.D. thesis, IME USP

  17. Ebadzadeh MM, Salimi-Badr A (2015) Cfnn: correlated fuzzy neural network. Neurocomputing 148:430–444

    Article  Google Scholar 

  18. Ester M, Kriegel HP, Sander J, Xu X et al. (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd, vol. 96, pp 226–231

  19. Fan Q, Wang Z, Li D, Gao D, Zha H (2017) Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl Based Syst 115:87–99

    Article  Google Scholar 

  20. Forghani Y, Yazdi HS (2014) Robust support vector machine-trained fuzzy system. Neural Netw 50:154–165

    Article  MATH  Google Scholar 

  21. Fraley C, Raftery AE (1998) How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput J 41(8):578–588

    Article  MATH  Google Scholar 

  22. Fraley C, Raftery AE (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97(458):611–631

    Article  MathSciNet  MATH  Google Scholar 

  23. Gabrys B, Bargiela A (2000) General fuzzy min-max neural network for clustering and classification. IEEE Trans Neural Netw 11(3):769–783

    Article  Google Scholar 

  24. Guevara J, Hirata R, Canu S (2013) Kernel functions in takagi-sugeno-kang fuzzy system with nonsingleton fuzzy input. In: 2013 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–8

  25. Guevara J, Hirata R, Canu S (2014) Positive definite kernel functions on fuzzy sets. In: 2014 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 439–446

  26. Guevara J, Hirata R, Canu S (2017) Cross-product kernels for fuzzy set similarity. Working paper or preprint

  27. Heo G, Gader P (2011) Robust kernel discriminant analysis using fuzzy memberships. Pattern Recognit 44(3):716–723

    Article  MATH  Google Scholar 

  28. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Article  Google Scholar 

  29. Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134

    Article  Google Scholar 

  30. Jawanpuria P, Nath JS, Ramakrishnan G (2015) Generalized hierarchical kernel learning. J Mach Learn Res 16:617–652

    MathSciNet  MATH  Google Scholar 

  31. Juang CF (2002) A tsk-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms. IEEE Trans Fuzzy Syst 10(2):155–170

    Article  Google Scholar 

  32. Juang CF, Chen GC (2012) A ts fuzzy system learned through a support vector machine in principal component space for real-time object detection. IEEE Trans Ind Electron 59(8):3309–3320

    Article  Google Scholar 

  33. Juang CF, Chiu SH, Chang SW (2007) A self-organizing ts-type fuzzy network with support vector learning and its application to classification problems. IEEE Trans Fuzzy Syst 15(5):998–1008

    Article  Google Scholar 

  34. Juang CF, Lin CT (1998) An online self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–32

    Article  Google Scholar 

  35. Kanzawa Y, Endo Y, Miyamoto S (2010) On kernel fuzzy c-means for data with tolerance using explicit mapping for kernel data analysis. In: 2010 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–6

  36. Karypis G, Han EH, Kumar V (1999) Chameleon: hierarchical clustering using dynamic modeling. Computer 32(8):68–75

    Article  Google Scholar 

  37. Khan NM, Ksantini R, Ahmad IS, Guan L (2014) Sn-svm: a sparse nonparametric support vector machine classifier. Signal Image Video Process 8(8):1625–1637

    Article  Google Scholar 

  38. Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12:953–997

    MathSciNet  MATH  Google Scholar 

  39. Lanckriet GR, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72

    MathSciNet  MATH  Google Scholar 

  40. Lee HM, Chen CM, Chen JM, Jou YL (2001) An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans Syst Man Cybern Part B Cybern 31(3):426–432

    Article  Google Scholar 

  41. Lee MM, Keerthi SS, Ong CJ, DeCoste D (2004) An efficient method for computing leave-one-out error in support vector machines with gaussian kernels. IEEE Trans Neural Netw 15(3):750–757

    Article  Google Scholar 

  42. Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471

    Article  Google Scholar 

  43. Lin CT, Lee CSG (1996) Neural fuzzy systems: a neural-fuzzy synergism to intelligent systems. Prentice-Hall, Upper Saddle River

    Google Scholar 

  44. Lin CT, Yeh CM, Hsu CF (2004) Fuzzy neural network classification using support vector machine. In: Proceedings of IEEE international symposium on circuits and systems, p 724727

  45. Lin CT, Yeh CM, Liang SF, Chung JF, Kumar N (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE TransFuzzy Syst 14(1):31–41

    Google Scholar 

  46. Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, pp 415–446

  47. Michie D, Spiegelhalter DJ, Taylor CC (1994) Machine learning, neural and statistical classification. ftp://ftp.stams.strath.ac.uk/pub/

  48. Ong CS, Williamson RC, Smola AJ (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6:1043–1071

    MathSciNet  MATH  Google Scholar 

  49. Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) Simplemkl. J Mach Learn Res 9(11):2491–2521

    MathSciNet  MATH  Google Scholar 

  50. Saitoh S, Saitoh S (1988) Theory of reproducing kernels and its applications, vol 189. Longman Scientific and Technical Harlow, Harlow

    MATH  Google Scholar 

  51. Sollich P (2002) Bayesian methods for support vector machines: evidence and predictive class probabilities. Mach Learn 46(1–3):21–52

    Article  MATH  Google Scholar 

  52. Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Netw 16(5):785–792

    Article  Google Scholar 

  53. Vapnik V (2000) The nature of statistical learning theory. Springer, Berlin

    Book  MATH  Google Scholar 

  54. Vlassis N, Likas A (2002) A greedy em algorithm for gaussian mixture learning. Neural Process Lett 15(1):77–87

    Article  MATH  Google Scholar 

  55. Wang J, Hua J, Guo J (2010) Fuzzy maximum scatter discriminant analysis with kernel methods. In: 2010 seventh international conference on fuzzy systems and knowledge discovery (FSKD), vol. 2. IEEE, pp 560–564

  56. Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831

    Article  Google Scholar 

  57. Xiong H, Swamy M, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460–474

    Article  Google Scholar 

  58. Zhong S, Chen D, Xu Q, Chen T (2013) Optimizing the gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification. Pattern Recognit 46(7):2045–2054

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering Department, Amirkabir University of Technology, No. 424, Hafez Ave, Tehran, Iran

    Hamed Ganji, Shahram Khadivi & Mohammad Mehdi Ebadzadeh

Authors
  1. Hamed Ganji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shahram Khadivi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Mehdi Ebadzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shahram Khadivi.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Additional information

This work has been done when Shahram Khadivi was with Amirkabir University of Technology.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ganji, H., Khadivi, S. & Ebadzadeh, M.M. Support vector-based fuzzy classifier with adaptive kernel. Neural Comput & Applic 31, 2117–2130 (2019). https://doi.org/10.1007/s00521-017-3170-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3170-x

Keywords

Search

Navigation