Abstract
Fuzzy neural network (FNN) and support vector machine (SVM) are popular techniques for pattern classification. FNN has a good local representation power and human-like reasoning benefit. But the learning algorithms used in most FNN classifiers only focus on minimizing empirical risk. SVM has excellent generalization performance due to focusing on simultaneously minimizing both empirical and expected risks. But this performance is dependent on choosing an appropriate kernel function. To overcome these challenges some efforts have been made to combine SVM and FNN; however, the methods proposed so far are faced with some major problems such as significantly increasing number of fuzzy rules. In this paper a support vector-based fuzzy classifier with adaptive kernel (SVFC-AK) is proposed. SVFC-AK implements the mentioned combining idea and yet not have disadvantages of the previous methods. This is realized by leveraging an underlying relation between FNN and kernel SVM. More precisely, SVFC-AK is a TS-type FNN for which the parameters are tuned through a new adaptive fuzzy kernel SVM. Also, the proposed fuzzy kernel, which is calculated automatically, is inspired by the fuzzy rule-based representation of FNN. Moreover, the fuzzy rules of SVFC-AK are generated using a modified clustering method based on Gaussian mixture model, which determines the number of required rules automatically. The experimental results illustrate that the classification performance of SVFC-AK is competitive or even better than some related methods, while the number of its fuzzy rules or hyper-parameters is substantially smaller.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Aeberhard S, Coomans D, De Vel O (1992) The classification performance of rda. Department of Computer Science and Department of Mathematics and Statistics, James Cook University of North Queensland, Technical Report, pp 92–01
Aiolli F, Donini M (2015) Easymkl: a scalable multiple kernel learning algorithm. Neurocomputing 169:215–224
Argyriou A, Micchelli CA, Pontil M (2005) Learning convex combinations of continuously parameterized basic kernels. In: Auer P, Meir R (eds) Learning theory. Springer, Berlin, pp 338–352
Batuwita R, Palade V (2010) Fsvm-cil: fuzzy support vector machines for class imbalance learning. IEEE Trans Fuzzy Syst 18(3):558–571
Berthold MR, Diamond J (1998) Constructive training of probabilistic neural networks. Neurocomputing 19(1):167–183
Blake CL, Merz CJ (1998) Uci repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html
Chai Y, Jia L, Zhang Z (2009) Mamdani model based adaptive neural fuzzy inference system and its application. Int J Comput Intell 5(1):22–29
Chang CC, Lin CJ (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):27. http://www.csie.nte.edu.tw/~cjlin/libsvm/
Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1–3):131–159
Chen Y, Wang JZ (2003) Support vector learning for fuzzy rule-based classification systems. IEEE Trans Fuzzy Syst 11(6):716–728
Cheng W, Juang C (2013) A fuzzy model with online incremental svm and margin-selective gradient descent learning for classification problems. IEEE Trans Fuzzy Syst 21(99):324–337
Chiang JH, Hao PY (2004) Support vector learning mechanism for fuzzy rule-based modeling: a new approach. IEEE Trans Fuzzy Syst 12(1):1–12
Cortes C, Mohri M, Rostamizadeh A (2010) Two-stage learning kernel algorithms. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 239–246
Cristianini N, Kandola J, Elisseeff A, Shawe-Taylor J (2006) On kernel target alignment. In: Holmes DE, Jain LC (eds) Innovations in machine learning. Springer, Berlin, pp 205–256
Cristianini N, Shawe-Taylor J (2000) An Introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge
Diaz JG (2016) Supervised machine learning with kernel embeddings of fuzzy sets and probability measures. Ph.D. thesis, IME USP
Ebadzadeh MM, Salimi-Badr A (2015) Cfnn: correlated fuzzy neural network. Neurocomputing 148:430–444
Ester M, Kriegel HP, Sander J, Xu X et al. (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Kdd, vol. 96, pp 226–231
Fan Q, Wang Z, Li D, Gao D, Zha H (2017) Entropy-based fuzzy support vector machine for imbalanced datasets. Knowl Based Syst 115:87–99
Forghani Y, Yazdi HS (2014) Robust support vector machine-trained fuzzy system. Neural Netw 50:154–165
Fraley C, Raftery AE (1998) How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput J 41(8):578–588
Fraley C, Raftery AE (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97(458):611–631
Gabrys B, Bargiela A (2000) General fuzzy min-max neural network for clustering and classification. IEEE Trans Neural Netw 11(3):769–783
Guevara J, Hirata R, Canu S (2013) Kernel functions in takagi-sugeno-kang fuzzy system with nonsingleton fuzzy input. In: 2013 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–8
Guevara J, Hirata R, Canu S (2014) Positive definite kernel functions on fuzzy sets. In: 2014 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 439–446
Guevara J, Hirata R, Canu S (2017) Cross-product kernels for fuzzy set similarity. Working paper or preprint
Heo G, Gader P (2011) Robust kernel discriminant analysis using fuzzy memberships. Pattern Recognit 44(3):716–723
Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425
Huang HC, Chuang YY, Chen CS (2012) Multiple kernel fuzzy clustering. IEEE Trans Fuzzy Syst 20(1):120–134
Jawanpuria P, Nath JS, Ramakrishnan G (2015) Generalized hierarchical kernel learning. J Mach Learn Res 16:617–652
Juang CF (2002) A tsk-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms. IEEE Trans Fuzzy Syst 10(2):155–170
Juang CF, Chen GC (2012) A ts fuzzy system learned through a support vector machine in principal component space for real-time object detection. IEEE Trans Ind Electron 59(8):3309–3320
Juang CF, Chiu SH, Chang SW (2007) A self-organizing ts-type fuzzy network with support vector learning and its application to classification problems. IEEE Trans Fuzzy Syst 15(5):998–1008
Juang CF, Lin CT (1998) An online self-constructing neural fuzzy inference network and its applications. IEEE Trans Fuzzy Syst 6(1):12–32
Kanzawa Y, Endo Y, Miyamoto S (2010) On kernel fuzzy c-means for data with tolerance using explicit mapping for kernel data analysis. In: 2010 IEEE international conference on fuzzy systems (FUZZ). IEEE, pp 1–6
Karypis G, Han EH, Kumar V (1999) Chameleon: hierarchical clustering using dynamic modeling. Computer 32(8):68–75
Khan NM, Ksantini R, Ahmad IS, Guan L (2014) Sn-svm: a sparse nonparametric support vector machine classifier. Signal Image Video Process 8(8):1625–1637
Kloft M, Brefeld U, Sonnenburg S, Zien A (2011) Lp-norm multiple kernel learning. J Mach Learn Res 12:953–997
Lanckriet GR, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72
Lee HM, Chen CM, Chen JM, Jou YL (2001) An efficient fuzzy classifier with feature selection based on fuzzy entropy. IEEE Trans Syst Man Cybern Part B Cybern 31(3):426–432
Lee MM, Keerthi SS, Ong CJ, DeCoste D (2004) An efficient method for computing leave-one-out error in support vector machines with gaussian kernels. IEEE Trans Neural Netw 15(3):750–757
Lin CF, Wang SD (2002) Fuzzy support vector machines. IEEE Trans Neural Netw 13(2):464–471
Lin CT, Lee CSG (1996) Neural fuzzy systems: a neural-fuzzy synergism to intelligent systems. Prentice-Hall, Upper Saddle River
Lin CT, Yeh CM, Hsu CF (2004) Fuzzy neural network classification using support vector machine. In: Proceedings of IEEE international symposium on circuits and systems, p 724727
Lin CT, Yeh CM, Liang SF, Chung JF, Kumar N (2006) Support-vector-based fuzzy neural network for pattern classification. IEEE TransFuzzy Syst 14(1):31–41
Mercer J (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, pp 415–446
Michie D, Spiegelhalter DJ, Taylor CC (1994) Machine learning, neural and statistical classification. ftp://ftp.stams.strath.ac.uk/pub/
Ong CS, Williamson RC, Smola AJ (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6:1043–1071
Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2008) Simplemkl. J Mach Learn Res 9(11):2491–2521
Saitoh S, Saitoh S (1988) Theory of reproducing kernels and its applications, vol 189. Longman Scientific and Technical Harlow, Harlow
Sollich P (2002) Bayesian methods for support vector machines: evidence and predictive class probabilities. Mach Learn 46(1–3):21–52
Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Netw 16(5):785–792
Vapnik V (2000) The nature of statistical learning theory. Springer, Berlin
Vlassis N, Likas A (2002) A greedy em algorithm for gaussian mixture learning. Neural Process Lett 15(1):77–87
Wang J, Hua J, Guo J (2010) Fuzzy maximum scatter discriminant analysis with kernel methods. In: 2010 seventh international conference on fuzzy systems and knowledge discovery (FSKD), vol. 2. IEEE, pp 560–564
Wang Y, Wang S, Lai KK (2005) A new fuzzy support vector machine to evaluate credit risk. IEEE Trans Fuzzy Syst 13(6):820–831
Xiong H, Swamy M, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460–474
Zhong S, Chen D, Xu Q, Chen T (2013) Optimizing the gaussian kernel function with the formulated kernel target alignment criterion for two-class pattern classification. Pattern Recognit 46(7):2045–2054
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Additional information
This work has been done when Shahram Khadivi was with Amirkabir University of Technology.
Rights and permissions
About this article
Cite this article
Ganji, H., Khadivi, S. & Ebadzadeh, M.M. Support vector-based fuzzy classifier with adaptive kernel. Neural Comput & Applic 31, 2117–2130 (2019). https://doi.org/10.1007/s00521-017-3170-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-017-3170-x