Skip to main content
SpringerLink
Log in

Locality Density-Based Fuzzy Multiple Empirical Kernel Learning

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Multiple Empirical Kernel Learning (MEKL) explicitly maps the samples to empirical feature spaces, in which the feature vectors of the mapped samples are explicitly presented. Thus with the explicit representation of the samples, almost all algorithms can be kernelized directly, which is much easier in processing and analyzing the structure of the empirical feature spaces. However, in conventional MEKL, samples are treated to belong to one exact class, and contribute the same importance to the decision surface. However, in many real-world applications, input samples may not be fully assigned to one class. MEKL suffers the instinct drawbacks in representing these samples. To overcome this problem, we assign a fuzzy membership to each mapped sample in each feature space and reformulate MEKL to the Fuzzy MEKL (FMEKL) in which different samples in different feature spaces can make different contributions to the learning of decision surface. Moreover, we propose a novel fuzzy membership evaluation approach named locality density-based fuzzy membership evaluation, which assign larger fuzzy membership to the samples with higher local density. Thus, FMEKL by adopting the locality density-based fuzzy membership evaluation is named as Locality Density-based Fuzzy Multiple Empirical Kernel Learning (LD-FMEKL). Experimental results on both balanced and imbalanced real-world datasets validate that LD-FMEKL outperforms the compared algorithms. The contributions of this work are: (i) reformulating traditional MEKL to fuzzy multiple empirical kernel learning; (ii) introducing an alternative locality density-based fuzzy membership evaluation approach; (iii) proposing the locality density-based FMEKL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. The value can be computed in the website http://graphpad.com/quickcalcs/PValue1.cfm.

  2. Yale Face database is publicly available at http://vision.ucsd.edu/datasets/yale_face_dataset_original/yalefaces.zip.

  3. Letter digit dataset is publicly available at http://www.cam-orl.co.uk.

  4. Coil-20 database is publicly available at http://www.cs.columbia.edu/CAVE/coil-20.html.

References

  1. Alcalá-Fdez J, Fernandez A, Luengo J, Derrac J, Garcıa S, Sánchez L, Herrera F (2011) KEEL: A Software Tool to Assess Evolutionary Algorithms for Data Mining Problems. Journal of Multiple-Valued Logic and Soft Computing 17:2–3

    Google Scholar 

  2. Alcalá-Fdez J, Sanchez L, Garcia S, del Jesus MJ, Ventura S, Garrell JM, Otero J, Romero C, Bacardit J, Rivas VM (2009) KEEL: A Software Tool to Assess Evolutionary Algorithms for Data Mining Problems. Soft Computing 13(3):307–318

    Article  Google Scholar 

  3. Althloothi S, Mahoor MH, Zhang X, Voyles RM (2014) Human Activity Recognition Using Multi-features and Multiple Kernel Learning. Pattern Recognition 47:1800–1812

    Article  Google Scholar 

  4. Bach FR, Lanckriet GR, Jordan MI (2004) Multiple Kernel Learning, Conic Duality, and the SMO Algorithm. In: International Conference on Machine Learning, pp 6–13 ACM

  5. Bache K, Lichman M (2013) UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, http://archive.ics.uci.edu/ml

  6. Barandela R, Sánchez JS, Garcıa V, Rangel E (2003) Strategies for Learning in Class Imbalance Problems. Pattern Recognition 36(3):849–851

    Article  Google Scholar 

  7. Berline N, Getzler E, Vergne M (2004) Heat Kernels and Dirac Operators. Springer-Verlag, Berlin, New York

    MATH  Google Scholar 

  8. Bucak SS, Jin R, Jain AK (2014) Multiple Kernel Learning for Visual Object Eecognition: A Review. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(7):1354–1369

    Article  Google Scholar 

  9. Chen D, He Q, Wang X (2010) FRSVMs: Fuzzy Rough Set based Support Vector Machines. Fuzzy Sets and Systems 161(4):596–607

    Article  MathSciNet  Google Scholar 

  10. Cortes C, Vapnik V (1995) Support Vector Machine. Machine Learning 20(3):273–297

    MATH  Google Scholar 

  11. Demšar J (2006) Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research 7:1–30

    MathSciNet  MATH  Google Scholar 

  12. Deng Z, Choi K, Jiang Y, Wang S (2014) Generalized Hidden-Mapping Ridge Regression, Knowledge-Leveraged Inductive Transfer Learning for Neural Networks, Fuzzy Systems and Kernel Methods. IEEE Transactions on Cybernetics 44(12):2585–2599

    Article  Google Scholar 

  13. Galar M, Fernandez A, Barrenechea E, Bustince H, Herrera F (2012) A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-based Approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42(4):463–484

    Article  Google Scholar 

  14. Gönen M, Alpaydın E (2013) Localized Algorithms for Multiple Kernel Learning. Pattern Recognition 46(3):795–807

    Article  MATH  Google Scholar 

  15. He H, Garcia EA (2009) Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering 21(9):1263–1284

    Article  Google Scholar 

  16. Huang J, Ling CX (2005) Using AUC and Accuracy in Evaluating Learning Algorithms. IEEE Transactions on Knowledge and Data Engineering 17(3):299–310

    Article  Google Scholar 

  17. Jian L, Xia Z, Liang X, Gao C (2011) Design of a Multiple Kernel Learning Algorithm for LS-SVM by Convex Programming. Neural Networks 24(5):476–483

    Article  MATH  Google Scholar 

  18. Jiang X, Yi Z, Lv JC (2006) Fuzzy SVM with a new Fuzzy Membership Function. Neural Computing and Applications 15(3–4):268–276

    Article  Google Scholar 

  19. Khalilia MA, Bezdek J, Popescu M, Keller JM (2014) Improvements to the Relational Fuzzy C-means Clustering Algorithm. Pattern Recognition 47(12):3920–3930

    Article  Google Scholar 

  20. Lam HK (2011) Polynomial Fuzzy-Model-Based Control Systems: Stability Analysis Via Piecewise-Linear Membership Functions. IEEE Transactions on Fuzzy Systems 19(3):588–593

    Article  Google Scholar 

  21. Lam HK, Narimani M (2010) Quadratic-Stability Analysis of Fuzzy-Model-Based Control Systems Using Staircase Membership Functions. IEEE Transactions on Fuzzy Systems 18(1):125–137

    Article  Google Scholar 

  22. Leski J (2004) Kernel Ho-Kashyap Classifier with Generalization Control. International Journal of Applied Mathematics and Computer Science 14(1):53–62

    MathSciNet  MATH  Google Scholar 

  23. Leski JM (2004) An \(\varepsilon \)-margin Nonlinear Classifier based on Fuzzy If-then Rules. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 34(1):68–76

    Article  MathSciNet  Google Scholar 

  24. Li K, Lu X (2011) A Double Margin based Fuzzy Support Vector Machine Algorithm. Journal of Computers 6(9):1962–1970

    Google Scholar 

  25. Li X, Yang G (2014) Fault Detection for TCS Fuzzy Systems With Unknown Membership Functions. IEEE Transactions on Fuzzy Systems 22(1):139–152

    Article  Google Scholar 

  26. Liang Z, Xia S, Zhou Y, Zhang L (2013) Training Lp-norm Multiple Kernel Learning in the Primal. Neural Networks 46:172–182

    Article  MATH  Google Scholar 

  27. Lin C, Wang S (2002) Fuzzy Support Vector Machines. IEEE Transactions on Neural Networks 13(2):464–471

    Article  Google Scholar 

  28. Lin KP (2014) A Novel Evolutionary Kernel Intuitionistic Fuzzy \(C\)-means Clustering Algorithm. IEEE Transactions on Fuzzy Systems 22(5):1074–1087

    Article  Google Scholar 

  29. Łkeski J (2003) Ho-Kashyap Classifier with Generalization Control. Pattern Recognition Letters 24(14):2281–2290

    Article  MATH  Google Scholar 

  30. Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face Recognition Using Kernel Direct Discriminant Analysis Algorithms. IEEE Transactions on Neural Networks 14(1):117–126

    Article  Google Scholar 

  31. Maldonado S, López J (2014) Imbalanced Data Classification Using Second-order Cone Programming Support Vector Machines. Pattern Recognition 47(5):2070–2079

    Article  MATH  Google Scholar 

  32. Nazarpour A, Adibi P (2014) Two-stage Multiple Kernel Learning for Supervised Dimensionality Reduction. Pattern Recognition 48(5):1854–1862

    Article  Google Scholar 

  33. Sonnenburg S, Rätsch G, Schäfer C (2006) A General and Efficient Multiple Kernel Learning Algorithm. Neural Information Processing Systems 18:1273–1280

    Google Scholar 

  34. Tsang ECC, Yeung DS, Chan PPK (2003) Fuzzy Support Vector Machines for Solving Two-class Problems. In: International Conference on Machine Learning and Cybernetics, volume 2, pp 1080–1083. IEEE, (2003)

  35. Vapnik V (2000) The Nature of Statistical Learning Theory. Springer Science and Business Media,

  36. Wang Y, Wang S, Lai KK (2005) A new Fuzzy Support Vector Machine to Evaluate Credit Risk. IEEE Transactions on Fuzzy Systems 13(6):820–831

    Article  Google Scholar 

  37. Wang Z, Chen SC, Sun TK (2008) MultiK-MHKS: A Novel Multiple Kernel Learning Algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2):348–353

    Article  Google Scholar 

  38. Wong T (2015) Performance Evaluation of Classification Algorithms by K-fold and Leave-one-out Cross Validation. Pattern Recognition 48(9):2839–2846

    Article  MATH  Google Scholar 

  39. Wu Z, Zhang H, Liu J (2014) A Fuzzy Support Vector Machine Algorithm for Classification based on a Novel PIM Fuzzy Clustering Method. Neurocomputing 125:119–124

    Article  Google Scholar 

  40. Xiong H, Swamy MNS, Ahmad MO (2005) Optimizing the Kernel in the Empirical Feature Space. IEEE Transactions on Neural Networks 16(2):460–474

    Article  Google Scholar 

  41. Yang X, Zhang G, Lu J, Ma J (2011) A Kernel Fuzzy C-means Clustering-based Fuzzy Support Vector Machine Algorithm for Classification Problems with Outliers or Noises. IEEE Transactions on Fuzzy Systems 19(1):105–115

    Article  Google Scholar 

  42. Ye J (2005) Characterization of a Family of Algorithms for Generalized Discriminant Analysis on Undersampled Problems. Journal of Machine Learning Research 6:483–502

    MathSciNet  MATH  Google Scholar 

  43. Ye J, Li T, Xiong T, Janardan R (2004) Using Uncorrelated Discriminant Analysis for Tissue Classification With Gene Expression Data. IEEE Transactions on Computational Biology and Bioinformatics 1(4):181–190

    Article  Google Scholar 

  44. Zhang H, Liu D (2006) Fuzzy Modeling and Fuzzy Control. Springer Science and Business Media, Boston, Birkhäuser

    MATH  Google Scholar 

  45. Zhang H, Quan Y (2001) Modeling, Identification, and Control of a Class of Nonlinear Systems. IEEE Transactions on Fuzzy Systems 9(2):349–354

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Natural Science Foundations of China under Grant No. 61672227, Shuguang Program supported by Shanghai Education Development Foundation and Shanghai Municipal Education Commission, and Action Plan for Innovation on Science and Technology Projects of Shanghai under Grant No. 16511101000.

Author information

Authors and Affiliations

  1. Key Laboratory of Advanced Control and Optimization for Chemical Processes, Ministry of Education, East China University of Science and Technology, Shanghai, 200237, People’s Republic of China

    Qi Fan & Zhe Wang

  2. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, 200237, People’s Republic of China

    Qi Fan, Zhe Wang & Daqi Gao

Authors
  1. Qi Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhe Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daqi Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Zhe Wang or Daqi Gao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, Q., Wang, Z. & Gao, D. Locality Density-Based Fuzzy Multiple Empirical Kernel Learning. Neural Process Lett 49, 1485–1509 (2019). https://doi.org/10.1007/s11063-018-9881-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-9881-x

Keywords

Search

Navigation