Skip to main content
SpringerLink
Log in

Generalized eigenvalue extreme learning machine for classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Extreme learning machine (ELM) has attracted widespread attention due to its simple, quick and good performance. In this work, in order to deal with cross data quickly and efficiently, we first propose generalized eigenvalue proximal extreme learning machine (GEPELM). It takes the form of ratio into consideration to seek for two non-parallel separating hyperplanes in ELM feature space, each of which is close to the samples of its own class and far away from the others simultaneously. Then generalized eigenvalue algorithm is adopted to solve, which incurs GEPELM to enjoy faster calculation speed than TELM. Further, improved generalized eigenvalue proximal extreme learning machine (IGEPELM) is proposed, which uses minus instead of ratio to avoid singular value phenomenon and further mitigate the computational burden by solving two standard eigenvalue problems. To further improve classification performance, generalized eigenvalue proximal extreme learning machine based on inter-class graph (GGEPELM) is proposed, which incorporates the geometric structure information of dissimilar samples into the guideline of GEPELM. In addition, the proposed classifiers are all extended to kernel ELM framework to handle non-linear data more precisely. Moreover, Sherman-Morrison-Woodbury formula is utilized to reduce time complexity of matrix inversion. Simultaneously, a quick solution strategy is incorporated into GEPELMs and GGEPELMs to mitigate the burden of solving large-scale problems. The numerical simulations are carried out on three databases including a benchmark database, an artificial database and a practical application database, which demonstrates the proposed classifiers enjoy high computational speed, good generalization performance and insensitivity to parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Huang G, Zhu Q, Siew C (2006) Extreme learning machine Theory and applications. Neurocomputing 70:489–501

    Article  Google Scholar 

  2. Huang G, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–C163

    Article  Google Scholar 

  3. Wang Y, Yang L, Yuan C (2019) A robust outlier control framework for classification designed with family of homotopy loss function. Neural Netw Offic J Int Neural Netw Soc 112:41–53

    Article  Google Scholar 

  4. Barreto G, Barros AP (2016) A robust extreme learning machine for pattern classification with outliers. Neurocomputing 176:3–13

    Article  Google Scholar 

  5. Ri J, Tian G, Liu Y, Xu W, Lou J (2020) Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning. Int J Mach Learn Cybern 1–14

  6. Yang L, Zhang S (2016) A sparse extreme learning machine framework by continuous optimization algorithms and its application in pattern recognition. Eng Appl Artif Intell 53:176–189

    Article  Google Scholar 

  7. Mariani V, Och S, Coelho L, Domingues E (2019) Pressure prediction of a spark ignition single cylinder engine using optimized extreme learning machine models. Appl Energy 249:204–221

    Article  Google Scholar 

  8. Chacko BP, Babu AP (2011) Online sequential extreme learning machine based handwritten character recognition. IEEE Technol Stud Symp 142–147

  9. Yahia S, Said S, Zaied M (2019) Deep wavelet extreme learning machine for data classification. CISIS-ICEUTE

  10. Li Y, Wang Y, Chen Z, Zou R (2020) Bayesian robust multi-extreme learning machine. Knowl Based Syst 210:106468

    Article  Google Scholar 

  11. Boswell D (2002) Introduction to support vector machines

  12. Sanz H, Reverter F, Valim C (2020) Enhancing SVM for survival data using local invariances and weighting. BMC Bioinforma 21

  13. Song Q, Yan G, Tang G, Ansari F (2020) Robust principal component analysis and support vector machine for detection of microcracks with distributed optical fiber sensors. Mech Syst Signal Process 146:107019

    Article  Google Scholar 

  14. Singh S, Parmar KS, Makkhan SJ, Kaur J, Peshoria S, Kumar J (2020) Study of ARIMA and least square support vector machine (LS-SVM) models for the prediction of SARS-cov-2 confirmed cases in the most affected countries. Chaos, Solitons Fract 139:110086–110086

    Article  MathSciNet  Google Scholar 

  15. Chorowski J, Wang J, Zurada J (2014) Review and performance comparison of SVM- and ELM-based classifiers. Neurocomputing 128:507–516

    Article  Google Scholar 

  16. Huang G, Huang G, Song S, You K (2015) Trends in extreme learning machines a review. Neural Netw Offic J Int Neur Netw Soc 61:32–48

    Article  Google Scholar 

  17. Huang G, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163

    Article  Google Scholar 

  18. Mangasarian OL, Wild EW (2006) Multisurface proximal support vector machine classification via generalized eigenvalues.[J]. IEEE Trans Pattern Anal Mach Intell 28(1):69

    Article  Google Scholar 

  19. Guarracino M, Cifarelli C, Seref O, Pardalos P (2007) A classification method based on generalized eigenvalue problems. Optim Methods Softw 22:73–81

    Article  MathSciNet  Google Scholar 

  20. Xu Y (2007) Localized Proximal Support Vector Machine via Generalized Eigenvalues. Chinese J Comput

  21. Shao Y, Deng N, Chen W, Wang Z (2013) Improved generalized eigenvalue proximal support vector machine. IEEE Signal Process Lett 20:213–216

    Article  Google Scholar 

  22. Saigal P, Khemchandani R (2015) Nonparallel hyperplane classifiers for multi-category classification. In: 2015 IEEE workshop on computational intelligence theories applications and future directions (WCI), pp 1–6

  23. Kumar D, Thakur M (2016) Weighted multicategory nonparallel planes SVM classifiers. Neurocomputing 211:106–116

    Article  Google Scholar 

  24. Chen W, Shao Y, Xu D, Fu Y (2013) Manifold proximal support vector machine for semi-supervised classification. Appl Intell 40:623–638

    Article  Google Scholar 

  25. Viola M, Sangiovanni M, Toraldo G, Guarracino M (2019) Semi-supervised generalized eigenvalues classification. Ann Oper Res 276:249–266

    Article  MathSciNet  Google Scholar 

  26. Sun S, Xie X, Dong C (2019) Multiview learning with generalized eigenvalue proximal support vector machines. IEEE Trans Cybern 49:688–697

    Article  Google Scholar 

  27. Li C, Ren P, Shao Y, Ye Y, Guo Y (2020) Generalized elastic net Lp-norm nonparallel support vector machine. Eng Appl Artif Intell 88

  28. Jayadeva KR, Chandra S (2007) Twin support vector machines for pattern classification. IEEE Trans Pattern Anal Mach Intell 29:905–910

    Article  Google Scholar 

  29. Wang Z, Shao Y, Bai L, Li C, Liu L, Deng N (2018) Insensitive stochastic gradient twin support vector machines for large scale problems. Inf Sci 462:114–131

    Article  MathSciNet  Google Scholar 

  30. Tanveer M, Tiwari A, Choudhary R, Jalan S (2019) Sparse pinball twin support vector machines. Appl Soft Comput 78:164– 175

    Article  Google Scholar 

  31. Mir A, Nasiri J (2018) KNN-Based least squares twin support vector machine for pattern classification. Appl Intell 48:4551–4564

    Article  Google Scholar 

  32. Ding XJ, Lan Y, Zhang ZF et al (2017) Optimization extreme learning machine with ν regularization. Neurocomputing 261:11–C19

    Article  Google Scholar 

  33. Hu R, Ratner ER, Stewart D, Björk K, Lendasse A (2020) A modified Lanczos Algorithm for fast regularization of extreme learning machines. Neurocomputing 414:172–181

    Article  Google Scholar 

  34. Wan Y, Song S, Huang G, Li S (2017) Twin extreme learning machines for pattern classification. Neurocomputing 260:235– 244

    Article  Google Scholar 

  35. Ma J (2020) Supervised and semi-supervised twin parametric-margin regularized extreme learning machine. Pattern Anal Appl (6): 1–24

  36. Rastogi R, Bharti A (2019) Least Squares Twin Extreme Learning Machine for Pattern Classification Proceedings of ICIIf 2018. Innovations in Infrastructure

  37. Parlett B (1998) The symmetric eigenvalue Problem. Philadelphia, PA, USA:SIAM

  38. Yang J, Cao J, Wang T, Xue A, Chen B (2020) Regularized correntropy criterion based semi-supervised ELM. Neural Netw Offic J Int Neural Netw Soc 122:117–129

    Article  Google Scholar 

  39. Liu HY, Hu J, Li YF, Wen ZW (2020) Optimization modeling, algorithms and theory

  40. Li L (1995) Fast parallel power method and inverse power method. Computat Math 17(003):253–259

    MathSciNet  MATH  Google Scholar 

  41. Yuan C, Yang L, Sun P (2021) Correntropy-based metric for robust twin support vector machine. Inf Sci 545:82–101

    Article  MathSciNet  Google Scholar 

  42. Karal O (2017) Maximum likelihood optimal and robust Support Vector Regression with lncosh loss function. Neural Netw Offic J Int Neural Netw Soc 94:1–12

    Article  Google Scholar 

  43. Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874

    Article  Google Scholar 

  44. Blake C, Merz C (1998) UCI Repository for Machine Learning Databases. [On-line]. Available http://www.ics.uci.edu/mlearn/MLRepository.html

  45. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  46. Schmidt F (1996) Statistical significance testing and cumulative knowledge in psychology implications for training of researchers. Psychol Methods 1:115–129

    Article  Google Scholar 

  47. Benavoli A, Corani G, Mangili F (2016) Should we really use Post-Hoc tests based on Mean-Ranks. J Mach Learn Res 17:5:1-5:10

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

This work was supported in part by National Natural Science Foundation of China (No11471010, No 11271367). Moreover, the authors thank the referees and editor for their constructive comments to improve the paper.

Author information

Authors and Affiliations

  1. College of Science, China Agricultural University, Beijing, 100083, China

    Ping Sun & Liming Yang

Authors
  1. Ping Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liming Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, P., Yang, L. Generalized eigenvalue extreme learning machine for classification. Appl Intell 52, 6662–6691 (2022). https://doi.org/10.1007/s10489-021-02654-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02654-2

Keywords

Search

Navigation