Skip to main content
SpringerLink
Log in

Multilayer discriminative extreme learning machine for classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The representation learning is the key to deep learning. As a special deep learning algorithm, the generalization performance of the multilayer extreme learning machine (ML-ELM) is influenced by the feature extraction capability of the extreme learning machine autoencoder (ELM-AE). But the ELM-AE does not consider class labels, and it is difficult for the ELM-AE to build the discriminative feature space in the classification tasks. Thus, to improve the class separability of abstract feature, the discriminant regularization is introduced into the ELM-AE and a new regularized ELM-AE called discriminative extreme learning machine autoencoder (DELM-AE) is proposed. And by stacking the ELM-AE for feature extraction and adopting the ELM for classifying, the multilayer discriminative extreme learning machine (ML-DELM) is proposed for classification tasks. The DELM-AE adds the discriminant regularization term to the loss function to reduce the distance between samples and their class centers in feature space. Compared with the ELM-AE and ML-ELM, the DELM-AE and ML-DELM indirectly utilizes the category information contained in the discriminant regularization term to guide the feature extraction. Empirical evaluations and experiments on various benchmark datasets show that feature representation learned by the DELM-AE is discriminative and sparse, and the generalization performance and structural sparsity of ML-DELM are comparable with the other state-of-the-art ML-ELM algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability statement

The data used to support the findings of this study are available from the corresponding author upon request.

References

  1. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  2. Zhu S, Wang H, Lv H, Zhang HS (2021) Augmented online sequential quaternion extreme learning machine. Neural Process Lett 53(2):1161–1186

    Article  Google Scholar 

  3. Zabala-Blanco D, Mora M, Barrientos RJ, Hernandez-Garcia R, Naranjo-Torres J (2020) Fingerprint classification through standard and weighted extreme learning machines. Appl Sci Basel 10(12):4125

    Article  Google Scholar 

  4. Akyol K (2020) Comparing of deep neural networks and extreme learning machines based on growing and pruning approach. Expert Syst Appl 140:112875

    Article  Google Scholar 

  5. Yan DQ, Chu YH, Zhang HY, Liu DS (2018) Information discriminative extreme learning machine. Soft Comput 22(2):677–689

    Article  Google Scholar 

  6. Li R, Wang XD, Lei L, Song YF (2019) L21-norm based loss function and regularization extreme learning machine. IEEE Access 7:6575–6586

    Article  Google Scholar 

  7. Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  8. Huang GB, Zhou HM, Ding XJ, Zhang R (2011) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  9. Chen Y, Tong CD, Ge YH, Lan T (2021) Fault detection based on auto-regressive extreme learning machine for nonlinear dynamic processes. Appl Soft Comput 106:107319

    Article  Google Scholar 

  10. Sikder N, Mohammad Arif AS, Islam MM, Nahid AA (2021) Induction motor bearing fault classification using extreme learning machine based on power features. Arab J Sci Eng 46(9):8475–8491

    Article  Google Scholar 

  11. Bhatti YK, Jamil A, Nida N, Yousaf MH, Viriri S, Velastin SA (2021) Facial expression recognition of instructor using deep features and extreme learning machine. Comput Intell Neurosci 2021:5570870

    Article  Google Scholar 

  12. Goel T, Murugan R (2020) Classifier for face recognition based on deep convolutional-optimized kernel extreme learning machine. Comput Electr Eng 85:106640

    Article  Google Scholar 

  13. Lahoura V, Singh H, Aggarwal A, Mohammed MA, Damasevicius R (2021) Cloud computing-based framework for breast cancer diagnosis using extreme learning machine. Diagnostics 11(2):241

    Article  Google Scholar 

  14. Zhao LL, Qian JH, Tian FC, Liu R, Liu B (2021) A weighted discriminative extreme learning machine design for lung cancer detection by an electronic nose system. IEEE Trans Instrum Meas 70:1–9

    Google Scholar 

  15. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    Article  MathSciNet  MATH  Google Scholar 

  16. Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292

    Article  Google Scholar 

  17. Kasun L, Zhou H, Huang GB, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34

    Google Scholar 

  18. Zhang J, Xiao WD, Li YJ, Zhang S, Zhang ZQ (2020) Multilayer probability extreme learning machine for device-free localization. Neurocomputing 396:383–393

    Article  Google Scholar 

  19. Khan MA, Abbas S, Khan KM, Al Ghamdi MA, Rehman A (2020) Intelligent forecasting model of COVID-19 novel coronavirus outbreak empowered with deep extreme learning machine. CMC Comput Mat Contin 64(3):1329–1342

    Google Scholar 

  20. Li Q, Zhang XY, Ma TJ, Jiao CL, Wang H, Hu W (2021) A multi-step ahead photovoltaic power prediction model based on similar day, enhanced colliding bodies optimization, variational mode decomposition, and deep extreme learning machine. Energy 224:120094

    Article  Google Scholar 

  21. Cao F, Yang Z, Ren J, Chen WZ, Han GJ, Shen YS (2019) Local block multilayer sparse extreme learning machine for effective feature extraction and classification of hyperspectral images. IEEE Trans Geosci Remote Sens 57(8):5580–5594

    Article  Google Scholar 

  22. Li H, Zhao H, Li H (2018) Neural-response-based extreme learning machine for image classification. IEEE Trans Neural Netw Learn Syst 30(2):539–552

    Article  MathSciNet  Google Scholar 

  23. Zhang N, Ding S, Shi Z (2016) Denoising Laplacian multi-layer extreme learning machine. Neurocomputing 171:1066–1074

    Article  Google Scholar 

  24. Wong CM, Vong CM, Wong PK, Cao JW (2016) Kernel-based multilayer extreme learning machines for representation learning. IEEE Trans Neural Netw Learn Syst 29(3):757–762

    Article  MathSciNet  Google Scholar 

  25. Sun K, Zhang JS, Zhang CX, Hu JY (2017) Generalized extreme learning machine autoencoder and a new deep neural network. Neurocomputing 230:374–381

    Article  Google Scholar 

  26. Li R, Wang XD, Song Y, Lei L (2021) Hierarchical extreme learning machine with L21-norm loss and regularization. Int J Mach Learn Cybern 12(5):1297–1310

    Article  Google Scholar 

  27. Liu Y, Wang J (2022) Transfer learning based multi-layer extreme learning machine for probabilistic wind power forecasting. Appl Energy 312:118729

    Article  Google Scholar 

  28. Luo X, Li Y, Wang WP, Ban XJ, Wang JH, Zhao WB (2020) A robust multilayer extreme learning machine using kernel risk-sensitive loss criterion. Int J Mach Learn Cybern 11(1):197–216

    Article  Google Scholar 

  29. Wu D, Li T, Wan Q (2021) A hybrid deep kernel incremental extreme learning machine based on improved coyote and beetle swarm optimization methods. Complex Intell Syst 7(6):3015–3032

    Article  Google Scholar 

  30. Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Department of Information Computer Science, University of California, Irvine, CA. [Online]. http://archive.ics.uci.edu/m. Accessed 21 Jan 2022

  31. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  32. Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747

  33. Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554

    Article  Google Scholar 

  34. Du H, Song DQ, Chen Z, Shu HP, Guo ZZ (2020) Prediction model oriented for landslide displacement with step-like curve by applying ensemble empirical mode decomposition and the PSO-ELM method. J Clean Prod 270:122248

    Article  Google Scholar 

  35. Krishnan GS, Kamath S (2019) A novel GA-ELM model for patient-specific mortality prediction over large-scale lab event data. Appl Soft Comput 80:525–533

    Article  Google Scholar 

Download references

Acknowledgements

The research is supported by the National Natural Science Foundation of China (61876189, 61273275, 61806219, and 61703426) and the Natural Science Basic Research Plan in Shaanxi Province (no. 2021JM—226).

Author information

Authors and Affiliations

  1. College of Air and Missile Defense, Air Force Engineering University, Xi’an, 710051, China

    Jie Lai, Xiaodan Wang, Qian Xiang & Yafei Song

  2. College of Air Traffic Control and Navigation, Air Force Engineering University, Xi’an, 710051, China

    Wen Quan

Authors
  1. Jie Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qian Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yafei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wen Quan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaodan Wang.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lai, J., Wang, X., Xiang, Q. et al. Multilayer discriminative extreme learning machine for classification. Int. J. Mach. Learn. & Cyber. 14, 2111–2125 (2023). https://doi.org/10.1007/s13042-022-01749-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01749-7

Keywords

Search

Navigation