Abstract
Recently, a novel learning algorithm for single-hidden-layer feedforward neural networks (SLFNs) named extreme learning machine (ELM) was proposed by Huang et al. The essence of ELM is that the learning parameters of hidden nodes, including input weights and biases, are randomly assigned and need not be tuned while the output weights can be analytically determined by the simple generalized inverse operation. The only parameter needed to be defined is the number of hidden nodes. Compared with other traditional learning algorithms for SLFNs, ELM provides extremely faster learning speed, better generalization performance and with least human intervention. This paper firstly introduces a brief review of ELM, describing the principle and algorithm of ELM. Then, we put emphasis on the improved methods or the typical variants of ELM, especially on incremental ELM, pruning ELM, error-minimized ELM, two-stage ELM, online sequential ELM, evolutionary ELM, voting-based ELM, ordinal ELM, fully complex ELM, and symmetric ELM. Next, the paper summarized the applications of ELM on classification, regression, function approximation, pattern recognition, forecasting and diagnosis, and so on. In the last, the paper discussed several open issues of ELM, which may be worthy of exploring in the future.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Xu XZ, Ding SF, Shi ZZ, Zhu H (2012) Optimizing radial basis function neural network based on rough set and AP clustering algorithm. J Zhejiang Univ Sci A 13(2):131–138
Chen Y, Zheng WX (2012) Stochastic state estimation for neural networks with distributed delays and Markovian jump. Neural Netw 25:14–20
Ding SF, Su CY, Yu JZ (2011) An optimizing BP neural network algorithm based on genetic algorithm. Artif Intell Rev 36(2):153–162
Francisco FN, César HM, Gutiérrez PA, Carbonero-Ruz M (2011) Evolutionary q-Gaussian radial basis function neural networks for multiclassification. Neural Netw 24(7):779–784
Ding SF, Jia WK, Su CY, Zhang LW (2011) Research of neural network algorithm based on factor analysis and cluster analysis. Neural Comput Appl 20(2):297–302
Razavi S, Tolson BA (2011) A new formulation for feedforward neural networks. IEEE Trans Neural Netw 22(10):1588–1598
Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN2004), vol 2, no 25–29, pp 985–990
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892
Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3060–3068
Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062
Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72:359–366
Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163
Lim JS, Lee S, Pang HS (2013) Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimations. Neural Comput Appl 22(3–4):569–576
Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529
Wang L, Huang YP, Luo XY, Wang Z, Luo SW (2011) Image deblurring with filters learned by extreme learning machine. Neurocomputing 74:2464–2474
Cao JW, Lin ZP, Huang GB, Liu N (2012) Voting based extreme learning machine. Inf Sci 185(1, 15):66–77
Zhu QY, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38:1759–1763
Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357
Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73(16–18):3028–3038
Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate on-line sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74(1–3):447–456
Li MB, Huang GB, Saratchandran P, Sundararajan N (2005) Fully complex extreme learning machine. Neurocomputing 68:306–314
Liu XY, Li P, Gao CH (2013) Symmetric extreme learning machine. Neural Comput Appl 22(3–4):551–558
Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468
Huang GB, Li MB, Chen L, Siew CK (2008) Incremental extreme learning machine with fully complex hidden nodes. Neurocomputing 71:576–583
Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395
Zhao JW, Wang ZH, Park DS (2012) Online sequential extreme learning machine with forgetting mechanism. Neurocomputing 87(15):79–89
Castano A, Fernandez-Navarro F, Hervas-Martinez C (2013) PCA-ELM: a robust and pruned extreme learning machine approach based on principal component analysis. Neural Process Lett 37(3):377–392
Zhang WB, Ji HB (2013) Fuzzy extreme learning machine for classification. Electron Lett 49(7):448–449
Horata P, Chiewchanwattana S, Sunat K (2013) Robust extreme learning machine. Neurocomputing 102(SI):31–44
He Q, Shang TF, Zhuang FZ (2013) Parallel extreme learning machine for regression based on MapReduce. Neurocomputing 102(SI):52–58
Yu Qi, Miche Yoan, Eirola Emil (2013) Regularized extreme learning machine for regression with missing data. Neurocomputing 102(SI):45–51
Zong WW, Huang GB, Chen YQ (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242
Wang BT, Wang GR, Li JJ, Wang B (2012) Update strategy based on region classification using ELM for mobile object index. Soft Comput 16(9):1607–1615
Zheng WB, Qian YT, Lu HJ (2013) Text categorization based on regularization extreme learning machine. Neural Comput Appl 22(3–4):447–456
Karpagachelvi S, Arthanari M, Sivakumar M (2012) Classification of electrocardiogram signals with support vector machines and extreme learning machine. Neural Comput Appl 21(6):1331–1339
Kim J, Shin HS, Shin K, Lee M (2009) Robust algorithm for arrhythmia classification in ECG using extreme learning machine. Biomed Eng. doi:10.1186/1475-925X-8-31
Lee Y, Lee H, Kim J, Shin HC, Lee M (2009) Classification of BMI control commands from rat’s neural signals using extreme learning machine. Biomed Eng. doi:10.1186/1475-925X-8-29
Li GQ, Niu PF (2013) An enhanced extreme learning machine based on ridge regression for regression. Neural Comput Appl 22(3–4):803–810
Balasundaram S (2013) On extreme learning machine for e-insensitive regression in the primal by Newton method. Neural Comput Appl. doi:10.1007/s00521-011-0798-9
Feng GR, Qian ZX, Zhang XP (2012) Evolutionary selection extreme learning machine optimization for regression. Soft Comput 16(9):1485–1491
Zong WW, Huang GB (2011) Face recognition based on extreme learning machine. W. Zong, G.-B. Huang Neurocomput 74:2541–2551
Mohammed AA, Minhas R, Jonathan WuQM, Sid-Ahmed MA (2011) Human face recognition based on multidimensional PCA and extreme learning machine. Pattern Recognit 44:2588–2597
Minhas R, Baradarani A, Seifzadeh S, Jonathan WuQM (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73:1906–1917
Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. J Mach Learn Cyber 3:149–161
Lan Y, Hu ZJ, Soh YC, Huang GB (2013) An extreme learning machine approach for speaker recognition. Neural Comput Appl 22(3–4):417–425
Nian R, He B, Lendasse A (2013) 3D object recognition based on a geometrical topology model and extreme learning machine. Neural Comput Appl 22(3–4):427–433
Zhou ZH, Zhao JW, Cao FL (2013) Surface reconstruction based on extreme learning machine. Neural Comput Appl 23(2):283–292
Yang JC, Jiao YB, Xiong NX (2013) Fast face gender recognition by using local ternary pattern and extreme learning machine. KSII Trans Intern Inf Syst 7(7):1705–1720
Yang JC, Xie SJ, Yoon S (2013) Fingerprint matching based on extreme learning machine. Neural Comput Appl 22(3–4):435–445
Chen FL, Ou TY (2011) Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry. Expert Syst Appl 38:1336–1345
Sun ZL et al (2008) Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 46:411–419
Hu XF, Zhao Z, Wang S, Wang FL, He DK, Wu SK (2008) Multi-stage extreme learning machine for fault diagnosis on hydraulic tube tester. Neural Comput Appl 17:399–403
Daliri MR (2012) A hybrid automatic system for the diagnosis of lung cancer based on genetic algorithm and fuzzy extreme learning machines. J Med Syst 36:1001–1005
Xu Y, Dai YY, Dong ZY, Zhang R, Meng K (2013) Extreme learning machine-based predictor for real-time frequency stability assessment of electric power systems. IET Gener Transm Distrib 7(4):391–397
Pan C, Park DS, Yang Y, Yoo HM (2012) Leukocyte image segmentation by visual attention and extreme learning machine. Neural Comput Appl 21(6):1217–1227
Pan C, Park DS, Lu HJ, Wu XP (2012) Color image segmentation by fixation-based active learning with ELM. Soft Comput 16(9):1569–1584
Malathi V, Marimuthu NS, Baskar S, Ramar K (2011) Application of extreme learning machine for series compensated transmission line protection. Eng Appl Artif Intell 24:880–887
Zhao LJ, Wang DH, Chai TY (2013) Estimation of effluent quality using PLS-based extreme learning machines. Neural Comput Appl 22(3–4):509–519
Li YJ, Li Y, Zhai JH, Shiu S (2012) RTS game strategy evaluation using extreme learning machine. Soft Comput 16(9):1627–1637
Li LN, Ouyang JH, Chen HL, Liu DY (2012) A computer aided diagnosis system for thyroid disease using extreme learning machine. J Med Syst 36(5):3327–3337
Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cyber 2:107–122
Acknowledgments
This work is supported by the National Natural Science Foundation (No. 61379101), the 973 Program (No. 2013CB329502), the Basic Research Program (Natural Science Foundation) of Jiangsu Province of China (No. BK20130209), the Opening Foundation of the Key Laboratory of Intelligent Information Processing of Chinese Academy of Sciences (No. IIP2010-1), and the Opening Foundation of Beijing Key Lab of Intelligent Telecommunications Software and Multimedia, Beijing University of Posts and Telecommunications.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ding, S., Xu, X. & Nie, R. Extreme learning machine and its applications. Neural Comput & Applic 25, 549–556 (2014). https://doi.org/10.1007/s00521-013-1522-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-013-1522-8