Abstract
To improve the learning performance, a novel online sequential extreme learning machine (ELM) algorithm for single-hidden layer feedforward networks is proposed with regularization mechanism in a unified framework. The proposed algorithm is called timeliness online regularized extreme learning machine (TORELM). Like the timeliness managing extreme learning machine which improves online sequential extreme learning machine by incorporating timeliness management scheme into ELM approach for the incremental training samples, TORELM also analyzes the training data one-by-one or chunk-by-chunk (a block of data) with fixed or varied chunk size under the similar framework. Meanwhile, the newly incremental training data could be prior to the historical data by maximizing the contribution of the newly increasing training data, since in some cases it may be more feasible that the incremental data can contribute reasonable weights to represent the current system situation in accordance with the practical analysis. Furthermore, in consideration of the disproportion between empirical risk and structural risk in some traditional learning methods, we add regularization technique to the timeliness scheme of TORELM through the use of a weight factor to balance them to achieve better generalization performance. Hence, TORELM has its unique feature of higher generalization capability in most cases with a small testing error while implementing online sequential learning. In addition, this algorithm is still competitive in training time compared with other schemes. Finally, the simulation results show that TORELM can achieve higher learning accuracy and better stability than other ELM-based machine learning methods.














Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122
Luo X, Chang XH, Liu H (2014) A Taylor based localization algorithm for wireless sensor network using extreme learning machine. IEICE Trans Inf Syst E97D(10):2652–2659
Luo X, Chang XH (2015) A novel data fusion scheme using grey model and extreme learning machine in wireless sensor networks. Int J Control Autom Syst 13(3):539–546
You ZH, Lei YK, Zhu L, Xia JF, Wang B (2013) Prediction of protein-protein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis. BMC Bioinform 14(Suppl. 8):S10
Huang GB, Bai Z, Kasun L, Vong C (2015) Local receptive fields based extreme learning machine. IEEE Comput Intell Magn 10(2):18–29
Luo X, Chang XH, Ban XJ (2016) Regression and classification using extreme learning machine based on L1-norm and L2-norm. Neurocomputing. 174:179–186
He YL, Wang XZ, Huang JZX (2016) Fuzzy nonlinear regression analysis using a random weight network. Inf Sci. doi:10.1016/j.ins.2016.01.037
Cambria E, Huang GB, Kasun LLC, Zhou H, Vong CM, Lin J, Yin J, Cai Z, Liu Q, Li K, Leung VCM, Feng L, Ong YS, Lim MH, Akusok A, Lendasse A, Corona F, Nian R, Miche Y, Gastaldo P, Zunino R, Decherchi S, Yang X, Mao K, Oh BS, Jeon J, Toh KA, Teoh ABJ, Kim J, Yu H, Chen Y, Liu J (2013) Extreme learning machines. IEEE Intell Syst 28(6):30–59
Wang XZ, Chen AX, Feng HM (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525
Lu SX, Wang XZ, Zhang GQ, Zhou X (2015) Effective algorithms of the Moore–Penrose inverse matrices for extreme learning machine. Intell Data Anal 19(4):743–760
Chacko BP, Krishnan VRV, Raju G (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern 3(2):149–161
Fu A, Dong C, Wang L (2015) An experimental study on stability and generalization of extreme learning machines. Int J Mach Learn Cybern 6(1):129–135
Viskan K, Mahesan N (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5(6):954–975
Nobrega JP, Oliveira ALI (2015) Kalman filter-based method for online sequential extreme learning machine for regression problems. Eng Appl Artif Intell 44:101–110
Kadirkamanathan V, Niranjan M (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5(6):954–975
Liang N, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Scardapane S, Comminiello D, Scarpiniti M (2015) Online sequential extreme learning machine with kernels. IEEE Trans Neural Netw Learn Syst 26(9):2214–2220
Matias T, Souza F, Araújo R, Gonçalves N, Barreto JP (2015) On-line sequential extreme learning machine based on recursive partial least squares. J Process Control 27:15–21
Yuan L, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13):3391–3395
Liu JF, Gu Y, Chen YQ, Cao YS (2013) Incremental localization in WLAN environment with timeliness management. Chin J Comput 36(7):1448–1455
Vapnik VN (1995) The nature of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999
Deng WY, Zheng QH, Chen L (2009) Regularized extreme learning machine. In: Proceedings of the IEEE symposium on computational intelligence and data mining, pp 389–395
Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3):155–163
Huang GB, Zhou H, Ding X (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529
Gu Y, Liu JF, Chen YQ, Jiang XL, Yu HC (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127
Glenn F, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of the 7th ACM SIGKDD international conference on knowledge discovery and data mining, pp 77–86
David HA (1998) Early sample measures of variability. Stat Sci 13(4):368–377
Rousseeuw PJ, Leroy A (1987) Robut regression and outlier detection. Wiley, New York
Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162
Wang XZ, Shao QY, Qing M, Zhai JH (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3–9
Stock Data (2016) YAHOO Finance API. http://ichart.yahoo.com/table.csv?s=600005.SS&a=00&b=01&c=2003&d=04&e=40&f=2012&g=d. Accessed 5 May 2016
Graham L, Oppacher F (2008) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/Hill-Valley. Accessed 5 May 2016
Dias DB, Peres SM, Bíscaro HH (2009) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/Libras+Movement. Accessed 5 May 2016
Roesler O (2013) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/EEG+Eye+State. Accessed 5 May 2016
Acknowledgments
This research is funded by the National Natural Science Foundation of China under Grants 61174103, 61272357 and 61300074, the National Key Technologies R&D Program of China under Grant 2015BAK38B01, the Aerospace Science Foundation of China under Grant 2014ZA74001, and the Fundamental Research Funds for the Central Universities under Grant 06500025.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Luo, X., Yang, X., Jiang, C. et al. Timeliness online regularized extreme learning machine. Int. J. Mach. Learn. & Cyber. 9, 465–476 (2018). https://doi.org/10.1007/s13042-016-0544-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-016-0544-9