Skip to main content
SpringerLink
Log in

Timeliness online regularized extreme learning machine

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

To improve the learning performance, a novel online sequential extreme learning machine (ELM) algorithm for single-hidden layer feedforward networks is proposed with regularization mechanism in a unified framework. The proposed algorithm is called timeliness online regularized extreme learning machine (TORELM). Like the timeliness managing extreme learning machine which improves online sequential extreme learning machine by incorporating timeliness management scheme into ELM approach for the incremental training samples, TORELM also analyzes the training data one-by-one or chunk-by-chunk (a block of data) with fixed or varied chunk size under the similar framework. Meanwhile, the newly incremental training data could be prior to the historical data by maximizing the contribution of the newly increasing training data, since in some cases it may be more feasible that the incremental data can contribute reasonable weights to represent the current system situation in accordance with the practical analysis. Furthermore, in consideration of the disproportion between empirical risk and structural risk in some traditional learning methods, we add regularization technique to the timeliness scheme of TORELM through the use of a weight factor to balance them to achieve better generalization performance. Hence, TORELM has its unique feature of higher generalization capability in most cases with a small testing error while implementing online sequential learning. In addition, this algorithm is still competitive in training time compared with other schemes. Finally, the simulation results show that TORELM can achieve higher learning accuracy and better stability than other ELM-based machine learning methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  2. Luo X, Chang XH, Liu H (2014) A Taylor based localization algorithm for wireless sensor network using extreme learning machine. IEICE Trans Inf Syst E97D(10):2652–2659

    Article  Google Scholar 

  3. Luo X, Chang XH (2015) A novel data fusion scheme using grey model and extreme learning machine in wireless sensor networks. Int J Control Autom Syst 13(3):539–546

    Article  Google Scholar 

  4. You ZH, Lei YK, Zhu L, Xia JF, Wang B (2013) Prediction of protein-protein interactions from amino acid sequences with ensemble extreme learning machines and principal component analysis. BMC Bioinform 14(Suppl. 8):S10

    Article  Google Scholar 

  5. Huang GB, Bai Z, Kasun L, Vong C (2015) Local receptive fields based extreme learning machine. IEEE Comput Intell Magn 10(2):18–29

    Article  Google Scholar 

  6. Luo X, Chang XH, Ban XJ (2016) Regression and classification using extreme learning machine based on L1-norm and L2-norm. Neurocomputing. 174:179–186

    Article  Google Scholar 

  7. He YL, Wang XZ, Huang JZX (2016) Fuzzy nonlinear regression analysis using a random weight network. Inf Sci. doi:10.1016/j.ins.2016.01.037

    Google Scholar 

  8. Cambria E, Huang GB, Kasun LLC, Zhou H, Vong CM, Lin J, Yin J, Cai Z, Liu Q, Li K, Leung VCM, Feng L, Ong YS, Lim MH, Akusok A, Lendasse A, Corona F, Nian R, Miche Y, Gastaldo P, Zunino R, Decherchi S, Yang X, Mao K, Oh BS, Jeon J, Toh KA, Teoh ABJ, Kim J, Yu H, Chen Y, Liu J (2013) Extreme learning machines. IEEE Intell Syst 28(6):30–59

    Article  Google Scholar 

  9. Wang XZ, Chen AX, Feng HM (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525

    Article  Google Scholar 

  10. Lu SX, Wang XZ, Zhang GQ, Zhou X (2015) Effective algorithms of the Moore–Penrose inverse matrices for extreme learning machine. Intell Data Anal 19(4):743–760

    Article  Google Scholar 

  11. Chacko BP, Krishnan VRV, Raju G (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern 3(2):149–161

    Article  Google Scholar 

  12. Fu A, Dong C, Wang L (2015) An experimental study on stability and generalization of extreme learning machines. Int J Mach Learn Cybern 6(1):129–135

    Article  Google Scholar 

  13. Viskan K, Mahesan N (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5(6):954–975

    Article  Google Scholar 

  14. Nobrega JP, Oliveira ALI (2015) Kalman filter-based method for online sequential extreme learning machine for regression problems. Eng Appl Artif Intell 44:101–110

    Article  Google Scholar 

  15. Kadirkamanathan V, Niranjan M (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5(6):954–975

    Article  Google Scholar 

  16. Liang N, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  17. Scardapane S, Comminiello D, Scarpiniti M (2015) Online sequential extreme learning machine with kernels. IEEE Trans Neural Netw Learn Syst 26(9):2214–2220

    Article  MathSciNet  Google Scholar 

  18. Matias T, Souza F, Araújo R, Gonçalves N, Barreto JP (2015) On-line sequential extreme learning machine based on recursive partial least squares. J Process Control 27:15–21

    Article  Google Scholar 

  19. Yuan L, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13):3391–3395

    Google Scholar 

  20. Liu JF, Gu Y, Chen YQ, Cao YS (2013) Incremental localization in WLAN environment with timeliness management. Chin J Comput 36(7):1448–1455

    Article  Google Scholar 

  21. Vapnik VN (1995) The nature of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Article  MathSciNet  MATH  Google Scholar 

  22. Deng WY, Zheng QH, Chen L (2009) Regularized extreme learning machine. In: Proceedings of the IEEE symposium on computational intelligence and data mining, pp 389–395

  23. Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3):155–163

    Article  Google Scholar 

  24. Huang GB, Zhou H, Ding X (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529

    Article  Google Scholar 

  25. Gu Y, Liu JF, Chen YQ, Jiang XL, Yu HC (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127

    Article  Google Scholar 

  26. Glenn F, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of the 7th ACM SIGKDD international conference on knowledge discovery and data mining, pp 77–86

  27. David HA (1998) Early sample measures of variability. Stat Sci 13(4):368–377

    Article  MathSciNet  MATH  Google Scholar 

  28. Rousseeuw PJ, Leroy A (1987) Robut regression and outlier detection. Wiley, New York

    Book  MATH  Google Scholar 

  29. Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162

    Article  Google Scholar 

  30. Wang XZ, Shao QY, Qing M, Zhai JH (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3–9

    Article  Google Scholar 

  31. Stock Data (2016) YAHOO Finance API. http://ichart.yahoo.com/table.csv?s=600005.SS&a=00&b=01&c=2003&d=04&e=40&f=2012&g=d. Accessed 5 May 2016

  32. Graham L, Oppacher F (2008) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/Hill-Valley. Accessed 5 May 2016

  33. Dias DB, Peres SM, Bíscaro HH (2009) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/Libras+Movement. Accessed 5 May 2016

  34. Roesler O (2013) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/EEG+Eye+State. Accessed 5 May 2016

Download references

Acknowledgments

This research is funded by the National Natural Science Foundation of China under Grants 61174103, 61272357 and 61300074, the National Key Technologies R&D Program of China under Grant 2015BAK38B01, the Aerospace Science Foundation of China under Grant 2014ZA74001, and the Fundamental Research Funds for the Central Universities under Grant 06500025.

Author information

Authors and Affiliations

  1. School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, 100083, People’s Republic of China

    Xiong Luo, Xiaona Yang, Changwei Jiang & Xiaojuan Ban

  2. Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing, 100083, People’s Republic of China

    Xiong Luo, Xiaona Yang, Changwei Jiang & Xiaojuan Ban

Authors
  1. Xiong Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaona Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Changwei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaojuan Ban
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiong Luo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Luo, X., Yang, X., Jiang, C. et al. Timeliness online regularized extreme learning machine. Int. J. Mach. Learn. & Cyber. 9, 465–476 (2018). https://doi.org/10.1007/s13042-016-0544-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-016-0544-9

Keywords

Search

Navigation