Skip to main content
Log in

An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In order to curb the model expansion of the kernel learning methods and adapt the nonlinear dynamics in the process of the nonstationary time series online prediction, a new online sequential learning algorithm with sparse update and adaptive regularization scheme is proposed based on kernel-based incremental extreme learning machine (KB-IELM). For online sparsification, a new method is presented to select sparse dictionary based on the instantaneous information measure. This method utilizes a pruning strategy, which can prune the least “significant” centers, and preserves the important ones by online minimizing the redundancy of dictionary. For adaptive regularization scheme, a new objective function is constructed based on basic ELM model. New model has different structural risks in different nonlinear regions. At each training step, new added sample could be assigned optimal regularization factor by optimization procedure. Performance comparisons of the proposed method with other existing online sequential learning methods are presented using artificial and real-word nonstationary time series data. The results indicate that the proposed method can achieve higher prediction accuracy, better generalization performance and stability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Mourad E, Amiya N (2012) Comparison-based system-level fault diagnosis: a neural network approach. IEEE Trans Parallel Distrib Syst 23(6):1047–1059

    Article  Google Scholar 

  2. Tian Z, Qian C, Gu B, Yang L, Liu F (2015) Electric vehicle air conditioning system performance prediction based on artificial neural network. Appl Therm Eng 89:101–104

    Article  Google Scholar 

  3. Cambria E, Huang GB (2013) Extreme learning machine [trends and controversies]. IEEE Intell Syst 28(6):30–59

    Article  Google Scholar 

  4. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and application. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  5. Yin G, Zhang YT, Li ZN, Ren GQ, Fan HB (2014) Online fault diagnosis method based on incremental support vector data description and extreme learning machine with incremental output structure. Neurocomputing 128:224–231

    Article  Google Scholar 

  6. Rong HJ, Huang GB, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Transactions on systems, man and cybernetics—part B: cybernetics 39(4):1067–1072

    Article  Google Scholar 

  7. Mirza B, Lin ZP, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329

    Article  Google Scholar 

  8. Xia SX, Meng FR, Liu B, Zhou Y (2015) A kernel clustering-based possibilistic fuzzy extreme learning machine for class imbalance learning. Cogn Comput 7:74–85

    Article  Google Scholar 

  9. Li XD, Mao WJ, Wei Jiang (2016) Multiple-kernel-learning- based extreme learning machine for classification design. Neural Comput Appl 27:175–184

    Article  Google Scholar 

  10. Zhao SL, Chen BD, Zhu PP, Principe JC (2013) Fixed budget quantized kernel least-mean-square algorithm. Sig Process 93:2759–2770

    Article  Google Scholar 

  11. Huang GB, Zhou H, Ding X, Zhang R (2011) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern—Part B: Cybern 42(2):513–529

    Article  Google Scholar 

  12. Wang XY, Han M (2014) Online sequential extreme learning machine with kernels for nonstationary time series prediction. Neurocomputing 145:90–97

    Article  Google Scholar 

  13. Deng WY, Ong YS, Tan PS, Zheng QH (2016) Online sequential reduced kernel extreme learning machine. Neurocomputing 174:72–84

    Article  Google Scholar 

  14. Wong SY, Yap KS, Yap HJ, Tan SC (2015) A truly online learning algorithm using hybird fuzzy ARTMAP and online extreme learning machine for pattern classification. Neural Process Lett 42:585–602

    Article  Google Scholar 

  15. Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  16. Huynh HT, Won Y (2011) Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks. Pattern Recogn Lett 32:1930–1935

    Article  Google Scholar 

  17. Guo L, Hao JH, Liu M (2014) An incremental extreme learning machine for online sequential learning problems. Neurocomputing 128:50–58

    Article  Google Scholar 

  18. Fan HJ, Song Q, Yang XL, Xu Z (2015) Kernel online learning algorithm with state feedbacks. Knowl Based Syst 89:173–180

    Article  Google Scholar 

  19. Fan HJ, Song Q (2013) A sparse kernel algorithm for online time series data prediction. Expert Syst Appl 40:2174–2181

    Article  Google Scholar 

  20. Zhou XR, Liu ZJ, Zhu CX (2014) Online regularized and kernelized extreme learning machines with forgetting mechanism. Math Probl Eng. doi:10.1155/2014/938548

    Google Scholar 

  21. Zhou XR, Wang CS (2016) Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism. Neurocomputing 174:1147–1155

    Article  Google Scholar 

  22. Gu Y, Liu JF, Chen YQ, Jiang XL, Yu HC (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127

    Article  Google Scholar 

  23. Lim J, Lee S, Pang HS (2013) Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimation. Neural Comput Appl 22:569–576

    Article  Google Scholar 

  24. He X, Wang HL, Lu JH, Jiang W (2015) Online fault diagnosis of analog circuit based on limited-samples sequence extreme learning machine. Control Decis 30(3):455–460

    MATH  Google Scholar 

  25. Scardapance S, Comminiello D, Scarpiniti M, Uncini A (2015) Online sequential extreme learning machine with kernel. IEEE Trans Neural Netw Learn. Syst 26(9):2214–2220

    Article  MathSciNet  Google Scholar 

  26. Zhang YT, Ma C, Li ZN, Fan HB (2014) Online modeling of kernel extreme learning machine based on fast leave-one-out cross-validation. J Shanghai Jiaotong Univ 48(5):641–646

    Google Scholar 

  27. Shao ZF, Meng JE (2016) An online sequential learning algorithm for regularized extreme learning machine. Neurocomputing 173:778–788

    Article  Google Scholar 

  28. Lu XJ, Zhou C, Huang MH, Lv WB (2016) Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system. Neurocomputing 174:617–626

    Article  Google Scholar 

  29. Lin M, Zhang LJ, Jin R, Weng SF, Zhang CS (2016) Online kernel learning with nearly constant support vectors. Neurocomputing 179:26–36

    Article  Google Scholar 

  30. Honeine P (2015) Analyzing sparse dictionaries for online learning with kernels. IEEE Trans Signal Process 63(23):6343–6353

    Article  MathSciNet  MATH  Google Scholar 

  31. Platt J (1991) A resource-allocating network for function interpolation. Neural Comput 3(2):213–225

    Article  MathSciNet  Google Scholar 

  32. Engel Y, Mannor S, Meir R (2004) The kernel recursive least-squares algorithm. IEEE Trans Signal Process 52(8):2275–2285

    Article  MathSciNet  MATH  Google Scholar 

  33. Richard C, Bermudez JCM, Honeine P (2009) Online prediction of time series data with kernels. IEEE Trans Signal Process 57(3):1058–1067

    Article  MathSciNet  MATH  Google Scholar 

  34. Fan HJ, Song Q, Xu Z (2014) Online learning with kernel regularized least mean square algorithms. Expert Syst Appl 41:4349–4359

    Article  Google Scholar 

  35. Liu WF, Park I, Principe JC (2009) An information theoretic approach of designing sparse kernel adaptive filters. IEEE Trans Neural Netw 20(12):1950–1961

    Article  Google Scholar 

  36. Fan HJ, Song Q, Shrestha SB (2016) Kernel online learning with adaptive kernel width. Neurocomputing 175:233–242

    Article  Google Scholar 

  37. Zhao YP, Wang KK (2014) Fast cross validation for regularized extreme learning machine. J Syst Eng Electron 25(5):895–900

    Article  Google Scholar 

Download references

Acknowledgements

The research was supported by National Science Foundation of China under Grant Nos. 61571454.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Aiqiang Xu.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interests regarding the publication of this paper.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, W., Xu, A., Ping, D. et al. An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction. Neural Comput & Applic 31, 637–652 (2019). https://doi.org/10.1007/s00521-017-3096-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-017-3096-3

Keywords

Navigation