Abstract
In order to curb the model expansion of the kernel learning methods and adapt the nonlinear dynamics in the process of the nonstationary time series online prediction, a new online sequential learning algorithm with sparse update and adaptive regularization scheme is proposed based on kernel-based incremental extreme learning machine (KB-IELM). For online sparsification, a new method is presented to select sparse dictionary based on the instantaneous information measure. This method utilizes a pruning strategy, which can prune the least “significant” centers, and preserves the important ones by online minimizing the redundancy of dictionary. For adaptive regularization scheme, a new objective function is constructed based on basic ELM model. New model has different structural risks in different nonlinear regions. At each training step, new added sample could be assigned optimal regularization factor by optimization procedure. Performance comparisons of the proposed method with other existing online sequential learning methods are presented using artificial and real-word nonstationary time series data. The results indicate that the proposed method can achieve higher prediction accuracy, better generalization performance and stability.
Similar content being viewed by others
References
Mourad E, Amiya N (2012) Comparison-based system-level fault diagnosis: a neural network approach. IEEE Trans Parallel Distrib Syst 23(6):1047–1059
Tian Z, Qian C, Gu B, Yang L, Liu F (2015) Electric vehicle air conditioning system performance prediction based on artificial neural network. Appl Therm Eng 89:101–104
Cambria E, Huang GB (2013) Extreme learning machine [trends and controversies]. IEEE Intell Syst 28(6):30–59
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and application. Neurocomputing 70(1–3):489–501
Yin G, Zhang YT, Li ZN, Ren GQ, Fan HB (2014) Online fault diagnosis method based on incremental support vector data description and extreme learning machine with incremental output structure. Neurocomputing 128:224–231
Rong HJ, Huang GB, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Transactions on systems, man and cybernetics—part B: cybernetics 39(4):1067–1072
Mirza B, Lin ZP, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329
Xia SX, Meng FR, Liu B, Zhou Y (2015) A kernel clustering-based possibilistic fuzzy extreme learning machine for class imbalance learning. Cogn Comput 7:74–85
Li XD, Mao WJ, Wei Jiang (2016) Multiple-kernel-learning- based extreme learning machine for classification design. Neural Comput Appl 27:175–184
Zhao SL, Chen BD, Zhu PP, Principe JC (2013) Fixed budget quantized kernel least-mean-square algorithm. Sig Process 93:2759–2770
Huang GB, Zhou H, Ding X, Zhang R (2011) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern—Part B: Cybern 42(2):513–529
Wang XY, Han M (2014) Online sequential extreme learning machine with kernels for nonstationary time series prediction. Neurocomputing 145:90–97
Deng WY, Ong YS, Tan PS, Zheng QH (2016) Online sequential reduced kernel extreme learning machine. Neurocomputing 174:72–84
Wong SY, Yap KS, Yap HJ, Tan SC (2015) A truly online learning algorithm using hybird fuzzy ARTMAP and online extreme learning machine for pattern classification. Neural Process Lett 42:585–602
Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Huynh HT, Won Y (2011) Regularized online sequential learning algorithm for single-hidden layer feedforward neural networks. Pattern Recogn Lett 32:1930–1935
Guo L, Hao JH, Liu M (2014) An incremental extreme learning machine for online sequential learning problems. Neurocomputing 128:50–58
Fan HJ, Song Q, Yang XL, Xu Z (2015) Kernel online learning algorithm with state feedbacks. Knowl Based Syst 89:173–180
Fan HJ, Song Q (2013) A sparse kernel algorithm for online time series data prediction. Expert Syst Appl 40:2174–2181
Zhou XR, Liu ZJ, Zhu CX (2014) Online regularized and kernelized extreme learning machines with forgetting mechanism. Math Probl Eng. doi:10.1155/2014/938548
Zhou XR, Wang CS (2016) Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism. Neurocomputing 174:1147–1155
Gu Y, Liu JF, Chen YQ, Jiang XL, Yu HC (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127
Lim J, Lee S, Pang HS (2013) Low complexity adaptive forgetting factor for online sequential extreme learning machine (OS-ELM) for application to nonstationary system estimation. Neural Comput Appl 22:569–576
He X, Wang HL, Lu JH, Jiang W (2015) Online fault diagnosis of analog circuit based on limited-samples sequence extreme learning machine. Control Decis 30(3):455–460
Scardapance S, Comminiello D, Scarpiniti M, Uncini A (2015) Online sequential extreme learning machine with kernel. IEEE Trans Neural Netw Learn. Syst 26(9):2214–2220
Zhang YT, Ma C, Li ZN, Fan HB (2014) Online modeling of kernel extreme learning machine based on fast leave-one-out cross-validation. J Shanghai Jiaotong Univ 48(5):641–646
Shao ZF, Meng JE (2016) An online sequential learning algorithm for regularized extreme learning machine. Neurocomputing 173:778–788
Lu XJ, Zhou C, Huang MH, Lv WB (2016) Regularized online sequential extreme learning machine with adaptive regulation factor for time-varying nonlinear system. Neurocomputing 174:617–626
Lin M, Zhang LJ, Jin R, Weng SF, Zhang CS (2016) Online kernel learning with nearly constant support vectors. Neurocomputing 179:26–36
Honeine P (2015) Analyzing sparse dictionaries for online learning with kernels. IEEE Trans Signal Process 63(23):6343–6353
Platt J (1991) A resource-allocating network for function interpolation. Neural Comput 3(2):213–225
Engel Y, Mannor S, Meir R (2004) The kernel recursive least-squares algorithm. IEEE Trans Signal Process 52(8):2275–2285
Richard C, Bermudez JCM, Honeine P (2009) Online prediction of time series data with kernels. IEEE Trans Signal Process 57(3):1058–1067
Fan HJ, Song Q, Xu Z (2014) Online learning with kernel regularized least mean square algorithms. Expert Syst Appl 41:4349–4359
Liu WF, Park I, Principe JC (2009) An information theoretic approach of designing sparse kernel adaptive filters. IEEE Trans Neural Netw 20(12):1950–1961
Fan HJ, Song Q, Shrestha SB (2016) Kernel online learning with adaptive kernel width. Neurocomputing 175:233–242
Zhao YP, Wang KK (2014) Fast cross validation for regularized extreme learning machine. J Syst Eng Electron 25(5):895–900
Acknowledgements
The research was supported by National Science Foundation of China under Grant Nos. 61571454.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interests regarding the publication of this paper.
Rights and permissions
About this article
Cite this article
Zhang, W., Xu, A., Ping, D. et al. An improved kernel-based incremental extreme learning machine with fixed budget for nonstationary time series prediction. Neural Comput & Applic 31, 637–652 (2019). https://doi.org/10.1007/s00521-017-3096-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-017-3096-3