Abstract
An ensemble of neural networks has been proved to be an effective machine learning framework. However, very limited studies in the current literature examined the neural network ensemble for online regression; furthermore, these methods were combination of online individual models and did not consider the ensemble diversity. In this paper, a novel online sequential learning algorithm for neural network ensembles for online regression is proposed. The algorithm is built upon the decorrelated neural network ensembles (DNNE) and thus referred to as Online-DNNE; so it uses single-hidden layer feed-forward neural networks with random hidden nodes’ parameters as ensemble components and introduces negative correlation learning to train base models simultaneously in a cooperative manner which can effectively maintain the ensemble diversity. The Online-DNNE only learns the newly arrived data, and the computation complexity is thus reduced. The results of the experiments with benchmarks show the effectiveness and significant advantages of the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alhamdoosh M, Wang D (2014) Fast decorrelated neural network ensembles with random weights. Inf Sci 264:104–117
Boris I, Yoh H-P (1995) Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans Neural Netw 6(6):1320–1329
Bouziane H, Messabih B, Chouarfia A (2015) Effect of simple ensemble methods on protein secondary structure prediction. Soft Comput 19:1663–1678
Brown G, Wyatt JL, Tiňo P (2005) Managing diversity in regression ensembles. J Mach Learn Res 6:1621–1650
Bruce R (1996) Ensemble learning using decorrelated neural networks. Connect Sci 8:373–384
Brzezinski D, Stefanowski J (2014) Combining block-based and online methods in learning ensembles from concept drifting data streams. Inf Sci 265:50–67
Cheng W, Ding J, Kong W et al (2011) An adaptive chaotic PSO for parameter optimization and feature extraction of LS–SVM based modelling. In: American control conference (ACC), San Francisco, CA, pp 3263–3268
Grabner MNH, Bischof H (2006) On-line boosting and vision. In: Computer vision and pattern recognition, 2006 IEEE Computer Society Conference on, vol 1. IEEE, pp 260–267
Hansen LK, Salamon P (1990) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12(10):993–1001
Huang GB (2003) Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans Neural Netw 14(2):274–281
Huang GB, Babri HA (1998) Upper bounds on the number of hidden neurons in feedforward networks with arbitrary bounded nonlinear activation functions. IEEE Trans Neural Netw 9(1):224–229
Husmeier D, Taylor JG (1998) Predicting conditional probability densities with the Gaussian mixture-RVFL network. In: C Artificial neural nets and genetic algorithms. Springer, Vienna, pp 477–481
Ikonomovska E, Gama J, Džeroski S (2015) Online tree-based ensembles and option trees for regression on evolving data streams. Neurocomputing 150:458–470
Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13):3391–3395
Liang NY, Huang GB, Saratchandran P et al (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Liu Y, Yao X (1997) Negatively correlated neural networks can produce best ensembles. Aust J Intell Inf Process Syst 4(3/4):176–185
Liu Y, Yao X (1999) Ensemble learning via negative correlation. Neural Netw 12:1399–1404
Liu C, Ding J, Jiang B, Chai T (2014) Adaptive online support vector regression prediction model for concentrate grade of the ore-dressing process. Control Theory Appl 31(3):386–391
Oza NC (2005) Online bagging and boosting. In: IEEE international conference on systems, man and cybernetics, vol 3. IEEE, pp 2340–2345
Ramezani F, Nikoo M (2015) Artificial neural network weights optimization based on social-based algorithm to realize sediment over the river. Soft Comput 19:375–387
Schmidt WF, Kraaijveld MA, Duin RPW (1992) Feed forward neural networks with random weights. In: Proceedings of 11th IAPR international conference on pattern recognition methodology and systems, pp 1–4
Stephan W, Susanne S et al (2015) Data-based prediction of sentiments using heterogeneous model ensembles. Soft Comput 19:3401–3412
Wang D, Alhamdoosh M (2013) Evolutionary extreme learning machine ensembles with size control. Neurocomputing 102:98–110
Acknowledgments
This work is partly supported by the NSFC Project under Grants 61273031, 61525302, 61590922 and 71571156, and the Science Foundation of Liaoning Province Project under Grant 2014020021. Junwei Wang has received the Seed Funding Programme for basic research from University of Hong Kong (201409159015), and supported by the open project funded by State Key Laboratory of Synthetical Automation for Process Industries (PAL-N201505).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Communicated by Y. Jin.
Rights and permissions
About this article
Cite this article
Ding, J., Wang, H., Li, C. et al. An online learning neural network ensembles with random weights for regression of sequential data stream. Soft Comput 21, 5919–5937 (2017). https://doi.org/10.1007/s00500-016-2269-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-016-2269-9