Abstract
In recent years, Extreme Learning Machine (ELM) has attracted comprehensive attentions as a universal function approximator. Comparing to other single layer feedforward neural networks, its input parameters of hidden neurons can be randomly generated rather than tuned, and thereby saving a huge amount of computational power. However, it has been pointed out that the randomness of ELM parameters would result in fluctuating performances. In this paper, we intensively investigate the randomness reduction effect by using a regularized version of ELM, named Ridge ELM (RELM). Previously, RELM has been shown to achieve generally better generalization than the original ELM. Furthermore, we try to demonstrate that RELM can also greatly reduce the fluctuating performance with 12 real world regression tasks. An insight into this randomness reduction effect is also given.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Zhu, Q.-Y., Qin, A.K., Suganthan, P.N., Huang, G.-B.: Evolutionary extreme learning machine. Pattern Recognition 38, 1759–1763 (2005)
Lan, Y., Soh, Y., Huang, G.-B.: Ensemble of online sequential extreme learning machine. Neurocomputing 72(13-15), 3391–3395 (2009)
Suresh, S., Saraswathi, S., Sundararajan, N.: Performance enhancement of extreme learning machine for multi-category sparse data classification problems. Engineering Applications of Artificial Intelligence 23(7), 1149–1157 (2010)
Liu, N., Wang, H.: Ensemble based extreme learning machine. IEEE Signal Processing Letters 17(8), 754–757 (2010)
Miche, Y., Sorjamaa, A., Bas, P., Simula, O., Jutten, C., Lendasse, A.: Op-elm: optimally pruned extreme learning machine. IEEE Transactions on Neural Networks 21(1), 158–162 (2010)
Sun, Z., Choi, T., Au, K., Yu, Y.: Sales forecasting using extreme learning machine with applications in fashion retailing. Decision Support Systems 46(1), 411–419 (2008)
Feng, G., Huang, G.-B., Lin, Q., Gay, R.: Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Transactions on Neural Networks 20(8), 1352–1357 (2009)
Lan, Y., Soh, Y., Huang, G.-B.: Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73, 3191–3199 (2010)
Lan, Y., Soh, Y., Huang, G.-B.: Two-stage extreme learning machine for regression. Neurocomputing 73(16-18), 3028–3038 (2010)
Hoerl, A., Kennard, R.: Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 55–67 (1970)
Huang, G.-B., Wang, D., Lan, Y.: Extreme learning machines: a survey. International Journal of Machine Learning and Cybernetics (2011)
Huang, G.-B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics (99), 1–17 (2010)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12(10), 993–1001 (1990)
Toh, K.: Deterministic neural classification. Neural Computation 20(6), 1565–1595 (2008)
Newman, D., Asuncion, A.: UCI machine learning repository (2007)
Vlachos, P.: Statlib project repository. Carnegie Mellon University (2000)
Mardikyan, S., Cetin, E.: Efficient choice of biasing constant for ridge regression. Int. J. Contemp. Math. Sciences 3(11), 527–536 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Er, M.J., Shao, Z., Wang, N. (2013). A Study on the Randomness Reduction Effect of Extreme Learning Machine with Ridge Regression. In: Guo, C., Hou, ZG., Zeng, Z. (eds) Advances in Neural Networks – ISNN 2013. ISNN 2013. Lecture Notes in Computer Science, vol 7951. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39065-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-39065-4_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39064-7
Online ISBN: 978-3-642-39065-4
eBook Packages: Computer ScienceComputer Science (R0)