Abstract
“Echo State” neural networks, which are a special case of recurrent neural networks, are studied from the viewpoint of their learning ability, with a goal to achieve their greater predictive ability. In this paper we study the influence of the memory length on predictive abilities of Echo State neural networks. The conclusion is that Echo State neural networks with fixed memory length can have troubles with adaptation of its intrinsic dynamics to dynamics of the prediction task. Therefore, we have tried to create complex prediction system as a combination of the local expert Echo State neural networks with different memory length and one special gating Echo State neural network. This approach was tested in laser fluctuations prediction. The prediction error achieved by this approach was substantially smaller in comparison with prediction error achieved by standard Echo State neural networks.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Jacobs, R., Jordan, M., Nowlan, S., Hinton, G.: Adaptive mixtures of local experts. Neural Computation 3, 79–87 (1991)
Jaeger, H.: The Echo State Approach to Analysing and Training Recurrent Neural Networks. German National Research Center for Information Technology, GMD report 148 (2001)
Haykin, S.: Neural networks - A comprehensive foundation. Macmillan Publishing, New York (1994)
Jordan, M., Jacobs, R.: Hierarchical mixtures of experts and the EM algorithm. Neural Computation 6, 181–214 (1994)
Carvalho, A., Tanner, M.: Mixtures-of-experts of autoregressive time series: asymptotic normality and model specification. IEEE Transactions on Neural Networks 16(1), 39–56 (2005)
Jaeger, H.: Short Term Memory in Echo State Networks. German National Research Center for Information Technology, GMD report 152 (2002)
Natschlager, T., Maass, W., Markram, H.: The ”liquid computer”: A novel strategy for real-time computing on time series. Special Issue on Foundations of Information Processing of TELEMATIK 8(1), 39–43 (2002)
Babinec, S., Pospichal, J.: Optimization in Echo state neural networks by Metropolis algorithm. In: Matousek, R., Osmera, P. (eds.) Proceedings of the 10th International Conference on Soft Computing, Mendel 2004, pp. 155–160. VUT Brno Publishing (2004)
Babinec, S., Pospichal, J.: Two approaches to optimize echo state neural networks. In: Matousek, R., Osmera, P. (eds.) Proceedings of the 11th International Conference on Soft Computing, Mendel 2005, pp. 39–44. VUT Brno Publishing (2005)
Babinec, S., Pospichal, J.: Improving the Prediction Accuracy of Echo State Neural Networks by Anti-Oja’s Learning. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D.P. (eds.) ICANN 2007. LNCS, vol. 4668, pp. 19–28. Springer, Heidelberg (2007)
Goldenholz, D.: Liquid computing: A real effect. Technical report, Boston University Department of Biomedical Engineering (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Babinec, Š., Pospíchal, J. (2009). Gating Echo State Neural Networks for Time Series Forecasting. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5506. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02490-0_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-02490-0_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02489-4
Online ISBN: 978-3-642-02490-0
eBook Packages: Computer ScienceComputer Science (R0)