Abstract
A lot of attention is now being focused on connectionist models known under the name “reservoir computing”. The most prominent example of these approaches is a recurrent neural network architecture called an echo state network (ESN). ESNs were successfully applied in several time series modeling tasks and according to the authors they performed exceptionally well. Multiple enhancements to standard ESN were proposed in the literature. In this paper we follow the opposite direction by suggesting several simplifications to the original ESN architecture. ESN reservoir features contractive dynamics resulting from its’ initialization with small weights. Sometimes it serves just as a simple memory of inputs and provides only negligible “extra-value” over much simple methods. We experimentally support this claim and we show that many tasks modeled by ESNs can be handled with much simple approaches.
This work was supported by the grants VG-1/0848/08 and VG-1/0822/08.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks. Technical Report GMD 148, German National Research Center for Information Technology (2001)
Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, vol. 15, pp. 593–600. MIT Press, Cambridge (2003)
Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)
Frank, S.L.: Learn more by training less: Systematicity in sentence processing by recurrent networks. Connection Science (in press, 2006)
Prokhorov, D.: Echo state networks: Appeal and challenges. In: Proceedings of International Joint Conference on Neural Networks, IJCNN 2005, Montreal, Canada, pp. 1463–1466 (2005)
Jaeger, H.: Reservoir riddles: Suggestions for echo state network research. In: Proceedings of International Joint Conference on Neural Networks IJCNN 2005, Montreal, Canada, pp. 1460–1462 (2005)
Čerňanský, M., Makula, M.: Feed-forward echo state networks. In: Proceedings of International Joint Conference on Neural Networks IJCNN 2005, Montreal, Canada, pp. 1479–1482 (2005)
Jaeger, H.: Adaptive nonlinear system identification with echo state networks. In: Proceedings of Neural Information Processing Systems NIPS 2002, Vancouver, Canada (2002)
Xue, Y., Yang, L., Haykin, S.: Decoupled echo state network with lateral inhibition. IEEE Transactions on Neural Network (January 2007) (in press)
Wierstra, D., Gomez, F.J., Schmidhuber, J.: Modeling systems with internal state using evolino. In: GECCO 2005: Proceedings of the 2005 conference on Genetic and evolutionary computation, pp. 1795–1802. ACM, New York (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Čerňanský, M., Tiňo, P. (2008). Predictive Modeling with Echo State Networks. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9_80
Download citation
DOI: https://doi.org/10.1007/978-3-540-87536-9_80
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87535-2
Online ISBN: 978-3-540-87536-9
eBook Packages: Computer ScienceComputer Science (R0)