Improving the Prediction Accuracy of Echo State Neural Networks by Anti-Oja’s Learning

Abstract

Echo state neural networks, which are a special case of recurrent neural networks, are studied from the viewpoint of their learning ability, with a goal to achieve their greater prediction ability. A standard training of these neural networks uses pseudoinverse matrix for one-step learning of weights from hidden to output neurons. This regular adaptation of Echo State neural networks was optimized by updating the weights of the dynamic reservoir with Anti-Oja’s learning. Echo State neural networks use dynamics of this massive and randomly initialized dynamic reservoir to extract interesting properties of incoming sequences. This approach was tested in laser fluctuations and Mackey-Glass time series prediction. The prediction error achieved by this approach was substantially smaller in comparison with prediction error achieved by a standard algorithm.