Abstract
The echo state network introduces a large and sparse reservoir to replace the hidden layer of the traditional recurrent neural network, which can solve the gradient-based problem of most recurrent neural networks in the training process. However, there may be an ill-posed problem when the least square method is used to calculate the output weight. In this paper, we proposed an echo state network based on L0 norm regularization. The main idea is to limit the number of output connections to compute the output effectively by removing unimportant ones. The simulation results of the chaotic time series prediction show the effectiveness of the proposed model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Haykin, S., Principe, J.: Making sense of a complex world [chaotic events modeling]. IEEE Signal Process. Magaz. 15(3), 66–81 (1998)
Yu, Z.: Chaos and symbol analysis and practice of time series (in Chinese). National University of Defense Technology Press, Changsha (2007)
Namikawa, J., Tani, J.: Building recurrent neural networks to implement multiple attractor dynamics using the gradient descent method. Advances in Artificial Neural Systems (2009)
Jaeger, H.: The “Echo State” approach to analyzing and training recurrent neural networks. Bonn, Germany: German National Research Center for Information Technology GMD Technical Report 148(34), 13 (2001)
Qiao, J.F., Bo, Y.C., Han, G.: Application of ESN-based multi indices dual heuristic dynamic programming on wastewater treatment process. Acta Automatica Sinica 39(7), 1146–1151 (2013)
Ongenae, F., Van, L.S., Verstraeten, D., Verplancke, T., Benoit, D., De, T.F., Dhaene, T., Schrauwen, B., Decruyenaere, J.: Time series classification for the prediction of dialysis in critically ill patients using echo state networks. Eng. Appl. Artif. Intell. 26(3), 984–996 (2013)
Sheng, C., Zhao, J., Liu, Y., et al.: Prediction for noisy nonlinear time series by echo state network based on dual estimation. Neurocomputing 82, 186–195 (2012)
Peng, Y., Wang, J.M., Peng, X.: Researches on time series prediction with echo state networks. Acta Electronica Sinica 38(b02), 148–154 (2010)
Chatzis, S.P., Demiris, Y.: Echo state Gaussian process. IEEE Trans. Neural Netw. 22(9), 1435–1445 (2011)
Haykin, S.: Neural Networks: a compressive foundation. Tsinghua University Press, Beijing (2001)
Han, M., Ren, W.J., Xu, M.L.: An Improved Echo State Network via L1-norm Regularization. J. Autom. 000(011), 2428–2435 (2014)
Xu, M., Han, M., Kanae, S.: L1/2norm regularized echo state network for chaotic time series prediction. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, Minho, Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9949, pp. 12–19. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46675-0_2
Iosifidis, A., Tefas, A.: DropELM: Fast Neural Network Regularization with Dropout and DropConnect. Neurocomputing 162, 57–66 (2015)
Dicker, L., Huang, B.S., Lin, X.L.: Variable selection and estimation with the seamless-L0 penalty. Statistica Sinica 23(2), 929–962 (2013)
Goodfellow, I., Haykin, Y., Courville, A.: Deep learning. MIT press, US (2016)
Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Proceedings of 27th Asilomar Conference on Signals, Systems and Computers, IEEE, pp. 40–44 (1993)
Manat, S., Zhang, Z.: Matching pursuit in a time-frequency dictionary. IEEE Trans. Signal Process. 12, 3397–3451 (1993)
Bianchi, F.M., Maiorino, E., Kampffmeyer, M.C., et al.: An overview and comparative analysis of recurrent neural networks for short term load forecasting. arXiv preprint arXiv:1705.04378 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Li, L., Huang, F., Yu, Z. (2020). Echo State Network Based on L0 Norm Regularization for Chaotic Time Series Prediction. In: Yu, Z., Becker, C., Xing, G. (eds) Green, Pervasive, and Cloud Computing. GPC 2020. Lecture Notes in Computer Science(), vol 12398. Springer, Cham. https://doi.org/10.1007/978-3-030-64243-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-64243-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64242-6
Online ISBN: 978-3-030-64243-3
eBook Packages: Computer ScienceComputer Science (R0)