Abstract
We have expected that dynamic higher functions such as “thinking” emerge through the growth from exploration in the framework of reinforcement learning (RL) using a chaotic Neural Network (NN). In this frame, the chaotic internal dynamics is used for exploration and that eliminates the necessity of giving external exploration noises. A special RL method for this framework has been proposed in which “traces” were introduced. On the other hand, reservoir computing has shown its excellent ability in learning dynamic patterns. Hoerzer et al. showed that the learning can be done by giving rewards and exploration noises instead of explicit teacher signals. In this paper, aiming to introduce the learning ability into our new RL framework, it was shown that the memory-required task in the work of Hoerzer et al. could be learned without giving exploration noises by utilizing the chaotic internal dynamics while the exploration level was adjusted flexibly and autonomously. The task could be learned also using “traces”, but still with problems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Shibata, K., Okabe, Y.: Reinforcement learning when visual signals are directly given as inputs. In: Proceedings of ICNN 1997, vol. 3, pp. 1716–1720 (1997)
Shibata, K.: Emergence of intelligence through reinforcement learning with a neural network. In: Mellouk, A. (ed.) Advances in Reinforcement Learning, pp. 99–120. InTech, Rijeka (2011)
Shibata, K., Utsunomiya, H.: Discovery of pattern meaning from delayed rewards by reinforcement learning with a recurrent neural network. In: Proceedings of IJCNN, pp. 1445–1452 (2011)
Shibata, K., Goto, K.: Emergence of flexible prediction-based discrete decision making and continuous motion generation through actor-q-learning. In: Proceedings of ICDL-Epirob, ID 15 (2013)
Sawatsubashi, Y., et al.: Emergence of discrete and abstract state representation in continuous input task. In: Robot Intelligence Technology and Applications, pp. 13–22 (2012)
Shibata, K., Sakashita, Y.: Reinforcement learning with internal-dynamics-based exploration using a chaotic neural network. In: Proceedings of International Joint Conference on Neural Networks (IJCNN) (2015). 2015.7
Goto, Y., Shibata, K.: Emergence of higher exploration in reinforcement learning using a chaotic neural network. In: Akira, H., Seiichi, O., Kenji, D., Kazushi, I., Minho, L., Derong, L. (eds.) ICONIP 2016. LNCS, pp. 40–48. Springer, Heidelberg (2016)
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks. GMD report 148, p. 43 (2001)
Maass, W., Natschlger, T., Markram, H.: Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14(11), 2531–2560 (2002)
Sussillo, D., Abbott, L.F.: Generating coherent patterns of activity from chaotic neural networks. Neuron 63(4), 544–557 (2009)
Hoerzer, G.M., Legenstein, R., Maass, W.: Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb. Cortex 24(3), 677–690 (2014)
Acknowledgement
The authors wish to thank Prof. Hiromichi Suetani for introducing FORCE Learning and the work of Hoerzer et al. to us. This work was supported by JSPS KAKENHI Grant Number 15K00360.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Matsuki, T., Shibata, K. (2016). Reward-Based Learning of a Memory-Required Task Based on the Internal Dynamics of a Chaotic Neural Network. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9947. Springer, Cham. https://doi.org/10.1007/978-3-319-46687-3_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-46687-3_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46686-6
Online ISBN: 978-3-319-46687-3
eBook Packages: Computer ScienceComputer Science (R0)