Abstract
In this paper, we demonstrate that goal-directed behavior unfolds in recurrent spiking neural networks (RSNNs) when intentions are projected onto continuously progressing spike dynamics encoding the recent history of an agent’s state. The projections, which can either be realized via backpropagation through time (BPTT) over a certain time window or even directly and temporally local in an online fashion using a biologically inspired inference rule. In contrast to previous studies that use, for instance, LSTM-like models, our approach is biologically more plausible as it fully relies on spike-based processing of sensorimotor experiences. Specifically, we show that precise control of a flying vehicle in a 3D environment is possible. Moreover, we show that more complex mental traces of foresighted movement imagination unfold that effectively help to circumvent learned obstacles.
We thank the International Max Planck Research School for Intelligent Systems (IMPRS-IS) for supporting Manuel Traub. Martin Butz is part of the Machine Learning Cluster of Excellence, EXC number 2064/1 – Project number 390727645.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bellec, G., Salaj, D., Subramoney, A., Legenstein, R., Maass, W.: Long short-term memory and learning-to-learn in networks of spiking neurons. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 795–805. Curran Associates Inc., Red Hook (2018)
Bellec, G., et al.: A solution to the learning dilemma for recurrent networks of spiking neurons. Nat. Commun. 11(1), 1–15 (2020)
Butz, M.V., Bilkey, D., Humaidan, D., Knott, A., Otte, S.: Learning, planning, and control in a monolithic neural event inference architecture. Neural Netw. 117, 135–144 (2019)
Butz, M.V., Kutter, E.F.: How the Mind Comes Into Being: Introducing Cognitive Science from a Functional and Computational Perspective. Oxford University Press, Oxford (2016)
Butz, M.V., Menge, T., Humaidan, D., Otte, S.: Inferring event-predictive goal-directed object manipulations in REPRISE. In: Tetko, I.V., Kurková, V., Karpov, P., Theis, F. (eds.) ICANN 2019, Part I. LNCS, vol. 11727, pp. 639–653. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30487-4_49
Clark, A.: Surfing Uncertainty: Prediction, Action, and the Embodied Mind. Oxford University Press, Oxford (2015)
Friston, K.: The free-energy principle: a rough guide to the brain? Trends Cogn. Sci. 13(7), 293–301 (2009)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Mnih, V., et al.: Playing atari with deep reinforcement learning. arXiv:1312.5602 (2013)
Nichol, A., Pfau, V., Hesse, C., Klimov, O., Schulman, J.: Gotta learn fast: a new benchmark for generalization in RL. arXiv:1804.03720 (2018)
OpenAI, et al.: Dota 2 with large scale deep reinforcement learning. arXiv:1912.06680 (2019)
Otte, S., Schmitt, T., Friston, K., Butz, M.V.: Inferring adaptive goal-directed behavior within recurrent neural networks. In: Lintas, A., Rovetta, S., Verschure, P.F.M.J., Villa, A.E.P. (eds.) ICANN 2017, Part I. LNCS, vol. 10613, pp. 227–235. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68600-4_27
Otte, S., Stoll, J., Butz, M.V.: Incorporating adaptive RNN-based action inference and sensory perception. In: Tetko, I.V., Kůrková, V., Karpov, P., Theis, F. (eds.) ICANN 2019. LNCS, vol. 11730, pp. 543–555. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30490-4_44
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv:1707.06347 (2017)
Traub, M., Legenstein, R., Otte, S.: Many-joint robot arm control with recurrent spiking neural networks. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2021). accepted for publication, preprint available (arXiv:2104.04064)
Yin, B., Corradi, F., Bohte, S.M.: Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. arXiv:2103.12593 (2021)
Yin, B., Corradi, F., Bohté, S.M.: Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks. bioRxiv:2021.03.22.436372 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Traub, M., Butz, M.V., Legenstein, R., Otte, S. (2021). Dynamic Action Inference with Recurrent Spiking Neural Networks. In: Farkaš, I., Masulli, P., Otte, S., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2021. ICANN 2021. Lecture Notes in Computer Science(), vol 12895. Springer, Cham. https://doi.org/10.1007/978-3-030-86383-8_19
Download citation
DOI: https://doi.org/10.1007/978-3-030-86383-8_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86382-1
Online ISBN: 978-3-030-86383-8
eBook Packages: Computer ScienceComputer Science (R0)