Skip to main content

Advertisement

Springer Nature Link
Log in

Biomorphic robot controls: event driven model free deep SNNs for complex visuomotor tasks

  • Invited Article
  • Published:
Artificial Life and Robotics Aims and scope Submit manuscript

Abstract

The human brain surpass conventional computer architectures in regard to energy efficiency, robustness, connectivity and adaptivity. These aspects are inspiring today’s emerging biomorphic technologies into both directions, software tools, and hardware systems. Thus, it is worthwhile to investigate into biological processes which enable the brain to perform computations and how they can be modelled and implemented in silicon. Taking inspiration from how the brain performs information processing requires a shift of computational paradigms compared to conventional computer architectures. Indeed, the brain is composed of nervous cells, called neurons, connected with synapses and forming self-organized networks. Neurons and synapses are complex dynamical systems ruled by biochemical and electrical reactions. As a result, they rely on local information forming functional neural clusters. Additionally, neurons communicate with each other with short electrical pulses, called spikes, which travel across synapses. Computational neuroscientists and roboticists attempt to model these computations with spiking neural networks (SNNs) and to ground them with real robots. When implemented on dedicated neuromorphic hardware, spiking neural networks can perform time delay free, energy efficient computations in analogy to the brain. Until recently, the advantages of this technology were limited due to the lack of efficient methods for programming spiking neural networks. Reinforcement learning is one paradigm for programming spiking neural networks, in which neural clusters self-organize them towards functional networks. In this paper a survey on our research at FZI on design, implementation and experiments with SNNs, solving visuomotor tasks for biomorphic robots is given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5(4):115–133

    Article  MathSciNet  MATH  Google Scholar 

  2. Hornik K (1991) Approximation capabilities of multilayer feedforward networks. Neural Netw 4(2):251–257

    Article  MathSciNet  Google Scholar 

  3. Elman JL (1990) Finding structure in time. Cogn Sci 14(2):179–211

    Article  Google Scholar 

  4. Siegelmann HT, Sontag ED (1991) On the computational powerof neural nets. In: Proceedings of the fifth annual workshop on computational learning theory, pp 440–449, ACM

  5. Eickenberg M, Gramfort A, Varoquaux G, Thirion B (2017) Seeing it all: convolutional network layers map the function of the human visual system. Neuroimage 152:184–194

    Article  Google Scholar 

  6. Yamins DLK, DiCarlo JJ (2016) Using goal-driven deep learning models to understand sensory cortex. Nature Neurosci 19(3):356

    Article  Google Scholar 

  7. Sjöström PJ, Turrigiano GG, Nelson SB (2001) Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32(6):1149–1164

    Article  Google Scholar 

  8. Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529

    Article  Google Scholar 

  9. Hodgkin AL, Huxley AF (1952) A quantitative description of membrane current and its application to conduction and excitation in nerve. J Physiol 117(4):500–544

    Article  Google Scholar 

  10. Izhikevich EM (2003) Simple model of spiking neurons. IEEE Trans Neural Netw 14(6):1569–1572

    Article  MathSciNet  Google Scholar 

  11. Lapicque L (1907) Recherches quantitatives sur l’excitation electrique des nerfs traitee comme une polari-zation. J Physiol Pathol Gener 9:620–635

    Google Scholar 

  12. Abbott LF (1999) Lapicque’s introduction of the integrate-and-fire model neuron. Brain Res Bull 50(5–6):303–304

    Article  Google Scholar 

  13. Burkitt AN (2006) A review of the integrate-and-fire neuron model: I. Homogeneous synaptic input. Biol Cybern 95(1):1–19

    Article  MathSciNet  MATH  Google Scholar 

  14. Gerstner W, Kistler WM (2002) Spiking neuron models: single neurons, populations, plasticity. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  15. Gerstner W, Kistler WM, Naud R, Paninski L (2014) Neuronal dynamics: from single neurons to networks and models of cognition. Cambridge University Press, Cambridge

    Book  Google Scholar 

  16. Andreopoulos A, Kashyap HJ, Nayak TK, Amir A, Flickner MD (2018) A low power, high throughput, fully event-based stereo system. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7532–7542

  17. Haessig G, Cassidy A, Alvarez R, Benosman R, Orchard G (2018) Spiking optical flow for event-based sensors using IBM’s TrueNorth neurosynaptic system. IEEE Trans Biomed Circ Syst 12(4):860–870

    Article  Google Scholar 

  18. Lahiri S, Ganguli S (2013) A memory frontier for complex synapses. In: Advances in neural information processing systems, pp 1034–1042

  19. Wolpert DM, Ghahramani Z, Flanagan JR (2001) Perspectives and problems in motor learning. Trends Cogn Sci 5(11):487–494

    Article  Google Scholar 

  20. Zenke F, Ganguli S (2018) Superspike: supervised learning in multilayer spiking neural networks. Neural Comput 30(6):1514–1541

    Article  MathSciNet  MATH  Google Scholar 

  21. Bellec G, Scherr F, Hajek E, Salaj D, Legenstein R, Maass W (2019) Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets. arXiv:1901.09049

  22. Hansen N, Ostermeier A (2001) Completely derandomized self adaptation in evolution strategies. Evol Comput 9(2):159–195

    Article  Google Scholar 

  23. Baldi P, Sadowski P (2016) A theory of local learning, the learning channel, and the optimality of backpropagation. Neural Netw 83:51–74

    Article  Google Scholar 

  24. Eckenberg M, Gramford A, Varoquaux G, Thirion B (2017) Seeing it all: convolutional netwörk layers map the function of the human visual system. Neuroimage 152:184–194

    Article  Google Scholar 

  25. Maass W, Natschläger T, Markram H (2002) Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput 14(11):2531–2560

    Article  MATH  Google Scholar 

  26. Florian RV (2007) Reinforcement learning through modulation of spiketiming-dependent synaptic plasticity. Neural Comput 19(6):1468–1502

    Article  MathSciNet  MATH  Google Scholar 

  27. Lillicrap TP, Santoro A, Marris L, Akerman CJ, Hinton G (2020) Backpropagation and the brain. Nature Rev Neurosci 2020:5

    Google Scholar 

  28. Mueggler E, Huber B, Scaramuzza D (2014) Event-based, 6-dof pose tracking for high-speed maneuvers. In: 2014 IEEE/RSJ international conference on intelligent robots and systems, pp 2761–2768, IEEE

  29. Mead CA, Mahowald MA (1988) A silicon model of early visual processing. Neural Networks 1(1):91–97

    Article  Google Scholar 

  30. Lichtsteiner P, Posch C, Delbruck T (2008) A 128x128 100dB 15µs latency asynchronous temporal contrast vision sensor. J Solid-State Circuits 43(2):566–576

    Article  Google Scholar 

  31. Pfeiffer M, Pfeil T (2018) Deep learning with spiking neurons:opportunities and challenges. Front Neurosci 2018:12

    Google Scholar 

  32. Sironi A, Brambilla M, Bourdis N, Lagorce X, Benosman R (2018) Hats: histograms of averaged time surfaces for robust event-based object classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1731–1740

  33. Davies M, Shrinavas N, Lin T-H, Chinya G, Cao Y, Choday SH, Dinou G, Joshi P, Imam N, Jain S et al (2018) Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38(1):82–99

    Article  Google Scholar 

  34. Bing Z, Meschede C, Huang K, Chen G, Rohrbein F, Akl M, Knoll A (2018) End to end learning of spiking neural network based on r-stdp for a lane keeping vehicle. In: 2018 IEEE international conference on robotics and automation (ICRA). IEEE, pp 1–8

  35. Jaeger H (2007) Echo state network. Scholarpedia 2(9):2330

    Article  Google Scholar 

  36. Kaiser J et al (2016) Towards a framework for end-to-end control of a simulated vehicle with spiking neural networks. In: International conference on simulation, modeling, and programming for autonomous robots (SIMPAR), pp 127–134, IEEE

  37. Kaiser J, Hoff M, Konle A, Tieck JCV, Kappel D, Reichard D, Subramoney A, Legenstein R, Roennau A, Maass W et al (2019) Embodied synaptic plasticity with online reinforcement learning. Front Neurorobot 13:81

    Article  Google Scholar 

  38. Kaiser J, Melbaum S, Tieck JCV, Roennau A, Butz MV, Dillmann R (2018) Learning to reproduce visually similar movements by minimizing event-based prediction error. In: International conference on biomedical robotics and biomechatronics (BIOROB), IEEE

  39. Kaiser J, Stal R, Subramoney A, Roennau A, Dillmann R (2017) Scaling up liquid state machines to predict over address events from dynamic vision sensors. Bioinspiration Biomimet 12(5):055001

    Article  Google Scholar 

  40. Falotico E et al (2017) Connecting artificial brains to robots in a comprehensive simulation framework: the neurorobotics platform. Front Neurorobot 11:2

    Article  Google Scholar 

  41. Jiang Z, Bing Z, Huang K, Knoll AC (2019) Retina based pipe-like object tracking implemented through spiking neural network on a snake robot. Front Neurorobot 13:29

    Article  Google Scholar 

  42. Kaiser J, Weinland J, Keller P, Steffen L, Tieck JCV, Reichard D, Roennau A, Conradt J, Dillmann R (2018). Microsaccades for neuromorphic stereo vision. In: International conference on artificial neural networks (ICANN), pp 244–252, Springer

  43. Kaiser J, Friedrich A, Tieck JCV, Reichard D, Roennau A, Neftci E, Dillmann R (2019) Embodied neuromorphic vision with event-driven random backpropagation. arXiv:1904.04805

  44. Kaiser J, Lindner G, Tieck JCV, Schulze M, Hoff M, Roennau A, Dillmann R (2018) Microsaccades for asynchronous feature extraction with spiking networks. In: International conference on development and learning and epigenetic robotics (ICDL-EPIROB), IEEE

  45. Kaiser J, Mostafa H, Neftci E (2020) Synaptic plasticity dynamics for deep continuous local learning (decolle). Front Neurosci 14:424

    Article  Google Scholar 

  46. Neftci EO, Augustine C, Paul S, Detorakis G (2017) Event-driven random back-propagation: enabling neuromorphic deep learning machines. Front Neurosci 11:324

    Article  Google Scholar 

  47. Bizzi E, Cheung V, Avella AD, Saltiel P, Tresch M (2008) Combining modules for movement. Brain Res Rev 57:1

    Article  Google Scholar 

  48. Avella AD, Saltiel P, Bizzi E (2003) Combinations of muscle synergies in the construction of a natural motor behavior. Nature Neurosci 6:3

    Article  Google Scholar 

  49. Bogdan P, Pineda Garcia G, Davidson S, Hopkins M, James R, Furber S (2019) Event-based computation: Unsupervised elementary motion decomposition. In: Proceedings of the 2019 emerging technology conference. EMiT/University of Huddersfield/High End Compute Ltd/University of Manchester

  50. Ijspeert AJ, Nakanishi J, Hoffmann H, Pastor P, Schaal S (2013) Dynamical movement primitives: Learning attractor models for motor behaviors. Neural Comput 25:2

    Article  MathSciNet  MATH  Google Scholar 

  51. Ciocarlie M, Goldfeder C, Allen P (2018) Dexterous grasping via eigengrasps: A low-dimensional approach to a high-complexity problem. In: Robotics: science and systems manipulation workshop-sensing and adapting to the real world, Citeseer

  52. Sburlea AI, Mueller-Putz GR (2018) Exploring re-presentations of human grasping in neural, muscle and kinematic signals. Sci Rep 8:1

    Article  Google Scholar 

  53. Crago PE, Nakai RJ, Chizeck HJ (1991) Feedback regulation of hand grasp opening and contact force during stimulation of paralyzed muscle. IEEE Trans Biomed Eng 38:1

    Article  Google Scholar 

  54. Cutkosky MR (1989) On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Trans Robot Autom 5:3

    Google Scholar 

  55. Scano A, Chiavenna A, Tosatti LM, Mueller H, Atzori M (2018) Muscle synergy analysis of a hand-grasp dataset: a limited subset of motor modules may underlie a large variety of grasps. Front Neurorobot 2018:12

    Google Scholar 

  56. Kim S-M, Hyun S-Y, Sohn J-W, Chae S, Kim S-P (2019) Neural response to grasp of robot hand from M1 area of Rhesus monkey. In: 2019 7th international winter conference on brain-computer interface (BCI), IEEE

  57. Ruehl SW, Parlitz C, Heppner G, Hermann A, Roennau A, Dillmann R (2014) Experimental evaluation of the Schunk 5-finger gripping hand for grasping tasks. In: 2014 IEEE international conference on robotics and biomimetics (ROBIO 2014), IEEE

  58. Ficuciello F, Federico A, Lippiello V, Siciliano B (2016) Synergies evaluation of the SCHUNK S5FH for grasping control. In: Advances in robot kinematics 2016. Springer

  59. Starke J, Eichmann C, Ottenhaus S, Asfour T (2020) Human-inspired representation of object-specific grasps for anthropomorphic hands. Int J Human Robot 17:2

    Article  Google Scholar 

  60. Andrychowicz OM, Baker B, Chociej M, Jozefowicz R, Mc-Grew B, Pachocki J, Petron A, Plappert M, Powell G, Ray A et al (2019) Learning dexterous in-hand manipulation. Int J Robot Res 39:1

    Google Scholar 

  61. Santina CD, Arapi V, Averta G, Damiani F, Fiore G, Settimi A, Catalano MG, Bacciu D, Bicchi A, Bianchi M (2019) Learning from humans how to grasp: a data-driven architecture for autonomous grasping with anthropomorphic soft hands. IEEE Robot Autom Lett 4(2):1533–1540

    Article  Google Scholar 

  62. Levine S, Pastor P, Krizhevsky A, Ibarz J (2018) Large-scale data collection. Int J Robot Res 205:5

    Google Scholar 

  63. Scherzinger S, Roennau A, Dillmann R (2017) Forward dynamics compliance control (FDCC): a new approach to cartesian compliance for robotic manipulators. In: 2017 IEEE/RSJ International conference on intelligent robots and systems (IROS), IEEE

  64. DeWolf T, Stewart TC, Slotine J-J, Eliasmith C (2016) A spiking neural model of adaptive arm control. Proc R Soc B Biol Sci 283:1843

    Google Scholar 

  65. Capolei MC, Angelidis E, Falotico E, Lund HH, Tolu S (2019) A biomimetic control method increases the adaptability of a humanoid robot acting in a dynamic environment. Front Neurorobot 13:5

    Article  Google Scholar 

  66. Urbain G, Barasuol V, Semini C, Dambre J et al (2018) Stance control inspired by cerebellum stabilizes reflex-based locomotion on HyQrobot. arXiv:2003.09327

  67. Tieck JCV, Donat H, Kaiser J, Peric I, Ulbrich S, Roennau A, Zoellner M, Dillmann R (2018) Towards grasping with spiking neural networks for anthropomorphic robot hands. In: International conference on artificial neural networks ICANN, vol 10613, LNCS

  68. Tieck JCV, Weber S, Stewart TC, Kaiser J, Roennau A, Dillmann R (2018) A spiking network classifies human sEMG signals and triggers finger reflexes on a robotic hand. Robot Auton Syst 2018:5

    Google Scholar 

  69. Tieck JCV, Rutschke J, Kaiser J, Schulze M, Buettner T, Reichard D, Roennau A, Dillmann R (2018) Combining spiking motor primitives with a behavior-based architecture to model locomotion for six-legged robots. In: IEEE/RSJ international conference on intelligent robots and systems (IROS)

  70. Tieck JCV, Steffen L, Kaiser J, Roennau A, Dillmann R (2018) Multi-modal motion activation for robot control using spiking neurons. Biomed Robot Biomechatron (BioRob) 2018:5

    Google Scholar 

  71. Eliasmith C (2020) How to build a brain: a neural architecture for biological cognition. Oxford University Press, Oxford

    Google Scholar 

  72. Furber SB, Temple S, Brown A (2016) High-performance computing for systems of spiking neurons. In: AISB’06 Workshop, GC5: Archit Brain Mind

  73. Davies M, Srinivasa N, Lin T-H, Chinya G, Cao Y, Choday SH, Dimou G, Joshi P, Imam N, Jain S et al (2018) Loihi: a neuromorphic manycore processor with on-chip learning. IEEE Micro 38:1

    Article  Google Scholar 

  74. Rhodes O, Peres L, Rowley AG, Gait A, Plana LA, Brenninkmeijer C, Furber SB (2019) Real-time cortical simulation on neuromorphic hardware. Philos Trans R Soc 378:2164

    Google Scholar 

  75. Donati E, Perez-Pena F, Bartolozzi C, Indiveri G, Chicca E (2018) Open-Loop neuromorphic controller implemented on VLSI devices. In: 2018 7th IEEE international conference on biomedical robotics and biomechatronics (Biorob), IEEE

  76. Haessig G, Milde MB, Aceituno PV, Oubari O, Knight JC, van Schaik A, Benosman RB, Indiveri G (2020) Event-based computation for touch localization based on precise spike timing. Front Neurosci 2020:14

    Google Scholar 

  77. Kaiser J, Weinland J, Keller P, Steffen L, Tieck JCV, Reichard D, Roennau A, Conradt J, Dillmann R (2018) Microsaccades for neuromorphic stereo vision. In: 2018 7th IEEE international conference on biomedical robotics and biomechatronics (BIOROB)

  78. Steffen L, Ulbrich S, Roennau A, Dillmann R (2019) Multi-view 3D reconstruction with self-organizing maps on event-based data. In: 2019,19th international conference on advanced robotics (ICAR). IEEE

Download references

Acknowledgements

This research has received funding from the European Union’s Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement No. 785907 Human Brain Project SGA2 and the Baden-Wuerttemberg Science Foundation

Author information

Authors and Affiliations

  1. Research Center Informatics FZI, Interactive Diagnosis Systems IDS, Karlsruhe, Germany

    Rüdiger Dillmann & Arne Rönnau

Authors
  1. Rüdiger Dillmann
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Arne Rönnau
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Rüdiger Dillmann.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was presented in part as a plenary speech at the joint symposium of the 27th International Symposium on Artificial Life and Robotics, the 7th International Symposium on BioComplexity, and the 5th International Symposium on Swarm Behavior and Bio-Inspired Robotics (Online, January 25–27, 2022).

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dillmann, R., Rönnau, A. Biomorphic robot controls: event driven model free deep SNNs for complex visuomotor tasks. Artif Life Robotics 27, 429–440 (2022). https://doi.org/10.1007/s10015-022-00769-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10015-022-00769-4

Keywords