Skip to main content
SpringerLink
Log in

Brain-Inspired Spike Timing Model of Dynamic Visual Information Perception and Decision Making with STDP and Reinforcement Learning

  • Conference paper
  • First Online:
Machine Learning, Optimization, and Data Science (LOD 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12566))

Abstract

The paper presents a brain-inspired spike timing neural network model of dynamic visual information processing and decision making implemented in the NEST simulator. It consists of multiple layers with functionality corresponding to the main visual information processing structures up to the areas responsible for decision making based on accumulated sensory evidence as well as the basal ganglia that modulate its response due to the feedback from the environment. The model has rich feedforward and feedback connectivity based on the knowledge about involved brain structures and their connections. The introduced spike timing-dependent plasticity and dopamine-dependent synapses allowed for its adaptation to external reinforcement signal. Simulations with specific visual stimuli and external reinforcement signal demonstrated that our model is able to change its decision via the considered as biologically plausible reinforcement learning.

This work is supported by the Bulgarian Science Fund project No DN02/3/2016 “Modelling of voluntary saccadic eye movements during decision making”.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Barto, A.G.: Adaptive Critics and the Basal Ganglia. in Models of Information Processing in the Basal Ganglia. In: Houk, J.C., Davis, J.L., Beiser, DG., (eds.) MIT Press, Cambridge, pp. 215–232 (1995)

    Google Scholar 

  2. Barto, A.G., Sutton, R.S., Anderson, C.W.: Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans. Syst. Man Cybern. B Cybern. 13(5), 834–846 (1983)

    Article  Google Scholar 

  3. Bocheva, N., Genova, B., Stefanova, M.: Drift diffusion modeling of response time in heading estimation based on motion and form cues. Int. J. Biol. Biomed. Eng. 12, 75–83 (2018)

    Google Scholar 

  4. Bogacz, R., Larsen, T.: Integration of reinforcement learning and optimal decision-making theories of the basal ganglia. Neural Comput. 23(4), 817–851 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  5. Casti, A., Hayot, F., Xiao, Y., Kaplan, E.: A simple model of retina-LGN transmission. J. Comput. Neurosci. 24, 235–252 (2008)

    Article  Google Scholar 

  6. Dunovan, K., Lynch, B., Molesworth, T., Verstynen, T.: Competing basal-ganglia pathways determine the difference between stopping and deciding not to go. ELife, pp. 1–24 (2015) https://doi.org/10.7554/eLife.08723

  7. Dunovan, K., Verstynen, T.: Believer-Skeptic meets Actor-Critic : Rethinking the role of basal ganglia pathways during decision-making and reinforcement learning. Frontiers Neurosci. 10(March), 1–15 (2016). https://doi.org/10.1101/037085

    Article  Google Scholar 

  8. Escobar, M.-J., Masson, G.S., Vieville, T., Kornprobst, P.: Action recognition using a bio-inspired feedforward spiking network. Int. J. Comput. Vis. 82, 284–301 (2009)

    Article  Google Scholar 

  9. Frank, M.J., Seeberger, L.C., O’Reilly, R.C.: By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306(5703), 1940–1943 (2004). https://doi.org/10.1126/science.1102941

    Article  Google Scholar 

  10. Fregnac, Y., Bathellier, B.: Cortical correlates of low-level perception: from neural circuits to percepts. Neuron 88, 110–126 (2015)

    Article  Google Scholar 

  11. Ghodratia, M., Khaligh-Razavic, S.-M., Lehky, S.R.: Towards building a more complex view of the lateral geniculate nucleus: recent advances in understanding its role. Prog. Neurobiol. 156, 214–255 (2017)

    Article  Google Scholar 

  12. Gleeson, P., Martinez, R., Davison, A.: Network models of V1. http://www.opensourcebrain.org/projects/111 2016)

  13. Herz, D.M., Zavala, B.A., Bogacz, R., Brown, P.: Neural Correlates of Decision Thresholds in the Human Subthalamic Nucleus. Current Biology 1–5, (2016). https://doi.org/10.1016/j.cub.2016.01.051

  14. Igarashi, J., Shounob, O., Fukai, T., Tsujino, H.: Real-time simulation of a spiking neural network model of the basal ganglia circuitry using general purpose computing on graphics processing units. Neural Networks 24, 950–960 (2011)

    Article  Google Scholar 

  15. Joel, D., Niv, Y., Ruppin, E.: Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Netw. 15, 535–547 (2002)

    Article  Google Scholar 

  16. Koprinkova-Hristova, P., Bocheva, N., Nedelcheva, S.: Investigation of feedback connections effect of a spike timing neural network model of early visual system. In: Innovations in Intelligent Systems and Applications (INISTA), 3–5 July 2018, Thessaloniki, Greece, (2018) https://doi.org/10.1109/INISTA.2018.8466292

  17. Koprinkova-Hristova, P., Bocheva, N., Nedelcheva, S., Stefanova, M.: A model of self-motion perception developed in NEST. Frontiers Comput. Neurosci. (2019). https://doi.org/10.3389/fncom.2019.00020

  18. Koprinkova-Hristova, P., et al.: STDP plasticity in TRN within hierarchical spike timing model of visual information processing, AIAI 2020. IFIP AICT 583, 1–12 (2020)

    Google Scholar 

  19. Koprinkova-Hristova, P., Bocheva, N.: Spike Timing Neural Model of Eye Movement Motor Response with Reinforcement Learning, BG SIAM 2018 (accepted paper for LNCS volume, to appear), (2020)

    Google Scholar 

  20. Kremkow, J., et al.: Push-Pull Receptive Field Organization and Synaptic Depression: Mechanisms for Reliably Encoding Naturalistic Stimuli in V1. Frontiers in Neural Circuits (2016). https://doi.org/10.3389/fncir.2016.00037

  21. Krishnan, R., Ratnadurai, S., Subramanian, D., Chakravarthy, V.S., Rengaswamyd, M.: Modeling the role of basal ganglia in saccade generation: Is the indirect pathway the explorer? Neural Networks 24, 801–813 (2011)

    Article  Google Scholar 

  22. Kunkel, S. et al.: NEST 2.12.0. Zenodo. (2017) https://doi.org/10.5281/zenodo.259534

  23. Layton, O.W., Fajen, B.R.: Possible role for recurrent interactions between expansion and contraction cells in MSTd during self-motion perception in dynamic environments. J. Vis. 17(5), 1–21 (2017)

    Article  Google Scholar 

  24. Nedelcheva, S., Koprinkova-Hristova, P.: Orientation selectivity tuning of a spike timing neural network model of the first layer of the human visual cortex. In: Georgiev, K., Todorov, M., Georgiev, I. (eds.) BGSIAM 2017. SCI, vol. 793, pp. 291–303. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-97277-0_24

    Chapter  Google Scholar 

  25. Plotkin, J.L., Goldberg, L.A.: Thinking Outside the Box (and Arrow): Current Themes in Striatal Dysfunction in Movement Disorders, The Neuroscientist. https://doi.org/10.1177/1073858418807887. (2018)

  26. Potjans, W., Morrison, A., Diesmann, M.: Enabling functional neural circuit simulations with distributed computing of neuromodulated plasticity. Front. Comp. Neurosci. 4, (2010). https://doi.org/10.3389/fncom.2010.00141many

  27. Rubin, J., Lee, D.D., Sompolinsky, H.: Equilibrium properties of temporally asymmetric Hebbian plasticity. Phys. Rev. Lett. 86(2), 364–367 (2001)

    Article  Google Scholar 

  28. Sadeh, S., Rotter, S.: Statistics and geometry of orientation selectivity in primary visual cortex. Biol. Cybern. 108, 631–653 (2014)

    Article  MATH  Google Scholar 

  29. Shadlen, M.N., Newsome, W.T.: Motion perception: seeing and deciding. Proc. Natl. Acad. Sci. USA 93(2), 628–633 (1996)

    Article  Google Scholar 

  30. Troyer, T.W., Krukowski, A.E., Priebe, N.J., Miller, K.D.: Contrast invariant orientation tuning in cat visual cortex: thalamocortical input tuning and correlation-based intracortical connectivity. J. Neurosci. 18, 5908–5927 (1998)

    Article  Google Scholar 

  31. Tsodyks, M., Uziel, A., Markram, H.: Synchrony generation in recurrent networks with frequency-dependent synapses. J. Neurosci. 20 RC50, 1–5 (2000)

    Google Scholar 

  32. Webb, B.S., Ledgeway, T.Y., McGraw, P.V.: Relating spatial and temporal orientation pooling to population decoding solutions in human vision. Vis. Res. 50, 2274–2283 (2010)

    Article  Google Scholar 

  33. Wei, W., Rubin, J.E., Wang, X.-J.: Role of the indirect pathway of the basal ganglia in perceptual decision making. J. Neurosci. 35(9), 4052–4064 (2015)

    Article  Google Scholar 

  34. Yan, H., Wang, J.: Quantification of motor network dynamics in Parkinson’s disease by means of landscape and flux theory. PLoS ONE 12(3), e0174364 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Information and Communication Technologies, Bulgarian Academy of Sciences, Sofia, Bulgaria

    Petia Koprinkova-Hristova

  2. Institute of Neurobiology, Bulgarian Academy of Sciences, Sofia, Bulgaria

    Nadejda Bocheva

Authors
  1. Petia Koprinkova-Hristova
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nadejda Bocheva
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Petia Koprinkova-Hristova .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. University of Reading, Reading, UK

    Varun Ojha

  3. University of Oxford, Oxford, UK

    Emanuele La Malfa

  4. University of Cambridge, Cambridge, UK

    Giorgio Jansen

  5. Almawave, Rome, Italy

    Vincenzo Sciacca

  6. University of Florida, Gainesville, FL, USA

    Panos Pardalos

  7. University of Catania, Catania, Italy

    Giovanni Giuffrida

  8. Harvard University, Cambridge, MA, USA

    Renato Umeton

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Koprinkova-Hristova, P., Bocheva, N. (2020). Brain-Inspired Spike Timing Model of Dynamic Visual Information Perception and Decision Making with STDP and Reinforcement Learning. In: Nicosia, G., et al. Machine Learning, Optimization, and Data Science. LOD 2020. Lecture Notes in Computer Science(), vol 12566. Springer, Cham. https://doi.org/10.1007/978-3-030-64580-9_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-64580-9_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-64579-3

  • Online ISBN: 978-3-030-64580-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation