Abstract
Match decision-making problem is one of the hot topics in the field of computational neuroscience. In this paper, we propose a spiking SHESN model with reward-modulated reinforcement learning so as to conduct computational modeling and prediction of such an open problem in a manner that has more neurophysiological characteristics. Neural coding of two sequentially-presented stimuli is read out from a collection of clustered neural populations in state reservoir through reward-modulated reinforcement learning. To evaluate match decision-making performance of our computational model, we set up three kinds of test datasets with different spike timing trains and present a criterion of maximum correlation coefficient for assessing whether match/nonmatch decision-making is successful or not. Finally, extensive experimental results show that the proposed model has strong robustness on interval of both spike timings and spike shift, which is consistent with monkey’s behavior records exhibited in match decision-making experiment [1].
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Qi, X.L., Meyer, T., Stanford, T.R., Constantinidis, C.: Neural correlates of a decision variable before learning to perform a match/nonmatch task. J. Neurosci. 32(18), 6161–6169 (2012)
Gerstner, W., Kistler, W.: Spiking Neuron Models: Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge (2002)
Caporale, N., Dan, Y.: Spike timing-dependent plasticity: a hebbian learning rule. Annu. Rev. Neurosci. 31, 25–46 (2008)
Wills, T.J., Lever, C., Cacucci, F., Burgess, N., O’Keefe, J.: Attractor dynamics in the hippocampal representation of the local environment. Science 308(5723), 873–876 (2005)
Jaeger, H.: The “echo state” approach to analysing and training recurrent neural networks - with an erratum note. German National Research Center for Information Technology GMD Technical report 48 (2001)
Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless telecommunication. Science 304(5667), 78–80 (2004)
Smolinski, T.G., Prinz, A.A.: Multi-objective evolutionary algorithms for model neuron parameter value selection matching biological behavior under different simulation scenarios. BMC Neurosci. 10(Suppl. 1), P260 (2009)
Krimm, R.F., Hill, D.L.: Neuron/target matching between chorda tympani neurons and taste buds during postnatal rat development. J. Neurobiol. 43(1), 98–106 (2000)
Deng, Z.D., Zhang, Y.: Collective behavior of a small-world recurrent neural system with scale-free distribution. IEEE Trans. Neural Networks 18(5), 1364–1375 (2007)
Jaeger, H., Lukoševičius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20(3), 335–352 (2007)
Legenstein, R., Chase, S.M., Schwartz, A.B., Maass, W.: A reward-modulated hebbian learning rule can explain experimentally observed network reorganization in a brain control task. J. Neurosci. 30(25), 8400–8410 (2010)
Dayan, P., Abbott, L.F.: Theoretical neuroscience: computational and mathematical modeling of neural systems. J. Cogn. Neurosci. 15(1), 154–155 (2003)
Acknowledgment
This work was supported in part by the National Science Foundation of China (NSFC) under Grant Nos. 91420106, 90820305, and 60775040.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Deng, Z., Yang, G. (2015). A Computational Model of Match Decision-Making Problem Using Spiking SHESN with Reward-Modulated Reinforcement Learning. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9489. Springer, Cham. https://doi.org/10.1007/978-3-319-26532-2_56
Download citation
DOI: https://doi.org/10.1007/978-3-319-26532-2_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26531-5
Online ISBN: 978-3-319-26532-2
eBook Packages: Computer ScienceComputer Science (R0)