Abstract
A biologically inspired computational model of rodent repre-sentation–based (locale) navigation is presented. The model combines visual input in the form of realistic two dimensional grey-scale images and odometer signals to drive the firing of simulated place and head direction cells via Hebbian synapses. The space representation is built incrementally and on-line without any prior information about the environment and consists of a large population of location-sensitive units (place cells) with overlapping receptive fields. Goal navigation is performed using reinforcement learning in continuous state and action spaces, where the state space is represented by population activity of the place cells. The model is able to reproduce a number of behavioral and neuro-physiological data on rodents. Performance of the model was tested on both simulated and real mobile Khepera robots in a set of behavioral tasks and is comparable to the performance of animals in similar tasks.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Franz, M.O., Mallot, H.A.: Biomimetic robot navigation. Robotics and Autonomous Systems 30, 133–153 (2000)
Jeffery, K.J. (ed.): The neurobiology of spatial behavior. Oxford University Press, Oxford (2003)
Sutton, R., Barto, A.G.: Reinforcement Learning - An Introduction. MIT Press, Cambridge (1998)
O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat. Brain Research 34, 171–175 (1971)
Wilson, M.A., McNaughton, B.L.: Dynamics of the hippocampal ensemble code for space. Science 261, 1055–1058 (1993)
Taube, J.S., Muller, R.I., Ranck Jr., J.B.: Head direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis. Journal of Neuroscience 10, 420–435 (1990)
Knierim, J.J., Kudrimoti, H.S., McNaughton, B.L.: Place cells, head direction cells, and the learning of landmark stability. Journal of Neuroscience 15, 1648–1659 (1995)
Muller, R.U., Kubie, J.L.: The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells. Journal of Neuroscience 7, 1951–1968 (1987)
McNaughton, B.L., Barnes, C.A., Gerrard, J.L., Gothard, K., Jung, M.W., Knierim, J.J., Kudrimoti, H., Qin, Y., Skaggs, W.E., Suster, M., Weaver, K.L.: Deciphering the hippocampal polyglot: the hippocampus as a path integration system. J. Exp. Biol. 199, 173–185 (1996)
Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)
Schultz, W.: Predictive Reward Signal of Dopamine Neurons. Journal of Neurophysiology 80, 1–27 (1998)
Freund, T.F., Powell, J.F., Smith, A.D.: Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13, 1189–1215 (1984)
Sesack, S.R., Pickel, V.M.: In the rat medial nucleus accumbens, hippocampal and catecholaminergic terminals converge on spiny neurons and are in apposition to each other. Brain Res. 527, 266–279 (1990)
Eichenbaum, H., Stewart, C., Morris, R.G.M.: Hippocampal representation in place learning. Journal of Neuroscience 10(11), 3531–3542 (1990)
Sutherland, R.J., Rodriguez, A.J.: The role of the fornix/fimbria and some related subcortical structures in place learning and memory. Behavioral and Brain Research 32, 265–277 (1990)
Redish, A.D.: Beyond the Cognitive Map, From Place Cells to Episodic Memory. MIT Press-Bradford Books, London (1999)
Morris, R.G.M.: Spatial localization does not require the presence of local cues. Learning and Motivation 12, 239–260 (1981)
Packard, M.G., McGaugh, J.L.: Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: Further evidence for multiple memory systems. Behavioral Neuroscience 106(3), 439–446 (1992)
Trullier, O., Wiener, S.I., Berthoz, A., Meyer, J.A.: Biologically-based artificial navigation systems: Review and prospects. Progress in Neurobiology 51, 483–544 (1997)
Recce, M., Harris, K.D.: Memory for places: A navigational model in support of Marr’s theory of hippocampal function. Hippocampus 6, 85–123 (1996)
Burgess, N., Donnett, J.G., Jeffery, K.J., O’Keefe, J.: Robotic and neuronal simulation of the hippocampus and rat navigation. Phil. Trans. R. Soc. Lond. B 352, 1535–1543 (1997)
Burgess, N., Jackson, A., Hartley, T., O’Keefe, J.: Predictions derived from modelling the hippocampal role in navigation. Biol. Cybern. 83, 301–312 (2000)
Burgess, N., Recce, M., O’Keefe, J.: A model of hippocampal function. Neural Networks 7, 1065–1081 (1994)
O’Keefe, J., Burgess, N.: Geometric determinants of the place fields of hippocampal neurons. Nature 381, 425–428 (1996)
Georgopoulos, A.P., Kettner, R.E., Schwartz, A.: Primate motor cortex and free arm movements to visual targets in three-dimensional space. II. Coding of the direction of movement by a neuronal population. Neuroscience 8, 2928–2937 (1988)
Gaussier, P., Leprêtre, S., Joulain, C., Revel, A., Quoy, M., Banquet, J.P.: Animal and robot learning: Experiments and models about visual navigation. In: 7th European Workshop on Learning Robots, Edinburgh, UK (1998)
Gaussier, P., Joulain, C., Banquet, J.P., Leprêtre, S., Revel, A.: The visual homing problem: An example of robotics/biology cross fertilization. Robotics and Autonomous Systems 30, 155–180 (2000)
Gaussier, P., Revel, A., Banquet, J.P., Babeau, V.: From view cells and place cells to cognitive map learning: processing stages of the hippocampal system. Biol. Cybern. 86, 15–28 (2002)
Arleo, A., Gerstner, W.: Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity. Biological Cybernetics, Special Issue on Navigation in Biological and Artificial Systems 83, 287–299 (2000)
Arleo, A., Smeraldi, F., Hug, S., Gerstner, W.: Place cells and spatial navigation based on 2d visual feature extraction, path integration, and reinforcement learning. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems 13, pp. 89–95. MIT Press, Cambridge (2001)
Arleo, A., Smeraldi, F., Gerstner, W.: Cognitive navigation based on nonuniform gabor space sampling, unsupervised growing networks, and reinforcement learning. IEEE Transactions on Neural Networks 15, 639–652 (2004)
Zhang, K.: Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: A theory. Journal of Neuroscience 16(6), 2112–2126 (1996)
Arleo, A., Gerstner, W.: Spatial orientation in navigating agents: Modeling head-direction cells. Neurocomputing 38–40, 1059–1065 (2001)
Skaggs, W.E., Knierim, J.J., Kudrimoti, H.S., McNaughton, B.L.: A model of the neural basis of the rat’s sense of direction. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems 7, pp. 173–180. MIT Press, Cambridge (1995)
Etienne, A.S., Jeffery, K.J.: Path integration in mammals. Hippocampus (2004)
Daugman, J.G.: Two-dimensional spectral analysis of cortical receptive field profiles. Vision Research 20, 847–856 (1980)
Pouget, A., Dayan, P., Zemel, R.S.: Inference and computation with population codes. Annu. Rev. Neurosci. 26, 381–410 (2003)
Strösslin, T., Gerstner, W.: Reinforcement learning in continuous state and action space. In: Kaynak, O., Alpaydın, E., Oja, E., Xu, L. (eds.) ICANN 2003 and ICONIP 2003. LNCS, vol. 2714. Springer, Heidelberg (2003)
Foster, D.J., Morris, R.G.M., Dayan, P.: A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10(1), 1–16 (2000)
Doya, K.: Reinforcement learning in continuous time and space. Neural Computation 12, 219–245 (2000)
Tolman, E.C.: Cognitive maps in rats and men. Psychological Review 55, 189–208 (1948)
Quirk, G.J., Muller, R.U., Kubie, J.L.: The firing of hippocampal place cells in the dark depends on the rat’s recent experience. Journal of Neuroscience 10, 2008–2017 (1990)
Morris, R.G.M., Garrud, P., Rawlins, J.N.P., O’Keefe, J.: Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683 (1982)
Hughes, A.: The topography of vision in mammals of contrasting life style: Comparative optics and retinal organisation. In: Crescitelli, F. (ed.) The Visual System in Vertebrates. Handbook of Sensory Physiology, vol. 7/5, pp. 613–756. Springer, Heidelberg (1977)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Sheynikhovich, D., Chavarriaga, R., Strösslin, T., Gerstner, W. (2005). Spatial Representation and Navigation in a Bio-inspired Robot. In: Wermter, S., Palm, G., Elshaw, M. (eds) Biomimetic Neural Learning for Intelligent Robots. Lecture Notes in Computer Science(), vol 3575. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11521082_15
Download citation
DOI: https://doi.org/10.1007/11521082_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27440-7
Online ISBN: 978-3-540-31896-5
eBook Packages: Computer ScienceComputer Science (R0)