Abstract
Flexible planning is necessary for reaching goals and adapting when conditions change. We introduce a biologically plausible path planning model that learns its environment, rapidly adapts to change, and plans efficient routes to goals. Our model addresses the decision-making process when faced with uncertainty. We tested the model in simulations of human and rodent navigation in mazes. Like the human and rat, the model was able to generate novel shortcuts, and take detours when familiar routes were blocked. Similar to rodent hippocampus recordings, the neural activity of the model resembles neural correlates of Vicarious Trial and Error (VTE) during early learning or during uncertain conditions and preplay predicting a future path after learning. We suggest that VTE, in addition to weighing possible outcomes, is a way in which an agent may gather information for future use.
Supported by the Air Force Office of Scientific Research (AFOSR) Contract No. FA9550-19-1-0306, by the National Science Foundation (NSF-FO award ID 2024633), and by the United States Air Force Research Laboratory (AFRL) and Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-18-C-0103.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alvernhe, A., Save, E., Poucet, B.: Local remapping of place cell firing in the Tolman detour task. Eur. J. Neurosci. 33(9), 1696–705 (2011). https://doi.org/10.1111/j.1460-9568.2011.07653.x
Bellec, G., et al.: A solution to the learning dilemma for recurrent networks of spiking neurons. Nat. Commun. 11(1), 3625 (2020). https://doi.org/10.1038/s41467-020-17236-y
Boone, A.P., Maghen, B., Hegarty, M.: Instructions matter: individual differences in navigation strategy and ability. Mem. Cogn. 47(7), 1401–1414 (2019). https://doi.org/10.3758/s13421-019-00941-5
Dragoi, G., Tonegawa, S.: Preplay of future place cell sequences by hippocampal cellular assemblies. Nature 469(7330), 397–401 (2011). https://doi.org/10.1038/nature09633
Fields, R.D.: A new mechanism of nervous system plasticity: activity-dependent myelination. Nat. Rev. Neurosci. 16(12), 756–67 (2015). https://doi.org/10.1038/nrn4023
Fischl, K.D., Fair, K., Tsai, W., Sampson, J., Andreou, A.: Path planning on the truenorth neurosynaptic system. In: 2017 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–4 (2017). https://doi.org/10.1109/ISCAS.2017.8050932
Hwu, T., Wang, A.Y., Oros, N., Krichmar, J.L.: Adaptive robot path planning using a spiking neuron algorithm with axonal delays. IEEE Trans. Cogn. Develop. Syst. 10(2), 126–137 (2018). https://doi.org/10.1109/Tcds.2017.2655539
Johnson, A., Redish, A.D.: Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27(45), 12176–89 (2007). https://doi.org/10.1523/JNEUROSCI.3761-07.2007
Kahn, G., Abbeel, P., Levine, S.: BADGR: an autonomous self-supervised learning-based navigation system. arXiv:2002.05700 (2020)
Krichmar, J.L., He, C.: Importance of path planning variability: a simulation study. Top. Cogn. Sci. (2021). https://doi.org/10.1111/tops.12568
Mattar, M.G., Daw, N.D.: Prioritized memory access explains planning and hippocampal replay. Nat. Neurosci. 21(11), 1609–1617 (2018). https://doi.org/10.1038/s41593-018-0232-z
Miller, K.J., Venditto, S.J.C.: Multi-step planning in the brain. Curr. Opin. Behav. Sci. 38, 29–39 (2021). https://doi.org/10.1016/j.cobeha.2020.07.003
Pfeiffer, B.E., Foster, D.J.: Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497(7447), 74–9 (2013). https://doi.org/10.1038/nature12112
Redish, A.D.: Vicarious trial and error. Nat. Rev. Neurosci. 17(3), 147–59 (2016). https://doi.org/10.1038/nrn.2015.30
Stachenfeld, K.L., Botvinick, M.M., Gershman, S.J.: The hippocampus as a predictive map. Nat. Neurosci. 20(11), 1643–1653 (2017). https://doi.org/10.1038/nn.4650
Tolman, E.C.: Cognitive maps in rats and men. Psychol. Rev. 55(4), 189–208 (1948). https://doi.org/10.1037/H0061626
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Krichmar, J.L., Ketz, N.A., Pilly, P.K., Soltoggio, A. (2022). Flexible Path Planning in a Spiking Model of Replay and Vicarious Trial and Error. In: Cañamero, L., Gaussier, P., Wilson, M., Boucenna, S., Cuperlier, N. (eds) From Animals to Animats 16. SAB 2022. Lecture Notes in Computer Science(), vol 13499. Springer, Cham. https://doi.org/10.1007/978-3-031-16770-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-16770-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16769-0
Online ISBN: 978-3-031-16770-6
eBook Packages: Computer ScienceComputer Science (R0)