Abstract
Taking inspiration from how the brain coordinates multiple learning systems is an appealing strategy to endow robots with more flexibility. One of the expected advantages would be for robots to autonomously switch to the least costly system when its performance is satisfying. However, to our knowledge no study on a real robot has yet shown that the measured computational cost is reduced while performance is maintained with such brain-inspired algorithms. We present navigation experiments involving paths of different lengths to the goal, dead-end, and non-stationarity (i.e., change in goal location and apparition of obstacles). We present a novel arbitration mechanism between learning systems that explicitly measures performance and cost. We find that the robot can adapt to environment changes by switching between learning systems so as to maintain a high performance. Moreover, when the task is stable, the robot also autonomously shifts to the least costly system, which leads to a drastic reduction in computation cost while keeping a high performance. Overall, these results illustrates the interest of using multiple learning systems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Meyer, J.-A., Guillot, A.: Biologically-inspired robots. In: Handbook of Robotics (B. Siciliano and O. Khatib, eds.), pp. 1395–1422. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-30301-5_61
Dollé, L., Khamassi, M., Girard, B., Guillot, A., Chavarriaga, R.: Analyzing interactions between navigation strategies using a computational model of action selection. In: International Conference on Spatial Cognition, pp. 71–86 (2008)
Caluwaerts, K., et al.: A biologically inspired meta-control navigation system for the Psikharpax rat robot. Bioinspiration Biomimetics 7, 025009 (2012)
Zambelli, M., Demiris, Y.: Online multimodal ensemble learning using self-learned sensorimotor representations. IEEE Trans. Cogn. Dev. Syst. 9(2), 113–126 (2016)
Banquet, J.-P., Hanoune, S., Gaussier, P., Quoy, M.: From cognitive to habit behavior during navigation, through cortical-basal ganglia loops. In: Villa, A.E.P., Masulli, P., Pons Rivero, A.J. (eds.) ICANN 2016. LNCS, vol. 9886, pp. 238–247. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-44778-0_28
Lowrey, K., Rajeswaran, A., Kakade, S., Todorov, E., Mordatch, I.: Plan online, learn offline: efficient learning and exploration via model-based control. In: International Conference on Learning Representations (2019)
Daw, N., Niv, Y., Dayan, P.: Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8(12), 1704–1711 (2005)
Khamassi, M., Humphries, M.: Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies. Front. Behav. Neurosci. 6, 79 (2012)
Renaudo, E., Girard, B., Chatila, R., Khamassi, M.: Respective advantages and disadvantages of model-based and model-free reinforcement learning in a robotics neuro-inspired cognitive architecture. In: Biologically Inspired Cognitive Architectures BICA 2015, (Lyon, France), pp. 178–184 (2015)
Renaudo, E., Girard, B., Chatila, R., Khamassi, M.: Which criteria for autonomously shifting between goal-directed and habitual behaviors in robots? In: 5th International Conference on Development and Learning and on Epigenetic Robotics (ICDL-EPIROB), pp. 254–260. (Providence, RI, USA) (2015)
Gat, E.: On three-layer architectures. In: Artificial Intelligence and Mobile Robots. MIT Press (1998)
Alami, R., Chatila, R., Fleury, S., Ghallab, M., Ingrand, F.: An architecture for autonomy. IJRR J. 17, 315–337 (1998)
Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, 1st edn. MIT Press, Cambridge (1998)
Viejo, G., Khamassi, M., Brovelli, A., Girard, B.: Modelling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning. Front. Behav. Neurosci. 9(225) (2015)
Powell, T., Sammut-Bonnici, T.: Pareto Analysis (2015)
Quigley, M., et al.: ROS: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)
Grisetti, G., Stachniss, C., Burgard, W.: Improved techniques for grid mapping with Rao-blackwellized particle filters. Trans. Rob. 23, 34–46 (2007)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Dromnelle, R., Girard, B., Renaudo, E., Chatila, R., Khamassi, M.: Coping with the variability in humans reward during simulated human-robot interactions through the coordination of multiple learning strategies. In: The 29th IEEE International Conference on Robot & Human Interactive Communication (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Dromnelle, R., Renaudo, E., Pourcel, G., Chatila, R., Girard, B., Khamassi, M. (2020). How to Reduce Computation Time While Sparing Performance During Robot Navigation? A Neuro-Inspired Architecture for Autonomous Shifting Between Model-Based and Model-Free Learning. In: Vouloutsi, V., Mura, A., Tauber, F., Speck, T., Prescott, T.J., Verschure, P.F.M.J. (eds) Biomimetic and Biohybrid Systems. Living Machines 2020. Lecture Notes in Computer Science(), vol 12413. Springer, Cham. https://doi.org/10.1007/978-3-030-64313-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-64313-3_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64312-6
Online ISBN: 978-3-030-64313-3
eBook Packages: Computer ScienceComputer Science (R0)