Abstract
This work explores the effects of the introduction of variational autoencoder based representation learning, and of its resulting latent spaces, within a robotic cognitive architecture to be able to efficiently learn models and policies when raw perceptual dimensionality is very high. The main focus of the paper is on the decision processes of the robots used for action selection. To this end we propose a procedure to obtain from autonomously produced latent state spaces the world and utility models necessary for deliberative operation as a first type of decision process. Additionally, we present a neuroevolutionary based approach to generate policies, for reactive operation, based on the information of the latent state space and using the previously obtained world and utility models to permit offline learning. A set of experiments over a real robot using vision, with the consequent high dimensional raw perceptual space, are carried out in order to validate the proposal.
This work has been partially funded by the Ministerio de Ciencia, Innovación y Universidades of Spain/FEDER (grant RTI2018-101114-B-I00), Xunta de Galicia (EDC431C-2021/39) and the Centro de Investigación de Galicia “CITIC”, funded by Xunta de Galicia and the European Union (European Regional Development Fund- Galicia 2014-2020 Program), by grant ED431G 2019/01, and by the Spanish Ministry of Education, Culture and Sports for the FPU grant of Alejandro Romero.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alvernaz, S., Togelius, J.: Autoencoder-augmented neuroevolution for visual doom playing. In: 2017 IEEE Conference on Computational Intelligence and Games (CIG), pp. 1–8. IEEE (2017)
Bellas, F., Duro, R.J., Faiña, A., Souto, D.: Multilevel Darwinist brain (MDB): artificial evolution in a cognitive architecture for real robots. IEEE Trans. Auton. Ment. Dev. 2(4), 340–354 (2010)
Deisenroth, M.P., Neumann, G., Peters, J., et al.: A survey on policy search for robotics. Found. Trends Robot. 2(1–2), 388–403 (2013)
Doncieux, S., et al.: Open-ended learning: a conceptual framework based on representational redescription. Front. Neurorobot. 59 (2018)
Grondman, I., Busoniu, L., Lopes, G.A., Babuska, R.: A survey of actor-critic reinforcement learning: standard and natural policy gradients. IEEE Trans. Syst. Man Cybern. Part C 42(6), 1291–1307 (2012)
Karl, M., Soelch, M., Bayer, J., van der Smagt, P.: Deep variational bayes filters: unsupervised learning of state space models from raw data. arXiv:1605.06432 (2016)
Kim, S., Coninx, A., Doncieux, S.: From exploration to control: learning object manipulation skills through novelty search and local adaptation. Robot. Auton. Syst. 136, 103710 (2021)
Kotseruba, I., Tsotsos, J.K.: A review of 40 years of cognitive architecture research: Core cognitive abilities and practical applications. arXiv:1610.08602 (2016)
Lesort, T., Díaz-Rodríguez, N., Goudou, J.F., Filliat, D.: State representation learning for control: an overview. Neural Netw. 108, 379–392 (2018)
Meden, B., Prieto, A., Peer, P., Bellas, F.: First steps towards state representation learning for cognitive robotics. In: International Conference on HAIS, pp. 499–510 (2020)
Plaat, A., Kosters, W., Preuss, M.: Model-based deep reinforcement learning for high-dimensional problems, a survey. arXiv preprint arXiv:2008.05598 (2020)
Romero, A., Bellas, F., Becerra, J.A., Duro, R.J.: Motivation as a tool for designing lifelong learning robots. Int. Comput.-Aid. Eng. 27(4), 353–372 (2020)
Romero, A., Prieto, A., Bellas, F., Duro, R.J.: Simplifying the creation and management of utility models in continuous domains for cognitive robotics. Neurocomputing 353, 106–118 (2019)
Zhang, A., Satija, H., Pineau, J.: Decoupling dynamics and reward for transfer learning. arXiv:1804.10689 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Romero, A., Meden, B., Bellas, F., Duro, R.J. (2022). Autonomous Knowledge Representation for Efficient Skill Learning in Cognitive Robots. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds) Bio-inspired Systems and Applications: from Robotics to Ambient Intelligence. IWINAC 2022. Lecture Notes in Computer Science, vol 13259. Springer, Cham. https://doi.org/10.1007/978-3-031-06527-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-06527-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06526-2
Online ISBN: 978-3-031-06527-9
eBook Packages: Computer ScienceComputer Science (R0)