Abstract
A central role in the development process of children is played by self-exploratory activities. Through a playful interaction with the surrounding environment, they test their own capabilities, explore novel situations, and understand how their actions affect the world. During this kind of exploration, interesting situations may be discovered. By learning to reach these situations, a child incrementally develops more and more complex skills. Inspired by studies from psychology, neuroscience, and machine learning, we designed SMILe (Self-Motivated Incremental Learning), a learning framework that allows artificial agents to autonomously identify and learn a set of abilities useful to face several different tasks, through an iterated three phase process: by means of a random exploration of the environment (babbling phase), the agent identifies interesting situations and generates an intrinsic motivation (motivating phase) aimed at learning how to get into these situations (skill acquisition phase). This process incrementally increases the skills of the agent, so that new interesting configurations can be experienced. We present results on two gridworld environments to show how SMILe makes it possible to learn skills that enable the agent to perform well and robustly in many different tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barto, A.G., Mahadevan, S.: Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems 13(4), 341–379 (2003)
Barto, A.G., Singh, S., Chentanez, N.: Intrinsically motivated learning of hierarchical collections of skills. In: Proceedings of ICDL (2004)
Berlyne, D.E.: Conflict, Arousal, and Curiosity. McGraw-Hill, New York (1960)
Bonarini, A., Lazaric, A., Restelli, M.: Smile: Self-motivated incremental learning. Technical report, Politecnico di Milano (2006), www.airlab.elet.polimi.it/papers/bonarini06smile.pdf
Kakade, S., Dayan, P.: Dopamine: Generalization and bonuses. Neural Networks 15, 549–559 (2002)
Konidaris, G.D., Hayes, G.M.: An architecture for behavior-based reinforcement learning. Adaptive Behavior 13(1), 5–32 (2005)
Lungarella, M., Metta, G., Pfeifer, R., Sandini, C.: Developmental robotics: a survey. Connection Science 15(4), 151–190 (2003)
Marshall, J., Blank, D., Meeden, L.: An emergent framework for self-motivation in developmental robotics. In: Proceedings of ICDL (2004)
McGovern, A., Barto, A.G.: Automatic discovery of subgoals in reinforcement learning using diverse density. In: Proceedings of ICML (2001)
Meltzoff, A., Moore, M.: Explaining facial imitation: a theoretical model. Early Development and Parenting 6, 179–192 (1997)
Menache, I., Mannor, S., Shimkin, N.: Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning. In: Elomaa, T., Mannila, H., Toivonen, H. (eds.) ECML 2002. LNCS, vol. 2430, p. 295. Springer, Heidelberg (2002)
Oudeyer, P.-Y., Kaplan, F., Hafner, V.: The playground experiment: Task-independent development of a curious robot. In: AAAI Spring Symposium Workshop on Developmental Robotics (2005)
Piaget, J.: The Origins of Intelligence in Children. Norton, New York (1952)
Ratitch, B., Precup, D.: Using mdp characteristics to guide exploration in reinforcement learning. In: European Conference on Reinforcement Learning (2003)
Schmidhuber, J.: Self-motivated development through rewards for predictor errors / improvements. In: AAAI Spring Symposium on Developmental Robotics (2005)
Stout, A., Konidaris, G., Barto, A.: Intrinsically motivated reinforcement learning: A promising framework for developmental robot learning. In: AAAI Spring Symposium on Developmental Robotics (2005)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Sutton, R.S., Precup, D., Singh, S.P.: Between mdps and semi-mdps: a framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112, 181–211 (1999)
Uchibe, E., Doya, K.: Reinforcement learning with multiple heterogeneous modules: A framework for developmental robot learning. In: Proceedings of ICDL (2005)
Weng, J., McClelland, A., Sporns, O., Stockman, I., Sur, M., Thelen, E.: Autonomous mental development by robots and animals. Science 291, 599–600 (2001)
Weng, J., Zhang, Y.: Novelty and reinforcement learning in the value system of developmental robots. In: International Workshop on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bonarini, A., Lazaric, A., Restelli, M. (2006). Incremental Skill Acquisition for Self-motivated Learning Animats. In: Nolfi, S., et al. From Animals to Animats 9. SAB 2006. Lecture Notes in Computer Science(), vol 4095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840541_30
Download citation
DOI: https://doi.org/10.1007/11840541_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38608-7
Online ISBN: 978-3-540-38615-5
eBook Packages: Computer ScienceComputer Science (R0)