Abstract
We present a robot motivational system design framework. The framework represents the underlying (possibly conflicting) goals of the robot as a set of drives, while ensuring comparable drive levels and providing a mechanism for drive priority adaptation during the robot’s lifetime. The resulting drive reward signals are compatible with existing reinforcement learning methods for balancing multiple reward functions. We illustrate the framework with an experiment that demonstrates some of its benefits.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bersini, H.: Reinforcement learning for homeostatic endogenous variables. In: From Animals to Animats 3: Proceedings of the Third International Conference on the Simulation of Adaptive Behavior, pp. 325–333 (1994)
Blumberg, B.M., Todd, P.M., Maes, P.: No bad dogs: Ethological lessons for learning in hamsterdam. In: From Animals to Animats 4: Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior, pp. 295–304 (1996)
Brooks, R.A.: The role of learning in autonomous robots. In: Proceedings of the Fourth Annual Workshop on Computational Learning Theory (COLT 1991), pp. 5–10 (1991)
Cañamero, L.: Modeling motivations and emotions as a basis for intelligent behavior. In: Lewis Johnson, W. (ed.) Proceedings of the First International Conference on Autonomous Agents, pp. 148–155. ACM Press, New York (1997)
Cos-Aguilera, I., Cañamero, L., Hayes, G.M.: Motivation-driven learning of object affordances: First experiments using a simulated khepera robot. In: Detjer, F., Dörner, D., Schaub, H. (eds.) The Logic of Cognitive Systems: Proceedings of the Fifth International Conference on Cognitive Modeling (ICCM 2003), pp. 57–62 (2003)
Doya, K., Uchibe, E.: The cyber rodent project: Exploration of adaptive mechanisms for self-preservation and self-reproduction. Adaptive Behavior 13(2), 149–160 (2005)
Halperin, J.R.P.: Machine motivation. In: From Animals to Animats: Proceedings of the First International Conference on the Simulation of Adaptive Behavior, pp. 213–221 (1990)
Humphrys, M.: Action selection methods using reinforcement learning. In: From Animats to Animats 4: Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior, pp. 135–144 (1996)
Konidaris, G.D., Hayes, G.M.: Anticipatory learning for focusing search in reinforcement learning agents. In: The Second Workshop on Anticipatory Behavior in Adaptive Learning Systems (July 2004)
Konidaris, G.D., Hayes, G.M.: Estimating future reward in reinforcement learning animats using associative learning. In: From Animals to Animats 8: Proceedings of the 8th International Conference on the Simulation of Adaptive Behavior, pp. 297–304 (2004)
Konidaris, G.D., Hayes, G.M.: An architecture of behavior-based reinforcement learning. Adaptive Behavior 13(1), 5–32 (2005)
McFarland, D., Bosser, T.: Intelligent Behavior in Animals and Robots. MIT Press, Cambridge (1994)
Savage, T.: Artificial motives: a review of motivation in artificial creatures. Connection Science 12(3/4), 211–277 (2000)
Spier, E.: From Reactive Behaviour to Adaptive Behaviour: Motivational Models for Behaviour in Animals and Robots. PhD thesis, Balliol College, University of Oxford (1997)
Spier, E., McFarland, D.: Possibly optimal decision-making under self-sufficiency and autonomy. Journal of Theoretical Biology 189(3), 317–331 (1997)
Sprague, N., Ballard, D.H.: Multiple-goal reinforcement learning with modular Sarsa(0). In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI 2003), pp. 1445–1447 (2003)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Toates, F.: Motivational Systems, ch. 4. Cambridge University Press, Cambridge (1986)
Whitehead, S., Karlsson, J., Tenenberg, J.: Learning multiple goal behavior via task decomposition and dynamic policy merging. In: Connell, J.H., Mahadevan, S. (eds.) Robot Learning, pp. 45–78. Kluwer Academic Publishers, Dordrecht (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Konidaris, G., Barto, A. (2006). An Adaptive Robot Motivational System. In: Nolfi, S., et al. From Animals to Animats 9. SAB 2006. Lecture Notes in Computer Science(), vol 4095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840541_29
Download citation
DOI: https://doi.org/10.1007/11840541_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38608-7
Online ISBN: 978-3-540-38615-5
eBook Packages: Computer ScienceComputer Science (R0)