Skip to main content

An Adaptive Robot Motivational System

  • Conference paper
From Animals to Animats 9 (SAB 2006)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4095))

Included in the following conference series:

  • 1819 Accesses

Abstract

We present a robot motivational system design framework. The framework represents the underlying (possibly conflicting) goals of the robot as a set of drives, while ensuring comparable drive levels and providing a mechanism for drive priority adaptation during the robot’s lifetime. The resulting drive reward signals are compatible with existing reinforcement learning methods for balancing multiple reward functions. We illustrate the framework with an experiment that demonstrates some of its benefits.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Bersini, H.: Reinforcement learning for homeostatic endogenous variables. In: From Animals to Animats 3: Proceedings of the Third International Conference on the Simulation of Adaptive Behavior, pp. 325–333 (1994)

    Google Scholar 

  2. Blumberg, B.M., Todd, P.M., Maes, P.: No bad dogs: Ethological lessons for learning in hamsterdam. In: From Animals to Animats 4: Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior, pp. 295–304 (1996)

    Google Scholar 

  3. Brooks, R.A.: The role of learning in autonomous robots. In: Proceedings of the Fourth Annual Workshop on Computational Learning Theory (COLT 1991), pp. 5–10 (1991)

    Google Scholar 

  4. Cañamero, L.: Modeling motivations and emotions as a basis for intelligent behavior. In: Lewis Johnson, W. (ed.) Proceedings of the First International Conference on Autonomous Agents, pp. 148–155. ACM Press, New York (1997)

    Chapter  Google Scholar 

  5. Cos-Aguilera, I., Cañamero, L., Hayes, G.M.: Motivation-driven learning of object affordances: First experiments using a simulated khepera robot. In: Detjer, F., Dörner, D., Schaub, H. (eds.) The Logic of Cognitive Systems: Proceedings of the Fifth International Conference on Cognitive Modeling (ICCM 2003), pp. 57–62 (2003)

    Google Scholar 

  6. Doya, K., Uchibe, E.: The cyber rodent project: Exploration of adaptive mechanisms for self-preservation and self-reproduction. Adaptive Behavior 13(2), 149–160 (2005)

    Article  Google Scholar 

  7. Halperin, J.R.P.: Machine motivation. In: From Animals to Animats: Proceedings of the First International Conference on the Simulation of Adaptive Behavior, pp. 213–221 (1990)

    Google Scholar 

  8. Humphrys, M.: Action selection methods using reinforcement learning. In: From Animats to Animats 4: Proceedings of the Fourth International Conference on the Simulation of Adaptive Behavior, pp. 135–144 (1996)

    Google Scholar 

  9. Konidaris, G.D., Hayes, G.M.: Anticipatory learning for focusing search in reinforcement learning agents. In: The Second Workshop on Anticipatory Behavior in Adaptive Learning Systems (July 2004)

    Google Scholar 

  10. Konidaris, G.D., Hayes, G.M.: Estimating future reward in reinforcement learning animats using associative learning. In: From Animals to Animats 8: Proceedings of the 8th International Conference on the Simulation of Adaptive Behavior, pp. 297–304 (2004)

    Google Scholar 

  11. Konidaris, G.D., Hayes, G.M.: An architecture of behavior-based reinforcement learning. Adaptive Behavior 13(1), 5–32 (2005)

    Article  Google Scholar 

  12. McFarland, D., Bosser, T.: Intelligent Behavior in Animals and Robots. MIT Press, Cambridge (1994)

    Google Scholar 

  13. Savage, T.: Artificial motives: a review of motivation in artificial creatures. Connection Science 12(3/4), 211–277 (2000)

    Article  Google Scholar 

  14. Spier, E.: From Reactive Behaviour to Adaptive Behaviour: Motivational Models for Behaviour in Animals and Robots. PhD thesis, Balliol College, University of Oxford (1997)

    Google Scholar 

  15. Spier, E., McFarland, D.: Possibly optimal decision-making under self-sufficiency and autonomy. Journal of Theoretical Biology 189(3), 317–331 (1997)

    Article  Google Scholar 

  16. Sprague, N., Ballard, D.H.: Multiple-goal reinforcement learning with modular Sarsa(0). In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI 2003), pp. 1445–1447 (2003)

    Google Scholar 

  17. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)

    Google Scholar 

  18. Toates, F.: Motivational Systems, ch. 4. Cambridge University Press, Cambridge (1986)

    Google Scholar 

  19. Whitehead, S., Karlsson, J., Tenenberg, J.: Learning multiple goal behavior via task decomposition and dynamic policy merging. In: Connell, J.H., Mahadevan, S. (eds.) Robot Learning, pp. 45–78. Kluwer Academic Publishers, Dordrecht (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Konidaris, G., Barto, A. (2006). An Adaptive Robot Motivational System. In: Nolfi, S., et al. From Animals to Animats 9. SAB 2006. Lecture Notes in Computer Science(), vol 4095. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840541_29

Download citation

  • DOI: https://doi.org/10.1007/11840541_29

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-38608-7

  • Online ISBN: 978-3-540-38615-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics