Abstract
Goal-driven agents are generally expected to be capable of pursuing simultaneously a variety of goals. As these goals may compete in certain circumstances, the agent must be able to constantly trade them off and shift their priorities in a rational way. One aspect of rationality is to evaluate its needs and make decisions accordingly. We endow the agent with a set of needs, or drives, that change over time as a function of external stimuli and internal consumption, and the decision making process hast to generate actions that maintain balance between these needs. The proposed framework pursues an approach in which decision making is considered as a multiobjective problem and approximately solved using a hierarchical reinforcement learning architecture. At a higher-level, a Q-learning learns to select the best learning strategy that improves the well-being of the agent. At a lower-level, an actor-critic design executes the selected strategy while interacting with a continuous, partially observable environment. We provide simulation results to demonstrate the efficiency of the approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Ram, A., Leake, D.: Goal-Driven Learning. MIT Press (1995)
Choi, D.: Reactive goal management in a cognitive architecture. Cognitive Systems Research 12(3-4), 293–308 (2011)
Jaidee, U., Munoz-Avila, H., Aha, D.: Integrated learning for goal-driven autonomy. In: IJCAI, pp. 2450–2455 (2011)
Zilberstein, S.: Metareasoning and Bounded Rationality. In: AAAI Workshop on Metareasoning: Thinking about Thinking (2008)
da Costa Pereira, C., Tettamanzi, A.: An integrated possibilistic framework for goal generation in cognitive agents. In: AAMAS, International Foundation for Autonomous Agents and Multiagent Systems, pp. 1239–1246 (2010)
Michalski, R.: Inferential Theory of Learning: Developing Foundations for Multistrategy Learning. In: Machine Learning, A Multistrategy Approach. Morgan K (1994)
Konidaris, G., Barto, A.: An adaptive robot motivational system. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 346–356. Springer, Heidelberg (2006)
Dayan, P.: Goal-directed control and its antipodes. Neural Networks 22(3), 213–219 (2009)
Dezfouli, Balleine: Actions, action sequences and habits: Evidence that goal-directed and habitual action control are hierarchically organized. PLoS Comp. Biol. 9(12) (2013)
Butz, M., Shirinov, E., Reif, K.: Self-organizing sensorimotor maps plus internal motivations yield animal-like behavior. Adaptive Behaviour 18(3-4), 315–337 (2010)
Sutton, R., Precup, D., Singh, S.: Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artif. Intel. 112(1-2), 181–211 (1999)
Salichs, M., Malfaz, M.: A new approach to modeling emotions and their use on a decision-making system for artificial agents. IEEE Trans. Affect. Comput. 3(1), 56–68 (2012)
Cos-Aguilera, I., Canamero, L., Hayes, G., Gillies, A.: Hedonic value: Enhancing adaptation for motivated agents. Adaptive Behaviour 21(6), 465–483 (2013)
Oubbati, M., Kord, B., Koprinkova-Hristova, P., Palm, G.: Learning of embodied interaction dynamics with recurrent neural networks: some exploratory experiments. Journal of Neural Engineering 11(2), 026019 (2014)
Bellman, R.E.: Dynamic Programming. Princeton Univ. Press, NJ (1957)
Deb, K.: Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evolutionary Computation 7(3), 205–230 (1999)
Prokhorov, D., Wunsch, D.: Adaptive critic designs. IEEE Transactions on Neural Networks 8, 997–1007 (1997)
Jaeger, H.: The ’echo state’ approach to analysing and training recurrent neural networks. Technical Report 148, AIS Fraunhofer, St. Augustin, Germany (2001)
Parisi, D.: Internal robotics. Connection Science 16, 325–338 (2004)
Konidaris, G.D., Hayes, G.M.: An architecture for behavior-based reinforcement learning. Adaptive Behavior 13(1), 5–32 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Oubbati, M., Fischer, C., Palm, G. (2014). Intrinsically Motivated Decision Making for Situated, Goal-Driven Agents. In: del Pobil, A.P., Chinellato, E., Martinez-Martin, E., Hallam, J., Cervera, E., Morales, A. (eds) From Animals to Animats 13. SAB 2014. Lecture Notes in Computer Science(), vol 8575. Springer, Cham. https://doi.org/10.1007/978-3-319-08864-8_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-08864-8_16
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-08863-1
Online ISBN: 978-3-319-08864-8
eBook Packages: Computer ScienceComputer Science (R0)