Abstract
This paper evaluates two new strategies for investigating artificial animals called animats. Animats are homeostatic agents with the objective of keeping their internal variables as close to optimal as possible. Steps towards the optimal are rewarded and steps away punished. Using reinforcement learning for exploration and decision making, the animats can consider predetermined optimal/acceptable levels in light of current levels, giving them greater flexibility for exploration and better survival chances. This paper considers the resulting strategies as evaluated in a range of environments, showing them to outperform common reinforcement learning, where internal variables are not taken into consideration.
Research supported by the Torsten Söderberg Foundation Ö110/17.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bersini, H.: Reinforcement learning for homeostatic endogenous variables. In: From Animals to Animats 3: Proceedings of the Third International Conference on the Simulation of Adaptive Behavior, pp. 325–333 (1994)
Davies, K.J.: Adaptive homeostasis. Mol. Asp. Med. 49, 1–7 (2016). Hormetic and regulatory effects of lipid oxidation products
Keramati, M., Gutkin, B.S.: A reinforcement learning theory for homeostatic regulation. In: Advances in Neural Information Processing Systems, pp. 82–90 (2011)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014)
Kompella, V.R., Kazerounian, S., Schmidhuber, J.: An anti-hebbian learning rule to represent drive motivations for reinforcement learning. In: del Pobil, A.P., Chinellato, E., Martinez-Martin, E., Hallam, J., Cervera, E., Morales, A. (eds.) SAB 2014. LNCS (LNAI), vol. 8575, pp. 176–187. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08864-8_17
Konidaris, G., Barto, A.: An adaptive robot motivational system. In: Nolfi, S., Baldassarre, G., Calabretta, R., Hallam, J.C.T., Marocco, D., Meyer, J.-A., Miglino, O., Parisi, D. (eds.) SAB 2006. LNCS (LNAI), vol. 4095, pp. 346–356. Springer, Heidelberg (2006). https://doi.org/10.1007/11840541_29
Konidaris, G.D., Hayes, G.M.: An architecture for behavior-based reinforcement learning. Adapt. Behav. 13(1), 5–32 (2005)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)
Oubbati, M., Fischer, C., Palm, G.: Intrinsically motivated decision making for situated, goal-driven agents. In: del Pobil, A.P., Chinellato, E., Martinez-Martin, E., Hallam, J., Cervera, E., Morales, A. (eds.) SAB 2014. LNCS (LNAI), vol. 8575, pp. 166–175. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-08864-8_16
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (2018)
Wilson, S.W.: Knowledge growth in an artificial animal. In: Narendra, K.S. (ed.) Adaptive and Learning Systems, pp. 255–264. Springer, Boston (1986). https://doi.org/10.1007/978-1-4757-1895-9_18
Wilson, S.W.: The animat path to AI. In: Meyer, J.A., Wilson, S.W. (eds.) From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, pp. 15–21. MIT Press (1991)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Andersson, P., Strandman, A., Strannegård, C. (2019). Exploration Strategies for Homeostatic Agents. In: Hammer, P., Agrawal, P., Goertzel, B., Iklé, M. (eds) Artificial General Intelligence. AGI 2019. Lecture Notes in Computer Science(), vol 11654. Springer, Cham. https://doi.org/10.1007/978-3-030-27005-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-27005-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27004-9
Online ISBN: 978-3-030-27005-6
eBook Packages: Computer ScienceComputer Science (R0)