Skip to main content

Advertisement

Springer Nature Link
Log in

Towards intrinsic autonomy through evolutionary computation

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

This paper presents an embodied open-ended environment driven evolutionary algorithm capable of evolving behaviors of autonomous agents without any explicit description of objectives, evaluation metrics or cooperative dynamics. The main novelty of our technique is obtaining intrinsically motivated autonomy of a virtual robot in continuous activity, by internalizing evolutionary dynamics in order to achieve adaptation of a neural controller, and with no need to frequently restart the agent’s initial conditions as in traditional training methods. Our work is grounded on ideas from the enactive artificial intelligence field and the biological concept of enaction, from which it is argued that what makes a living being “intentional” is the ability to autonomously, adaptively rearrange their microstructure to suit some function in order to maintain its own constitution. We bring an alternative understanding of intrinsic motivation than that traditionally approached by intrinsic motivated reinforcement learning, and so we also make a brief discussion of the relationship between both paradigms and the autonomy of an agent. We show the autonomous development of foraging and navigation behaviors of a virtual robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://irrlicht.sourceforge.net/.

  2. http://bulletphysics.org/.

  3. Watch videos of other runs in https://www.youtube.com/watch?v=X8DGK9ZIULA and https://www.youtube.com/watch?v=Kil_MaAp64s.

References

  • Baldassarre G (2011) What are intrinsic motivations? A biological perspective. In: 2011 IEEE international conference on development and learning (ICDL), vol 2, pp 1–8

  • Baldassarre G, Mirolli M (eds) (2013) Intrinsically motivated learning in natural and artificial systems. Springer, Berlin

    Google Scholar 

  • Barandiaran X, Moreno A (2008) Adaptivity: from metabolism to behavior. Adapt Behav Anim Animats Softw Agents Robots Adapt Syst 16(5):325–344

    Google Scholar 

  • Barandiaran XE, Di Paolo E, Rohde M (2009) Defining agency: individuality, normativity, asymmetry, and spatio-temporality in action. Adapt Behav Anim Animats Softw Agents Robots Adapt Syst 17(5):367–386

    Google Scholar 

  • Barto AG (2013) Intrinsic motivation and reinforcement learning. In: Baldassarre G, Mirolli M (eds) Intrinsically motivated learning in natural and artificial systems. Springer, Berlin, pp 17–47

    Chapter  Google Scholar 

  • Barto AG, Singh S, Chentanez N (2004) Intrinsically motivated learning of hierarchical collections of skills. In: International conference on developmental learning

  • Beer RD (1995) On the dynamics of small continuous-time recurrent neural networks. Adapt Behav 3(4):469–509

    Article  Google Scholar 

  • Bredeche N, Montanier J (2010) Environment-driven embodied evolution in a population of autonomous agents. In: Parallel problem solving from nature, pp 290–299

  • Bredeche N, Haasdijk E, Eiben A (2009) On-line, on-board evolution of robot controllers. In: Evolution artificielle/artificial evolution, Strasbourg. https://hal.inria.fr/inria-00413259

  • Bredeche N, Montanier J, Liu W, Winfield A (2012) Environment-driven distributed evolutionary adaptation in a population of autonomous robotic agents. Math Comput Model Dyn Syst 18(1):101–129

    Article  Google Scholar 

  • Bredeche N, Haasdijk E, Prieto A (2018) Embodied evolution in collective robotics: a review. Front Robot AI 5:12. https://doi.org/10.3389/frobt.2018.00012

    Article  Google Scholar 

  • Dürr P, Mattiussi C, Floreano D (2006) Neuroevolution with analog genetic encoding. In: Runarsson TP, Beyer HG, Burke E, Merelo-Guervós JJ, Whitley LD, Yao X (eds) Parallel problem solving from nature—PPSN IX. Springer, Berlin, pp 671–680

    Chapter  Google Scholar 

  • Egbert MD, Barandiaran XE (2011) Quantifying normative behaviour and precariousness in adaptive agency. In: Advances in artificial life, proceedings of the 11th European conference on artificial life, ECAL 11 pp 210–218

  • Eiben A, Haasdijk E, Bredeche N (2010) Embodied, on-line, on-board evolution for autonomous robotics. In: Levi SKEP (ed) Symbiotic multi-robot organisms: reliability, adaptability, evolution. Cognitive systems monographs, vol 7. Springer, Berlin, pp 361–382. https://hal.inria.fr/inria-00531455

  • Elfwing S, Uchibe E, Doya K, Christensen HI (2011) Darwinian embodied evolution of the learning ability for survival. Adapt Behav Anim Animats Softw Agents Robots Adapt Syst 19(2):101–120

    Google Scholar 

  • Fitch WT (2008) Nano-intentionality: a defense of intrinsic intentionality. Biol Philos 23(2):157–177

    Article  Google Scholar 

  • Froese T, Ziemke T (2009) Enactive artificial intelligence: investigating the systemic organization of life and mind. Artif Intell 173(3–4):466–500

    Article  Google Scholar 

  • Froese T, Virgo N, Izquierdo E (2007) Autonomy: a review and a reappraisal. In: Almeida e Costa F, Rocha LM, Costa E, Harvey I, Coutinho A (eds) Advances in artificial life. Springer, Berlin, pp 455–464

    Chapter  Google Scholar 

  • Gay S, Mille A, Georgeon O, Dutech A (2016) Autonomous construction and exploitation of a spatial memory by a self-motivated agent. Cogn Syst Res 41:1–35

    Article  Google Scholar 

  • Georgeon OL, Marshall JB, Gay S (2012) Interactional motivation in artificial systems: between extrinsic and intrinsic motivation. In: 2012 IEEE international conference on development and learning and epigenetic robotics (ICDL), pp 1–2. https://doi.org/10.1109/DevLrn.2012.6400833

  • Haasdijk E, Eiben AE, Karafotias G (2010a) On-line evolution of robot controllers by an encapsulated evolution strategy. In: IEEE CEC 2010. IEEE, New York, pp 1–7

  • Haasdijk E, Eiben AE, Karafotias G (2010b) On-line evolution of robot controllers by an encapsulated evolution strategy. In: Proceedings of the IEEE congress on evolutionary computation, CEC 2010, Barcelona, Spain, 18–23 July 2010, pp 1–7. https://doi.org/10.1109/CEC.2010.5585926

  • Haasdijk E, Bredeche N, Eiben A (2014) Combining environment-driven adaptation and task-driven optimisation in evolutionary robotics. PLoS ONE 9(6):e98466

    Article  Google Scholar 

  • Kaplan F, Oudeyer P (2006) Intrinsically motivated machines. In: 50 Years of artificial intelligence, essays dedicated to the 50th anniversary of artificial intelligence, pp 303–314. https://doi.org/10.1007/978-3-540-77296-5_27

  • Klyne A, Merrick KE (2016) Intrinsically motivated particle swarm optimisation applied to task allocation for workplace hazard detection. Adapt Behav 24(4):219–236

    Article  Google Scholar 

  • Kompella VR, Luciw MD, Stollenga MF, Pape L, Schmidhuber J (2012) Autonomous learning of abstractions using curiosity-driven modular incremental slow feature analysis. In: 2012 IEEE international conference on development and learning and epigenetic robotics, ICDL-EPIROB 2012, pp 1–8

  • Kompella VR, Stollenga MF, Luciw MD, Schmidhuber J (2014) Explore to see, learn to perceive, get the actions for free: SKILLABILITY. In: 2014 international joint conference on neural networks, IJCNN 2014, pp 2705–2712

  • Lehman J, Miikkulainen R (2014) Overcoming deception in evolution of cognitive behaviors. In: Proceedings of the 2014 annual conference on genetic and evolutionary computation, GECCO’14. ACM, New York, pp 185–192

  • Lehman J, Stanley KO (2011) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 19(2):189–223

    Article  Google Scholar 

  • Merrick KE (2017) Value systems for developmental cognitive robotics: a survey. Cogn Syst Res 41:38–55

    Article  Google Scholar 

  • Nogueira YLB, Fisch de Brito CE, Vidal CA, Cavalcante-Neto JB (2013a) Emergence of autonomous behaviors of virtual characters through simulated reproduction. In: Advances in artificial life: ECAL 2013. The MIT Press, Berlin, pp 750–757

  • Nogueira YLB, Fisch de Brito CE, Vidal CA, Cavalcante-Neto JB (2013b) Evolving plastic neuromodulated networks for behavior emergence of autonomous virtual characters. In: Advances in artificial life, ECAL 2013. The MIT Press, Berlin, pp 577–584

  • Nogueira YLB, de Brito CEF, Vidal CA, Neto JBC (2016) Emergent vision system of autonomous virtual characters. Neurocomputing 173:1851–1867

    Article  Google Scholar 

  • Oudeyer PY, Kaplan F (2008) How can we define intrinsic motivation? In: the 8th international conference on epigenetic robotics: modeling cognitive development in robotic systems, Lund university cognitive studies. LUCS, Lund. https://hal.inria.fr/inria-00420175

  • Oudeyer PY, Smith L (2016) How evolution may work through curiosity-driven developmental process. Top Cogn Sci 8. https://doi.org/10.1111/tops.12196. https://hal.inria.fr/hal-01404334

  • Oudeyer PY, Kaplan F, Hafner V (2007) Intrinsic motivation systems for autonomous mental development. IEEE Trans Evol Comput 11(2):265–286. https://doi.org/10.1109/TEVC.2006.890271. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4141061

  • Schmidhuber J (2010) Formal theory of creativity, fun, and intrinsic motivation (1990–2010). IEEE Trans Auton Mental Dev 2(3):230–247

    Article  Google Scholar 

  • Silva F, Duarte M, Correia L, Oliveira SM, Christensen AL (2016) Open issues in evolutionary robotics. Evol Comput 24(2):205–236

    Article  Google Scholar 

  • Trueba P, Prieto A, Bellas F, Duro RJ (2015) Applying the canonical distributed embodied evolution algorithm in a collective indoor navigation task. In: IJCNN. IEEE, New York, pp 1–8

  • Varela FJ (1979) Principles of biological autonomy. North Holland, Amsterdam

    Google Scholar 

  • Varela FJ (1992) Autopoiesis and a biology of intentionality. In: McMullin B (ed) Autopoiesis and perception. Dublin City University, Dublin, pp 4–14

    Google Scholar 

  • Vernon D, Lowe R, Thill S, Ziemke T (2015) Embodied cognition and circular causality: on the role of constitutive autonomy in the reciprocal coupling of perception and action. Front Psychol 6:1660. https://doi.org/10.3389/fpsyg.2015.01660

    Article  Google Scholar 

  • Watson RA, Ficici SG, Pollack JB (2002) Embodied evolution: distributing an evolutionary algorithm in a population of robots. Robot Auton Syst 39:1–18

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computing, Federal University of Ceará, Campus do Pici - Bl. 910, Fortaleza, CE, CEP: 60440-900, Brazil

    Yuri Lenon Barbosa Nogueira, Carlos Eduardo Fisch de Brito, Creto Augusto Vidal & Joaquim Bento Cavalcante-Neto

Authors
  1. Yuri Lenon Barbosa Nogueira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Eduardo Fisch de Brito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Creto Augusto Vidal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joaquim Bento Cavalcante-Neto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuri Lenon Barbosa Nogueira.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nogueira, Y.L.B., de Brito, C.E.F., Vidal, C.A. et al. Towards intrinsic autonomy through evolutionary computation. Artif Intell Rev 53, 4449–4473 (2020). https://doi.org/10.1007/s10462-019-09798-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-019-09798-1

Keywords

Search

Navigation