ABSTRACT
The diversity and quality of natural systems have been a puzzle and inspiration for communities studying artificial life. It is now widely admitted that the adaptation mechanisms enabling these properties are largely influenced by the environments they inhabit. Organisms facing environmental variability have two alternative adaptation mechanisms operating at different timescales: plasticity, the ability of a phenotype to survive in diverse environments and evolvability, the ability to adapt through mutations. Although vital under environmental variability both mechanisms are associated with fitness costs hypothesized to render them unnecessary in stable environments. In this work, we study the interplay between environmental dynamics and adaptation in a minimal model of the evolution of plasticity and evolvability. We experiment with different types of environments characterized by the presence of niches and a climate function that determines the fitness landscape. We empirically show that environmental dynamics affect plasticity and evolvability differently and that the presence of diverse ecological niches favors adaptability even in stable environments. We perform ablation studies of the selection mechanisms to separate the role of fitness-based selection and niche-limited competition. Results obtained from our minimal model allow us to propose promising research directions in the study of open-endedness in biological and artificial systems.
Supplemental Material
Available for Download
Supplemental material.
- Josh R. Auld, Anurag A. Agrawal, and Rick A. Relyea. 2010. Re-evaluating the costs and limits of adaptive phenotypic plasticity. Proceedings of the Royal Society B: Biological Sciences 277, 1681 (Feb. 2010), 503--511. Google ScholarCross Ref
- Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. 2020. Emergent Tool Use From Multi-Agent Autocurricula. https://openreview.net/forum?id=SkxpxJBKwS tex.ids: Baker2019 arXiv: 1909.07528.Google Scholar
- Gillian R. Brown, Thomas E. Dickins, Rebecca Sear, and Kevin N. Laland. 2011. Evolutionary accounts of human behavioural diversity. Philosophical Transactions of the Royal Society B: Biological Sciences 366, 1563 (Feb. 2011), 313--324. Google ScholarCross Ref
- Luis-Miguel Chevin, Russell Lande, and Georgina M. Mace. 2010. Adaptation, Plasticity, and Extinction in a Changing Environment: Towards a Predictive Theory. PLoS Biology 8, 4 (April 2010), e1000357. Google ScholarCross Ref
- Jeff Clune. 2020. AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence. arXiv:1905.10985 [cs] (Jan. 2020). http://arxiv.org/abs/1905.10985 arXiv: 1905.10985.Google Scholar
- Karl Cobbe, Oleg Klimov, Chris Hesse, Taehoon Kim, and John Schulman. 2019. Quantifying Generalization in Reinforcement Learning. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, California, USA, 1282--1289. http://proceedings.mlr.press/v97/cobbe19a.htmlGoogle Scholar
- Thomas D. Cuypers, Jacob P. Rutten, and Paulien Hogeweg. 2017. Evolution of evolvability and phenotypic plasticity in virtual cells. BMC Evolutionary Biology 17, 1 (Dec. 2017), 60. Google ScholarCross Ref
- Charles Darwin. 1859. On the Origin of Species by Means of Natural Selection or the Natural Selection of Favoured Races in the Struggle for Life. New York: D. Appleton and Company.Google Scholar
- Emily L. Dolson, Anya E. Vostinar, Michael J. Wiser, and Charles A Ofria. 2019. The MODES Toolbox: Measurements of Open-Ended Dynamics in Evolving Systems. Artificial Life 25 (2019), 50--73.Google ScholarDigital Library
- Stephane Doncieux, Giuseppe Paolo, Alban Laflaquière, and Alexandre Coninx. 2020. Novelty Search makes Evolvability Inevitable. arXiv:2005.06224 [cs] (May 2020). http://arxiv.org/abs/2005.06224 arXiv: 2005.06224.Google Scholar
- D. J. Earl and M. W. Deem. 2004. Evolvability is a selectable trait. Proceedings of the National Academy of Sciences 101, 32 (Aug. 2004), 11531--11536. Google ScholarCross Ref
- Manfred Eppe and Pierre-Yves Oudeyer. 2021. Intelligent Behavior Depends on the Ecological Niche. KI - Künstliche Intelligenz (Jan. 2021). Google ScholarCross Ref
- Kevin Frans and Olaf Witkowski. 2021. Population-Based Evolution Optimizes a Meta-Learning Objective. arXiv:2103.06435 [cs] (March 2021). http://arxiv.org/abs/2103.06435 arXiv: 2103.06435.Google Scholar
- Cameron K Ghalambor, Lisa M Angeloni, and Scott P Carroll, [n. d.]. Behavior as Phenotypic Plasticity. ([n.d.]), 19.Google Scholar
- Antoine Giraud, Ivan Matic, Olivier Tenaillon, Antonio Clara, Miroslav Radman, Michel Fons, and François Taddei. 2001. Costs and Benefits of High Mutation Rates: Adaptive Evolution of Bacteria in the Mouse Gut. Science 291, 5513 (2001), 2606--2608. arXiv:https://www.science.org/doi/pdf/10.1126/science.1056421 Google ScholarCross Ref
- Matt Grove. 2011. Speciation, diversity, and Mode 1 technologies: The impact of variability selection. Journal of Human Evolution 61, 3 (Sept. 2011), 306--319. Google ScholarCross Ref
- Matt Grove. 2014. Evolution and dispersal under climatic instability: a simple evolutionary algorithm. Adaptive Behavior 22, 4 (Aug. 2014), 235--254. Google ScholarDigital Library
- Kelley Harris. 2015. Evidence for recent, population-specific evolution of the human mutation rate. Proceedings of the National Academy of Sciences 112, 11 (March 2015), 3439--3444. Google ScholarCross Ref
- Timothy D. Johnston. 1982. Selective Costs and Benefits in the Evolution of Learning. In Advances in the Study of Behavior. Vol. 12. Elsevier, 65--106. Google ScholarCross Ref
- Grgur Kovac, Rémy Portelas, Katja Hofmann, and Pierre-Yves Oudeyer. 2021. SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents. CoRR abs/2107.00956 (2021). arXiv:2107.00956 https://arxiv.org/abs/2107.00956Google Scholar
- Robert Tjarko Lange and Henning Sprekeler. 2021. Learning not to learn: Nature versus nurture in silico. arXiv:2010.04466 [cs, q-bio] (March 2021). http://arxiv.org/abs/2010.04466 arXiv: 2010.04466.Google Scholar
- Joel Lehman and Risto Miikkulainen. 2015. Extinction Events Can Accelerate Evolution. PLOS ONE 10, 8 (Aug. 2015), e0132886. Google ScholarCross Ref
- Joel Lehman and Kenneth O. Stanley. 2013. Evolvability Is Inevitable: Increasing Evolvability without the Pressure to Adapt. PLoS ONE 8, 4 (April 2013), e62186. Google ScholarCross Ref
- Joel Z Leibo, Edward Hughes, Marc Lanctot, and Thore Graepel. 2019. Autocurricula and the emergence of innovation from social interaction: A manifesto for multi-agent intelligence research. arXiv preprint arXiv:1903.00742 (2019).Google Scholar
- Michael Lynch. 2011. The Lower Bound to the Evolution of Mutation Rates. Genome Biology and Evolution 3 (Jan. 2011), 1107--1118. Google ScholarCross Ref
- Michael Lynch, Matthew S. Ackerman, Jean-Francois Gout, Hongan Long, Way Sung, W. Kelley Thomas, and Patricia L. Foster. 2016. Genetic drift, selection and the evolution of the mutation rate. Nature Reviews Genetics 17, 11 (Nov. 2016), 704--714. Google ScholarCross Ref
- Michael Lynch and Wilfried Gabriel. 1987. Environmental Tolerance. The American Naturalist 129, 2 (1987), 283--303. arXiv:https://doi.org/10.1086/284635 Google ScholarCross Ref
- Mark A. Maslin, Susanne Shultz, and Martin H. Trauth. 2015. A synthesis of the theories and concepts of early human evolution. Philosophical Transactions of the Royal Society B: Biological Sciences 370, 1663 (March 2015), 20140064. Publisher: Royal Society. Google ScholarCross Ref
- Eleni Nisioti, Katia Jodogne-del Litto, and Clément Moulin-Frier. 2021. Grounding an Ecological Theory of Artificial Intelligence in Human Evolution. In NeurIPS 2021 - Conference on Neural Information Processing Systems / Workshop: Ecological Theory of Reinforcement Learning. virtual event, France. https://hal.archives-ouvertes.fr/hal-03446961 Submitted paper.Google Scholar
- Pierre-Yves Oudeyer, Frdric Kaplan, and Verena V. Hafner. 2007. Intrinsic Motivation Systems for Autonomous Mental Development. IEEE Transactions on Evolutionary Computation 11, 2 (April 2007), 265--286. Conference Name: IEEE Transactions on Evolutionary Computation. Google ScholarDigital Library
- Paul N Pearson. 2001. Red Queen Hypothesis. John Wiley & Sons, Ltd. arXiv:https://novel-coronavirus.onlinelibrary.wiley.com/doi/pdf/10.1038/npg.els.0001667 Google ScholarCross Ref
- Remy Portelas, Cédric Colas, Lilian Weng, Katja Hofmann, and Pierre-Yves Oudeyer. 2020. Automatic Curriculum Learning For Deep RL: A Short Survey. arXiv:2003.04664 [cs, stat] (May 2020). http://arxiv.org/abs/2003.04664 arXiv:2003.04664.Google Scholar
- Richard Potts. 2013. Hominin evolution in settings of strong environmental variability. Quaternary Science Reviews 73 (Aug. 2013), 1--13. Google ScholarCross Ref
- Justin K. Pugh, Lisa B. Soros, and Kenneth O. Stanley. 2016. Quality Diversity: A New Frontier for Evolutionary Computation. Frontiers in Robotics and AI 3 (July 2016). Google ScholarCross Ref
- Ricard Solé. 2022. Revisiting Leigh Van Valen's "A New Evolutionary Law" (1973). Biological Theory (Jan. 2022), s13752-021-00391-w. Google ScholarCross Ref
- Joseph Suarez, Yilun Du, Phillip Isola, and Igor Mordatch. 2019. Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents. arXiv:1903.00784 [cs, stat] (March 2019). http://arxiv.org/abs/1903.00784 arXiv: 1903.00784.Google Scholar
- Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michaël Mathieu, Nat McAleese, Nathalie Bradley-Schmieg, Nathaniel Wong, Nicolas Porcel, Roberta Raileanu, Steph Hughes-Fitt, Valentin Dalibard, and Wojciech Marian Czarnecki. 2021. Open-Ended Learning Leads to Generally Capable Agents. CoRR abs/2107.12808 (2021). arXiv:2107.12808 https://arxiv.org/abs/2107.12808Google Scholar
- Martin H. Trauth, Mark A. Maslin, Alan L. Deino, Annett Junginger, Moses Lesoloyia, Eric O. Odada, Daniel O. Olago, Lydia A. Olaka, Manfred R. Strecker, and Ralph Tiedemann. 2010. Human evolution in a variable environment: the amplifier lakes of Eastern Africa. Quaternary Science Reviews 29, 23-24 (Nov. 2010), 2981--2988. Google ScholarCross Ref
- Elisabeth S. Vrba. 1985. Environment and evolution: alternative causes of the temporal distribution of evolutionary events. South African Journal of Science 81 (1985), 229--236.Google Scholar
- Rui Wang, Joel Lehman, Jeff Clune, and Kenneth O. Stanley. 2019. Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions. arXiv:1901.01753 [cs] (Feb. 2019). http://arxiv.org/abs/1901.01753 arXiv: 1901.01753.Google Scholar
Recommendations
Program evolvability under environmental variations and neutrality
GECCO '07: Proceedings of the 9th annual conference companion on Genetic and evolutionary computationBiological organisms employ various mechanisms to cope with the dynamic environments they live in. One recent research reported that depending on the rates of environmental variation, populations evolve toward genotypes in different regions of the ...
Exploiting environmental differentiation to promote evolvability in artificial evolution
GECCO '17: Proceedings of the Genetic and Evolutionary Computation Conference CompanionIn this work we investigate the possibility to exploit environmental differentiation to promote evolvability in artificial evolution. More specifically we propose a new algorithm and demonstrate how agents evolved for the ability to solve the double-...
Evolvability Search: Directly Selecting for Evolvability in order to Study and Produce It
GECCO '16: Proceedings of the Genetic and Evolutionary Computation Conference 2016One hallmark of natural organisms is their significant evolvability, i.e.,their increased potential for further evolution. However, reproducing such evolvability in artificial evolution remains a challenge, which both reduces the performance of ...
Comments