Abstract
Redundancy is a ubiquitous feature of genetic programming (GP), with many-to-one mappings commonly observed between genotype and phenotype, and between phenotype and fitness. If a representation is redundant, then neutral mutations are possible. A mutation is phenotypically-neutral if its application to a genotype does not lead to a change in phenotype. A mutation is fitness-neutral if its application to a genotype does not lead to a change in fitness. Whether such neutrality has any benefit for GP remains a contentious topic, with reported experimental results supporting both sides of the debate. Most existing studies use performance statistics, such as success rate or search efficiency, to investigate the utility of neutrality in GP. Here, we take a different tack and use a measure of robustness to quantify the neutrality associated with each genotype, phenotype, and fitness value. We argue that understanding the influence of neutrality on GP requires an understanding of the distributions of robustness at these three levels, and of the interplay between robustness, evolvability, and accessibility amongst genotypes, phenotypes, and fitness values. As a concrete example, we consider a simple linear genetic programming system that is amenable to exhaustive enumeration and allows for the full characterization of these quantities, which we then relate to the dynamical properties of simple mutation-based evolutionary processes. Our results demonstrate that it is not only the distribution of robustness amongst phenotypes that affects evolutionary search, but also (1) the distributions of robustness at the genotypic and fitness levels and (2) the mutational biases that exist amongst genotypes, phenotypes, and fitness values. Of crucial importance is the relationship between the robustness of a genotype and its mutational bias toward other phenotypes.












Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Note that a phenotype may comprise multiple, independent genotype networks that cannot be reached from one another via phenotypically-neutral point mutations. Such collections of genotype networks are referred to as genotype sets [53]. Since it is known that the phenotypes of this LGP system each comprise a single genotype network [21], we do not present the mathematical formalisms needed to describe genotype sets.
Note that genotypic accessibility is not a useful concept. This is because it is equivalent to the total number of possible point mutations to a genotype. Specifically, if we used \(f^{\rm g}_{x^{\rm g}y^{\rm g}}\) to denote the fraction of point mutations to genotype x g that result in genotype y g, then \(\sum\nolimits_{y^{\rm g}}{f^{\rm g}_{y^{\rm g}x^{\rm g}}}=40\forall x^{\rm g}\in \Upphi^{\rm g}\).
References
W. Banzhaf, in Proceedings of Parallel Problem Solving from Nature III. Genotype-phenotype mapping and neutral variation—a case study in genetic programming. Lecture Notes in Computer Science, vol. 866 (1994) pp. 322–332
W. Banzhaf, A. Leier, Genetic Programming Theory and Practice III, Chapter 14: Evolution on Neutral Networks in Genetic Programming. (Springer, Berlin, 2006), pp. 207–221
D. Barber, Bayesian Reasoning and Machine Learning. (Cambridge University Press, Cambridge, 2011)
J.D. Bloom, S.T. Labthavikul, C.R. Otey, F.H. Arnold, Protein stability promotes evolvability. Proc. Natl. Acad. Sci. 103, 5869–5874 (2006)
S. Bullock, in Proceedings of the 6th European Conference on Artificial Life. Smooth operator? Understanding and visualising mutation bias. Lecture Notes in Computer Science, vol. 2159 (2001), pp. 602–612
S. Ciliberti, O.C. Martin, A. Wagner, Innovation and robustness in complex regulatory gene networks. Proc. Nat. Acad. Sci. 104, 13591–13596 (2007)
M. Collins, Finding needles in haystacks is harder with neutrality. Genet. Prog. Evolvable Mach. 7, 131–144 (2006)
M.C. Cowperthwaite, E.P. Economo, W.R. Harcombe, E.L. Miller, L.A. Meyers, The ascent of the abundant: How mutational networks constrain evolution. PLoS Comput. Biol. 4(7), e10000110 (2008)
B. Dorronsoro, E. Alba, in IEEE Congress on Evolutionary Computation. A simple cellular genetic algorithm for continuous optimization (2006), pp. 2838–2844
J.A. Draghi, T.L. Parsons, G.P. Wagner, J.B. Plotkin, Mutational robustness can facilitate adaptation. Nature 463, 353–355 (2010)
M. Ebner, P. Langguth, J. Albert, M. Shackleton, R. Shipman, in Proceedings of the IEEE Congress on Evolutionary Computation. On neutral networks and evolvability (2001), pp 1–8
M. Ebner, M. Shackleton, R. Shipman, How neutral networks influence evolvability. Complexity 7(2), 19–33 (2002)
E. Ferrada, A. Wagner, Protein robustness promotes evolutionary innovations on large evolutionary time-scales. Proc. R. Soc. Lond. B 275, 1595–1602 (2008)
W. Fontana, P. Schuster, Continuity in evolution: On the nature of transitions. Science 280, 1451–1455 (1998)
E. Galván-López, R. Poli, in Proceedings of the Genetic and Evolutionary Computation Conference. An empirical investigation of how and why neutrality affects evolutionary search (2006), pp. 1149–1156
E. Galván-López, R. Poli, A. Kattan, M. O’Neill, A. Brabazon, Neutrality in evolutionary algorithms… What do we know?. Evol. Syst. 2, 145–163 (2011)
C. Grinstead, J. Snell, Introduction to Probability. (American Mathematical Society, Providence, 1991)
I. Harvey, A. Thompson, in Proceedings of the First International Conference on Evolvable Systems: From Biology to Hardware. Through the labyrinth evolution finds a way: A silicon ridge (1996), pp. 406–422
E.J. Hayden, E. Ferrada, A. Wagner, Cryptic genetic variation promotes rapid evolutionary adaptation in an RNA enzyme. Nature 474, 92–95 (2011)
T. Hu, W. Banzhaf, in Proceedings of the Genetic and Evolutionary Computation Conference. Neutrality and variability: two sides of evolvability in linear genetic programming (2009), pp. 963–970
T. Hu, J.L. Payne, W. Banzhaf, J.H. Moore, in Proceedings of the 14th European Conference on Genetic Programming. Robustness, evolvability, and accessibility in linear genetic programming. Lecture Notes in Computer Science, vol. 6621 (2011), pp. 13–24
M.A. Huynen, P.F. Stadler, W. Fontana, Smoothness within ruggedness: the role of neutrality in adaptation. Proc. Natl. Acad. Sci. 93, 397–401 (1996)
M. Isalan, C. Lemerle, K. Michalodimitrakis, C. Horn, P. Beltrao, E. Raineri, M. Garriga-Canut, L. Serrano, Evolvability and hierarchy in rewired bacterial gene networks. Nature 452, 840–846 (2008)
H. Ishibuchi, N. Tsukamoto, Y. Nojima, in Proceedings of Parallel Problem Solving from Nature X. Examining the effect of elitism in cellular genetic algorithms using two neighborhood structures. Lecture Notes in Computer Science, vol. 5199 (2008), pp. 458–467
T. Jörg, O.C. Martin, A. Wagner, Neutral network sizes of biological RNA molecules can be computed and are not atypically small. BMC Bioinform. 9, 464 (2008)
M. Kirschner, J. Gerhart, Evolvability. Proc. Natl. Acad. Sci. 95, 8420–8427 (1998)
P.K. Lehre, P.C. Haddow, in Proceedings of the IEEE Congress on Evolutionary Computation. Accessibility between neutral networks in indirect genotype-phenotype mappings (2005), pp. 419–426
P.K. Lehre, P.C. Haddow, in Simulated Evolution and Learning. Accessibility and runtime between convex neutral networks. Lecture Notes in Computer Science, vol. 4247 (2006), pp. 734–741
O.C. Martin, A. Wagner, Effects of recombination on complex regulatory circuits. Genetics 138, 673–684 (2009)
T. Mihaljev, B. Drossel, Evolution of a population of random Boolean networks. Eur. Phys. J. B 67, 259–267 (2009)
E. Murphy, M. O’Neill, A. Brabazon, in Proceedings of the 14th European Conference on Genetic Programming. Examining mutation landscapes in grammar based genetic programming. Lecture Notes in Computer Science, vol. 6621 (2011), pp. 130–141
M. Newman, Mixing patterns in networks. Phys. Rev. E 67, 026126 (2003)
M. Newman, Networks: An Introduction. (Oxford University Press, Oxford, 2010)
M. Newman, R. Engelhardt, Effects of selective neutrality on the evolution of molecular species. Proc. R. Soc. Lond. B 265, 1333–1338 (1998)
J.L. Payne, J.H. Moore, in Proceedings of the European Conference on Artificial Life. Robustness, evolvability, and accessibility in the signal-integration space of gene regulatory circuits (2011), pp. 638–645
D.A. Pechenick, J.L. Payne, J.H. Moore, The influence of assortativity on the robustness of signal-integration logic in gene regulatory networks. J. Theor. Biol. 296, 21–32 (2012)
K. Raman, A. Wagner, The evolvability of programmable hardware. J. R. Soc. Interf. 8, 269–281 (2011)
C. Reidys, P.F. Stadler, P. Schuster, Generic properties of combinatory maps: Neutral networks of RNA secondary structures. Bull. Math. Biol. 59, 339–397 (1997)
J. Rodrigues, A. Wagner, Genotype networks, innovation, and robustness in sulfur metabolism. BMC Syst. Biol. 5, 39 (2011)
F. Rothlauf, D.E. Goldberg, Redundant representations in evolutionary computation. Evol. Comput. 11, 381–415 (2003)
P. Schuster, W. Fontana, P.F. Stadler, I.L. Hofacker, From sequences to shapes and back: a case study in RNA secondary structures. Proc. R. Soc. Lond. B 255, 279–284 (1994)
R. Shipman, M. Shackleton, M. Ebner, R. Watson, in Proceedings of the 7th International Conference on Artificial Life. Neutral search spaces for artificial evolution: a lesson from life (2001), pp 162–169
T. Smith, P. Husbands, P. Layzell, M. O’Shea, Fitness landscapes and evolvability. Evol. Comput. 10, 1–34 (2002)
T. Smith, P. Husbands, M. O’Shea, in Proceedings of the 6th European Conference on Artificial Life. Neutral networks and evolvability with complex genotype-phenotype mapping. Lecture Notes in Computer Science, vol. 2159 (2001), pp. 272–282
T. Smith, P. Husbands, M. O’Shea, in Proceedings of the IEEE Congress on Evolutionary Computation. Neutral networks in an evolutionary robotics search space (2001), pp. 136–145
T. Soule, Resilient individuals improve evolutionary search. Artif. Life 12, 17–34 (2006)
B.M.R. Stadler, P.F. Stadler, G.P. Wagner, W. Fontana, The topology of the possible: Formal spaces underlying patterns of evolutionary change. J. Theor. Biol. 213, 241–274 (2001)
van E. Nimwegen, J. Crutchfield, M. Huynen, Neutral evolution of mutational robustness. Proc. Natl. Acad. Sci. 96, 9716–9720 (1999)
L. Vanneschi, Y. Pirola, P. Collard, M. Tomassini, S. Verel, G. Mauri, in Proceedings of the Genetic and Evolutionary Computation Conference. A quantitative study of neutrality in GP Boolean landscapes (2006), pp. 895–902
A. Wagner, Robustness and Evolvability in Living Systems. (Princeton University Press, Princeton, 2005)
A. Wagner, Neutralism and selectionism: a network-based reconciliation. Nat. Rev. Genet. 9, 965–974 (2008)
A. Wagner, Robustness and evolvability: a paradox resolved. Proc. R. Soc. Lond. B 275, 91–100 (2008)
A. Wagner, The Origins of Evolutionary Innovations. (Oxford University Press, Oxford, 2011)
G. Wagner, L. Altenberg, Complex adaptation and the evolution of evolvability. Evolution 50, 967–976 (1996)
J. Whitacre, A. Bender, Degeneracy: a design principle for achieving robustness and evolvability. J. Theor. Biol. 263, 143–153 (2010)
C.O. Wilke, Adaptive evolution on neutral networks. Bull. Math. Biol. 63, 715–730 (2001)
T. Yu, J.F. Miller, Through the interaction of neutral and adaptive mutations, evolutionary search finds a way. Artif. Life 12, 525–551 (2006)
Acknowledgments
This work was partially supported by NIH grants R01-LM009012, R01-LM010098, and R01-AI59694. J.L.P. was supported by NIH grant R25-CA134286. W.B. acknowledges support from NSERC Discovery Grants, under RGPIN 283304-07. The authors would like to thank the three anonymous reviewers for their scrutiny, Davnah Urbach for her thoughtful comments on earlier versions of this manuscript, and Bill Langdon for fruitful discussions at Evo* 2011.
Author information
Authors and Affiliations
Corresponding author
Additional information
Ting Hu and Joshua L. Payne contributed equally to this work.
Appendices
Appendices
In the appendix, we lay out the details of our analytical treatment, which is based on the techniques described in [3, 17]. Both texts provide exceptionally lucid expositions of absorbing Markov chains and their applications.
1.1 Appendix 1: Markov chains to determine the mean first passage time
The mean waiting time and mean adaptation time can be obtained analytically with absorbing Markov chains [17], using the stochastic transition matrix P whose elements
denote the probability of a transition from phenotype x p to phenotype y p.
The target phenotype t p is the absorbing state of the Markov chain; all other phenotypes are transient. The row corresponding to phenotype t p is therefore modified such that \(p_{t^{\rm p}t^{\rm p}} = 1\) and \(p_{t^{\rm p}x^{\rm p}} = 0 \forall x^{\rm p}\ne t^{\rm p}\). Placing P in canonical form, we have
where Q is a \((|\Upphi^{\rm p}|-1)\times (|\Upphi^{\rm p}|-1)\) matrix, R is a \((|\Upphi^{\rm p}|-1)\times 1\) column vector, and 0 is a \(1\times (|\Upphi^{\rm p}|-1)\) row vector. To obtain the mean waiting time, we use Q to calculate the fundamental matrix
where I is the identity matrix and entry \(n_{x^{\rm p}y^{\rm p}}\) is the expected time spent in transient phenotype x p, given that the random walk started in the transient phenotype y p. The fundamental matrix is then used to calculate
where e is a column vector of ones and \(\tau_{x^{\rm p}}\) is the mean waiting time to reach phenotype t p from phenotype x p.
1.2 Appendix 2: Markov chains to determine the most common path
Absorbing Markov chains can also be used to determine the most common path from a transient phenotype s p to a target phenotype t p. Since the most common path must visit each phenotype at most once, the length of the path must be less than or equal to the total number of phenotypes \(|\Upphi^{\rm p}|\). Every path has an associated probability, which can be calculated using the entries of the transition matrix P. For example, the path \(s^{\rm p}\rightarrow x^{\rm p}\rightarrow y^{\rm p} \rightarrow t^{\rm p}\) has probability \(p_{s^{\rm p}x^{\rm p}} p_{x^{\rm p}y^{\rm p}} p_{y^{\rm p}t^{\rm p}}\). The most common path is the one with the highest such joint probability, which can be determined efficiently using the message passing approach described in [3].
Rights and permissions
About this article
Cite this article
Hu, T., Payne, J.L., Banzhaf, W. et al. Evolutionary dynamics on multiple scales: a quantitative analysis of the interplay between genotype, phenotype, and fitness in linear genetic programming. Genet Program Evolvable Mach 13, 305–337 (2012). https://doi.org/10.1007/s10710-012-9159-4
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10710-012-9159-4