Abstract
The choice of genetic representation crucially determines the capability of evolutionary processes to find complex solutions in which many variables interact. The question is how good genetic representations can be found and how they can be adapted online to account for what can be learned about the structure of the problem from previous samples. We address these questions in a scenario that we term indirect Estimation-of-Distribution: We consider a decorrelated search distribution (mutational variability) on a variable length genotype space. A one-to-one encoding onto the phenotype space then needs to induce an adapted phenotypic variability incorporating the dependencies between phenotypic variables that have been observed successful previously. Formalizing this in the framework of Estimation-of-Distribution Algorithms, an adapted phenotypic variability can be characterized as minimizing the Kullback-Leibler divergence to a population of previously selected individuals (parents). Our core result is a relation between the Kullback-Leibler divergence and the description length of the encoding in the specific scenario, stating that compact codes provide a way to minimize this divergence. A proposed class of Compression Evolutionary Algorithms and preliminary experiments with an L-system compression scheme illustrate the approach. We also discuss the implications for the self-adaptive evolution of genetic representations on the basis of neutrality (σ-evolution) towards compact codes.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Amari, S.: Information geometry on hierarchy of probability distributions. IEEE Transactions on Information Theory 47(5), 1701–1711 (2001)
Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Technical Report CMU-CS-94-163, Comp. Sci. Dep., Carnegie Mellon U. (1994)
Baluja, S., Davies, S.: Using optimal dependency-trees for combinatorial optimization: Learning the structure of the search space. In: Proc. of Fourteenth Int. Conf. on Machine Learning (ICML 1997), pp. 30–38 (1997)
Barbulescu, L., Watson, J.-P., Whitley, D.: Dynamic representations and escaping local optima: Improving genetic algorithms and local search. In: Seventeenth National Conference on Artificial Intelligence (AAAI), pp. 879–884 (2000)
Barron, A., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory 44, 2743–2760 (1998)
de Bonet, J.S., Isbell Jr., C.L., Viola, P.: MIMIC: Finding optima by estimating probability densities. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, p. 424. The MIT Press, Cambridge (1997)
de Jong, E.D.: Representation development from Pareto-Coevolution. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2724, Springer, Heidelberg (2003)
Halder, G., Callaerts, P., Gehring, W.: Induction of ectopic eyes by targeted expression of the eyeless gene in Drosophila. Science 267, 1788–1792 (1995)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaption in evolutionary strategies. Evolutionary Computation 9, 159–195 (2001)
Heckendorn, R.B., Wright, A.H.: Efficient linkage discovery by limited probing. Evolutionary Computation (2004) (accepted for publication)
Hornby, G.S., Pollack, J.B.: The advantages of generative grammatical encodings for physical design. In: Proc. of 2001 Congress on Evolutionary Computation (CEC 2001), pp. 600–607. IEEE Press, Los Alamitos (2001)
Liepins, G.E., Vose, M.D.: Representation issues in Genetic Algorithms. Journal of Experimental and Theoretical Artificial Intelligence 2 (1990)
Nevill-Manning, C.G., Witten, I.H.: Identifying hierarchical structure in sequences: A linear-time algorithm. Journal of Artificial Intelligence Research 7, 67–82 (1997)
Nordin, P., Banzhaf, W.: Complexity compression and evolution. In: Eshelman, L. (ed.) Genetic Algorithms: Proc. of Sixth International Conf (ICGA 1995), pp. 310–317, 15-19. Morgan Kaufmann, Pittsburgh (1995)
Pelikan, M., Goldberg, D.E.: Hierarchical BOA solves Ising spin glasses and MAXSAT. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2724, pp. 1271–1282. Springer, Heidelberg (2003)
Pelikan, M., Goldberg, D.E., Cantú-Paz, E.: Linkage problem, distribution estimation, and Bayesian networks. Evolutionary Computation 9, 311–340 (2000)
Pelikan, M., Goldberg, D.E., Lobo, F.: A survey of optimization by building and using probabilistic models. Technical Report IlliGAL-99018, Illinois Genetic Algorithms Laboratory (1999)
Rothlauf, F., Goldberg, D.E.: Redundant representations in Evolutionary Computation. Evolutionary Computation 11, 381–415 (2003)
Stephens, C.R., Vargas, J.M.: Effective fitness as an alternative paradigm for evolutionary computation I: General formalism. Genetic Programming and Evolvable Machines 1, 363–378 (2000)
Toussaint, M.: Demonstrating the evolution of complex genetic representations: An evolution of artificial plants. In: Cantú-Paz, E., Foster, J.A., Deb, K., Davis, L., Roy, R., O’Reilly, U.-M., Beyer, H.-G., Kendall, G., Wilson, S.W., Harman, M., Wegener, J., Dasgupta, D., Potter, M.A., Schultz, A., Dowsland, K.A., Jonoska, N., Miller, J., Standish, R.K. (eds.) GECCO 2003. LNCS, vol. 2724, pp. 86–97. Springer, Heidelberg (2003)
Toussaint, M.: The evolution of genetic representations and modular neural adaptation, April 2003. PhD thesis, Institut für Neuroinformatik, Ruhr-Universiät- Bochum, Germany. Published with the Logos Verlag Berlin, 173 pages (2004) ISBN 3-8325- 0579-2
Toussaint, M.: On the evolution of phenotypic exploration distributions. In: Cotta, C., De Jong, K., Poli, R., Rowe, J. (eds.) Foundations of Genetic Algorithms 7 (FOGA VII), pp. 169–182. Morgan Kaufmann, San Francisco (2003)
Toussaint, M.: Notes on information geometry and evolutionary processes (2004) Los Alamos pre-print nlin.AO/0408040
Vitányi, P.M.B., Li, M.: Minimum Description Length induction, Bayesianism, and Kolmogorov complexity. IEEE Trans. Inform. Theory IT-46, 446–464 (2000)
Wagner, G.P., Altenberg, L.: Complex adaptations and the evolution of evolvability. Evolution 50, 967–976 (1996)
Watson, R.A., Pollack, J.B.: Hierarchically consistent test problems for genetic algorithms: Summary and additional results. In: Late breaking papers at the Genetic and Evolutionary Computation Conference, pp. 292–297 (1999)
Whitley, D., Rana, S., Heckendorn, R.: Representation issues in neighborhood search and evolutionary algorithms. In: Genetic Algorithms and Evolution Strategy in Engineering and Computer Science, pp. 39–58. John Wiley & Sons Ltd., Chichester (1997)
Wright, H., Poli, R., Stephens, C.R., Langdon, W.B., Pulavarty, S.: An Estimation of Distribution Algorithm based on maximum entropy. In: Deb, K., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 343–354. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Toussaint, M. (2005). Compact Genetic Codes as a Search Strategy of Evolutionary Processes. In: Wright, A.H., Vose, M.D., De Jong, K.A., Schmitt, L.M. (eds) Foundations of Genetic Algorithms. FOGA 2005. Lecture Notes in Computer Science, vol 3469. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11513575_5
Download citation
DOI: https://doi.org/10.1007/11513575_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27237-3
Online ISBN: 978-3-540-32035-7
eBook Packages: Computer ScienceComputer Science (R0)