Abstract
Recently, several evolutionary algorithms have been proposed that build and use an explicit distribution model of the population to perform optimization. One of the main issues in this class of algorithms is how to estimate the distribution of selected samples. In this paper, we present a Bayesian evolutionary algorithm (BEA) that learns the sample distribution by a probabilistic graphical model known as Helmholtz machines. Due to the generative nature and availability of the wake-sleep learning algorithm, the Helmholtz machines provide an effective tool for modeling and sampling from the distribution of selected individuals. The proposed method has been applied to a suite of GA-deceptive functions. Experimental results show that the BEA with the Helmholtz machine outperforms the simple genetic algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baluja, S. and Caruana, R., “Removing the genetics from the standard genetic algorithm”, Technical Report CMU-CS-95-141, Carnegie Mellon University, 1995.
Baluja, S. and Davies, S., “Using optimal dependency-trees for combinatorial optimization: learning the structure of the search space”, Proc. 14th Int. Conf. on Machine Learning, pp. 30–38, Morgan-Kaufmann, 1997.
Bäck, T., Evolutionary Algorithms in Theory and Practice. Oxford Univ. Press, 1996.
Dayan, P., Neal, G.E., and Zemel, R.S., “The Helmholtz machine”, Neural Computation, 7: 1022–1037, 1995.
De Bonet, J.S., Isbell, C.L., and Viola, P., “MIMIC: Finding optima by estimating probability densities”, NIPS 9, pp. 424–430, The MIT Press, 1997.
Hinton, G.E., Dayan, P., Frey, B.J., Neal R. M., “The wake-sleep algorithm for unsupervised neural networks”, Science, 268: 1158–1160, 1995.
Mühlenbein, H. and Paaß, G., “From recombination of genes to the estimation of distributions I: Binary parameters”, PPSN IV, LNCS 1141, pp. 178–187, Springer, 1996.
Mühlenbein, H., Mahnig, T., and A. Ochoa, “Schemata, distributions and graphical models in evolutionary optimization”, Journal of Heuristics, 5:215–247, 1999.
Mühlenbein, H. and Mahnig, T., “FDA-A scalable evolutionary algorithm for the optimization of additively decomposed functions”, Evolutionary Computation, 7(4):353–376, 1999.
Neal, R.M. and Dayan, P., “Factor analysis using delta-rule wake-sleep learning”, Neural Computation, 9:1781–1803, 1997.
Pelikan, M. and Muhlenbein, H., “The bivariate marginal distribution algorithm”, Advances in Soft Computing-Engineering Design and Manufacturing, pp. 521–535, London: Springer-Verlag, 1999.
Pelikan, M., Goldberg, D.E., and Cantii-Paz, E., “BOA: The Bayesian optimization algorithm”, GECCO-99: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 525–532, CA: Morgan Kaufmann, 1999.
Zhang, B.-T., A Bayesian framework for evolutionary computation. In Proc. 1999 Congress on Evolutionary Computation (CEC99), IEEE Press, pp. 722–727, 1999.
Zhang, B.-T., Bayesian methods for efficient genetic programming. Genetic Programming and Evolvable Machines, 1(3):217–242, 2000.
Zhang, B.-T. and Joung, J.-G., Efficient model induction by a Bayesian evolutionary algorithm with incremental data inheritance, Technical Report SCAI-98-017, Artificial Intelligence Lab (SCAI), Seoul National University, August 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, BT., Shin, SY. (2000). Bayesian Evolutionary Optimization Using Helmholtz Machines. In: Schoenauer, M., et al. Parallel Problem Solving from Nature PPSN VI. PPSN 2000. Lecture Notes in Computer Science, vol 1917. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45356-3_81
Download citation
DOI: https://doi.org/10.1007/3-540-45356-3_81
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41056-0
Online ISBN: 978-3-540-45356-7
eBook Packages: Springer Book Archive