Luigi Malagò
ABSTRACT
Estimation of Distribution Algorithms are a recent new meta-heuristic used in Genetics-Based Machine Learning to solve combinatorial and continuous optimization problems. One of the distinctive features of this family of algorithms is that the search for the optimum is performed within a candidate space of probability distributions associated to the problem rather than over the population of possible solutions. A framework based on Information Geometry [3] is applied in this paper to propose a geometrical interpretation of the different operators used in EDAs and provide a better understanding of the underlying behavior of this family of algorithms from a novel point of view. The analysis carried out and the simple examples introduced show the importance of the boundary of the statistical model w.r.t. the distributions and EDA may converge to.
- A. Agresti. Categorical Data Analysis. Wiley, New York, NY, second edition, 2002.Google Scholar
- S.-I. Amari. Information Geometry of the EM and em Algorithms for Neural Networks. Neural Networks, 8(9):1379--1408, 1995. Google ScholarDigital Library
- S.-I. Amari. Information Geometry on Hierarchy of Probability Distributions. IEEE Trans. on Information Theory, 47(5):1701--1711, 2001. Google ScholarDigital Library
- S.-I. Amari, K. Kurata, and H. Nagaoka. Information Geometry of Boltzmann Machines. IEEE Transaction on Neural Networks, 3(2):260--271, 1992.Google ScholarDigital Library
- S.-I. Amari and H. Nagaoka. Methods of Information Geometry (Translations of Mathematical Monographs). AMS, Oxford University Press, 2000.Google Scholar
- S. Baluja. Population-Based Incremental Learning: A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning. Technical Report CMU-CS-94-163, Pittsburgh, PA, 1994. Google ScholarDigital Library
- J. S. De Bonet, C. L. Isbell, Jr., and P. Viola. MIMIC: Finding Optima by Estimating Probability Densities. In Advances in Neural Information Processing Systems, volume 9, pages 424--430, 1996.Google Scholar
- M. Hohfeld and G. Rudolph. Towards a Theory of Population-Based Incremental Learning. In Proceedings of The IEEE Conference on Evolutionary Computation, pages 1--5, 1997.Google Scholar
- P. Larrañaga and J. A. Lozano. Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers, Norwell, MA, 2002. Google ScholarDigital Library
- G. Lebanon. Riemannian Geometry and Statistical Machine Learning. PhD thesis, CMU, Pittsburgh, PA, 2005. Google ScholarDigital Library
- L. Malagò. An Information Geometry Perspective on Estimation of Distribution Algorithms. Master's thesis, Politecnico di Milano, Italy, 2007.Google Scholar
- M. Pelikan, D. E. Goldberg, and E. Cantu-Paz. Linkage Problem, Distribution Estimation, and Bayesian Networks. Evolutionary Computation, 8(3):311--340, 2000. Google ScholarDigital Library
- M. Pelikan and H. M¨uhlenbein. The Bivariate Marginal Distribution Algorithm. In R. Roy, T. Furuhashi, and P. K. Chawdhry, editors, Advances in Soft Computing: Engineering Design and Manufacturing, pages 521--535, London, UK, 1999.Google ScholarCross Ref
- S. Shakya and J. McCall. Optimization by Estimation of Distribution with DEUM Framework Based on Markov Random Fields. International Journal of Automation and Computing, 4(3):262--272, 2007.Google ScholarCross Ref
- M. Toussaint. The Structure of Evolutionary Exploration: On Crossover, Buildings Blocks, and Estimation-Of-Distribution Algorithms. In Proceedings of GECCO'03, pages 1444--1456, 2003. Google ScholarDigital Library
- M. Toussaint. Notes on Information Geometry and Evolutionary Processes, 2004.Google Scholar
- Q. Zhang and H. Mühlenbein. On the Convergence of a Class of Estimation of Distribution Algorithms. IEEE Trans. on Evolutionary Computation, 8(2):127--136, 2004. Google ScholarDigital Library
Index Terms
- An information geometry perspective on estimation of distribution algorithms: boundary analysis
Recommendations
Sub-structural niching in estimation of distribution algorithms
GECCO '05: Proceedings of the 7th annual conference on Genetic and evolutionary computationWe propose a sub-structural niching method that fully exploits the problem decomposition capability of linkage-learning methods such as the estimation distribution algorithms and concentrate on maintaining diversity at the sub-structural level. The ...
Analysis of computational time of simple estimation of distribution algorithms
Estimation of distribution algorithms (EDAs) are widely used in stochastic optimization. Impressive experimental results have been reported in the literature. However, little work has been done on analyzing the computation time of EDAs in relation to ...
Cooperative coevolution and univariate estimation of distribution algorithms
FOGA '09: Proceedings of the tenth ACM SIGEVO workshop on Foundations of genetic algorithmsIn this paper, we discuss a curious relationship between Cooperative Coevolutionary Algorithms (CCEAs) and univariate Estimation of Distribution Algorithms (EDAs). Specifically, the distribution model for univariate EDAs is equivalent to the infinite ...
Comments