Skip to main content

Advertisement

SpringerLink
Log in

A review of message passing algorithms in estimation of distribution algorithms

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

Message passing algorithms (MPAs) have been traditionally used as an inference method in probabilistic graphical models. Some MPA variants have recently been introduced in the field of estimation of distribution algorithms (EDAs) as a way to improve the efficiency of these algorithms. Multiple developments on MPAs point to an increasing potential of these methods for their application as part of hybrid EDAs. In this paper we review recent work on EDAs that apply MPAs and propose ways to further extend the useful synergies between MPAs and EDAs. Furthermore, we analyze some of the implications that MPA developments can have in their future application to EDAs and other evolutionary algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Abbeel P, Koller D, Ng AY (2006) Learning factor graphs in polynomial time and sample complexity. J Mach Learn Res 7:1743–1788

    MathSciNet  MATH  Google Scholar 

  • Baluja S, Davies S (1997) Using optimal dependency-trees for combinatorial optimization: learning the structure of the search space. In: Fisher DH (ed) Proceedings of the 14th international conference on machine learning. Morgan Kaufmann, San Francisco, pp 30–38

    Google Scholar 

  • Batra D, Gallagher A, Parikh D, Chen T (2010) Beyond trees: MRF inference via outer-planar decomposition. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, San Francisco, pp 2496–2503

  • Bickson D (2008) Gaussian belief propagation: theory and application. arXiv preprint arXiv:0811.2518. Accessed 18 Dec 2014

  • Braunstein A, Mézard M, Zecchina R (2005) Survey propagation: an algorithm for satisfiability. Random Struct Algorithms 27(2):201–226

    Article  MATH  Google Scholar 

  • Braunstein A, Mézard M, Zecchina R (2006) Constraint satisfaction by survey propagation. In: Percus A, Istrate G, Moore C (eds) Computational complexity and statistical physics. Oxford University Press, Oxford, pp 107–124

    Google Scholar 

  • Brownlee AE, McCall JA, Shakya SK, Zhang Q (2010) Structure learning and optimisation in a Markov network based estimation of distribution algorithm. In: Exploitation of linkage learning in evolutionary algorithms. Springer, Berlin, pp 45–69

  • Brownlee AEI (2009) Multivariate Markov networks for fitness modelling in an estimation of distribution algorithm. PhD Thesis, The Robert Gordon University, School of Computing, Aberdeen

  • Brownlee AEI, McCall J, Pelikan M (2012) Influence of selection on structure learning in Markov network EDAs: an empirical study. MEDAL Report No. 2012006. Missouri Estimation of Distribution Algorithms Laboratory (MEDAL)

  • Brownlee AEI, McCall J, Zhang Q, Brown D (2008) Approaches to selection and their effect on fitness modelling in an estimation of distribution algorithm. In: Proceedings of the 2008 congress on evolutionary computation CEC-2008. IEEE Press, Hong Kong, pp 2621–2628

  • Ceberio J, Irurozki E, Mendiburu A, Lozano JA (2012) A review on estimation of distribution algorithms in permutation-based combinatorial optimization problems. Prog Artif Intell 1(1):103–117

    Article  Google Scholar 

  • Ceberio J, Mendiburu A, Lozano JA (2013) The Plackett–Luce ranking model on permutation-based optimization problems. In: 2013 IEEE congress on evolutionary computation (CEC). IEEE, Cancún, pp 494–501

  • Chen B, Hu J (2010a) An adaptive niching EDA based on clustering analysis. In: 2010 IEEE congress on evolutionary computation (CEC). IEEE, Barcelona, pp 1–7

  • Chen B, Hu J (2010b) A novel clustering based niching EDA for protein folding. In: Proceedings of the world congress on nature and biologically inspired computing, 2009. NaBIC 2009. IEEE, Coimbatore, pp 748–753

  • Crick C, Pfeffer A (2003) Loopy belief propagation as a basis for communication in sensor networks. In: Proceedings of the 19th annual conference on uncertainty in artificial intelligence (UAI-2003). Morgan Kaufmann Publishers, San Francisco, pp 159–166

  • Deming WE, Stephan FF (1940) On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Ann Math Stat 11(4):427–444

    Article  MathSciNet  Google Scholar 

  • Dolev D, Bickson D, Johnson J (2009) Fixing convergence of Gaussian belief propagation. In: IEEE international symposium on information theory, 2009. ISIT 2009. IEEE, Seoul, pp 1674–1678

  • Domínguez E, Lage-Castellanos A, Mulet R, Ricci-Tersenghi F, Rizzo T (2011) Characterizing and improving generalized belief propagation algorithms on the 2D Edwards–Anderson model. J Stat Mech Theory Exp 2011(12):P12007

    Article  Google Scholar 

  • Dong W, Yao X (2008) NichingEDA: utilizing the diversity inside a population of EDAs for continuous optimization. In: Proceedings of the 2008 congress on evolutionary computation CEC-2008. IEEE Press, Hong Kong, pp 1260–1267

  • Duchi J, Tarlow D, Elidan G, Koller D (2007) Using combinatorial optimization within max-product belief propagation. In: Advances in neural information processing systems 19: proceedings of the 2006 conference, vol 19. The MIT Press, Cambridge, p 369

  • Echegoyen C (2012) Contributions to the analysis and understanding of estimation of distribution algorithms. PhD Thesis, Department of Computer Science and Artificial Intelligence, University of the Basque Country

  • Echegoyen C, Lozano JA, Santana R, Larrañaga P (2007) Exact Bayesian network learning in estimation of distribution algorithms. In: Proceedings of the 2007 congress on evolutionary computation CEC-2007. IEEE Press, Los Alamitos, pp 1051–1058. doi:10.1109/CEC.2007.4424586

  • Echegoyen C, Santana R, Lozano JA, Larrañaga P (2008) The impact of probabilistic learning algorithms in EDAs based on Bayesian networks. In: Linkage in evolutionary computation, studies in computational intelligence. Springer, Berlin, pp 109–139. doi:10.1007/978-3-540-85068-7_6

  • Echegoyen C, Mendiburu A, Santana R, Lozano JA (2009) Analyzing the probability of the optimum in EDAs based on Bayesian networks. In: Proceedings of the 2009 congress on evolutionary computation CEC-2009. IEEE Press, Trondheim, pp 1652–1659. doi:10.1109/CEC.2009.4983140

  • Echegoyen C, Mendiburu A, Santana R, Lozano JA (2010a) Analyzing the k most probable solutions in EDAs based on Bayesian networks. In: Exploitation of linkage learning in evolutionary algorithms, evolutionary learning and optimization. Springer, pp 163–189. doi:10.1007/978-3-642-12834-9_8

  • Echegoyen C, Mendiburu A, Santana R, Lozano JA (2010b) Estimation of Bayesian networks algorithms in a class of complex networks. In: Proceedings of the 2010 congress on evolutionary computation CEC-2010. IEEE Press, Barcelona. doi:10.1109/CEC.2010.5586511

  • Echegoyen C, Zhang Q, Mendiburu A, Santana R, Lozano JA (2011) On the limits of effectiveness in estimation of distribution algorithms. In: Proceedings of the 2011 congress on evolutionary computation CEC-2007. IEEE Press, New Orleans, pp 1573–1580. doi:10.1109/CEC.2011.5949803

  • Echegoyen C, Mendiburu A, Santana R, Lozano JA (2012) Toward understanding EDAs based on Bayesian networks through a quantitative analysis. IEEE Trans Evol Comput 16(2):173–189. doi:10.1109/TEVC.2010.2102037

    Article  Google Scholar 

  • Etxeberria R, Larrañaga P (1999) Global optimization using Bayesian networks. In: Ochoa A, Soto MR, Santana R (eds) Proceedings of the second symposium on artificial intelligence (CIMAF-99). Editorial Academia, Havana, pp 332–339

    Google Scholar 

  • Frey BJ, Dueck D (2007) Clustering by passing messages between data points. Science 315:972–976

    Article  MathSciNet  MATH  Google Scholar 

  • Furtlehner C, Schoenauer M (2010) Multi-objective 3-SAT with survey-propagation. In: Proceedings of the NIPS 2010 workshop on discrete optimization in machine learning: structures, algorithms and applications (DISCML), Whistler, Canada. http://hal.inria.fr/inria-00533149 Accessed 18 Dec 2014

  • Gao Y, Culberson JC (2005) Space complexity of estimation of distribution algorithms. Evol Comput 13(1):125–143

    Article  MathSciNet  Google Scholar 

  • Givoni IE, Frey BJ (2009) A binary variable model for affinity propagation. Neural Comput 21(6):1589–1600

    Article  MathSciNet  MATH  Google Scholar 

  • Givoni IE, Chung C, Frey BJ (2011) Hierarchical affinity propagation. In: Proceedings of the 27th annual conference on uncertainty in artificial intelligence (UAI-2011). Morgan Kaufmann, Barcelona

  • Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading

    MATH  Google Scholar 

  • González C, Lozano JA, Larrañaga P (2002) Mathematical modeling of UMDAc algorithm with tournament selection. Behaviour on linear and quadratic functions. Int J Approx Reason 31(4):313–340

    Article  MATH  Google Scholar 

  • Grahl J, Minner S, Bosman PA (2008) Learning structure illuminates black boxes—an introduction to estimation of distribution algorithms. In: Advances in metaheuristics for hard optimization. Springer, Berlin, pp 365–395

  • Harik G (1999) Linkage learning via probabilistic modeling in the ECGA. IlliGAL Report 99010. University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana

    Google Scholar 

  • Hartmann AK, Weigt M (2005) Phase transitions in combinatorial optimization problems: basics, algorithms and statistical mechanics. Wiley, Weinheim

    Book  Google Scholar 

  • Helmi BH, Rahmani AT, Pelikan M (2014) A factor graph based genetic algorithm. Int J Appl Math Comput Sci 24(3):621–633

    Article  MathSciNet  Google Scholar 

  • Henrion M (1988) Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In: Lemmer JF, Kanal LN (eds) Proceedings of the second annual conference on uncertainty in artificial intelligence. Elsevier, Amsterdam, pp 149–164

    Google Scholar 

  • Heskes T (2004) On the uniqueness of belief propagation fixed points. Neural Comput 16:2379–2413

    Article  MATH  Google Scholar 

  • Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press, Ann Arbor

    Google Scholar 

  • Höns R (2006) Estimation of distribution algorithms and minimum relative entropy. PhD Thesis, University of Bonn, Bonn

  • Höns R (2012) Using maximum entropy and generalized belief propagation in estimation of distribution algorithms. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 175–190

    Chapter  Google Scholar 

  • Höns R, Santana R, Larrañaga P, Lozano JA (2007) Optimization by max-propagation using Kikuchi approximations. Technical Report EHU-KZAA-IK-2/07. Department of Computer Science and Artificial Intelligence, University of the Basque Country

  • Ihler AT, Fisher J, Willsky AS (2006) Loopy belief propagation: convergence and effects of message errors. J Mach Learn Res 6(1):905

    MathSciNet  Google Scholar 

  • Jiroušek R, Přeučil S (1995) On the effective implementation of the iterative proportional fitting procedure. Comput Stat Data Anal 19:177–189

    Article  MATH  Google Scholar 

  • Johnson A, Shapiro JL (2002) The importance of selection mechanisms in distribution estimation algorithms. In: Collet P (ed) Proceedings of EA 2001, lecture notes in computer science, vol 2310. Springer, pp 91–103

  • Kaban A, Bootkrajang J, Durrant RJ (2013) Towards large scale continuous EDA: a random matrix theory perspective. In: Proceedings of the genetic and evolutionary computation conference GECCO-2013. ACM, Amsterdam, pp 383–390

  • Karshenas H, Santana R, Bielza C, Larrañaga P (2011) Multi-objective optimization with joint probabilistic modeling of objectives and variables. In: Evolutionary multi-criterion optimization: sixth international conference, EMO 2011, lecture notes in computer science. Springer, Berlin, pp 298–312. doi:10.1007/978-3-642-19893-9_21

  • Karshenas H, Santana R, Bielza C, Larrañaga P (2012) Continuous estimation of distribution algorithms based on factorized Gaussian Markov networks. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 157–173. doi:10.1007/978-3-642-28900-2-10

    Chapter  Google Scholar 

  • Kim K, McKay BR, Punithan D (2010) Sampling bias in estimation of distribution algorithms for genetic programming using prototype trees. In: PRICAI 2010: trends in artificial intelligence. Springer, Berlin, pp 100–111

  • Kroc L, Sabharwal A, Selman B (2009) Message-passing and local heuristics as decimation strategies for satisfiability. In: Proceedings of the 2009 ACM symposium on applied computing. ACM, Honolulu, pp 1408–1414

  • Kschischang FR, Frey BJ, Loeliger HA (2001) Factor graphs and the sum–product algorithm. IEEE Trans Inf Theory 47(2):498–519

    Article  MathSciNet  MATH  Google Scholar 

  • Larrañaga P, Lozano JA (eds) (2002) Estimation of distribution algorithms. A new tool for evolutionary computation. Kluwer Academic Publishers, Boston

    MATH  Google Scholar 

  • Leone M, Weigt S, Weigt M (2007) Clustering by soft-constraint affinity propagation: applications to gene-expression data. Bioinformatics 23(20):2708–2715

    Article  Google Scholar 

  • Lima CF (2009) Substructural local search in discrete estimation of distribution algorithms. PhD Thesis, University of Algarve

  • Lima CF, Pelikan M, Goldberg DE, Lobo FG, Sastry K, Hauschild M (2007) Influence of selection and replacement strategies on linkage learning in BOA. In: Proceedings of the 2007 congress on evolutionary computation CEC-2007. IEEE Press, Los Alamitos, pp 1083–1090

  • Lima CF, Pelikan M, Lobo FG, Goldberg DE (2009) Loopy substructural local search for the Bayesian optimization algorithm. In: Engineering stochastic local search algorithms. Designing, implementing and analyzing effective heuristics. Springer, Berlin, pp 61–75

  • Lozano JA, Larrañaga P, Inza I, Bengoetxea E (eds) (2006) Towards a new evolutionary computation: advances on estimation of distribution algorithms. Springer, Heidelberg

    Google Scholar 

  • Mahfoud SW (1995) Niching methods for genetic algorithms. PhD Thesis, University of Illinois at Urbana-Champaign, Urbana. Also IlliGAL Report No. 95001

  • Malioutov DM, Johnson JK, Willsky AS (2006) Walk-sums and belief propagation in Gaussian graphical models. J Mach Learn Res 7:2031–2064

    MathSciNet  MATH  Google Scholar 

  • Meltzer T, Yanover C, Weiss Y (2005) Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Tenth IEEE international conference on computer vision, pp 428–435

  • Mendiburu A, Lozano J, Miguel-Alonso J (2005) Parallel implementation of EDAs based on probabilistic graphical models. IEEE Trans Evol Comput 9(4):406–423

    Article  Google Scholar 

  • Mendiburu A, Santana R, Bengoetxea E, Lozano, J (2007a) A parallel framework for loopy belief propagation. In: Thierens D et al (eds) Proceedings of the genetic and evolutionary computation conference GECCO-2007, vol II. Companion material. ACM Press, London, pp 2843–2850. http://dl.acm.org/citation.cfm?id=1274084 Accessed 18 Dec 2014

  • Mendiburu A, Santana R, Lozano JA (2007b) Introducing belief propagation in estimation of distribution algorithms: a parallel framework. Technical Report EHU-KAT-IK-11/07. Department of Computer Science and Artificial Intelligence, University of the Basque Country

  • Mendiburu A, Santana R, Lozano JA (2012) Fast fitness improvements in estimation of distribution algorithms using belief propagation. In: Santana R, Shakya S (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 141–155. doi:10.1007/978-3-642-28900-2-9

    Chapter  Google Scholar 

  • Mézard M, Parisi G, Zechina R (2002) Analytic and algorithmic solution of random satisfiability problems. Science 297:812–812. doi:10.1126/science.1073287

    Article  Google Scholar 

  • Minka T (2001) A family of algorithms for approximate bayesian inference. PhD Thesis, Massachusetts Institute of Technology

  • Minka T (2005) Divergence measures and message passing. Technical Report TR-2005-173. Mitsubishi Electric Research Laboratories

  • Mooij JM (2005) Validity estimates for loopy belief propagation on binary real-world networks. In: Advances in neural information processing systems 17. MIT Press, Cambridge, pp 945–952

  • Mooij J (2010) libDAI: a free and open source C++ library for discrete approximate inference in graphical models. J Mach Learn Res 11:2169–2173

    MATH  Google Scholar 

  • Mühlenbein H (2012) Convergence theorems of estimation of distribution algorithms. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 91–108

    Chapter  Google Scholar 

  • Mühlenbein H, Höns R (2006) The factorized distributions and the minimum relative entropy principle. In: Pelikan M, Sastry K, Cantú-Paz E (eds) Scalable optimization via probabilistic modeling: from algorithms to applications, studies in computational intelligence. Springer, Berlin, pp 11–38

  • Mühlenbein H, Mahnig T (2002) Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning. Int J Approx Reason 31(3):157–192

    Article  MATH  Google Scholar 

  • Mühlenbein H, Paaß G (1996) From recombination of genes to the estimation of distributions I. Binary parameters. In: Voigt HM, Ebeling W, Rechenberg I, Schwefel HP (eds) Parallel problem solving from nature—PPSN IV, lectures notes in computer science, vol 1141. Springer, Berlin, pp 178–187

    Chapter  Google Scholar 

  • Mühlenbein H, Mahnig T, Ochoa A (1999) Schemata, distributions and graphical models in evolutionary optimization. J Heuristics 5(2):213–247

    Article  Google Scholar 

  • Munetomo M, Murao N, Akama K (2008) Introducing assignment functions to Bayesian optimization algorithms. Inf Sci 178(1):152–163

    Article  MATH  Google Scholar 

  • Murphy KP, Weiss Y, Jordan MI (1999) Loopy belief propagation for approximate inference: an empirical study. In: Proceedings of the fifteenth conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers, Inc., San Francisco, pp 467–475

  • Nilsson D (1998) An efficient algorithm for finding the M most probable configurations in probabilistic expert systems. Stat Comput 2:159–173

    Article  Google Scholar 

  • Ocenasek J, Schwarz J (2002) Estimation of distribution algorithm for mixed continuous–discrete optimization problems. In: Proceedings of the 2nd Euro-international symposium on computational intelligence. IOS Press, Kosice, pp 227–232

  • Ochoa A, Höns R, Soto MR, Mühlenbein H (2003) A maximum entropy approach to sampling in EDA—the single connected case. In: Progress in pattern recognition, speech and image analysis, lectures notes in computer science, vol 2905. Springer, Berlin, pp 683–690

  • Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Mateo

    Google Scholar 

  • Pelikan M, Goldberg DE, Lobo F (2002) A survey of optimization by building and using probabilistic models. Comput Optim Appl 21(1):5–20

    Article  MathSciNet  MATH  Google Scholar 

  • Peña J, Lozano JA, Larrañaga P (2005) Globally multimodal problem optimization via an estimation of distribution algorithm based on unsupervised learning of Bayesian networks. Evol Comput 13(1):43–66

    Article  Google Scholar 

  • Ponce de León E, Díaz E (2012) Adaptive evolutionary algorithm based on a cliqued gibbs sampling over graphical Markov model structure. In: Shakya S, Santana R (eds) Markov networks in evolutionary computation. Springer, Berlin, pp 109–123

    Chapter  Google Scholar 

  • Regnier-Coudert O (2013) Bayesian network structure learning using characteristic properties of permutation representations with applications to prostate cancer treatment. PhD Thesis, Robert Gordon University

  • Rivera JP, Santana R (2000) Design of an algorithm based on the estimation of distributions to generate new rules in the XCS classifier system. Technical Report ICIMAF 2000-100, CEMAFIT 2000-78. Institute of Cybernetics, Mathematics and Physics, Havana

    Google Scholar 

  • Santana R (2003) Factorized Distribution Algorithms: selection without selected population. Technical Report ICIMAF 2003-240. Institute of Cybernetics, Mathematics and Physics, Havana

    Google Scholar 

  • Santana R (2006) Advances in probabilistic graphical models for optimization and learning. Applications in protein modelling. PhD Thesis, University of the Basque Country

  • Santana R, Larrañaga P, Lozano JA (2005) Interactions and dependencies in estimation of distribution algorithms. In: Proceedings of the 2005 congress on evolutionary computation CEC-2005. IEEE Press, Edinburgh, pp 1418–1425. doi:10.1109/CEC.2005.1554856

  • Santana R, Mendiburu A, Lozano JA (2008) An empirical analysis of loopy belief propagation in three topologies: grids, small-world networks and random graphs. In: Jaeger M, Nielsen TD (eds) Proceedings of the fourth European workshop on probabilistic graphical models (PGM-2008), pp 249–256

  • Santana R, Larrañaga P, Lozano JA (2010) Learning factorizations in estimation of distribution algorithms using affinity propagation. Evol Comput 18(4):515–546. http://www.mitpressjournals.org/doi/abs/10.1162/EVCO_a_00002. Accessed 18 Dec 2014

  • Santana R, Mendiburu A, Lozano JA (2012a) Evolving NK-complexity for evolutionary solvers. In: Companion proceedings of the 2012 genetic and evolutionary computation conference GECCO-2012. ACM Press, Philadelphia, pp 1473–1474. http://dl.acm.org/citation.cfm?id=2330997. Accessed 18 Dec 2014

  • Santana R, Mendiburu A, Lozano JA (2012b) New methods for generating populations in Markov network based EDAs: decimation strategies and model-based template recombination. Technical Report EHU-KZAA-TR:2012-05. Department of Computer Science and Artificial Intelligence, University of the Basque Country. http://hdl.handle.net/10810/9180. Accessed 18 Dec 2014

  • Santana R, Mendiburu A, Lozano JA (2013) Message passing methods for estimation of distribution algorithms based on Markov networks. In: Proceedings of the 4th conference on swarm, evolutionary, and memetic computing (SEMCCO-2013), lectures notes in computer science. Springer, Chennai, pp 419–430 (in press)

  • Sastry K, Abbass HA, Goldberg DE, Johnson D (2005) Sub-structural niching in estimation of distribution algorithms. In: Proceedings of the 2005 conference on genetic and evolutionary computation. ACM, Washington, DC, pp 671–678

  • Sastry K, Lima CF, Goldberg DE (2006) Evaluation relaxation using substructural information and linear estimation. In: Proceedings of the 8th annual conference on genetic and evolutionary computation. ACM, New York, pp 419–426

  • Sastry K, Goldberg DE, Llorá X (2007) Towards billion-bit optimization via a parallel estimation of distribution algorithm. In: Thierens D et al (eds) Proceedings of the genetic and evolutionary computation conference GECCO-2007, vol I. ACM Press, London, pp 577–584

    Google Scholar 

  • Sato H, Hasegawa Y, Bollegala D, Iba H (2012) Probabilistic model building GP with belief propagation. In: 2012 IEEE congress on evolutionary computation (CEC). IEEE, pp 1–8

  • Shakya S, Santana R, Lozano JA (2012) A Markovianity based optimisation algorithm. Genet Program Evol Mach 13(2):159–195. doi:10.1007/s10710-011-9149-y

    Article  Google Scholar 

  • Soto MR (2003) A single connected factorized distribution algorithm and its cost of evaluation. PhD Thesis, University of Havana, Havana (in Spanish)

  • Suwannik W, Chongstitvatana P (2008) Solving one-billion-bit noisy OneMax problem using estimation distribution algorithm with arithmetic coding. In: Proceedings of the 2008 congress on evolutionary computation CEC-2008. IEEE Press, Hong Kong, pp 1203–1206

  • Tanaka K, Shouno H, Okada M, Titterington D (2004) Accuracy of the Bethe approximation for hyperparameter estimation in probabilistic image processing. J Phys A 37(36):8675

    Article  MathSciNet  MATH  Google Scholar 

  • Valdez-Peña IS, Hernández-Aguirre A, Botello-Rionda S (2009) Approximating the search distribution to the selection distribution in EDAs. In: Proceedings of the genetic and evolutionary computation conference GECCO-2009. ACM, New York, pp 461–468

  • Van Hoyweghen C, Goldberg D, Naudts B (2002a) From twomax to the Ising model: easy and hard symmetrical problems. In: Proceedings of the genetic and evolutionary computation conference GECCO-2002. Morgan Kaufmann Publishers, Inc., San Francisco, pp 626–633

  • Van Hoyweghen C, Naudts B, Goldberg D (2002b) Spin-flip symmetry and synchronization. Evol Comput 10(4):317–344

    Article  Google Scholar 

  • Wainwright MJ, Jordan MI (2003) Graphical models, exponential families, and variational inference. Technical Report 649. Department of Statistics, University of California, Berkeley

    Google Scholar 

  • Wainwright M, Jaakkola T, Willsky A (2002) Exact MAP estimates by (hyper) tree agreement. Adv Neural Inf Process Syst 15:809–816

    Google Scholar 

  • Wainwright M, Jaakkola T, Willsky A (2004) Tree consistency and bounds on the performance of the max-product algorithm and its generalizations. Stat Comput 14:143–166

    Article  MathSciNet  Google Scholar 

  • Wang Z, Zoghi M, Hutter F, Matheson D, De Freitas N (2013) Bayesian optimization in high dimensions via random embeddings. In: Proceedings of the Twenty-Third international joint conference on Artificial Intelligence. AAAI Press, Chicago, pp 1778–1784

  • Weiss Y (2000) Correctness of local probability propagation in graphical models with loops. Neural Comput 12:1–41

    Article  MATH  Google Scholar 

  • Weiss Y, Freeman WT (2001) On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Trans Inf Theory 47(2):723–735

    Article  MathSciNet  Google Scholar 

  • Welling M (2004) On the choice of regions for generalized belief propagation. In: Proceedings of the 20th conference on uncertainty in artificial intelligence (UAI-2004). Morgan Kaufmann Publishers, Banff, pp 585–592

  • Wiegerinck W, Heskes T (2003) Fractional belief propagation. In: Advances in neural information processing systems. MIT, Vancouver, pp 455–462

  • Xing EP, Jordan MI (2003) Graph partition strategies for generalized mean field inference. Technical Report CSD-03-1274. Division of Computer Science, University of California, Berkeley

    Google Scholar 

  • Yanover C, Weiss Y (2003) Approximate inference and protein-folding. In: Becker S, Thrun S, Obermayer K (eds) Advances in neural information processing systems 15. MIT Press, Cambridge, pp 1457–1464

    Google Scholar 

  • Yedidia JS, Freeman WT, Weiss Y (2005) Constructing free energy approximations and generalized belief propagation algorithms. IEEE Trans Inf Theory 51(7):2282–2312

    Article  MathSciNet  MATH  Google Scholar 

  • Yuille A (2001) A double-loop algorithm to minimize the Bethe and Kikuchi free energies. Neural Comput 14(6):1691–1722

    Google Scholar 

  • Zilberstein S (1996) Using anytime algorithms in intelligent systems. AI Mag 17(3):73

    Google Scholar 

Download references

Acknowledgments

This work has been partially supported by the Saiotek and Research Groups 2013–2018 (IT-609-13) programs (Basque Government), TIN2013-41272P (Ministry of Science and Technology of Spain), COMBIOMED network in computational bio-medicine (Carlos III Health Institute), and by the NICaiA Project PIRSES-GA-2009-247619 (European Commission).

Author information

Authors and Affiliations

  1. Intelligent Systems Group, Department of Computer Science and Artificial Intelligence, University of the Basque Country (UPV/EHU), Paseo Manuel de Lardizabal 1, 20080, San Sebastián, Guipúzcoa, Spain

    Roberto Santana, Alexander Mendiburu & Jose A. Lozano

Authors
  1. Roberto Santana
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexander Mendiburu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jose A. Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Roberto Santana.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Santana, R., Mendiburu, A. & Lozano, J.A. A review of message passing algorithms in estimation of distribution algorithms. Nat Comput 15, 165–180 (2016). https://doi.org/10.1007/s11047-014-9473-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-014-9473-2

Keywords

Search

Navigation