Skip to main content
SpringerLink
Log in

Saving computational budget in Bayesian network-based evolutionary algorithms

  • Published:
Natural Computing Aims and scope Submit manuscript

Abstract

During the evolutionary process, algorithms based on probability distributions for generating new individuals suffer from computational burden due to the intensive computation of probability distribution estimations, particularly when using Probabilistic Graph Models (PGMs). In the Bayesian Optimisation Algorithm (BOA), for instance, determining the optimal Bayesian network structure by a given solution sample is an NP-hard problem. To overcome this issue, we consider a new BOA-based optimisation approach (FBOA) which explores the fact that patterns of PGM adjustments can be used as a guide to reduce the frequency of PGM updates because significant changes in PGM structure might not occur so frequently, and because they can be particularly sparse at the end of evolution. In the present paper, this new approach is scrutinised in the search space of an NK-landscape optimisation problem for medium and large-size instances. Average gaps and success rates as well as the correlation between the landscape ruggedness of the problem and the expected runtime of FBOA and BOA are presented for medium-size instances. For large-size instances, optimisation results from FBOA and BOA are compared. The experiments show that, despite our FBOA being of almost three times faster than BOA, it still produces competitive optimisation results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In terms of complexity theory, finding the optimal PGM structure is NP-complete. However, some real-world problems could require more wallclock time for calculating the objective value of a single solution, in which case surrogate models are often used.

References

  • Aliferis CF, Statnikov A, Tsamardinos I, Mani S, Koutsoukos XD (2010) Local causal and Markov blanket induction for causal discovery and feature selection for classification part I: algorithms and empirical evaluation. J Mach Learn Res 11(Jan):171–234

    MathSciNet  MATH  Google Scholar 

  • Bengoetxea E (2002) Inexact graph matching using estimation of distribution algorithms. Ph.D. thesis, University of the Basque Country, Basque Country (2002)

  • Bengoetxea E, Larrañaga P, Bielza C, Del Pozo JF (2011) Optimal row and column ordering to improve table interpretation using estimation of distribution algorithms. J Heuristics 17(5):567–588

    Article  Google Scholar 

  • Bresler G (2015) Efficiently learning ising models on arbitrary graphs. In: Proceedings of the forty-seventh annual ACM symposium on theory of computing (STOC). ACM, pp 771–782

  • Casella G, Berger RL (2002) Statistical inference, 2nd edn. Duxbury, Pacific Grove

    MATH  Google Scholar 

  • Cheng Y, Diakonikolas I, Kane D, Stewart A (2018) Robust learning of fixed-structure Bayesian networks. In: NeurIPS, pp 10304–10316

  • Conover W (1999) Practical nonparametric statistics, 3rd edn. Wiley, New York

    Google Scholar 

  • Cooper G, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9(4):309–347

    MATH  Google Scholar 

  • El Yafrani M, Martins M, Wagner M, Ahiod B, Delgado M, Lüders R (2018) A hyperheuristic approach based on low-level heuristics for the travelling thief problem. Genet Program Evol Mach 19(1):121–150

    Article  Google Scholar 

  • El Yafrani M, Martins M, Delgado M, LÃijders R, Sung I, Wagner M, Oliva D (2019) On updating probabilistic graphical models in Bayesian optimisation algorithm. In: 9th Brazilian conference on intelligent systems (BRACIS). Salvador, Brasil, pp 311–316

  • Etxeberria R, Larrañaga P (1999) Global optimization using Bayesian networks. In: Proceedings of the second symposium on artificial intelligence, CIMAF’99. Editorial Academia, Havana, Cuba, pp 332–339

  • Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675–701

    Article  Google Scholar 

  • Heckerman D, Geiger D, Chickering D (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Mach Learn 20(3):197–243

    MATH  Google Scholar 

  • Henrion M (1988) Propagating uncertainty in Bayesian networks by probabilistic logic sampling. In: Machine intelligence and pattern recognition, vol. 5. Elsevier, pp. 149–163

  • Hollander M, Wolfe DA, Chicken E (2013) Nonparametric statistical methods, vol 751. Wiley, Hoboken

    MATH  Google Scholar 

  • Kauffman SA (1993) The origins of order: self-organization and selection in evolution. Oxford University Press, Oxford

    Google Scholar 

  • Kollat JB, Reed P, Kasprzyk J (2008) A new epsilon-dominance hierarchical Bayesian optimization algorithm for large multiobjective monitoring network design problems. Adv Water Resour 31(5):828–845

    Article  Google Scholar 

  • Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. The MIT Press, Cambridge

    MATH  Google Scholar 

  • Larrañaga P, Lozano JA (2002) Estimation of distribution algorithms: a new tool for evolutionary computation, vol 2. Springer, Amsterdam

    Book  Google Scholar 

  • Larrañaga P, Karshenas H, Bielza C, Santana R (2012) A review on probabilistic graphical models in evolutionary computation. J Heuristics 18:795–819

    Article  Google Scholar 

  • Liaw R, Ting C (2013) Effect of model complexity for estimation of distribution algorithm in nk landscapes. In: 2013 IEEE symposium on foundations of computational intelligence (FOCI), pp 76–83. https://doi.org/10.1109/FOCI.2013.6602458

  • Liefooghe A, Verel S, Daolio F, Aguirre H, Tanaka K (2015) A feature-based performance analysis in evolutionary multiobjective optimization. In: International conference on evolutionary multi-criterion optimization. Springer, Guimaraes, Portugal, pp 95–109

  • Martins JP, Delbem AC (2016) Pairwise independence and its impact on estimation of distribution algorithms. Swarm Evol Comput 27:80–96

    Article  Google Scholar 

  • Martins MSR, El Yafrani M, Delgado MRBS, Wagner M, Ahiod B, Lüders R (2017) HSEDA: a heuristic selection approach based on estimation of distribution algorithm for the travelling thief problem. In: Proceedings of the genetic and evolutionary computation conference, GECCO’17. ACM, New York, NY, USA, pp 361–368

  • Martins MS, El Yafrani M, Santana R, Delgado MR, Lüders R, Ahiod B (2018) On the performance of multi-objective estimation of distribution algorithms for combinatorial problems. In: IEEE conference on evolutionary computation, CEC’18, pp. 1–8. arXiv:1806.09935

  • McNemar Q (1947) Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2):153–157

    Article  Google Scholar 

  • Mühlenbein H, Paab G (1996) From recombination of genes to the estimation of distributions I. Binary parameters. Parallel problem solving from nature. PPSN IV: lecture notes in computer science, vol 1411. Springer, London, UK, pp 178–187

  • Pearl J (1988) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Mateo

    MATH  Google Scholar 

  • Pelikan M (2008) Analysis of estimation of distribution algorithms and genetic algorithms on NK landscapes. In: Proceedings of the 10th annual conference on genetic and evolutionary computation, GECCO’08. ACM, Atlanta, pp 1033–1040

  • Pelikan M, Goldberg DE, Cantú-Paz E (1999) BOA: the Bayesian optimization algorithm. In: Proceedings of the genetic and evolutionary computation conference, GECCO’99, vol I. Morgan Kaufmann Publishers, San Francisco, CA, pp 525–532

  • Pham N (2011) Investigations of constructive approaches for examination timetabling and 3d-strip packing. Ph.D. thesis, School of Computer Science and Information Technology, University of Nottingham, UK

  • Santana R, Larrañaga P, Lozano JA (2008) Combining variable neighborhood search and estimation of distribution algorithms in the protein side chain placement problem. J Heuristics 14:519–547

    Article  Google Scholar 

  • Santana R, Mendiburu A, Lozano JA (2015) Evolving MNK-landscapes with structural constraints. IEEE Congress on Evolutionary Computation. CEC’15. IEEE, Sendai, Japan, pp 1364–1371

  • Siegel S, Castellan N (1988) The friedman two-way analysis of variance by ranks. Nonparametric statistics for the behavioral sciences, pp 174–184

  • Tsamardinos I, Aliferis CF, Statnikov AR, Statnikov E (2003) Algorithms for Large Scale Markov Blanket Discovery. In: FLAIRS conference, vol 2. AAAI Press, St. Augustine, Florida, USA, pp 376–380

  • Tsamardinos I, Brown LE, Aliferis CF (2006) The max–min hill-climbing Bayesian network structure learning algorithm. Mach Learn 65(1):31–78

    Article  Google Scholar 

  • Yuan C, Malone B (2013) Learning optimal Bayesian networks: a shortest path perspective. J Artif Intell Res 48(1):23–65

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Federal University of Technology - Paraná (UTFPR), Curitiba, Brazil

    Marcella Scoczynski, Myriam Delgado & Ricardo Lüders

  2. IN3 - Computer Science Department, Universitat Oberta de Catalunya, Barcelona, Spain

    Diego Oliva

  3. Depto. de Ciencias Computacionales, Universidad de Guadalajara, CUCEI, Guadalajara, Mexico

    Diego Oliva

  4. Optimisation and Logistics, The University of Adelaide, Adelaide, Australia

    Markus Wagner

  5. Operations Research Group, Aalborg University (AAU), Aalborg, Denmark

    Inkyung Sung & Mohamed El Yafrani

Authors
  1. Marcella Scoczynski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Myriam Delgado
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo Lüders
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Diego Oliva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Markus Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Inkyung Sung
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Mohamed El Yafrani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Diego Oliva.

Ethics declarations

Conflict of interest

The author declare that they have no conflict of interest.

Human and animal rights

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Scoczynski, M., Delgado, M., Lüders, R. et al. Saving computational budget in Bayesian network-based evolutionary algorithms. Nat Comput 20, 775–790 (2021). https://doi.org/10.1007/s11047-021-09849-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11047-021-09849-z

Keywords

Search

Navigation