Abstract
Bayesian networks (BNs) can be easily refined (or learn) using data given prior knowledge about a changing environment. Furthermore, by exploring multiple diverse BNs in parallel, it is expected that an intelligent system may adapt quickly to changes in the environment, resulting in robust prediction. Recently, there have been attempts to design BN structures using evolutionary algorithms; however, most of these have used only the fittest solution from the final generation. Because it is difficult to combine all of the important factors into a single evaluation function, the solution is often biased and of limited adaptability. Here we describe a method of generating diverse BN structures via speciation and selective combination for adaptive prediction. Experiments using the seven benchmark networks show that the proposed method can result in improved accuracy in handling uncertainty by exploiting ensembles of BNs evolved by speciation.
Similar content being viewed by others
References
Barber D (2012) Bayesian reasoning and machine learning. Cambridge University Press, Cambridge
Beinlich IA, Suermondt HJ, Chavez RM, Cooper GF (1989) The ALARM monitoring system: a case study with two probabilistic inference techniques for belief networks. In: Proceedings of the Second European Conference on Artificial Intelligence in Medicine, pp 247–256
Binder J, Koller D, Russell S, Kanazawa K (1997) Adaptive probabilistic networks with hidden variables. Mach Learn 29(2–3):213–244
Chickering DM, Geiger D, Heckerman D (1994) Learning Bayesian networks is NP-hard, Technical Report MSR-TR-94-17, Microsoft Research
Colace F, De Santo M, Greco L (2014) Learning Bayesian network structure using a multiexpert approach. Int J Softw Eng Knowl Eng 24(2):269–284
Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9:309–347
Daly R, Shen Q, Aitken S (2011) Learning Bayesian networks: approaches and issues. Knowl Eng Rev 26(2):99–157
Feng G, Zhang J-D, Liao SS (2014) A novel method for combining Bayesian networks, theoretical analysis, and its applications. Pattern Recognit 47(5):2057–2069
Gamez JA, Mateo JL, Puerta JM (2011) Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood. Data Min Knowl Discov 22(1–2):106–148
Garg A, Pavlovic V, Rehg JM (2003) Boo sted learning in dynamic Bayesian networks for multimodal speaker detection. Proc IEEE 91(9):1355–1369
GENIE & SMILE. http://genie.sis.pitt.edu
Goldberg DE (2008) Genetic algorithms in search, optimization, and machine learning, 1st edn. Addison-Wesley Professional
Gouvea MM Jr., Araujo AFR (2010) Diversity-based adaptive evolutionary algorithms, Chapter 1. New Achievements in Evolutionary Computation
Heckerman D (2008) A tutorial on learning with Bayesian networks. Innov Bayesian Netw 156:33–82
Hu L, Wang L (2013) Using consensus Bayesian network to model the reactive oxygen species regulatory pathway. PLOS One 8(2):e56832. doi:10.1371/journal.pone.0056832
Hwang K-S, Cho S-B (2009) Landmark detection from mobile life log using a modular Bayesian network model. Expert Syst Appl 36:12065–12076
Jensen FV, Kjærulff U, Olesen KG, Pedersen J (1989) An expert system for control of waste water treatment—a pilot project. Technical report. Judex Datasystemer A/S, Aalborg (in Danish)
Kim K-J, Cho S-B (2005) Systematically incorporating domain-specific knowledge into evolutionary speciated checkers players. IEEE Trans Evol Comput 9(6):615–627
Kim K-J, Cho S-B (2008) Evolutionary ensemble of diverse artificial neural networks using speciation. Neurocomputing 71(7–9):1604–1618
Kim K-J, Cho S-B (2012) Automated synthesis of multiple analog circuits using evolutionary computation for redundancy-based fault-tolerance. Appl Soft Comput 12(4):1309–1321
Kim K, Mckay R (2012) Stochastic diversity loss and scalability in estimation of distribution genetic programming. IEEE Trans Evol Comput 17(3):301–320
Kim K-J, Park J-G, Cho S-B (2011) Correlation analysis and performance evaluation of distance measures for evolutionary neural networks. J Intell Fuzzy Syst 22:83–92
Kim KJ, Yoo JO, Cho SB (2005) Robust inference of Bayesian networks using speciated evolution and ensemble. In: International Symposium on Methodologies for Intelligent Systems, pp 92–101
Koller D, Friedman N (2009) Probabilistic graphical models: principles and techniques. MIT Press, Cambridge
Korb KB, Nicholson AE (2010) Bayesian artificial intelligence, 2nd edn. CRC Press, Boca Raton
Larranaga P, Karshenas H, Bielza C, Santana R (2013) A review on evolutionary algorithms in Bayesian network learning and inference tasks. Inf Sci 233(1):109–125
Larranaga P, Kuijpers CMH, Murga RH, Yurramendi Y (1996) Learning Bayesian network structures by searching for the best ordering with genetic algorithm. IEEE Trans Syst Man Cybern Part A 26(4):487–493
Larranaga P, Poza M, Yurramendi Y, Murga RH, Kuijpers CMH (1996) Structure learning of Bayesian networks by genetic algorithms: a performance analysis of control parameters. IEEE Trans Pattern Anal Mach Intell 18(9):912–926
Lauritzen S-L, Spiegelhalter DJ (1988) Local computations with probabilities on graphical structures and their applications on expert systems. J R Stat Soc B 50(2):157–224
Li XL, He XD, Yuan SM (2005) Learning Bayesian networks structures from incomplete data based on extending evolutionary programming. In: Proceedings of the Fourth International Conference on Machine Learning and Cybernetics, pp 2039–2043
Li W, Liu W, Yue K (2008) Recovering the global structure from multiple local Bayesian networks. Int J Artif Intell Tools 17(6):1067–1088
Luo X, Ouyang Y, Xiong Z (2011) Improving matrix factorization-based recommender via ensemble methods. Int J Inf Technol Decision Making 10(3):539–561
Mahfoud SW (1995) Niching methods for genetic algorithms. Ph.D. Dissertation, University of Illinois at Urbana-Champaign
Muruzabal J, Cotta C (2007) A Study on the evolution of Bayesian network graph structures. Adv Probab Graph Models 193–214
Myers JW, Laskey KB, Dejong KA (1999) Learning Bayesian networks from incomplete data using evolutionary algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp 458–465
Na Y, Yang J (2010) Distributed Bayesian network structure learning. In: IEEE International Symposium on Industrial Electronics, pp 1607–161
Pena JM, Kocka T, Nielsen JD (2004) Featuring multiple local optima to assist the user in the interpretation of induced Bayesian network models. In: Proceedings of the Tenth International Conference on Information Processing and Management of Uncertainty in Knowledge-based Systems, pp 1683–1690
Peng Y, Kou G, Wang G, Wu W, Shi Y (2011) Ensemble of software defect predictors: an AHP-based evaluation method. Int J Inf Technol Decision Making 10(1):187–206
Robles V, Larranaga P, Pena JM, Menasalvas E, Perez MS, Herves V, Wasilewska A (2004) Bayesian network multi-classifiers for protein secondary structure prediction. Artif Intell Med 31(2):117–136
Rogers A, Prugel-Bennett A (1999) Genetic drift in genetic algorithm selection schemes. IEEE Trans Evol Comput 3(4):298–303
Schadt EE, Linderman MD, Sorenson J, Lee L, Nolan GP (2010) Computational solutions to large-scale data management and analysis. Nat Rev Genet 11(9):647–657
Scutari M, Denis JB (2014) Bayesian networks: with examples in R. Chapman & Hall, London
Shen C-W (2009) A Bayesian networks approach to modeling financial risks of e-logistics investments. Int J Inf Technol Decision Making 8(4):711–726
Su X, Khoshgoftaar TM (2008) Collaborative filtering for multi-class data using Bayesian networks. Int J Artif Intell Tools 17(1):71–85
Vafaee F (2014) Learning the structure of large-scale Bayesian networks using genetic algorithm. In: Proceedings of the Conference on Genetic and Evolutionary Computation, pp 855–862
Wong ML, Lam W, Leung KS (1999) Using evolutionary programming and minimum description length principle for data mining of Bayesian networks. IEEE Trans Pattern Anal Mach Intell 21(2):174–178
Wong ML, Lee SY, Leung KS (2004) Data mining of Bayesian networks using cooperative coevolution. Decision Support Syst 38:451–472
Zhou ZH (2012) Ensemble methods: foundations and algorithms. Chapman & Hall/CRC, London
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIP) (2013 R1A2A2A01016589) and the Industrial Strategic Technology Development Program, 10044828, Development of augmenting multisensory technology for enhancing significant effect on service industry, funded by the Ministry of Trade, Industry & Energy (MI, Korea).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by V. Loia.
Rights and permissions
About this article
Cite this article
Kim, KJ., Cho, SB. Ensemble bayesian networks evolved with speciation for high-performance prediction in data mining. Soft Comput 21, 1065–1080 (2017). https://doi.org/10.1007/s00500-015-1841-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-015-1841-z