Skip to main content
SpringerLink
Log in

Bayesian belief network learning algorithms for modeling contextual relationships in natural imagery: a comparative study

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

This paper presents a comparative study of Bayesian belief network structure learning algorithms with a view to identify a suitable algorithm for modeling the contextual relations among objects typically found in natural imagery. Four popular structure learning algorithms are compared: two constraint-based algorithms (PC proposed by Spirtes and Glymour and Fast Incremental Association Markov Blanket proposed by Yaramakala and Margaritis), a score-based algorithm (Hill Climbing as implemented by Daly), and a hybrid algorithm (Max-Min Hill Climbing proposed by Tsamardinos et al.). Contrary to the belief regarding the superiority of constraint-based approaches, our empirical results show that a score-based approach performs better on our context dataset in terms of prediction power and learning time. The hybrid algorithm could achieve similar prediction performance as the score-based approach, but requires longer time to learn the desired network. Another interesting fact the study has revealed is the existence of strong correspondence between the linear correlation pattern within the dataset and the edges found in the learned networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abul Hasanat MH, Ramachandram D, Rajeswari M (2009) ConVeS: a context verification framework for object recognition system. In: ISTA ’09: proceedings of the 2009 conference on information science, technology and applications, Kuwait. ACM, New York, USA, pp 78–83

  • Acid S, de Campos LM, Fernández-Luna JM, Rodríguez S, Rodríguez JM, Salcedo JL (2004) A comparison of learning algorithms for Bayesian networks: a case study based on data from an emergency medical service. Artif Intell Med 30(3): 215–232 Bayesian Networks in Biomedicince and Health-Care

    Article  Google Scholar 

  • Alcobé JR (2007) Learning Bayesian networks with an approximated MDL score. In: Lucas P, Gmez JA, Salmern A (eds) Advances in probabilistic graphical models, volume 214/2007 of studies in fuzziness and soft computing. Springer, Berlin, pp 215–234

    Google Scholar 

  • Blake C (1998) UCI repository of machine learning databases. Available at: http://ci.nii.ac.jp/naid/10018044499/en/

  • Brown L, Tsamardinos I, Aliferis C (2005) A comparison of novel and state-of-the-art polynomial Bayesian network learning algorithms. In: Proceedings of the national conference on artificial intelligence, vol. 20. AAAI Press; MIT Press; Menlo Park; Cambridge, 1999, pp 739

  • Cheng J, Greiner R, Kelly J, Bell D, Liu W (2002) Learning Bayesian networks from data: an information-theory based approach. Artif Intell 137: 43–90

    Article  MATH  MathSciNet  Google Scholar 

  • Cheng J, Greiner R (1999) Comparing Bayesian network classifiers. In: Proceedings of the 15th conference on uncertainty in artificial intelligence (UAIâ99), pp 101–107

  • Cheng J, Greiner R (2001) Learning Bayesian belief network classifiers: algorithms and system. Adv Artif Intell 2056/2001: 141–151

    Article  Google Scholar 

  • Chickering DM, Heckerman D, Meek C (2004) Large-sample learning of Bayesian networks is NP-Hard. J Mach Learn Res 5: 1287–1330

    MathSciNet  Google Scholar 

  • Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9(4): 309–347

    MATH  Google Scholar 

  • Cowell RG (2001) Conditions under which conditional independence and scoring methods lead to identical selection of Bayesian network models. In: UAI ’01: Proceedings of the 17th conference in uncertainty in artificial Intelligence, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc, Los Altos, pp 91–97

  • Daly R, Shen Q (2007) Methods to accelerate the learning of Bayesian network structures. In: Proceedings of the 2007 UK workshop on computational intelligence. Citeseer

  • de Campos LM, Fernández-Luna JM, Huete JF (2004) Bayesian networks and information retrieval: an introduction to the special issue. Inf Process Manag 40(5): 727–733 Bayesian Networks and Information Retrieval

    Article  Google Scholar 

  • Friedman N, Geiger D, Goldszmidt M (1997) Bayesian network classifiers. Mach Learn 29(2): 131–163

    Article  MATH  Google Scholar 

  • Friedman N, Goldszmidt M (1996) Building classifiers using Bayesian networks. In: Proceedings of the national conference on artificial intelligence, pp 1277–1284

  • Friedman N, Linial M, Nachman I, Pe’er D (2000) Using Bayesian networks to analyze expression data. J Comput Biol 7(3–4): 601–620

    Article  Google Scholar 

  • Galleguillos C, Rabinovich A, Belongie S (2008) Object categorization using co-occurrence, location and appearance. In: IEEE conference on computer vision and pattern recognition

  • Heckerman D, Geiger D, Chickering DM (1995) Learning Bayesian networks: the combination of knowledge and statistical data. Mach Learn 20(3): 197–243

    MATH  Google Scholar 

  • Heckerman D, Meek C, Cooper G (1999) Computation, causation, and discovery, Chapter A Bayesian approach to causal discovery. AAAI Press, pp 143–67

  • Jeon G, Falcon R, Kim D, Lee R, Jeong J (2008) Application of Bayesian belief network in reliable analysis for video deinterlacing. IEEE Trans Consumer Electron 54(1): 123–130

    Article  Google Scholar 

  • Ji J-Z, Zhang H-X, Hu R-B, Liu C-N (2009) A Bayesian network learning algorithm based on independence test and ant colony optimization. Acta Automatica Sinica 35(3): 281–288

    Article  MATH  Google Scholar 

  • Kabli R, Herrmann F, McCall J (2007) A chain-model genetic algorithm for Bayesian network structure learning. In: GECCO ’07: proceedings of the 9th annual conference on genetic and evolutionary computation, ACM, New York, NY, USA, pp 1264–1271

  • Koller D, Sahami M (1996) Toward optimal feature selection. In: Machine learning international workshop then conference, Citeseer, pp 284–292.

  • Kullback S (1997) Information theory and statistics. Dover publications, Mineola

    MATH  Google Scholar 

  • Kullback S, Leibler R (1951) On information and sufficiency. Ann Math Stat 22(1): 79–86

    Article  MATH  MathSciNet  Google Scholar 

  • Lam W, Bacchus F (1994) Learning Bayesian belief networks: an approach based on the MDL principle. Comput Intell 10(3): 269–293

    Article  Google Scholar 

  • Langley P, Iba W, Thompson K (1992) An analysis of Bayesian classifiers. In: Proceedings of the tenth national conference on artificial intelligence. MIT Press, pp 223–228

  • Lauría EJ, Duchessi PJ (2006) A Bayesian belief network for it implementation decision support. Decis Support Syst 42(3): 1573–1588

    Article  Google Scholar 

  • Lee KJ, Chang W (2009) Bayesian belief network for box-office performance: a case study on Korean movies. Expert Syst Appl 36(1): 280–291

    Article  Google Scholar 

  • Leray P, Francois O (2004) BNT structure learning package: documentation and experiments. Technical Report 2004/PhLOF, PSI, LITIS Laboratory, INSA de Rouen, Avenue de l’Université—BP 8, 76801 Saint-Étienne-du-Rouvray Cedex

  • Madden M (2002) A new Bayesian network structure for classification tasks. Artif Intell Cogn Sci 2464/2002: 183–197

    Google Scholar 

  • Madden MG (2002b) Evaluation of the performance of the markov blanket Bayesian classifier algorithm. Technical Report NUIG-IT-011002, Department of Information Technology, National University of Ireland, Galway, Ireland

  • Madden MG (2003) The performance of Bayesian network classifiers constructed using different techniques. In: Proceedings of European conference on machine learning, workshop on probabilistic graphical models for classification, pp 59–70

  • Margaritis D (2003) Learning Bayesian network model structure from data. Ph. D. thesis, School of Computer Science, Carnegie-Mellon University, Pittsburgh, PA. Available as Technical Report CMU-CS-03-153

  • Murphy KP (2001) The Bayes net toolbox for Matlab. Comput Sci Stat 33(2): 1024–1034

    Google Scholar 

  • Niculescu-Mizil A, Caruana R (2007) Inductive transfer for Bayesian network structure learning. In: Proceedings of the 11th international conference on AI and statistics (AISTATS 2007), vol. 2, pp 339–346

  • Pelikan M, Goldberg D (2006) Hierarchical Bayesian optimization algorithm. In: Scalable optimization via probabilistic modeling, volume 33/2006 of studies in computational intelligence. Springer, Berlin, pp 63–90

  • Pham D, Ruz G (2009) Unsupervised training of Bayesian networks for data clustering. Proc Royal Soci A 465(2109): 2927–2948

    Article  MATH  MathSciNet  Google Scholar 

  • Russell BC, Torralba A, Murphy KP, Freeman WT (2008) LabelMe: a database and web-based tool for image annotation. Int J Comput Vision 77(1–3): 157–173

    Article  Google Scholar 

  • Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2): 461–464

    Article  MATH  Google Scholar 

  • Scutari M (2009) bnlearn: Bayesian network structure learning. R package version 1.6. Available at: http://www.bnlearn.com/

  • Spirtes P, Glymour C, Scheines R (2000) Causation, prediction, and search, 2nd edn. MIT press, Cambridge

    Google Scholar 

  • Tape TG (2009) The area under an ROC curve. Available at: http://gim.unmc.edu/dxtests/roc3.htm. Accessed on February 02, 2010

  • Tsamardinos I, Aliferis CF, Statnikov A (2003a) Algorithms for large scale markov blanket discovery. In: Proceedings of the sixteenth international Florida artificial intelligence research society conference. AAAI Press, pp 376–381

  • Tsamardinos I, Aliferis CF, Statnikov A (2003b) Time and sample efficient discovery of markov blankets and direct causal relations. In: KDD ’03: proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 673–678

  • Tsamardinos I, Brown L, Aliferis C (2006) The max- min hill-climbing Bayesian network structure learning algorithm. Mach Learn 65(1): 31–78

    Article  Google Scholar 

  • Verma T, Pearl J (1990) Equivalence and synthesis of causal models. In: Proceedings of the sixth annual conference on uncertainty in artificial intelligence table of contents. Elsevier Science Inc. New York, pp 255–270

  • Wemmenhove B, Mooij J, Wiegerinck W, Leisink M, Kappen H, Neijt J (2007) Inference in the promedas medical expert system. In: Artificial intelligence in medicine, volume 4594/2007 of lecture notes in computer science. Springer, Berlin, pp 456–460

  • Wong ML, Guo YY (2008) Learning Bayesian networks from incomplete databases using a novel evolutionary algorithm. Decis Support Syst 45(2): 368–383 I.T. and Value Creation

    Article  Google Scholar 

  • Yaramakala S, Margaritis D (2005) Speculative markov blanket discovery for optimal feature selection. In: Fifth IEEE international conference on Data Mining, p 4

Download references

Author information

Authors and Affiliations

  1. Computer Vision Research Group, School of Computer Sciences, Universiti Sains Malaysia, 11800, Pulau, Penang, Malaysia

    Mozaherul Hoque Abul Hasanat, Dhanesh Ramachandram & Rajeswari Mandava

Authors
  1. Mozaherul Hoque Abul Hasanat
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dhanesh Ramachandram
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajeswari Mandava
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mozaherul Hoque Abul Hasanat.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Abul Hasanat, M.H., Ramachandram, D. & Mandava, R. Bayesian belief network learning algorithms for modeling contextual relationships in natural imagery: a comparative study. Artif Intell Rev 34, 291–308 (2010). https://doi.org/10.1007/s10462-010-9176-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-010-9176-8

Keywords

Search

Navigation