Skip to main content
SpringerLink
Log in

Machine Learning of Bayesian Networks Using Constraint Programming

  • Conference paper
  • First Online:
Principles and Practice of Constraint Programming (CP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 9255))

Abstract

Bayesian networks are a widely used graphical model with diverse applications in knowledge discovery, classification, prediction, and control. Learning a Bayesian network from discrete data can be cast as a combinatorial optimization problem and there has been much previous work on applying optimization techniques including proposals based on ILP, A* search, depth-first branch-and-bound (BnB) search, and breadth-first BnB search. In this paper, we present a constraint-based depth-first BnB approach for solving the Bayesian network learning problem. We propose an improved constraint model that includes powerful dominance constraints, symmetry-breaking constraints, cost-based pruning rules, and an acyclicity constraint for effectively pruning the search for a minimum cost solution to the model. We experimentally evaluated our approach on a representative suite of benchmark data. Our empirical results compare favorably to the best previous approaches, both in terms of number of instances solved within specified resource bounds and in terms of solution time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Witten, I.H., Frank, E., Hall, M.A.: Data Mining, 3rd edn. Morgan Kaufmann (2011)

    Google Scholar 

  2. Chickering, D., Meek, C., Heckerman, D.: Large-sample learning of Bayesian networks is NP-hard. In: Proc. of UAI, pp. 124–133 (2003)

    Google Scholar 

  3. Koivisto, M.: Parent assignment is hard for the MDL, AIC, and NML costs. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 289–303. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  4. Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. J. Mach. Learn. Res. 5, 549–573 (2004)

    MATH  MathSciNet  Google Scholar 

  5. Silander, T., Myllymäki, P.: A simple approach for finding the globally optimal Bayesian network structure. In: Proc. of UAI, pp. 445–452 (2006)

    Google Scholar 

  6. Malone, B., Yuan, C., Hansen, E.A.: Memory-efficient dynamic programming for learning optimal Bayesian networks. In: Proc. of AAAI, pp. 1057–1062 (2011)

    Google Scholar 

  7. Jaakkola, T., Sontag, D., Globerson, A., Meila, M.: Learning Bayesian network structure using LP relaxations. In: Proc. of AISTATS, pp. 358–365 (2010)

    Google Scholar 

  8. Barlett, M., Cussens, J.: Advances in Bayesian network learning using integer programming. In: Proc. of UAI, pp. 182–191 (2013)

    Google Scholar 

  9. Yuan, C., Malone, B.: Learning optimal Bayesian networks: A shortest path perspective. J. of Artificial Intelligence Research 48, 23–65 (2013)

    MATH  MathSciNet  Google Scholar 

  10. Fan, X., Malone, B., Yuan, C.: Finding optimal Bayesian network structures with constraints learned from data. In: Proc. of UAI, pp. 200–209 (2014)

    Google Scholar 

  11. Fan, X., Yuan, C.: An improved lower bound for Bayesian network structure learning. In: Proc. of AAAI (2015)

    Google Scholar 

  12. Tian, J.: A branch-and-bound algorithm for MDL learning Bayesian networks. In: Proc. of UAI, pp. 580–588 (2000)

    Google Scholar 

  13. Malone, B., Yuan, C.: A depth-first branch and bound algorithm for learning optimal bayesian networks. In: Croitoru, M., Rudolph, S., Woltran, S., Gonzales, C. (eds.) GKR 2013. LNCS, vol. 8323, pp. 111–122. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  14. de Campos, C.P., Ji, Q.: Efficient structure learning of Bayesian networks using constraints. Journal of Machine Learning Research 12, 663–689 (2011)

    MATH  Google Scholar 

  15. Fan, X., Yuan, C., Malone, B.: Tightening bounds for Bayesian network structure learning. In: Proc. of AAAI, pp. 2439–2445 (2014)

    Google Scholar 

  16. Darwiche, A.: Modeling and Reasoning with Bayesian Networks. Cambridge University Press (2009)

    Google Scholar 

  17. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. The MIT Press (2009)

    Google Scholar 

  18. Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978)

    Article  MATH  Google Scholar 

  19. Lam, W., Bacchus, F.: Using new data to refine a Bayesian network. In: Proc. of UAI, pp. 383–390 (1994)

    Google Scholar 

  20. Buntine, W.L.: Theory refinement of Bayesian networks. In: Proc. of UAI, pp. 52–60 (1991)

    Google Scholar 

  21. Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)

    MATH  Google Scholar 

  22. Larranaga, P., Kuijpers, C., Murga, R., Yurramendi, Y.: Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Trans. Syst., Man, Cybern. 26, 487–493 (1996)

    Article  Google Scholar 

  23. Teyssier, M., Koller, D.: Ordering-based search: a simple and effective algorithm for learning Bayesian networks. In: Proc. of UAI, pp. 548–549 (2005)

    Google Scholar 

  24. Chickering, D.M.: A transformational characterization of equivalent Bayesian network structures. In: Proc. of UAI, pp. 87–98 (1995)

    Google Scholar 

  25. Chickering, D.M.: Learning equivalence classes of Bayesian network structures. Journal of Machine Learning Research 2, 445–498 (2002)

    MATH  MathSciNet  Google Scholar 

  26. Michie, D.: "memo" functions and machine learning. Nature 218, 19–22 (1968)

    Article  Google Scholar 

  27. Smith, B.M.: Caching search states in permutation problems. In: van Beek, P. (ed.) CP 2005. LNCS, vol. 3709, pp. 637–651. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  28. Cussens, J.: Integer programming for Bayesian network structure learning. Quality Technology & Quantitative Management 1, 99–110 (2014)

    Google Scholar 

  29. Kitching, M., Bacchus, F.: Symmetric component caching. In: Proc. of IJCAI, pp. 118–124 (2007)

    Google Scholar 

  30. Kitching, M., Bacchus, F.: Exploiting decomposition in constraint optimization problems. In: Stuckey, P.J. (ed.) CP 2008. LNCS, vol. 5202, pp. 478–492. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  31. Friedman, N.: Learning belief networks in the presence of missing values and hidden variables. In: Proc. of ICML, pp. 125–133 (1997)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, N2L 3G1, Canada

    Peter van Beek & Hella-Franziska Hoffmann

Authors
  1. Peter van Beek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hella-Franziska Hoffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter van Beek .

Editor information

Editors and Affiliations

  1. École Polytechnique de Montréal, Montréal, Québec, Canada

    Gilles Pesant

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

van Beek, P., Hoffmann, HF. (2015). Machine Learning of Bayesian Networks Using Constraint Programming. In: Pesant, G. (eds) Principles and Practice of Constraint Programming. CP 2015. Lecture Notes in Computer Science(), vol 9255. Springer, Cham. https://doi.org/10.1007/978-3-319-23219-5_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-23219-5_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-23218-8

  • Online ISBN: 978-3-319-23219-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation