Abstract
Bayesian networks are a widely used graphical model with diverse applications in knowledge discovery, classification, prediction, and control. Learning a Bayesian network from discrete data can be cast as a combinatorial optimization problem and there has been much previous work on applying optimization techniques including proposals based on ILP, A* search, depth-first branch-and-bound (BnB) search, and breadth-first BnB search. In this paper, we present a constraint-based depth-first BnB approach for solving the Bayesian network learning problem. We propose an improved constraint model that includes powerful dominance constraints, symmetry-breaking constraints, cost-based pruning rules, and an acyclicity constraint for effectively pruning the search for a minimum cost solution to the model. We experimentally evaluated our approach on a representative suite of benchmark data. Our empirical results compare favorably to the best previous approaches, both in terms of number of instances solved within specified resource bounds and in terms of solution time.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Witten, I.H., Frank, E., Hall, M.A.: Data Mining, 3rd edn. Morgan Kaufmann (2011)
Chickering, D., Meek, C., Heckerman, D.: Large-sample learning of Bayesian networks is NP-hard. In: Proc. of UAI, pp. 124–133 (2003)
Koivisto, M.: Parent assignment is hard for the MDL, AIC, and NML costs. In: Lugosi, G., Simon, H.U. (eds.) COLT 2006. LNCS (LNAI), vol. 4005, pp. 289–303. Springer, Heidelberg (2006)
Koivisto, M., Sood, K.: Exact Bayesian structure discovery in Bayesian networks. J. Mach. Learn. Res. 5, 549–573 (2004)
Silander, T., Myllymäki, P.: A simple approach for finding the globally optimal Bayesian network structure. In: Proc. of UAI, pp. 445–452 (2006)
Malone, B., Yuan, C., Hansen, E.A.: Memory-efficient dynamic programming for learning optimal Bayesian networks. In: Proc. of AAAI, pp. 1057–1062 (2011)
Jaakkola, T., Sontag, D., Globerson, A., Meila, M.: Learning Bayesian network structure using LP relaxations. In: Proc. of AISTATS, pp. 358–365 (2010)
Barlett, M., Cussens, J.: Advances in Bayesian network learning using integer programming. In: Proc. of UAI, pp. 182–191 (2013)
Yuan, C., Malone, B.: Learning optimal Bayesian networks: A shortest path perspective. J. of Artificial Intelligence Research 48, 23–65 (2013)
Fan, X., Malone, B., Yuan, C.: Finding optimal Bayesian network structures with constraints learned from data. In: Proc. of UAI, pp. 200–209 (2014)
Fan, X., Yuan, C.: An improved lower bound for Bayesian network structure learning. In: Proc. of AAAI (2015)
Tian, J.: A branch-and-bound algorithm for MDL learning Bayesian networks. In: Proc. of UAI, pp. 580–588 (2000)
Malone, B., Yuan, C.: A depth-first branch and bound algorithm for learning optimal bayesian networks. In: Croitoru, M., Rudolph, S., Woltran, S., Gonzales, C. (eds.) GKR 2013. LNCS, vol. 8323, pp. 111–122. Springer, Heidelberg (2014)
de Campos, C.P., Ji, Q.: Efficient structure learning of Bayesian networks using constraints. Journal of Machine Learning Research 12, 663–689 (2011)
Fan, X., Yuan, C., Malone, B.: Tightening bounds for Bayesian network structure learning. In: Proc. of AAAI, pp. 2439–2445 (2014)
Darwiche, A.: Modeling and Reasoning with Bayesian Networks. Cambridge University Press (2009)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. The MIT Press (2009)
Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6, 461–464 (1978)
Lam, W., Bacchus, F.: Using new data to refine a Bayesian network. In: Proc. of UAI, pp. 383–390 (1994)
Buntine, W.L.: Theory refinement of Bayesian networks. In: Proc. of UAI, pp. 52–60 (1991)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)
Larranaga, P., Kuijpers, C., Murga, R., Yurramendi, Y.: Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Trans. Syst., Man, Cybern. 26, 487–493 (1996)
Teyssier, M., Koller, D.: Ordering-based search: a simple and effective algorithm for learning Bayesian networks. In: Proc. of UAI, pp. 548–549 (2005)
Chickering, D.M.: A transformational characterization of equivalent Bayesian network structures. In: Proc. of UAI, pp. 87–98 (1995)
Chickering, D.M.: Learning equivalence classes of Bayesian network structures. Journal of Machine Learning Research 2, 445–498 (2002)
Michie, D.: "memo" functions and machine learning. Nature 218, 19–22 (1968)
Smith, B.M.: Caching search states in permutation problems. In: van Beek, P. (ed.) CP 2005. LNCS, vol. 3709, pp. 637–651. Springer, Heidelberg (2005)
Cussens, J.: Integer programming for Bayesian network structure learning. Quality Technology & Quantitative Management 1, 99–110 (2014)
Kitching, M., Bacchus, F.: Symmetric component caching. In: Proc. of IJCAI, pp. 118–124 (2007)
Kitching, M., Bacchus, F.: Exploiting decomposition in constraint optimization problems. In: Stuckey, P.J. (ed.) CP 2008. LNCS, vol. 5202, pp. 478–492. Springer, Heidelberg (2008)
Friedman, N.: Learning belief networks in the presence of missing values and hidden variables. In: Proc. of ICML, pp. 125–133 (1997)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
van Beek, P., Hoffmann, HF. (2015). Machine Learning of Bayesian Networks Using Constraint Programming. In: Pesant, G. (eds) Principles and Practice of Constraint Programming. CP 2015. Lecture Notes in Computer Science(), vol 9255. Springer, Cham. https://doi.org/10.1007/978-3-319-23219-5_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-23219-5_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23218-8
Online ISBN: 978-3-319-23219-5
eBook Packages: Computer ScienceComputer Science (R0)