Abstract
In the Probabilistic Graphical Model (PGM) community there is an interest around tractable models, i.e., those that can guarantee exact inference even at the price of expressiveness. Structure learning algorithms are interesting tools to automatically infer both these architectures and their parameters from data. Even if the resulting models are efficient at inference time, learning them can be very slow in practice. Here we focus on Cutset Networks (CNets), a recently introduced tractable PGM representing weighted probabilistic model trees with tree-structured models as leaves. CNets have been shown to be easy to learn, and yet fairly accurate. We propose a learning algorithm that aims to improve their average test log-likelihood while preserving efficiency during learning by adopting a random forest approach. We combine more CNets, learned in a generative Bayesian framework, into a generative mixture model. A thorough empirical comparison on real word datasets, against the original learning algorithms extended to our ensembling approach, proves the validity of our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Source code is available at http://www.di.uniba.it/~ndm/dcsn/.
- 2.
All experiments have been run on a 4-core Intel Xeon E312xx (Sandy Bridge) @2.0 GHz with 8 Gb of RAM and Ubuntu 14.04.1, kernel 3.13.0-39.
References
Ammar, S., Leray, P., Defourny, B., Wehenkel, L.: Probability density estimation by perturbing and combining tree structured Markov networks. In: Sossai, C., Chemello, G. (eds.) ECSQARU 2009. LNCS, vol. 5590, pp. 156–167. Springer, Heidelberg (2009)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Choi, M.J., Tan, V.Y.F., Anandkumar, A., Willsky, A.S.: Learning latent tree graphical models. J. Mach. Learn. Res. 12, 1771–1812 (2011)
Chow, C., Liu, C.: Approximating discrete probability distributions with dependence trees. IEEE Trans. Inf. Theory 14(3), 462–467 (1968)
Criminisi, A., Shotton, J., Konukoglu, E.: Decision forests: a unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends Comput. Graph. Vis. 7, 81–227 (2011)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 39(1), 1–38 (1977)
Di Mauro, N., Vergari, A., Esposito, F.: Learning accurate cutset networks by exploiting decomposability. In: Gavanelli, M., Lamma, E., Riguzzi, F. (eds.) AI*IA 2015: Advances in Artificial Intelligence (2015)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2–3), 131–163 (1997)
Haaren, J.V., Davis, J.: Markov network structure learning: a randomized feature generation approach. In: Proceedings of the 26th Conference on Artificial Intelligence. AAAI Press (2012)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2009)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)
Lowd, D., Davis, J.: Learning Markov network structure with decision trees. In: Proceedings of the 10th IEEE International Conference on Data Mining, pp. 334–343. IEEE Computer Society Press (2010)
Lowd, D., Domingos, P.: Learning arithmetic circuits. CoRR, abs/1206.3271 (2012)
Lowd, D., Rooshenas, A.: Learning Markov networks with arithmetic circuits. In: Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, JMLR Workshop Proceedings, vol. 31, pp. 406–414 (2013)
Lowd, D., Rooshenas, A.: The Libra Toolkit for Probabilistic Models. CoRR, abs/1504.00110 (2015)
Meilă, M., Jordan, M.I.: Learning with mixtures of trees. J. Mach. Learn. Res. 1, 1–48 (2000)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)
Poon, H., Domingos, P.: Sum-product network: a new deep architecture. In: NIPS 2010 Workshop on Deep Learning and Unsupervised Feature Learning (2011)
Rahman, T., Kothalkar, P., Gogate, V.: Cutset networks: a simple, tractable, and scalable approach for improving the accuracy of chow-liu trees. In: Calders, T., Esposito, F., Hüllermeier, E., Meo, R. (eds.) ECML PKDD 2014, Part II. LNCS, vol. 8725, pp. 630–645. Springer, Heidelberg (2014)
Ridgeway, G.: Looking for lumps: boosting and bagging for density estimation. Comput. Stat. Data Anal. 38(4), 379–392 (2002)
Rooshenas, A., Lowd, D.: Learning sum-product networks with direct and indirect variable interactions. In: Proceedings of the 31st International Conference on Machine Learning, JMLR Workshop and Conference Proceedings, pp. 710–718 (2014)
Roth, D.: On the hardness of approximate reasoning. Artif. Intell. 82(1–2), 273–302 (1996)
Acknowledgements
Work supported by the project PUGLIA@SERVICE (PON02 00563 3489339) financed by the Italian Ministry of University and Research (MIUR) and by the European Commission through the project MAESTRA, grant no. ICT-2013-612944.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Di Mauro, N., Vergari, A., Basile, T.M.A. (2015). Learning Bayesian Random Cutset Forests. In: Esposito, F., Pivert, O., Hacid, MS., Rás, Z., Ferilli, S. (eds) Foundations of Intelligent Systems. ISMIS 2015. Lecture Notes in Computer Science(), vol 9384. Springer, Cham. https://doi.org/10.1007/978-3-319-25252-0_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-25252-0_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25251-3
Online ISBN: 978-3-319-25252-0
eBook Packages: Computer ScienceComputer Science (R0)