Abstract
This article presents MCTS-BN, an adaptation of the Monte Carlo Tree Search (MCTS) algorithm for the structural learning of Bayesian Networks (BNs). Initially designed for game tree exploration, MCTS has been repurposed to address the challenge of learning BN structures by exploring the search space of potential ancestral orders in Bayesian Networks. Then, it employs Hill Climbing (HC) to derive a Bayesian Network structure from each order. In large BNs, where the search space for variable orders becomes vast, using completely random orders during the rollout phase is often unreliable and impractical. We adopt a semi-randomized approach to address this challenge by incorporating variable orders obtained from other heuristic search algorithms such as Greedy Equivalent Search (GES), PC, or HC itself. This hybrid strategy mitigates the computational burden and enhances the reliability of the rollout process. Experimental evaluations demonstrate the effectiveness of MCTS-BN in improving BNs generated by traditional structural learning algorithms, exhibiting robust performance even when base algorithm orders are suboptimal and surpassing the gold standard when provided with favorable orders.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alonso, J.I., de la Ossa, L., Gámez, J.A., Puerta, J.M.: On the use of local search heuristics to improve GES-based Bayesian network learning. Appl. Soft Comput. 64, 366–376 (2018)
Bai, F., Ju, X., Wang, S., Zhou, W., Liu, F.: Wind farm layout optimization using adaptive evolutionary algorithm with Monte Carlo Tree Search reinforcement learning. Energy Convers. Manage. 252, 115047 (2022)
Bryant, P., Pozzati, G., Zhu, W., Shenoy, A., Kundrotas, P., Elofsson, A.: Predicting the structure of large protein complexes using AlphaFold and Monte Carlo tree search. Nat. Commun. 13(1) (2022)
Chaslot, G.M.J.-B., Winands, M.H.M., van den Herik, H.J.: Parallel Monte-carlo tree search. In: van den Herik, H.J., Xu, X., Ma, Z., Winands, M.H.M. (eds.) CG 2008. LNCS, vol. 5131, pp. 60–71. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87608-3_6
Chickering, D.M.: Optimal structure identification with greedy search. J. Mach. Learn. Res. 3, 507–554 (2002)
Fenton, N., Neil, M.: Risk Assessment and Decision Analysis with Bayesian Networks. Chapman and Hall/CRC (2018)
Gámez, J.A., Mateo, J.L., Puerta, J.M.: Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood. Data Min. Knowl. Disc. 22(1), 106–148 (2011)
Heckerman, D., Geiger, D., Chickering, D.M.: Learning bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)
Howard, R.A.: Dynamic Programming and Markov Processes. MIT Press, Cambridge, MA (1960)
Jensen, F.V., Nielsen, T.D.: Bayesian Networks and Decision Graphs, 2nd edn. Springer, New York (2007)
Kajita, S., Kinjo, T., Nishi, T.: Autonomous molecular design by Monte-Carlo tree search and rapid evaluations using molecular dynamics simulations. Commun. Phy. 3(1) (2020)
Kjaerulff, U.B., Madsen, A.L.: Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis. Springer Publishing Company, 2nd edn. (2013)
Kocsis, L., Szepesvári, C.: Bandit Based Monte-Carlo Planning, pp. 282–293. Springer Berlin Heidelberg (2006)
Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning. The MIT Press (2009)
Labbe, Y., et al.: Monte-Carlo tree search for efficient visually guided rearrangement planning. IEEE Rob. Autom. Lett. 5(2), 3715–3722 (2020)
Leurent, E., Maillard, O.A.: Monte-carlo graph search: the value of merging similar states. In: Pan, S.J., Sugiyama, M. (eds.) Proceedings of The 12th Asian Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 129, pp. 577–592. PMLR (2020)
Li, A., van Beek, P.: Bayesian network structure learning with side constraints. In: Kratochvíl, V., Studený, M. (eds.) Proceedings of the Ninth International Conference on Probabilistic Graphical Models. Proceedings of Machine Learning Research, vol. 72, pp. 225–236. PMLR (2018)
McLachlan, S., Dube, K., Hitman, G.A., Fenton, N.E., Kyrimi, E.: Bayesian networks in healthcare: distribution by medical condition. Artif. Intell. Med. 107, 101912 (2020)
Mo, S., Pei, X., Wu, C.: Safe reinforcement learning for autonomous vehicle using monte carlo tree search. IEEE Trans. Intell. Transp. Syst. 23(7), 6766–6773 (2022)
Perez, D., Mostaghim, S., Samothrakis, S., Lucas, S.M.: Multiobjective monte carlo tree search for real-time games. IEEE Trans. Comput. Intell. AI Games 7(4), 347–360 (2015)
Ramsey, J., Glymour, M., Sanchez-Romero, R., Glymour, C.: A million variables and more: the fast greedy equivalence search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images. Int. J. Data Sci. Anal. 3, 121–129 (2017)
Scanagatta, M., Salmerón, A., Stella, F.: A survey on bayesian network structure learning from data. Prog. Artif. Intell. 8(4), 425–439 (2019)
Scutari, M.: Learning bayesian networks with the bnlearn R Package. J. Stat. Softw. 35(3), 1–22 (2010)
Sevinc, V., Kucuk, O., Goltas, M.: A bayesian network model for prediction and analysis of possible forest fire causes. For. Ecol. Manage. 457, 117723 (2020)
Silver, D., Huang, A., Maddison, C.J., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)
Silver, D., Hubert, T., Schrittwieser, J., et al.: A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362(6419), 1140–1144 (2018)
Spirtes, P., Glymour, C., Scheimes, R.: Causation. Prediction and Search. Springer- Verlag, New York, USA (1993)
Weng, D., Chen, R., Zhang, J., Bao, J., Zheng, Y., Wu, Y.: Pareto-optimal transit route planning with multi-objective monte-carlo tree search. IEEE Trans. Intell. Transp. Syst. 22(2), 1185–1195 (2021)
Xie, X., et al.: New theoretical ISM-K2 Bayesian network model for evaluating vaccination effectiveness. J. Ambient. Intell. Humaniz. Comput. 14(9), 12789–12805 (2022)
Świechowski, M., Godlewski, K., Sawicki, B., Mańdziuk, J.: Monte Carlo Tree Search: a review of recent modifications and applications. Artif. Intell. Rev. 56(3), 2497–2562 (2022)
Acknowledgements
The following projects have funded this work: SBPLY/21/180225/000062 by the Government of Castilla-La Mancha and “ERDF A way of making Europe”; PID2022-139293NB-C32, TED2021-131291B-I00 and FPU21/01074 by MCIN/AEI/10.13039/501100011033 and “ESF Investing your future”; 2022-GRIN-34437 and 2019-PREDUCLM-10188 by Universidad de Castilla-La Mancha and ERDF funds.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Laborda, J.D., Torrijos, P., Puerta, J.M., Gámez, J.A. (2024). Enhancing Bayesian Network Structural Learning with Monte Carlo Tree Search. In: Lesot, MJ., et al. Information Processing and Management of Uncertainty in Knowledge-Based Systems. IPMU 2024. Lecture Notes in Networks and Systems, vol 1174. Springer, Cham. https://doi.org/10.1007/978-3-031-74003-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-031-74003-9_32
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-74002-2
Online ISBN: 978-3-031-74003-9
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)