research-article

PSL: An Algorithm for Partial Bayesian Network Structure Learning

Authors:

Xindong WuAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data (TKDD), Volume 16, Issue 5

Article No.: 93, Pages 1 - 25

https://doi.org/10.1145/3508071

Published: 09 March 2022 Publication History

Abstract

Learning partial Bayesian network (BN) structure is an interesting and challenging problem. In this challenge, it is computationally expensive to use global BN structure learning algorithms, while only one part of a BN structure is interesting, local BN structure learning algorithms are not a favourable solution either due to the issue of false edge orientation. To address the problem, this article first presents a detailed analysis of the false edge orientation issue with local BN structure learning algorithms and then proposes PSL, an efficient and accurate Partial BN Structure Learning (PSL) algorithm. Specifically, PSL divides V-structures in a Markov blanket (MB) into two types: Type-C V-structures and Type-NC V-structures, then it starts from the given node of interest and recursively finds both types of V-structures in the MB of the current node until all edges in the partial BN structure are oriented. To further improve the efficiency of PSL, the PSL-FS algorithm is designed by incorporating Feature Selection (FS) into PSL. Extensive experiments with six benchmark BNs validate the efficiency and accuracy of the proposed algorithms.

References

[1]

Constantin F. Aliferis, Alexander Statnikov, Ioannis Tsamardinos, Subramani Mani, and Xenofon D. Koutsoukos. 2010. Local causal and markov blanket induction for causal discovery and feature selection for classification part ii: Analysis and extensions. Journal of Machine Learning Research 11, 1 (2010), 235–284.

Digital Library

[2]

Constantin F. Aliferis, Alexander Statnikov, Ioannis Tsamardinos, Subramani Mani, and Xenofon D. Koutsoukos. 2010. Local causal and markov blanket induction for causal discovery and feature selection for classification part i: Algorithms and empirical evaluation. Journal of Machine Learning Research 11, 1 (2010), 171–234.

Digital Library

[3]

Constantin F. Aliferis, Ioannis Tsamardinos, and Alexander Statnikov. 2003. HITON: A novel markov blanket algorithm for optimal variable selection. In Proceedings of the AMIA Annual Symposium. American Medical Informatics Association, 21–25.

[4]

Steen Andreassen, Annelise Rosenfalck, Bjørn Falck, Kristian G. Olesen, and Stig Kjær Andersen. 1996. Evaluation of the diagnostic performance of the expert EMG assistant MUNIN. Electroencephalography and Clinical Neurophysiology/Electromyography and Motor Control 101, 2 (1996), 129–144.

[5]

Ingo A. Beinlich, Henri Jacques Suermondt, R. Martin Chavez, and Gregory F. Cooper. 1989. The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks. In Proceedings of the AIME 89. Springer, 247–256.

[6]

David Maxwell Chickering. 2002. Learning equivalence classes of Bayesian-network structures. The Journal of Machine Learning Research 2, 2 (2002), 445–498.

Digital Library

[7]

David Maxwell Chickering, David Heckerman, and Christopher Meek. 2004. Large-sample learning of bayesian networks is NP-hard. Journal of Machine Learning Research 5, 10 (2004), 1287–1330.

Digital Library

[8]

Diego Colombo and Marloes H. Maathuis. 2014. Order-independent constraint-based causal structure learning. Journal of Machine Learning Research 15, 1 (2014), 3741–3782.

Digital Library

[9]

Gregory F. Cooper and Edward Herskovits. 1992. A bayesian method for the induction of probabilistic networks from data. Machine Learning 9, 4 (1992), 309–347.

[10]

Shunkai Fu and Michel C. Desmarais. 2008. Fast markov blanket discovery algorithm via local learning within single pass. In Proceedings of the Conference of the Canadian Society for Computational Studies of Intelligence. Springer, 96–107.

[11]

Tian Gao, Kshitij Fadnis, and Murray Campbell. 2017. Local-to-global bayesian network structure learning. In Proceedings of the International Conference on Machine Learning. 1193–1202.

[12]

Tian Gao and Qiang Ji. 2015. Local causal discovery of direct causes and effects. In Proceedings of the Advances in Neural Information Processing Systems. 2512–2520.

[13]

Tian Gao and Qiang Ji. 2017. Efficient markov blanket discovery and its application. IEEE Transactions on Cybernetics 47, 5 (2017), 1169–1179.

[14]

Tian Gao and Qiang Ji. 2017. Efficient score-based markov blanket discovery. International Journal of Approximate Reasoning 80, 1 (2017), 277–293.

Digital Library

[15]

Isabelle Guyon, Constantin Aliferis, and André Elisseeff. 2007. Causal feature selection. In Proceedings of the Computational Methods of Feature Selection. Chapman and Hall/CRC, 75–97.

[16]

Isabelle Guyon and André Elisseeff. 2003. An introduction to variable and feature selection. Journal of Machine Learning Research 3, 3 (2003), 1157–1182.

Digital Library

[17]

Zhaolong Ling, Kui Yu, Hao Wang, Lin Liu, Wei Ding, and Xindong Wu. 2019. Bamb: A balanced markov blanket discovery approach to feature selection. ACM Transactions on Intelligent Systems and Technology 10, 5 (2019), 1–25.

Digital Library

[18]

Zhaolong Ling, Kui Yu, Yiwen Zhang, Lin Liu, and Jiuyong Li. 2021. Causal learner: A toolbox for causal structure and markov blanket learning. arXiv:2103.06544. Retrieved from https://arxiv.org/abs/2103.06544.

[19]

Dimitris Margaritis and Sebastian Thrun. 2000. Bayesian network induction via local neighborhoods. In Proceedings of the Advances in Neural Information Processing Systems. 505–511.

[20]

Christopher Meek. 1995. Causal inference and causal explanation with background knowledge. In Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 403–410.

[21]

J. Pearl. 1988. Morgan kaufmann series in representation and reasoning. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann.

[22]

Judea Pearl. 2014. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann.

Digital Library

[23]

Mauro Scanagatta, Giorgio Corani, Cassio P. de Campos, and Marco Zaffalon. 2016. Learning treewidth-bounded bayesian networks with thousands of variables. In Proceedings of the Advances in Neural Information Processing Systems. 1462–1470.

[24]

Marco Scutari. 2010. Learning bayesian networks with the bnlearn r package. Journal of Statistical Software 35, 3 (2010), 1–22. Retrieved from http://www.jstatsoft.org/v35/i03/.

[25]

Marco Scutari and Adriana Brogini. 2012. Bayesian network structure learning with permutation tests. Communications in Statistics-Theory and Methods 41, 16–17 (2012), 3233–3243.

[26]

Marco Scutari, Catharina Elisabeth Graafland, and José Manuel Gutiérrez. 2019. Who learns better Bayesian network structures: Accuracy and speed of structure learning algorithms. International Journal of Approximate Reasoning 115, 12 (2019), 235–253.

Digital Library

[27]

Marco Scutari, Phil Howell, David J. Balding, and Ian Mackay. 2014. Multiple quantitative trait analysis using bayesian networks. Genetics 198, 1 (2014), 129–137.

[28]

Peter Spirtes, Clark N. Glymour, and Richard Scheines. 2000. Causation, Prediction, and Search. MIT press.

[29]

Ioannis Tsamardinos, Constantin F. Aliferis, and Alexander Statnikov. 2003. Time and sample efficient discovery of Markov blankets and direct causal relations. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 673–678.

Digital Library

[30]

Ioannis Tsamardinos, Laura E. Brown, and Constantin F. Aliferis. 2006. The max-min hill-climbing bayesian network structure learning algorithm. Machine Learning 65, 1 (2006), 31–78.

Digital Library

[31]

Diego Vidaurre, Concha Bielza, and Pedro Larrañaga. 2010. Learning an L1-regularized gaussian bayesian network in the equivalence class space. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 40, 5 (2010), 1231–1242.

Digital Library

[32]

Xianchao Xie, Zhi Geng, and Qiang Zhao. 2006. Decomposition of structural learning about directed acyclic graphs. Artificial Intelligence 170, 4–5 (2006), 422–439.

Digital Library

[33]

Shuai Yang, Hao Wang, Kui Yu, Fuyuan Cao, and Xindong Wu. 2021. Towards efficient local causal structure learning. IEEE Transactions on Big Data (2021). DOI:

[34]

Sandeep Yaramakala and Dimitris Margaritis. 2005. Speculative markov blanket discovery for optimal feature selection. In Proceedings of the 5th IEEE International Conference on Data Mining. IEEE, 809–812.

Digital Library

[35]

Jianxin Yin, You Zhou, Changzhang Wang, Ping He, Cheng Zheng, and Zhi Geng. 2008. Partial orientation and local structural learning of causal networks for prediction. In Proceedings of the Causation and Prediction Challenge. 93–105.

[36]

Kui Yu, Xianjie Guo, Lin Liu, Jiuyong Li, Hao Wang, Zhaolong Ling, and Xindong Wu. 2020. Causality-based feature selection: Methods and evaluations. ACM Computing Surveys 53, 5 (2020), 1–36.

Digital Library

[37]

Kui Yu, Lin Liu, and Jiuyong Li. 2021. A unified view of causal and non-causal feature selection. ACM Transactions on Knowledge Discovery from Data 15, 4 (2021), 1–46.

Digital Library

[38]

Kui Yu, Lin Liu, Jiuyong Li, Wei Ding, and Thuc Duy Le. 2019. Multi-source causal feature selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 9 (2019), 2240–2256.

Digital Library

[39]

Kui Yu, Xindong Wu, Wei Ding, and Jian Pei. 2016. Scalable and accurate online feature selection for big data. ACM Transactions on Knowledge Discovery from Data 11, 2 (2016), 1–39.

Digital Library

[40]

Lei Yu and Huan Liu. 2004. Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 10 (2004), 1205–1224.

Digital Library

[41]

Yue Yu, Jie Chen, Tian Gao, and Mo Yu. 2019. Dag-gnn: Dag structure learning with graph neural networks. In Proceedings of the International Conference on Machine Learning. 7154–7163.

[42]

Xun Zheng, Bryon Aragam, Pradeep K. Ravikumar, and Eric P. Xing. 2018. DAGs with NO TEARS: Continuous optimization for structure learning. In Proceedings of the 32nd International Conference on Neural Information Processing Systems. 9472–9483.

Cited By

Yu KRong CWang HCao FLiang J(2025)Federated local causal structure learningScience China Information Sciences10.1007/s11432-023-4203-668:3Online publication date: 16-Jan-2025
https://doi.org/10.1007/s11432-023-4203-6
Ling ZLi BZhang YZhou PWu XHuang YYu KWu X(2024)Causal Discovery Using Weight-Based Conditional Independence TestACM Transactions on Knowledge Discovery from Data10.1145/368746719:1(1-24)Online publication date: 28-Aug-2024
https://dl.acm.org/doi/10.1145/3687467
Liang JWang JYu GDomeniconi CZhang XGuo M(2024)Gradient-Based Local Causal Structure LearningIEEE Transactions on Cybernetics10.1109/TCYB.2023.323763554:1(486-495)Online publication date: Jan-2024
https://doi.org/10.1109/TCYB.2023.3237635
Show More Cited By

Index Terms

PSL: An Algorithm for Partial Bayesian Network Structure Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
      1. Causal reasoning and diagnostics

Recommendations

Feature Selection for Efficient Local-to-global Bayesian Network Structure Learning
Local-to-global learning approach plays an essential role in Bayesian network (BN) structure learning. Existing local-to-global learning algorithms first construct the skeleton of a DAG (directed acyclic graph) by learning the MB (Markov blanket) or PC (...
Discrete Bayesian Network Classifiers: A Survey

We have had to wait over 30 years since the naive Bayes model was first introduced in 1960 for the so-called Bayesian network classifiers to resurge. Based on Bayesian networks, these classifiers have many strengths, like model interpretability, ...
Efficient and effective Bayesian network local structure learning

In this paper, we propose a more efficient Bayesian network structure learning algorithm under the framework of score based local learning (SLL). Our algorithm significantly improves computational efficiency by restricting the neighbors of each variable ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 16, Issue 5

October 2022

532 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3514187

Editor:
Charu Aggarwal
IBM T.J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 March 2022

Accepted: 01 December 2021

Revised: 01 October 2021

Received: 01 July 2021

Published in TKDD Volume 16, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Key Research and Development Program of China
Natural Science Foundation of Anhui Province of China
Australian Research Council

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
537
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu KRong CWang HCao FLiang J(2025)Federated local causal structure learningScience China Information Sciences10.1007/s11432-023-4203-668:3Online publication date: 16-Jan-2025
https://doi.org/10.1007/s11432-023-4203-6
Ling ZLi BZhang YZhou PWu XHuang YYu KWu X(2024)Causal Discovery Using Weight-Based Conditional Independence TestACM Transactions on Knowledge Discovery from Data10.1145/368746719:1(1-24)Online publication date: 28-Aug-2024
https://dl.acm.org/doi/10.1145/3687467
Liang JWang JYu GDomeniconi CZhang XGuo M(2024)Gradient-Based Local Causal Structure LearningIEEE Transactions on Cybernetics10.1109/TCYB.2023.323763554:1(486-495)Online publication date: Jan-2024
https://doi.org/10.1109/TCYB.2023.3237635
Wang NLiu HZhang LCai YShi Q(2024)An efficient skeleton learning approach-based hybrid algorithm for identifying Bayesian network structureEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.108105133:PBOnline publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2024.108105

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents