Skip to main content
SpringerLink
Log in

Improving greedy local search methods by switching the search space

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Bayesian networks play a vital role in human understanding of the world. Finding a precise equivalence class of a Bayesian network is an effective way to represent causality. However, as one of the most widely used methods of searching for equivalence classes, greedy equivalence search (GES), can easily fall into a local optimum. To address this problem, we explore the reasons why GES becomes stuck in a local optimum by analyzing its operators and search strategies in detail. Moreover, we demonstrate that converting the search space into another space can address the drawbacks of local search in the space of the equivalence class. Accordingly, two novel frameworks based on switching spaces are proposed to improve GES. Finally, the effectiveness, scalability, and stability of the proposed methods are verified by extensive experiments through which our frameworks are compared with state-of-the-art methods on different benchmarks. The results show that our methods significantly strengthen the performance of GES.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data Availibility Statement

The data used in this manuscript are generated from public networks from https://www.bnlearn.com/bnreposi-tory/.

Notes

  1. Since variables are represented by nodes in the graph, we do not distinguish between nodes and variables

  2. Also called a pattern in [9].

  3. This is also called the essential graph in [46]

  4. https://www.bnlearn.com/bnrepository/

  5. https://github.com/fishmoon1234/DAG-GNN

  6. https://www.cs.york.ac.uk/aig/sw/gobnilp/

References

  1. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, (1988)

  2. Nott C, Ölçmen SM, Karr CL, Trevino LC (2007) Sr-30 turbojet engine real-time sensor health monitoring using neural networks, and bayesian belief networks. Applied Intelligence 26(3):251–265

    Article  Google Scholar 

  3. Mani S, Valtorta M, McDermott S (2005) Building bayesian network models in medicine: The mentor experience. Applied Intelligence 22(2):93–108

    Article  Google Scholar 

  4. Zhang B, Gaiteri C, Bodea L-G, Wang Z, McElwee J, Podtelezhnikov AA, Zhang C, Xie T, Tran L, Dobrin R et al (2013) Integrated systems approach identifies genetic nodes and networks in late-onset alzheimer’s disease. Cell 153(3):707–720

    Article  Google Scholar 

  5. Runge J, Bathiany S, Bollt E, Camps-Valls G, Coumou D, Deyle E, Glymour C, Kretschmer M, Mahecha MD, Muñoz-Marí J et al (2019) Inferring causation from time series in earth system sciences. Nature communications 10(1):1–13

    Article  Google Scholar 

  6. Gao X-G, Mei J-F, Chen H-Y, Chen D-Q (2014) Approximate inference for dynamic bayesian networks: sliding window approach. Applied intelligence 40(4):575–591

    Article  Google Scholar 

  7. Luo G, Zhao B, Du S (2019) Causal inference and bayesian network structure learning from nominal data. Applied Intelligence 49(1):253–264

    Article  Google Scholar 

  8. Chickering M, Heckerman D, Meek C (2004) Large-sample learning of bayesian networks is np-hard. Journal of Machine Learning Research 5:1287–1330

    MathSciNet  MATH  Google Scholar 

  9. Spirtes, P., Glymour, C.N., Scheines, R., Heckerman, D.: Causation, Prediction, and Search, (2000)

  10. Colombo D, Maathuis MH et al (2014) Order-independent constraint-based causal structure learning. J. Mach. Learn. Res. 15(1):3741–3782

    MathSciNet  MATH  Google Scholar 

  11. Le TD, Hoang T, Li J, Liu L, Liu H, Hu S (2016) A fast pc algorithm for high dimensional causal discovery with multi-core pcs. IEEE/ACM transactions on computational biology and bioinformatics 16(5):1483–1495

    Article  Google Scholar 

  12. Schwarz, G.: Estimating the dimension of a model. The annals of statistics, 461–464 (1978)

  13. Suzuki, J.: A construction of bayesian networks from databases based on an mdl principle. In: Uncertainty in Artificial Intelligence, pp. 266–273 (1993). Elsevier

  14. Buntine, W.: Theory refinement on bayesian networks. In: Uncertainty Proceedings 1991, pp. 52–60 (1991)

  15. De Campos CP, Ji Q (2011) Efficient structure learning of bayesian networks using constraints. The Journal of Machine Learning Research 12:663–689

    MathSciNet  MATH  Google Scholar 

  16. Cussens, J.: Bayesian network learning with cutting planes. In: Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence (UAI 2011), pp. 153–160 (2011). AUAI Press

  17. Yuan, C., Malone, B., Wu, X.: Learning optimal bayesian networks using a* search. In: Twenty-Second International Joint Conference on Artificial Intelligence (2011)

  18. Wang Z, Gao X, Yang Y, Tan X, Chen D (2021) Learning bayesian networks based on order graph with ancestral constraints. Knowledge-Based Systems 211:106515

    Article  Google Scholar 

  19. Tan X, Gao X, Wang Z, Han H, Liu X, Chen D (2022) Learning the structure of bayesian networks with ancestral and/or heuristic partition. Information Sciences 584:719–751

    Article  Google Scholar 

  20. Chickering, M., Geiger, D., Heckerman, D.: Learning bayesian networks: Search methods and experimental results. In: Proceedings of the Fifth International Workshop on Artificial Intelligence and Statistics (1995)

  21. MEEK, C.: Casual inference and causal explanation with background knowledge. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 403–410 (1995). Morgan Kaufmann

  22. Chickering, D.M.: Optimal structure identification with greedy search. Journal of machine learning research 3(Nov), 507–554 (2002)

  23. Chickering DM (2002) Learning equivalence classes of bayesian-network structures. The Journal of Machine Learning Research 2:445–498

    MathSciNet  MATH  Google Scholar 

  24. Teyssier, M., Koller, D.: Ordering-based search: a simple and effective algorithm for learning bayesian networks. In: Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, pp. 584–590 (2005)

  25. Lee S, Kim SB (2019) Parallel simulated annealing with a greedy algorithm for bayesian network structure learning. IEEE Transactions on Knowledge and Data Engineering 32(6):1157–1166

    Article  Google Scholar 

  26. Ye Q, Amini AA, Zhou Q (2020) Optimizing regularized cholesky score for order-based learning of bayesian networks. IEEE transactions on pattern analysis and machine intelligence 43(10):3555–3572

    Article  Google Scholar 

  27. Wang Z, Gao X, Tan X, Liu X (2021) Determining the direction of the local search in topological ordering space for bayesian network structure learning. Knowledge-Based Systems 234:107566

    Article  Google Scholar 

  28. Xie F, Cai R, Zeng Y, Gao J, Hao Z (2019) An efficient entropy-based causal discovery method for linear structural equation models with iid noise variables. IEEE transactions on neural networks and learning systems 31(5):1667–1680

    Article  MathSciNet  Google Scholar 

  29. Zheng, X., Aragam, B., Ravikumar, P.K., Xing, E.P.: Dags with no tears: Continuous optimization for structure learning. Advances in Neural Information Processing Systems 31 (2018)

  30. Yu, Y., Chen, J., Gao, T., Yu, M.: Dag-gnn: Dag structure learning with graph neural networks. In: International Conference on Machine Learning, pp. 7154–7163 (2019). PMLR

  31. Zhu, S., Ng, I., Chen, Z.: Causal discovery with reinforcement learning. In: International Conference on Learning Representations (2019)

  32. Zhang, M., Jiang, S., Cui, Z., Garnett, R., Chen, Y.: D-vae: A variational autoencoder for directed acyclic graphs. Advances in Neural Information Processing Systems 32 (2019)

  33. Nielsen, J.D., Kocka, T., Pena, J.M.: On local optima in learning bayesian networks. In: Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence, pp. 435–442 (2003)

  34. Alonso-Barba JI, Gámez JA, Puerta JM et al (2013) Scaling up the greedy equivalence search algorithm by constraining the search space of equivalence classes. International journal of approximate reasoning 54(4):429–451

    Article  MathSciNet  MATH  Google Scholar 

  35. Nandy P, Hauser A, Maathuis MH (2018) High-dimensional consistency in score-based and hybrid structure learning. The Annals of Statistics 46(6A):3151–3183

    Article  MathSciNet  MATH  Google Scholar 

  36. Ramsey J, Glymour M, Sanchez-Romero R, Glymour C (2017) A million variables and more: the fast greedy equivalence search algorithm for learning high-dimensional graphical causal models, with an application to functional magnetic resonance images. International journal of data science and analytics 3(2):121–129

  37. Alonso JI, de la Ossa L, Gamez JA, Puerta JM (2018) On the use of local search heuristics to improve ges-based bayesian network learning. Applied Soft Computing 64:366–376

    Article  Google Scholar 

  38. Lourenço, H.R., Martin, O.C., Stützle, T.: Iterated local search. In: Handbook of Metaheuristics, pp. 320–353 (2003)

  39. Mladenović N, Hansen P (1997) Variable neighborhood search. Computers & operations research 24(11):1097–1100

    Article  MathSciNet  MATH  Google Scholar 

  40. De Campos LM, Fernández-Luna JM, Puerta JM (2003) An iterated local search algorithm for learning bayesian networks with restarts based on conditional independence tests. International Journal of Intelligent Systems 18(2):221–235

    Article  MATH  Google Scholar 

  41. Alonso-Barba, J.I., delaOssa, L., Puerta, J.M.: Structural learning of bayesian networks using local algorithms based on the space of orderings. Soft Computing 15(10), 1881–1895 (2011)

  42. Scanagatta, M., de Campos, C.P., Corani, G., Zaffalon, M.: Learning bayesian networks with thousands of variables. Advances in neural information processing systems 28 (2015)

  43. Lee, C., Beek, P.v.: Metaheuristics for score-and-search bayesian network structure learning. In: Canadian Conference on Artificial Intelligence, pp. 129–141 (2017). Springer

  44. Puerta JM, Aledo JA, Gámez JA, Laborda JD (2021) Efficient and accurate structural fusion of bayesian networks. Information Fusion 66:155–169

    Article  Google Scholar 

  45. VERMA, T.: Equivalence and synthesis of causal models. In: Proceedings of the Sixth Conference on Uncertainty in Artificial Intelligence, 1991 (1991). Elsevier

  46. Andersson SA, Madigan D, Perlman MD (1997) A characterization of markov equivalence classes for acyclic digraphs. The Annals of Statistics 25(2):505–541

    Article  MathSciNet  MATH  Google Scholar 

  47. Dor, D., Tarsi, M.: A simple algorithm to construct a consistent extension of a partially oriented graph. Technicial Report R-185, Cognitive Systems Laboratory, UCLA (1992)

  48. Chickering, D.M.: A transformational characterization of equivalent bayesian network structures. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 87–98 (1995)

  49. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques, (2009)

  50. Jiang L, Zhang L, Li C, Wu J (2018) A correlation-based feature weighting filter for naive bayes. IEEE transactions on knowledge and data engineering 31(2):201–213

    Article  Google Scholar 

  51. Jiang L, Zhang L, Yu L, Wang D (2019) Class-specific attribute weighted naive bayes. Pattern recognition 88:321–330

    Article  Google Scholar 

  52. Nadeau, C., Bengio, Y.: Inference for the generalization error. Advances in neural information processing systems 12 (1999)

  53. Fan G-F, Zhang L-Z, Yu M, Hong W-C, Dong S-Q (2022) Applications of random forest in multivariable response surface for short-term load forecasting. International Journal of Electrical Power & Energy Systems 139:108073

    Article  Google Scholar 

  54. Scanagatta, M., Corani, G., Zaffalon, M.: Improved local search in bayesian networks structure learning. In: Advanced Methodologies for Bayesian Networks, pp. 45–56 (2017). PMLR

  55. Raftery, A.E.: Bayesian model selection in social research. Sociological methodology, 111–163 (1995)

Download references

Funding

This work was supported by the National Natural Science Foundation of China (61573285). This work was also Sponsored by Innovation Foundation for Doctor Dissertation of Northwestern Polytechnical University (CX2022047). This work was also supported by the Fundamental Research Funds for the Central Universities (G2022KY0602).

Author information

Authors and Affiliations

  1. School of Electronics and Information, Northwestern Polytechnical University, Dongxiang Road, Xi’an, 710072, Shaanxi, China

    Xiaohan Liu, Xiaoguang Gao, Xinxin Ru & Zidong Wang

  2. AVIC Chengdu Aircraft Design and Research Institute, Riyue Road, Chengdu, 610091, Sichuan, China

    Xiangyuan Tan

Authors
  1. Xiaohan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoguang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xinxin Ru
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiangyuan Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zidong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Xiaohan Liu: Conceptualization, Software, Methodology, Writing- Original draft preparation. XiaoGuang Gao: Methodology, Supervision. Xinxin Ru: Software ,Validation. Xiangyuan Tan: Writing-Reviewing and Editing. Zidong Wang: Software, Validation.

Corresponding author

Correspondence to Xiaoguang Gao.

Ethics declarations

Ethics approval and informed consent

The manuscript does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This is not required as no human participants are involved.

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Gao, X., Ru, X. et al. Improving greedy local search methods by switching the search space. Appl Intell 53, 22143–22160 (2023). https://doi.org/10.1007/s10489-023-04693-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04693-3

Keywords

Search

Navigation