skip to main content
research-article

Probabilistic Rule Learning Systems: A Survey

Published: 03 May 2021 Publication History

Abstract

This survey provides an overview of rule learning systems that can learn the structure of probabilistic rules for uncertain domains. These systems are very useful in such domains because they can be trained with a small amount of positive and negative examples, use declarative representations of background knowledge, and combine efficient high-level reasoning with the probability theory. The output of these systems are probabilistic rules that are easy to understand by humans, since the conditions for consequences lead to predictions that become transparent and interpretable. This survey focuses on representational approaches and system architectures, and suggests future research directions.

References

[1]
Elena Bellodi and Fabrizio Riguzzi. 2013. Expectation maximization over binary decision diagrams for probabilistic logic programs. Intelligent Data Analysis 17, 2 (2013), 343--363.
[2]
Elena Bellodi and Fabrizio Riguzzi. 2015. Structure learning of probabilistic logic programs by searching the clause space. Theory and Practice of Logic Programming 15, 2 (2015), 169--212.
[3]
William W. Cohen. 2016. TensorLog: A differentiable deductive database. arXiv:1605.06523.
[4]
Andrew Cropper and Stephen H. Muggleton. 2016. Metagol system. Retrieved May 25, 2020 from https://github.com/metagol/metagol.
[5]
Luc De Raedt, Anton Dries, Ingo Thon, Guy Van den Broeck, and Mathias Verbeke. 2015. Inducing probabilistic relational rules from probabilistic examples. In Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI’15).
[6]
Luc De Raedt and Kristian Kersting. 2008. Probabilistic inductive logic programming. In Probabilistic Inductive Logic Programming: Theory and Applications, Luc De Raedt, Paolo Frasconi, Kristian Kersting, and Stephen Muggleton (Eds.). Vol. 4911. Springer, Berlin, Germany, 1--27.
[7]
Luc De Raedt and Angelika Kimmig. 2015. Probabilistic (logic) programming concepts. Machine Learning 100, 1 (2015), 5--47.
[8]
Anton Dries, Angelika Kimmig, Wannes Meert, Joris Renkens, Guy Van den Broeck, Jonas Vlasselaer, and Luc De Raedt. 2015. Problog2: Probabilistic logic programming. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 312--315.
[9]
Kevin Ellis, Lucas Morales, Mathias Sablé-Meyer, Armando Solar-Lezama, and Josh Tenenbaum. 2018. Learning libraries of subroutines for neurally--guided Bayesian program induction. In Advances in Neural Information Processing Systems. 7805--7815.
[10]
Arnaud Nguembang Fadja and Fabrizio Riguzzi. 2019. Lifted discriminative learning of probabilistic logic programs. Machine Learning 108, 7 (2019), 1111--1135.
[11]
Artur d’Avila Garcez, Marco Gori, Luis C. Lamb, Luciano Serafini, Michael Spranger, and Son N. Tran. 2019. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv:1905.06088.
[12]
Arcchit Jain, Tal Friedman, Ondrej Kuzelka, Guy Van den Broeck, and Luc De Raedt. 2019. Scalable rule learning in probabilistic knowledge bases. In Proceedings of the 1st Conference on Automated Knowledge Base Construction (AKBC’19).
[13]
Ashwin Kalyan, Abhishek Mohta, Oleksandr Polozov, Dhruv Batra, Prateek Jain, and Sumit Gulwani. 2018. Neural-guided deductive search for real-time program synthesis from examples. In Proceedings of the International Conference on Learning Representations.
[14]
Muhammad Taimoor Khan and Howard Shrobe. 2019. Security of cyberphysical systems: Chaining induction and deduction. Computer 52, 7 (2019), 72--75.
[15]
Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. 2018. DeepProbLog: Neural probabilistic logic programming. In Advances in Neural Information Processing Systems. 3749--3759.
[16]
Eric McCreath and Arun Sharma. 1995. Extraction of meta-knowledge to restrict the hypothesis space for ILP systems. In Proceedings of the 8th Australian Joint Conference on Artificial Intelligence. 75--82.
[17]
Jack Minker. 1982. On indefinite databases and the closed world assumption. In Proceedings of the International Conference on Automated Deduction. 292--308.
[18]
Stephen Muggleton. 1991. Inductive logic programming. New Generation Computing 8, 4 (1991), 295--318.
[19]
Stephen Muggleton. 1995. Inverse entailment and Progol. New Generation Computing 13, 3--4 (1995), 245--286.
[20]
Stephen Muggleton. 1996. Learning from positive data. In Proceedings of the International Conference on Inductive Logic Programming. 358--376.
[21]
Stephen Muggleton, Wang-Zhou Dai, Claude Sammut, Alireza Tamaddoni-Nezhad, Jing Wen, and Zhi-Hua Zhou. 2018. Meta-interpretive learning from noisy images. Machine Learning 107, 7 (2018), 1097--1118.
[22]
Stephen Muggleton and Luc De Raedt. 1994. Inductive logic programming: Theory and methods. Journal of Logic Programming 19 (1994), 629--679.
[23]
Stephen H. Muggleton, Dianhuan Lin, and Alireza Tamaddoni-Nezhad. 2015. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited. Machine Learning 100, 1 (2015), 49--73.
[24]
Stephen H. Muggleton, Ute Schmid, Christina Zeller, Alireza Tamaddoni-Nezhad, and Tarek Besold. 2018. Ultra-strong machine learning: Comprehensibility of programs learned with ILP. Machine Learning 107, 7 (2018), 1119--1140.
[25]
David Poole. 2003. First-order probabilistic inference. In Proceedings of the International Joint Conference on Artificial Intelligence, Vol. 3. 985--991.
[26]
J. Ross Quinlan. 1990. Learning logical definitions from relations. Machine Learning 5, 3 (1990), 239--266.
[27]
Johannes Rabold, Hannah Deininger, Michael Siebers, and Ute Schmid. 2019. Enriching visual with verbal explanations for relational concepts—Combining LIME with Aleph. In Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 180--192.
[28]
Luc De Raedt, Sebastijan Dumancic, Robin Manhaeve, and Giuseppe Marra. 2020. From statistical relational to neuro-symbolic artificial intelligence. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI’20). 4943--4950.
[29]
Luc De Raedt, Kristian Kersting, Sriraam Natarajan, and David Poole. 2016. Statistical Relational Artificial Intelligence: Logic, Probability, and Computation. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool.
[30]
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. Why should I trust you? Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 1135--1144.
[31]
Fabrizio Riguzzi. 2018. Foundations of Probabilistic Logic Programming. River Publishers.
[32]
Fabrizio Riguzzi and Damiano Azzolini. 2020. cplint Documentation. SWI-Prolog Version. Retrieved May 25, 2020 from http://friguzzi.github.io/cplint/_build/latex/cplint.pdf.
[33]
Fabrizio Riguzzi, Elena Bellodi, and Riccardo Zese. 2014. A history of probabilistic inductive logic programming. Frontiers in Robotics and AI 1 (2014), 6.
[34]
Fabrizio Riguzzi and Theresa Swift. 2018. A survey of probabilistic logic programming. In Declarative Logic Programming: Theory, Systems, and Applications. ACM Books, 185--228.
[35]
Tim Rocktäschel and Sebastian Riedel. 2017. End-to-end differentiable proving. In Advances in Neural Information Processing Systems. 3788--3800.
[36]
Tim Rocktäschel, Sameer Singh, and Sebastian Riedel. 2015. Injecting logical background knowledge into embeddings for relation extraction. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 1119--1129.
[37]
Taisuke Sato. 1995. A statistical learning method for logic programs with distribution semantics. In Proceedings of the 12th International Conference on Logic Programming (ICLP’95). 715--729.
[38]
Taisuke Sato and Yoshitaka Kameya. 1997. PRISM: A language for symbolic-statistical modeling. In Proceedings of the International Joint Conference on Artificial Intelligence, Vol. 97. 1330--1339.
[39]
Maria I. Sessa. 2002. Approximate reasoning by similarity-based SLD resolution. Theoretical Computer Science 275, 1--2 (2002), 389--426.
[40]
Ashwin Srinivasan. 2007. The Aleph Manual. Retrieved May 25, 2020 from http://www.cs.ox.ac.uk/activities/programinduction/Aleph/aleph.html.
[41]
Joost Vennekens, Sofie Verbaeten, and Maurice Bruynooghe. 2004. Logic programs with annotated disjunctions. In Proceedings of the International Conference on Logic Programming. 431--445.
[42]
Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni. 2020. Generalizing from a few examples: A survey on few-shot learning. ACM Computing Surveys 53, 3 (2020), 1--34.
[43]
Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, and Tim Rocktäschel. 2019. NLProlog: Reasoning with weak unification for question answering in natural language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 6151--6161.
[44]
Dirk Weissenborn, Tomáš Kočiskỳ, and Chris Dyer. 2017. Dynamic integration of background knowledge in neural NLU systems. arXiv:1706.02596.
[45]
Fan Yang, Zhilin Yang, and William W. Cohen. 2017. Differentiable learning of logical rules for knowledge base reasoning. In Advances in Neural Information Processing Systems. 2319--2328.

Cited By

View all
  • (2024)Towards a neuro-symbolic cycle for human-centered explainabilityNeurosymbolic Artificial Intelligence10.3233/NAI-240740(1-13)Online publication date: 28-Aug-2024
  • (2023)Logical Rule-Based Knowledge Graph Reasoning: A Comprehensive SurveyMathematics10.3390/math1121448611:21(4486)Online publication date: 30-Oct-2023
  • (2023)A General Paradigm of Knowledge-driven and Data-driven Fusion2023 15th International Conference on Advanced Computational Intelligence (ICACI)10.1109/ICACI58115.2023.10146138(1-7)Online publication date: 6-May-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys
ACM Computing Surveys  Volume 54, Issue 4
May 2022
782 pages
ISSN:0360-0300
EISSN:1557-7341
DOI:10.1145/3464463
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 May 2021
Accepted: 01 January 2021
Revised: 01 November 2020
Received: 01 May 2020
Published in CSUR Volume 54, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Probabilistic rule learning
  2. probabilistic logic programming
  3. sub-symbolic rule learning
  4. symbolic rule learning

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)60
  • Downloads (Last 6 weeks)3
Reflects downloads up to 18 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Towards a neuro-symbolic cycle for human-centered explainabilityNeurosymbolic Artificial Intelligence10.3233/NAI-240740(1-13)Online publication date: 28-Aug-2024
  • (2023)Logical Rule-Based Knowledge Graph Reasoning: A Comprehensive SurveyMathematics10.3390/math1121448611:21(4486)Online publication date: 30-Oct-2023
  • (2023)A General Paradigm of Knowledge-driven and Data-driven Fusion2023 15th International Conference on Advanced Computational Intelligence (ICACI)10.1109/ICACI58115.2023.10146138(1-7)Online publication date: 6-May-2023

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media