Skip to main content
SpringerLink
Log in

Probabilistic modelling, inference and learning using logical theories

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

This paper provides a study of probabilistic modelling, inference and learning in a logic-based setting. We show how probability densities, being functions, can be represented and reasoned with naturally and directly in higher-order logic, an expressive formalism not unlike the (informal) everyday language of mathematics. We give efficient inference algorithms and illustrate the general approach with a diverse collection of applications. Some learning issues are also considered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Allison, L.: Models for machine learning and data mining in functional programming. J. Funct. Program. 15(1), 15–32 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  2. Andrews, P.B.: An Introduction to Mathematical Logic and Type Theory: To Truth Through Proof. Academic, London (1986)

    MATH  Google Scholar 

  3. Baral, C., Gelfond, M., Rushton, J.N.: Probabilistic reasoning with answer sets. In: Proceedings of the 7th International Conference on Logic Programming and Nonmonotonic Reasoning, pp. 21–33, Fort Lauderdale, 6–8 January 2004

  4. Boole, G.: An Investigation of the Laws of Thought on which are founded the Mathematical Theories of Logic and Probabilities. Walton and Maberly, London (1854)

    Google Scholar 

  5. Boole, G.: Studies in Logic and Probability. Watts, London (1952)

    MATH  Google Scholar 

  6. Church, A.: A formulation of the simple theory of types. J. Symb. Log. 5, 56–68 (1940)

    Article  MATH  MathSciNet  Google Scholar 

  7. Clark, K.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and Databases, pp. 293–322. Plenum, New York (1978)

    Google Scholar 

  8. Cole, J.J., Gray, M., Lloyd, J.W., Ng, K.S.: Personalisation for user agents. In: Proceedings of the 4th International Joint Conference on Autonomous Agents and Multi Agent Systems, pp. 603–610, Utrecht, 25–29 July 2005

  9. De Raedt, L., Kersting, K.: Probabilistic logic learning. SIGKDD Explorations 5(1), 31–48 (2003)

    Article  Google Scholar 

  10. de Salvo Braz, R.: Lifted First-Order Probabilistic Inference. PhD thesis, University of Illinois at Urbana-Champaign (2007)

  11. de Salvo Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence, pp. 1319–1325, Edinburgh, 30 July–5 August 2005

  12. de Salvo Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning, chapter 15. MIT, Cambridge (2007)

    Google Scholar 

  13. Dekhtyar, A., Subrahmanian, V.S.: Hybrid probabilistic programs. J. Log. Program. 43(3), 187–250 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  14. Dowek, G.: Higher-order unification and matching. In: Handbook of automated reasoning, pp. 1009–1062. Elsevier Science, Amsterdam (2001)

    Chapter  Google Scholar 

  15. Ebbinghaus, H.D., Flum, J., Thomas, W.: Mathematical Logic. Springer, New York (1984)

    MATH  Google Scholar 

  16. Enderton, H.B.: A Mathematical Introduction to Logic, 2nd edn. Harcourt Academic, San Diego (2001)

    MATH  Google Scholar 

  17. Erwig, M., Kollmansberger, S.: Probabilistic functional programming in Haskell. J. Funct. Program. 16(1), 21–34 (2006)

    Article  MATH  Google Scholar 

  18. Fagin, R., Halpern, J.Y.: Reasoning about knowledge and probability. J. ACM 41(2), 340–367 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  19. Farmer, W.M.: The seven virtues of simple type theory. J. Appl. Logic 6(3), 267–286 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  20. Gelfond, M., Lifschitz, V.: Classical negation in logic programs and disjunctive databases. New Gener. Comput. 9, 365–387 (1991)

    Google Scholar 

  21. Goodman, N., Mansinghka, V.K., Roy, D., Bonawitz, K., Tenenbaum, J.B.: Church: a language for generative models. In: McAllester, D.A., Myllymäki, P. (eds.) Proceedings of the 24th Conference in Uncertainty in Artificial Intelligence, pp. 220–229. AUAI, Corvallis (2008)

    Google Scholar 

  22. Gyftodimos, E., Flach, P.A.: Combining Bayesian networks with higher-order data representations. In: 6th International Symposium on Intelligent Data Analysis, pp. 145–156, Madrid, 8–10 September 2005

  23. Hailperin, T.: Sentential Probability Logic. Lehigh University Press, Bethlehem (1996)

    MATH  Google Scholar 

  24. Hájek, A.: Probability, logic and probability logic. In: Goble, L. (ed.) The Blackwell Guide to Philosophical Logic, chapter 16, pp. 362–384. Blackwell, Oxford (2001)

    Google Scholar 

  25. Halpern, J.Y.: An analysis of first-order logics of probability. Artif. Intell. 46(3), 311–350 (1990)

    Article  MATH  Google Scholar 

  26. Halpern, J.Y.: Reasoning about Uncertainty. MIT, Cambridge (2003)

    MATH  Google Scholar 

  27. Henkin, L.: Completeness in the theory of types. J. Symb. Log. 15(2), 81–91 (1950)

    Article  MATH  MathSciNet  Google Scholar 

  28. Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(D), 35–45 (1960)

    Google Scholar 

  29. Kersting, K., De Raedt, L.: Bayesian logic programming: theory and tool. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning, chapter 10. MIT, Cambridge (2007)

    Google Scholar 

  30. Kschischang, F.R., Frey, B.J., Loeliger, H.-A.: Factor graphs and the sum-product algorithm. IEEE Trans. Inf. Theory 47(2) (2001)

  31. Lakshmanan, L.V.S., Sadri, F.: Modeling uncertainty in deductive databases. In: Karagiannis, D. (ed.) Proceedings of the International Conference on Database and Expert Systems Applications, DEXA’94, pp. 724–733, Athens, 7–9 September 1994

  32. Leivant, D.: Higher-order logic. In: Gabbay, D.M., Hogger, C.J., Robinson, J.A., Siekmann, J. (eds.) Handbook of Logic in Artificial Intelligence and Logic Programming, vol. 2, pp. 230–321. Oxford University Press, Oxford (1994)

    Google Scholar 

  33. Lloyd, J.W.: Programming in an integrated functional and logic language. J. Funct. Logic Program. 3 (1999)

  34. Lloyd, J.W.: Knowledge Representation, Computation, and Learning in Higher-order Logic. http://rsise.anu.edu.au/~jwl/ (2002)

  35. Lloyd, J.W.: Logic for Learning: Learning Comprehensible Theories from Structured Data. Springer, New York (2003)

    MATH  Google Scholar 

  36. Lloyd, J.W.: Knowledge Representation and Reasoning in Modal Higher-order Logic. http://rsise.anu.edu.au/~jwl (2007)

  37. Lloyd, J.W., Ng, K.S.: Learning modal theories. In: Muggleton, S., Otero, R., Tamaddoni-Nezhad, A. (eds.) Proceedings of the 16th International Conference on Inductive Logic Programming. LNAI, vol. 4455, pp. 320–334 (2007)

  38. Lloyd, J.W., Ng, K.S., Veness, J.: Modal Functional Logic Programming. http://rsise.anu.edu.au/~kee (2007)

  39. Lloyd, J.W., Sears, T.D.: An architecture for rational agents. In: Declarative Agent Languages and Technologies (DALT-2005). LNAI, vol. 3904, pp. 51–71. Springer, New York (2006)

    Chapter  Google Scholar 

  40. Milch, B., Marthi, B., Russell, S., Sontag, D., Ong, D.L., Kolobov, A.: Blog: probabilistic models with unknown objects. In: Kaelbling, L.P., Saffiotti, A. (eds.) Proceedings of the 19th International Joint Conference on Artificial Intelligence, pp. 1352–1359 (2005)

  41. Milch, B., Russell, S.: First-order probabilistic languages: into the unknown. In: Muggleton, S., Otero, R., Tamaddoni-Nezhad, A. (eds.) Proceedings of the 16th International Conference on Inductive Logic Programming. LNAI, vol. 4455, pp. 10–24 (2007)

  42. Muggleton, S.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS, Amsterdam (1996)

    Google Scholar 

  43. Ng, K.S.: Learning Comprehensible Theories from Structured Data. PhD thesis, Computer Sciences Laboratory, The Australian National University (2005)

  44. Ng, K.S., Lloyd, J.W.: Probabilistic reasoning in a classical logic. J. Appl. Logic (2008). doi:10.1016/j.jal.2007.11.008

    Google Scholar 

  45. Ng, R.T., Subrahmanian, V.S.: Probabilistic logic programming. Inf. Comput. 101(2), 150–201 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  46. Nilsson, N.J.: Probabilistic logic. Artif. Intell. 28(1), 71–88 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  47. Park, S., Pfenning, F., Thrun, S.: A probabilistic language based upon sampling functions. In: Conference Record of the 32nd Annual ACM Symposium on Principles of Programming Languages (2005)

  48. Pfeffer, A.: The design and implementation of IBAL: a general-purpose probabilistic language. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning, chapter 14. MIT, Cambridge (2007)

    Google Scholar 

  49. Poole, D.: Logic, knowledge representation, and Bayesian decision theory. In: Proceedings of the 1st International Conference on Computational Logic. LNCS, vol. 1861, pp. 70–86 (2000)

  50. Poole, D.: First-order probabilistic inference. In: Proceedings of the 18th International Joint Conference on Artificial Intelligence, pp. 985–991 (2003)

  51. Ramsey, N., Pfeffer, A.: Stochastic lambda calculus and monads of probability distributions. In: Conference Record of the 29th Annual ACM Symposium on Principles of Programming Languages (2002)

  52. Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62, 107–136 (2006)

    Article  Google Scholar 

  53. Sato, T., Kameya, Y.: PRISM: a language for symbolic-statistical modeling. In: Proceedings of the 15th International Joint Conference on Artificial Intelligence, pp. 1330–1335 (1997)

  54. Shapiro, S.: Classical logic II—higher-order logic. In: Goble, L. (ed.) The Blackwell Guide to Philosophical Logic, pp. 33–54. Blackwell, Oxford (2001)

    Google Scholar 

  55. Shirazi, A., Amir, E.: Probabilistic modal logic. In: Holte, R.C., Howe, A. (eds.) Proceedings of the 22nd AAAI Conference on Artificial Intelligence, pp. 489–495 (2007)

  56. Thompson, S.: Type Theory and Functional Programming. Addison-Wesley, Reading (1991)

    MATH  Google Scholar 

  57. van Benthem, J., Doets, K.: Higher-order logic. In: Gabbay, D.M., Guenther, F. (eds.) Handbook of Philosophical Logic, vol. 1, pp. 275–330. Reidel, Dordrecht (1983)

    Google Scholar 

  58. Williamson, J.: Probability logic. In: Gabbay, D., Johnson, R., Ohlbach, H.J., Woods, J. (eds.) Handbook of the Logic of Inference and Argument: The Turn Toward the Practical. Studies in Logic and Practical Reasoning, vol. 1, pp. 397–424. Elsevier, Amsterdam (2002)

    Chapter  Google Scholar 

  59. Williamson, J.: Philosophies of probability. In: Irvine, A., (ed.) Handbook of the Philosophy of Mathematics. Handbook of the Philosophy of Science, vol. 4. Elsevier, Amsterdam (2009, in press)

    Google Scholar 

  60. Zhang, N.L., Poole, D.: A simple approach to Bayesian network computations. In: Proceedings of the 10th Biennial Canadian Artificial Intelligence Conference, pp. 171–178 (1994)

Download references

Author information

Authors and Affiliations

  1. National ICT Australia and The Australian National University, Acton, Australian Capital Territory, Australia

    K. S. Ng

  2. College of Engineering and Computer Science, The Australian National University, Acton, Australian Capital Territory, Australia

    J. W. Lloyd

  3. National ICT Australia and University of New South Wales, Kensington, New South Wales, Australia

    W. T. B. Uther

Authors
  1. K. S. Ng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. W. Lloyd
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. W. T. B. Uther
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. S. Ng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ng, K.S., Lloyd, J.W. & Uther, W.T.B. Probabilistic modelling, inference and learning using logical theories. Ann Math Artif Intell 54, 159–205 (2008). https://doi.org/10.1007/s10472-009-9136-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-009-9136-7

Keywords

Mathematics Subject Classifications (2000)

Search

Navigation