Skip to main content
SpringerLink
Log in
Book cover

Probabilistic Inductive Logic Programming pp 92–117Cite as

Markov Logic

  • Chapter

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 4911))

Abstract

Most real-world machine learning problems have both statistical and relational aspects. Thus learners need representations that combine probability and relational logic. Markov logic accomplishes this by attaching weights to first-order formulas and viewing them as templates for features of Markov networks. Inference algorithms for Markov logic draw on ideas from satisfiability, Markov chain Monte Carlo and knowledge-based model construction. Learning algorithms are based on the conjugate gradient algorithm, pseudo-likelihood and inductive logic programming. Markov logic has been successfully applied to problems in entity resolution, link prediction, information extraction and others, and is the basis of the open-source Alchemy system.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Becker, S., Le Cun, Y.: Improving the convergence of back-propagation learning with second order methods. In: Proceedings of the 1988 Connectionist Models Summer School, San Mateo, CA, pp. 29–37. Morgan Kaufmann, San Francisco (1989)

    Google Scholar 

  2. Besag, J.: Statistical analysis of non-lattice data. The Statistician 24, 179–195 (1975)

    Article  Google Scholar 

  3. Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, UK, pp. 1319–1325. Morgan Kaufmann, San Francisco (2005)

    Google Scholar 

  4. Collins, M.: Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In: Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing, Philadelphia, PA, pp. 1–8. ACL (2002)

    Google Scholar 

  5. Damien, P., Wakefield, J., Walker, S.: Gibbs sampling for Bayesian non-conjugate and hierarchical models by auxiliary variables. Journal of the Royal Statistical Society, Series B 61 (1999)

    Google Scholar 

  6. De Raedt, L., Dehaspe, L.: Clausal discovery. Machine Learning 26, 99–146 (1997)

    Article  MATH  Google Scholar 

  7. Della Pietra, S., Della Pietra, V., Lafferty, J.: Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 380–392 (1997)

    Article  Google Scholar 

  8. Dietterich, T.: Experience with Markov logic networks in a large AI system. In: Probabilistic, Logical and Relational Learning - Towards a Synthesis, number 05051 in Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Dagstuhl, Germany (2007)

    Google Scholar 

  9. Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, Stockholm, Sweden, pp. 1300–1307. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  10. Genesereth, M.R., Nilsson, N.J.: Logical Foundations of Artificial Intelligence. Morgan Kaufmann, San Mateo (1987)

    MATH  Google Scholar 

  11. Getoor, L., Taskar, B. (eds.): Introduction to Statistical Relational Learning. MIT Press, Cambridge (2007)

    MATH  Google Scholar 

  12. Gilks, W.R., Richardson, S., Spiegelhalter, D.J.: Markov Chain Monte Carlo in Practice. Chapman and Hall, London, UK (1996)

    MATH  Google Scholar 

  13. Grenager, T., Klein, D., Manning, C.D.: Unsupervised learning of field segmentation models for information extraction. In: Proceedings of the Forty-Third Annual Meeting on Association for Computational Linguistics, Ann Arbor, Michigan, pp. 371–378. Association for Computational Linguistics (2005)

    Google Scholar 

  14. Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning 20, 197–243 (1995)

    MATH  Google Scholar 

  15. Jaimovich, A., Meshi, O., Friedman, N.: Template based inference in symmetric relational markov random fields. In: Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence, Vancouver, Canada, AUAI Press (2007)

    Google Scholar 

  16. Kautz, H., Selman, B., Jiang, Y.: A general stochastic approach to solving problems with hard and soft constraints. In: Gu, D., Du, J., Pardalos, P. (eds.) The Satisfiability Problem: Theory and Applications, pp. 573–586. American Mathematical Society, New York (1997)

    Google Scholar 

  17. Kersting, K., De Raedt, L.: Towards combining inductive logic programming with Bayesian networks. In: Proceedings of the Eleventh International Conference on Inductive Logic Programming, Strasbourg, France, pp. 118–131. Springer, Heidelberg (2001)

    Google Scholar 

  18. Kok, S., Domingos, P.: Learning the structure of Markov logic networks. In: Proceedings of the Twenty-Second International Conference on Machine Learning, Bonn, Germany, pp. 441–448. ACM Press, New York (2005)

    Google Scholar 

  19. Kok, S., Domingos, P.: Statistical predicate invention. In: Proceedings of the Twenty-Fourth International Conference on Machine Learning, Corvallis, OR, pp. 433–440. ACM Press, New York (2007)

    Google Scholar 

  20. Kok, S., Sumner, M., Richardson, M., Singla, P., Poon, H., Lowd, D., Domingos, P.: The Alchemy system for statistical relational AI. Technical report, Department of Computer Science and Engineering, University of Washington, Seattle, WA (2007), http://alchemy.cs.washington.edu

  21. Kushmerick, N.: Wrapper induction: Efficiency and expressiveness. Artificial Intelligence 118(1-2), 15–68 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  22. Lavrač, N., Džeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood, Chichester (1994)

    MATH  Google Scholar 

  23. Lawrence, S., Bollacker, K., Giles, C.L.: Autonomous citation matching. In: Proceedings of the Third International Conference on Autonomous Agents, ACM Press, New York (1999)

    Google Scholar 

  24. Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Mathematical Programming 45(3), 503–528 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  25. Lloyd, J.W.: Foundations of Logic Programming. Springer, Berlin, Germany (1987)

    MATH  Google Scholar 

  26. Lowd, D., Domingos, P.: Efficient weight learning for Markov logic networks. In: Proceedings of the Eleventh European Conference on Principles and Practice of Knowledge Discovery in Databases, Warsaw, Poland, pp. 200–211. Springer, Heidelberg (2007)

    Google Scholar 

  27. Lowd, D., Domingos, P.: Recursive random fields. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence, Hyderabad, India, AAAI Press, Menlo Park (2007)

    Google Scholar 

  28. Lunn, D.J., Thomas, A., Best, N., Spiegelhalter, D.: WinBUGS – a Bayesian modeling framework: Concepts, structure, and extensibility. Statistics and Computing 10, 325–337 (2000)

    Article  Google Scholar 

  29. Matuszek, C., Witbrock, M.: Personal communication (2006)

    Google Scholar 

  30. McCallum, A.: Efficiently inducing features of conditional random fields. In: Proceedings of the Nineteenth Conference on Uncertainty in Artificial Intelligence, Acapulco, Mexico, Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  31. Mihalkova, L., Mooney, R.: Bottom-up learning of Markov logic network structure. In: Proceedings of the Twenty-Fourth International Conference on Machine Learning, Corvallis, OR, pp. 625–632. ACM Press, New York (2007)

    Google Scholar 

  32. Møller, M.: A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks 6, 525–533 (1993)

    Article  Google Scholar 

  33. Muggleton, S.: Stochastic logic programs. In: De Raedt, L. (ed.) Advances in Inductive Logic Programming, pp. 254–264. IOS Press, Amsterdam, Netherlands (1996)

    Google Scholar 

  34. Neville, J., Jensen, D.: Dependency networks for relational data. In: Proceedings of the Fourth IEEE International Conference on Data Mining, Brighton, UK, pp. 170–177. IEEE Computer Society Press, Los Alamitos (2004)

    Chapter  Google Scholar 

  35. Nocedal, J., Wright, S.: Numerical Optimization. Springer, New York (2006)

    MATH  Google Scholar 

  36. Pasula, H., Marthi, B., Milch, B., Russell, S., Shpitser, I.: Identity uncertainty and citation matching. In: Advances in Neural Information Processing Systems 14, MIT Press, Cambridge (2002)

    Google Scholar 

  37. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)

    Google Scholar 

  38. Pearlmutter, B.: Fast exact multiplication by the Hessian. Neural Computation 6(1), 147–160 (1994)

    Article  Google Scholar 

  39. Poole, D.: First-order probabilistic inference. In: Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, Acapulco, Mexico, pp. 985–991. Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  40. Poon, H., Domingos, P.: Sound and efficient inference with probabilistic and deterministic dependencies. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, Boston, MA, pp. 458–463. AAAI Press, Menlo Park (2006)

    Google Scholar 

  41. Poon, H., Domingos, P.: Joint inference in information extraction. In: Proceedings of the Twenty-Second National Conference on Artificial Intelligence, Vancouver, Canada, pp. 913–918. AAAI Press, Menlo Park (2007)

    Google Scholar 

  42. Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)

    Google Scholar 

  43. Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286 (1989)

    Article  Google Scholar 

  44. Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–136 (2006)

    Article  Google Scholar 

  45. Riedel, S., Klein, E.: Genic interaction extraction with semantic and syntactic chains. In: Proceedings of the Fourth Workshop on Learning Language in Logic, Bonn, Germany, pp. 69–74. IMLS (2005)

    Google Scholar 

  46. Robinson, J.A.: A machine-oriented logic based on the resolution principle. Journal of the ACM 12, 23–41 (1965)

    Article  MATH  Google Scholar 

  47. Roth, D.: On the hardness of approximate reasoning. Artificial Intelligence 82, 273–302 (1996)

    Article  MathSciNet  Google Scholar 

  48. Sha, F., Pereira, F.: Shallow parsing with conditional random fields. In: Proceedings of the 2003 Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics (2003)

    Google Scholar 

  49. Shewchuck, J.: An introduction to the conjugate gradient method without the agonizing pain. Technical Report CMU-CS-94-125, School of Computer Science, Carnegie Mellon University (1994)

    Google Scholar 

  50. Singla, P., Domingos, P.: Discriminative training of Markov logic networks. In: aaai05, Pittsburgh, PA, pp. 868–873. AAAI Press, Menlo Park (2005)

    Google Scholar 

  51. Singla, P., Domingos, P.: Entity resolution with Markov logic. In: Proceedings of the Sixth IEEE International Conference on Data Mining, Hong Kong, pp. 572–582. IEEE Computer Society Press, Los Alamitos (2006)

    Chapter  Google Scholar 

  52. Singla, P., Domingos, P.: Memory-efficient inference in relational domains. In: Proceedings of the Twenty-First National Conference on Artificial Intelligence, Boston, MA, AAAI Press, Menlo Park (2006)

    Google Scholar 

  53. Singla, P., Domingos, P.: Markov logic in infinite domains. In: Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence, Vancouver, Canada, pp. 368–375. AUAI Press (2007)

    Google Scholar 

  54. Srinivasan, A.: The Aleph manual. Technical report, Computing Laboratory, Oxford University (2000)

    Google Scholar 

  55. Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge (1994)

    Google Scholar 

  56. Wei, W., Erenrich, J., Selman, B.: Towards efficient sampling: Exploiting random walk strategies. In: aaai04, San Jose, CA, AAAI Press, Menlo Park (2004)

    Google Scholar 

  57. Wellman, M., Breese, J.S., Goldman, R.P.: From knowledge bases to decision models. Knowledge Engineering Review 7 (1992)

    Google Scholar 

  58. Wellner, B., McCallum, A., Peng, F., Hay, M.: An integrated, conditional model of information extraction and coreference with application to citation matching. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence, Banff, Canada, pp. 593–601. AUAI Press (2004)

    Google Scholar 

  59. Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation. In: Leen, T., Dietterich, T., Tresp, V. (eds.) Advances in Neural Information Processing Systems 13, pp. 689–695. MIT Press, Cambridge (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, University of Washington, Seattle, WA, 98195-2350, USA

    Pedro Domingos, Stanley Kok, Daniel Lowd, Hoifung Poon & Parag Singla

  2. Microsoft Research, Redmond, WA, 98052,  

    Matthew Richardson

Authors
  1. Pedro Domingos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stanley Kok
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Lowd
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hoifung Poon
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Matthew Richardson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Parag Singla
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Luc De Raedt Paolo Frasconi Kristian Kersting Stephen Muggleton

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Domingos, P., Kok, S., Lowd, D., Poon, H., Richardson, M., Singla, P. (2008). Markov Logic. In: De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S. (eds) Probabilistic Inductive Logic Programming. Lecture Notes in Computer Science(), vol 4911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-78652-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-78652-8_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-78651-1

  • Online ISBN: 978-3-540-78652-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation