Skip to main content
SpringerLink
Log in

Learning first-order probabilistic models with combining rules

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

Many real-world domains exhibit rich relational structure and stochasticity and motivate the development of models that combine predicate logic with probabilities. These models describe probabilistic influences between attributes of objects that are related to each other through known domain relationships. To keep these models succinct, each such influence is considered independent of others, which is called the assumption of “independence of causal influences” (ICI). In this paper, we describe a language that consists of quantified conditional influence statements and captures most relational probabilistic models based on directed graphs. The influences due to different statements are combined using a set of combining rules such as Noisy-OR. We motivate and introduce multi-level combining rules, where the lower level rules combine the influences due to different ground instances of the same statement, and the upper level rules combine the influences due to different statements. We present algorithms and empirical results for parameter learning in the presence of such combining rules. Specifically, we derive and implement algorithms based on gradient descent and expectation maximization for different combining rules and evaluate them on synthetic data and on a real-world task. The results demonstrate that the algorithms are able to learn both the conditional probability distributions of the influence statements and the parameters of the combining rules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Altendorf, E.E., Restificar, A.C., Dietterich, T.G.: Learning from sparse data by exploiting monotonicity constraints. In: Proceedings of UAI 05 (2005)

  2. Binder, J., Koller, D., Russell, S., Kanazawa, K.: Adaptive probabilistic networks with hidden variables. Mach. Learn. 29(2–3), 213–244 (1997) ISSN 0885-6125

    Article  MATH  Google Scholar 

  3. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39, 185–197 (1977)

    MathSciNet  Google Scholar 

  4. Díez, F.J., Galán, S.F.: Efficient computation for the noisy MAX. Int. J. Approx. Reason. 18, 165–177 (2003)

    MATH  Google Scholar 

  5. Domingos, P., Richardson, M.: Markov logic: a unifying framework for statistical relational learning. In: Proceedings of the SRL Workshop in ICML, Banff, July 2004

  6. Dragunov, A.N., Dietterich, T.G., Johnsrude, K., McLaughlin, M., Li, L., Herlocker, J.L.: Tasktracer: a desktop environment to support multi-tasking knowledge workers. In: Proceedings of IUI, San Diego, January 2005

  7. Fierens, D., Blockeel, H., Bruynooghe, M., Ramon, J.: Logical Bayesian networks and their relation to other probabilistic logical models. In: Proceedings of ILP, Bonn, 10–13 August 2005

  8. Getoor, L., Grant, J.: PRL: a probabilistic relational language. Mach. Learn. 62(1–2), 7–31 (2006)

    Article  Google Scholar 

  9. Getoor, L., Taskar, B.: Introduction to Statistical Relational Learning. MIT, Cambridge (2007)

    MATH  Google Scholar 

  10. Getoor, L., Friedman, N., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Dzeroski, S., Lavrac, N. (eds.) Relational Data Mining. Springer, New York (2001)

    Google Scholar 

  11. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001)

    MATH  Google Scholar 

  12. Heckerman, D., Breese, J.S.: Causal independence for probability assessment and inference using Bayesian networks. Technical Report MSR-TR-94-08, Microsoft Research (1994)

  13. Heckerman, D., Meek, C., Koller, D.: Probabilistic models for relational data. Technical Report MSR-TR-2004-30, March (2004)

  14. Jaeger, M.: Relational Bayesian networks. In: Proceedings of UAI-97, Providence, 1–3 August 1997

  15. Jaeger, M.: Parameter learning for relational Bayesian networks. In: Proceedings of the International Conference in Machine Learning, Corvalis, 20–24 June 2007

  16. Kersting, K., De Raedt, L.: Bayesian logic programs. In: Proceedings of the Work-in-Progress Track at the 10th International Conference on Inductive Logic Programming, London, 24–27 July 2000

  17. Kersting, K., De Raedt, L.: Adaptive Bayesian logic programs. In: Proceedings of the ILP ’01, pp. 104–117. Springer, New York (2001)

    Google Scholar 

  18. Koller, D., Pfeffer, A.: Learning probabilities for noisy first-order rules. In: IJCAI, pp. 1316–1323. Nagoya, 23–29 August 1997

  19. Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proc. 18th International Conf. on Machine Learning, pp. 282–289 (2001)

  20. Laskey, K.B.: MEBN: a language for first-order Bayesian knowledge bases. Artif. Intell. 172(2–3), 140–178 (2008)

    Article  MathSciNet  Google Scholar 

  21. Muggleton, S.: Stochastic logic programs. In: Advances in Inductive Logic Programming, pp. 254–264 (1996)

  22. Natarajan, S., Tadepalli, P., Altendorf, E., Dietterich, T.G., Fern, A., Restificar, A.: Learning first-order probabilistic models with combining rules. In: Proceedings of the International Conference in Machine Learning, Bonn, 7–11 August 2005

  23. Natarajan, S., Tadepalli, P., Fern, A.: A relational hierarchical model for decision-theoretic assistance. In: Proceedings of 17th Annual International Conference on Inductive Logic Programming, Corvallis, 19–21 June 2007

  24. Neville, J., Jensen, D., Friedland, L., Hay, M.: Learning relational probability trees. In: KDD ’03: proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, pp. 625–630. Washington, DC, 24–27 August 2003

  25. Ngo, L., Haddawy, P.: Probabilistic logic programming and Bayesian networks. In: Proceedings ACSC95, Pathumthani, 11–13 December 1995

  26. Poole, D.: Probabilistic Horn abduction and Bayesian networks. Artif. Intell. 64(1), 81–129 (1993)

    Article  MATH  Google Scholar 

  27. Sato, T., Kameya, Y.: Parameter learning of logic programs for symbolic-statistical modeling. J. Artif. Intell. Res. 15, 391–454 (2001)

    MATH  MathSciNet  Google Scholar 

  28. Vomlel, J.: Noisy-or classifier: research articles. Int. J. Intell. Syst. 21(3), 381–398 (2006)

    Article  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR, 97331-3202, USA

    Sriraam Natarajan, Prasad Tadepalli, Thomas G. Dietterich & Alan Fern

Authors
  1. Sriraam Natarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prasad Tadepalli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas G. Dietterich
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alan Fern
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sriraam Natarajan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Natarajan, S., Tadepalli, P., Dietterich, T.G. et al. Learning first-order probabilistic models with combining rules. Ann Math Artif Intell 54, 223–256 (2008). https://doi.org/10.1007/s10472-009-9138-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-009-9138-5

Keywords

Mathematics Subject Classification (2000)

Search

Navigation