Abstract
Markov Logic Networks (MLNs) combine Markov networks and first-order logic by attaching weights to first-order formulas and viewing these as templates for features of Markov networks. Learning the structure of MLNs is performed by state-of-the-art methods by maximizing the likelihood of a relational database. This can lead to suboptimal results given prediction tasks. On the other hand better results in prediction problems have been achieved by discriminative learning of MLNs weights given a certain structure. In this paper we propose an algorithm for learning the structure of MLNs discriminatively by maximimizing the conditional likelihood of the query predicates instead of the joint likelihood of all predicates. The algorithm chooses the structures by maximizing conditional likelihood and sets the parameters by maximum likelihood. Experiments in two real-world domains show that the proposed algorithm improves over the state-of-the-art discriminative weight learning algorithm for MLNs in terms of conditional likelihood. We also compare the proposed algorithm with the state-of-the-art generative structure learning algorithm for MLNs and confirm the results in [22] showing that for small datasets the generative algorithm is competitive, while for larger datasets the discriminative algorithm outperfoms the generative one.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Besag, J.: Statistical analysis of non-lattice data. Statistician 24, 179–195 (1975)
Davis, J., Burnside, E., de Castro Dutra, I., Page, D., Santos Costa, V.: An integrated approach to learning Bayesian networks of rules. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 84–95. Springer, Heidelberg (2005)
Davis, J., Goadrich, M.: The relationship between precision-recall and ROC curves. In: Proc. of 23rd Intl. Conf. on Machine Learning, pp. 233–240 (2006)
Dehaspe, L.: Maximum entropy modeling with clausal constraints. In: Proc. of ILP 1997, pp. 109–124 (1997)
Della Pietra, S., Pietra, V.D., Laferty, J.: Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 380–392 (1997)
De Raedt, L., Dehaspe, L.: Clausal discovery. Machine Learning 26, 99–146 (1997)
De Raedt, L., Frasconi, P., Kersting, K., Muggleton, S.: Probabilistic Inductive Logic Programming - Theory and Applications. Springer, Heidelberg (2008)
Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29, 103–130 (1997)
Friedman, J.H.: On bias, variance, 0/1 - loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery 1, 55–77 (1997)
Getoor, L., Taskar, B.: Introduction to statistical relational learning. MIT Press, Cambridge (2007)
Greiner, R., Su, X., Shen, S., Zhou, W.: Structural extension to logistic regression: Discriminative parameter learning of belief net classifiers. Machine Learning 59, 297–322 (2005)
Grossman, D., Domingos, P.: Learning bayesian network classiers by maximizing conditional likelihood. In: Proc. 21st Int’l Conf. on Machine Learning, pp. 361–368. ACM Press, Banf (2004)
Hoos, H.H., Stutzle, T.: Stochastic local search: Foundations and applications. Morgan Kaufmann, San Francisco (2005)
Kok, S., Domingos, P.: Learning the structure of markov logic networks. In: Proc, 22nd Int’l Conf. on Machine Learning, pp. 441–448 (2005)
Kok, S., Singla, P., Richardson, M., Domingos, P.: The alchemy system for statistical relational ai (Technical Report). Department of Computer Science and Engineering, University of Washington, Seattle, WA (2005), http://alchemy.cs.washington.edu/
Laferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proc. 18th Int’l Conf. on Machine Learning, pp. 282–289 (2001)
Landwehr, N., Kersting, K., De Raedt, L.: nFOIL: Integrating Naive Bayes and FOIL. In: Proc. 20th Nat’l Conf. on Artificial Intelligence, pp. 795–800. AAAI Press, Menlo Park (2005)
Landwehr, N., Kersting, K., De Raedt, L.: Integrating Naive Bayes and FOIL. Journal of Machine Learning Research, 481–507 (2007)
Landwehr, N., Passerini, A., De Raedt, L., Frasconi, P.: kFOIL: Learning Simple Relational Kernels. In: Proc. 21st Nat’l Conf. on Artificial Intelligence. AAAI Press, Menlo Park (2006)
Loureno, H.R., Martin, O., Stutzle, T.: Iterated local search. In: Handbook of Metaheuristics, pp. 321–353. Kluwer Academic Publishers, Dordrecht (2002)
Lowd, D., Domingos, P.: Efficient weight learning for markov logic networks. In: Proc. of the 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, pp. 200–211 (2007)
Ng, A.Y., Jordan, M.I.: On discriminative vs. generative: A comparison of logistic regression and naive Bayes. In: Advances in Neural Information Processing Systems, vol. 14, pp. 841–848. MIT Press, Cambridge (2002)
McCallum, A.: Efficiently inducing features of conditional random fields. In: Proc. 19th Conf. on Uncertainty in Artificial Intelligence, pp. 403–410 (2003)
Mihalkova, L., Mooney, R.J.: Bottom-up learning of markov logic network structure. In: Proc. 24th Int’l Conf. on Machine Learning, pp. 625–632 (2007)
Pernkopf, F., Bilmes, J.: Discriminative versus generative parameter and structure learning of Bayesian network classifiers. In: Proc, 22nd Int’l Conf. on Machine Learning, pp. 657–664 (2005)
Popescul, A., Ungar, L., Lawrence, S., Pennock, D.: Statistical Relational Learning for Document Mining. In: Proc. 3rd Int’l Conf. on Data Mining, pp. 275–282 (2003)
Poon, H., Domingos, P.: Sound and efficient inference with probabilistic and deterministic dependencies. In: Proc. 21st Nat’l Conf. on Artificial Intelligence, pp. 458–463. AAAI Press, Menlo Park (2006)
Poon, H., Domingos, P., Sumner, M.: A General Method for Reducing the Complexity of Relational Inference and its Application to MCMC. In: Proc. 23rd Nat’l Conf. on Artificial Intelligence. AAAI Press, Chicago (to appear, 2008)
Quinlan, J.R.: Learning logical definitions from relations. Machine Learning 5, 239–266 (1990)
Richardson, M., Domingos, P.: Markov logic networks. Machine Learning 62, 107–236 (2006)
Singla, P., Domingos, P.: Discriminative training of markov logic networks. In: Proc. 20th Nat’l Conf. on Artificial Intelligence, pp. 868–873. AAAI Press, Menlo Park (2005)
Singla, P., Domingos, P.: Entity resolution with markov logic. In: Proc. 6th Int’l Conf. on Data Mining, pp. 572–582. IEEE Computer Society Press, Los Alamitos (2006)
Singla, P., Domingos, P.: Memory-efficient inference in relational domains. In: Proc. 21st Nat’l Conf. on Artificial Intelligence, pp. 488–493. AAAI Press, Menlo Park (2006)
Sha, F., Pereira, F.: Shallow parsing with conditional random fields. In: Proc. HLT-NAACL, pp. 134–141 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Biba, M., Ferilli, S., Esposito, F. (2008). Discriminative Structure Learning of Markov Logic Networks. In: Železný, F., Lavrač, N. (eds) Inductive Logic Programming. ILP 2008. Lecture Notes in Computer Science(), vol 5194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-85928-4_9
Download citation
DOI: https://doi.org/10.1007/978-3-540-85928-4_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-85927-7
Online ISBN: 978-3-540-85928-4
eBook Packages: Computer ScienceComputer Science (R0)