Abstract
Genetic Algorithms (GAs) are known for their capacity to explore large search spaces and due to this ability they were applied (to some extent) to Inductive Logic Programming (ILP). Although Estimation of Distribution Algorithms (EDAs) generally perform better than standard GAs, they have not been applied to ILP. This work presents EDA-ILP, an ILP system based on EDA and inverse entailment, and also its extension, the REDA-ILP, which employs the Reduce algorithm in bottom clauses to considerably reduce the search space. Experiments in real-world datasets showed that both systems were successfully compared to Aleph and GA-ILP (another variant of EDA-ILP created replacing the EDA by a standard GA). EDA-ILP was also successfully compared to Progol-QG/GA (and its other variants) in phase transition benchmarks. Additionally, we found that REDA-ILP usually obtains simpler theories than EDA-ILP, more efficiently and with equivalent accuracies. These results show that EDAs provide a good base for stochastic search in ILP.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Pitangui, C., Zaverucha, G.: Inductive Logic Programming Through Estimation Distribution Algorithm. In: Proceedings of IEEE Congress of Evolutionary Computation (CEC 2011), New Orleans, LA, EUA, pp. 54–61 (2011) 978-1-4244-7834-7
Muggleton, S., De Raedt, L.: Inductive Logic Programming: Theory and Methods. Journal of Logic Programming 19(20) (1994)
Mühlenbein, H., Paaß, G.: From Recombination of Genes to the Estimation of Distributions I. Binary Parameters. In: Ebeling, W., Rechenberg, I., Voigt, H.-M., Schwefel, H.-P. (eds.) PPSN 1996. LNCS, vol. 1141, pp. 178–187. Springer, Heidelberg (1996)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers Inc., San Francisco (1988)
Baluja, S.: Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Carnegie Mellon University, Pittsburgh (1994); Technical Report: CMU-CS-94-163
Holland, J.: Adaptation in natural and artificial systems. MIT Press, Cambridge (1975)
Muggleton, S.H., Feng, C.: Efficient induction of logic programs. In: Proceedings of the First Conference on Algorithmic Learning Theory, pp. 368–381. Ohmsha, Tokyo (1990)
Srinivasan, A.: The Aleph Manual, www.comlab.ox.ac.uk/activities/machinelearning/Aleph/ (last access September 29, 2011)
Alphonse, E., Rouveirol, C.: Lazy propositionalisation for Relational Learning. In: 14th European Conference on Artificial Intelligence (ECAI 2000), pp. 256–260. IOS Press (2000)
Muggleton, S., Tamaddoni-Nezhad, A.: QG/GA: A stochastic search approach for Progol. Machine Learning 70(2-3), 123–133 (2007), doi:10.1007/s10994-007-5029-3
Muggleton, S.: Inverse entailment and Progol. New Generation Computing, Special issue on Inductive Logic Programming 13(3-4), 245–286 (1995)
Oliphant, L., Shavlik, J.: Using Bayesian Networks to Direct Stochastic Search in Inductive Logic Programming. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 191–199. Springer, Heidelberg (2008)
Srinivasan, A., King, R.D.S.H., Muggleton, S., Sternberg, M.: Carcinogenesis Predictions using ILP. In: Džeroski, S., Lavrač, N. (eds.) ILP 1997. LNCS (LNAI), vol. 1297, pp. 273–287. Springer, Heidelberg (1997)
King, R.D., Srinivasan, A., Sternberg, M.J.E.: Relating chemical activity to structure: an examination of ILP successes. New Gen. Comp. 13, 411–433 (1995)
Nadeau, C., Bengio, Y.: Inference for the Generalization Error. Machine Learning 52(3), 239–281 (2003)
Huynh, T., Mooney, R.: Discriminative Structure and Parameter Learning for Markov Logic Networks. In: Proceedings of the 25th International Conference on Machine Learning (ICML 2008), Helsinki, Finland, pp. 416–423 (2008)
Muggleton, S.H., Santos, J.C.A., Tamaddoni-Nezhad, A.: TopLog: ILP Using a Logic Program Declarative Bias. In: Garcia de la Banda, M., Pontelli, E. (eds.) ICLP 2008. LNCS, vol. 5366, pp. 687–692. Springer, Heidelberg (2008)
Bratko, I.: Refining Complete Hypotheses in ILP. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 44–55. Springer, Heidelberg (1999)
Paes, A., Zaverucha, G., Santos Costa, V.: Revising First-Order Logic Theories from Examples Through Stochastic Local Search. In: Blockeel, H., Ramon, J., Shavlik, J., Tadepalli, P. (eds.) ILP 2007. LNCS (LNAI), vol. 4894, pp. 200–210. Springer, Heidelberg (2008)
Srinivasan, A.: A study of two probabilistic methods for searching large spaces with ILP(Technical Report PRG-TR-16-00). Oxford University Computing Laboratory, Oxford (2000)
Paes, A., Železný, F., Zaverucha, G., Page, D.L., Srinivasan, A.: ILP Through Propositionalization and Stochastic k-Term DNF Learning. In: Muggleton, S.H., Otero, R., Tamaddoni-Nezhad, A. (eds.) ILP 2006. LNCS (LNAI), vol. 4455, pp. 379–393. Springer, Heidelberg (2007)
Tamaddoni-Nezhad, A., Muggleton, S.H.: Searching the Subsumption Lattice by a Genetic Algorithm. In: Cussens, J., Frisch, A.M. (eds.) ILP 2000. LNCS (LNAI), vol. 1866, pp. 243–252. Springer, Heidelberg (2000)
Pelikan, M.: Hierarchical Bayesian Optimization Algorithm Toward a New Generation of Evolutionary Algorithms, 1st edn. STUDFUZZ, vol. 170. Springer (2005)
Železný, F., Srinivasan, A., Page, D.: Lattice-Search Runtime Distributions May Be Heavy-Tailed. In: Matwin, S., Sammut, C. (eds.) ILP 2002. LNCS (LNAI), vol. 2583, pp. 333–345. Springer, Heidelberg (2003)
Goadrich, M., Oliphant, L., Shavlik, J.: Gleaner: Creating Ensembles of First-Order Clauses to Improve Recall-Precision Curves. Machine Learning 64(1-3), 231–261 (2006)
Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational learning as search in a critical region. Journal of Machine Learning Research 4, 431–463 (2003)
Alphonse, E., Osmani, A.: On the connection between the phase transition of the covering test and the learning success rate in ILP. Machine Learning Journal 70(2-3), 135–150 (2008)
Henrion, M.: Propagating Uncertainty in Bayesian Networks by Probabilistic Logic Sampling. In: Lemmer, J.F., Kanal, L.N. (eds.) Uncertainty in Artificial Intelligence, vol. 2, pp. 149–163. North Holland (1988)
Pitangui, C., Zaverucha, G.: Genetic local search for rule learning. In: Genetic And Evolutionary Computation Conference (GECCO) Atlanta, GA, USA, pp. 1427–1428 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Pitangui, C.G., Zaverucha, G. (2012). Learning Theories Using Estimation Distribution Algorithms and (Reduced) Bottom Clauses. In: Muggleton, S.H., Tamaddoni-Nezhad, A., Lisi, F.A. (eds) Inductive Logic Programming. ILP 2011. Lecture Notes in Computer Science(), vol 7207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31951-8_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-31951-8_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31950-1
Online ISBN: 978-3-642-31951-8
eBook Packages: Computer ScienceComputer Science (R0)