Abstract
One challenge faced by many Inductive Logic Programming (ILP) systems is poor scalability to problems with large search spaces and many examples. Randomized search methods such as stochastic clause selection (SCS) and rapid random restarts (RRR) have proven somewhat successful at addressing this weakness. However, on datasets where hypothesis evaluation is computationally expensive, even these algorithms may take unreasonably long to discover a good solution. We attempt to improve the performance of these algorithms on datasets by learning an approximation to ILP hypothesis evaluation. We generate a small set of hypotheses, uniformly sampled from the space of candidate hypotheses, and evaluate this set on actual data. These hypotheses and their corresponding evaluation scores serve as training data for learning an approximate hypothesis evaluator. We outline three techniques that make use of the trained evaluation-function approximator in order to reduce the computation required during an ILP hypothesis search. We test our approximate clause evaluation algorithm using the popular ILP system Aleph. Empirical results are provided on several benchmark datasets. We show that the clause evaluation function can be accurately approximated.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lavrac, N., Dzeroski, S.: Inductive Logic Programming. Ellis Horwood (1994)
King, R., Muggleton, S., Sternberg, M.: Predicting protein secondary structure using inductive logic programming. Protein Engineering 5, 647–657 (1992)
Srinivasan, A., King, R., Muggleton, S., Sternberg, M.: The predictive toxicology evaluation challenge. In: Proc. 15th Intl. Joint Conf. on Artificial Intelligence, pp. 1–6 (1997)
Dolsak, B., Muggleton, S.: The application of ILP to finite element mesh design. In: Proc. 1st Intl. Workshop on ILP, pp. 225–242 (1991)
Zelle, J., Mooney, R.: Learning semantic grammars with constructive inductive logic programming. In: Proc. 11th Natl. Conf. on Artificial Intelligence, pp. 817–822 (1993)
Bratko, I., Grobelnik, M.: Inductive learning applied to program construction and verification. In: Proc. 3rd Intl. Workshop on Inductive Logic Programming, pp. 169–182 (1993)
Nienhuys-Cheng, S., de Wolf, R.: Foundations of Inductive Logic Programming. Springer, Heidelberg (1997)
Schmidt-Schauss, M.: Implication of clauses is undecidable. Theoretical Computer Science 59, 287–296 (1988)
Quinlan, J.: Learning logical definitions from relations. Machine Learning, 239–266 (1990)
Muggleton, S., Feng, C.: Efficient induction of logic programs. In: Proc. 1st Conf. on Algorithmic Learning Theory, pp. 368–381 (1990)
Muggleton, S.: Inverse Entailment and Progol. New Generation Computing 13, 245–286 (1995)
Srinivasan, A.: A study of two probabilistic methods for searching large spaces with ILP. Tech. Report PRG-TR-16-00. Oxford Univ. Computing Lab (2000)
Zelezny, F., Srinivasan, A., Page, D.: Lattice-search runtime distributions may be heavy-tailed. In: Proc. 12th Intl. Conf. on Inductive Logic Programming,pp. 333-345 (2002)
Giordana, L.: Saitta & F. Zini, Learning disjunctive concepts by means of genetic algorithms. In: Proc. 11th Intl. Conf. on Machine Learning ,pp. 96-104 (1994)
Hanschke, P., Wurtz, J.: Satisfiability of the smallest binary program. Info. Proc. Letters 496, 237–241 (1993)
Dantsin, E., Eiter, T., Gottlob, G., Voronkov, A.: Complexity and expressive power of logic programming. ACM Computing Surveys 33, 374–425 (2001)
Rückert, U., Kramer, S.: Stochastic local search in k-term DNF learning. In: Proc. 20th Intl. Conf. on Machine Learning,pp. 648-655 (2003)
Blockeel, H., Dehasp, L., Demoen, B., Janssens, G., Ramon, J., Vandecasteele, H.: Improving the efficiency of inductive logic programming through the use of query packs. J. AI Research 16, 135–166 (2002)
Santos Costa, V., Srinivasan, A., Camacho, R., Blockeel, H., Demoen, B., Janssens, G., Struyf, J., Van de casteele, H., Van Laer, W.: Query transformations for improving the efficiency of ILP systems. J. Machine Learning Research 4, 465–491 (2003)
Srinivasan, A.: A study of two sampling methods for analysing large datasets with ILP. Data Mining and Knowledge Discovery 3, 95–123 (1999)
Sebag, M., Rouveirol, C.: Resource-bounded relational reasoning: induction and deduction through stochastic matching. Machine Learning 38, 41–62 (2000)
Maloberti, J., Sebag, M.: Theta-subsumption in a constraint satisfaction perspective. In: Proc. 11th Intl. Conf. on Inductive Logic Programming, pp. 164–178 (2001)
Boyan, J., Moore, A.: Learning evaluation functions to improve optimization by local search. J. Machine Learning Research 1, 77–112 (2000)
Hornik, K., Stinchcombe, M., White, H.: Multilayer feedforward networks are universal approximators. Neural Networks 2, 359–366 (1989)
Nix, D., Weigend, A.: Learning local error bars for nonlinear regression. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge (1995)
Craven, M., Shavlik, J.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge (1995)
King, R., Muggleton, S., Srinivasan, A., Sternberg, M.: Structure-activity relationships derived by machine learning. PNAS 93, 438–442 (1996)
Srinivasan, A., King, R., Muggleton, S., Sternberg, M.: Carcinogenesis predictions using ILP. In: Proc. 7th Intl. Workshop on Inductive Logic Programming, pp. 273–287 (1997)
Witten, I., Frank, E.: Data Mining. Morgan Kaufmann Publishers, San Francisco (1999)
Goadrich, M., Oliphant, L., Shavlik, J.: Learning ensembles of first-order clauses for recall-precision curves: a case study in biomedical information extraction. In: Proc. 14th Intl. Conf. on Inductive Logic Programming (2004)
Botta, M., Giordana, A., Saitta, L., Sebag, M.: Relational learning as search in a critical region. J. Machine Learning Research 4, 431–463 (2003)
Caruana, R., Baluja, S.: Using the future to ’sort out’ the present. In: Advances in Neural Information Processing Systems, MIT Press, Cambridge (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
DiMaio, F., Shavlik, J. (2004). Learning an Approximation to Inductive Logic Programming Clause Evaluation. In: Camacho, R., King, R., Srinivasan, A. (eds) Inductive Logic Programming. ILP 2004. Lecture Notes in Computer Science(), vol 3194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30109-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-30109-7_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22941-4
Online ISBN: 978-3-540-30109-7
eBook Packages: Springer Book Archive