Abstract
Inductive logic programming is a form of machine learning from examples which employs the representation formalism of clausal logic. One of the earliest inductive logic programming systems was Ehud Shapiro’s Model Inference System [90], which could synthesise simple recursive programs like append/3. Many of the techniques devised by Shapiro, such as top-down search of program clauses by refinement operators, the use of intensional background knowledge, and the capability of inducing recursive clauses, are still in use today. On the other hand, significant advances have been made regarding dealing with noisy data, efficient heuristic and stochastic search methods, the use of logical representations going beyond definite clauses, and restricting the search space by means of declarative bias. The latter is a general term denoting any form of restrictions on the syntactic form of possible hypotheses. These include the use of types, input/output mode declarations, and clause schemata. Recently, some researchers have started using alternatives to Prolog featuring strong typing and real functions, which alleviate the need for some of the above ad-hoc mechanisms. Others have gone beyond Prolog by investigating learning tasks in which the hypotheses are not definite clause programs, but for instance sets of indefinite clauses or denials, constraint logic programs, or clauses representing association rules. The chapter gives an accessible introduction to the above topics. In addition, it outlines the main current research directions which have been strongly influenced by recent developments in data mining and challenging real-life applications.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and A.I. Verkamo. Fast discovery of association rules. In U.M. Fayyad, G. Piatetski-Shapiro, P. Smyth, and R. Uthurusamy (eds.), Advances in Knowledge Discovery and Data Mining, pp. 307–328. AAAI Press, 1996.
J. J. Alferes, J. A. Leite, L. M. Pereira, H. Przymusinska, and T. C. Przymusinski. Dynamic logic programming, In A. Cohn, L. Schubert and S. Shapiro (eds.), Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning, pp. 98–109. Morgan Kaufmann, 1998.
D. Angluin, M. Frazier, and L. Pitt. Learning conjunctions of Horn clauses. Machine Learning, 9(2/3): 147–164, 1992.
H. Blockeel and L. De Raedt. Top-down induction of first-order logical decision trees. Artificial Intelligence 101(1–2): 285–297, June 1998.
H. Blockeel, L. De Raedt, N. Jacobs, and B. Demoen. Scaling up inductive logic programming by learning from interpretations. Data Mining and Knowledge Discovery, 3(1): 59–93, 1999.
H. Blockeel, L. Dehaspe, B. Demoen, G. Janssens, J. Ramon, and H. Vandecasteele. Executing query packs in ILP. In J. Cussens and A. Frisch (eds.), Proceedings of the Tenth International Conference on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1866, pp. 60–77. Springer-Verlag, 2000.
A.F. Bowers, C. Giraud-Carrier, and J.W. Lloyd. Classification of individuals with complex structure. In P. Langley (ed.), Proceedings of the Seventeenth International Conference on Machine Learning, pp. 81–88. Morgan Kaufmann, 2000.
I. Bratko and S. Muggleton. Applications of Inductive Logic Programming. Communications of the A CM 38(11): 65–70, November 1995.
L. Breiman. Bagging predictors. Machine Learning 24(2): 123–140, 1996.
G. Brewka. Well-founded semantics for extended logic programs with dynamic preferences. Journal of Artificial Intelligence Research, 4: 19–36, 1996.
G. Brewka and T. Eiter. Preferred answer sets. In A. Cohn, L. Schubert and S. Shapiro (eds.), Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning, pp. 89–97. Morgan Kaufmann, 1998.
W.W. Cohen. Recovering software specifications with inductive logic programming. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 142–148. The MIT Press, 1994.
W.W. Cohen and H. Hirsh. Learning the CLASSIC Description Logic: Theoretical and Experimental Results. In J. Doyle, E. Sandewall, and P. Torasso (eds.), Proceedings of the Fourth International Conference on Principles of Knowledge Representation and Reasoning, pp. 121–133. Morgan Kaufmann, 1994.
J. Cussens. Notes on inductive logic programming methods in natural language processing (European work). Unpublished manuscript, 1998. ftp://ftp.cs.york.ac.uk/pub/ML_GROUP/Papers/ilp98tut.ps.gz.
J. Cussens and S. Džeroski (eds.). Learning Language in Logic. Lecture Notes in Artificial Intelligence 1925, Springer-Verlag, 2000.
L. Dehaspe and L. De Raedt. Mining association rules in multiple relations. In S. Džeroski and N. Lavrač (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 125–132. Springer-Verlag, 1997.
L. Dehaspe, H. Toivonen, and R.D. King. Finding frequent substructures in chemical compounds. In R. Agrawal, P. Stolorz, and G. Piatetsky-Shapiro (eds.), Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 30–36. AAAI Press, 1998.
L. De Raedt (ed.). Advances in Inductive Logic Programming. IOS Press, 1996.
L. De Raedt. Logical settings for concept-learning. Artificial Intelligence, 95(1): 187–201, 1997.
L. De Raedt and H. Blockeel. Using logical decision trees for clustering. In N. Lavrač and S. Džeroski (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 133–140. Springer-Verlag, 1997.
L. De Raedt and L. Dehaspe. Clausal discovery. Machine Learning, 26(2/3): 99–146, 1997.
L. De Raedt. An inductive logic programming query language for database mining (extended abstract). In J. Calmet and J. Plaza (eds.), Proceedings of the Fourth Workshop on Artificial Intelligence and Symbolic Computation, Lecture Notes in Artificial Intelligence 1476. Springer-Verlag, 1998.
L. De Raedt. A perspective on inductive logic programming. In K. Apt, V. Marek, M. Truszezynski, and D.S. Warren (eds.), The logic programming paradigm: current trends and future directions. Springer-Verlag, 1999.
L. De Raedt. A logical database mining query language. In J. Cussens and A. Frisch, Proceedings of the Tenth International Conference on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1866, pp. 78–92. Springer-Verlag, 2000.
Y. Dimopoulos and A.C. Kakas. Learning non-monotonic logic programs: learning exceptions. In N. Lavrač and S. Wrobel (eds.), Proceedings of the Eighth European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 912, pp. 122–138. Springer-Verlag, 1995.
Y. Dimopoulos and A.C. Kakas. Abduction and inductive learning. In [18], pp. 144–171.
Y. Dimopoulos, S. Džeroski, and A.C. Kakas. Integrating Explanatory and Descriptive Induction in ILP. In M.E. Pollack (ed.), Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pp. 900–907. Morgan Kaufmann, 1997.
B. Dolšak and S. Muggleton. The application of inductive logic programming to finite-element mesh design. In [76], pp. 453–472.
S. Džeroski and L. Todorovski. Discovering dynamics: From inductive logic programming to machine discovery. In Proceedings of the Tenth International Conference on Machine Learning, pp. 97–103. Morgan Kaufmann, 1993.
S. Džeroski and I. Bratko. Applications of Inductive Logic Programming. In [18], pp. 65–81.
S. Džeroski, L. De Raedt, and H. Blockeel. Relational reinforcement learning. In J. Shavlik (ed.), Proceedings of the Fifteenth International Conference on Machine Learning, pp. 136–143. Morgan Kaufmann, 1998.
S. Džeroski and N. Lavrač, eds. Relational Data Mining. Springer-Verlag, 2001. In press.
W. Emde. Learning of characteristic concept descriptions from small sets to classified examples. In F. Bergadano and L. De Raedt (eds.), Proceedings of the Seventh European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 784, pp. 103–121. Springer-Verlag, 1994.
W. Emde and D. Wettschereck. Relational instance-based learning. In L. Saitta (ed.), Proceedings of the Thirteenth International Conference on Machine Learning, pp. 122–130. Morgan Kaufmann, 1996.
P.A. Flach. Predicate invention in inductive data engineering. In P. Brazdil (ed.), Proceedings of the Sixth European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 667, pp. 83–94. Springer-Verlag, 1993.
P.A. Flach. Simply Logical-intelligent reasoning by example. John Wiley, 1994.
P.A. Flach. Conjectures-an inquiry concerning the logic of induction. PhD thesis, Tilburg University, April 1995.
P.A. Flach. Rationality postulates for induction. In Y. Shoham (ed.), Proceedings of the Sixth International Conference on Theoretical Aspects of Rationality and Knowledge, pp. 267–281. Morgan Kaufmann, 1996.
P.A. Flach. Normal forms for Inductive Logic Programming. In N. Lavrač and S. Džeroski (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 149–156. Springer-Verlag, 1997.
P.A. Flach, C. Giraud-Carrier, and J.W. Lloyd. Strongly typed inductive concept learning. In D. Page (ed.), Proceedings of the Eighth International Conference on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1446, pp. 185–194. Springer-Verlag, 1998.
P.A. Flach and I. Savnik. Database dependency discovery: a machine learning approach. AI Communications, 12(3): 139–160, November 1999.
P.A. Flach and N. Lachiche. 1BC: A first-order Bayesian classifier. In S. Džeroski and P.A. Flach (eds.), Proceedings of the Ninth International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1634, pp. 92–103. Springer-Verlag, 1999.
P.A. Flach and A.C. Kakas (eds.) Abduction and Induction: Essays on their Relation and Integration. Kluwer, 2000.
P.A. Flach and N. Lachiche. Confirmation-guided discovery of first-order rules with Tertius. Machine Learning, 42(1/2): 61–95, 2001.
Y. Freund and R.E. Shapire. Experiments with a new boosting algorithm. In L. Saitta (ed.), Proceedings of the Thirteenth International Conference on Machine Learning, 148–156. Morgan Kaufmann, 1996.
A. Giordana and C. Sale. Learning structured concepts using genetic algorithms. In D. Sleeman (ed.), Proceedings of the Ninth International Workshop on Machine Learning, pp. 169–178. Morgan Kaufmann, 1992.
G. Gottlob. Subsumption and implication. Information Processing Letters 24: 109–111, 1987.
D.T. Hau and E.W. Coiera. Learning qualitative models of dynamic systems. Machine Learning, 26(2/3): 177–212, 1997.
N. Helft. Induction as nonmonotonic inference. In R.J. Brachman, H.J. Levesque, and R. Reiter (eds.), Proceedings of the First International Conference on Principles of Knowledge Representation and Reasoning, pp. 149–156. Morgan Kaufmann, 1989.
P. Idestam-Almquist. Generalization of clauses. PhD thesis, Stockholm University, October 1993.
P. Idestam-Almquist. Generalization of clauses under implication. Journal of Artificial Intelligence Research, 3: 467–489, 1995.
A.C. Kakas and F. Riguzzi. Learning with abduction. In S. Džeroski and N. Lavrač (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 181–188. Springer-Verlag, 1997.
A. Karalič and I. Bratko. First-order regression. Machine Learning, 26(2/3): 147–176, 1997.
M. Kifer and V.S. Subrahmanian. Generalized annotated logic programs. Journal of Logic Programming, 1992.
R.D. King, S. Muggleton, R. Lewis, and M.J.E. Sternberg. Drug design by machine learning: The use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. In Proceedings of the National Academy of Sciences of the USA 89(23): 11322–11326, 1992.
R.D. King, A. Karwath, A. Clare, and L. Dehaspe. Genome scale prediction of protein functional class from sequence using data mining. In Proceedings of the Sixth International Conference on Knowledge Discovery and Data Mining, pp. 384–398. ACM Press, New York, 2000.
R.D. King, A. Karwath, A. Clare, and L. Dehaspe. Accurate prediction of protein functional class in the M.tuberculosis and E.coli genomes using data mining. Yeast (Comparative and Functional Genomics, 17: 283–293, 2000.
M. Kirsten and S. Wrobel. Relational distance-based clustering. In D. Page (ed.) Proceedings of the Eighth International Conference on Inductive Logic Programming, pp. 261–270, Lecture Notes in Artificial Intelligence 1446. Springer-Verlag, 1998.
P. van der Laag. An analysis of refinement operators in Inductive Logic Programming. PhD Thesis, Erasmus University Rotterdam, December 1995.
E. Lamma, F. Riguzzi, and L. M. Pereira. Agents learning in a three-valued logical setting. In A. Panayiotopoulos (ed.), Proceedings of the Workshop on Machine Learning and Intelligent Agents, in conjunction with Machine Learning and Applications, Advanced Course on Artificial Intelligence (ACAI-99), Chania, Greece, 1999.
E. Lamma, F. Riguzzi, and L. M. Pereira. Strategies in combined learning via Logic Programs. Machine Learning, 38(1/2): 63–87, 2000.
N. Lavrač, S. Džeroski, and M. Grobelnik. Learning nonrecursive definitions of relations with LINUS. In Y. Kodratoff (ed.) Proceedings of the Fifth European Working Session on Learning, Lecture Notes in Artificial Intelligence 482, pp. 265–281. Springer-Verlag, 1991.
N. Lavrač and S. Džeroski. Inductive Logic Programming: techniques and applications. Ellis Horwood, 1994.
N. Lavrač, S. Džeroski, and I. Bratko. Handling imperfect data in Inductive Logic Programming. In [18], pp. 48–64.
N. Lavrač and P.A. Flach. An extended transformation approach to Inductive Logic Programming. ACM Transactions on Computational Logic, 2(4): 458–494, 2001.
C.J. Kennedy. Strongly typed evolutionary programming. PhD Thesis, University of Bristol, 2000.
J.W. Lloyd. Programming in an integrated functional and logic programming language. Journal of Functional and Logic Programming, 1999(3).
D.W. Loveland and G. Nadathur. Proof procedures for logic programming. Handbook of Logic in Artificial Intelligence and Logic Programming, Vol. 5, D.M. Gabbay, C.J. Hogger, and J.A. Robinson (eds.), Oxford University Press, pp. 163–234, 1998.
D. Michie, S. Muggleton, D. Page, and A. Srinivasan. To the international computing community: A new East-West challenge. Technical report, Oxford University Computing laboratory, Oxford,UK, 1994.
T.M. Mitchell. Machine Learning. McGraw-Hill, 1997.
T.M. Mitchell. Does machine learning really work? AI Magazine 18(3): 11–20, 1997.
F. Mizoguchi, H. Ohwada, M. Daidoji, and S. Shirato. Using inductive logic programming to learn classification rules that identify glaucomatous eyes. In N. Lavrač, E. Keravnou, and B. Zupan (eds.), Intelligent Data Analysis in Medicine and Pharmacology, pp. 227–242. Kluwer, 1997.
R.J. Mooney and M.E. Califf. Induction of first-order decision lists: Results on learning the past tense of English verbs. Journal of Artificial Intelligence Research 3: 1–24, 1995.
I. Mozetič. Learning of qualitative models. In I. Bratko and N. Lavrač (eds.) Progress in Machine Learning, pp. 201–217. Sigma Press, 1987.
S. Muggleton. Inductive Logic Programming. New Generation Computing, 8(4): 295–317, 1991. Also in [76], pp. 3–27.
S. Muggleton (ed.). Inductive Logic Programming. Academic Press, 1992.
S. Muggleton and C. Feng. Efficient induction of logic programs. In [76], pp. 281–298.
S. Muggleton, R.D. King, and M.J.E. Sternberg. Protein secondary structure prediction using logic. Protein Engineering 7: 647–657, 1992.
S. Muggleton and L. De Raedt. Inductive Logic Programming: theory and methods. Journal of Logic Programming, 19/20: 629–679, 1994.
S. Muggleton. Inverse entailment and Progol. New Generation Computing, 13: 245–286, 1995.
C. Nédellec, C. Rouveirol, H. Adé, F. Bergadano, and B. Tausend. Declarative bias in Inductive Logic Programming. In [18], pp. 82–103.
D. Page. ILP: Just do it. In J.W. Lloyd (ed.), Proceedings of the First International Conference on Computational Logic, Lecture Notes in Artificial Intelligence 1861, pp. 25–40. Springer-Verlag, 2000.
G. Plotkin. A note on inductive generalisation. Machine Intelligence 5, B. Meltzer and D. Michie (eds.), pp. 153–163. North-Holland, 1970.
G. Plotkin. A further note on inductive generalisation. Machine Intelligence 6, B. Meltzer and D. Michie (eds.), pp. 101–124. North-Holland, 1971.
F. Provost and T. Fawcett. Robust classification for imprecise environments. Machine Learning 42(3): 203–231, 2001.
J.R. Quinlan. Learning logical definitions from relations. Machine Learning, 5(3): 239–266, 1990.
J.R. Quinlan. Boosting, bagging, and C4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725–730. AAAI Press, 1996.
I. Savnik and P.A. Flach. Discovery of multivalued dependencies from relations. Intelligent Data Analysis, 4(3,4): 195–211, 2000.
M. Sebag and C. Rouveirol. Constraint Inductive Logic Programming. In [18], pp. 277–294.
E.Y. Shapiro. Inductive inference of theories from facts. Technical Report 192, Computer Science Department, Yale University, 1981.
E.Y. Shapiro. Algorithmic program debugging. MIT Press, 1983.
E. Sommer. Rulebase stratifications: an approach to theory restructuring. In S. Wrobel (ed.), Proceedings of the Fourth International Workshop on Inductive Logic Programming, GMD-Studien 237, pp. 377–390, 1994.
A. Srinivasan, S. Muggleton, R.D. King, and M.J.E. Sternberg. Mutagenesis: ILP experiments in a non-determinate biological domain. In S. Wrobel (ed.), Proceedings of the Fourth International Workshop on Inductive Logic Programming, GMD-Studien 237, pp. 217–232, 1994.
A. Srinivasan, R.D. King, S. Muggleton, and M.J.E. Sternberg. Carcinogenesis prediction using inductive logic programming. In N. Lavrač, E. Keravnou, and B. Zupan (eds.), Intelligent Data Analysis in Medicine and Pharmacology, pp. 243–260. Kluwer, 1997.
I. Stahl. Compression measures in ILP. In [18], pp. 295–307.
L. Valiant. A theory of the learnable. Communications of the ACM 27: 1134–1142, 1984.
I.H. Witten and E. Frank. Data Mining: Practical machine learning tools and techniques with Java implementations. Morgan Kauffman, 2000.
S. Wrobel. An algorithm for multi-relational discovery of subgroups. In Proceedings of the First European Symposium on Principles of Data Mining and Knowledge Discovery, pp. 78–87. Springer-Verlag, 1997.
XSB Group Home Page: http://www.cs.sunysb.edu/~sbprolog/.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Flach, P., Lavrač, N. (2002). Learning in Clausal Logic: A Perspective on Inductive Logic Programming. In: Kakas, A.C., Sadri, F. (eds) Computational Logic: Logic Programming and Beyond. Lecture Notes in Computer Science(), vol 2407. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45628-7_17
Download citation
DOI: https://doi.org/10.1007/3-540-45628-7_17
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43959-2
Online ISBN: 978-3-540-45628-5
eBook Packages: Springer Book Archive