Skip to main content

Learning in Clausal Logic: A Perspective on Inductive Logic Programming

  • Chapter
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2407))

Abstract

Inductive logic programming is a form of machine learning from examples which employs the representation formalism of clausal logic. One of the earliest inductive logic programming systems was Ehud Shapiro’s Model Inference System [90], which could synthesise simple recursive programs like append/3. Many of the techniques devised by Shapiro, such as top-down search of program clauses by refinement operators, the use of intensional background knowledge, and the capability of inducing recursive clauses, are still in use today. On the other hand, significant advances have been made regarding dealing with noisy data, efficient heuristic and stochastic search methods, the use of logical representations going beyond definite clauses, and restricting the search space by means of declarative bias. The latter is a general term denoting any form of restrictions on the syntactic form of possible hypotheses. These include the use of types, input/output mode declarations, and clause schemata. Recently, some researchers have started using alternatives to Prolog featuring strong typing and real functions, which alleviate the need for some of the above ad-hoc mechanisms. Others have gone beyond Prolog by investigating learning tasks in which the hypotheses are not definite clause programs, but for instance sets of indefinite clauses or denials, constraint logic programs, or clauses representing association rules. The chapter gives an accessible introduction to the above topics. In addition, it outlines the main current research directions which have been strongly influenced by recent developments in data mining and challenging real-life applications.

This is a preview of subscription content, log in via an institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. R. Agrawal, H. Mannila, R. Srikant, H. Toivonen, and A.I. Verkamo. Fast discovery of association rules. In U.M. Fayyad, G. Piatetski-Shapiro, P. Smyth, and R. Uthurusamy (eds.), Advances in Knowledge Discovery and Data Mining, pp. 307–328. AAAI Press, 1996.

    Google Scholar 

  2. J. J. Alferes, J. A. Leite, L. M. Pereira, H. Przymusinska, and T. C. Przymusinski. Dynamic logic programming, In A. Cohn, L. Schubert and S. Shapiro (eds.), Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning, pp. 98–109. Morgan Kaufmann, 1998.

    Google Scholar 

  3. D. Angluin, M. Frazier, and L. Pitt. Learning conjunctions of Horn clauses. Machine Learning, 9(2/3): 147–164, 1992.

    Article  MATH  Google Scholar 

  4. H. Blockeel and L. De Raedt. Top-down induction of first-order logical decision trees. Artificial Intelligence 101(1–2): 285–297, June 1998.

    Google Scholar 

  5. H. Blockeel, L. De Raedt, N. Jacobs, and B. Demoen. Scaling up inductive logic programming by learning from interpretations. Data Mining and Knowledge Discovery, 3(1): 59–93, 1999.

    Article  Google Scholar 

  6. H. Blockeel, L. Dehaspe, B. Demoen, G. Janssens, J. Ramon, and H. Vandecasteele. Executing query packs in ILP. In J. Cussens and A. Frisch (eds.), Proceedings of the Tenth International Conference on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1866, pp. 60–77. Springer-Verlag, 2000.

    Google Scholar 

  7. A.F. Bowers, C. Giraud-Carrier, and J.W. Lloyd. Classification of individuals with complex structure. In P. Langley (ed.), Proceedings of the Seventeenth International Conference on Machine Learning, pp. 81–88. Morgan Kaufmann, 2000.

    Google Scholar 

  8. I. Bratko and S. Muggleton. Applications of Inductive Logic Programming. Communications of the A CM 38(11): 65–70, November 1995.

    Google Scholar 

  9. L. Breiman. Bagging predictors. Machine Learning 24(2): 123–140, 1996.

    MATH  MathSciNet  Google Scholar 

  10. G. Brewka. Well-founded semantics for extended logic programs with dynamic preferences. Journal of Artificial Intelligence Research, 4: 19–36, 1996.

    MATH  MathSciNet  Google Scholar 

  11. G. Brewka and T. Eiter. Preferred answer sets. In A. Cohn, L. Schubert and S. Shapiro (eds.), Proceedings of the Sixth International Conference on Principles of Knowledge Representation and Reasoning, pp. 89–97. Morgan Kaufmann, 1998.

    Google Scholar 

  12. W.W. Cohen. Recovering software specifications with inductive logic programming. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 142–148. The MIT Press, 1994.

    Google Scholar 

  13. W.W. Cohen and H. Hirsh. Learning the CLASSIC Description Logic: Theoretical and Experimental Results. In J. Doyle, E. Sandewall, and P. Torasso (eds.), Proceedings of the Fourth International Conference on Principles of Knowledge Representation and Reasoning, pp. 121–133. Morgan Kaufmann, 1994.

    Google Scholar 

  14. J. Cussens. Notes on inductive logic programming methods in natural language processing (European work). Unpublished manuscript, 1998. ftp://ftp.cs.york.ac.uk/pub/ML_GROUP/Papers/ilp98tut.ps.gz.

  15. J. Cussens and S. Džeroski (eds.). Learning Language in Logic. Lecture Notes in Artificial Intelligence 1925, Springer-Verlag, 2000.

    Google Scholar 

  16. L. Dehaspe and L. De Raedt. Mining association rules in multiple relations. In S. Džeroski and N. Lavrač (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 125–132. Springer-Verlag, 1997.

    Google Scholar 

  17. L. Dehaspe, H. Toivonen, and R.D. King. Finding frequent substructures in chemical compounds. In R. Agrawal, P. Stolorz, and G. Piatetsky-Shapiro (eds.), Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining, pp. 30–36. AAAI Press, 1998.

    Google Scholar 

  18. L. De Raedt (ed.). Advances in Inductive Logic Programming. IOS Press, 1996.

    Google Scholar 

  19. L. De Raedt. Logical settings for concept-learning. Artificial Intelligence, 95(1): 187–201, 1997.

    Article  MATH  MathSciNet  Google Scholar 

  20. L. De Raedt and H. Blockeel. Using logical decision trees for clustering. In N. Lavrač and S. Džeroski (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 133–140. Springer-Verlag, 1997.

    Google Scholar 

  21. L. De Raedt and L. Dehaspe. Clausal discovery. Machine Learning, 26(2/3): 99–146, 1997.

    Article  MATH  Google Scholar 

  22. L. De Raedt. An inductive logic programming query language for database mining (extended abstract). In J. Calmet and J. Plaza (eds.), Proceedings of the Fourth Workshop on Artificial Intelligence and Symbolic Computation, Lecture Notes in Artificial Intelligence 1476. Springer-Verlag, 1998.

    Google Scholar 

  23. L. De Raedt. A perspective on inductive logic programming. In K. Apt, V. Marek, M. Truszezynski, and D.S. Warren (eds.), The logic programming paradigm: current trends and future directions. Springer-Verlag, 1999.

    Google Scholar 

  24. L. De Raedt. A logical database mining query language. In J. Cussens and A. Frisch, Proceedings of the Tenth International Conference on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1866, pp. 78–92. Springer-Verlag, 2000.

    Google Scholar 

  25. Y. Dimopoulos and A.C. Kakas. Learning non-monotonic logic programs: learning exceptions. In N. Lavrač and S. Wrobel (eds.), Proceedings of the Eighth European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 912, pp. 122–138. Springer-Verlag, 1995.

    Google Scholar 

  26. Y. Dimopoulos and A.C. Kakas. Abduction and inductive learning. In [18], pp. 144–171.

    Google Scholar 

  27. Y. Dimopoulos, S. Džeroski, and A.C. Kakas. Integrating Explanatory and Descriptive Induction in ILP. In M.E. Pollack (ed.), Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence, pp. 900–907. Morgan Kaufmann, 1997.

    Google Scholar 

  28. B. Dolšak and S. Muggleton. The application of inductive logic programming to finite-element mesh design. In [76], pp. 453–472.

    Google Scholar 

  29. S. Džeroski and L. Todorovski. Discovering dynamics: From inductive logic programming to machine discovery. In Proceedings of the Tenth International Conference on Machine Learning, pp. 97–103. Morgan Kaufmann, 1993.

    Google Scholar 

  30. S. Džeroski and I. Bratko. Applications of Inductive Logic Programming. In [18], pp. 65–81.

    Google Scholar 

  31. S. Džeroski, L. De Raedt, and H. Blockeel. Relational reinforcement learning. In J. Shavlik (ed.), Proceedings of the Fifteenth International Conference on Machine Learning, pp. 136–143. Morgan Kaufmann, 1998.

    Google Scholar 

  32. S. Džeroski and N. Lavrač, eds. Relational Data Mining. Springer-Verlag, 2001. In press.

    Google Scholar 

  33. W. Emde. Learning of characteristic concept descriptions from small sets to classified examples. In F. Bergadano and L. De Raedt (eds.), Proceedings of the Seventh European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 784, pp. 103–121. Springer-Verlag, 1994.

    Google Scholar 

  34. W. Emde and D. Wettschereck. Relational instance-based learning. In L. Saitta (ed.), Proceedings of the Thirteenth International Conference on Machine Learning, pp. 122–130. Morgan Kaufmann, 1996.

    Google Scholar 

  35. P.A. Flach. Predicate invention in inductive data engineering. In P. Brazdil (ed.), Proceedings of the Sixth European Conference on Machine Learning, Lecture Notes in Artificial Intelligence 667, pp. 83–94. Springer-Verlag, 1993.

    Google Scholar 

  36. P.A. Flach. Simply Logical-intelligent reasoning by example. John Wiley, 1994.

    Google Scholar 

  37. P.A. Flach. Conjectures-an inquiry concerning the logic of induction. PhD thesis, Tilburg University, April 1995.

    Google Scholar 

  38. P.A. Flach. Rationality postulates for induction. In Y. Shoham (ed.), Proceedings of the Sixth International Conference on Theoretical Aspects of Rationality and Knowledge, pp. 267–281. Morgan Kaufmann, 1996.

    Google Scholar 

  39. P.A. Flach. Normal forms for Inductive Logic Programming. In N. Lavrač and S. Džeroski (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 149–156. Springer-Verlag, 1997.

    Google Scholar 

  40. P.A. Flach, C. Giraud-Carrier, and J.W. Lloyd. Strongly typed inductive concept learning. In D. Page (ed.), Proceedings of the Eighth International Conference on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1446, pp. 185–194. Springer-Verlag, 1998.

    Google Scholar 

  41. P.A. Flach and I. Savnik. Database dependency discovery: a machine learning approach. AI Communications, 12(3): 139–160, November 1999.

    Google Scholar 

  42. P.A. Flach and N. Lachiche. 1BC: A first-order Bayesian classifier. In S. Džeroski and P.A. Flach (eds.), Proceedings of the Ninth International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1634, pp. 92–103. Springer-Verlag, 1999.

    Google Scholar 

  43. P.A. Flach and A.C. Kakas (eds.) Abduction and Induction: Essays on their Relation and Integration. Kluwer, 2000.

    Google Scholar 

  44. P.A. Flach and N. Lachiche. Confirmation-guided discovery of first-order rules with Tertius. Machine Learning, 42(1/2): 61–95, 2001.

    Article  MATH  Google Scholar 

  45. Y. Freund and R.E. Shapire. Experiments with a new boosting algorithm. In L. Saitta (ed.), Proceedings of the Thirteenth International Conference on Machine Learning, 148–156. Morgan Kaufmann, 1996.

    Google Scholar 

  46. A. Giordana and C. Sale. Learning structured concepts using genetic algorithms. In D. Sleeman (ed.), Proceedings of the Ninth International Workshop on Machine Learning, pp. 169–178. Morgan Kaufmann, 1992.

    Google Scholar 

  47. G. Gottlob. Subsumption and implication. Information Processing Letters 24: 109–111, 1987.

    Article  MATH  MathSciNet  Google Scholar 

  48. D.T. Hau and E.W. Coiera. Learning qualitative models of dynamic systems. Machine Learning, 26(2/3): 177–212, 1997.

    Article  MATH  Google Scholar 

  49. N. Helft. Induction as nonmonotonic inference. In R.J. Brachman, H.J. Levesque, and R. Reiter (eds.), Proceedings of the First International Conference on Principles of Knowledge Representation and Reasoning, pp. 149–156. Morgan Kaufmann, 1989.

    Google Scholar 

  50. P. Idestam-Almquist. Generalization of clauses. PhD thesis, Stockholm University, October 1993.

    Google Scholar 

  51. P. Idestam-Almquist. Generalization of clauses under implication. Journal of Artificial Intelligence Research, 3: 467–489, 1995.

    MATH  Google Scholar 

  52. A.C. Kakas and F. Riguzzi. Learning with abduction. In S. Džeroski and N. Lavrač (eds.), Proceedings of the Seventh International Workshop on Inductive Logic Programming, Lecture Notes in Artificial Intelligence 1297, pp. 181–188. Springer-Verlag, 1997.

    Google Scholar 

  53. A. Karalič and I. Bratko. First-order regression. Machine Learning, 26(2/3): 147–176, 1997.

    Article  MATH  Google Scholar 

  54. M. Kifer and V.S. Subrahmanian. Generalized annotated logic programs. Journal of Logic Programming, 1992.

    Google Scholar 

  55. R.D. King, S. Muggleton, R. Lewis, and M.J.E. Sternberg. Drug design by machine learning: The use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. In Proceedings of the National Academy of Sciences of the USA 89(23): 11322–11326, 1992.

    Article  Google Scholar 

  56. R.D. King, A. Karwath, A. Clare, and L. Dehaspe. Genome scale prediction of protein functional class from sequence using data mining. In Proceedings of the Sixth International Conference on Knowledge Discovery and Data Mining, pp. 384–398. ACM Press, New York, 2000.

    Chapter  Google Scholar 

  57. R.D. King, A. Karwath, A. Clare, and L. Dehaspe. Accurate prediction of protein functional class in the M.tuberculosis and E.coli genomes using data mining. Yeast (Comparative and Functional Genomics, 17: 283–293, 2000.

    Google Scholar 

  58. M. Kirsten and S. Wrobel. Relational distance-based clustering. In D. Page (ed.) Proceedings of the Eighth International Conference on Inductive Logic Programming, pp. 261–270, Lecture Notes in Artificial Intelligence 1446. Springer-Verlag, 1998.

    Google Scholar 

  59. P. van der Laag. An analysis of refinement operators in Inductive Logic Programming. PhD Thesis, Erasmus University Rotterdam, December 1995.

    Google Scholar 

  60. E. Lamma, F. Riguzzi, and L. M. Pereira. Agents learning in a three-valued logical setting. In A. Panayiotopoulos (ed.), Proceedings of the Workshop on Machine Learning and Intelligent Agents, in conjunction with Machine Learning and Applications, Advanced Course on Artificial Intelligence (ACAI-99), Chania, Greece, 1999.

    Google Scholar 

  61. E. Lamma, F. Riguzzi, and L. M. Pereira. Strategies in combined learning via Logic Programs. Machine Learning, 38(1/2): 63–87, 2000.

    Article  MATH  Google Scholar 

  62. N. Lavrač, S. Džeroski, and M. Grobelnik. Learning nonrecursive definitions of relations with LINUS. In Y. Kodratoff (ed.) Proceedings of the Fifth European Working Session on Learning, Lecture Notes in Artificial Intelligence 482, pp. 265–281. Springer-Verlag, 1991.

    Google Scholar 

  63. N. Lavrač and S. Džeroski. Inductive Logic Programming: techniques and applications. Ellis Horwood, 1994.

    Google Scholar 

  64. N. Lavrač, S. Džeroski, and I. Bratko. Handling imperfect data in Inductive Logic Programming. In [18], pp. 48–64.

    Google Scholar 

  65. N. Lavrač and P.A. Flach. An extended transformation approach to Inductive Logic Programming. ACM Transactions on Computational Logic, 2(4): 458–494, 2001.

    Article  Google Scholar 

  66. C.J. Kennedy. Strongly typed evolutionary programming. PhD Thesis, University of Bristol, 2000.

    Google Scholar 

  67. J.W. Lloyd. Programming in an integrated functional and logic programming language. Journal of Functional and Logic Programming, 1999(3).

    Google Scholar 

  68. D.W. Loveland and G. Nadathur. Proof procedures for logic programming. Handbook of Logic in Artificial Intelligence and Logic Programming, Vol. 5, D.M. Gabbay, C.J. Hogger, and J.A. Robinson (eds.), Oxford University Press, pp. 163–234, 1998.

    Google Scholar 

  69. D. Michie, S. Muggleton, D. Page, and A. Srinivasan. To the international computing community: A new East-West challenge. Technical report, Oxford University Computing laboratory, Oxford,UK, 1994.

    Google Scholar 

  70. T.M. Mitchell. Machine Learning. McGraw-Hill, 1997.

    Google Scholar 

  71. T.M. Mitchell. Does machine learning really work? AI Magazine 18(3): 11–20, 1997.

    Google Scholar 

  72. F. Mizoguchi, H. Ohwada, M. Daidoji, and S. Shirato. Using inductive logic programming to learn classification rules that identify glaucomatous eyes. In N. Lavrač, E. Keravnou, and B. Zupan (eds.), Intelligent Data Analysis in Medicine and Pharmacology, pp. 227–242. Kluwer, 1997.

    Google Scholar 

  73. R.J. Mooney and M.E. Califf. Induction of first-order decision lists: Results on learning the past tense of English verbs. Journal of Artificial Intelligence Research 3: 1–24, 1995.

    Article  Google Scholar 

  74. I. Mozetič. Learning of qualitative models. In I. Bratko and N. Lavrač (eds.) Progress in Machine Learning, pp. 201–217. Sigma Press, 1987.

    Google Scholar 

  75. S. Muggleton. Inductive Logic Programming. New Generation Computing, 8(4): 295–317, 1991. Also in [76], pp. 3–27.

    MATH  Google Scholar 

  76. S. Muggleton (ed.). Inductive Logic Programming. Academic Press, 1992.

    Google Scholar 

  77. S. Muggleton and C. Feng. Efficient induction of logic programs. In [76], pp. 281–298.

    Google Scholar 

  78. S. Muggleton, R.D. King, and M.J.E. Sternberg. Protein secondary structure prediction using logic. Protein Engineering 7: 647–657, 1992.

    Google Scholar 

  79. S. Muggleton and L. De Raedt. Inductive Logic Programming: theory and methods. Journal of Logic Programming, 19/20: 629–679, 1994.

    Article  Google Scholar 

  80. S. Muggleton. Inverse entailment and Progol. New Generation Computing, 13: 245–286, 1995.

    Article  Google Scholar 

  81. C. Nédellec, C. Rouveirol, H. Adé, F. Bergadano, and B. Tausend. Declarative bias in Inductive Logic Programming. In [18], pp. 82–103.

    Google Scholar 

  82. D. Page. ILP: Just do it. In J.W. Lloyd (ed.), Proceedings of the First International Conference on Computational Logic, Lecture Notes in Artificial Intelligence 1861, pp. 25–40. Springer-Verlag, 2000.

    Google Scholar 

  83. G. Plotkin. A note on inductive generalisation. Machine Intelligence 5, B. Meltzer and D. Michie (eds.), pp. 153–163. North-Holland, 1970.

    Google Scholar 

  84. G. Plotkin. A further note on inductive generalisation. Machine Intelligence 6, B. Meltzer and D. Michie (eds.), pp. 101–124. North-Holland, 1971.

    Google Scholar 

  85. F. Provost and T. Fawcett. Robust classification for imprecise environments. Machine Learning 42(3): 203–231, 2001.

    Article  MATH  Google Scholar 

  86. J.R. Quinlan. Learning logical definitions from relations. Machine Learning, 5(3): 239–266, 1990.

    Google Scholar 

  87. J.R. Quinlan. Boosting, bagging, and C4.5. In Proceedings of the Thirteenth National Conference on Artificial Intelligence, pp. 725–730. AAAI Press, 1996.

    Google Scholar 

  88. I. Savnik and P.A. Flach. Discovery of multivalued dependencies from relations. Intelligent Data Analysis, 4(3,4): 195–211, 2000.

    MATH  Google Scholar 

  89. M. Sebag and C. Rouveirol. Constraint Inductive Logic Programming. In [18], pp. 277–294.

    Google Scholar 

  90. E.Y. Shapiro. Inductive inference of theories from facts. Technical Report 192, Computer Science Department, Yale University, 1981.

    Google Scholar 

  91. E.Y. Shapiro. Algorithmic program debugging. MIT Press, 1983.

    Google Scholar 

  92. E. Sommer. Rulebase stratifications: an approach to theory restructuring. In S. Wrobel (ed.), Proceedings of the Fourth International Workshop on Inductive Logic Programming, GMD-Studien 237, pp. 377–390, 1994.

    Google Scholar 

  93. A. Srinivasan, S. Muggleton, R.D. King, and M.J.E. Sternberg. Mutagenesis: ILP experiments in a non-determinate biological domain. In S. Wrobel (ed.), Proceedings of the Fourth International Workshop on Inductive Logic Programming, GMD-Studien 237, pp. 217–232, 1994.

    Google Scholar 

  94. A. Srinivasan, R.D. King, S. Muggleton, and M.J.E. Sternberg. Carcinogenesis prediction using inductive logic programming. In N. Lavrač, E. Keravnou, and B. Zupan (eds.), Intelligent Data Analysis in Medicine and Pharmacology, pp. 243–260. Kluwer, 1997.

    Google Scholar 

  95. I. Stahl. Compression measures in ILP. In [18], pp. 295–307.

    Google Scholar 

  96. L. Valiant. A theory of the learnable. Communications of the ACM 27: 1134–1142, 1984.

    Article  MATH  Google Scholar 

  97. I.H. Witten and E. Frank. Data Mining: Practical machine learning tools and techniques with Java implementations. Morgan Kauffman, 2000.

    Google Scholar 

  98. S. Wrobel. An algorithm for multi-relational discovery of subgroups. In Proceedings of the First European Symposium on Principles of Data Mining and Knowledge Discovery, pp. 78–87. Springer-Verlag, 1997.

    Google Scholar 

  99. XSB Group Home Page: http://www.cs.sunysb.edu/~sbprolog/.

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Flach, P., Lavrač, N. (2002). Learning in Clausal Logic: A Perspective on Inductive Logic Programming. In: Kakas, A.C., Sadri, F. (eds) Computational Logic: Logic Programming and Beyond. Lecture Notes in Computer Science(), vol 2407. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45628-7_17

Download citation

  • DOI: https://doi.org/10.1007/3-540-45628-7_17

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-43959-2

  • Online ISBN: 978-3-540-45628-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics