Skip to main content
SpringerLink
Log in

A Pathology of Bottom-Up Hill-Climbing in Inductive Rule Learning

  • Conference paper
  • First Online:
Algorithmic Learning Theory (ALT 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2533))

Included in the following conference series:

Abstract

In this paper, we close the gap between the simple and straight-forwardimplemen tations of top-down hill-climbing that can be foundin the literature, andthe rather complex strategies for greedy bottom-up generalization. Our main result is that the simple bottom-up counterpart to the top-down hill-climbing algorithm is unable to learn in domains with dispersed examples. In particular, we show that guided greedy generalization is impossible if the seed example differs in more than one attribute value from its nearest neighbor. We also perform an empirical study of the commonness of this problem is in popular benchmark datasets, andpresen t average-case andw orst-case results for the probability of drawing a pathological seed example in binary domains.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Zhang] F. Bergadano, S. Matwin, R. S. Michalski, and J. Zhang. Learning twotieredd escriptions of flexible concepts: The POSEIDON system. Machine Learning, 8:5–43, 1992.

    Google Scholar 

  2. C. L. Blake and C. J. Merz. UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html, 1998. Department of Information andComputer Science, University of California at Irvine, Irvine CA.

  3. Y. Chevaleyre and J.-D. Zucker. A framework for learning rules from multiple instance data. In L. D. Raedt and P. Flach, editors, Proceedings ofthe 12th European Conference on Machine Learning (ECML-01), pages 49–60, Freiburg, Germany, 2001. Springer-Verlag.

    Google Scholar 

  4. P. Clark and R. Boswell. Rule induction with CN2: Some recent improvements. In Proceedings ofthe 5th European Working Session on Learning (EWSL-91), pages 151–163, Porto, Portugal, 1991. Springer-Verlag.

    Google Scholar 

  5. P. Clark and T. Niblett. The CN2 induction algorithm. Machine Learning, 3(4): 261–283, 1989.

    Google Scholar 

  6. W. W. Cohen. Fast effective rule induction. In A. Prieditis and S. Russell, editors, Proceedings ofthe 12th International Conference on Machine Learning (ML-95), pages 115–123, Lake Tahoe, CA, 1995. Morgan Kaufmann.

    Google Scholar 

  7. L. De Raedt. Attribute value learning versus inductive logic programming: The missing links (extended abstract). In D. Page, editor, Proceedings ofthe 8th International Conference on Inductive Logic Programming (ILP-98), pages 1–8. Springer-Verlag, 1998.

    Google Scholar 

  8. L. De Raedt and W. Van Laer. Inductive constraint logic. In Proceedings ofthe 5th Workshop on Algorithmic Learning Theory (ALT-95). Springer-Verlag, 1995.

    Google Scholar 

  9. P. Domingos. Unifying instance-basedandrule-based induction. Machine Learning, 24:141–168, 1996.

    Google Scholar 

  10. S. Džeroski and I. Bratko. Handling noise in Inductive Logic Programming. In S. H. Muggleton and K. Furukawa, editors, Proceedings ofthe 2nd International Workshop on Inductive Logic Programming (ILP-92), number TM-1182 in ICOT Technical Memorandum, pages 109–125, Tokyo, Japan, 1992. Institue for New Generation Computer Technology.

    Google Scholar 

  11. J. Fürnkranz. Fossil: A robust relational learner. In F. Bergadano and L. De Raedt, editors, Proceedings ofthe 7th European Conference on Machine Learning (ECML-94), volume 784 of Lecture Notes in Artificial Intelligence, pages 122–137, Catania, Italy, 1994. Springer-Verlag.

    Google Scholar 

  12. J. Fürnkranz. Pruning algorithms for rule learning. Machine Learning, 27(2): 139–171, 1997.

    Article  Google Scholar 

  13. J. Fürnkranz. Separate-and-conquer rule learning. Artificial Intelligence Review, 13(1):3–54, February 1999.

    Article  MATH  Google Scholar 

  14. J. Fürnkranz and G. Widmer. Incremental Reduced Error Pruning. In W. Cohen and H. Hirsh, editors, Proceedings ofthe 11th International Conference on Machine Learning (ML-94), pages 70–77, New Brunswick, NJ, 1994. Morgan Kaufmann.

    Google Scholar 

  15. K. A. Kaufman and R. S. Michalski. An adjustable rule learner for pattern discovery using the AQ methodology. Journal ofIntelligent Information Systems, 14:199–216, 2000.

    Article  Google Scholar 

  16. R. Kohavi and G.H. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1-2):273–324, 1997. Special Issue on Relevance.

    Article  MATH  Google Scholar 

  17. N. Lavrač, P. Flach, and B. Zupan. Rule evaluation measures: A unifying view. In S. Džeroski and P. Flach, editors, Ninth International Workshop on Inductive Logic Programming (ILP’99), volume 1634 of Lecture Notes in Artificial Intelligence, pages 174–185. Springer-Verlag, June 1999.

    Google Scholar 

  18. R. S. Michalski. On the quasi-minimal solution of the covering problem. In Proceedings of the 5th International Symposium on Information Processing (FCIP-69), volume A3 (Switching Circuits), pages 125–128, Bled, Yugoslavia, 1969.

    Google Scholar 

  19. R. S. Michalski. Pattern recognition andrule-guided inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2:349–361, 1980.

    MATH  Google Scholar 

  20. R. S. Michalski, I. Mozetič, J. Hong, and N. Lavrač. The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In Proceedings ofthe 5th National Conference on Artificial Intelligence (AAAI-86), pages 1041–1045, Philadelphia, PA, 1986.

    Google Scholar 

  21. S. H. Muggleton. Inverse entailment and Progol. New Generation Computing, 13 (3,4):245–286, 1995. Special Issue on Inductive Logic Programming.

    Article  Google Scholar 

  22. S. H. Muggleton and C. Feng. Efficient induction of logic programs. In Proceedings ofthe 1st Conference on Algorithmic Learning Theory, pages 1–14, Tokyo, Japan, 1990.

    Google Scholar 

  23. J. R. Quinlan. Learning logical definitions from relations. Machine Learning, 5: 239–266, 1990.

    Google Scholar 

  24. J. R. Quinlan. Determinate literals in inductive logic programming. In Proceedings of the 8th International Workshop on Machine Learning (ML-91), pages 442–446, 1991.

    Google Scholar 

  25. J. R. Quinlan and R. M. Cameron-Jones. Induction of logic programs: FOIL and relatedsystems. New Generation Computing, 13(3,4):287–312, 1995. Special Issue on Inductive Logic Programming.

    Google Scholar 

  26. G. I. Webb. Learning disjunctive class descriptions by least generalisation. Technical Report TR C92/9, Deakin University, School of Computing & Mathematics, Geelong, Australia, September 1992.

    Google Scholar 

  27. G. I. Webb and J. W. M. Agar. Inducing diagnostic rules for glomerular disease with the DLG machine learning algorithm. Artificial Intelligence in Medicine, 4: 419–430, 1992.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Austrian Research Institute for Artificial Intelligence, Schottengasse 3, A-1010, Vienna, Austria

    Johannes Fürnkranz

Authors
  1. Johannes Fürnkranz
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Dipartimento di Tecnologie dell’Informazione, Università degli Studi di Milano, via Bramante 65, 26013, Crema (CR), Italy

    Nicolò Cesa-Bianchi

  2. Department of Computer Science, Tokyo Institute of Technology, 2-12-1, Ohokayama Meguro Ward, 152-8552, Tokyo, Japan

    Masayuki Numao

  3. Institut für Theoretische Informatik, Universität zu Lübeck, Wallstr. 40, 23560, Lübeck, Germany

    Rüdiger Reischuk

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Fürnkranz, J. (2002). A Pathology of Bottom-Up Hill-Climbing in Inductive Rule Learning. In: Cesa-Bianchi, N., Numao, M., Reischuk, R. (eds) Algorithmic Learning Theory. ALT 2002. Lecture Notes in Computer Science(), vol 2533. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36169-3_22

Download citation

  • DOI: https://doi.org/10.1007/3-540-36169-3_22

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00170-6

  • Online ISBN: 978-3-540-36169-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation