Skip to main content
SpringerLink
Log in

An Empirical Evaluation of Bagging in Inductive Logic Programming

  • Conference paper
  • First Online:
Inductive Logic Programming (ILP 2002)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2583))

Included in the following conference series:

Abstract

Ensembles have proven useful for a variety of applications, with a variety of machine learning approaches. While Quinlan has applied boosting to FOIL, the widely-used approach of bagging has never been employed in ILP. Bagging has the advantage over boosting that the di.erent members of the ensemble can be learned and used in parallel. This advantage is especially important for ILP where run-times often are high. We evaluate bagging on three di.erent application domains using the complete-search ILP system, Aleph. We contrast bagging with an approach where we take advantage of the non-determinism in ILP search, by simply allowing Aleph to run multiple times, each time choosing “seed” examples at random.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. E. Alpaydin. Multiple networks for function learning. In IEEE International Conference on Neural Networks, pages 9–14, 1993.

    Google Scholar 

  2. J. Basney and M. Livny. Managing network resources in Condor. In Proceedings of the Ninth IEEE Symposium on High Performance Distributed Computing (HPDC9), Pittsburgh, Pennsylvania, pages 298–299, Aug 2000.

    Google Scholar 

  3. H. Blockeel, L. Dehaspe, B. Demoen, G. Janssens, J. Ramon, and H. Vandecasteele. Executing query packs in ILP. In J. Cussens and A. Frisch, editors, Proceedings of the 10th International Conference on Inductive Logic Programming, volume 1866 of Lecture Notes in Arti.cial Intelligence, pages 60–77. Springer-Verlag, 2000.

    Google Scholar 

  4. H. Blockeel, B. Demoen, G. Janssens, H. Vandecasteele, and W. Van Laer. Two advanced transformations for improving the efficiency of an ILP system. In J. Cussens and A. Frisch, editors, Proceedings of the Work-in-Progress Track at the 10th International Conference on Inductive Logic Programming, pages 43–59, 2000.

    Google Scholar 

  5. I. Bratko and M. Grobelnik. Inductive learning applied to program construction and verification. In S. Muggleton, editor, Proceedings of the 3rd International Workshop on Inductive Logic Programming, pages 279–292. J. Stefan Institute, 1993.

    Google Scholar 

  6. L. Breiman. Bagging Predictors. Machine Learning, 24(2):123–140, 1996.

    MATH  MathSciNet  Google Scholar 

  7. L. Breiman. Stacked Regressions. Machine Learning, 24(1):49–64, 1996.

    MATH  MathSciNet  Google Scholar 

  8. L. Dehaspe and L. De Raedt. Parallel inductive logic programming. In Proceedings of the MLnet Familiarization Workshop on Statistics, Machine Learning and Knowledge Discovery in Databases, 1995.

    Google Scholar 

  9. T. Dietterich. Ensemble methods in machine learning. In J. Kittler and F. Roli, editors, First International Workshop on Multiple Classifier Systems, Lecture Notes in Computer Science, pages 1–15. Springer-Verlag, 2000.

    Google Scholar 

  10. B. Dolisak and S. Muggleton. The application of ILP to finite element mesh design. In S. Muggleton, editor, Proceedings of the 1st International Workshop on Inductive Logic Programming, pages 225–242, 1991.

    Google Scholar 

  11. S. Džeroski, L. Dehaspe, B. Ruck, and W. Walley. Classification of river water quality data using machine learning. In Proceedings of the 5th International Conference on the Development and Application of Computer Techniques to Environmental Studies, 1995.

    Google Scholar 

  12. Y. Freund and R. Shapire. Experiments with a new boosting algorithm. In Proceedings of the 14th National Conference on Arti.cial Intelligence, pages 148–156. Morgan Kaufman, 1996.

    Google Scholar 

  13. J. Graham, D. Page, and A. Wild. Parallel inductive logic programming. In Proceedings of the Systems, Man, and Cybernetics Conference, 2000.

    Google Scholar 

  14. L. Hansen and P. Salamon. Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(10):993–1001, October 1990.

    Article  Google Scholar 

  15. S. Hoche and S. Wrobel. Relational learning using constrained confidence-rated boosting. In Céline Rouveirol and Michèle Sebag, editors, Proceedings of the 11th International Conference on Inductive Logic Programming, volume 2157 of Lecture Notes in Artificial Intelligence, pages 51–64. Springer-Verlag, September 2001.

    Google Scholar 

  16. R. King, S. Muggleton, and M. Sternberg. Predicting protein secondary structure using inductive logic programming. Protein Engineering, 5:647–657, 1992.

    Article  Google Scholar 

  17. A. Krogh and J. Vedelsby. Neural network ensembles, cross validation, and active learning. In G. Tesauro, D. Touretzky, and T. Leen, editors, Advances in Neural Information Processing Systems, volume 7, pages 231–238. The MIT Press, 1995.

    Google Scholar 

  18. N. Lincoln and J. Skrzypek. Synergy of clustering multiple backpropagation networks. In Advances in Neural Information Processing Systems. Morgan Kaufmann, 1989.

    Google Scholar 

  19. T. Matsui, N. Inuzuka, H. Seki, and H. Ito. Parallel induction algorithms for large samples. In S. Arikawa and H. Motoda, editors, Proceedings of the First International Conference on Discovery Science, volume 1532of Lecture Notes in Artificial Intelligence, pages 397–398. Springer-Verlag, December 1998.

    Google Scholar 

  20. J. Metz. The epidemic in a closed population with all susceptibles equally vulnerable; some results for large susceptible populations and small initial infections. Acta Biotheoretica, 27:75–123, 1978.

    Article  Google Scholar 

  21. D. W. Opitz and R. Maclin. Popular ensemble methods: An empirical study. Journal of Arti.cial Intelligence Research, 11:169–198, 1999.

    MATH  Google Scholar 

  22. D. W. Opitz and J. W. Shavlik. Actively searching for an effective neural-network ensemble. Connection Science, 8(3/4):337–353, 1996.

    Article  Google Scholar 

  23. F. J. Provost and T. Fawcett. Robust classi.cation systems for imprecise environments. In Proceedings of the 16th National Conference on Artificial Intelligence, pages 706–713, 1998.

    Google Scholar 

  24. J. R. Quinlan. Bagging, boosting, and c4.5. In Proceedings of the 14th National Conference on Artificial Intelligence, volume 1, pages 725–730, 1996.

    Google Scholar 

  25. J. R. Quinlan. Boosting first-order learning. Algorithmic Learning Theory, 7th International Workshop, Lecture Notes in Computer Science, 1160:143–155, 1996.

    Google Scholar 

  26. V. Santos Costa, A. Srinivasan, and R. Camacho. A note on two simple transformations for improving the efficiency of an ILP system. In J. Cussens and A. Frisch, editors, Proceedings of the 10th International Conference on Inductive Logic Programming, volume 1866 of Lecture Notes in Artificial Intelligence, pages 225–242. Springer-Verlag, 2000.

    Google Scholar 

  27. M. Sebag and C. Rouveirol. Tractable induction and classification in first-order logic via stochastic matching. In Proceedings of the 15th International Joint Conference on Artificial Intelligence, pages 888–893. Morgan Kaufmann, 1997.

    Google Scholar 

  28. A. Srinivasan. A study of two sampling methods for analysing large datasets with ILP. Data Mining and Knowledge Discovery, 3(1):95–123, 1999.

    Article  Google Scholar 

  29. A. Srinivasan. The Aleph Manual, 2001.

    Google Scholar 

  30. A. Srinivasan, R. King, S. Muggleton, and M. Sternberg. Carcinogenesis predictions using ILP. In S. Džeroski and N. Lavrač, editors, Proceedings of the 7th International Workshop on Inductive Logic Programming, volume 1297 of Lecture Notes in Artificial Intelligence, pages 273–287. Springer-Verlag, 1997.

    Google Scholar 

  31. J. Struyf and H. Blockeel. Efficient cross-validation in ILP. In Céline Rouveirol and Michèle Sebag}, editors, Proceedings of the 11th International Conference on Inductive Logic Programming, volume 2157 of Lecture Notes in Arti.cial Intelligence, pages 228–239. Springer-Verlag, September 2001.

    Google Scholar 

  32. F. Zelezny, A. Srinivasan, and D. Page. Lattice-search runtime distributions may be heavy-tailed. In The Twelfth International Conference on Inductive Logic Programming. Springer Verlag, July 2002.

    Google Scholar 

  33. J. Zelle and R. Mooney. Learning semantic grammars with constructive inductive logic programming. In Proceedings of the 11th National Conference on Artificial Intelligence, pages 817–822, Washington, D.C., July 1993. AAAI Press/MIT Press.

    Google Scholar 

  34. S. Zemke. Bagging imperfect predictors. In Proceedings of the International Conference on Artificial Neural Networks in Engineering, St. Louis, MI, USA. ASME Press, 1999.

    Google Scholar 

  35. M. Zweig and G. Campbell. Receiver-operative characteristic. Clinical Chemistry, 39:561–577, 1993.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Biostatistics and Medical Informatics and Department of Computer Sciences, University of Wisconsin-Madison, USA

    Inês de Castro Dutra, David Page, Vítor Santos Costa & Jude Shavlik

Authors
  1. Inês de Castro Dutra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. David Page
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vítor Santos Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jude Shavlik
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Information Technology and Engineering, University of Ottawa, 800 King Edward Ave., K1N 6N5, Ottawa, ON, Canada

    Stan Matwin

  2. School of Computer Science and Engineering, University of New South Wales, 2052, Sydney, NSW, Australia

    Claude Sammut

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

de Castro Dutra, I., Page, D., Santos Costa, V., Shavlik, J. (2003). An Empirical Evaluation of Bagging in Inductive Logic Programming. In: Matwin, S., Sammut, C. (eds) Inductive Logic Programming. ILP 2002. Lecture Notes in Computer Science(), vol 2583. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36468-4_4

Download citation

  • DOI: https://doi.org/10.1007/3-540-36468-4_4

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-00567-4

  • Online ISBN: 978-3-540-36468-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation