Skip to main content
Springer Nature Link
Log in

Preceding Rule Induction with Instance Reduction Methods

  • Conference paper
Machine Learning and Data Mining in Pattern Recognition (MLDM 2013)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7988))

  • 4419 Accesses

Abstract

A new prepruning technique for rule induction is presented which applies instance reduction before rule induction.  An empirical evaluation records the predictive accuracy and size of rule-sets generated from 24 datasets from the UCI Machine Learning Repository. Three instance reduction algorithms (Edited Nearest Neighbour, AllKnn and DROP5) are compared. Each one is used to reduce the size of the training set, prior to inducing a set of rules using Clark and Boswell’s modification of CN2. A hybrid instance reduction algorithm (comprised of AllKnn and DROP5) is also tested.  For most of the datasets, pruning the training set using ENN, AllKnn or the hybrid significantly reduces the number of rules generated by CN2, without adversely affecting the predictive performance. The hybrid achieves the highest average predictive accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Aha, D.W., Kibler, D., Albert, M.K.: Instance – based learning algorithm. Machine Learning 6, 37–66 (1991)

    Google Scholar 

  2. Brunk, C., Pazzini, M.: An investigation of noise-tolerant relational concept learning algorithms. In: Proceedings of the 8th International Workshop on Machine Learning, Evanston, Illinois, pp. 389–393 (1991)

    Google Scholar 

  3. Clark, P., Boswell, R.: Rule induction with CN2: some recent improvements. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS, vol. 482, pp. 151–163. Springer, Heidelberg (1991)

    Chapter  Google Scholar 

  4. Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3, 261–283 (1989)

    Google Scholar 

  5. Cohen, W.: Efficient pruning methods for separate-and-conquer rule learning systems. In: Bajcsy, R. (ed.) Proceedings of the 13th International Joint Conference on Artificial Intelligence, pp. 988–994. Morgan Kaufmann, Chambery (1993)

    Google Scholar 

  6. Cohen, W.: Fast effective rule induction. In: Prieditis, A., RussellIn, S.J. (eds.) Machine Learning: Proceedings of the 12th International Conference, vol. 3, pp. 115–123. Morgan Kaufmann, Lake Tahoe (1995)

    Google Scholar 

  7. Cohen, W., Singer, Y.: A simple, fast and effective rule learner. In: Hendler, J., Subramanian, D. (eds.) Proceedings of the Sixteenth National Conference on Artificial Intelligence, pp. 335–342. AAAI/MIT Press, Menlo Park (1999)

    Google Scholar 

  8. Dain, O., Cunningham, R., Boyer, S.: IREP++ a faster rule learning algorithm. In: Michael, W., Dayal, U., Kamath, C., Davis, B. (eds.) Proceeding Fourth SIAM Int. Conf. Data Mining, Lake Buena Vista, FL, USA, pp. 138–146 (2004)

    Google Scholar 

  9. Gamberger, D., Lavrac, N., Dzeroski, S.: Noise Elimination in inductive concept learning: A case study in medical diagnosis. In: Arikawa, S., Sharma, A.K. (eds.) ALT 1996. LNCS, vol. 1160, pp. 199–212. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  10. El Hindi, K., Alakhras, M.: Eliminating border instance to avoid overfitting. In: dos Reis, A.P. (ed.) Proceeding of Intelligent Systems and Agents 2009, pp. 93–99. IADIS press, Algarve (2009)

    Google Scholar 

  11. Fürnkranz, J., Widmer, G.: Incremental reduced error pruning. In: Cohen, W., Hirsh, H. (eds.) Proceedings of the 11th International Conference on Machine learning (ML 1994), pp. 70–77. Morgan Kaufmann, New Brunswick (1994)

    Google Scholar 

  12. Gates, G.W.: The reduced nearest neighbor rule. Institute of Electrical and Electronics Engineers Transactions on Information Theory 18(3), 431–433 (1972)

    Article  Google Scholar 

  13. Grudziński, K., Grochowski, M., Duch, W.: Pruning Classification Rules with Reference Vector Selection Methods. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part I. LNCS, vol. 6113, pp. 347–354. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  14. Grudzinski, K.: EkP: A fast minimization – based prototype selection algorithm. In: Intelligent Information System XVI, pp. 45–53. Academic Publishing House EXIT, Warsaw (2008)

    Google Scholar 

  15. Hart, P.E.: The condensed nearest neighbor rules. Institute of Electrical and Electronics Engineers Transactions on Information Theory 14(3), 515–516 (1968)

    Article  Google Scholar 

  16. Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Mellish, C. (ed.) Proceedings of 14th International Joint Conference on Artificial Intelligence, pp. 1137–1143. Morgan Kaufmann, San Francisco (1995)

    Google Scholar 

  17. Lukasz, A., Krzysztof, J.: Highly scalable and robust rule learner: performance evaluation and comparison. IEEE Transactions on Systems, Man, and Cybernetics, Part B 36(1), 32–53 (2006)

    Article  Google Scholar 

  18. Murphy, P.M., Aha, D.W.: UCI repository of Machine Learning Data bases. available by anonymous ftp to ics.uci.edu in the pub/machine-learning-databases directory (1994)

    Google Scholar 

  19. Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  20. Othman, O., El Hindi, K.: Rule reduction technique for RISE algorithm. Advances in Modeling, Series B: Signal Processing and Pattern Recognition 47, 2 (2004)

    Google Scholar 

  21. Pham, D.T., Bigot, S., Dimov, S.: A rule merging technique for handling noise in inductive learning. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 218 (C), 1255–1268 (2004)

    Article  Google Scholar 

  22. Ritter, G.L., Woodruff, H.B., Lowry, S.R., Isenhour, T.L.: An Algorithm for a Selective Nearest Neighbor Decision Rule. IEEE Transactions on Information Theory 21(6), 665–669 (1975)

    Article  MATH  Google Scholar 

  23. Schapire, R., Singer, Y.: Improved boosting algorithms using confidence-rated predictions. In: Bartlett, P.L., Mansour, Y. (eds.) Proceeding COLT 1998 Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 80–91. ACM press, New York (1998)

    Chapter  Google Scholar 

  24. Shehzad, K.: Simple Hybrid and Incremental Post-Pruning Techniques for Rule Induction. IEEE Transactions on Knowledge and Data Engineering (99), 1–6 (2011)

    Google Scholar 

  25. Tomek, I.: An experiment with the edited nearest-neighbor rule. IEEE Transactions on Systems, Man, and Cybernetics 6(6), 448–452 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  26. Weiss, S., Indurkhya, N.: Reduced complexity rule induction. In: Mylopouslos, J., Reiter, R. (eds.) Proceedings of 12th International Joint Conference on Artificial Intelligence, pp. 678–684. Morgan Kauffmann, Sydney (1991)

    Google Scholar 

  27. Wilson, D.L.: Asymptotic properties of nearest neighbor rules Using Edited Data. IEEE Transactions on Systems, Man, and Cybernetics 2(3), 408–421 (1972)

    Article  MATH  Google Scholar 

  28. Wilsson, D.R., Martinez, T.R.: Instance Pruning Technique. In: Fisher, D.H. (ed.) Machine Learning: Proceedings of the Fourteenth International Conference (ICML 1997), pp. 403–411. Morgan Kauffmann, San Francisco (1997)

    Google Scholar 

  29. Wilsson, D.R., Martinez, T.R.: Reduction techniques for instance based learning algorithms. Machine Learning 38(3), 257–286 (2000)

    Article  Google Scholar 

  30. Zhao, K.P., Zhou, S.G., Guan, J.H., Zhou, A.Y.: C-Pruner: An improved instance pruning algorithm. In: Proceedings of the 2th International Conference on Machine Learning and Cybernetics, Sheraton Hotel, Xi’an, China, vol. 1, pp. 94–99. IEEE, Piscataway (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. King Abdullah II School for Information Technology, Jordan University, Amman, Jordan

    Osama Othman

  2. School of Computing, Scienc and Engineering, Newton Building, The University of Salford, Greater Manchester, M5 4WT, England, UK

    Christopher H. Bryant

Authors
  1. Osama Othman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher H. Bryant
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Vision and Applied Computer Sciences, IBaI, Leipzig, Germany

    Petra Perner

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Othman, O., Bryant, C.H. (2013). Preceding Rule Induction with Instance Reduction Methods. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2013. Lecture Notes in Computer Science(), vol 7988. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39712-7_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39712-7_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39711-0

  • Online ISBN: 978-3-642-39712-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics