Abstract
Compression measures used in inductive learners, such as measures based on the MDL (Minimum Description Length) principle, provide a theoretically justified basis for grading candidate hypotheses. Compression-based induction is appropriate also for handling of noisy data. This paper shows that a simple compression measure can be used to detect noisy examples. A technique is proposed in which noisy examples are detected and eliminated from the training set, and a hypothesis is then built from the set of remaining examples. The separation of noise detection and hypothesis formation has the advantage that noisy examples do not influence hypothesis construction as opposed to most standard approaches to noise handling in which the learner typically tries to avoid overfitting the noisy example set. Experimental results in a KRK (king-rook-king) chess endgame domain show the potential of this novel approach to noise handling.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
B. Cestnik and I. Bratko. On estimating probabilities in tree pruning. In Proc. 5th European Working Session on Learning, 138–150, 1991.
P. Clark and T. Niblett. The CN2 induction algorithm. Machine Learning, 3(4):261–283, 1989.
K.A. De Jong and W.M. Spears. Learning concept classification rules using genetic algorithms. In Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI-91), Morgan Kaufmann, (1991) 651–656.
S. Džeroski, N. Lavrac. Inductive learning in deductive databases. IEEE Transactions on Knowledge and Data Engineering, 5(6), 939–949, 1993.
D. Gamberger. A minimization approach to propositional inductive learning. In Proceedings of the 8th European Conference on Machine Learning (ECML-95), Springer, (1995) 151–160.
D. Gamberger. Specific rule induction for medical domains. In Proc. Computer-Aided Data Analysis in Medicine, CADAM-95, 136–145. US Scientific Publishing IJS-SP-95-1, 1995.
D. Gamberger and N. Lavrač. Towards a theory of relevance in inductive concept learning. Technical report IJS-DP-7310. J. Stefan Institute, Ljubljana, 1995.
D. Gamberger, N. Lavrač and S. Džeroski. Noise elimination in inductive concept learning: A case study in medical diagnosis. In Proc. Seventh International Workshop on Algorithmic Learning Theory ALT'96, Springer 1996 (in press).
N. Lavrač, S. Dzeroski and M. Grobelnik. Learning nonrecursive definitions of relations with LINUS. In Proc. Fifth European Working Session on Learning, pages 265–281, Springer, Berlin, 1991.
N. Lavrač and S. Džeroski. Inductive learning of relations from noisy examples. In S. Muggleton (ed.) Inductive Logic Programming, 495–516. Academic Press, 1992.
N. Lavrač and S. Džeroski. Inductive Logic Programming: Techniques and Applications. Ellis Horwood (Simon & Schuster), Ellis Horwood Series in Artificial Intelligence. UK: Chichester, 1994.
N. Lavrač, D. Gamberger and S. Džeroski. An approach to dimensionality reduction in learning from deductive databases. In Proceedings of the 5th International Workshop on Inductive Logic Programming (ILP-95), Technical report, Katholieke Universiteit Leuven, 1995.
N. Lavrač, S. Džroski and I. Bratko. Handling imperfect data in inductive logic programming. In L. De Raedt (ed.) Advances in Inductive Logic Programming, 48–64. IOS Press, 1996.
J. Mingers. An empirical comparison of pruning methods for decision tree induction. Machine Learning, 4(2):227–243, 1989.
J. Mingers. An empirical comparison of selection measures for decision-tree induction. Machine Learning, 3(4):319–342, 1989.
S.H. Muggleton, M. Bain, J. Hayes-Michie and D.Michie. An experimental comparison of human and machine learning formalisms. In Proc. Sixth International Workshop on Machine Learning, 113–118, Morgan Kaufmann, San Mateo, CA, 1989.
S. Muggleton, A. Srinivasan and M. Bain. Compression, significance and accuracy. In Proc. 9th International Conference on Machine Learning, 338–347. Morgan Kaufmann, 1992.
T. Niblett T and I. Bratko. Learning decision rules in noisy domains. In M. Bramer (ed.) Research and Development in Expert Systems III, 24–25. Cambridge University Press, 1986.
R. Quinlan. Simplifying decision trees. International Journal of Man-Machine Studies, 27(3):221–234, 1987.
J.R. Quinlan. Learning logical definitions from relations. Machine Learning, 5(3): 239–266, 1990.
J. Rissanen. Modeling by shortest data description. Automatica, 14: 465–471, 1978.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gamberger, D., Lavrač, N. (1997). Noise detection and elimination applied to noise handling in a KRK chess endgame. In: Muggleton, S. (eds) Inductive Logic Programming. ILP 1996. Lecture Notes in Computer Science, vol 1314. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-63494-0_49
Download citation
DOI: https://doi.org/10.1007/3-540-63494-0_49
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-63494-2
Online ISBN: 978-3-540-69583-7
eBook Packages: Springer Book Archive