Abstract
The Occam's razor principle suggests that among all the correct hypotheses, the simplest hypothesis is the one which best captures the structure of the problem domain and has the highest prediction accuracy when classifying new instances. This principle is implicitly used also for dealing with noise, in order to avoid overfitting a noisy training set by rule truncation or by pruning of decision trees. This work gives a theoretical framework for the applicability of Occam's razor, developed into a procedure for eliminating noise from a training set. The results of empirical evaluation show the usefulness of the presented approach to noise elimination.
Chapter PDF
Similar content being viewed by others
References
Brodley, M.: Recursive automatic bias selection for classifier construction. Machine Learning 20 (1995) 63–94
Clark, P., Niblett, T.: The CN2 Induction Algorithm. Machine Learning 3 (1989) 261–284
Gamberger, D.: A minimization approach to propositional inductive learning. In Proc. of the 8th European Conference on Machine Learning (1995) 151–160
Gamberger, D., Lavrač, N.: Noise detection and elimination applied to noise handling in a KRK chess endgame. In Proc. of the 5th International Workshop on Inductive Logic Programming (1996) 59–75
Gamberger, D., Lavrač, N., Džeroski, S.: Noise Elimination in Inductive Concept Learning: A case study in medical diagnosis. In Proc. of the 7th International Workshop on Algorithmic Learning Theory (1996) 199–212.
Kohavi, R., Wolpert, D.H.: Bias Plus Variance Decomposition for Zero-One Loss Functions. In Proc. of the 13th International Conference on Machine Learning (1996) 275–283
Kononenko, I., Bratko, I.: Information-based evaluation criterion for classifier performance. Machine Learning 6 (1991) 67–80
Lavrač, N., Džeroski, S.: Inductive Logic Programming: Techniques and Applications. Ellis Horwood (1994)
Li, M., Vitányi, P.: An Introduction to Kolmogorov Complexity and its Applications. Springer (1993)
Quinlan, J.R.: Induction of Decision Trees. Machine Learning 1 (1986) 81–106
Quinlan, J.R.: Learning Logical Definitions from Relations. Machine Learning 5 (1990) 239–266
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1992)
Rao, R., Gordon, D., Spears, W.: For every generalization action, is there really an equal or opposite reaction? Analysis of conservation law. In Proc. of the 12th International Conference on Machine Learning (1995) 471–479
Rissanen, J.: Modeling by the shortest data description. Automatica 14 (1978) 465–471
Schaffer, C.: A conservation law for generalization performance. In Proc. of the 11th International Conference on Machine Learning (1994) 259–265
Stahl, I.: Compression Measures in ILP. In L. De Raedt (ed.): Advances in Inductive Logic Programming IOS Press (1996) 295–307
Webb, G.I.: Further Experimental Evidence against the Utility of Occam's razor. Journal of Artificial Intelligence Research 4 (1996) 397–417
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1997 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gamberger, D., Lavrač, N. (1997). Conditions for Occam's razor applicability and noise elimination. In: van Someren, M., Widmer, G. (eds) Machine Learning: ECML-97. ECML 1997. Lecture Notes in Computer Science, vol 1224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-62858-4_76
Download citation
DOI: https://doi.org/10.1007/3-540-62858-4_76
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-62858-3
Online ISBN: 978-3-540-68708-5
eBook Packages: Springer Book Archive