Abstract
Decision trees provide a powerful method of pattern classification. At each node in a binary tree a decision is made based upon the value of one of many possible attributes or features. The leaves of the tree represent the various classes that can be recognized. Various techniques have been used to select the feature and threshold to use at each node based upon a set of training data. Information theoretic methods are the most popular techniques used for designing each node in the tree. An alternate method uses the Kolmogorov-Smirnov test to design classification trees involving two classes. This paper presents an extension of this method that can produce a single decision tree when there are multiple classes. The relationship between this generalized Kolmogorov-Smirnov method and entropy minimization methods will be described. Experiments comparing classification results using this decision tree with results of using a Bayesian classifier will be presented.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
References
G. R. Dattatreya and L. N. Kanal, "Decision Trees in Pattern Recognition," Progress in Pattern Recognition 2, L. N. Kanal and A Rosenfeld (Editors), Elsevier Science Publishers B. V. (North-Holland), 1985.
L. Breiman, J. H. Friedman, R. A. Olshen and C. J. Stone, Classification and Regression Trees, Wadsworth & Brooks/Cole, Monterey, CA, 1984.
I. K. Sethi and G. P. R. Sarvarayudu, "Hierarchical Classifier Design Using Mutual Information," IEEE Trans. on Pattern Anal. and Machine Intell., Vol. PAMI-4, pp 441–445, 1982.
J. R. Quinlan, "Learning Efficient Classification Procedures and their Application to Chess End Games," in Machine Learning, An Artificial Intelligence Approach, R. S. Michalski, et al. Eds., Tioga Publishing Co., Palo Alto, CA pp. 463–482, 1983.
J. L. Talmon, "A multiclass nonparametric partitioning algorithm,” Pattern Recognition Letters, vol. 4, pp 31–38, 1986.
J. H. Friedman, "A Recursive Partitioning Decision Rule for Nonparametric Classification," IEEE Trans. on Computers, Vol C-26, pp. 404–408, April 1977.
E. M. Rounds, "A Combined Nonparametric Approach to Feature Selection and Binary Decision Tree Design," Proc. 1979 IEEE Computer Society Conf. on Pattern Recognition and Image Processign, pp. 38–43, 1979.
R. E. Haskell, G. Castelino and B. Mirshab, "Computer Learning Using Binary Tree Classifiers," Proc. 1988 Rochester Forth Conference on Programming Environments, pp. 77–78, June 14–18, 1988.
C. E. Shannon, "A Mathematical Theory of Communication," Bell Syst. Tech. J., Vol. 27, pp. 379–423, 1948.
S. Watanabe, "Pattern Recognition as a Quest for Minimum Entropy," Pattern Recognition, Vol. 13, pp. 381–387, 1981.
J. R. Quinlan, "Decision Trees as Probabilistic Classifiers," Proc. Fourth Int. Workshop on Machine Learning, U. of Cal, Irvine, pp. 31–37, June 22–25, 1987.
B. Mirshab, "A Computer-Based Pattern Learning System With Application to Printed Text Recognition," PhD Dissertation, Oakland University, Rochester, MI, 1989.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1991 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Haskell, R.E., Noui-Mehidi, A. (1991). Design of hierarchical classifiers. In: Sherwani, N.A., de Doncker, E., Kapenga, J.A. (eds) Computing in the 90's. Great Lakes CS 1989. Lecture Notes in Computer Science, vol 507. Springer, New York, NY. https://doi.org/10.1007/BFb0038482
Download citation
DOI: https://doi.org/10.1007/BFb0038482
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-97628-0
Online ISBN: 978-0-387-34815-5
eBook Packages: Springer Book Archive