A decision theoretic approach to hierarchical classifier design

doi:10.1016/0031-3203(84)90087-6

Pattern Recognition

Volume 17, Issue 3, 1984, Pages 359-369

https://doi.org/10.1016/0031-3203(84)90087-6 Get rights and content

Abstract

The design of tree classifiers is considered from the statistical point of view. The procedure for calculating the a posteriori probabilities is decomposed into a sequence of steps. In every step the a posteriori probabilities for a certain subtask of the given pattern recognition task are calculated. The resulting tree classifier realizes a soft-decision strategy in contrast to the hard-decision strategy of the conventional decision tree. At the different nonterminal nodes, mean square polynomial classifiers are applied having the property of estimating the desired a posteriori probabilities together with an integrated feature selection capability.

References (19)

T.A. Bailey et al.
Cluster validity profiles
Pattern Recognition
(1982)
P. Argentiero et al.
An automated approach to the design of decision tree classifiers
IEEE Trans. Pattern Anal. Mach. Intell. PAMI-4
(1982)
A.V. Kulkarni et al.
An optimization approach to hierarchical classifier design
J.K. Mui et al.
Automated classification of nucleated blood cells using a binary tree classifier
IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2
(1980)
E.M. Rounds
A combined nonparametric approach to feature selection and binary decision tree design
H.J. Payne et al.
An algorithm for constructing optimal binary decision trees
IEEE Trans. Comput. C-26
(1977)
I.K. Sethi et al.
Hierarchical classifier design using mutual information
IEEE Trans. Pattern Anal. Mach. Intell. PAMI-4
(1982)
G.R. Dattatreya et al.
Decision tree design for pattern recognition including feature measurement cost
W.W. Bledsoe et al.
Discussion of problems in pattern recognition

There are more references available in the full text version of this article.

Cited by (28)

A low-cost and efficient electronic nose system for quantification of multiple indoor air contaminants utilizing HC and PLSR
2022, Sensors and Actuators B: Chemical
Citation Excerpt :
Thus, HC gives better classification results with fewer training samples than single complex classifiers [14], such as neural network and k-nearest neighbor (KNN) classifier. Therefore, hierarchical classifier, based on the divide-and-conquer strategy, seems to be a good alternative for the classification of single and mixed gases [16]. Then, the concentration of single gases or individual components in mixture gases could be estimated utilizing neural network or multiple regression models.
A novel quantification technique is presented for electronic nose (E-nose), which is based on a double-step strategy combined with hierarchical classifier (HC) and partial least squares regression (PLSR). With the tree structure of HC, the complexity of classifier training process can be reduced in the case of unbalanced samples. For each level of the class hierarchy, the extreme learning machine-based artificial neural network (ELM-ANN) is applied for classification. In order to improve the classification performance of ELM-ANN, the multiple time-domain features are selected as training inputs, and a novel optimization method of the number of hidden layer neurons is given. To validate the effectiveness of the presented quantification technique, an E-nose system is designed to quantify the gases including six toxic gases (hydrogen sulfide, carbon monoxide, ammonia, toluene, formaldehyde, acetone) and three kinds of binary gas mixtures. This presented hierarchical classifier has demonstrated outstanding performance for the identification of target gases, such as the macro-averaged precision for unlabeled data is improved from 80% to 92% compared with non-hierarchical classifiers. Furthermore, an excellent performance of concentration estimation is obtained utilizing PLSR, where average values of the coefficient of determination for training and test samples are equal to 0.957 and 0.927, respectively. Overall, our work demonstrates that the proposed approach is applicable in E-nose-based odor quantification.
Fuzzy classification trees for data analysis
2002, Fuzzy Sets and Systems
Overly generalized predictions are a serious problem in concept classification. In particular, the boundaries among classes are not always clearly defined. For example, there are usually uncertainties in diagnoses based on data from biochemical laboratory examinations. Such uncertainties make the prediction be more difficult than noise-free data. To avoid such problems, the idea of fuzzy classification is proposed. This paper presents the basic definition of fuzzy classification trees along with their construction algorithm. Fuzzy classification trees is a new model that integrates the fuzzy classifiers with decision trees, that can work well in classifying the data with noise. Instead of determining a single class for any given instance, fuzzy classification predicts the degree of possibility for every class.
Some empirical results the dataset from UCI Repository are given for comparing FCT and C4.5. Generally speaking, FCT can obtain better results than C4.5.
On mapping decision trees and neural networks
1999, Knowledge-Based Systems
There exist several methods for transforming decision trees to neural networks. These methods typically construct the networks by directly mapping decision nodes or rules to the neural units. As a result, the networks constructed are often larger than necessary. This article describes a pruning-based method for mapping decision trees to neural networks, which can compress the network by removing unimportant and redundant units and connections. In addition, equivalent decision trees extracted from the pruned networks are simpler than those induced by well-known algorithms such as ID3 and C4.5.
Multiclass discriminant mappings
1994, Signal Processing
Multiclass mappings oriented to the classification of multivariate data are discussed. Parametric and nonparametric mapping criteria are used for single level (one shot) projections. A method for reducing the computation complexity of the optimization procedure is suggested. In the case of a large number of classes, a sequential mapping based on the binary tree scheme is proposed. An algorithm for interactive binary tree design is presented. It combines automated and manual composition of the groups of the classes at the nodes of the tree. An example of the application of the tree mapping is presented.
Es werden Multiklassen-Mappings diskutiert, die sich an einer Klassifikation von Multivariate-Daten orientieren. Parametrische und nichtparametrische Mapping-Kriterien werden benutzt fur Single-Level (one shot)-Projektionen. Zur Reduktion des Rechenaufwandes für die Optimierungsprozedur wird eine Methode vorgeschlagen. Im Falle einer großen Anzahl von Klassen wird ein sequentielles Mapping vorgeschlagen, das auf einem binären Baum-Schema basiert. Es wird ein Algorithmus für den interaktiven Entwurf eines binären Baumes vorgestellt. Er kombiniert die automatische und manuelle Zusammensetzung von Gruppen von Klassen an den Knoten des Baumes. Ein Beispiel für die Anwendung des Baum-Mappings wird angeführt.
Nous discutons dans cet article les mises en correspondance multi-classes dans le contexte de la classification de données multi-variables. Nous utilisons des critères fonctionnels paramétriques et non paramètriques pour des projections à niveau unique (immédiates). Nous suggérons une méthode de réduction de la complexité de calcul de la procédure d'optimisation. Nous proposons dans le cas d'un nombre élevé de classes une mise en correspondance séquentielle basée sur une technique d'arbre binaire. Nous présentons un algorithme de conception interactive d'arbre binaire. Cet algorithme combine les compositions automatique et manuelle de groupes de classes au niveau des noeuds de l'arbre. Nous présentons une exemple d'application de cette mise en correspondance par arbre.
Parallel multi-layer classifier architectures of increasing hierarchical order
1993, Pattern Recognition Letters
Multi-level classifier architectures provide a means of improving error-rate performance in many classification tasks such as machine printed or handwritten character recognition. It is argued that appropriate multi-level structures are well suited to parallel implementation, and results are presented to characterise the performance of such structures in a practical character recognition environment for a range of configurations and, in particular, for hierarchies of increasing order.
Decision tree performance enhancement using an artificial neural network implementation
1991, Machine Intelligence and Pattern Recognition
Decision tree classifiers represent a popular classification methodology that has been successfully used in many pattern recognition tasks. Noting many similarities between the tree classifiers and the multilayer perceptron classifiers, an artificial neural network based implementation of decision tree classifiers to enhance their performance is described here. Several issues related to decision tree classifiers are discussed and it is shown how these can be dealt with following the proposed implementation.

View all citing articles on Scopus

View full text

A decision theoretic approach to hierarchical classifier design

Abstract

Pattern Recognition

An automated approach to the design of decision tree classifiers

IEEE Trans. Pattern Anal. Mach. Intell. PAMI-4

An optimization approach to hierarchical classifier design

Automated classification of nucleated blood cells using a binary tree classifier

IEEE Trans. Pattern Anal. Mach. Intell. PAMI-2

A combined nonparametric approach to feature selection and binary decision tree design

An algorithm for constructing optimal binary decision trees

IEEE Trans. Comput. C-26

Hierarchical classifier design using mutual information

IEEE Trans. Pattern Anal. Mach. Intell. PAMI-4

Decision tree design for pattern recognition including feature measurement cost

Discussion of problems in pattern recognition