Architecture for Classifier Combination Using Entropy Measures

Abstract

In this paper we emphasize the need for a general theory of combination. Presently, most systems combine recognizers in an ad hoc manner. Recognizers can be combined in series and/or in parallel. Empirical methods can become extremely time consuming, given the very large number of combination possibilities. We have developed a method of systematically arriving at the optimal architecture for combination of classifiers that can include both parallel and serial methods. Our focus in this paper, however, will be on serial methods. We also derive some theoretical results to lay the foundation for our experiments. We show how a greedy algorithm that strives for entropy reduction at every stage leads to results superior to combination methods which are ad hoc. In our experiments we have seen an advantage of about 5% in certain cases.