Classifier Ensemble Using a Heuristic Learning with Sparsity and Diversity

Abstract

Classifier ensemble has been intensively studied with the aim of overcoming the limitations of individual classifier components in two prevalent directions, i.e., to diversely generate classifier components, and to sparsely combine multiple classifiers. Currently, most approaches are emphasized only on sparsity or on diversity. In this paper, we investigated classifier ensemble with learning both sparsity and diversity using a heuristic method. We formulated the sparsity and diversity learning problem in a general mathematical framework which is beneficial for learning sparsity and diversity while grouping classifiers. Moreover, we proposed a practical approach based on the genetic algorithm for the optimization process. In order to conveniently evaluate the diversity of component classifiers, we introduced the diversity contribution ability to select proper classifier components and evolve classifier weights. Experimental results on several UCI classification data sets confirm that our approach has a promising sparseness and the generalization performance.