A novel training weighted ensemble (TWE) with application to face recognition

Abstract

Individual classifiers that are fully trained are unstable especially when the database conditions are changed. Moreover, designing a unique classifier with the suitable parameters to achieve acceptable performance is a non-trivial task. Combined classifiers, which consist of a set of individually trained classifiers, are introduced to avoid the previous problems. There are two key issues in the combination of classifiers. The first issue is how to obtain the set of base classifiers to combine. The second issue is how to fuse the decisions of those classifiers. In this paper, weak Learning Vector Quantization (LVQ) neural networks have been used as base classifiers. Also, a new combination technique which is based on training-weighted voting is introduced. Other factors that greatly affect the performance of a combined classifier are related to the type of the individual classifiers, the training parameters, database size and nature, etc. These factors have been considered in the design of the proposed combined classifier. TWE has been experimentally tested on five standard face databases: Yale, ORL, Grimace, Faces94 and Faces95 and has demonstrated excellent performance. Analysis of the ensemble stability has shown promising results.

Introduction

Many researchers have solved pattern recognition problems by introducing solutions that are based on a single classifier. In their studies, that unique classifier has been trained very well, and then used to recognize the unseen (test) instances. For each test instance, the decision that is taken by this classifier is considered to be the final decision of the introduced solution. The design of a unique classifier that results in excellent percentage of correct classification is very complex especially when each instance has a huge number of attributes, which is the case in face recognition problems. In [16], the problems that face the design of that unique classifier are discussed in detail.

On the other hand, a lot of studies gave other solutions to pattern recognition problems which are based on a combination of multiple classifiers. In their researches, the final decision of the whole solution is taken after combining the decisions of all the individual classifiers. These studies agreed that the decision taken by a combination of multiple classifiers is better than the decision of only one classifier regardless of the strength of this unique classifier. In [16], the advantages of combining multiple classifiers or what is called a multiple classifier system are given. Face recognition, as one of the pattern recognition problems, is considered to be one of the most important fields especially after the 11th of September 2001 events. The need to automatically recognize the people from their faces becomes much imperative than before. In this paper, we introduce a new approach based on combining multiple classifiers instead of depending on one classifier to achieve better classification results. In Section 2, a literature review is given to explain the used techniques in combining the individual classifiers. Section 3 describes the Learning Vector Quantization (LVQ) neural network as a base classifier and explains the proposed approach. In Section 4, the different image databases used in this research like Yale, ORL and Essex are described. In Section 5, in addition to discussing the implementation of the proposed approach, comparisons against other different approaches are presented. Finally, a conclusion about the proposed approach and its ability to reach the objective of this paper is discussed.