Fusion of FLIR automatic target recognition algorithms

Abstract

In this paper, we investigate several fusion techniques for designing a composite classifier to improve the performance (probability of correct classification) of forward-looking infrared (FLIR) automatic target recognition (ATR). The motivation behind the fusion of ATR algorithms is that if each contributing technique in a fusion algorithm (composite classifier) emphasizes on learning at least some features of the targets that are not learned by other contributing techniques for making a classification decision, a fusion of ATR algorithms may improve overall probability of correct classification of the composite classifier. In this research, we propose to use four ATR algorithms for fusion. The individual performance of the four contributing algorithms ranges from 73.5% to about 77% of probability of correct classification on the testing set. The set of correctly classified targets by each contributing algorithm usually has a substantial overlap with the set of correctly identified targets by other algorithms (over 50% for the four algorithms being used in this research). There is also a significant part of the set of correctly identified targets that is not shared by all contributing algorithms. The size of this subset of correctly identified targets generally determines the extent of the potential improvement that may result from the fusion of the ATR algorithms. In this research, we propose to use Bayes classifier, committee of experts, stacked-generalization, winner-takes-all, and ranking-based fusion techniques for designing the composite classifiers. The experimental results show an improvement of more than 6.5% over the best individual performance.

Introduction

Automatic target recognition (ATR) systems generally consist of three stages as shown in Fig. 1 [1]: (1) a preprocessing stage (target detection stage) that operates on the entire image and extracts regions containing potential targets, (2) a clutter¹ rejection stage that uses a sophisticated classification technique to identify true targets by discarding the clutter images (false alarms) from the potential target images provided by the detection stage, and (3) a classification stage that determines the type of the target.

ATR using forward-looking infrared (FLIR) imagery is an integral part of the ongoing research at US Army Research Laboratory (ARL) for digitization of the battlefield. The real-life FLIR imagery (for example the database available at the ARL, see Fig. 2, Fig. 3) demonstrates a significantly high level of variability of target thermal signatures. The high variability of target thermal signatures is due to several reasons, including meteorological conditions, times of the day, locations, ranges, etc. This highly unpredictable nature of thermal signatures makes FLIR ATR a very challenging problem. In recent years a number of FLIR ATR algorithms have been developed by the scientists at ARL as well as by the researchers in academia and industry working under ARL-sponsored research. These research activities have used a common development set of FLIR data (17,318 target images). The performance of these independently developed algorithms is measured in terms of the probability of correct classification using a common testing FLIR data collected under relatively unfavorable conditions. The testing FLIR data is not used during the algorithm development. The performance of these algorithms seems to be topped off around 77% of probability of correct classification.

In this paper, we investigate several fusion techniques for designing a composite classifier to improve the performance (probability of correct classification) of FLIR ATR. The motivation behind the fusion of ATR algorithms is that if each contributing technique in a fusion algorithm (composite classifier) emphasizes on learning at least some features of the targets that are not learned by other contributing techniques for making a classification decision, a fusion of ATR algorithms may improve overall probability of correct classification of the composite classifier. In this research, we propose to use four ATR algorithms for fusion with individual performance of the four contributing algorithms ranging from 73.5% to about 77% of probability of correct classification on the testing set (ROI data set with 3456 target images).

The first algorithm uses a multi-layer convolution neural network (MLCNN) [2] for designing the ATR system [3], [4]. An MLCNN is a variation of time-delay neural network (TDNN), which has been applied for speech and phoneme recognition [5]. MLCNN has been used for handwritten word recognition [6]. An MLCNN is a multi-layer feed-forward structure having three different kinds of layers: a convolution layer, a subsampling layer, and a summing layer that makes a partition of the results of the convolution operations and sums the results in each block. A nonlinear squashing function is usually applied to the outputs of each kind of layer. For the first convolution layer, each convolution operation can be interpreted as extracting the same feature in different parts of the input. The final output of the MLCNN gives scores for the classification of the tested target chip (image).

The next classification algorithm is based on learning vector quantization (LVQ) algorithm. The LVQ-based classifier consists of four stages [7]: a set of aspect windows of different size, a stage in which the extracted area is enlarged to a fixed size, a stage for wavelet decomposition of the enlarged extraction, and a dedicated VQ for each subband within each aspect window. In the first stage, each aspect window is a background-clipping rectangle whose size is determined by the type of target and the range of aspects that it operates on. After the background removal in the first stage, each extracted target area is enlarged in the second stage into a fixed dimension that is common to all the aspect windows. In the third stage, the enlarged extraction is decomposed into four subbands based on a wavelet decomposition process. After the wavelet decomposition, the final stage uses a set of VQ codebooks for feature matching and target recognition. In this stage, the target recognition problem is treated as a template-matching task through a similarity metric-based approach. A set of subband templates or code vectors is constructed for each subband of a particular target at a specific range of aspects. Each set of code vectors forms a codebook, representing the target signatures for a given subband of a particular target at a specific range of aspects.

The third classification algorithm used in the fusion is based on modular neural network (MNN) [8] approach in which the classification of targets is realized hierarchically [9]. The local structure of a target image is captured by extraction of the directional variance features at different resolutions in small image blocks. These image blocks are organized, by location, into several larger image regions, called receptive fields. An individual neural network (expert network) in the modular network is designed to classify targets using only the local features in a single receptive field. At the final level, the classification results of the expert networks for the individual receptive fields are further combined to produce the final classification decision.

The fourth technique uses expansion matching (EXM) filtering and Karhunen–Loeve transform (KLT) for feature extraction [10], [11]. The extracted features (feature vectors) are then used to train supporting vector machines (SVM), which finally identify the class of the input target image. In this research, we propose to use averaged Bayes classifier, committee of experts, stacked-generalization, winner-takes-all, and ranking-based fusion techniques for designing the composite classifiers. The experimental results show an improvement of more than 6.5% over the best individual performance.

This paper is organized as follows. Section 2 presents several fusion techniques proposed in this paper. Section 3 presents the experimental results and Section 4 concludes the paper discussing the future research directions.