An Innovative Approach for Detection of Armoured Vehicle in Airborne Thermal Imagery Using Morphological Processing and Texture Feature Extraction

1 Introduction

In defence applications, detection of a vehicle, whether truck or tank, in thermal imagery is important to track its movement. However, detection of targets based on a hot spot in airborne thermal imagery has its own challenges because the appearance of target changes with weather condition and running time of the vehicle. Additionally, fire and warm objects could produce false alarm, for example, a running diesel generator in a field could be identified as a vehicle. Very few literature [24, 28, 30] have explored target detection and classification using single-band thermal imagery. Most of the papers offered pattern recognition and real-time tracking using forward-looking infrared imagery (FLIR) images [13, 29] and multiband thermal IR imagery [10, 23].

The human visual system (HVS) has been used to improve various detection capabilities like detection rate, false alarm rate, and speed. Current algorithms based on HVS usually improve one or two of the aforementioned detection capabilities while sacrificing the others. The use of a robust IR small-target detection algorithm based on HVS is proposed to pursue good performance [15]. It shows that the proposed algorithm has good robustness and efficiency for real IR small-target detection.

Bharadwaj and Carin [5] showed that FLIR image changes with target orientation, temperature of vehicles (depends on how long engine has been on or off), weather condition, and background clutter. They proposed IR image classification techniques using hidden Markov model in which relationship between different target subcomponents are modelled in a tree-like fashion. In lower-resolution IR imagery, a small target has very small imaging area. At the same time, in the image of complex background and low signal-to-noise ratio images, the target also fluctuates likely by the amount of noise. These adverse factors make the detection performance difficult for small moving targets in thermal imagery. To tackle these types of situations, multiresolution techniques like wavelet-based multiresolution analysis [16] has been used for target detection. For real-time target detection and tracking in FLIR image sequences, two methodologies, intensity variation function (IVF) and template modelling (TM), are proposed [1]. IVF and TM are both formulated based on target intensity feature and shape information, respectively, with the second triggered when the first fails due to clutter and noise.

In the recent years, image gradient has become one of the favourite techniques for target investigation in IR imagery. The histogram of oriented gradient features and the bag-of-words model have been used for automatic target recognition in IR Imagery [20]. It is easy to implement and is less time consuming. However, hot spot detection of similar objects requires rotationally symmetric and linear filters that works on both spatial and frequency domains.

Gabor filters have found many applications in image processing and pattern recognition problems. The two-dimensional (2D) Gabor filters have been applied for edge and line detection [8, 22], features extraction [14, 31] classification [4, 7, 17], and object recognition [19, 25], and so forth. Gabor filtering provides simultaneous inspection of images in both spatial and frequency domains. The Gabor filter representation increases the dimensions of the feature space such that salient features can effectively be discriminated in the extended feature space [18].

Texture feature-based object detection and discrimination has been performed in various literature [26]. In this paper, the targets are detected using fuzzy clustering, edge fusion, and local texture energy. The position and size of the detected targets are then used to initialise the tracking algorithm. In our implementation, texture features also play an important role in tank detection using single-band thermal imagery. The tracking of targets is out of the scope of this paper.

The present study uses single-band thermal imagery for detection of targets as hot spot. The major challenges are differentiating the object with background noise and from any other hot spot or object. However, this paper is dedicated only for tank detection, but analysis with other hot objects existing in the field also explored to validate our algorithm. In our implementation, we started with segmentation to discriminate background and foreground objects. A pre-processing algorithm based on top-hat filtering used to reduce uneven illumination and image enhancement. A window size double the size of a tank is used to select all the possible targets that are identified after segmentation. Gabor filter banks have been designed and used to extract the energy features of all targets to discriminate similar and dissimilar objects in thermal imagery.

2 Image Description and Field Preparation

In this paper, a Jenoptik single-band thermal imager was used for producing a single 640×480-pixel image. Airborne images were captured in wavelength range of 7.5–14 μm, with 60 cm spatial resolution from a height of 3000 ft (914.4 m).

The area near Bhatinda in the state of Punjab, India, was used as a test site for both airborne survey and field studies. Objects like tanks and trucks were spread out in various locations in running and non-running condition. Aside from this, steel pipes filled with both hot and cold water, separately, firelight, and diesel electric generator were also used for the experiment. Airborne survey and field measurement were performed during day time from 11:00 to 14:00 h and during night time from 20:00 to 22:00 h.

3 Mean Shift Segmentation

In this paper, the main goal of the segmentation is not only to create visually distinct and homogeneous regions, but the discrimination between objects and background in thermal images is also our major concern. According to literature, in this situation, the parametric methods are not adequate. As the mean shift algorithm [9, 11, 12, 21] is a non-parametric clustering technique that does not require prior knowledge of the number of clusters or constrain the shape of the clusters. It is well-suited for our application where the hot targets (hot spot) and background should be clearly delineated. It is obvious that a running tank has the highest intensity (like other hot spots) over the image in comparison to the background. The intensity values of a tank in a thermal image depend on the actual temperature of tank in the field, material of the tank, and other environmental conditions. Image segmentation has been performed using the EDISON software, which integrates the literature [12, 21]. Segmentation results and boundary overlay are shown in Figure 1. We are only targeting objects that have higher contrast than the background. To identify possible targets, we are ignoring dark cluster regions and uniform large clusters that are bigger than the tank. Boundary overlay is used to mark possible targets after segmentation. Closed loop boundaries that have similar size as tank are only marked as possible targets.

Figure 1:

Image Segmentation Results. (A) Original Images. (B) Segmentation. (C) Boundary Overlays.

4 Image Enhancement Pre-Processing Algorithm

Most of the time, thermal images are severely affected by noise. Because of noise, the hot spot targets become blurry and identification and tracking become complex. Temperature distribution depends upon material and weather condition as well. Normally, hot spot targets have higher intensity values than surroundings, but clutter background can create confusion, which could be mistaken as a target. Thus, it is necessary to pre-process the image to reduce the false background. The block diagram in Figure 2 describes the pre-processing steps for thermal imagery.

Figure 2:

Block Diagram of Proposed Pre-Processing Method.

Pre-processing steps will transform data into a low-intensity space, but the actual pattern of hot spot targets do not vary.

For thermal image processing and dim target identification, morphological operations have been used successfully in literature [2, 3, 6, 27]. Top-hat transform is a preferred operation for target enhancement in high clutter thermal images. Basically, it performs high-pass filtering to improve image contrast. There are two types of operations, open top-hat filtering, which preserves sharp peaks, and close top-hat filtering, which preserves sharp bottom. Particularly, opening smoothes bright regions of image and closing smoothes dark regions. All of the mathematical morphological operations work with two sets. One set is the original image to be analysed and the other set is called structuring element. For gray-scale image, the dilation, erosion, opening, closing, open top-hat transformation, and close top-hat transformation of input image f(x, y) by flat structuring element B(i, j) are defined as follows:

3D plots of the pre-processed image have been used for new possible target selection. Hot spot targets have high-intensity values than background. Thus, uniform high-intensity response areas equal to or less than window size that fall very rapidly are selected as possible targets (Figure 3, A2, B2). If a high-intensity response is uniform over wide regions, it is possibly a false alarm (Figure 3, C2, D2). For further processing of possible targets, window selection comprises the following steps:

1. The highest pixel value is selected as the centre of the window.

2. Two windows could be overlapped in many situations.

3. The size of window is double the size of object (in our case, 40×40 pixels).

Figure 3:

(A1, C1) Original Image with Possible Targets. (A2, C2) Pre-Processed Image. (B1, B2, D1, D2) 3D Plot of Both Unprocessed and Pre-Processed Images.

5 Feature Extraction Using Gabor Filter Banks

A number of authors have used Gabor filter banks to extract local image features. Gabor filters are basically band-pass filters that can be configured to extract a specific band of frequency components from an image. A 2D Gabor filter consists of a sinusoidal plane wave of some frequency and orientation modulated by 2D Gaussian envelopes. A cosine-modulated Gabor filter in the spatial domain is given by the following:

The standard deviation (σ) of the Gaussian factor determines the effective size of the surrounding of a pixel where a weighted summation takes place. The eccentricity of the Gaussian, and herewith the eccentricity of the convolution kernel g, is determined by the parameter called the spatial aspect ratio. The parameter λ is the wavelength and 1/λ is the spatial frequency of the harmonic factor S. As the spatial frequency tuning curve of a filter with an impulse response g has a maximum at 1/λ (as Figure 4), ee refer to 1/λ as the preferred spatial frequency of the Gabor filter. The ratio σ/λ determines the spatial frequency bandwidth of the Gabor filters. The half-response spatial frequency bandwidth b (in octaves) and the ratio σ/λ are related as follows:

Figure 4:

3D Plot of Gabor Filters with Two Different Value of Theta.

Finally, the parameter ϕ, which is a phase offset in the argument of the harmonic factor S, determines the symmetry of the function g(x,y) for ϕ=0 and ϕ=π; it is symmetric, or even, with respect to the centre point (0,0); for ϕ=–π/2 and ϕ=–π/2, g(x,y) is anti-symmetric, or odd. The value b=2, γ=0.5, and λ=3, 5, 7, and 9 are used in our experiment. The response of the Gabor filter with six orientation values (θ=0, π/5, 2π/5, 3π/5, 4π/5, π) and four spatial frequency (1/λ=1/3, 1/5, 1/7, 1/9), which are used for implementation of Gabor filters, is shown in Figure 5.

Figure 5:

2D Plot of Gabor Filter with Various Values of Theta and Spatial Frequency.

The texture of an image can be analysed through its spectral content of the sub-band signals, which are obtained by filtering the image with filter banks. An input image I(x,y) is convolved with a two-dimensional Gabor function g(x,y) to obtain a Gabor feature image r(x,y), as follows:

The energy feature is very useful for texture image discrimination, and an efficient way to calculate spectral content is by computing the energy

where N and M are the number of rows and columns of the window. To define the texture content of the target, an energy vector is constructed E=(e₁, e₂, e₃, e₄, …, e_k)^T for the target window. Unless a camouflaged target is present, the texture of the target different from the neighbourhood of the target region is selected. To utilize this, eight overlapping windows in the neighbourhood of the target are selected and their energy vectors, E₁, E₂, E₃, …, E₈, are computed. Selection of overlapping windows is done by eight directions, as shown in Figure 6.

Figure 6:

Selection of Fixed Size Target and Background Window for Texture Analysis.

The similarity between target and neighbouring window can be computed by Chebychev distance measuring method. A minimum value of distance has been used as a threshold for tank detection. The Chebychev distance measure between tank and other objects that appears similar to tanks in thermal imagery such as diesel generators, firelights, and trucks has been calculated for better discrimination.

6 Distance Analysis for Target Discrimination

As given in Table 1, T1–T6 are tanks, and their engines are running on the ground for the last 12 h. T7 is a diesel generator, which is also running on the field for the last 12 h. Trucks have different visuals than tanks in thermal imagery, which could be easily recognised and discriminated from the tank-like signatures. Figure 7 shows the thermal signature of 3-ton army trucks and T-72 tanks. The experiment has been carried out to compare the texture between these objects and tanks. Table 1 shows the Chebychev distance measure between six tanks and one diesel power generator. Similarly, the distance between the tanks and two trucks, three fire lights, and hot water tank have been calculated. The average distance values are given in Table 2.

Figure 7:

Thermal images of (A, B) tanks (C, D) Truck.

Chebychev distance measure gives appreciable results. This method is also able to differentiate tanks from a group of other similar targets. The chosen window size for texture analysis is same for all the objects.

This experiment shows that one texture feature is not enough to discriminate all targets. Objects could not be linearly separable because distance is not consistent in a particular range or group. In this case, extraction of multiple features could be the possible option for improving detection. True-positive rate (TPR) and false-positive rate (FPR) are calculated for testing algorithm performance, which are defined as follows:

As a field preparation, 10 tanks, 2 trucks, 3 firelight, 1 diesel generator, 3 containers, and 3 hollow pipes filled with hot water were installed in the field. A confusion matrix shown in Table 3 has been formed to calculate TPR and FPR values. A TPR of 90% and an FTR of 0.8%, i.e. a very low FTR compared with TPR, show good performance of our algorithm. In this study, we have a limited number of tanks, but if we increase the number of tanks in the field, the TPR could give a more realistic value. An image where both tanks and other hot spots like fire, which have high contrast than background, are present could misidentify a tank. In this situation, the aforementioned study suggests the calculation of distance to discriminate false alarm. Figure 8 shows the images with correct targets that are identified as a tank.

Figure 8:

Output Images with Detected Tanks.

7 Conclusion and Results

In this paper, we have designed an algorithm for the tank detection using single-band thermal imagery. First, segmentation of a 300×300-pixel image using mean shift has been performed to discriminate the background and possible targets available in the ground. After pre-processing using proposed top-hat filtering scheme, few false targets have been removed out of the possible target set. After segmentation, a new possible target set was identified. New possible targets were selected using a fixed size window, which was double the size of the tank in the pixel domain. A window size of 40×40 pixels was used for tank detection. Gabor filter banks were used for the texture feature extraction of the target image. As the Gabor-energy operator responded to contours and edges, the energy features were calculated and compared with the same features extracted from various hot spot (Tables 1 and 2).

Matlab R2009A was used for image pre-processing, designing of Gabor filter banks, and feature extraction, and EDISON software used for image segmentation. All experiments were implemented on a PC with 4-GB memory and 3.4-GHz Intel^® i7 Core™ duo processor. Approximately 50 single-band thermal images (300×300) of same size with and without tanks have been processed with this algorithm. Result shows that this algorithm has good performance. The calculated accuracy of tank detection is >90% with low false alarm rate. The total time consumption, including segmentation, pre-processing, and one possible target processing, is <0.1 ms. The proposed algorithm can achieve not only a fast detection speed but also the best performance in terms of TPR and FPR. The algorithm provides higher TPR with lower FPR (Table 3). Our algorithm also gives good results particularly when the background is complex and bright, like the last two scenes in Figure 1. As a future scope, multiple texture features could be extracted, and based on these featured, a set of classifier could be designed to classify hot spot objects to improve overall accuracy.