Target tracking in airborne forward looking infrared imagery

https://doi.org/10.1016/S0262-8856(03)00059-3Get rights and content

Abstract

In this paper, we propose a robust approach for tracking targets in forward looking infrared (FLIR) imagery taken from an airborne moving platform. First, the targets are detected using fuzzy clustering, edge fusion and local texture energy. The position and the size of the detected targets are then used to initialize the tracking algorithm. For each detected target, intensity and local standard deviation distributions are computed, and tracking is performed by computing the mean-shift vector that minimizes the distance between the kernel distribution for the target in the current frame and the model. In cases when the ego-motion of the sensor causes the target to move more than the operational limits of the tracking module, we perform a multi-resolution global motion compensation using the Gabor responses of the consecutive frames. The decision whether to compensate the sensor ego-motion is based on the distance measure computed from the likelihood of target and candidate distributions. To overcome the problems related to the changes in the target feature distributions, we automatically update the target model. Selection of the new target model is based on the same distance measure that is used for motion compensation. The experiments performed on the AMCOM FLIR data set show the robustness of the proposed method, which combines automatic model update and global motion compensation into one framework.

Introduction

Detection and tracking of moving or stationary targets in FLIR imagery are challenging research topics in computer vision. Though a great deal of effort has been expended on detecting and tracking objects in visual images, there has been only limited amount of work on thermal images in the computer vision community.

The thermal images are obtained by sensing the radiation in the infrared (IR) spectrum, which is either emitted or reflected by the object in the scene. Due to this property, IR images can provide information which is not available in visual images. However, in contrast to visual images, the images obtained from an IR sensor have extremely low signal to noise ratio (SNR), which results in limited information for performing detection or tracking tasks. In addition, in airborne forward looking infrared (FLIR) images, non-repeatability of the target signature, competing background clutter, lack of a priori information, high ego-motion of the sensor and the artifacts due to the weather conditions make detecting or tracking targets even harder.

To overcome the shortcomings of the nature of the FLIR imagery, different approaches impose different constraints to provide solutions for a limited number of situations. For instance, many target detection methods require that the targets are hot spots which appear as bright regions in the images [1], [2], [3]. Similarly, several target tracking algorithms require one or both of the following assumptions to be satisfied: (1) no sensor ego-motion [4] and (2) target features do not drastically change over the course of tracking [3], [5], [6]. However, in realistic tracking scenarios, neither of these assumptions are applicable, and a robust tracking method must successfully deal with these problems. To the best of our knowledge, there is no such published method that provides a solution to both of these problems in one framework.

In this paper, we present an approach for real-time target tracking in FLIR imagery in the presence of high global motion and changes in target features, i.e. shape and intensity. Moreover, the targets are not required to have constant velocity or acceleration. The proposed tracking algorithm uses the positions and the sizes of targets determined by the target detection scheme. For target detection, we apply steerable filters and compute texture energies of the targets, which are located using a segmentation-based approach. Once the targets are detected, the tracking method employs three modules to perform tracking. The first module, which is a modified version of Ref. [7], is based on finding the translation vector in the image space that minimizes the distance between the distributions of the model and the candidate. The distributions are obtained from the intensity and local standard deviation measure of the frames. The local standard deviation measure is obtained in the neighborhood of each pixel in the frame and provides a very good representation of frequency content of the local image structure. Based on the distance measure computed from the target feature distributions, the other two modules compensate for the sensor ego-motion and update the target model. The global motion estimation module uses the multi-resolution scheme of Ref. [8] assuming a planar scene under perspective projection. It uses Gabor filter responses of two consecutive frames to obtain the pseudo-perspective motion parameters.

The remainder of the paper is organized as follows: Section 2 discusses the recent literature on detecting and tracking targets in FLIR imagery. In Section 3, the target detector which is used to initialize the tracking algorithm with the position and the size of the target is described. Section 4 presents a discussion on the tracking problems and gives the details of the tracking algorithm which uses the two modules (1) automatic target model update (Section 4.3), (2) the sensor ego-motion compensation (Section 4.4). The implementation details are outlined in Section 4.5. Finally, experimental results for the proposed tracking method are presented in Section 5 and conclusions are drawn in Section 6.

Section snippets

Related work

In this section, we examine some of the representative works reported in the literature on detecting and tracking targets in FLIR imagery. In general, existing methods on IR images work for a limited number of situations due to the constraints imposed on the solution.

For detection of FLIR targets, many methods rely on the ithot spot technique, which assumes that the target IR radiation is much stronger than the radiation of the background and the noise. The goal of the target detectors is then

Target detection

Detection of targets in the FLIR sequences is a hard problem because of the variability of the appearance of targets due to atmospheric conditions, background, and thermodynamic state of the targets. Most of the time, the background forms similar shapes to those of the actual targets, and the targets become obscured.

Since the theme of this research is target tracking, we perform an initial target detection similar to Ref. [11]. In our implementation, we focused only on hot targets, which appear

Target tracking

We can categorize target tracking approaches into two classes. The first class is the correspondence based approaches, where the moving objects are detected in each frame and then the correspondences between the detected targets in the current and the previous frames are established. In contrast, the second class of approaches require target detection only in the first frame, and a target model, e.g. target template or intensity distribution, is extracted and used in performing tracking in the

Experiments

We have applied the proposed tracking method to the AMCOM FLIR data set. The data set was made available to us in grayscale format and has 41 sequences where each frame in each sequence is 128×128.

The proposed approach was developed using C++ on a Pentium III platform and the current implementation of the algorithm is capable of tracking at 10 fps. On all sequences, the detection algorithm is executed until a target is detected. For dark targets, we manually marked the target and performed

Conclusions

We propose a robust approach for tracking targets in airborne FLIR imagery. The tracking method requires the position and the size of the target in the first frame. The target detection scheme, which is used to initialize the tracking algorithm, finds target candidates using fuzzy clustering, edge fusion and texture measures. We employ a texture analysis on these candidates to select the correct targets. The experimental results for 200 target and 200 non-target regions that were manually

Acknowledgements

The authors wish to thank to Richard Sims of US Army AMCOM for providing us FLIR sequences. This research was partially funded by a grant from Lockheed Martin Corporation.

References (25)

  • J.Y. Chen et al.

    A detection algorithm for optical targets in clutter

    IEEE Transactions on Aerospace and Electronic Systems

    (1987)
  • M.S. Longmire et al.

    Lms and matched digital filters for optical clutter suppression

    Applied Optics

    (1988)
  • H. Shekarforoush et al.

    In: IEEE International Conference on Image Processing

    (2000)
  • D. Davies et al.

    In: Ninth British Machine Vision Conference, September

    (1998)
  • A. Strehl et al.

    Detecting moving objects in airborne forward looking infra-red sequences

    Machine Vision Applications Journal

    (2000)
  • U. Braga-Neto et al.

    In: 33rd Conference of Information Sciences and Systems, March

    (1999)
  • D. Comaniciu et al.

    Real-time tracking of non-rigid objects using mean shift

    (2000)
  • J.R. Bergen et al.

    Hierarchical model-based motion estimation

    In: Europen Conference on Computer Vision

    (1992)
  • E.T. Lim et al.

    Adaptive spatial filtering techniques for the detection of targets in infrared imaging seekers

    (2000)
  • A.P. Tzannes et al.

    In: Proceedings of ICASSP

    (1999)
  • Y.W. Lim et al.

    On the color image segmentation algorithm based on the thresholding and the fuzzy c-means techniques

    Pattern Recognition Journal

    (1990)
  • E. Saber et al.

    Fusion of color and edge information for improved segmentation and edge linking

    Image and Vision Computing Journal

    (1997)
  • Cited by (269)

    • Multispectral person re-identification using gan for color-to-thermal image translation

      2019, Multimodal Scene Understanding: Algorithms, Applications and Deep Learning
    • Infrared image interest point detector for few samples Based on Superpoint

      2023, ICAIT 2023 - 2023 IEEE 15th International Conference on Advanced Infocomm Technology
    • Overview on infrared dim target tracking

      2022, Proceedings of SPIE - The International Society for Optical Engineering
    View all citing articles on Scopus
    View full text