Elsevier

Pattern Recognition Letters

Volume 32, Issue 9, 1 July 2011, Pages 1317-1327
Pattern Recognition Letters

A robust template tracking algorithm with weighted active drift correction

https://doi.org/10.1016/j.patrec.2011.03.010Get rights and content

Abstract

In this paper, we propose a novel algorithm for object template tracking and its drift correction. It can prevent the tracking drift effectively, and save the time of an additional correction tracking. In our algorithm, the total energy function consists of two terms: the tracking term and the drift correction term. We minimize the total energy function synchronously for template tracking and weighted active drift correction. The minimization of the active drift correction term is achieved by the inverse compositional algorithm with a weighted L2 norm, which is incorporated into traditional affine image alignment (AIA) algorithm. Its weights can be adaptively updated for each template. For diminishing the accumulative error in tracking, we design a new template update strategy that chooses a new template with the lowest matching error. Finally, we will present various experimental results that validate our algorithm. These results also show that our algorithm achieves better performance than the inverse compositional algorithm for drift correction.

Highlights

► We propose a novel algorithm for object template tracking and its drift correction. ► The tracking term and the drift correction term constitute the total energy function. ► The template tracking and weighted active drift correction are achieved synchronously by minimizing the total energy. ► We design a new template updating strategy to diminish the accumulative error during the tracking. ► Verified on the PETS2001 datasets, the proposed algorithm can prevent the tracking drift effectively, and save the time of an additional correction tracking, and achieve better performance than the inverse compositional algorithm for drift correction.

Introduction

The goal of automatic object tracking is to find the targets in image sequences. The tracking process is usually affected by many factors, such as the deformation of the object, the change of environment illumination in images, partial or full object occlusions. Many tracking algorithms have been proposed and implemented in applications to overcome these disturbing problems. These algorithms can be divided into three main categories: the feature-based tracking algorithms (Mitra et al., 2002; Tissainayagam and Suter, 2005; Nickels and Hutchinson, 2002), the contour-based tracking algorithms (Paragios and Deriche, 2000; Freedman and Zhang, 2004; Linlin et al., 2009), and the region-based tracking algorithms (Bascle and Deriche, 1995; Jepson et al., 2003). In the last category, the region content is either used directly with template tracking or represented by a nonparametric description, such as a histogram. The well-known region-based tracking methods are the mean-shift algorithm (Comaniciu et al., 2003) and particle filtering tracking algorithm (Perez et al., 2002; Zhou et al., 2004). Without the influence of occlusion, the region-based methods have high robustness and high accuracy, because they can carry both spatial and appearance information.

Template tracking is widely studied in computer vision, which can date back to Lucas and Kanade (1981). It is to track an object through a video sequence by extracting the template in the first frame and finding the region which matches the chosen template as closely as possible in the following frames. Based on the affine image alignment (AIA) algorithm, template tracking has been extended in a variety of ways, which include: (1) allowing for arbitrary parametric transformations of the template (Bergen et al., 1992); (2) allowing for linear appearance variation (Black and Jepson, 1998; Hager and Belhumeur, 1998); and (3) dealing with special cases, such as occlusion and containing background pixels (Ishikawa et al., 2002). Based on the combination of these extensions, some non-rigid appearance models for template tracking are proposed, such as active appearance models (AAM) (Cootes et al., 2001; Gross et al., 2006) and active blobs (Sclaroff and Isidoro, 1998). The underlying assumption behind template tracking is that the appearance of the object remains the same throughout the entire video sequence, as shown in Fig. 1. This assumption is generally reasonable for a certain period of time, but eventually the template will be no longer an accurate model of the object as time goes on. Therefore, the assumption is often violated (Matthews et al., 2004). One solution to this problem is to update the template every frame or after an interval of several frames. However, the simple updating strategy often brings the problem of template drift, which consists two parts: spatial drift and feature drift (Lankton et al., 2008). The spatial drift is the change in the model such that the model and object are misaligned. The feature drift is the change of object appearance as it diverges from the appearance of the model over time.

A drift-correcting algorithm is proposed in (Matthews et al., 2004), in which the template is updated in every frame, and still stays firmly attached to the original object. The algorithm is a simple extension of the naive algorithm. As well as maintaining a current estimate of the template, it also retains the first template from the first frame. The template is first updated as in the naive algorithm with the image at the current template location (Matthews et al., 2004). However, to eliminate drift, this updated template is then aligned with the first template to complete the final update. This is a passive drift correction algorithm. Obviously, the second tracking needs additional calculating time, which greatly decreases the efficiency of the algorithm. Besides, it is still sensitive to the variation in the object appearance relative to the first template.

Following the Matthews’s algorithm (2004), Schreiber (2007) presents a robust version of the drift correction, which adds weights to the template pixels and inserts affine parameters into updating step when the template is updated. Although this algorithm improves the robustness of the drift-correcting algorithm to some extent, it still has two tracking steps, and due to calculating the updating weights, its efficiency is even much lower than the passive drift correction algorithm.

To handle the problems of partial occlusions, appearance variation and presence of background pixels, some robust template matching algorithms are proposed (Hager and Belhumeur, 1998; Ishikawa et al., 2002; Baker et al., 2003). These robust tracking algorithms can be regarded as a weighted least square process, such that occluded regions, background pixels and regions where brightness has changed would be considered as outliers and would be suppressed (Schreiber, 2007). However, these algorithms require a trade-off between efficiency and accuracy. In (Baker and Mathews, 2003), the Hessian matrix contains a weighting function which is updated every iteration and thus can not be pre-computed. In (Ishikawa et al., 2002), the template is divided into a few blocks, and the Hessian matrix of each block is given dependently as a constant value, so it need not be re-computed in each iteration. In (Hager and Belhumeur, 1998), they propose that only the pixels of the object are counted during AIA to prevent the outliers, such as noise and background. In the algorithm, the Hessian matrix is related to the division of the external pixels, so that it must be re-computed each iteration. It reduces the efficiency of the algorithm.

In this paper, we propose a novel robust template tracking algorithm. In this algorithm, the tracking term and the drift correction term constitute the total energy. The template tracking and weighted active drift correction are achieved synchronously by minimizing the total energy. An active drift correction term is integrated into the affine image alignment algorithm. Our algorithm can prevent the tracking drift effectively and save the time of an additional correction tracking. Meanwhile, the robust weights are adaptively updated for each template. In addition, we adopt a new template updating strategy to diminish the accumulative error in the template updating.

The rest of this paper is organized as follows: In Section 2, we review the inverse compositional algorithm and its robust version, and the passive drift correction algorithm. In Section 3, we describe our weighted active drift correction algorithm and our new template updating strategy to diminish the accumulative error. In Section 4, we present our experimental results and compare them with those of the passive drift correction algorithm. The parameters used in this paper are discussed in Section 5. Section 6 contains the conclusion and the outline of our future work.

Section snippets

Related work: inverse compositional algorithms with drift correction

The goal of the AIA is to minimize the sum of squared error between two images, the template T and the image I warped back onto the coordinate frame of the template (Baker and Mathews, 2004):XT[I(W(X;P))-T(X)]2where X = (x, y)T denotes the image coordinates, P = (pp6) is a vector of affine transformation parameter, and W(X; P) denotes the parameterized set of allowed warps. The warp W(X; P) maps the pixel X in the template to the sub-pixel location W(X; P) in the image I.

The minimization of formula

Robust template tracking with weighted active drift correction

In this Section, we introduce our robust template tracking algorithm with weighted active drift correction. In this algorithm, we regard the tracking information and the drift-correction information as different energy components of the total target energy.

Let α denote the coefficient of the drift correction. α ϵ (0, 1). Tn is the updated template. T0 is the first template. EupdateT is the energy component of the updated template Tn in formula (11), which represents the difference between the

Experimental results

We compare the experimental results of our proposed tracking algorithm with those of the PDC algorithm, and confirm its robustness and efficiency under many kinds of object changes in appearance.

The programs of two algorithms are implemented in Matlab 7.0. Experimental results are obtained on a computer with a 2.0 GHz Pentium-IV, 512 M RAM. The test video sequences are from dataset4 and dataset5 in PETS2001 (http://www.cvg.rdg.ac.uk/slides/pets.html). More information about the sequences is

Parameters in our algorithm

Our algorithm has two parameters which influence its performance: the weight coefficient α and the length N used for the update of the template.

The parameter α controls the drift correction behavior. In order to test its sensitivity in our algorithm, we set the parameter α to different values. The test results of two used sequences (Sequences 3 and 4) are shown in Fig. 10. The different color lines (blue, black, red-purple, Blue-green, green, red, yellow) stand for different values of the

Conclusion

This paper proposes a novel and robust template tracking algorithm with weighted active drift correction. The experimental results in different environments show that our algorithm can correct the drift effectively and does not need an additional correction tracking. It achieves better performance compared with the passive drift correction algorithm. For this research topic and our algorithm, there is still plenty of room for the further research. Our future work will focus on automatic

Acknowledgement

This work is funded by the Natural Science Foundation of China under Grant No. 60805046 and 60835004.

References (26)

  • R. Gross et al.

    Active appearance models with occlusion

    Image Vis. Comput.

    (2006)
  • K. Nickels et al.

    Estimating uncertainty in SSD-based feature tracking

    Image Vis. Comput.

    (2002)
  • D. Schreiber

    Robust template tracking with drift correction

    Pattern Recognition Lett.

    (2007)
  • P. Tissainayagam et al.

    Object tracking in image sequences using point feature

    Pattern Recognition

    (2005)
  • Baker, S., Gross, R., Matthews, I., 2002. Lucas–Kanade 20 Years on: A Unifying Framework: Part 1. Technical Report...
  • Baker, S., Gross, R., Matthews, I., 2003. Lucas–Kanade 20 Years on: A Unifying Framework: Part 2. Technical Report...
  • S. Baker et al.

    Lucas–Kanade 20 years on: A unifying framework

    Internat. J. Comput. Vision

    (2004)
  • Bascle, B., Deriche, R., 1995. Region tracking through image sequences. In: Proc. IEEE Internat. Conf. on Computer...
  • Bergen, J.R., Anandan, P., Hanna, K.J., Hingorani, R., 1992. Hierarchical model-based motion estimation. In: Proc....
  • M. Black et al.

    Eigen-tracking: robust matching and tracking of articulated objects using a view-based representation

    IJCV

    (1998)
  • D. Comaniciu et al.

    Kernel based object tracking

    IEEE Trans. Pattern Anal. Machine Intell.

    (2003)
  • T. Cootes et al.

    Active appearance models

    IEEE Trans. Pattern Anal. Machine Intell.

    (2001)
  • D. Freedman et al.

    Active contours for tracking distributions

    IEEE Trans. Image Process.

    (2004)
  • Cited by (12)

    • Motion tracking in infrared imaging for quantitative medical diagnostic applications

      2014, Infrared Physics and Technology
      Citation Excerpt :

      Based on the experiences gained in our previous clinical studies, in this study, a template-based tracking scheme is applied for the dynamic IR imaging. In the field of computer vision, automatic methods for tracking the moving object in an image sequence fall into three main categories [17,18]: feature-based tracking [19,20], contour-based tracking [21,22], and region-based tracking [23,24]. In the absence of occlusions, region-based tracking methods perform well in terms of robustness and accuracy.

    • A new optimal seam finding method based on tensor analysis for automatic panorama construction

      2013, Pattern Recognition Letters
      Citation Excerpt :

      Finally, in image stitching, the aligned images are merged to form a composite image, and the intensity values of the images are adjusted if necessary. Since the first and second steps have been extensively studied in the past few decades (Luong et al., 2011; Fan et al., 2011; Zitová and Flusser, 2003), our work is mainly focusing on the final step. There are two main categories of image stitching approaches: transition smoothing and optimal seam finding.

    • Fast template matching based on grey prediction for real-time object tracking

      2017, Proceedings of SPIE - The International Society for Optical Engineering
    • Motion-Based Feature Selection and Adaptive Template Update Strategy for Robust Visual Tracking

      2016, Proceedings - 2016 3rd International Conference on Information Science and Control Engineering, ICISCE 2016
    View all citing articles on Scopus
    View full text