Elsevier

Pattern Recognition

Volume 33, Issue 2, February 2000, Pages 237-249
Pattern Recognition

Adaptive window method with sizing vectors for reliable correlation-based target tracking

https://doi.org/10.1016/S0031-3203(99)00056-4Get rights and content

Abstract

We propose an adaptive window method that can provide a tracker with a tight reference window by adaptively adjusting its window size independently into all four side directions for enhancing the reliability of correlation-based image tracking in complex cluttered environments. When the size and shape of a moving object changes in an image, a correlator often accumulates walk-off error. A success of correlation-based tracking depends largely on choosing the suitable window size and position and thus transferring the proper reference image template to the next frame. We generate sizing vectors from the corners and sides, and then decompose the sizing vector from the corner into two corresponding sides. Since our tracker is capable of adjusting a reference image size more properly, stable tracking has been achieved minimizing the influence of complex background and clutters. We tested the performance of our method using 39 artificial image sequences made of 4260 images and 45 real image sequences made of more than 3400 images, and had the satisfactory results for most of them.

Introduction

Correlation-based tracking [1], [2], [3], [4], [5] acquires a reference template from the previous frame called a reference image and searches for the most similar area to estimate the target position in the current frame. Although the correlator is said to be robust against the cluttered noise, its real application has some limitations, too. Usually, it is desirable that searching area should be chosen to be small due to its large computation involved. Another problem is its tendency to accumulate walk-off error especially when the object of interest is changing in size, shape, or orientation from frame to frame.

If walk-off error is accumulated beyond a certain critical point, correlation-based tracking often fails. It is quite important that the size and position of a window should be determined precisely to guarantee that a proper reference image can be transferred to the next frame. In order to increase correlation reliability, the transferred reference image is usually desired to have a high occupancy rate of an object, which means that the window encloses the object properly. For this, it is quite desirable that the window should adjust its size as circumstances of the object have been under change.

A concept of an adaptive window could be found also in stereo matching. In the case of stereo matching, the disparity boundaries become sharp for the smaller window but the computed disparity becomes usually noisy. The larger window means that the computed disparity becomes clean but the disparity boundaries can be blurred. Kanade and Okutomi [6] determined the adaptive window size using local intensity and disparity patterns to minimize uncertainty in the disparity computed at each point. Lotti and Giraudon [7] made four non-centered adaptive windows associated to each image point in the thresholded edge image.

In correlation-based tracking, the adaptive window studies based on estimating an object size are not much reported in the technical literature. To automatically adapt the reference area, Hughes and Moy [1] designed the edge walking algorithm for the boundary detection to operate on a segmented binary image. This algorithm scans the binary image in a raster fashion looking for illuminated pixels. Once an illuminated pixel has been found, the edge walking algorithm searches for another illuminated pixels connected to the initial pixel. They used a boundary detection method for estimating the size of an object and thus determining the size of window that would enclose the object. Similarly, Montera et al. [2] determined an object region through expanding from the inner point of an object to the outer in the image. To yield the boundary of an object, they searched for the areas where pixel values vary from above the threshold to below the threshold. However, both methods could be difficult to be applied to an object having internal edges in non-homogeneous cluttered environment. Chodos et al. [3] developed a window algorithm embedded in a tracker, which is able to adjust the track gate size in four directions using the gradient function formed from the correlation weighting function. However, we expect this method to be unsuitable for being applied to a large object moving fast near an observer since it is able to adjust only by one pixel in each direction.

An adaptive window without a proper sizing mechanism can hardly accommodate itself to environment variations when a window size is much larger or smaller than an object size or an object size is abruptly changed. To adjust a window size more rapidly and efficiently, we propose the adaptive window method which is able to control the size continuously with four directional sizing vectors in a complex background. Our method introduces eight sizing vectors estimated from eight districts (four side districts and four corner districts), and decomposes each sizing vector from a corner district into the two orthogonal vectors to estimate the final sizing vectors in four side directions.

In the proposed window method, positive difference of edges (PDOE) image rather than the conventional edge image is adopted as a basic correlation feature, since its use is found to be quite useful in compressing background components. The detailed description of the PDOE is beyond the viewpoint of this paper. Thus we briefly introduce the PDOE and the applied correlator in Section 2. In Section 3, we detail the structure and procedure of the proposed window method. Then in Section 4, we provide experimental results using artificial and real image sequences. Finally, we include a conclusion in Section 5.

Section snippets

Applied image tracker architecture

The image tracking block diagram we propose is described in Fig. 1. The overall system is largely divided into the PDOE, the correlator for the main tracking process, the proposed adaptive window block, and the recursive updating loop. First, we acquire the background-reduced image using the PDOE as a tracking feature and then track the object by applying the correlator. Finally, the adaptive window method determines a reference image region tightly enclosing the object to be used in the next

Proposed adaptive window method

Fig. 5 describes an overall block diagram of our proposed adaptive window scheme. The feature set used for adjusting an adaptive window is the PDOE image previously described. Our method needs mainly several steps: the adaptive window setting, the sizing vector estimation from corners and sides, and the window size determination and the reference center point relocation. Here, the adaptive window can expand or shrink independently in four side directions, each side having the sizing magnitude

Experimental results

We have applied our proposed method based on four independent sizing vectors to 45 real image sequences made of more than 3400 images and 39 artificial image sequences made of 4260 images and obtained satisfactory results for most of them. Now, the aim of this Section is to evaluate of the performance of our tracker with adaptive window sizing. However, when we actually performed tracking experiments based on the fixed size window for many cases in which the size of an object image underwent

Conclusions

We have presented an image tracking architecture employing the four-direction adaptive window method with independent sizing vectors for enhancing the performance of correlation-based tracking in the cluttered surroundings. Our method could control the sizing magnitudes fast enough in four directions to reduce the influence of the background and increase the occupancy rate of the target. At the onset of tracking, generally a human operator establishes an initial window with its size much larger

About the Author—SUNG-IL CHIEN recieved the B.S. degree from Seoul National University, Seoul, Korea, in 1977, and the M.S. degree from the Korea Advanced Institute of Science and Technology, Seoul, Korea, in 1981, and Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University in 1988. Since 1981, he has been with the School of Electronic and Electrical Engineering, Kyungpook National University, Taegu, Korea, where he is currently a professor. His research interests

References (8)

  • D. Hughes, A.J.E. Moy, Advances in automatic electro-optical tracking systems, Proc. SPIE 1697 (1992)...
  • D.A. Montera et al.

    Object tracking through adaptive correlation

    Opt. Eng.

    (1994)
  • S.L. Chodos, G.T. Pope, A.K. Rue, R.P. Verdes, Dual Mode Video Tracker, U.S. Patent No. 4849906,...
  • R.L. Brunson, D.L. Boesen, G.A. Crockett, J.F. Riker, Precision trackpoint control via correlation track referenced to...
There are more references available in the full text version of this article.

Cited by (14)

  • An adaptive template matching-based single object tracking algorithm with parallel acceleration

    2019, Journal of Visual Communication and Image Representation
    Citation Excerpt :

    Motion-detection based trackers [26–30] can successfully handle the tracking task in the structured environment and [31–33] have an excellent performance to solve the problem of the object continuous motion. Template-matching based trackers [34–36] are easy and have good robustness. In these methods, trackers based on template matching are the most classical methods in the field of VOT.

  • Corner detection and matching for visual tracking during power line inspection

    2003, Image and Vision Computing
    Citation Excerpt :

    The error between the predicted and measured position of the feature is then used to update the Kalman filter so that predictions are a function of both the model and past motion. There are many variations on this theme such as adaptively varying the size of the match window with the size of the object of interest [3] or taking into account the quality of an individual feature (which can vary with illumination or contrast) as an indication of its reliability for analysing object motion [13]. A method which has attracted attention lately is ‘multiple hypothesis testing’ (MHT) which is used to track multiple objects moving independently in a scene.

  • Deformable object tracking based on Lie algebra

    2012, Zidonghua Xuebao/Acta Automatica Sinica
View all citing articles on Scopus

About the Author—SUNG-IL CHIEN recieved the B.S. degree from Seoul National University, Seoul, Korea, in 1977, and the M.S. degree from the Korea Advanced Institute of Science and Technology, Seoul, Korea, in 1981, and Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University in 1988. Since 1981, he has been with the School of Electronic and Electrical Engineering, Kyungpook National University, Taegu, Korea, where he is currently a professor. His research interests are pattern recognition, computer vision, and neural networks.

About the Author—SI-HUN SUNG recieved the B.S. and M.S. degrees in Electronic Engineering from the Kyungpook National University, Taegu, Korea, in 1995 and 1997, respectively. He is currently working towards the Ph.D degree in Electronic Engineering at the Kyungpook National University as a research assistant. His research interests include the areas of the field application of computer and machine vision, image processing, pattern recognition, and neural networks. He is a member of the SPIE, the IEEE, and the Institute of Electronics Engineers of Korea.

View full text