Online classification for object tracking based on superpixel

Abstract

Treating object tracking as a binary classification problem has been greatly explored in recent years. State-of-the-art classification based trackers perform better robustness than many of the other existing trackers. In this paper, we propose a collaborative model by incorporating the local and holistic models together which is corresponding to discriminative and generative models in tracking-classification-framework. At the local level, an on-line Random Forest (RF) classifier is trained to distinguish the superpixels of the object from the background. A series of local superpixels are used to represent the target, so as to adapt the appearance variances. The discriminative model is used to classify superpixels in the next frame as either belonging to the object or background. A confidence map consisting of dependability and stability is formed to measure the probabilities of superpxiels pertaining to the target from the classifier. A modified mean-shift is proposed to work on the confidence map to find the peak, where is the position of the target. Meanwhile, a separate component for managing the training set dynamically is employed to control the updating of the RF model. At the global level, the target is represented by covariance matrix of multi-scale bounding boxes. The generative model is applied for protection measure, which can effectively reduce the drifts during tracking. Experimental results demonstrate that our method is effective and performs favorably in comparison to the state-of-the-art trackers.

Introduction

Object tracking is a focused research field in computer vision for the past few decades. It plays an important role in many computer vision applications such as activity analysis [1], intelligent transportation systems [2], and surveillance [3], etc. Tracking finds a region in the current image that matches the given object. If the matching function takes into account only the object without the background, it might not be able to correctly distinguish the object from the background and the tracking might fail.

Certainly, object tracking is quite a challenging research because the appearance of the same object will change differently under different illumination or viewpoints. Numbers of powerful algorithms have been proposed to track the object in the video over the past two decades. Yang et al. [4] survey the recent progress on visual tracking, including feature descriptors, online learning methods, integration of context information as well as sampling methods. Based on object modeling, a majority of the trackers can be divided into the following two categories: discriminative methods [5], [6], [7], [8], [9], [10] and generative methods [11], [12], [13], [14], [15]. Techniques under the generative approach attempt to build a model based on the object appearance. However, they are not equipped to distinguish an object of interest from the background patches because they consider only object similarity. Consequently, background pixels within a bounding-box are inevitably considered as parts of foreground, thereby bringing amount of inaccurate information for updating the appearance model.

Compared with the generative approach, the discriminative approach aims at separating the object from the background. This observation is evident from several recent developments in the literature [16], [17], [18] which illustrate better tracking results than the generative approach. However, the discriminative approach does not cater for arbitrary object tracking and adaptation to appearance changes. Babenko et al. [18] use multiple-instance learning to train the model with bags of positive examples. Kalal et al. [19] encode rules for additional supervision based on optical flow and a conservative appearance model. Other approaches avoid or delay making hard decisions. Tuzel et al. [20] address the problem of distractor resistance tracking for multiple nearby objects and also poses it as a binary pattern classification problem. Wu et al. [21] propose a regional deep learning tracker that observes the target by multiple sub-regions and each region is observed by a deep learning model. In contrast to most existing trackers which only exploit 2D color or gray images to learn the appearance model of the tracked object online, Zhong et al. [22] take a different approach. Inspired by the increased popularity of depth sensors, they put more emphasis on the 3D context to prevent model drift and handle occlusion. In addition, methods based on correlation filter [23], [24], [25] and methods based on deep learning [26], [27], [28], [29] have achieved excellent performance and draw more attention. Overall consideration, we take two models to complement each other.

In this paper, we try to integrate the merits of both models and the main contributions are summarized as the following aspects. Firstly, we propose a collaborative model by incorporating the local and holistic models together which is corresponding to discriminative and generative models in tracking-by-classification framework. Secondly, we propose an effective and efficient appearance model by superpixel and an online Random Forest (RF) model by incorporating the foreground and background information. This discriminative model utilizes both the local information of the target and spatial-temporal cues of the superpixel. Thirdly, we form a confidence map consisting of dependability and stability to measure the probabilities of superpxiels pertaining to the target. This measure can provide reliable evidence for target localization and discard any superpixels whose positions are far away from the target. Fourthly, we propose a new modified mean-shift algorithm to work on the confidence map for locating the target. The algorithm is verified to be more rapid and accurate. Finally, we represent the target by covariance matrix of multi-scale bounding boxes at the global level. The generative model is applied for protection measure to reduce the drifts. Furthermore, we utilize the strategy for dynamically managing training samples to control model updating. In conjunction with updating mechanism, managing component and protection measure, the proposed algorithm is able to handle complex scenarios. In addition, the target can be tracked by the boundary and video segmentation is also achieved as a by-product in our approach.