A unified model sharing framework for moving object detection

Highlights

• The sharing mechanism realizes many-to-one correspondence between pixels and models.

• The model sharing framework reduces the number of models and enhances precision.

• The model sharing framework embed existing approaches and improve their performance.

Abstract

Millions of surveillance cameras have been installed in public areas, producing vast amounts of video data every day. It is an urgent need to develop intelligent techniques to automatically detect and segment moving objects which have wide applications. Various approaches have been developed for moving object detection based on background modeling in the literature. Most of them focus on temporal information but partly or totally ignore spatial information, bringing about sensitivity to noise and background motion. In this paper, we propose a unified model sharing framework for moving object detection. To begin with, to exploit the spatial-temporal correlation across different pixels, we establish a many-to-one correspondence by model sharing between pixels, and a pixel is labeled into foreground or background by searching an optimal matched model in the neighborhood. Then a random sampling strategy is introduced for online update of the shared models. In this way, we can reduce the total number of models dramatically and match a proper model for each pixel accurately. Furthermore, existing approaches can be naturally embedded into the proposed sharing framework. Two popular approaches, statistical model and sample consensus model, are used to verify the effectiveness. Experiments and comparisons on ChangeDetection benchmark 2014 demonstrate the superiority of the model sharing solution.

Introduction

Nowadays an increasing number of surveillance cameras are installed in public places and produce massive video data every day. Therefore, efficiently detecting and analyzing objects of interest, such as person and vehicle, across large scale surveillance videos is of great significance and has become a hot spot of research in the field of computer vision. As a fundamental task in video processing, moving object detection has been widely investigated and background subtraction is the most common solution, which distinguishes moving objects (called foreground) from the scene (called background) typically on the basis of appearance modeling and updates in local or global areas. As the preprocessing step of the whole video process, background subtraction heavily affects the subsequent steps and the overall results.

Over the past years, various approaches, benchmarks and libraries have been developed, which witnesses the importance of background subtraction. Background subtraction can be approached in many different ways. Toward a convenient and high-speed implementation, most modern approaches of background subtraction are based on pixel level modeling. Among these approaches, temporal information is fully utilized to build background models. On the assumption that a pixel is irrelevant to its adjacent pixels, pixel based background subtraction is widely investigated, such as Gaussian Mixture Model (GMM) [1], [2], Kernel Density Estimation (KDE) [3], non-parametric approaches based on sample consensus: Visual Background Extractor (ViBe) [4], Pixel-Based Adaptive Segmenter (PBAS) [5] and Self-Balanced SENsitivity SEgmenter (SuBSENSE) [6]. Although pixel based approaches are effective and easy to bootstrap, they ignore the spatial relationship between pixels, thus they are sensitive to the noise and background motion. Traditional pixel based approaches establish and update a model for each pixel with historical values of this pixel. Ignoring the spatial relationship between pixels, a pixel is will be classified to foreground when its value is changed due to background noise or local motion. For example, the pixels of tree leaves are classified to foreground for pixel based approaches when the tree shakes with wind. Without considering the spatial correlation between adjacent pixels, pixel based models are not robust to noise and background motion. In order to exploit the information around one pixel, some region based approaches [7], [8] were proposed by incorporating the adjacent pixels around a central pixel. This context information enhances the robustness for background noise and illumination, e.g., block descriptors [8] and local binary similarity patterns (LBSP) [9]. Some other approaches [10], [11] clustered pixels into different classes to build models. However, the region based approaches are usually sensitive to the region size and the complexity of the video scene, which inevitably leads to precision loss.

Based on our wide observation, it is not necessary to build a background model for all positions since a model can be easily shared by the neighbor pixels with similar appearance. Pixel based models establish superfluous models and are sensitive to background noise and movement, while region based models suffer from region size and precision loss. To fully exploit the spatial-temporal correlation across different pixels and accurately find a model for each pixel, we propose a novel framework to learn shared models for moving object detection. On one hand, we argue that a model can be dynamically shared by different pixels in different frames because adjacent pixels have similar pattern in space and time. It is not necessary to build a background model for each pixel in a texture-consistent region. This kind of sharing framework could dynamically associate between pixels and models to reduce the total number of models. On the other hand, existing approaches can be naturally embedded into the model sharing framework. The pixel based background approaches such as GMM [1] and ViBe [4] can be embedded into our framework to exploit the relation of model sharing whatever feature or model are utilized. While region or block based approaches cannot be embedded. The reason is that our framework can realize the many-to-one sharing mechanism between pixels and models without influencing the feature or mathematical representation of background models. To verify the effectiveness of the proposed framework we apply it on the statistical model and the sample consensus model. With the sharing framework, the noises caused by local small movements can be effectively eliminated by dynamically searching for a shared model around. Meanwhile the number of models is reduced remarkably and the performance of shared models is superior to original models.