Moving cast shadow detection using online sub-scene shadow modeling and object inner-edges analysis

https://doi.org/10.1016/j.jvcir.2014.02.015Get rights and content

Highlights

  • We proposed an accurate and adaptive method for moving cast shadow detection.

  • We develop sub-scene shadow models to describe shadow appearance more accurately.

  • We employ object inner-edges analysis and shadow expanding for improvement.

  • Our method can adaptively handle shadow appearance changes and camouflages.

  • The proposed method outperforms some state-of-the-art methods.

Abstract

In this paper, we propose an adaptive and accurate moving cast shadow detection method employing online sub-scene shadow modeling and object inner-edges analysis for applications of static-camera video surveillance. To describe shadow appearance more accurately, the proposed method builds adaptive online shadow models for sub-scenes with different conditions of irradiance and reflectance. The online shadow models are learned by utilizing Gaussian functions to fit the significant peaks of accumulating histograms, which are calculated from Hue, Saturation and Intensity (HSI) difference of moving objects between background and foreground. Additionally, object inner-edges analysis is adopted to reject camouflages, which are misclassified foreground regions that are highly similar to shadows. Finally, the main shadow regions are expanded to recycle the misclassified shadow pixels based on local color constancy. The proposed algorithm can adaptively handle the shadow appearance changes and camouflages without prior information about illuminations and scenarios. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods.

Introduction

Moving cast shadow detection is one of the crucial issues in visual application on stationary camera surveillance videos. For detecting the moving objects from a stationary camera video, usually a background subtraction technique is utilized for foreground extraction, this leads to classifying shadows as parts of foreground, because shadows share the same movement pattern and have a similar magnitude of intensity change as that of foreground objects [1]. Cast shadows connect the foreground regions of different adjacent objects as one connected component when moving objects are detected. Therefore, cast shadows interfere with accurately describing each connected objects, such as extracting features (size, location, color histogram, etc.). Especially, object tracking and recognition degrade in video surveillance without shadow suppression [2]. Lots of methods are proposed in recent years to detect and remove moving cast shadows. A comprehensive survey work [3] classifies these methods into different categories based on the main used assumptions. Some approaches [4], [5], [6], [7] utilize the assumption that shadow pixels keep chrominance invariant but make intensity darker compared to corresponding background pixels to detect shadows. They tend to analyze the color feature in different color spaces. Cucchiara et al. [4] transform images from RGB space to HSI space, for further analyzing chrominance and intensity independently. Chen et al. [5] analyze the shadow property in YUV space. The work of Shan et al. [6] evaluate the performances in different color spaces like HSV, YUV, Lab, c1c2c3. Sun and Li [7] propose a novel moving cast shadow detection method using combined color models. These chromatic-based methods are simple to implement and can operate very fast, but they would be unreliable when the illumination condition of the applied scene changes or object regions which are very similar to shadows exist.

To deal with the influence of illumination changes, texture-based methods [2], [8], [9] are proposed based on the assumption that current frame covered by shadows becomes darker but retains the texture of corresponding background. Xu et al. [8] use the edge correlation to remove shadows in normal indoor scenes. Javed and Shahs method [9] employs color segmentation to obtain shadow candidate regions. The candidate regions with higher gradient direction correlation are considered to be cast shadow regions. But in most cases, the employed color segmentation will break the texture into small regions, leading to be more sensitive to noise. Then Sanin et al. [2] analyze shadow texture property in large regions using connected components of foreground as candidate shadow regions and achieve higher detection rates. The robustness to illuminations makes texture correlation to be a powerful method for detecting shadows, but it may fail when there are no distinctive textures between objects and background. Therefore, considering the disadvantages and advantages of the above two kinds of methods, some researchers [10], [11], [12] try to combine the shadow properties of chromatic and texture to achieve more robust shadow removal results.

The geometry-based methods are proposed according to proper knowledge of the applied scene conditions like object shapes, camera angles, and illumination sources. Many geometry-based methods [13], [14] are usually used as assistant methods because the geometry assumptions are often violated when the applied scene conditions change in real surveillance videos. However, some researches aim to find more general geometry properties for common applied scenes. Amato et al. [15] segment each connected component of foreground into candidate regions by local color constancy detection. Then they extract a general geometry feature named terminal pixel weight in each candidate region and use a scene-independent threshold to classify these candidate regions as shadow or foreground.

For most of the mentioned kinds of works, all related feature thresholds must be tuned in a static setting way to achieve satisfying results when the applied scene or illumination changes. To tackle this issue, learning-based methods are proposed as a new trend to dynamically adapt to varying conditions without any manual intervention. Most methods learning in a supervised or semi-supervised way need manual labeling work as prior information. For instance, El-Zahhar and Abukarelsedik [16] use an ensemble-driven semi-supervised learning approach for adaptive shadow detection. In their detection system offline-labeling part is very significant. Joshi and Papanikolopoulos [17] propose a more general semi-supervised learning technique that employs Support Vector Machines and Co-training algorithm. But their method still relies on small set of human-labeled data. Then unsupervised learning methods are adopted in cast shadow detection as needing no manually labeled data. Huang and Chen [18] propose an unsupervised way to online model shadows using a Gaussian mixture model of the 3D color features without manual inputting. Huang and Wu’s work [19] is based on the assumption that the color from all moving objects is distributed randomly, but the distribution of shadow is concentrated in a specific range based on the properties of shadow. Therefore, they use a Gaussian function to fit the most significant peak of each color feature accumulating histogram and regard the fitted Gaussian as a global likelihood function of shadows. These unsupervised methods learn the shadow distribution with a general global model, but neglects the fact that shadow appearances may be distinctive in different applied sub-scenes with varied conditions of irradiance and reflectance. This situation happens quite normally at environments with lots of light blocks and different surface materials of background. The global shadow model cannot update timely to accurately handle the shadow appearance changes in different scene regions. In addition, these methods fail to segment shadows from camouflages, which are foreground object regions similar to shadows.

In this paper, we propose an adaptive and accurate shadow detection method using online sub-scene shadow modeling and object inner-edges analysis. The contributions of this method are twofold. As the first contribution of our work, based on the same assumption as Ref. [19], for describing shadow appearances more accurately, we online learn adaptive sub-scene shadow models according to varied conditions of irradiance and reflectance in sub-scenes. The second contribution is that we regard moving cast shadow removal as a system task by incorporating object inner-edges analysis and shadow expanding into our detection system after primary classification. These two operations increase both detection and discrimination rate. In object inner-edges analysis, the object inner-edges properties are exploited to reject the camouflages. Finally, misclassified shadow pixels are recycled by shadow expanding based on the property of local color constancy. The proposed method can robustly deal with both outdoor and indoor shadows, and adaptively handle shadow appearance changes and camouflages without any prior information about scenario and illumination. The rest of this paper is organized as follows: We review the problematic issues of shadow appearance in Section 2. The proposed moving cast shadow detection method will be described in detail in Section 3. Then experimental results and comparisons are provided in Section 4. Finally, in Section 5 we conclude this paper.

Section snippets

Shadow appearances

Cast shadow appears on a surface when direct illuminations from light sources of the surface are partially or completely blocked by objects, producing an appearance change on the surface. Cast shadows are classified as visible and invisible shadows according to different spectral characteristics caused by different light sources between outdoor and indoor [6]. When light is fierceness (faintness), shadow is visible (invisible). Examples of moving cast shadow are presented in Fig. 1(a)–(f)

Proposed moving cast shadow detection method

The general workflow of our method is illustrated in Fig. 3. After foreground extraction, there are three crucial steps: (i) primary classification using online sub-scene learning shadow model, (ii) object inner-edges analysis and (iii) shadow expanding. The step (ii) and (iii) can be seen as correction process. Before the start of the proposed method, crude scene image segmentation is employed for obtaining masks of sub-scenes. The step (i) contains learning phase and classification phase. In

Experimental results

In this section, we evaluate the proposed method by exploring both qualitative and quantitative experimental results. We present the qualitative and quantitative results of the proposed method and compare our method with three state-of-the-art methods, including the method employing global shadow model (GSM) [19], a very well-performed method based on large texture region (LTR) [2], and an outstanding method using local color constancy (LCC) [15]. The C++ implementation of LTR is available

Conclusions

This paper proposed an adaptive and accurate shadow detection algorithm, which can successfully deal with shadow appearance changes and camouflages. According to the illumination conditions of different sub-scene regions, the proposed method develops online shadow models for each sub-scene region for describing shadow appearance more accurately. Moreover, in object inner-edges analysis, object inner-edges are detected and then external terminal pixel weights are calculated to reclassify the

Acknowledgments

This research is partially supported by Chinese advanced research project No. 51301040201, Chinese advanced research foundation Nos. 9140A01010411-JW0503 and 9140A01060113JW05016. The authors are also grateful for the valuable and constructive comments from the reviewers.

References (26)

  • B. Sun et al.

    Moving cast shadow detection of vehicle using combined color models

  • O. Javed et al.

    Tracking and object classification for automated surveillance

  • G. Fung et al.

    Effective moving cast shadow detection for monocular color traffic image sequences

    Opt. Eng.

    (2002)
  • Cited by (0)

    View full text