Cut set-based Dynamic Key frame selection and Adaptive Layer-based Background Modeling for background subtraction

https://doi.org/10.1016/j.jvcir.2018.06.024Get rights and content

Highlights

Abstract

Background subtraction has been widely discussed in video surveillance, but it still has open challenges such as dynamic background, illumination variation. To address these challenges a novel Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) approach for background subtraction is proposed which adaptively changes layers in the background model for each scenario such as static, dynamic background and high illumination.

The concept of key frame is used to choose representative frames from the video. In order to capture the invariant directional codes of each spatio-temporal patch symmetric operators such as line and rotational symmetry are used. The proposed method identifies highly similar static spatio-temporal patches and sets it as background there by reducing the computational complexity in the foreground detection step. Both qualitative and quantitative evaluations on challenging video sequences demonstrate that the proposed algorithm performs background subtraction more favorably than the state-of-the-art methods.

Introduction

In most video analytic systems, background subtraction is performed for moving foreground detection. In static cameras, background subtraction is performed by maintaining a statistical model of the background and then comparing its difference from each incoming video frame. Then, this background model is kept updated to reflect illumination variation or any structural change in the background over time [1], [3].

Existing Background Subtraction (BS) techniques are reliable and produce acceptable detection results either with scenario specific parameter tuning or when scene dynamics remains stable. However, due to over-reliance on statistical observations, these techniques show unpredictable performance in dynamic unconstrained scenarios where the characteristics of the operating environment are either unknown or change abruptly [17].

In this paper, a new BS technique, called Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) is proposed, that shows reliable detection performance across dynamic unconstrained scenarios without requiring any scenario-specific parameter tuning.

The contribution can be summarized as follows:

  • Adaptive layer-based strategy can be used to develop an accurate background model.

  • To overcome the limitation of key frame selection step at the time of cut transition, Cut set-based Dynamic Key frame selection (CDK) process is introduced which can be used to create a invariant and adaptive background model.

This is an extension to the already published work with RSDT (Rotational Symmetry Dynamic Texture) and SCD (Similar-Congruent-Dissimilar) based Scoring model [10]. In this approach symmetrical operators such as line and rotational symmetry can be used to overcome the problem in dynamic background. So, in the proposed approach, symmetrical operators are utilized to create an invariant background model. Even though it gives good results it has to overcome the limitation with key frame selection and background model creation.

To improve the accuracy and reduce computational complexity, Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) is proposed.

To overcome the limitations in key frame selection strategy of RSDT frame work at the time of cut transition, Cut set-based Dynamic Key frame selection (CDK) process is introduced. It can be used to reduce the error rate gradually with the help of adaptive key frame selection step using cut set. That is at the time of cut transition, cut set is formed to create a background model for the frames within the cut set. So, it is highly adaptable, and such an invariant background modeling is suitable for real world environment.

Adaptive Layer-based Background Modeling (ALBM) process is proposed to dynamically fix the number of layers for background modeling process. That is, it is capable of adaptively changing the layers of the background model with respect to environmental changes such as static, dynamic and illumination scenarios. So, the accuracy of the proposed work increases. Highly similar patches are set as background before foreground detection step. This has reduced computational complexity greatly.

The paper is structured as follows: a survey of background subtraction and object tracking algorithms is discussed in Section 2. Section 2 presents the concepts of the proposed ALBM. The experimental results are given in Section 3. Finally, conclusion and future work are presented in Section 4.

Section snippets

Background subtraction

Background subtraction is a fundamental step for many computer vision applications, such as indoor surveillance [27], anomaly detection [34], sports video analysis [36], traffic surveillance [28], [5] and so on. Usually, the scene experiences different impact including lighting changes and dynamic backgrounds. Owing to complex environment and real-time requirement of the surveillance system, numerous techniques [27], [25], [26] have been proposed to conquer the previously stated issues. These

Proposed work

The complete structure of the proposed system is shown in Fig. 1. There are three main sections: (i). Cut set-based Dynamic key frame selection (CDK) (ii). Adaptive Layer-based Background Modeling (ALBM) and (iii). Moving object detection. In the first step, key frame selection is based on cut set which is formed dynamically with the help of CDK. Background modeling involves adaptively changing between two layers for the creation of efficient subspaces.

Experiments

The performance of the proposed work CDK and ALBM is evaluated against six most widely used background subtraction techniques GMM [25], MOG [22], SCS-LBP [37], Bayesian Histogram [26], XCS-LBP [4], Sigma-Delta Z [18] and with the most latest methods Structured-Sparse Decomposition [14], CS-STLTP [12], Generalized Fused Lasso [35], FCDH [19], RSDT with SCD [10].

Conclusion

A Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) approach for background subtraction is proposed along with symmetry-based directional codes to accurately maintain the background model for background subtraction. This approach is suitable for environmental change such as static, dynamic and high illumination.

The proposed CDK step can be used to choose the best key frame for the above three environmental changes. The results of the proposed

Conflict of interest

The authors declare no conflict of interest.

Dr D. Jeyabharathi has completed her B.E degree in the department of computer science and Engineering from Jayaraj Annapackiam CSI College of Engineering, Nazareth, India, under Anna University Chennai in 2009. She has completed her M.E degree in the department of computer science and Engineering from Manonmaniam Sundaranar University, Tirunelveli, in 2013. She is completed her doctorate from Anna University: Tirunelveli Region in 2018. Tirunelveli, in the field of video processing. She is

References (43)

  • S. Andrews et al.

    A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos

    Comput. Vision Image Understanding (CVIU)

    (2014)
  • BMC:...
  • T. Bouwmans

    Traditional and recent approaches in background modeling for foreground detection: an overview

    Comput. Sci. Rev.

    (2014)
  • S. Caroline, B. Thierry, F. Carl, An extended center-symmetric local binary pattern for background modeling and...
  • V. Cevher, A. Sankaranarayanan, M.F. Duarte, D. Reddy, R.G. Baraniuk, R. Chellappa, Compressive sensing for background...
  • Diane L. Herrmann, Paul J. Sally, Number, Shape, & Symmetry, https://books.google.co.in/books?isbn=1466554649, pp....
  • B. Han et al.

    Density-based multifeature background subtraction with support vector machine

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2012)
  • ...
  • ...
  • D. Jeyabharathi et al.

    A novel Rotational Symmetry Dynamic Texture (RSDT) based sub space construction and SCD (Similar-Congruent-Dissimilar) based scoring model for background subtraction in real time videos

    Multimedia Tools Appl.

    (2016)
  • L. Li, W. Huang, I. Gu, Q. Tian, Foreground object detection from videos containing complex background, ACM...
  • Liang Lin et al.

    Complex background subtraction by pursuing dynamic spatio – temporal models

    Liang IEEE Trans. Image Process.

    (2014)
  • X. Liu et al.

    Contextualized trajectory parsing with spatio-temporal graph

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • Xi’an Liu et al.

    Background subtraction based on low-rank and structured sparse decomposition

    IEEE Trans. Image Process.

    (2015)
  • L. Maddalena et al.

    A self-organizing approach to background subtraction for visual surveillance applications

    IEEE Trans. Image Process.

    (2008)
  • V. Mahadevan et al.

    Spatiotemporal saliency in dynamic scenes

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2010)
  • Mahfuzul Haque et al.

    Perception-inspired background subtraction

    IEEE Trans. Circuits Syst. Video Technol.

    (2013)
  • A. Manzanera

    Σ-Δ Background Subtraction and the Zipf Law

    Pattern Recognit. Image Anal. Appl.

    (2007)
  • D. Panda et al.

    Detection of moving objects using fuzzy color difference histogram based background subtraction

    IEEE Signal Process. Lett.

    (2016)
  • M. Pawlak et al.

    Statistical assessment of image symmetries

    IEEE Stat. Signal Process. Workshop (SSP)

    (2011)
  • PETS:...
  • Cited by (0)

    Dr D. Jeyabharathi has completed her B.E degree in the department of computer science and Engineering from Jayaraj Annapackiam CSI College of Engineering, Nazareth, India, under Anna University Chennai in 2009. She has completed her M.E degree in the department of computer science and Engineering from Manonmaniam Sundaranar University, Tirunelveli, in 2013. She is completed her doctorate from Anna University: Tirunelveli Region in 2018. Tirunelveli, in the field of video processing. She is currently working as an Assistant Professor in the department of Information Technology, Sri Krishna College of Technology, Coimbatore. She is a member of IE(INDIA). Her research interest includes Image processing, Network security.

    Dr. Dejey received her B.E. and M.E. degrees in Computer Science and Engineering from Manonmaniam Sundaranar University, Tirunelveli, India, in 2003 and 2005, respectively. Later, she was with the Department of Computer Science and Engineering, Manonmaniam Sundaranar University, Tirunelveli, India, as a Junior Research Fellow under the UGC Research Grant. She completed her Ph.D in Computer Science and Engineering in 2011. She has been with the Department of Computer Science and Engineering, Anna University Regional Campus – Tirunelveli, as an Assistant Professor since 2010 and as the Head of the Department from 2011 to 2015. She is a member of IEEE, IE (INDIA) and ISTE. Her research interests include image and signal processing, watermarking, information hiding and multimedia security.

    This paper has been recommended for acceptance by Olivier Le Meur.

    View full text