Cut set-based Dynamic Key frame selection and Adaptive Layer-based Background Modeling for background subtraction☆
Introduction
In most video analytic systems, background subtraction is performed for moving foreground detection. In static cameras, background subtraction is performed by maintaining a statistical model of the background and then comparing its difference from each incoming video frame. Then, this background model is kept updated to reflect illumination variation or any structural change in the background over time [1], [3].
Existing Background Subtraction (BS) techniques are reliable and produce acceptable detection results either with scenario specific parameter tuning or when scene dynamics remains stable. However, due to over-reliance on statistical observations, these techniques show unpredictable performance in dynamic unconstrained scenarios where the characteristics of the operating environment are either unknown or change abruptly [17].
In this paper, a new BS technique, called Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) is proposed, that shows reliable detection performance across dynamic unconstrained scenarios without requiring any scenario-specific parameter tuning.
The contribution can be summarized as follows:
- •
Adaptive layer-based strategy can be used to develop an accurate background model.
- •
To overcome the limitation of key frame selection step at the time of cut transition, Cut set-based Dynamic Key frame selection (CDK) process is introduced which can be used to create a invariant and adaptive background model.
This is an extension to the already published work with RSDT (Rotational Symmetry Dynamic Texture) and SCD (Similar-Congruent-Dissimilar) based Scoring model [10]. In this approach symmetrical operators such as line and rotational symmetry can be used to overcome the problem in dynamic background. So, in the proposed approach, symmetrical operators are utilized to create an invariant background model. Even though it gives good results it has to overcome the limitation with key frame selection and background model creation.
To improve the accuracy and reduce computational complexity, Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) is proposed.
To overcome the limitations in key frame selection strategy of RSDT frame work at the time of cut transition, Cut set-based Dynamic Key frame selection (CDK) process is introduced. It can be used to reduce the error rate gradually with the help of adaptive key frame selection step using cut set. That is at the time of cut transition, cut set is formed to create a background model for the frames within the cut set. So, it is highly adaptable, and such an invariant background modeling is suitable for real world environment.
Adaptive Layer-based Background Modeling (ALBM) process is proposed to dynamically fix the number of layers for background modeling process. That is, it is capable of adaptively changing the layers of the background model with respect to environmental changes such as static, dynamic and illumination scenarios. So, the accuracy of the proposed work increases. Highly similar patches are set as background before foreground detection step. This has reduced computational complexity greatly.
The paper is structured as follows: a survey of background subtraction and object tracking algorithms is discussed in Section 2. Section 2 presents the concepts of the proposed ALBM. The experimental results are given in Section 3. Finally, conclusion and future work are presented in Section 4.
Section snippets
Background subtraction
Background subtraction is a fundamental step for many computer vision applications, such as indoor surveillance [27], anomaly detection [34], sports video analysis [36], traffic surveillance [28], [5] and so on. Usually, the scene experiences different impact including lighting changes and dynamic backgrounds. Owing to complex environment and real-time requirement of the surveillance system, numerous techniques [27], [25], [26] have been proposed to conquer the previously stated issues. These
Proposed work
The complete structure of the proposed system is shown in Fig. 1. There are three main sections: (i). Cut set-based Dynamic key frame selection (CDK) (ii). Adaptive Layer-based Background Modeling (ALBM) and (iii). Moving object detection. In the first step, key frame selection is based on cut set which is formed dynamically with the help of CDK. Background modeling involves adaptively changing between two layers for the creation of efficient subspaces.
Experiments
The performance of the proposed work CDK and ALBM is evaluated against six most widely used background subtraction techniques GMM [25], MOG [22], SCS-LBP [37], Bayesian Histogram [26], XCS-LBP [4], Sigma-Delta Z [18] and with the most latest methods Structured-Sparse Decomposition [14], CS-STLTP [12], Generalized Fused Lasso [35], FCDH [19], RSDT with SCD [10].
Conclusion
A Cut set-based Dynamic Key frame selection (CDK) and Adaptive Layer-based Background Modeling (ALBM) approach for background subtraction is proposed along with symmetry-based directional codes to accurately maintain the background model for background subtraction. This approach is suitable for environmental change such as static, dynamic and high illumination.
The proposed CDK step can be used to choose the best key frame for the above three environmental changes. The results of the proposed
Conflict of interest
The authors declare no conflict of interest.
Dr D. Jeyabharathi has completed her B.E degree in the department of computer science and Engineering from Jayaraj Annapackiam CSI College of Engineering, Nazareth, India, under Anna University Chennai in 2009. She has completed her M.E degree in the department of computer science and Engineering from Manonmaniam Sundaranar University, Tirunelveli, in 2013. She is completed her doctorate from Anna University: Tirunelveli Region in 2018. Tirunelveli, in the field of video processing. She is
References (43)
- et al.
A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos
Comput. Vision Image Understanding (CVIU)
(2014) - BMC:...
Traditional and recent approaches in background modeling for foreground detection: an overview
Comput. Sci. Rev.
(2014)- S. Caroline, B. Thierry, F. Carl, An extended center-symmetric local binary pattern for background modeling and...
- V. Cevher, A. Sankaranarayanan, M.F. Duarte, D. Reddy, R.G. Baraniuk, R. Chellappa, Compressive sensing for background...
- Diane L. Herrmann, Paul J. Sally, Number, Shape, & Symmetry, https://books.google.co.in/books?isbn=1466554649, pp....
- et al.
Density-based multifeature background subtraction with support vector machine
IEEE Trans. Pattern Anal. Mach. Intell.
(2012) - ...
- ...
- et al.
A novel Rotational Symmetry Dynamic Texture (RSDT) based sub space construction and SCD (Similar-Congruent-Dissimilar) based scoring model for background subtraction in real time videos
Multimedia Tools Appl.
(2016)
Complex background subtraction by pursuing dynamic spatio – temporal models
Liang IEEE Trans. Image Process.
Contextualized trajectory parsing with spatio-temporal graph
IEEE Trans. Pattern Anal. Mach. Intell.
Background subtraction based on low-rank and structured sparse decomposition
IEEE Trans. Image Process.
A self-organizing approach to background subtraction for visual surveillance applications
IEEE Trans. Image Process.
Spatiotemporal saliency in dynamic scenes
IEEE Trans. Pattern Anal. Mach. Intell.
Perception-inspired background subtraction
IEEE Trans. Circuits Syst. Video Technol.
Σ-Δ Background Subtraction and the Zipf Law
Pattern Recognit. Image Anal. Appl.
Detection of moving objects using fuzzy color difference histogram based background subtraction
IEEE Signal Process. Lett.
Statistical assessment of image symmetries
IEEE Stat. Signal Process. Workshop (SSP)
Cited by (0)
Dr D. Jeyabharathi has completed her B.E degree in the department of computer science and Engineering from Jayaraj Annapackiam CSI College of Engineering, Nazareth, India, under Anna University Chennai in 2009. She has completed her M.E degree in the department of computer science and Engineering from Manonmaniam Sundaranar University, Tirunelveli, in 2013. She is completed her doctorate from Anna University: Tirunelveli Region in 2018. Tirunelveli, in the field of video processing. She is currently working as an Assistant Professor in the department of Information Technology, Sri Krishna College of Technology, Coimbatore. She is a member of IE(INDIA). Her research interest includes Image processing, Network security.
Dr. Dejey received her B.E. and M.E. degrees in Computer Science and Engineering from Manonmaniam Sundaranar University, Tirunelveli, India, in 2003 and 2005, respectively. Later, she was with the Department of Computer Science and Engineering, Manonmaniam Sundaranar University, Tirunelveli, India, as a Junior Research Fellow under the UGC Research Grant. She completed her Ph.D in Computer Science and Engineering in 2011. She has been with the Department of Computer Science and Engineering, Anna University Regional Campus – Tirunelveli, as an Assistant Professor since 2010 and as the Head of the Department from 2011 to 2015. She is a member of IEEE, IE (INDIA) and ISTE. Her research interests include image and signal processing, watermarking, information hiding and multimedia security.
- ☆
This paper has been recommended for acceptance by Olivier Le Meur.