Abstract
We present a new approach for modeling background in complex scenes that contain unpredicted motions caused e.g. by wind over water surface, in tree branches, or over the grass. The background model of each pixel is defined based on the observation of its spatial neighborhood in a recent history, and includes up to \(K \ge 1\) modes, which defines the frequently appeared patterns at the given pixel position in the color space, ranked in decreasing order of occurrence frequency. Foreground regions can then be detected by comparing the intensity of an observed pixel to the high frequency modes of its background model. Experiments show that our spatial-temporal background model is superior to traditional related algorithms in cases for which a pixel encounters modes that are frequent in the spatial neighborhood without being frequent enough in the actual pixel position. As an additional contribution, our paper also proposes an original assessment method, which has the advantage of avoiding the use of costly handmade ground truth sequences of foreground objects silhouettes.








Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.References
Barnich O, Van Droogenbroeck M (2011) Vibe: a universal background subtraction algorithm for video sequences. Image Process IEEE Trans 20(6):1709–1724
Cho SH, Kang HB (2011) Panoramic background generation using mean-shift in moving camera environment. In: Proceedings of the international conference on image processing, computer vision, and pattern recognition
Cristani M, Farenzena M, Bloisi D, Murino V (2010) Background subtraction for automated multisensor surveillance: a comprehensive review. EURASIP J Adv Sig Process 2010:43
Elgammal A, Harwood D, Davis L (2000) Non-parametric model for background subtraction. In: Computer visionłECCV 2000. Springer, pp 751–767
Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW) 2012. IEEE, pp 38–43
Kim K, Chalidabhongse TH, Harwood D, Davis L (2005) Real-time foreground–background segmentation using codebook model. Real-Time Imaging 11(3):172–185
Kim W, Jung C, Kim C (2011) Spatiotemporal saliency detection and its applications in static and dynamic scenes. Circ Syst Video Technol IEEE Trans 21(4):446–456
Ko T, Soatto S, Estrin D (2008) Background subtraction on distributions. In: Computer vision–ECCV 2008. Springer, pp 276–289
Ko T, Soatto S, Estrin D (2010) Warping background subtraction. In: IEEE conference on computer vision and pattern recognition (CVPR) 2010. IEEE, pp 1331–1338
Liu Y, Yao H, Gao W, Chen X, Zhao D (2007) Nonparametric background generation. J Vis Commun Image Represent 18(3):253–263
Mahadevan V, Vasconcelos N (2010) Spatiotemporal saliency in dynamic scenes. Pattern Anal Mach Intell IEEE Trans 32(1):171–177
Mittal A, Paragios N (2004) Motion-based background subtraction using adaptive kernel density estimation. In: Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition 2004. CVPR 2004, vol 2. IEEE, pp II–302
Noriega P, Bernier O (2006) Real time illumination invariant background subtraction using local kernel histograms. In: British machine vision association (BMVC), pp 567–580
Pal A, Schaefer G, Celebi ME (2010) Robust codebook-based video background subtraction. In: IEEE international conference on acoustics speech and signal processing (ICASSP) 2010. IEEE, pp 1146–1149
Pless R (2005) Spatio-temporal background models for outdoor surveillance. EURASIP J Appl Sig Process 2005:2281–2291
Sidibé D, Strauss O, Puech W (2009) Automatic background generation from a sequence of images based on robust mode estimation. In: IS&T/SPIE electronic imaging. International Society for Optics and Photonics, pp 72 500P–72 500P
Stauffer C, Grimson WEL (1999) Adaptive background mixture models for real-time tracking. In: IEEE computer society conference on computer vision and pattern recognition 1999, vol 2. IEEE
Tian Y, Wang Y, Hu Z, Huang T (2013) Selective eigenbackground for background modeling and subtraction in crowded scenes. In: IEEE transactions on circuits and systems for video technology
Wang H, Suter D (2006) A novel robust statistical method for background initialization and visual surveillance. In: Computer vision–ACCV 2006. Springer, pp 328–337
Wren CR, Azarbayejani A, Darrell T, Pentland AP (1997) Pfinder: real-time tracking of the human body. IEEE Trans Pattern Anal Mach Intell 19(7):780–785
Wu M, Peng X (2010) Spatio-temporal context for codebook-based dynamic background subtraction. AEU-Int J Electron Commun 64(8):739–747
Acknowledgments
The authors would like to thank all the anonymous reviewers for their time and valuable comments. This work was supported in part by the National Natural Science Foundation of China, under Project 61302125, and in part by the Fundamental Research Funds for the Central Universities, China.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sun, L., Sheng, W. & Liu, Y. Background modeling and its evaluation for complex scenes. Multimed Tools Appl 74, 3947–3966 (2015). https://doi.org/10.1007/s11042-013-1806-0
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-013-1806-0