Skip to main content
SpringerLink
Log in

Spatiotemporal local compact binary pattern for background subtraction in complex scenes

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A variety of binary feature descriptors such as local binary pattern (LBP) and its variations have recently attracted considerable attention for modelling backgrounds, due to their robustness and strong discriminatory power. However, most existing binary feature descriptors fail to model complex scenes due to their sensitivity to noise. In this paper, we propose an effective local compact binary descriptor for background modelling. For each image, local compact binary patterns (LCBPs) are first extracted by computing a number of low-dimensional pixel difference vectors (PDVs). Then, the LCBP is extended to the spatiotemporal domain taking into account the temporal persistence of pixels, and a novel local compact binary descriptor, STLCBP, is proposed. Multiple color spaces are also considered in order to separate foreground from background pixels accurately. Finally, a joint domain-range adaptive kernel density estimate (KDE) model is used to estimate the background and foreground scores by combining texture features with color features. Experimental results on two well-known datasets, I2R and CDnet2014, demonstrate that the proposed approach significantly outperforms many state-of-the-art methods and works effectively on a wide range of complex videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Amraee S, Vafaei A, Jamshidi K, Adibi P (2018) Anomaly detection and localization in crowed scenes using connected component analysis. Multimed Tools Appl 77(12):14767–14782

    Article  Google Scholar 

  2. Balcilar M, Amasyali M, Sonmez A (2014) Moving object detection using lab2000HL color space with spatial and temporal smoothing. Applied Math Infor Sci 8(4):1755–1766

    Article  Google Scholar 

  3. Balcilar M, Karabiber F, Sonmez A (2013) Performance analysis of Lab2000HL color space for background subtraction. IEEE Inter Symon Innov in Intell Syst Appl:1–6

  4. Barnich O, Droogenbroeck M (2011) ViBe: A universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724

    Article  MathSciNet  Google Scholar 

  5. Bilodeau G, Jodoin J, Saunier N (2013) Change detection in feature space using local binary similarity patterns. Int Conf Comput Robot Vis:106–112

  6. Wu J, Huang F, Hu W, He W (2019) Study of multiple moving detection in fisheye video based on the moving blob model. Tools Appl 78:877-896

  7. Chen M, Wei X, Yang Q, Li Q, Wang G, Yang M (2018) Spatiotemporal GMM for background subtraction with superpixel hierarchy. IEEE Trans Pattern Anal Mach Intell 40(6):1518–1525

    Article  Google Scholar 

  8. Chengjun J, Guiran C, Wei C, Huiyan J (2011) Background extraction and update method based on histogram in ycbcr color space. Inter Conf E-Business E-Government:1–4

  9. Comaniciu D, Zhu Y, Davis L (2008) Sequential kernel density approximation and its application to real-time visual tracking. IEEE Trans Pattern Anal Mach Intell 30(7):1186–1197

    Article  Google Scholar 

  10. Cuevas C, Mohedano R, Garcia N (2012) Versatile Bayesian classifier for moving object detection by non-parametric background-foreground modeling. IEEE Conf on Image Process:313–316

  11. Duan Y, Lu J, Feng J, Zhou J (2018) Context-aware local binary feature learning for face recognition. IEEE Trans Pattern Anal Mach Intell 40(5):1139–1152

    Article  Google Scholar 

  12. Elgammal A, Duraiswami R, Harvood D, Davis L (2002) Background and foreground modeling using nonparametric kernel density estimation for visual surveillance. Proc IEEE 90(7):1151–1163

    Article  Google Scholar 

  13. Guo L, Xu D, Qiang Z (2016) Background subtraction using local SVD binary pattern. Proc IEEE Conf Comput Vis Patt Recogn:86–94

  14. He W, King Y, Wu J et al (2018) Local compact binary patterns for background subtraction in complex scenes. Proc Int Conf Pattern Recognit:1518–1523

  15. Heikkila M, Pietikainen M (2006) A texture-based method for modeling the background and detecting moving objects. IEEE Trans Pattern Anal Mach Intell 28(4):657–662

    Article  Google Scholar 

  16. Heikkila M, Pietikainen M, Heikkila J (2004) A texture-based method for detecting moving objects. IEEE Trans in British Mach Vis Conf:187–196

  17. Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: The pixel-based adaptive segmenter. IEEE Conf on Comput Vis Pattern Recogn:38–43

  18. Johnson G, Song X, Montag E, Fairchild M (2010) Derivation of a color space for image color difference measurement. Color Res Appl 35(6):387–400

    Article  Google Scholar 

  19. Karpagavalli P, Ramprasad AV (2017) An adaptive hybrid GMM for multiple human detection in crowd scenario. Multimed Tools Appl 76(122):14129–14149

    Article  Google Scholar 

  20. Li L, Huang W, Gu I, Tian Q (2003) Foreground object detection from videos containing complex background. Proc ACM Conf on Multi:2–10

  21. Liao S, Zhao G, Kellokumpu V, Pietikainen M, Li S (2010) Modeling pixel processing with scale invariant local patterns for background subtraction in complex scenes. IEEE Conf on Comput Vis Pattern Recogn:1301–1306

  22. Lin L, Xu Y, Liang X, Lai J (2014) Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 23(7):3191–3202

    Article  MathSciNet  Google Scholar 

  23. Lissner I, Preiss J, Urban P, Lichtenauer M, Zolliker P (2013) Image-Difference prediction: from grayscale to color. IEEE Trans Image Process 22(2):435–446

    Article  MathSciNet  Google Scholar 

  24. Lissner I, Urban P (2012) Toward a unified color space for perception-based image processing. IEEE Trans Image Process 21(3):1153–1168

    Article  MathSciNet  Google Scholar 

  25. Liu L, Lao S, Guo Y, Wang X, Pietikainen M (2016) Median robust extended local binary pattern for texture classification. IEEE Trans Image Process 25(3):1368–1381

    Article  MathSciNet  Google Scholar 

  26. Liu W, Lin S, Yang M, Huang C (2013) Real-time binary descriptor based background modeling. Asian Conf on Pattern Recogn:722–726

  27. Liu X, Zhao G, Yao J, Qi C (2015) Background subtraction based on low-rank and structured sparse decomposition. IEEE Trans Image Process 24(8):2502–2514

    Article  MathSciNet  Google Scholar 

  28. Lu J, Liong V, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056

    Article  Google Scholar 

  29. Maddalena L, Petrosino A (2012) The SOBS algorithm: what are the limits? IEEE Conf on Comput Vis Pattern Recogn Workshops:21–26

  30. Maddalena L, Petrosino A (2014) The 3dSOBS+ algorithm for moving object detection. Comput Vis Image Und 122:65–73

    Article  Google Scholar 

  31. Narayana M, Hanson A, Learned-Miller E (2012) Background modeling using adaptive pixelwise kernel variances in a hybrid feature space. IEEE Conf on Comput Vis Pattern Recogn:2104–2111

  32. Narayana M, Hanson A, Learned-Miller E (2012) Improvements in joint domain-range modeling for background subtraction. Proceed of the British Mach Vis Conf:1–11

  33. Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  34. Pan Z, Liu S, Fu W (2017) A review of visual moving target tracking. Multimed Tools Appl 76(16):16989–17018

    Article  Google Scholar 

  35. Roy S, Ghosh A (2018) Real-time adaptive histogram min-max bucket (HMMB) model for background subtraction. IEEE Trans Pattern Anal Mach Intell 28(7):1513–1525

    Google Scholar 

  36. Sajid H, Cheung S (2017) Universal multimode background subtraction. IEEE Trans Image Process 26(7):3249–3260

    Article  MathSciNet  Google Scholar 

  37. Seki M, Wada T, Fujiwara H, Sumi K (2003) Background subtraction based on co-occurrence of image variations. IEEE Conf on Comput Vis Pattern Recogn II:65–72

    Google Scholar 

  38. Sheikh Y, Shah M (2005) Bayesian modeling of dynamic scenes for object detection. IEEE Trans Pattern Anal Mach Intell 27(11):1178–1792

    Article  Google Scholar 

  39. Shu Y, Zhang H (2018) Multimodal information fusion based human movement recognition. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-6315-8

  40. Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. IEEE Conf on Comput Vis Pattern Recogn:246–252

  41. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650

    Article  MathSciNet  Google Scholar 

  42. Wang Y, Jodoin P, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) CDnet 2014: An expanded change detection benchmark dataset. IEEE Workshop Comput Vis Pattern Recogn:387–394

  43. Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Program 142(1-2):397–434

    Article  MathSciNet  Google Scholar 

  44. Wu J, Huang F, Hu W et al (2018) Study of multiple moving targets’ detection in fisheye video based on the moving blob model. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-5763-5

    Article  Google Scholar 

  45. Xue G, Song L, Sun J, Wu M (2011) Hybrid center-symmetric local pattern for dynamic background subtraction. IEEE Inter Conf on Multimed Expo:1–6

  46. Xue G, Sun J, Song L (2010) Dynamic background subtraction based on spatial extended center-symmetric local binary pattern. IEEE Inter Conf on Multimed Expo:1050–1054

  47. Zhang S, Yao H, Liu S (2008) Dynamic background Modeling and subtraction using spatio-temporal local binary patterns. IEEE Conf on Image Process:1556–1559

  48. Zhao G, Ahonen T, Matas J, Pietikainen M (2012) Rotation-invariant image and video description with local binary pattern features. IEEE Trans Image Process 21(4):1465–1477

    Article  MathSciNet  Google Scholar 

  49. Zhao G, Pietikaninen M (2007) Dynamic Texture Recognition using volume local binary patterns with an application to facial expressions. IEEE Trans Pattern Anal Mach Intell 29(6):915–928

    Article  Google Scholar 

  50. Zivkovic Z (2004) Improved adaptive Gaussian mixture model for background subtraction. Inter Conf on Pattern Recogn II:28–31

    Google Scholar 

Download references

Acknowledgements

This work has been supported in part by Hunan Provincial Natural Science Foundation of China (2019JJ40104, 2019JJ50211, 2019JJ50212), the Open Fund of Education Department of Hunan Province (18K086), the Science and Technology Program of Hunan Province (2016TP1021), the Development of Distributed Underwater Monitoring and Control Networks funded by the Ministry of Ocean and Fisheries, South Korea.

Author information

Authors and Affiliations

  1. School of Information and Communication Engineering, Hunan Institute of Science and Technology, Yueyang, 414006, China

    Wei He, Jianhui Wu, Guoyun Zhang, Qi Qi, Bing Tu & Xianfeng Ou

  2. Department of Information and Communication Engineering, Hoseo University, Asan, 31499, South Korea

    Wei He, Hak-Lim Ko & Yong Kwan Kim

Authors
  1. Wei He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hak-Lim Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Kwan Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianhui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guoyun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qi Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bing Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Xianfeng Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yong Kwan Kim or Xianfeng Ou.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, W., Ko, HL., Kim, Y.K. et al. Spatiotemporal local compact binary pattern for background subtraction in complex scenes. Multimed Tools Appl 78, 31415–31439 (2019). https://doi.org/10.1007/s11042-019-7688-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7688-z

Keywords

Search

Navigation