Skip to main content
SpringerLink
Log in

Challenging situations for background subtraction algorithms

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Background subtraction is the prerequisite for a wide range of applications including video surveillance, smart environments and content retrieval. Real environments present some challenging situations even for the most recent algorithms, such as shadows, illumination changes, dynamic background, among others. If a real environment is previously known and the challenging situations of this environment can be predicted, the choice of an appropriate algorithm to deal with such situations may be essential for obtaining better segmentation results. In our work, we identify the main situations that affect the performance of background subtraction algorithms and present a classification of these challenging situations. In addition, we present a solution that uses videos and ground-truths from existing datasets to evaluate the performance of segmentation algorithms when they need to deal with a specific challenging situation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Babaee M, Dinh DT, Rigoll G (2018) A deep convolutional neural network for video sequence background subtraction. Pattern Recog 76:635–649. https://doi.org/10.1016/j.patcog.2017.09.040

    Article  Google Scholar 

  2. Badura S, Lieskovsky A, Mokrys M (2011) With shadow elimination towards effective foreground extraction. In: 2011 IEEE international symposium on signal processing and information technology (ISSPIT). IEEE, pp 404–408. https://doi.org/10.1109/ISSPIT.2011.6151596

  3. Barnich O, Droogenbroeck MV (2011) Vibe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20 (6):1709–1724. https://doi.org/10.1109/TIP.2010.2101613

    Article  MathSciNet  MATH  Google Scholar 

  4. Berjón D, Cuevas C, Morán F, García N (2018) Real-time nonparametric background subtraction with tracking-based foreground update. Pattern Recog 74:156–170. https://doi.org/10.1016/j.patcog.2017.09.009

    Article  Google Scholar 

  5. Bianco S, Ciocca G, Schettini R (2017) Combination of video change detection algorithms by genetic programming. IEEE Trans Evol Comput 21(6):914–928. https://doi.org/10.1109/TEVC.2017.2694160

    Article  Google Scholar 

  6. Bloisi D, Iocchi L (2012) Independent multimodal background subtraction. In: Proceedings of the third international conference on computational modeling of objects presented in images: fundamentals, methods and applications. Rome, Italy. https://doi.org/10.1201/b12753-8, pp 39–44

  7. Bloisi DD, Pennisi A, Iocchi L (2017) Parallel multi-modal background modeling. Pattern Recogn Lett 96:45–54. https://doi.org/10.1016/j.patrec.2016.10.016

    Article  Google Scholar 

  8. Bouwmans T, Maddalena L, Petrosino A (2017) Scene background modeling and initialization: a taxonomy. Pattern Recog Lett 96:3–11. https://doi.org/10.1016/j.patrec.2016.12.024

    Article  Google Scholar 

  9. Brutzer S, Höferlin B, Heidemann G (2011) Evaluation of background subtraction techniques for video surveillance. In: CVPR 2011, pp 1937–1944. https://doi.org/10.1109/CVPR.2011.5995508

  10. Calderara S, Prati A, Cucchiara R (2008) HECOL: Homography And epipolar-based consistent labeling for outdoor park surveillance. Comput Vis Image Underst 111(1):21–42. https://doi.org/10.1016/j.cviu.2007.07.006

    Article  Google Scholar 

  11. Dey B, Kundu MK (2015) Efficient foreground extraction from HEVC compressed video for application to real-time analysis of surveillance ‘Big’ data. IEEE Trans Image Process 24(11):3574–3585. https://doi.org/10.1109/TIP.2015.2445631

    Article  MathSciNet  MATH  Google Scholar 

  12. Dou J, Li J (2014) Robust human action recognition based on spatio-temporal descriptors and motion temporal templates. Optik - Int J Light Electron Opt 125(7):1891–1896. https://doi.org/10.1016/j.ijleo.2013.10.022

    Article  Google Scholar 

  13. Fisher R (2016) CAVIAR test case scenarios

  14. Gallego J, Pardàs M, Haro G (2012) Enhanced foreground segmentation and tracking combining bayesian background, shadow and foreground modeling. Pattern Recog Lett 33(12):1558–1568. https://doi.org/10.1016/j.patrec.2012.05.004

    Article  Google Scholar 

  15. Ge W, Dong Y, Guo Z, Chen Y (2014) Background subtraction with dynamic noise sampling and complementary learning. In: 2014 22nd international conference on pattern recognition. IEEE. https://doi.org/10.1109/ICPR.2014.406, pp 2341–2346

  16. Ge W, Guo Z, Dong Y, Chen Y (2016) Dynamic background estimation and complementary learning for pixel-wise foreground/background segmentation. Pattern Recogn 59:112–125. https://doi.org/10.1016/j.patcog.2016.01.031

    Article  Google Scholar 

  17. Goyette N, Jodoin P, Porikli F, Konrad J, Ishwar P (2012) Changedetection.net: A new change detection benchmark dataset. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). https://doi.org/10.1109/CVPRW.2012.6238919, pp 1–8

  18. Gregorio M, Giordano M (2017) Wisardrp for change detection in video sequences. In: 25th European symposium on artificial neural networks, computational intelligence and machine learning, pp 453–458

  19. Hofmann M, Tiefenbacher P, Rigoll G (2012) Background segmentation with feedback: the pixel-based adaptive segmenter. In: 2012 IEEE computer society conference on computer vision and pattern recognition workshops. https://doi.org/10.1109/CVPRW.2012.6238925, pp 38–43

  20. Hsiao HH, Leou JJ (2011) An effective foreground/background segmentation approach for bootstrapping video sequences. In: 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, Foreground Extraction. https://doi.org/10.1109/ICASSP.2011.5946619, pp 1177–1180

  21. i-Lids (2007) i-Lids dataset for AVSS 2007

  22. Isik S, Ozkan K, Gunal S, Gerek ON (2018) Swcd: a sliding window and self-regulated learning-based background updating method for change detection in videos. J Electronic Imaging 27(2):27–27–11. https://doi.org/10.1117/1.JEI.27.2.023002

    Article  Google Scholar 

  23. Li J, Li X, Yang B, Sun X (2015) Segmentation-based image copy-move forgery detection scheme. IEEE Trans Inf Forensics Secur 10(3):507–518. https://doi.org/10.1109/TIFS.2014.2381872

    Article  Google Scholar 

  24. Liu Y, Pados DA (2016) Compressed-Sensed-Domain L 1 -PCA video surveillance. IEEE Trans Multimedia 18(3):351–363. https://doi.org/10.1109/TMM.2016.2514848

    Article  Google Scholar 

  25. Mandellos NA, Keramitsoglou I, Kiranoudis CT (2011) A background subtraction algorithm for detecting and tracking vehicles. Expert Syst Appl 38(3):1619–1631. https://doi.org/10.1016/j.eswa.2010.07.083

    Article  Google Scholar 

  26. Liu N, Wu H, Lin L (2015) Hierarchical ensemble of background models for PTZ-based video surveillance. IEEE Trans Cybern 45(1):89–102. https://doi.org/10.1109/TCYB.2014.2320493

    Article  Google Scholar 

  27. Noh S, Jeon M (2013) A new framework for background subtraction using multiple cues. In: Lee K M, Matsushita Y, Rehg J M, Hu Z (eds) Computer vision – ACCV, 2012. Springer, Berlin, pp 493–506

  28. Qin L, Sheng B, Lin W, Wu W, Shen R (2015) GPU-accelerated Video Background subtraction using gabor detector. J Vis Commun Image Represent 32:1–9. https://doi.org/10.1016/j.jvcir.2015.07.010

    Article  Google Scholar 

  29. Singh M, Nain N, Panwar S (2015) Foreground object extraction using thresholding with automatic shadow removal. In: 2015 11th international conference on signal-image technology & internet-based systems (SITIS), pp 655–662. https://doi.org/10.1109/SITIS.2015.73

  30. Sobral A, Bouwmans T (2014) Bgs library: a library framework for algorithm’s evaluation in foreground/background segmentation. In: Background modeling and foreground detection for video surveillance. CRC Press, Taylor and Francis Group

  31. Sobral A, Vacavant A (2014) A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput Vis Image Underst 122:4–21. https://doi.org/10.1016/j.cviu.2013.12.005

    Article  Google Scholar 

  32. St-Charles PL, Bilodeau GA (2014) Improving background subtraction using local binary similarity patterns. In: IEEE winter conference on applications of computer vision (WACV), 2014. https://doi.org/10.1109/WACV.2014.6836059, pp 509–515

  33. St-Charles PL, Bilodeau GA, Bergevin R (2015) A self-adjusting approach to change detection based on background word consensus. In: 2015 IEEE winter conference on applications of computer vision. https://doi.org/10.1109/WACV.2015.137, pp 990–997

  34. St-Charles PL, Bilodeau GA, Bergevin R (2015) Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans Image Process 24(1):359–373. https://doi.org/10.1109/TIP.2014.2378053

    Article  MathSciNet  MATH  Google Scholar 

  35. Stauffer C, Grimson W (1999) Adaptive background mixture models for real-time tracking. In: Proceedings. 1999 IEEE computer society conference on computer vision and pattern recognition (cat. no PR00149), IEEE comput. Soc, vol 2, pp 246–252. https://doi.org/10.1109/CVPR.1999.784637

  36. Thangarajah A, Wu Q J, Singh A, Mandon B, Chowdhury A (2015) Video foreground detection in non-static background using multi-dimensional color space. Procedia Computer Science 70:55–61. https://doi.org/10.1016/j.procs.2015.10.030

    Article  Google Scholar 

  37. Toyama K, Krumm J, Brumitt B, Meyers B (1999) Wallflower: principles and practice of background maintenance. In: Proceedings of the seventh IEEE international conference on computer vision, vol 1, pp 255–261. https://doi.org/10.1109/ICCV.1999.791228

  38. UK EPSRC REASON (2007) PETS 2007 benchmark data

  39. University of Reading (2001) PETS2001 DATASETS

  40. Vacavant A, Chateau T, Wilhelm A, Lequièvre L (2013) A benchmark dataset for outdoor Foreground/Background extraction. Springer, Berlin, pp 291–300. https://doi.org/10.1007/978-3-642-37410-4_25

    Google Scholar 

  41. Wang J, Wang Y, Jiang M, Yan X, Song M (2014) Moving cast shadow detection using online sub-scene shadow modeling and object inner-edges analysis. J Vis Commun Image Represent 25(5):978–993. https://doi.org/10.1016/j.jvcir.2014.02.015

    Article  Google Scholar 

  42. Wang K, Gou C, Wang FY (2018) m 4 c d : a robust change detection method for intelligent visual surveillance. IEEE Access 6:15505–15520. https://doi.org/10.1109/ACCESS.2018.2812880

    Article  Google Scholar 

  43. Wang L, Yin B, Guo A, Ma H, Cao J (2018) Skip-connection convolutional neural network for still image crowd counting. Appl Intell 48:3360–3371. https://doi.org/10.1007/s10489-018-1150-1

    Article  Google Scholar 

  44. Wang Y, Qi Y (2013) Memory-based cognitive modeling for robust object extraction and tracking. Appl Intell 39(3):614–629. https://doi.org/10.1007/s10489-013-0437-5

    Article  Google Scholar 

  45. Wang Y, Jodoin PM, Porikli F, Konrad J, Benezeth Y, Ishwar P (2014) CDnet 2014: an expanded change detection benchmark dataset. In: 2014 IEEE conference on computer vision and pattern recognition workshops. IEEE. https://doi.org/10.1109/CVPRW.2014.126, pp 393–400

  46. Wei W, Yang XL, Zhou B, Feng J, Shen PY (2012) Combined energy minimization for image reconstruction from few views. Math Probl Eng 2012:1–15. https://doi.org/10.1155/2012/154630

    MathSciNet  MATH  Google Scholar 

  47. Wei W, Qiang Y, Zhang J (2013) A bijection between lattice-valued filters and lattice-valued congruences in residuated lattices. Mathematical Problems in Engineering 2013. https://doi.org/10.1155/2013/908623

  48. Wei W, Fan X, Song H, Wang H (2017) Video tamper detection based on multi-scale mutual information. Multimed Tools Appl 9:1–18. https://doi.org/10.1007/s11042-017-5083-1

    Google Scholar 

  49. Yang D, Zhao C, Zhang X, Huang S (2018) Background modeling by stability of adaptive features in complex scenes. IEEE Trans Image Process 27(3):1112–1125. https://doi.org/10.1109/TIP.2017.2768828

    Article  MathSciNet  MATH  Google Scholar 

  50. Zhao Z, Bouwmans T, Zhang X, Fang Y (2012) Proceedings of second international conference multimedia and signal processing, Springer, Berlin, Heidelberg, chap a fuzzy background modeling approach for motion detection in dynamic backgrounds, pp 177–185. https://doi.org/10.1007/978-3-642-35286-7_23

Download references

Author information

Authors and Affiliations

  1. Universidade Tecnológica Federal do Paraná, Cornélio Procópio, Brazil

    Silvio R. R. Sanches & Claiton Oliveira

  2. Universidade Estadual Paulista “Júlio de Mesquita Filho”, Bauru, Brazil

    Antonio C. Sementille

  3. Universidade de São Paulo, São Paulo, Brazil

    Valdinei Freire

Authors
  1. Silvio R. R. Sanches
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claiton Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio C. Sementille
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Valdinei Freire
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Silvio R. R. Sanches.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sanches, S.R.R., Oliveira, C., Sementille, A.C. et al. Challenging situations for background subtraction algorithms. Appl Intell 49, 1771–1784 (2019). https://doi.org/10.1007/s10489-018-1346-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-018-1346-4

Keywords

Search

Navigation