Skip to main content
Springer Nature Link
Log in

SuperBE: computationally light background estimation with superpixels

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper presents a motion-based superpixel-level background estimation algorithm that aims to be competitively accurate while requiring less computation time for background modelling and updating. Superpixels are chosen for their spatial and colour coherency and can be grouped together to better define the shapes of objects in an image. RGB mean and colour covariance matrices are used as the discriminative features for comparing superpixels to their background model samples. The background model initialisation and update procedures are inspired by existing approaches, with the key aim of minimising computational complexity and therefore processing time. Experiments carried out with a widely used dataset show that SuperBE can achieve a high level of accuracy and is competitive against other state-of-the-art background estimation algorithms. The main contribution of this paper is the computationally efficient use of superpixels in background estimation while maintaining high accuracy, reaching 135 fps on 320 × 240 resolution images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Süsstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)

    Article  Google Scholar 

  2. Babaee, M., Dinh, D.T., Rigoll, G.: A deep convolutional neural network for background subtraction. CoRR (2017). http://arxiv.org/abs/1702.01731

  3. Bak, S., Corvée, E., Brémond, F., Thonnat, M.: Person re-identification using spatial covariance regions of human body parts. In: International Conference on Advanced Video and Signal Based Surveillance, pp. 435–440 (2010)

  4. Barnich, O., van Droogenbroeck, M.: Vibe: a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)

    Article  MathSciNet  Google Scholar 

  5. Bouwmans, T.: Traditional and recent approaches in background modeling for foreground detection: an overview. Comput. Sci. Rev. 11(12), 31–66 (2014)

    Article  Google Scholar 

  6. Bradski, G.: The OpenCV Library. Dr Dobb’s Journal of Software Tools (2000)

  7. Brutzer, S., Höferlin, B., Heidemann, G.: Evaluation of background subtraction techniques for video surveillance. In: International Conference on Computer Vision and Pattern Recognition, pp. 1937–1944 (2011)

  8. Cherian, A., Sra, S., Banerjee, A., Papanikolopoulos, N.: Jensen–Bregman logdet divergence with application to efficient similarity search for covariance matrices. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2161–2174 (2013)

    Article  Google Scholar 

  9. Devi, S.K., Malmurugan, N., Poornima, S.: Improving the efficiency of background subtraction using superpixel extraction and midpoint for centroid. Int. J. Comput. Appl. 43(10), 1–5 (2012)

    Google Scholar 

  10. El Baf, F., Bouwmans, T., Vachon, B.: Type-2 fuzzy mixture of Gaussians model: application to background modeling. In: International Symposium on Visual Computing, pp. 772–781 (2008)

  11. Figueira, D., Taiana, M., Nambiar, A., Nascimento, J., Bernardino, A.: The HDA+ data set for research on fully automated re-identification systems. In: European Conference on Computer Vision Workshops, pp. 241–255 (2015)

    Chapter  Google Scholar 

  12. Förstner, W., Moonen, B.: A metric for covariance matrices. In: Grafarend E.W., Krumm F.W., Schwarze V.S. (eds) Geodesy-The Challenge of the 3rd Millennium, pp. 299–309, Springer, Berlin, Heidelberg (2003)

    Chapter  Google Scholar 

  13. Giordano, D., Murabito, F., Palazzo, S., Spampinato, C.: Superpixel-based video object segmentation using perceptual organization and location prior. In: Conference on Computer Vision and Pattern Recognition, pp. 4814–4822 (2015)

  14. Gu, X., Deng, J.D., Purvis, M.K.: Improving superpixel-based image segmentation by incorporating color covariance matrix manifolds. In: International Conference on Image Processing, pp. 4403–4406 (2014)

  15. Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: Conference on Computer Vision and Pattern Recognition Workshops, pp. 38–43 (2012)

  16. Ivanov, Y., Bobick, A., Liu, J.: Fast lighting independent background subtraction. In: IEEE Workshop on Visual Surveillance (1998)

  17. Javed, S., Oh, S.H., Sobral, A., Bouwmans, T., Jung, S.K. Background subtraction via superpixel-based online matrix decomposition with structured foreground constraints. In: International Conference on Computer Vision Workshop, pp. 930–938 (2015)

  18. Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.S.: Real-time foreground-background segmentation using codebook model. Real Time Imaging 11(3), 172–185 (2005)

    Article  Google Scholar 

  19. Krumm, J., Harris, S., Meyers, B., Brumitt, B., Hale, M., Shafer, S.: Multi-camera multi-person tracking for easyliving. In: International Workshop on Visual Surveillance, pp. 3–10 (2000)

  20. Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M.: Video segmentation by tracking many figure-ground segments. In: International Conference on Computer Vision, pp. 2192–2199 (2013)

  21. Liem, M.C., Gavrila, D.M.: A comparative study on multi-person tracking using overlapping cameras. In: International Conference on Computer Vision Systems, pp. 203–212 (2013)

    Google Scholar 

  22. Liem, M.C., Gavrila, D.M.: Joint multi-person detection and tracking from overlapping cameras. Comput. Vis. Image Underst. 128, 36–50 (2014)

    Article  Google Scholar 

  23. Lim, J., Han, B.: Generalized background subtraction using superpixels with label integrated motion estimation. In: European Conference on Computer Vision, pp. 173–187 (2014)

    Chapter  Google Scholar 

  24. Lu, W., Tan, Y.P.: A color histogram based people tracking system. Int. Symp. Circuits Syst. 2, 137–140 (2001)

    Google Scholar 

  25. Luque, R.M., Domínguez, E., Palomo, E.J., Muñoz, J.: A neural network approach for video object segmentation in traffic surveillance. In: International Conference on Image Analysis and Recognition, pp. 151–158 (2008)

  26. Mittal, A., Davis, L.S.: M2tracker: a multi-view approach to segmenting and tracking people in a cluttered scene. Int. J. Comput. Vis. 51(3), 189–203 (2003)

    Article  Google Scholar 

  27. Papazoglou, A., Ferrari, V.: Fast object segmentation in unconstrained video. In: International Conference on Computer Vision, pp. 1777–1784 (2013)

  28. Piccardi, M.: Background subtraction techniques: a review. In: International Conference on Systems, Man, and Cybernetics, pp. 3099–3104 (2004)

  29. Schick, A., Bäuml, M., Stiefelhagen, R.: Improving foreground segmentations with probabilistic superpixel markov random fields. In: Conference on Computer Vision and Pattern Recognition Workshops, pp. 27–31 (2012)

  30. Shoushtarian, B., Bez, H.E.: A practical adaptive approach for dynamic background subtraction using an invariant colour model and object tracking. Pattern Recognit. Lett. 26(1), 5–26 (2005)

    Article  Google Scholar 

  31. Shu, G., Dehghan, A., Shah, M.: Improving an object detector and extracting regions using superpixels. In: Conference on Computer Vision and Pattern Recognition, pp. 3721–3727 (2013)

  32. Sobral, A., Vacavant, A.: A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput. Vis. Image Underst. 122, 4–21 (2014)

    Article  Google Scholar 

  33. Spampinato, C., Palazzo, S., Kavasidis, I.: A texton-based kernel density estimation approach for background modeling under extreme conditions. Comput. Vis. Image Underst. 122, 74–83 (2014)

    Article  Google Scholar 

  34. St-Charles, P., Bilodeau, G., Bergevin, R. A self-adjusting approach to change detection based on background word consensus. In: Winter Conference on Applications of Computer Vision, pp. 990–997 (2015)

  35. St-Charles, P., Bilodeau, G., Bergevin, R.: Subsense: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)

    Article  MathSciNet  Google Scholar 

  36. Tsai, D., Lai, S.: Independent component analysis-based background subtraction for indoor surveillance. IEEE Trans. Image Process. 18(1), 158–167 (2009)

    Article  MathSciNet  Google Scholar 

  37. Wang, H., Suter, D.: Background subtraction based on a robust consensus method. In: International Conference on Pattern Recognition, pp. 223–226 (2006)

  38. Wang, R., Bunyak, F., Seetharaman, G., Palaniappan, K.: Static and moving object detection using flux tensor with split gaussian models. In: Conference on Computer Vision and Pattern Recognition, pp. 420–424 (2014)

  39. Wang, Y., Jodoin, P., Porikli, F.M., Konrad, J., Benezeth, Y., Ishwar, P.: Cdnet 2014: an expanded change detection benchmark dataset. In: Conference on Computer Vision and Pattern Recognition, pp. 393–400 (2014)

  40. Xin, B., Tian, Y., Wang, Y., Gao, W.: Background subtraction via generalized fused lasso foreground modeling. In: Conference on Computer Vision and Pattern Recognition, pp. 4676–4684 (2015)

  41. Zivkovic, Z.: Improved adaptive gaussian mixture model for background subtraction. Int. Conf. on Pattern Recognit. 2, 28–31 (2004)

    Google Scholar 

Download references

Acknowledgements

This research was conducted with support from the New Zealand eScience Infrastructure (NeSI) Group and their high-performance computing facilities.

Author information

Authors and Affiliations

  1. Embedded Systems Research Group, Department of Electrical and Computer Engineering, The University of Auckland, Auckland, New Zealand

    Andrew Tzer-Yeu Chen, Morteza Biglari-Abhari & Kevin I-Kai Wang

Authors
  1. Andrew Tzer-Yeu Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Morteza Biglari-Abhari
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Kevin I-Kai Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Andrew Tzer-Yeu Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, A.TY., Biglari-Abhari, M. & Wang, K.IK. SuperBE: computationally light background estimation with superpixels. J Real-Time Image Proc 16, 2319–2335 (2019). https://doi.org/10.1007/s11554-018-0750-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0750-7

Keywords