Skip to main content

Advertisement

Log in

3D Local Spatio-temporal Ternary Patterns for Moving Object Detection in Complex Scenes

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Humans possess natural cognitive vision to perceive objects in a 3D space and are able to differentiate foreground and background moving objects using their shape, colour and texture. Moving object detection is a leading and challenging task in complex scenes which involve illumination variation, blurriness, camouflage, moving background objects, etc. Inspired by human cognitive vision, a novel descriptor named 3D local spatio-temporal ternary patterns (3D-LStTP) is proposed for moving object detection. The 3D-LStTP collects multidirectional spatio-temporal information from three consecutive frames in a video by forming a 3D grid structure. The background models are constructed by using texture and colour features. The results obtained after modelling are integrated for foreground moving object detection in complex scenes. The performance of proposed algorithm is validated by conducting five experiments on Fish4Knowledge dataset, four experiments on I2R dataset and four experiments on Change Detection dataset. Qualitative and quantitative analyses are carried out on benchmark datasets. The results after investigation prove that the proposed method outperforms the state-of-the-art techniques for moving object detection in terms of ROC, TPR, FPR, Precision and F- measure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Akilan T, Wu QJ, Yang Y. Fusion-based foreground enhancement for background subtraction using multivariate multi-model gaussian distribution. Inf Sci 2018;430:414–31.

    Article  Google Scholar 

  2. Bouwmans T, El Baf F, Vachon B. Background modeling using mixture of gaussians for foreground detection-a survey. Recent Patents on Computer Science 2008;1(3):219–37.

    Article  Google Scholar 

  3. El Baf F, Bouwmans T, Vachon B. Fuzzy integral for moving object detection. IEEE international conference on Fuzzy systems, 2008. FUZZ-IEEE 2008.(IEEE world congress on computational intelligence). IEEE; 2008. p. 1729–36.

  4. Goyette N, Jodoin PM, Porikli F, Konrad J, Ishwar P. Changedetection. net: a new change detection benchmark dataset. 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE; 2012. p. 1–8.

  5. Guo JM, Liu YF, Hsia CH, Shih MH, Hsu CS. Hierarchical method for foreground detection using codebook model. IEEE Transactions on Circuits and Systems for Video Technology 2011;21(6):804–15.

    Article  Google Scholar 

  6. Han B, Davis LS. Density-based multifeature background subtraction with support vector machine. IEEE Transactions on Pattern Analysis and Machine Intelligence 2012;34(5):1017–23.

    Article  PubMed  Google Scholar 

  7. Hati KK, Sa PK, Majhi B. Intensity range based background subtraction for effective object detection. IEEE Signal Processing Letters 2013;20(8):759–62.

    Article  Google Scholar 

  8. Heikkila M, Pietikainen M. A texture-based method for modeling the background and detecting moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 2006;28(4):657–62.

    Article  Google Scholar 

  9. Ji Z, Wang W. Detect foreground objects via adaptive fusing model in a hybrid feature space. Pattern Recogn 2014;47(9):2952–61.

    Article  Google Scholar 

  10. KaewTraKulPong P, Bowden R. An improved adaptive background mixture model for real-time tracking with shadow detection. Video-based surveillance systems. Boston: Springer; 2002. p. 135–144.

  11. Karasulu B, Korukoglu S. Moving object detection and tracking by using annealed background subtraction method in videos: performance optimization. Expert Syst Appl 2012;39(1):33–43.

    Article  Google Scholar 

  12. Kavasidis I, Palazzo S, Di Salvo R, Giordano D, Spampinato C. An innovative web-based collaborative platform for video annotation. Multimedia Tools and Applications 2014;70(1):413–32.

    Article  Google Scholar 

  13. Kim K, Chalidabhongse TH, Harwood D, Davis L. Real-time foreground–background segmentation using codebook model. Real-time imaging 2005;11(3):172–85.

    Article  Google Scholar 

  14. Kim W, Kim Y. Background subtraction using illumination-invariant structural complexity. IEEE Signal Processing Letters 2016;23(5):634–8.

    Article  Google Scholar 

  15. Lee DS. Effective gaussian mixture learning for video background subtraction. IEEE transactions on Pattern Analysis And Machine Intelligence 2005;27(5):827–32.

    Article  PubMed  Google Scholar 

  16. Li L, Huang W, Gu IY, Tian Q. Foreground object detection from videos containing complex background. Proceedings of the eleventh ACM international conference on multimedia. ACM; 2003. p. 2–10.

  17. Liao S, Zhao G, Kellokumpu V, Pietikäinen M, Li SZ. Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE; 2010 . p. 1301–06.

  18. Lin L, Xu Y, Liang X, Lai J. Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 2014;23(7):3191–202.

    Article  PubMed  Google Scholar 

  19. Nanni L, Brahnam S, Lumini A. Local ternary patterns from three orthogonal planes for human action classification. Expert Syst Appl 2011;38(5):5125–28.

    Article  Google Scholar 

  20. Noh S, Jeon M. A new framework for background subtraction using multiple cues. Asian conference on computer vision. Berlin, Heidelberg: Springer; 2012. p. 493–506.

  21. Ojala T, Pietikäinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recognit 1996;29(1):51–9.

    Article  Google Scholar 

  22. Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002;24(7):971–87.

    Article  Google Scholar 

  23. Pan J, Li X, Li X, Pang Y. Incrementally detecting moving objects in video with sparsity and connectivity. Cognitive Computation 2016;8(3):420–8.

    Article  Google Scholar 

  24. Sheikh Y, Shah M. Bayesian modeling of dynamic scenes for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005;27(11):1778–92.

    Article  PubMed  Google Scholar 

  25. Sobral A, Vacavant A. A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput Vis Image Underst 2014;122:4–21.

    Article  Google Scholar 

  26. Spampinato C, Palazzo S, Kavasidis I. A texton-based kernel density estimation approach for background modeling under extreme conditions. Comput Vis Image Underst 2014;122:74–83.

    Article  Google Scholar 

  27. St-Charles PL, Bilodeau GA, Bergevin R. Flexible background subtraction with self-balanced local sensitivity. Proceedings of the IEEE conference on computer vision and pattern recognition workshops; 2014. p. 408–13.

  28. Stauffer C, Grimson WEL. Adaptive background mixture models for real-time tracking. IEEE computer society conference on computer vision and pattern recognition, 1999. vol 2, IEEE; 1999. p. 246–52.

  29. Tan X, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 2010;19(6):1635–50.

    Article  PubMed  Google Scholar 

  30. Tu Z, Abel A, Zhang L, Luo B, Hussain A. A new spatio-temporal saliency-based video object segmentation. Cognitive Computation 2016;8(4):629–47.

    Article  Google Scholar 

  31. Tu Z, Zheng A, Yang E, Luo B, Hussain A. A biologically inspired vision-based approach for detecting multiple moving objects in complex outdoor scenes. Cognitive Computation 2015;7(5):539–51.

    Article  Google Scholar 

  32. Vasamsetti S, Setia S, Mittal N, Sardana HK, Babbar G. 2018. Automatic underwater moving object detection using multi-feature integration framework in complex backgrounds. IET Computer Vision.

  33. Wang Z, Liao K, Xiong J, Zhang Q. Moving object detection based on temporal information. IEEE Signal Processing Letters 2014;21(11):1403–07.

    Article  Google Scholar 

  34. Wren CR, Azarbayejani A, Darrell T, Pentland AP. Pfinder: Real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997;19(7):780–85.

    Article  Google Scholar 

  35. Yao J, Odobez JM. Multi-layer background subtraction based on color and texture. IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE; 2007. p. 1–8.

  36. Yubing T, Cheikh FA, Guraya FFE, Konik H, Trémeau A. A spatiotemporal saliency model for video surveillance. Cognitive Computation 2011;3(1):241–63.

    Article  Google Scholar 

  37. Zhang S, Li N, Cheng X, Wu Z. Adaptive object detection by implicit sub-class sharing features. Signal Proc 2013;93(6):1458–70.

    Article  Google Scholar 

  38. Zhang Y, Wang X, Qu B. Three-frame difference algorithm research based on mathematical morphology. Procedia Engineering 2012;29:2705–9.

    Article  Google Scholar 

  39. Zheng A, Xu M, Luo B, Zhou Z, Li C. Class: collaborative low-rank and sparse separation for moving object detection. Cognitive Computation 2017;9(2):180–93.

    Article  Google Scholar 

  40. Zivkovic Z, Van Der Heijden F. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn Lett 2006;27(7):773–80.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to express their sincere thanks to the funding agency Council for Scientific and Industrial Research, India under Network Project (ESC0113) for supporting this work. The authors are grateful to Dr. Maia Hoeberechts and team for providing the Ocean Networks Canada Dataset.

Funding

This study was funded by Council for Scientific and Industrial Research, India under Network Project (grant/project number: ESC0113).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Srikanth Vasamsetti.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent is not necessary for the present study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vasamsetti, S., Mittal, N., Neelapu, B.C. et al. 3D Local Spatio-temporal Ternary Patterns for Moving Object Detection in Complex Scenes. Cogn Comput 11, 18–30 (2019). https://doi.org/10.1007/s12559-018-9594-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-018-9594-5

Keywords

Navigation