Abstract
Humans possess natural cognitive vision to perceive objects in a 3D space and are able to differentiate foreground and background moving objects using their shape, colour and texture. Moving object detection is a leading and challenging task in complex scenes which involve illumination variation, blurriness, camouflage, moving background objects, etc. Inspired by human cognitive vision, a novel descriptor named 3D local spatio-temporal ternary patterns (3D-LStTP) is proposed for moving object detection. The 3D-LStTP collects multidirectional spatio-temporal information from three consecutive frames in a video by forming a 3D grid structure. The background models are constructed by using texture and colour features. The results obtained after modelling are integrated for foreground moving object detection in complex scenes. The performance of proposed algorithm is validated by conducting five experiments on Fish4Knowledge dataset, four experiments on I2R dataset and four experiments on Change Detection dataset. Qualitative and quantitative analyses are carried out on benchmark datasets. The results after investigation prove that the proposed method outperforms the state-of-the-art techniques for moving object detection in terms of ROC, TPR, FPR, Precision and F- measure.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Akilan T, Wu QJ, Yang Y. Fusion-based foreground enhancement for background subtraction using multivariate multi-model gaussian distribution. Inf Sci 2018;430:414–31.
Bouwmans T, El Baf F, Vachon B. Background modeling using mixture of gaussians for foreground detection-a survey. Recent Patents on Computer Science 2008;1(3):219–37.
El Baf F, Bouwmans T, Vachon B. Fuzzy integral for moving object detection. IEEE international conference on Fuzzy systems, 2008. FUZZ-IEEE 2008.(IEEE world congress on computational intelligence). IEEE; 2008. p. 1729–36.
Goyette N, Jodoin PM, Porikli F, Konrad J, Ishwar P. Changedetection. net: a new change detection benchmark dataset. 2012 IEEE computer society conference on computer vision and pattern recognition workshops (CVPRW). IEEE; 2012. p. 1–8.
Guo JM, Liu YF, Hsia CH, Shih MH, Hsu CS. Hierarchical method for foreground detection using codebook model. IEEE Transactions on Circuits and Systems for Video Technology 2011;21(6):804–15.
Han B, Davis LS. Density-based multifeature background subtraction with support vector machine. IEEE Transactions on Pattern Analysis and Machine Intelligence 2012;34(5):1017–23.
Hati KK, Sa PK, Majhi B. Intensity range based background subtraction for effective object detection. IEEE Signal Processing Letters 2013;20(8):759–62.
Heikkila M, Pietikainen M. A texture-based method for modeling the background and detecting moving objects. IEEE Transactions on Pattern Analysis and Machine Intelligence 2006;28(4):657–62.
Ji Z, Wang W. Detect foreground objects via adaptive fusing model in a hybrid feature space. Pattern Recogn 2014;47(9):2952–61.
KaewTraKulPong P, Bowden R. An improved adaptive background mixture model for real-time tracking with shadow detection. Video-based surveillance systems. Boston: Springer; 2002. p. 135–144.
Karasulu B, Korukoglu S. Moving object detection and tracking by using annealed background subtraction method in videos: performance optimization. Expert Syst Appl 2012;39(1):33–43.
Kavasidis I, Palazzo S, Di Salvo R, Giordano D, Spampinato C. An innovative web-based collaborative platform for video annotation. Multimedia Tools and Applications 2014;70(1):413–32.
Kim K, Chalidabhongse TH, Harwood D, Davis L. Real-time foreground–background segmentation using codebook model. Real-time imaging 2005;11(3):172–85.
Kim W, Kim Y. Background subtraction using illumination-invariant structural complexity. IEEE Signal Processing Letters 2016;23(5):634–8.
Lee DS. Effective gaussian mixture learning for video background subtraction. IEEE transactions on Pattern Analysis And Machine Intelligence 2005;27(5):827–32.
Li L, Huang W, Gu IY, Tian Q. Foreground object detection from videos containing complex background. Proceedings of the eleventh ACM international conference on multimedia. ACM; 2003. p. 2–10.
Liao S, Zhao G, Kellokumpu V, Pietikäinen M, Li SZ. Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. 2010 IEEE conference on computer vision and pattern recognition (CVPR). IEEE; 2010 . p. 1301–06.
Lin L, Xu Y, Liang X, Lai J. Complex background subtraction by pursuing dynamic spatio-temporal models. IEEE Trans Image Process 2014;23(7):3191–202.
Nanni L, Brahnam S, Lumini A. Local ternary patterns from three orthogonal planes for human action classification. Expert Syst Appl 2011;38(5):5125–28.
Noh S, Jeon M. A new framework for background subtraction using multiple cues. Asian conference on computer vision. Berlin, Heidelberg: Springer; 2012. p. 493–506.
Ojala T, Pietikäinen M, Harwood D. A comparative study of texture measures with classification based on featured distributions. Pattern Recognit 1996;29(1):51–9.
Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002;24(7):971–87.
Pan J, Li X, Li X, Pang Y. Incrementally detecting moving objects in video with sparsity and connectivity. Cognitive Computation 2016;8(3):420–8.
Sheikh Y, Shah M. Bayesian modeling of dynamic scenes for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005;27(11):1778–92.
Sobral A, Vacavant A. A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos. Comput Vis Image Underst 2014;122:4–21.
Spampinato C, Palazzo S, Kavasidis I. A texton-based kernel density estimation approach for background modeling under extreme conditions. Comput Vis Image Underst 2014;122:74–83.
St-Charles PL, Bilodeau GA, Bergevin R. Flexible background subtraction with self-balanced local sensitivity. Proceedings of the IEEE conference on computer vision and pattern recognition workshops; 2014. p. 408–13.
Stauffer C, Grimson WEL. Adaptive background mixture models for real-time tracking. IEEE computer society conference on computer vision and pattern recognition, 1999. vol 2, IEEE; 1999. p. 246–52.
Tan X, Triggs B. Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 2010;19(6):1635–50.
Tu Z, Abel A, Zhang L, Luo B, Hussain A. A new spatio-temporal saliency-based video object segmentation. Cognitive Computation 2016;8(4):629–47.
Tu Z, Zheng A, Yang E, Luo B, Hussain A. A biologically inspired vision-based approach for detecting multiple moving objects in complex outdoor scenes. Cognitive Computation 2015;7(5):539–51.
Vasamsetti S, Setia S, Mittal N, Sardana HK, Babbar G. 2018. Automatic underwater moving object detection using multi-feature integration framework in complex backgrounds. IET Computer Vision.
Wang Z, Liao K, Xiong J, Zhang Q. Moving object detection based on temporal information. IEEE Signal Processing Letters 2014;21(11):1403–07.
Wren CR, Azarbayejani A, Darrell T, Pentland AP. Pfinder: Real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997;19(7):780–85.
Yao J, Odobez JM. Multi-layer background subtraction based on color and texture. IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE; 2007. p. 1–8.
Yubing T, Cheikh FA, Guraya FFE, Konik H, Trémeau A. A spatiotemporal saliency model for video surveillance. Cognitive Computation 2011;3(1):241–63.
Zhang S, Li N, Cheng X, Wu Z. Adaptive object detection by implicit sub-class sharing features. Signal Proc 2013;93(6):1458–70.
Zhang Y, Wang X, Qu B. Three-frame difference algorithm research based on mathematical morphology. Procedia Engineering 2012;29:2705–9.
Zheng A, Xu M, Luo B, Zhou Z, Li C. Class: collaborative low-rank and sparse separation for moving object detection. Cognitive Computation 2017;9(2):180–93.
Zivkovic Z, Van Der Heijden F. Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recogn Lett 2006;27(7):773–80.
Acknowledgements
The authors would like to express their sincere thanks to the funding agency Council for Scientific and Industrial Research, India under Network Project (ESC0113) for supporting this work. The authors are grateful to Dr. Maia Hoeberechts and team for providing the Ocean Networks Canada Dataset.
Funding
This study was funded by Council for Scientific and Industrial Research, India under Network Project (grant/project number: ESC0113).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent is not necessary for the present study.
Rights and permissions
About this article
Cite this article
Vasamsetti, S., Mittal, N., Neelapu, B.C. et al. 3D Local Spatio-temporal Ternary Patterns for Moving Object Detection in Complex Scenes. Cogn Comput 11, 18–30 (2019). https://doi.org/10.1007/s12559-018-9594-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12559-018-9594-5