Skip to main content
SpringerLink
Log in

(MS)2EDNet: Multiscale Motion Saliency Deep Network for Moving Object Detection

  • Conference paper
  • First Online:
Computer Vision and Image Processing (CVIP 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1568))

Included in the following conference series:

  • 825 Accesses

Abstract

Foreground segmentation in videos is a perplexing task. Infrequent motion of objects, illumination, shadow, camouflage, etc. are major factors which degrades the quality of segmentation. Usage of visual features like color, texture or shape, deficiencies the acquaintance of semantic evidence for foreground segmentation. In this paper, a novel compact multiscale motion saliency encoder-decoder learning network, ((MS)2EDNet) is presented for moving object detection (MOD). Initially, the lengthy streaming video is split into several small video streams (SVS). The background for each SVS is estimated using proposed network. Further, the saliency map is estimated via the input frames and estimated background for each SVS. Further, a compact multiscale encoder–decoder network (MSEDNet) is presented to extract the multiscale foregrounds from saliency maps. The extracted multiscale foregrounds are integrated to estimate the final foreground of the video frame. The effectiveness of the proposed (MS)2EDNet is estimated on three standard datasets (CDnet-2014 [1], and Wallflower [3]) for MOD. The compactness of the (MS)2EDNet is analyzed based on computational complexity and compared with the present approaches. Experimental study shows that proposed network outpaces the present state-of-the-art approaches on three standard datasets for MOD in terms of both detection accuracy and computational complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Y. Wang, P. M. Jodoin, F. Porikli, J. Konrad, Y. Benezeth, and P. Ishwar.: An expanded change detection benchmark dataset. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 393–400 (2014)

    Google Scholar 

  2. Toyama, K., Krumm, J., Brumitt, B., Meyers, B.: Wallflower: principles and practice of background maintenance. In: Proceedings of the IEEE International Conference on Computer Vision, vol. 1(1), pp. 255–261 (1999)

    Google Scholar 

  3. Gupta, A.K., Seal, A., Prasad, M., Khanna, P.: Salient object detection techniques in computer vision—a survey. Entropy 22(10), 1174 (2020)

    Article  MathSciNet  Google Scholar 

  4. Wu, Z., Dahua Lin, X.T.: Adjustable bounded rectifiers: towards deep binary representations. arXiv Prepr. arXiv:1511.06201, pp. 1–11 (2015)

  5. Lin, H., Member, S., Liu, T., Chuang, J., Member, S.: Learning a scene background model via classification. IEEE Trans. Signal Process 57(5), 1641–1654 (2009)

    Google Scholar 

  6. Agarwala, A., et al.: Interactive digital photomontage. ACM SIGGRAPH 2004 Pap. - SIGGRAPH ’04 1(212), 294 (2004)

    Google Scholar 

  7. Xu, X., Huang, T.S.: A loopy belief propagation approach for robust background estimation. In: 26th IEEE Conference on Computer Vision and Pattern Recognition. CVPR (2008)

    Google Scholar 

  8. Liang, C.W., Juang, C.F.: Moving object classification using a combination of static appearance features and spatial and temporal entropy values of optical flows. IEEE Trans. Intell. Transpor. Syst. 16(6), 3453–3464 (2015)

    Google Scholar 

  9. Jiang, S., Lu, X.: WeSamBE: a weight-sample-based method for background subtraction. IEEE Trans. Circu. Syst. Video Technol. 8215, 1–10 (2017)

    Google Scholar 

  10. Xi, T., Zhao, W., Wang, H., Lin, W.: Salient object detection with spatiotemporal background priors for video. IEEE Trans. Image Process. 26(7), 3425–3436 (2017)

    Google Scholar 

  11. Chen, Y., Wang, J., Zhu, B., Tang, M., Lu, H.: Pixel-wise deep sequence learning for moving object detection. IEEE Trans. Circuits Syst. Video Technol. 8215, 1–13 (2017)

    Google Scholar 

  12. Yeh, C., Member, S., Lin, C., Muchtar, K., Lai, H., Motivation, A.: Three-pronged compensation and hysteresis thresholding for moving object detection in real-time video surveillance. IEEE Trans. Ind. Electron. 64(6), 4945–4955 (2017)

    Google Scholar 

  13. Liao, S., Zhao, G., Kellokumpu, V., Pietikäinen, M., Li, S.Z.: Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1301–1306 (2010)

    Google Scholar 

  14. Wang, R., Bunyak, F., Seetharaman, G., Palaniappan, K.: Static and moving object detection using flux tensor with split gaussian models. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 420–424 (2014)

    Google Scholar 

  15. St-Charles, P.-L., Bilodeau, G.-A., Bergevin, R.: SuBSENSE: a universal change detection method with local adaptive sensitivity. IEEE Trans. Image Process. 24(1), 359–373 (2015)

    Google Scholar 

  16. Bianco, S., Ciocca, G., Schettini, R.: Combination of video change detection algorithms by genetic programming. IEEE Trans. Evol. Comput. 21(6), 914–928 (2017)

    Google Scholar 

  17. Romero, J.D., Lado, M.J., Mendez, A.J.: A background modeling and foreground detection algorithm using scaling coefficients defined with a color model called lightness-red-green-blue. IEEE Trans. Image Process. 27(3), 1243–1258 (2017)

    Google Scholar 

  18. Lin, Y., Tong, Y., Cao, Y., Zhou, Y., Wang, S.: Visual-attention-based background modeling for detecting infrequently moving objects. IEEE Trans. Circuits Syst. Video Technol. 27(6), 1208–1221 (2017)

    Google Scholar 

  19. Aytekin, C., Possegger, H., Mauthner, T., Kiranyaz, S., Bischof, H., Gabbouj, M.: Spatiotemporal saliency estimation by spectral foreground detection. IEEE Trans. Multim. 20(1), 82–95 (2018)

    Google Scholar 

  20. Pang, Y., Member, S., Ye, L., Li, X., Pan, J.: Incremental learning with saliency map for moving object detection. IEEE Trans. Circu. Sys. Video Technol. (TCSVT) 1, 1–12 (2016)

    Google Scholar 

  21. Braham, M., Van Droogenbroeck, M.: Deep background subtraction with scene-specific convolutional neural networks. In: International Conference on Systems, Signals, and Image Processing, pp. 1–4 (2016)

    Google Scholar 

  22. Wang, W., Shen, J., Shao, L.: Video salient object detection via fully convolutional networks. IEEE Trans. Image Process. 27(1), 38–49 (2018)

    Google Scholar 

  23. Yang, L., Li, J., Member, S., Luo, Y., Member, S.: Deep background modeling using fully convolutional network. IEEE Trans. Intell. Transp. Syst. 19(1), 254–262 (2018)

    Google Scholar 

  24. Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv Prepr. arXiv:1409.1556, pp. 1–14 (2014)

  25. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)

    Google Scholar 

  26. Roy, S.M., Ghosh, A.: Real-time adaptive histogram min-max bucket (HMMB) model for background subtraction. IEEE Trans. Circuits Syst. Video Technol. 8215(c), 1–1 (2017)

    Google Scholar 

  27. Cai, B., Xu, X., Jia, K., Qing, C.: DehazeNet : an end-to-end system for single image haze removal. IEEE Trans. Image Proce.ss 25(11), 1–13 (2016)

    Google Scholar 

  28. Wu, Z., Lin, D., Tang, X.: Adjustable bounded rectifiers: towards deep binary representations. arXiv Prepr. arXiv1511.06201, pp. 1–11 (2015)

  29. Szegedy, C., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 07–12–June, pp. 1–9 (2015)

    Google Scholar 

  30. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  31. St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: A self-adjusting approach to change detection based on background word consensus. In: IEEE Winter Conference on Applications of Computer Vision, WACV 2015, pp. 990–997 (2015)

    Google Scholar 

  32. Babaee, M., Dinh, D.T., Rigoll, G.: A Deep Convolutional Neural Network for Background Subtraction. arXiv preprint arXiv:1702.01731 (2017)

  33. Zivkovic, Z., Van Der Heijden, F.: Efficient adaptive density estimation per image pixel for the task of background subtraction. Pattern Recognit. Lett. 27(7), 773–780 (2006)

    Google Scholar 

  34. Barnich, O., Van Droogenbroeck, M.: ViBe : a universal background subtraction algorithm for video sequences. IEEE Trans. Image Process. 20(6), 1709–1724 (2011)

    Google Scholar 

  35. Schindler, K., Wang, H.: Smooth foreground-background segmentation for video processing. In: Asian Conference on Computer Visio, pp. 581–590 (2006)

    Google Scholar 

  36. Pan, J., Li, X., Li, X., Pang, Y.: Incrementally detecting moving objects in video with sparsity and connectivity. Cognit. Comput. 8(3), 420–428 (2016)

    Google Scholar 

  37. He, J., Balzano, L., Szlam, A.: Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1568–1575 (2012)

    Google Scholar 

  38. Guo, X., Wang, X., Yang, L., Cao, X., Ma, Y.: Robust foreground detection using smoothness and arbitrariness constraints. In: European Conference on Computer Vision, pp. 535–550 (2014)

    Google Scholar 

  39. Dikmen, M., Huang, T.S.: Robust estimation of foreground in surveillance videos by sparse error estimation. In: 19th International Conference on Pattern Recognition, pp. 1–4 (2008)

    Google Scholar 

  40. Sheikh, Y., Shah, M.: Bayesian modeling of dynamic scenes for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 27(11), 1778–1792 (2005)

    Google Scholar 

  41. Kim, K., Chalidabhongse, T.H., Harwood, D., Davis, L.: Real-time foreground-background segmentation using codebook model. Real-Time Imaging 11(3), 172–185 (2005)

    Google Scholar 

  42. Hofmann, M., Tiefenbacher, P., Rigoll, G.: Background segmentation with feedback: the pixel-based adaptive segmenter. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Work, pp. 38–43 (2012)

    Google Scholar 

  43. Yang, L., Cheng, H., Su, J., Li, X.: Pixel-to-model distance for robust background reconstruction. IEEE Trans. Circuits Syst. Video Technol. 26(5), 903–916 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of E&TC, PimpriChinchwad College of Engineering & Research, Ravet, Pune, India

    Santosh Nagnath Randive, Kishor B. Bhangale, Rahul G. Mapari, Kiran M. Napte & Kishor B. Wane

Authors
  1. Santosh Nagnath Randive
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kishor B. Bhangale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rahul G. Mapari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kiran M. Napte
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kishor B. Wane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Santosh Nagnath Randive .

Editor information

Editors and Affiliations

  1. Indian Institute of Technology Roorkee, Roorkee, India

    Balasubramanian Raman

  2. Indian Institute of Technology Ropar, Ropar, India

    Subrahmanyam Murala

  3. Jadavpur University, Kolkata, India

    Ananda Chowdhury

  4. Indian Institute of Technology Ropar, Ropar, India

    Abhinav Dhall

  5. Indian Institute of Technology Ropar, Ropar, India

    Puneet Goyal

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Randive, S.N., Bhangale, K.B., Mapari, R.G., Napte, K.M., Wane, K.B. (2022). (MS)2EDNet: Multiscale Motion Saliency Deep Network for Moving Object Detection. In: Raman, B., Murala, S., Chowdhury, A., Dhall, A., Goyal, P. (eds) Computer Vision and Image Processing. CVIP 2021. Communications in Computer and Information Science, vol 1568. Springer, Cham. https://doi.org/10.1007/978-3-031-11349-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-11349-9_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-11348-2

  • Online ISBN: 978-3-031-11349-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation