Skip to main content
SpringerLink
Log in

A motion and lightness saliency approach for forest smoke segmentation and detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Most video-based detection systems rely on extracting visual features directly from divided original frames or detected motion regions achieved using conventional background subtraction. However, these approaches are not effective for smoke detection because conventional motion detection methods are sensitive to non-salient motion regions like waves, shaking tree leaves, camera jitter an so on. This tiny motion regions can be easily misclassified as smoke. Second, in the case of light smoke the background is visible with or without motion detection, such that it will deteriorate the feature quality. These regions usually occur far from smoke centers and present unstable and non-salient characteristics. This paper proposes an approach to separate the smoke region based on motion and lightness saliency detection. A low-rank and structured sparse decomposition method is used to extract the foreground regions. Saliency of smoke is then computed for further separation. These aforementioned measures ensure a reliable smoke component extraction. We propose a saliency measurement for group-sparse robust orthonormal subspace and learning (ROSL) in virtue of adaptive parameters. Experiments on challenging data sets demonstrate that the proposed method can work well on a wide range of smoke videos and give better smoke detection results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://signal.ee.bilkent.edu.tr/VisiFire/Demo/SmokeClips

References

  1. Barnich O, Van Droogenbroeck M (2009) Vibe: a powerful random technique to estimate the background in video sequences. In: ICASSP 2009. IEEE International Conference on Acoustics, Speech and Signal Processing, 2009, pp 945–948

  2. Barnich O, Van Droogenbroeck M (2011) Vibe: a universal background subtraction algorithm for video sequences. IEEE Trans Image Process 20(6):1709–1724

    Article  MathSciNet  Google Scholar 

  3. Bengio S, Pereira F, Singer Y, Strelow D (2009) Group sparse coding. In: Advances in Neural Information Processing Systems, pp 82–89

  4. Cai M, Lu X, Wu X, Feng Y (2016) Intelligent video analysis-based forest fires smoke detection algorithms. In: 2016 12th international conference on natural computation, fuzzy systems and knowledge discovery (ICNC-FSKD), IEEE, pp 1504–1508

  5. Calderara S, Piccinini P, Cucchiara R (2011) Vision based smoke detection system using image energy and color information. Mach Vis Appl 22(4):705–719

    Article  Google Scholar 

  6. Çetin AE, Dimitropoulos K, Gouverneur B, Grammalidis N, Günay O, Habibolu YH, Töreyin BU, Verstockt S (2013) Video fire detection–review. Digital Signal Process 23(6):1827–1843

    Article  Google Scholar 

  7. Chen TH, Kao CL, Chang SM (2003) An intelligent real-time fire-detection method based on video processing. In: Proceedings. IEEE 37th Annual 2003 International Carnahan Conference on Security Technology, 2003, IEEE, pp 104–111

  8. Chen TH, Yin YH, Huang SF, Ye YT (2006) The smoke detection for early fire-alarming system base on video processing. In: IIH-MSP’06. International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2006, IEEE, pp 427–430

  9. Chunyu Y, Jun F, Jinjun W, Yongming Z (2010) Video fire smoke detection using motion and color features. Fire Technol 46(3):651–663

    Article  Google Scholar 

  10. Cui X, Liu Q, Metaxas D (2009) Temporal spectral residual: fast motion saliency detection. In: Proceedings of the 17th ACM International Conference on Multimedia, ACM, pp 617–620

  11. Dimitropoulos K, Barmpoutis P, Grammalidis N (2017) Higher order linear dynamical systems for smoke detection in video surveillance applications. IEEE Trans Circuits Syst Video Technol 27(5):1143–1154

    Article  Google Scholar 

  12. Evangelio RH, Ptzold M, Sikora T (2012) Splitting gaussians in mixture models. In: 2012 IEEE Ninth international conference on advanced video and signal-based surveillance, pp 300–305

  13. Fang Y, Lin W, Chen Z, Tsai CM, Lin CW (2014) A video saliency detection model in compressed domain. IEEE Trans Circuits Syst Video Technol 24 (1):27–38

    Article  Google Scholar 

  14. Ganesh A, Wright J, Li X, Cands EJ, Ma Y (2010) Dense error correction for low-rank matrices via principal component pursuit. In: 2010 IEEE International symposium on information theory, pp 1513–1517

  15. Gao Z, Cheong LF, Shan M (2012) Block-sparse rpca for consistent foreground detection. In: European Conference on Computer Vision, Springer, pp 690–703

  16. Gao Z, Cheong LF, Wang YX (2014) Block-sparse rpca for salient motion detection. IEEE Trans Pattern Anal Mach Intell 36(10):1975–1987

    Article  Google Scholar 

  17. Gopalakrishnan V, Rajan D, Hu Y (2012) A linear dynamical system framework for salient motion detection. IEEE Trans Circuits Syst Video Technol 22(5):683–692

    Article  Google Scholar 

  18. Haines TSF, Xiang T (2014) Background subtraction with dirichletprocess mixture models. IEEE Trans Pattern Anal Mach Intell 36(4):670–683

    Article  Google Scholar 

  19. Ham S, Ko BC, Nam JY (2011) Vision based forest smoke detection using analyzing of temporal patterns of smoke and their probability models. In: Image Processing: Machine Vision Applications IV, vol 7877, International Society for Optics and Photonics, p 78770A

  20. Huang CR, Chang YJ, Yang ZX, Lin YY (2014) Video saliency map detection by dominant camera motion removal. IEEE Trans Circuits Syst Video Technol 24(8):1336–1349

    Article  Google Scholar 

  21. Jia Y, Yuan J, Wang J, Fang J, Zhang Q, Zhang Y (2016) A saliency-based method for early smoke detection in video sequences. Fire Technol 52 (5):1271–1292

    Article  Google Scholar 

  22. Kim W, Kim C (2014) Spatiotemporal saliency detection using textural contrast and its applications. IEEE Trans Circuits Syst Video Technol 24(4):646–659

    Article  Google Scholar 

  23. Ko B, Cheong KH, Nam JY (2010) Early fire detection algorithm based on irregular patterns of flames and hierarchical bayesian networks. Fire Saf J 45(4):262–270

    Article  Google Scholar 

  24. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.5055

  25. Liu X, Zhao G, Yao J, Qi C (2015) Background subtraction based on low-rank and structured sparse decomposition. IEEE Trans Image Process 24(8):2502–2514

    Article  MathSciNet  Google Scholar 

  26. Long C, Zhao J, Han S, Xiong L, Yuan Z, Huang J, Gao W (2010) Transmission: a new feature for computer vision based smoke detection. In: International Conference on Artificial Intelligence and Computational Intelligence, Springer, pp 389–396

  27. Mairal J, Jenatton R, Bach FR, Obozinski GR (2010) Network flow algorithms for structured sparsity. In: Advances in Neural Information Processing Systems, pp 1558–1566

  28. Park J, Ko B, Nam JY, Kwak S (2013) Wildfire smoke detection using spatiotemporal bag-of-features of smoke. In: 2013 IEEE Workshop on Applications of Computer Vision (WACV), IEEE, pp 200–205

  29. Shu X, Porikli F, Ahuja N (2014) Robust orthonormal subspace learning: Efficient recovery of corrupted low-rank matrices. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3874–3881

  30. Tian H, Li W, Ogunbona PO, Wang L (2018) Detection and separation of smoke from single image frames. IEEE Trans Image Process 27(3):1164–1177

    Article  MathSciNet  Google Scholar 

  31. Tian H, Li W, Wang L, Ogunbona P (2012) A novel video-based smoke detection method using image separation. In: 2012 IEEE International Conference on Multimedia and Expo (icme), IEEE, pp 532–537

  32. Tian H, Li W, Wang L, Ogunbona P (2014) Smoke detection in video: an image separation approach. Int J Comput Vis 106(2):192–209

    Article  Google Scholar 

  33. Toreyin BU, Dedeoglu Y, Cetin AE (2005) Flame detection in video using hidden markov models. In: ICIP 2005. IEEE International Conference on Image Processing, 2005, vol 2, IEEE, pp II–1230

  34. Töreyin B U, Dedeoġlu Y, Cetin AE (2005) Wavelet based real-time smoke detection in video. In: Signal Processing Conference, 2005 13th European, IEEE, pp 1–4

  35. Toreyin BU, Dedeoglu Y, Cetin AE (2006) Contour based smoke detection in video using wavelets. In: Signal Processing Conference, 2006 14th European, IEEE, pp 1–5

  36. Wang N, Yao T, Wang J, Yeung DY (2012) A probabilistic approach to robust matrix factorization. In: ECCV’12 Proceedings of the 12th European conference on Computer Vision - Volume Part VII, pp 126–139

    Chapter  Google Scholar 

  37. Wang S, He Y, Zou J, Duan B, Wang J (2014) A flame detection synthesis algorithm. Fire Technol 50(4):959–975

    Article  Google Scholar 

  38. Xiong Z, Caballero R, Wang H, Finn AM, Lelic MA, Peng PY (2007) Video-based smoke detection: possibilities, techniques, and challenges. In: IFPA, Fire Suppression and Detection Research and Applications—A Technical Working Conference (SUPDET), Orlando, FL

  39. Xue Y, Guo X, Cao X (2012) Motion saliency detection using low-rank and sparse decomposition. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp 1485–1488

  40. Ye W, Zhao J, Wang S, Wang Y, Zhang D, Yuan Z (2015) Dynamic texture based smoke detection using surfacelet transform and hmt model. Fire Saf J 73:91–101

    Article  Google Scholar 

  41. Yi X, Park D, Chen Y, Caramanis C (2016) Fast algorithms for robust pca via gradient descent. Neural Information Processing Systems pp 4152–4160

  42. Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recogn Lett 29(7):925–932

    Article  Google Scholar 

  43. Yuan F (2012) A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with adaboost for video smoke detection. Pattern Recogn 45(12):4326–4336

    Article  Google Scholar 

  44. Zhang Z, Shen T, Zou J (2014) An improved probabilistic approach for fire detection in videos. Fire Technol 50(3):745–752

    Article  Google Scholar 

  45. Zhao Y, Zhou Z, Xu M (2015) Forest fire smoke video detection using spatiotemporal and dynamic texture features. Can J Electr Comput Eng 2015:40

    Google Scholar 

  46. Zhou T, Tao D (2013) Shifted subspaces tracking on sparse outlier for motion segmentation. In: IJCAI ’13 Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence, pp 1946–1952

Download references

Acknowledgments

The authors would like to thank the editor and the anonymous reviewers for their valuable and helpful comments, as well as the important guiding significance to our researches.

This work was supported by the National Natural Science Foundation of China (No.61871123), Key Research and Development Program in Jiangsu Province (No.BE2016739) and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, China

    Xuehui Wu & Xiaobo Lu

  2. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing, 210096, China

    Xuehui Wu & Xiaobo Lu

  3. Department of Electrical and Computer Engineering, University of Calgary, 2500 University Dr N.W., Calgary, AB, T2N 1N4, Canada

    Henry Leung

Authors
  1. Xuehui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaobo Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Henry Leung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaobo Lu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Natural Science Foundation of China (No.61871123), Key Research and Development Program in Jiangsu Province (No.BE2016739) and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Lu, X. & Leung, H. A motion and lightness saliency approach for forest smoke segmentation and detection. Multimed Tools Appl 79, 69–88 (2020). https://doi.org/10.1007/s11042-019-08047-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08047-5

Keywords

Search

Navigation