Skip to main content

Advertisement

SpringerLink
Log in

Patchwise dictionary learning for video forest fire smoke detection in wavelet domain

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Fire smoke detection in forest faces much more challenges than in local areas or indoors. Conventional fire smoke methods using static features extracted form single image directly cannot handle some disturbed factors exist in forest, such as shaking trees and fog/haze. On the other hand, it is difficult to capture the dynamic features of fire smoke while they are vital for fire smoke detection. This paper proposes an effective method of extracting features based on sequential images for smoke detection, quasi-dynamic features are extracted using on pixel-block arranged image sets. First, consecutive frames are rearranged into a new big image based on a pixel-block rule, so that the new image still keep the whole structure of original image on block level, and the change of different pixels in the same block, which comes from the same position of each frame, can reflect the dynamic features by special process. Second, to process the new arranged big image, driven by single image wavelet transform for denoting local signal characteristics, Dual Tree-Complex Wavelet Transform is utilized to decompose the new big image into eight images with the same size of the source image after several level-decomposition, including four directions (HH, HL, LH, LL) with two decomposed images in each direction, so that the changes from smoke in different orientation can be extracted and depicted in these subbands. Third, a multi-scale dictionary learning method is proposed to learn dictionaries of each group images. Furthermore, elastic net is introduced for sparse coding and feature coefficients extraction. All operations above contribute to a significant improvement for smoke detection in this paper. Extensive experiments are performed to validate the effectiveness of the proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. http://signal.ee.bilkent.edu.tr/VisiFire/Demo/SmokeClips.

References

  1. Adib M, Eckstein R, Hernandez-Sosa G (2018) SnO2 nanowire-based aerosol jet printed electronic nose as fire detector. IEEE Sens J 18(2):494–500

    Article  Google Scholar 

  2. Aharon M, Elad M, Bruckstein A (2006) K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54(11):4311–4322

    Article  Google Scholar 

  3. Bahrampour S, Nasrabadi NM, Ray A, Jenkins WK (2015) Multimodal task-driven dictionary learning for image classification. IEEE Trans Image Process 25(1):24–38

    Article  MathSciNet  Google Scholar 

  4. Bahrampour S, Nasrabadi NM, Ray A, Jenkins WK (2016) Multimodal task-driven dictionary learning for image classification. IEEE Trans Image Process 25(1):24–38

    Article  MathSciNet  Google Scholar 

  5. Bradley DM, Bagnell JA (2008) Differentiable sparse coding. In: International conference on neural information processing systems, pp 113–120

  6. Cai M, Lu X, Wu X, Feng Y (2016) Intelligent video analysis-based forest fires smoke detection algorithms. In: International conference on natural computation, fuzzy systems and knowledge discovery, pp 1504–1508

  7. Calderara S, Piccinini P, Cucchiara R (2011) Vision based smoke detection system using image energy and color information. Mach Vis Appl 22(4):705–719

    Article  Google Scholar 

  8. Cheon J, Lee J, Lee I, Chae Y, Yoo Y, Han G (2009) A single-chip CMOS smoke and temperature sensor for an intelligent fire detector. IEEE Sens J 9(8):914–921

    Article  Google Scholar 

  9. Chunyu Y, Jun F, Jinjun W, Yongming Z (2010) Video fire smoke detection using motion and color features. Fire Technol 46(3):651–663

    Article  Google Scholar 

  10. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE Computer Society conference on computer vision and pattern recognition, pp 886–893

  11. Efron B, Hastie T, Johnstone IM, Tibshirani R, Ishwaran H, Knight K, Loubes J, Massart P, Madigan D, Ridgeway G et al (2004) Least angle regression. Ann Stat 32(2):407–499

    Article  MathSciNet  Google Scholar 

  12. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    Article  MathSciNet  Google Scholar 

  13. Fonollosa J, Solórzano A, Marco S (2018) Chemical sensor systems and associated algorithms for fire detection: a review. Sensors 18(2):553

    Article  Google Scholar 

  14. Fu SW, Li PC, Lai YH, Yang CC, Hsieh LC, Yu T (2016) Joint dictionary learning-based non-negative matrix factorization for voice conversion to improve speech intelligibility after oral surgery. IEEE Trans Biomed Eng PP(99):1–1

    Google Scholar 

  15. Grosse R, Raina R, Kwong H, Ng AY (2012) Shift-invariance sparse coding for audio classification. Comput Sci 9:8

    Google Scholar 

  16. Gu S, Zuo W, Xie Q, Meng D, Feng X, Zhang L (2015) Convolutional sparse coding for image super-resolution. In: IEEE international conference on computer vision, pp 1823–1831

  17. Gunay O, Toreyin BU, Kose K, Cetin AE (2012) Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans Image Process 21(5):2853–2865

    Article  MathSciNet  Google Scholar 

  18. Hu A (2007) Forest fire and smoke detection based on video image segmentation. In: Mippr: pattern recognition & computer vision

  19. Jakovcevic T, Stipanicev D, Krstinic D (2013) Visual spatial-context based wildfire smoke sensor. Mach Vis Appl 24(4):707–719

    Article  Google Scholar 

  20. Jiang Z, Lin Z, Davis LS (2011) Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: IEEE conference on computer vision & pattern recognition

  21. Kavukcuoglu K, Ranzato MA, Fergus R, Lecun Y (2009) Learning invariant features through topographic filter maps. In: IEEE conference on computer vision and pattern recognition. CVPR 2009, pp 1605–1612

  22. Kingsbury N (2015) The dual-tree complex wavelet transform: a new efficient tool for image restoration and enhancement. In: 9th European Signal Processing Conference (EUSIPCO 1998)

  23. Ko B, Park J, Nam J (2013) Spatiotemporal bag-of-features for early wildfire smoke detection. Image Vis Comput 31(10):786–795

    Article  Google Scholar 

  24. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. Computer Science Department, University of Toronto, Tech. Rep 1

  25. Li M, Xie Q, Zhao Q, Wei W, Gu S, Tao J, Meng D (2018) Video rain streak removal by multiscale convolutional sparse coding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 6644–6653

  26. Mairal J, Bach F, Ponce J (2012) Task-driven dictionary learning. IEEE Trans Pattern Anal Mach Intell 34(4):791

    Article  Google Scholar 

  27. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2008) Discriminative learned dictionaries for local image analysis. In: IEEE conference on computer vision and pattern recognition. In: CVPR 2008, pp 1–8

  28. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2008) Supervised dictionary learning. In: HAL-INRIA, pp. 1–8 (2008)

  29. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2010) Non-local sparse models for image restoration. In: IEEE International conference on computer vision, pp 2272–2279

  30. Mairal J, Bach FR, Ponce J (2012) Task-driven dictionary learning. IEEE Trans Pattern Anal Mach Intell 34(4):791–804

    Article  Google Scholar 

  31. Mairal J, Bach FR, Ponce J, Sapiro G (2009) Online dictionary learning for sparse coding. In: International conference on machine learning, pp 689–696

  32. Mairal J, Elad M, Sapiro G (2007) Sparse representation for color image restoration. IEEE Trans Image Process 17(1):53–69

    Article  MathSciNet  Google Scholar 

  33. Morerio P, Marcenaro L, Regazzoni CS, Gera G (2013) Early fire and smoke detection based on colour features and motion analysis. In: IEEE international conference on image processing, pp 1041–1044

  34. Muhammad K, Ahmad J, Baik SW (2017) Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288:30–42

    Article  Google Scholar 

  35. Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987

    Article  Google Scholar 

  36. Ophir B, Lustig M, Elad M (2011) Multi-scale dictionary learning using wavelets. IEEE J Sel Top Signal Process 5(5):1014–1024

    Article  Google Scholar 

  37. Osborne MR, Presnell B, Turlach BA (2000) A new approach to variable selection in least squares problems. IMA J Numer Anal 20(3):389–403

    Article  MathSciNet  Google Scholar 

  38. Park J, Ko B, Nam J, Kwak SY (2013) Wildfire smoke detection using spatiotemporal bag-of-features of smoke. In: Workshop on applications of computer vision, pp 200–205

  39. Raina R, Battle A, Lee H, Packer B, Ng AY (2007) Self-taught learning: transfer learning from unlabeled data, pp 759–766

  40. Rubinstein R, Zibulevsky M, Elad M (2010) Double sparsity: learning sparse dictionaries for sparse signal approximation. IEEE Trans Signal Process 58(3):1553–1564

    Article  MathSciNet  Google Scholar 

  41. Shu X, Porikli F, Ahuja N (2014) Robust orthonormal subspace learning: efficient recovery of corrupted low-rank matrices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3874–3881

  42. Soliman H, Sudan K, Mishra A (2010) A smart forest-fire early detection sensory system: another approach of utilizing wireless sensor and neural networks. In: 2010 IEEE Sensors pp. 1900–1904

  43. Tian H, Li W, Ogunbona P, Wang L (2014) Single image smoke detection. In: Asian conference on computer vision, pp 87–101

  44. Tian H, Li W, Ogunbona PO, Wang L (2018) Detection and separation of smoke from single image frames. IEEE Trans Image Process 27(3):1164–1177

    Article  MathSciNet  Google Scholar 

  45. Tian H, Li W, Wang L, Ogunbona P (2012) A novel video-based smoke detection method using image separation. In: 2012 IEEE International Conference on Multimedia and Expo (ICME). IEEE, pp 532–537

  46. Tian H, Li W, Wang L, Ogunbona P (2014) Smoke detection in video: an image separation approach. Int J Comput Vis 106(2):192–209

    Article  Google Scholar 

  47. Töreyin BU, Dedeoğlu Y, Cetin AE (2005) Wavelet based real-time smoke detection in video. In: 2005 13th European Signal Processing Conference. IEEE, pp 1–4

  48. Toreyin BU, Dedeoglu Y, Cetin AE (2010) Wavelet based real-time smoke detection in video. In: 2005 European Signal Processing Conference, pp 1–4

  49. Tran D, Bourdev L, Fergus R, Torresani L, Paluri M (2015) Learning spatiotemporal features with 3D convolutional networks. In: IEEE international conference on computer vision

  50. Wu X, Lu X, Leung H (2017) An adaptive threshold deep learning method for fire and smoke detection. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC). IEEE, pp 1954–1959

  51. Wu X, Lu X, Leung H (2018) A video based fire smoke detection using robust adaboost. Sensors 18(11):3780

    Article  Google Scholar 

  52. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  53. Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: IEEE conference on computer vision and pattern recognition. CVPR 2009, pp 1794–1801

  54. Yang M, Zhang D, Feng X, Zhang D (2012) Fisher discrimination dictionary learning for sparse representation. In: IEEE international conference on computer vision

  55. Yeganli F, Nazzal M, Ozkaramanli H (2015) Image super-resolution via sparse representation over multiple learned dictionaries based on edge sharpness and gradient phase angle. SIViP 9(1):285–293

    Article  Google Scholar 

  56. Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett 29(7):925–932

    Article  Google Scholar 

  57. Yuan F (2012) A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with adaboost for video smoke detection. Pattern Recognit 45(12):4326–4336

    Article  Google Scholar 

  58. Zhang H, Wang S, Zhao M, Xu X, Ye Y (2018) Locality reconstruction models for book representation. IEEE Trans Knowl Data Eng 30(10):1873–1886

    Article  Google Scholar 

  59. Zhang Q, Li B (2010) Discriminative K-SVD for dictionary learning in face recognition. In: Computer vision & pattern recognition

  60. Zhang, Q, Xu J, Xu L, Guo H (2016) Deep convolutional neural networks for forest fire detection. In: Proceedings of the 2016 International Forum on Management, Education and Information Technology Application. Atlantis Press

  61. Zhao Y, Zhou Z, Xu M (2015) Forest fire smoke video detection using spatiotemporal and dynamic texture features. J Electr Comput Eng 2015:40

    Google Scholar 

  62. Zhu Z, Guo F, Yu H, Chen C (2014) Fast single image super-resolution via self-example learning and sparse representation. IEEE Trans Multimed 16(8):2178–2190

    Article  Google Scholar 

  63. Zibulevsky M, Pearlmutter BA (2014) Blind source separation by sparse decomposition in a signal dictionary. Neural Comput 13(4):863–882

    Article  Google Scholar 

  64. Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B Stat Methodol 67(2):301–320

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editor and the anonymous reviewers for their valuable and helpful comments, as well as the important guiding significance to our researches. This work was supported by the National Natural Science Foundation of China (No. 61871123), Key Research and Development Program in Jiangsu Province (No.BE2016739) and a Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions.

Author information

Authors and Affiliations

  1. School of Automation, Southeast University, Nanjing, 210096, China

    Xuehui Wu, Yichao Cao & Xiaobo Lu

  2. College of Artificial Intelligence, Nanjing Agricultural University, Nanjing, 210095, China

    Xuehui Wu

  3. Key Laboratory of Measurement and Control of Complex Systems of Engineering, Ministry of Education, Southeast University, Nanjing, 210096, China

    Xuehui Wu, Yichao Cao & Xiaobo Lu

  4. Department of Electrical and Computer Engineering, University of Calgary, 2500 University Dr N.W., Calgary, AB, T2N 1N4, Canada

    Henry Leung

Authors
  1. Xuehui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yichao Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaobo Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Henry Leung
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaobo Lu.

Ethics declarations

Conflict of interest

The authors declared that they have no conflict of interest to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, X., Cao, Y., Lu, X. et al. Patchwise dictionary learning for video forest fire smoke detection in wavelet domain. Neural Comput & Applic 33, 7965–7977 (2021). https://doi.org/10.1007/s00521-020-05541-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05541-y

Keywords

Search

Navigation