Skip to main content
SpringerLink
Log in

Fire detection in video surveillance using superpixel-based region proposal and ESE-ShuffleNet

  • 1186: Pattern Recognition and Artificial Intelligence based Multimedia Systems and Applications
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a forest fire detection framework using superpixel-based suspicious fire region proposal and light-weight convolutional neural network. The proposed methodology contains two main steps. In suspicious fire region proposal, we introduce a novel superpixel algorithm (SCMM) driven by Cauchy mixture model. Then, the negative Under-segmentation Error (UE) of each superpixel is applied to inter-frame comparison for predicting varying superpixels. After that, by computing the features of motion superpixels using Local Difference Binary (LDB) descriptor for two adjacent frames, the suspicious fire regions are localized. In following fire identification, to improve network performance while reducing computational complexity, this study presents a light-weight network architecture, called Expanded Squeeze-and-Excitation ShuffleNet (ESE-ShuffleNet). All suspicious fire regions are sent into this network to identify as either fire or non-fire included. Experiments show that our framework performs well on fire detection tasks. Code is available at http://www.imagetech-polynomials.com/ESE-ShuffleNet.html.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. https://www.microsoft.com/en-us/research/project/image-understanding/

  2. http://cfdb.univ-corse.fr/

  3. https://github.com/UIA-CAIR/Fire-Detection-Image-Dataset

  4. https://collections.durham.ac.uk/files/r2d217qp536#.Xwl8g0UzaUn

  5. http://ivrl.epflfl.ch/research/superpixels

  6. https://github.com/shenjianbing/lrw14

  7. http://cmm.ensmp.fr/~machairas/waterpixels.html

  8. http://jschenthu.weebly.com/projects.html

  9. https://github.com/ahban/GMMSP

  10. https://github.com/YuejiaoGong/CAS

  11. https://github.com/wjc852456/pytorch-mobilenet-v1

  12. https://github.com/tensorflow/models/tree/master/research/slim/nets/mobilenet

  13. https://github.com/kuan-wang/pytorch-mobilenet-v3

  14. https://github.com/AlloNighthawk/ShuffleNet−v1

  15. https://github.com/ericsun99/Shufflenet-v2-Pytorch

  16. https://github.com/DeepScale/SqueezeNet

  17. https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet

  18. https://github.com/huawei-noah/CV-Backbones

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  2. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  3. Ban Z, Liu J, Cao L (2018) Superpixel segmentation using Gaussian mixture model. IEEE Trans Image Process 27(8):4105–4117

    Article  MathSciNet  MATH  Google Scholar 

  4. Barmpoutis P, Dimitropoulos K, Kaza K, Grammalidis N (2019) Fire detection from images using faster R-CNN and multidimensional texture analysis. In Proceedings of IEEE Int Conf Acoustics Speech Signal Process (ICASSP), pp. 8301–8305

  5. Bochkovskiy A, Wang CY, Liao HYM (2020) YOLO v4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  6. Borges PVK, Izquierdo E (2010) A probabilistic approach for vision-based fire detection in videos. IEEE Trans Circuits Syst Video Technol 20(5):721–731

    Article  Google Scholar 

  7. CA GS, Bhowmik N, Breckon TP (2019) Experimental exploration of compact convolutional neural network architectures for non-temporal real-time fire detection. In Proceedings of IEEE Int Conf Mach Learn Appl (ICMLA), pp. 653–658

  8. Chen J, Li Z, Huang B (2017) Linear spectral clustering superpixel. IEEE Trans Image Process 26(7):3317–3330

    Article  MathSciNet  MATH  Google Scholar 

  9. Dunnings AJ, Breckon TP (2018) Experimentally defined convolutional neural network architecture variants for non-temporal real-time fire detection. In Proceedings of IEEE Int Conf Image Process (ICIP), pp. 1558–1562

  10. Foggia P, Saggese A, Vento M (2015) Real-time fire detection for video-surveillance applications using a combination of experts based on color, shape, and motion. IEEE Trans Circuits Syst Video Technol 25(9):1545–1556

    Article  Google Scholar 

  11. Han K, Wang Y, Tian Q, Guo J, Xu C, Xu C (2020) GhostNet: More features from cheap operations. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 1580–1589

  12. Hashemzadeh M, Zademehdi A (2019) Fire detection for video surveillance applications using ICA k-medoids-based color model and efficient spatio-temporal visual features. Expert Syst Appl 130:60–78

    Article  Google Scholar 

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 770–778

  14. Howard A, Sandler M, Chu G, Chen LC, Chen B, Tan M, Wang W, Zhu Y, Pang R, Vasudevan V et al (2019) Searching for MobileNetV3. In Proceedings of IEEE Int Conf Comput Vis (ICCV), pp. 1314–1324

  15. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) MobileNets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861

  16. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 7132–7141

  17. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 4700–4708

  18. Huang H, Kuang P, Fan L, Shi H (2020) An improved multi-scale fire detection method based on convolutional neural network. In Proceedings of Int Comput Conf Wavelet Active Med Tech Inf Process (ICCWAMTIP), pp. 109–112

  19. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv preprint, arXiv:1602.07360

  20. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In Proceedings of Adv Neural Inf Process Syst (NIPS), pp. 1097–1105

  21. Kuczma M (2009) An introduction to the theory of functional equations and inequalities: Cauchy’s equation and Jensen’s inequality. Springer Science & Business Media

  22. Li P, Zhao W (2020) Image fire detection algorithms based on convolutional neural networks. Case Stud Therm Eng 19

    Article  Google Scholar 

  23. Ma N, Zhang X, Zheng H, Sun J (2018) ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In Proceedings of European Conf Comput Vision (ECCV), pp. 116–131

  24. Machairas V, Faessel M, Cárdenas-Peña D, Chabardes T, Walter T, Decencière E (2015) Waterpixels. IEEE Trans Image Process 24(11):3707–3716

    Article  MathSciNet  MATH  Google Scholar 

  25. Matukhina O, Amaeva L, Merzlyakov S (2020) Fire detection system with utilization of industrial video surveillance system. In Proceedings of Int Multi-Conf Ind Eng Modern Tech (FarEastCon), pp. 1–5

  26. Muhammad K, Ahmad J, Baik SW (2018) Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288:30–42

    Article  Google Scholar 

  27. Muhammad K, Ahmad J, Lv Z, Bellavista P, Yang P, Baik SW (2018) Efficient deep CNN-based fire detection and localization in video surveillance applications. IEEE Trans Syst Man Cybern A Syst Humans 49(7):1419–1434

  28. Muhammad K, Ahmad J, Mehmood I, Rho S, Baik SW (2018) Convolutional neural networks based fire detection in surveillance videos. IEEE Access 6:18174–18183

    Article  Google Scholar 

  29. Muhammad K, Khan S, Elhoseny M, Ahmed SH, Baik SW (2019) Efficient fire detection for uncertain surveillance environment. IEEE Trans Ind Inform 15(5):3113–3122

    Article  Google Scholar 

  30. Nguyen TM, Wu QJ (2012) Fast and robust spatially constrained Gaussian mixture model for image segmentation. IEEE Trans Circuits Syst Video Technol 23(4):621–635

    Article  Google Scholar 

  31. Redmon J, Farhadi A (2018) YOLO v3: An incremental improvement. arXiv preprint arXiv:1804.02767

  32. Ren S, He K, Girshick R, Sun J (2017) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 6:1137–1149

    Article  Google Scholar 

  33. Ren X, Malik J (2003) Learning a classification model for segmentation. In Proceedings of IEEE Int Conf Comput Vis (ICCV), pp. 10–17

  34. Saeed F, Paul A, Hong WH, Seo H (2020) Machine learning based approach for multimedia surveillance during fire emergencies. Multimed Tools Appl 79:16201–16217

    Article  Google Scholar 

  35. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2018) MobileNetV2: Inverted residuals and linear bottlenecks. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 4510–4520

  36. Saponara S, Elhanashi A, Gagliardi A (2020) Exploiting R-CNN for video smoke/fire sensing in antifire surveillance indoor and outdoor systems for smart cities. In Proceedings of IEEE Int Conf Smart Comput (SMARTCOMP), pp. 392–397

  37. Shen J, Du Y, Wang W, Li X (2014) Lazy random walks for superpixel segmentation. IEEE Trans Image Process 23(4):1451–1462

    Article  MathSciNet  MATH  Google Scholar 

  38. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, Inception-ResNet and the impact of residual connections on learning. In Proceedings of Thirty-first AAAI Conf Artif Intell (AAAI-17)

  39. Tan M, Chen B, Pang R, Vasudevan V, Sandler M, Howard A, Le QV (2019) MnasNet: Platform-aware neural architecture search for mobile. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 2820–2828

  40. Xiao X, Zhou Y, Gong Y (2018) Content-adaptive superpixel segmentation. IEEE Trans Image Process 27(6):2883–2896

    Article  MathSciNet  MATH  Google Scholar 

  41. Xie X, Xie G, Xu X, Cui L (2019) Adaptive high-precision superpixel segmentation. Multimed Tools Appl 78:12353–12371

    Article  Google Scholar 

  42. Yang TJ, Howard A, Chen B, Zhang X, Go A, Sandler M, Sze V, Adam H (2018) Netadapt: Platform-aware neural network adaptation for mobile applications. In Proceedings of European Conf Comput Vision (ECCV), pp. 285–300

  43. Yang X, Cheng KT (2012) LDB: An ultra-fast feature for scalable augmented reality on mobile devices. In Proceedings of IEEE Int Sym Mixed Augment Reality (ISMAR), pp. 49–57

  44. Zhang X, Qian K, Jing K, Yang J, Yu H (2020) Fire detection based on convolutional neural networks with channel attention. In Proceedings of Chinese Autom Congr (CAC), pp. 3080–3085

  45. Zhang X, Zhou X, Lin M, Sun J (2018) ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of IEEE Conf Comput Vis Pattern Recognit (CVPR), pp. 6848–6856

Download references

Acknowledgements

This work was supported by the National Nature Science Foundation of China under Grant 61872143.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China

    Pengyu Wang, Jianmei Zhang & Hongqing Zhu

Authors
  1. Pengyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianmei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongqing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongqing Zhu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, P., Zhang, J. & Zhu, H. Fire detection in video surveillance using superpixel-based region proposal and ESE-ShuffleNet. Multimed Tools Appl 82, 13045–13072 (2023). https://doi.org/10.1007/s11042-021-11261-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11261-9

Keywords

Search

Navigation