Skip to main content
SpringerLink
Log in

Real-time detection of flame and smoke using an improved YOLOv4 network

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Fire is one of the major disasters in the world, which seriously endangers the safety of life and property. Effective flame and smoke detection can provide timely warning information for firefighters. Existing flame and smoke detection algorithms are limited by processor performance and cannot operate large deep networks. This paper proposes a lightweight detector called Light-YOLOv4, which considers the balance between performance and efficiency. First, the backbone network CSPDarknet53 is replaced by MobileNetv3. Second, the path aggregation network is changed into bidirectional feature pyramid network (BiFPN) based on the idea of bidirectional cross-scale connections. Third, an efficient feature extraction module named depthwise separable attention module based on depthwise separable convolution and coordinate attention network is construed to replace the 3\(\times \)3 standard convolution of spatial pyramid pooling, BiFPN and YOLO head network. In comparison with YOLOv4, Light-YOLOv4 has only 19.1% of its trainable parameters while almost keeping the same accuracy. By combining multiple tricks, Light-YOLOv4 can achieve a better balance between performance (85.64% mAP) and efficiency (71 FPS), which meets flame and smoke detection tasks’ requirements on the accuracy and real time. Experiments on Nvidia Jetson TX2 further demonstrate that Light-YOLOv4 has good detection performance and speed on embedded scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Muhammad, K., Ahmad, J., Baik, S.W.: Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288, 30–42 (2018). https://doi.org/10.1016/j.neucom.2017.04.083

    Article  Google Scholar 

  2. Uǧur Töreyin, B., Dedeoǧlu, Y., Uǧur, G.A., Çetin, E.: Computer vision based method for real-time fire and flame detection. Signal Pattern Recognit. Lett. 27, 49–58 (2006). https://doi.org/10.1016/j.patrec.2005.06.015

    Article  Google Scholar 

  3. Gunay, O., Toreyin, B.U., Kivanc, K.A., Cetin, E.: Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans. Image Process. 21(5), 2853–2865 (2012). https://doi.org/10.1109/TIP.2012.2183141

    Article  MathSciNet  MATH  Google Scholar 

  4. Kuo, J.-Y., Lai, T.-Y., Fanjiang, Y.-Y., Huang, F.-C., Liao, Y.-H.: A behavior-based flame detection method for a real-time video surveillance system. J. Chin. Inst. Eng. 38, 947–958 (2015). https://doi.org/10.1080/02533839.2015.1047797

    Article  Google Scholar 

  5. Shamsoshoara, A., Afghaha, F., Razi, A., Zheng, L., Fulé, P.Z., Blasch, E.: Aerial imagery pile burn detection using deep learning: the FLAME dataset. Comput. Netw. (2021). https://doi.org/10.1016/j.comnet.2021.108001

    Article  Google Scholar 

  6. Wang, Y.E., Wei, G.-Y., Brooks D.: Benchmarking TPU, GPU, and CPU Platforms for Deep Learning. (2019) arXiv:1907.10701

  7. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: YOLOv4: Optimal Speed and Accuracy of Object Detection. (2020). arXiv:2004.10934

  8. Tan, M., Pang, R., Le, Q.V.: EfficientDet: scalable and efficient object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10778–10787 (2020). https://doi.org/10.1109/CVPR.2018.00913

  9. Hongyu, H., Ping, K., Li, F., Huaxin, S.: An improved multi-scale fire detection method based on convolutional neural network. In: 2020 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pp. 109–112 (2020). https://doi.org/10.1109/CVPR.2018.00913

  10. Chino, D.Y.T., Avalhais, L.P.S., Rodrigues Jr, J.F., Train, A.J.M.: BoWFire: detection of fire in still images by integrating pixel color and texture analysis. In: 2015 28th SIBGRAPI Conference on Graphics, Patterns and Images, 2015 28th SIBGRAPI Conference on Graphics, pp. 95–102 (2015). https://doi.org/10.1109/CVPR.2018.00913

  11. Wang, X., Li, Y., Li, Z.: Research on flame detection algorithm based on multi-feature fusion. In: 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), pp. 184–189 (2020). https://doi.org/10.1109/CVPR.2018.00913

  12. Günay, O., Enis Ç.A.: Real-time dynamic texture recognition using random sampling and dimension reduction. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3087–3091 (2015).https://doi.org/10.1109/ICIP.2015.7351371

  13. Yu, N., Chen, Y.: Video flame detection method based on TwoStream convolutional neural network. In: 2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), pp. 482–486 (2019). https://doi.org/10.1109/CVPR.2018.00913

  14. Chaoxia, C., Shang, W., Zhang, F.: Information-guided flame detection based on faster R-CNN. IEEE Access 8, 58923–58932 (2020). https://doi.org/10.1109/CVPR.2018.00913

    Article  Google Scholar 

  15. Pan, H., Badawi, D., Cetin, A.E.: Computationally efficient wildfire detection method using a deep convolutional network pruned via Fourier analysis. Sensors 20(10), 2891 (2020). https://doi.org/10.3390/s20102891

    Article  Google Scholar 

  16. Pan, H., Badawi, D., Zhang, X., Cetin, A.E.: Additive neural network for forest fire detection. Signal Image Video Process. SIViP 14, 675–682 (2020). https://doi.org/10.1007/s11760-019-01600-7

    Article  Google Scholar 

  17. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37(9), 1904–1916 (2015). https://doi.org/10.1109/TPAMI.2015.2389824

    Article  Google Scholar 

  18. Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017). https://doi.org/10.1109/CVPR.2017.106

  19. Howard, A.G., Zhu, M., Chen, B., et al.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017). arXiv:1704.04861

  20. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510–4520 (2018). https://doi.org/10.1109/CVPR.2018.00474

  21. Howard, A., Sandler, M., Chu, G., et al.: Searching for MobileNetV3. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1314–1324 (2019). https://doi.org/10.1109/CVPR.2018.00913

  22. Jie, H., Shen, L., Albanie, S., Sun, G., Enhua, W.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020). https://doi.org/10.1109/TPAMI.2019.2913372

    Article  Google Scholar 

  23. Hou, Q., Zhou, D., Feng, J.: Coordinate Attention for Efficient Mobile Network Design (2021). arXiv:2103.02907

  24. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Anal. Mach. Intell. 39(6), 1137–1149 (2017). https://doi.org/10.1109/CVPR.2018.00913

    Article  Google Scholar 

  25. Liu, W., Anguelov, D., Erhan, D., et al.: SSD: single shot multibox detector. In: European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 9905 (2016)

  26. Redmon, J., Redmon J.: YOLOv3: An Incremental Improvement (2018). arXiv:1804.02767

  27. Muhammad, K., Ahmad, J., Mehmood, I., Rho, S., Sung, W.B.: Convolutional neural networks based fire detection in surveillance videos. IEEE Access 6, 18174–18183 (2018). https://doi.org/10.1109/ACCESS.2018.2812835

    Article  Google Scholar 

  28. Muhammad, K., Ahmad, J., Lv, Z., Bellavista, P., Yang, P., Baik, S.W.: Efficient deep CNN-based fire detection and localization in video surveillance applications. IEEE Trans. Syst. Man Cybern. Syst. 49(7), 1419–1434 (2019). https://doi.org/10.1109/TSMC.2018.2830099

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key R & D Program of China (2019YFB1312104) and Key R & D Program of Hebei Province (20311803D).

Author information

Authors and Affiliations

  1. Institute of Electrical Engineering, Yanshan University, Qinhuangdao, China

    Yifan Wang, Changchun Hua, Weili Ding & Ruinan Wu

Authors
  1. Yifan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Changchun Hua
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weili Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ruinan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Hua, C., Ding, W. et al. Real-time detection of flame and smoke using an improved YOLOv4 network. SIViP 16, 1109–1116 (2022). https://doi.org/10.1007/s11760-021-02060-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-02060-8

Keywords

Search

Navigation