Skip to main content
SpringerLink
Log in

Flame and smoke recognition on smart edge using deep learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

This study develops real-time flame and smoke recognition on an intelligent edge using transfer learning and deep learning. In this case, the inference application was deployed on edge devices. There are devices and tools to accelerate the operation of neural networks, including Intel Neural Compute Stick v.2 (NCS2) on Raspberry Pi4 and DeepStream on NVIDIA Jetson Xavier NX, as edge computing devices. The inference process is also presented and conducted on the web. In terms of operating efficiency, we use NCS2 to accelerate detection which can reach 2.5FPS and DeepStream on Jetson Xavier which can reach 10FPS at an image resolution of \(1280 \times 720\). Yolo was used as the inference model because it has a low error value of 0.8 compared to Faster R-CNN at 2.3 and SSD MobileNetv1 at 1.2. Finally, an aerial camera and a webcam were tested to detect flame or smoke and recognize whether the detection effect will be reduced due to external light sources or other factors in actual applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author upon reasonable request.

References

  1. Gaur A, Singh A, Kumar A, Kumar A, Kapoor K (2020) Video flame and smoke based flame detection algorithms: a literature review. Flame Technol 56(5):1943–1980

    Google Scholar 

  2. Wu H, Wu D, Zhao J (2019) An intelligent flame detection approach through cameras based on computer vision methods. Process Saf Environ Prot 127:245–256

    Article  Google Scholar 

  3. Li P, Zhao W (2020) Image flame detection algorithms based on convolutional neural networks. Case Stud Therm Eng 19:100625

    Article  Google Scholar 

  4. Zhang T, Yang C-T, Lin H-H, Lo L-J, Chan Y-W (2020) the implementation of facial symmetry assessment before and after orthognathic surgery using transfer learning. In: Published in The 3rd International Conference on Innovative Computing (IC2020)

  5. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  6. Shi F et al (2020) A flame monitoring and alarm system based on YOLOv3 with OHEM. In: 2020 39th Chinese control conference (CCC). IEEE

  7. Zhang M, et al. (2020) Deep learning in the era of edge computing: challenges and opportunities. Fog Comput 67–78

  8. Chen YC, Fathoni H, Yang CT (2020) Implementation of fire and smoke detection using deepstream and edge computing approachs. In: 2020 International conference on pervasive artificial intelligence (ICPAI). IEEE, pp. 272–275

  9. Han D, Lee B (2009) Flame and smoke detection method for early real-time detection of a tunnel flame. Flame Saf J 44(7):951–961

    Google Scholar 

  10. Tang J, Liu S, Liu L, Yu B, Shi W (2020) LoPECS: a low power edge computing system for real-time autonomous driving services. IEEE Access 8:30467–30479

    Article  Google Scholar 

  11. Huynh LN, Lee Y, Balan RK (2017) DeepMon: mobile GPU based deep learning framework for continuous vision applications. In: MobiSys 2017 - Proceedings of the 15th annual international conference on mobile systems, applications, and services. pp 82–95

  12. Muhammad K, Khan S, Palade V, Mehmood I, De Albuquerque VHC (2019) Edge intelligence-assisted smoke detection in foggy surveillance environments. IEEE Trans Industr Inf 16(2):1067–1075

    Article  Google Scholar 

  13. Ke R et al (2020) A smart, efficient, and reliable parking surveillance system with edge artificial intelligence on IoT devices. IEEE Trans Intell Transp Syst 22(8):4962–4974

    Article  Google Scholar 

  14. Fernandez-Beltran AP, Plaza J (2019) Low-high-power consumption architectures for deep-learning models applied to hyperspectral image classification. IEEE Geosci. Remote Sens. Lett. 16(5):776–780

    Article  Google Scholar 

  15. Jadon A, Varshney A, Ansari MS (2020) Low-complexity highperformance deep learningdeep learning model for real-time low-cost embedded flame detection systems. Procedia Comput Sci 171(2019):418–426

    Article  Google Scholar 

  16. Zhao P, Li J, Xi J, Gou X (2012) A mobile real-time video system using RTMP. In: 2012 Fourth International Conference on Computational Intelligence and Communication Networks. IEEE, pp 61–64

  17. Wu H, Wu D, Zhao J (2019) An intelligent fire detection approach through cameras based on computer vision methods. Process Saf Environ Prot 127:245–256

    Article  Google Scholar 

  18. Flynn H, Cameron S (2013) Multi-modal people detection from aerial video. In: Proceedings of the 8th international conference on computer recognition systems CORES 2013. pp 815–824

  19. Pi Y, Nath ND, Behzadan AH (2020) Convolutional neural networks for object detection in aerial imagery for disaster response and recovery. Adv Eng Inf 43:101009

    Article  Google Scholar 

  20. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  21. Zhao T, Nevatia R (2003) Car detection in low resolution aerialimages. Image Vis Comput 21(8):693–703

    Article  Google Scholar 

  22. Seebamrungsat J, Praising S, Riyamongkol P (2014) Flame detection in the buildings using image processing. In: 2014 Third ICT International Student Project Conference (ICT-ISPC). pp 95–98. https://doi.org/10.1109/ICT-ISPC.2014.6923226.

  23. Rivera AJA, Villalobos ADC, Monje JCN, Mariñas JAG, Oppus CM (2016) Post-Disaster Rescue Facility: Human Detection and Geolocation Using Aerial Drones. In: 2016 IEEE Region 10 Conference (TENCON)

  24. Seebamrungsat J, Praising S, Riyamongkol P (2014) Flame detection in the buildings using image processing. In: Student Project Conference (ICT-ISPC) 2014 Third ICT International. pp 95–98

  25. Yang C-T, Zhang T, Kristiani E, Cheng C-T (2019) The implementation of objects detection and analysis using deep learning with GPU. In: Published in The 9th International Conference on Frontier Computing (FC2019)

  26. Singh A, Patil D, Omkar SN (2018) Eye in the Sky: real-time drone surveillance system (DSS) for violent individuals identification using ScatterNet hybrid deep learning network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

  27. Kristiani E, Yang C, Huang C (2020) iSEC: an optimized deep learning model for image classification on edge computing. IEEE Access 8:27267–27276. https://doi.org/10.1109/ACCESS.2020.2971566

    Article  Google Scholar 

  28. Oreifej O, Mehran R, Shah M (2010) Human identity recognition in aerial images. In: Computer Vision and Pattern Recognition (CVPR) 2010 IEEE Conference on. pp 709–716

  29. Adam (2022) https://keras.io/api/optimizers/adam/, Keras, Accessed on 6 Jan 2022

  30. Wang J, Zhang Y, Lu J, Li Y (2013) Target detection and pedestrian recognition in infrared images. J Comput 8(4):1050–1057

    Article  Google Scholar 

  31. Jiao Z et al (2019) A Deep learning based forest flame detection approach using uav and YOLOv3. In: 1st International Conference on Industrial Artificial Intelligence (IAI). pp 3–7

  32. Moranduzzo T, Melgani F (2014) Detecting cars in uav UAV images with a catalog-based approach. IEEE Trans Geosci Remote Sens 52(10):6356–6367

    Article  Google Scholar 

  33. Yi Z, Yongliang S, Jun Z (2019) An improved tiny YOLOv3 pedestrian detection algorithm. Optik 183:17–23

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Science and Technology Council (NSTC), Taiwan (R.O.C.), under Grants Number 111-2622-E-029-003-, 111-2811-E-029-001-, 111-2621-M-029-004-, and 110-2221-E-029-020-MY3.

Author information

Authors and Affiliations

  1. Department of Computer Science, Tunghai University, Taichung City, 407224, Taiwan, R.O.C.

    Endah Kristiani, Yi-Chun Chen, Chao-Tung Yang & Chia-Hsin Li

  2. Department of Informatics, Krida Wacana Christian University, Jakarta, 11470, Indonesia

    Endah Kristiani

  3. Research Center for Smart Sustainable Circular Economy, Tunghai University, No. 1727, Sec. 4, Taiwan Boulevard, Taichung City, 407224, Taiwan, R.O.C.

    Chao-Tung Yang

  4. iAMBITION TECHNOLOGY CO., LTD., 3F., No. 159-1, Sec. 1, Zhongxing Rd., Dali District, 41267, Taichung, Taiwan, R.O.C.

    Chia-Hsin Li

Authors
  1. Endah Kristiani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yi-Chun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao-Tung Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chia-Hsin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao-Tung Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kristiani, E., Chen, YC., Yang, CT. et al. Flame and smoke recognition on smart edge using deep learning. J Supercomput 79, 5552–5575 (2023). https://doi.org/10.1007/s11227-022-04884-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04884-8

Keywords

Search

Navigation