Skip to main content
SpringerLink
Log in

CAM-based non-local attention network for weakly supervised fire detection

  • Original Research Paper
  • Published:
Service Oriented Computing and Applications Aims and scope Submit manuscript

Abstract

Many available object detectors are already used in fire detection, such as Faster RCNN, SSD, YOLO, etc., to localize the fire in images. Although these approaches perform well, they require object-level annotations for training, which are manually labeled and very expensive. In this paper, we propose a method based on the Class Activation Map (CAM) and non-local attention to explore the Weakly Supervised Fire Detection (WSFD) given only image-level annotations. Specifically, we first train a deep neural network with non-local attention as the classifier for identifying fire and non-fire images. Then, we use the classifier to create a CAM for every fire image in the inference stage and finally generate a corresponding bounding box according to each connected domain of the CAM. To evaluate the availability of our method, a benchmark dataset named WS-FireNet is constructed, and comprehensive experiments are performed on the WS-FireNet dataset. The experimental results demonstrate that our approach is effective in image-level supervised fire detection.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Liau H, Yamini N, Wong Y (2018) Fire ssd: Wide fire modules based single shot detector on edge device. arXiv preprint arXiv:1806.05363

  2. Kang L.-W, Wang I.-S, Chou K.-L, Chen S.-Y, Chang C.-Y (2019) Image-based real-time fire detection using deep learning with data augmentation for vision-based surveillance applications. In: 2019 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pp. 1–4 . IEEE

  3. Barmpoutis P, Dimitropoulos K, Kaza K, Grammalidis N (2019) Fire detection from images using faster r-cnn and multidimensional texture analysis. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 8301–8305, IEEE

  4. Li P, Zhao W (2020) Image fire detection algorithms based on convolutional neural networks. Case Stud Thermal Eng 19:100625

    Article  Google Scholar 

  5. Goyal S, Shagill M, Kaur A, Vohra H, Singh A (2020) A yolo based technique for early forest fire detection. Int J Innov Technol Explor Eng (IJITEE) 9:1357–1362

    Article  Google Scholar 

  6. Qin Y-Y, Cao J-T, Ji X-F (2021) Fire detection method based on depthwise separable convolution and yolov3. Int J Autom Comput 18(2):300–310

    Article  Google Scholar 

  7. Krizhevsky A, Sutskever I, Hinton G.E (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

  8. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  9. Su Y, Lin G, Zhu J, Wu, Q (2020) Human interaction learning on 3d skeleton point clouds for video violence recognition. In: European conference on computer vision, Springer, pp 74–90

  10. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T0, Dehghani M, Minderer M, Heigold G, Gelly S, et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

  11. Smahi MI, Hadjila F, Tibermacine C, Benamar A (2021) A deep learning approach for collaborative prediction of web service Qos. SOCA 15(1):5–20

    Article  Google Scholar 

  12. Su Y, Sun R, Lin G, Wu Q (2021) Context decoupling augmentation for weakly supervised semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7004–7014

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  14. Su Y, Lin G, Wu Q (2021) Self-supervised 3d skeleton action representation learning with motion consistency and continuity. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 13328–13338

  15. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  16. Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99

  17. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. In: European conference on computer vision, Springer, pp 21–37

  18. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 779–788

  19. Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7263–7271

  20. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767

  21. Bochkovskiy A, Wang C-Y, Liao H-YM (2020) Yolov4: optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934

  22. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vision 88(2):303–338

    Article  Google Scholar 

  23. Bilen H, Vedaldi A (2016) Weakly supervised deep detection networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2846–2854

  24. Tang P, Wang X, Bai X, Liu W (2017) Multiple instance detection network with online instance classifier refinement. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2843–2851

  25. Tang P, Wang X, Bai S, Shen W, Bai X, Liu W, Yuille A (2018) Pcl: Proposal cluster learning for weakly supervised object detection. IEEE Trans Pattern Anal Mach Intell 42(1):176–191

    Article  Google Scholar 

  26. Wei Y, Shen Z, Cheng B, Shi H, Xiong J, Feng J, Huang T (2018) Ts2c: Tight box mining with surrounding segmentation context for weakly supervised object detection. In: Proceedings of the European conference on computer vision (ECCV), pp 434–450

  27. Zeng Z, Liu B, Fu J, Chao H, Zhang L (2019) Wsod2: Learning bottom-up and top-down objectness distillation for weakly-supervised object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 8292–8300

  28. Li X, Kan M, Shan S, Chen X (2019) Weakly supervised object detection with segmentation collaboration. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9735–9744

  29. Ren Z, Yu Z, Yang X, Liu M-Y, Lee YJ, Schwing AG, Kautz J (2020) Instance-aware, context-focused, and memory-efficient weakly supervised object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10598–10607

  30. Huang Z, Zou Y, Kumar B, Huang D (2020) Comprehensive attention self-distillation for weakly-supervised object detection. Adv Neural Inf Process Syst 33:16797–16807

    Google Scholar 

  31. Uijlings JR, Van De Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. Int J Comput Vision 104(2):154–171

    Article  Google Scholar 

  32. Zitnick CL, Dollár P (2014) Edge boxes: locating object proposals from edges. In: European conference on computer vision, Springer, pp 391–405

  33. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba, A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

  34. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  35. Lai L, Chen J, Huang H, Wu Q (2021) Exploring a cam-based approach for weakly supervised fire detection task

  36. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587

  37. Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp 1440–1448

  38. Namozov A, Im Cho Y (2018) An efficient deep learning algorithm for fire and smoke detection with limited data. Adv Electr Comput Eng 18(4):121–128

    Article  Google Scholar 

  39. Sadewa RP, Irawan B, Setianingsih C (2019) Fire detection using image processing techniques with convolutional neural networks. In: 2019 international seminar on research of information technology and intelligent systems (ISRITI), IEEE, pp 290–295

  40. Jadon A, Omama M, Varshney A, Ansari MS, Sharma R (2019) Firenet: a specialized lightweight fire & smoke detection model for real-time iot applications. arXiv preprint arXiv:1905.11922

  41. Valikhujaev Y, Abdusalomov A, Cho YI (2020) Automatic fire and smoke detection method for surveillance systems based on dilated cnns. Atmosphere 11(11):1241

    Article  Google Scholar 

  42. Huang R, Pedoeem J, Chen C (2018) Yolo-lite: a real-time object detection algorithm optimized for non-gpu computers. In: 2018 IEEE international conference on big data (Big Data), IEEE, pp 2503–2510

  43. Dai J, Li Y, He K, Sun J (2016) R-fcn: object detection via region-based fully convolutional networks. Adv Neural Inf Process Syst 29

  44. Celik T (2010) Fast and efficient method for fire detection using image processing. ETRI J 32(6):881–890

    Article  Google Scholar 

  45. Harkat H, Nascimento J, Bernardino A (2020) Fire segmentation using a deeplabv3+ architecture. In: Image and signal processing for remote sensing XXVI, vol 11533. International Society for Optics and Photonics, p 115330

  46. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  47. Wang Q, Wu B, Zhu P, Li P, Hu Q (2020) Eca-net: Efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR)

  48. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  49. Hu, J., Shen, L., Albanie, S., Sun, G., Vedaldi, A (2018) Gather-excite: Exploiting feature context in convolutional neural networks. Adv Neural Inf Process Syst 31:9401–9411

  50. Woo S, Park J, Lee J-Y, Kweon IS (2018) Cbam: convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  51. Park J, Woo S, Lee J-Y, Kweon I-S (2018) Bam: Bottleneck attention module. In: British Machine Vision Conference (BMVC). British Machine Vision Association (BMVA)

  52. Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ’squeeze & excitation’in fully convolutional networks. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 421–429

  53. Santoro A, Raposo D, Barrett DG, Malinowski M, Pascanu, R, Battaglia P, Lillicrap T (2017) A simple neural network module for relational reasoning. Adv Neural Inf Process Syst 30:4967–4976

  54. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Icml

  55. Fire-Detection-Image-Dataset. https://github.com/cair/Fire-Detection-Image-Dataset

  56. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  57. Everingham M, Eslami S, Van Gool L, Williams CK, Winn J, Zisserman A (2015) The pascal visual object classes challenge: a retrospective. Int J Comput Vision 111(1):98–136

    Article  Google Scholar 

  58. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: Common objects in context. In: European conference on computer vision, Springer, pp 740–755

Download references

Funding

This work was supported by the National Natural Science Foundation of China (Grant No. 61876208 and 62072186), the Guangdong Basic and Applied Basic Research Foundation (Grant No. 2019B1515130001), the Opening Project of Guangdong Key Laboratory of Big Data Analysis and Processing, and the Opening Project of Ministry of Education Key Laboratory of Big Data and Intelligent Robot (South China University of Technology) (No. 202105).

Author information

Author notes

  1. Wenjun Wang and Lvlong Lai have contributed equally to this work.

Authors and Affiliations

  1. School of Software Engineering, South China University of Technology, Guangzhou, China

    Wenjun Wang, Lvlong Lai, Jian Chen & Qingyao Wu

  2. School of Data Science and Information Engineering, Guizhou Minzu University, Guiyang, China

    Wenjun Wang

  3. Key Laboratory of Big Data and Intelligent Robot, Ministry of Education, Guangzhou, China

    Wenjun Wang & Lvlong Lai

  4. Pazhou Lab, Guangzhou, China

    Qingyao Wu

Authors
  1. Wenjun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lvlong Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingyao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Lai and Wang developed the proposed method and drafted the manuscript. Wu and Chen supervised the project, contributed to discussion and analysis, and provided important suggestions for the paper. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Jian Chen or Qingyao Wu.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 749 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, W., Lai, L., Chen, J. et al. CAM-based non-local attention network for weakly supervised fire detection. SOCA 16, 133–142 (2022). https://doi.org/10.1007/s11761-022-00336-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11761-022-00336-6

Keywords

Search

Navigation