Skip to main content

Advertisement

Log in

Hybrid CBAM-EfficientNetV2 Fire Image Recognition Method with Label Smoothing in Detecting Tiny Targets

  • Research Article
  • Published:
Machine Intelligence Research Aims and scope Submit manuscript

Abstract

Image fire recognition is of great significance in fire prevention and loss reduction through early fire detection and warning. Aiming at the problems of low accuracy of existing fire recognition and high error rate of tiny target detection, this study proposed a fire recognition model based on a channel space attention mechanism. First, the convolutional block attention module (CBAM) is introduced into the first and last convolutional layers EfficientNetV2, which shows strong feature extraction ability and high computational efficiency as the backbone network. In terms of channel and space aspects, the weights in the feature layer are increased, which enhances the semantic information of flame smoke features and makes the model pay more attention to the feature information of fire images. Then, label smoothing based on the cross-entropy loss function is introduced into this study to avoid predicting labels too confidently in the training process to improve the generalization ability of the recognition model. The experimental results show that the fire image recognition accuracy based on the CBAM-EfficientNetV2 model reaches 98.9%. The accuracy of smoke image recognition can reach 98.5%. The accuracy of small target detection can reach 96.1%. At the same time, we compared the existing methods and found that the proposed method achieved higher accuracy, precision, recall, and F1-score. Finally, the fire image results are visualized using the Grad-CAM technique, which makes the model more effective and more intuitive in detecting tiny targets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Y. F. Guo. Application of fire protection IOT in the fire causes investigation. Fire Protection Today, vol. 7, no. 4, pp. 112–114, 2022. (in Chinese)

    Google Scholar 

  2. Z. C. Liu, K. Zhang, C. Y. Wang, S. Y. Huang. Research on the identification method for the forest fire based on deep learning. Optik, vol. 223, Article number 165491, 2020. DOI: https://doi.org/10.1016/j.ijleo.2020.165491.

  3. P. Barmpoutis, T. Stathaki, K, Dimitropoulos, G. Nikos. Early fire detection based on aerial 360-degree sensors, deep convolution neural networks and exploitation of fire dynamic textures. Remote Sensing, vol. 12, no. 19, Article number 3177, 2020. DOI: https://doi.org/10.3390/rs12193177.

  4. H. Wu, D. Y. Wu, J. S. Zhao. An intelligent fire detection approach through cameras based on computer vision methods. Process Safety and Environmental Protection, vol. 127, pp. 245–256, 2019. DOI: https://doi.org/10.1016/j.psep.2019.05.016.

    Article  Google Scholar 

  5. T. X. Tung, J. M. Kim. An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Safety Journal, vol. 46, no. 5, pp. 276–282, 2011. DOI: https://doi.org/10.1016/j.firesaf.2011.03.003.

    Article  Google Scholar 

  6. P. Li, W. D. Zhao. Image fire detection algorithms based on convolutional neural networks. Case Studies in Thermal Engineering, vol. 19, Article number 100625, 2020. DOI: https://doi.org/10.1016/j.csite.2020.100625.

  7. M. Hashemzadeh. Hiding information in videos using motion clues of feature points. Computers & Electrical Engineering, vol. 68, pp. 14–25, 2018. DOI: https://doi.org/10.1016/j.compeleceng.2018.03.046.

    Article  Google Scholar 

  8. M. Hashemzadeh, B. Asheghi, N. Farajzadeh. Content-aware image resizing: An improved and shadow-preserving seam carving method. Signal Processing, vol. 155, pp. 233–246, 2018. DOI: https://doi.org/10.1016/j.sigpro.2018.09.037.

    Article  Google Scholar 

  9. M. Hashemzadeh, B. A. Azar. Retinal blood vessel extraction employing effective image features and combination of supervised and unsupervised machine learning methods. Artificial Intelligence in Medicine, vol. 95, pp. 1–15, 2019. DOI: https://doi.org/10.1016/j.artmed.2019.03.001.

    Article  Google Scholar 

  10. A. Khan, A. Sohail, U. Zahoora, A. S. Qureshi. A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, vol. 53, no. 8, pp. 5455–5516, 2020. DOI: https://doi.org/10.1007/s10462-020-09825-6.

    Article  Google Scholar 

  11. W. H. Xiong, J. F. Ren, Z. H. Wu, M. Jiang. Research on fire identification based on hybrid convolutional neural network. Computer Technology and Development, vol. 30, no. 7, pp. 81–86, 2020. DOI: https://doi.org/10.3969/j.issn.1673-629X.2020.07.018. (in Chinese)

    Google Scholar 

  12. J. Y. Ren. Research on Image Type Fire Identification Method Based on Convolutional Neural Network, Master dissertation, Xi’an University of Science and Technology, China, 2019. (in Chinese)

    Google Scholar 

  13. Q. Lu, J. Yu, Z. F. Wang. A color model based fire flame detection system. In Proceedings of the 7th Chinese Conference on Pattern Recognition, Springer, Chengdu, China, pp. 474–485, 2016. DOI: https://doi.org/10.1007/978-981-10-3002-4_40.

    Chapter  Google Scholar 

  14. T. Celik, H. Demirel. Fire detection in video sequences using a generic color model. Fire Safety Journal, vol. 44, no. 2, pp. 147–158, 2009. DOI: https://doi.org/10.1016/j.firesaf.2008.05.005.

    Article  Google Scholar 

  15. Y. H. Kim, A. Kim, H. Y. Jeong. RGB Color model based the fire detection algorithm in video sequences on wireless sensor network. International Journal of Distributed Sensor Networks, vol. 10, no. 4, Article number 923609, 2014. DOI: https://doi.org/10.1155/2014/923609.

  16. T. X. Truong, J. M. Kim. Fire flame detection in video sequences using multi-stage pattern recognition techniques. Engineering Applications of Artificial Intelligence, vol. 25, no. 7, pp. 1365–1372, 2012. DOI: https://doi.org/10.1016/j.engappai.2012.05.007.

    Article  Google Scholar 

  17. L. Rossi, M. Akhloufi, Y. Tison. Dynamic fire 3D modeling using a real-time stereovision system. Journal of Communication and Computer, vol. 6, no. 10, pp. 54–61, 2009.

    Google Scholar 

  18. Z. B. Mei, C. Y. Yu, X. Zhang. Machine vision based fire flame detection using multi-features. In Proceedings of the 24th Chinese Control and Decision Conference, IEEE, Taiyuan, China, pp. 2844–2848, 2012. DOI: https://doi.org/10.1109/CCDC.2012.6244453.

    Google Scholar 

  19. Y. K. Wang, A. G. Wu, J. Zhang, M. Zhao, W. S. Li, N. Dong. Fire smoke detection based on texture features and optical flow vector of contour. In Proceedings of the 12th World Congress on Intelligent Control and Automation, IEEE, Guilin, China, pp. 2879–2883, 2016. DOI: https://doi.org/10.1109/WCICA.2016.7578611.

    Google Scholar 

  20. J. Li, F. Li, L. Li. Research for a new fire image detection technology. Journal of Jiangsu Normal University (Natural Science Edition), vol. 31, no. 3, pp. 25–30, 2013. (in Chinese)

    Google Scholar 

  21. R. Bohush, N. Brouka. Smoke and flame detection in video sequences based on static and dynamic features. In Proceedings of Signal Processing: Algorithms, Architectures, Arrangements, and Applications, IEEE, Poznan, Poland, pp. 20–25, 2013.

    Google Scholar 

  22. G. P. Jiang, F. Shang, F. Wang, X. J. Liu, T. S. Qiu. A combined intelligent fire detector with BP networks. In Proceedings of the 6th World Congress on Intelligent Control and Automation, IEEE, Dalian, China, pp. 5417–5419, 2006. DOI: https://doi.org/10.1109/WCICA.2006.1714106.

    Google Scholar 

  23. A. Hosseini, M. Hashemzadeh, N. Farajzadeh. UFS-net: A unified flame and smoke detection method for early detection of fire in video surveillance applications using CNNs. Journal of Computational Science, vol. 61, Article number 101638, 2022. DOI: https://doi.org/10.1016/j.jocs.2022.101638.

  24. Ü. Atila, M. Uçar, K. Akyol, E. Uçar. Plant leaf disease classification using EfficientNet deep learning model. Ecological Informatics, vol. 61, Article number 101182, 2021. DOI: https://doi.org/10.1016/j.ecoinf.2020.101182.

  25. T. T. Tai, D. N. H. Thanh, N. Q. Hung. A dish recognition framework using transfer learning. IEEE Access, vol. 10, pp. 7793–7799, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3143119.

    Article  Google Scholar 

  26. Y. T. Li. Forest fire smoke recognition and detection based on EfficientNet. In Proceedings of Conference on Telecommunications, Optics and Computer Science, Dalian, China, pp. 712–719, 2022. DOI: https://doi.org/10.1109/TOCS56154.2022.10016028.

    Google Scholar 

  27. J. Fu, J. Liu, H. J. Tian, Y. Li, Y. J. Bao, Z. W. Fang, H. Q. Lu. Dual attention network for scene segmentation. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Long Beach, USA, pp. 3141–3149, 2020. DOI: https://doi.org/10.1109/CVPR.2019.00326.

    Google Scholar 

  28. J. Hi, L. Shen, G. Sun. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 8, pp. 2011–2023, 2020. DOI: https://doi.org/10.1109/TPAMI.2019.2913372.

    Article  Google Scholar 

  29. H. C. Li, P. F. Xiong, J. An, L. X. Wang. Pyramid attention network for semantic segmentation. In Proceedings of British Machine Vision Conference, Newcastle, UK, Article number. 285, 2018.

    Google Scholar 

  30. J. Park, S. Woo, J. Y. Lee, I. S. Kweon. BAM: Bottleneck attention module. In Proceedings of British Machine Vision Conference, Newcastle, UK, Article number 147, 2018.

    Google Scholar 

  31. H. J. Zhang, N. Zhang, N. F. Xiao. Fire detection and identification method based on visual attention mechanism. Optik, vol. 126, no. 24, pp. 5011–5018, 2015. DOI: https://doi.org/10.1016/j.ijleo.2015.09.167.

    Article  Google Scholar 

  32. S. Majid, F. Alenezi, S. Masood, M. Ahmad, E. S. Gündüz, K. Polat. Attention based CNN model for fire detection and localization in real-world images. Expert Systems with Applications, vol. 189, Article number 116114, 2022. DOI: https://doi.org/10.1016/j.eswa.2021.116114.

  33. L. J. He, X. L. Gong, S. R. Zhang, L. J. Wang, F. Li. Efficient attention based deep fusion CNN for smoke detection in fog environment. Neurocomputing, vol. 434, pp. 224–238, 2021. DOI: https://doi.org/10.1016/j.neucom.2021.01.024.

    Article  Google Scholar 

  34. Y. X. Mao, T. Z. Zhang, B. Fu, D. N. H. Thanh. A self-attention based wasserstein generative adversarial networks for single image inpainting. Pattern Recognition and Image Analysis, vol. 32, no. 3, pp. 591–599, 2022. DOI: https://doi.org/10.1134/S1054661822030245.

    Article  Google Scholar 

  35. S. Woo, J. Park, J. Y. Lee, I. S. Kweon. CBAM: Convolutional block attention module. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 3–19, 2018. DOI: https://doi.org/10.1007/978-3-030-01234-2_1.

    Google Scholar 

  36. M. X. Tan, Q. V. Le. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, USA, pp. 6105–6114, 2019.

    Google Scholar 

  37. M. X. Tan, Q. V. Le. EfficientNet: Rethinking M. X. Tan, Q. V. Le. EfficientNetV2: Smaller models and faster training. In Proceedings of the 38th International Conference on Machine Learning, pp. 10096–10106, 2021.

    Google Scholar 

  38. R. Müller, S. Kornblith, G. Hinton. When does label smoothing help? In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, Canada, Article number 422, 2019.

    Google Scholar 

  39. A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, USA, pp. 1097–1105, 2012.

    Google Scholar 

  40. M. X. Tan, Q. V. Le. EfficientNet: Rethinking K. Simonyan, A. Zisserman. Very deep convolutional networks for large-scale image recognition. [Online], Available: https://arxiv.org/abs/1409.1556, 2014.

    Google Scholar 

  41. K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90.

    Google Scholar 

  42. C. Szegedy, S. Ioffe, V. Vanhoucke, A. A. Alemi. Inception-v4, inception-ResNet and the impact of residual connections on learning. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, USA, pp. 4278–4284, 2017.

    Google Scholar 

  43. C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594.

    Google Scholar 

  44. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna. Rethinking the inception architecture for computer vision. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 2818–2826, 2016. DOI: https://doi.org/10.1109/CVPR.2016.308.

    Google Scholar 

  45. N. N. Ma, X. Y. Zhang, H. T. Zheng, J. Sun. ShuffleNet V2: Practical guidelines for efficient CNN architecture design. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 122–138, 2018. DOI: https://doi.org/10.1007/978-3-030-01264-9_8.

    Google Scholar 

  46. X. Y. Zhang, X. Y. Zhou, M. X. Lin, J. Sun. ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Salt Lake City, USA, pp. 6848–6856, 2017. DOI: https://doi.org/10.1109/CV-PR.2018.00716.

    Google Scholar 

  47. Q. B. Hou, D. Q. Zhou, J. S. Feng. Coordinate attention for efficient mobile network design. In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, Nashville, USA, pp. 13708–13717, 2021. DOI: https://doi.org/10.1109/CVPR46437.2021.01350.

    Google Scholar 

  48. R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra. Grad-CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, vol. 128, no. 2, pp. 336–359, 2020. DOI: https://doi.org/10.1007/s11263-019-01228-7.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Key Research and Development Program of China (No. 2021YFC15235 02-03) and Fundamental Research Funds for the Central Universities, China (No. FRF-IDRY-21-016).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xuehong Gao.

Ethics declarations

The authors declared that they have no conflicts of interest to this work.

Additional information

Bo Wang received the M. Sc. degree in safety engineering from School of Civil and Resource Engineering, University of Science and Technology Beijing, China in 2023. Currently, he is working as a safety engineer in Shanghai New Micro Semiconductor, China.

His research interests include image recognition and risk assessment.

Guozhong Huang received the B. Sc., M. Sc. and Ph. D. degrees in municipal engineering from Harbin Institute of Technology, China in 1994, 1997 and 2001, respectively. He is a full-time professor in Department of Safety and Science, University of Science and Technology Beijing, China. He has published more than 60 papers in academic journals.

His research interests include safety assessment and risk analysis, automotive product defect risk assessment and general consumer product safety management, public safety and emergency response, intelligent industrial equipment service safety risk warning and other directions.

Haoxuan Li received the M. Sc. degree in environmental engineering from University of New South Wales, UK in 2020. He is currently a Ph. D. candidate in safety science and engineering at School of Civil and Resource Engineering, University of Science and Technology Beijing, China.

His research interests include indirect injury, risk assessment and big data analysis.

Xiaolong Chen received the M. Sc. degree in security engineering from University of Science and Technology Beijing, China in 2022. He now works for Huawei, China.

His research interests include fire identification and network security.

Lei Zhang received the Ph.D. degree in metallurgical engineering from University of Science and Technology Beijing, China. He is currently a lecturer at School of Civil and Resource Engineering, University of Science and Technology Beijing, China.

His research interests include metal smelting safety and product collateral damage.

Xuehong Gao received the Ph. D. degree in industrial engineering from Pusan National University, Republic of Korea in 2021. He is an associate professor at University of Science and Technology Beijing, China. He has published more than 20 research articles in journals such as IJPR, IJPE, JCLP, CAIE, ANOR, etc.

His research interests include robust optimization, disaster management, safety science, and operations research.

Colored figures are available in the online version at https://link.springer.com/journal/11633

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, B., Huang, G., Li, H. et al. Hybrid CBAM-EfficientNetV2 Fire Image Recognition Method with Label Smoothing in Detecting Tiny Targets. Mach. Intell. Res. 21, 1145–1161 (2024). https://doi.org/10.1007/s11633-023-1445-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-023-1445-5

Keywords

Navigation