Skip to main content
SpringerLink
Log in

An AI-Based Action Detection UAV System to Improve Firefighter Safety

  • Conference paper
  • First Online:
Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14028))

Included in the following conference series:

  • 769 Accesses

Abstract

Human hazardous fires can inflict massive harm to life, property, and the environment. Close contact with fire sources threatens firefighters’ lives who are critical first responders to fire suppression and rescue. Unmanned aerial vehicles (UAVs) are recently introduced to improve firefighters’ performance by monitoring fire characteristics and inferring trajectory. Some UAVs studies used artificial intelligence (AI) for pattern recognition for wildfire prevention and other relevant tasks. However, how to coordinate firefighters is equally important and needs to be explored. Therefore, this research offers an analysis and comparison of AI-based computer vision methods that can annotate human movement automatically. This study proposes to use UAVs combined with AI-based human motion detection to recognize firefighters’ action patterns. Based on existing human motion datasets of firefighting videos, we have successfully trained a supervised machine learning algorithm recognizing firefighters’ forward movement and water splashing actions. The trained model was tested in the existing video and reached an 81.55% accuracy rate. Applying this model in the UAV system is able to improve firefighters’ functioning and safeguard their lives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Seraj, E., Silva, A., Gombolay, M.: Safe Coordination of Human-Robot Firefighting Teams (2019). https://arxiv.org/abs/1903.06847

  2. Hall, S., Evarts, B.: Fire loss in the United States during 2021. National Fire Protection Association (2022)

    Google Scholar 

  3. NOAA National Centers for Environmental Information, Monthly Wildfires Report for Annual 2022, published online January 2023. https://www.ncei.noaa.gov/access/monitoring/monthly-report/fire/202213. Accessed 10 Feb 2023

  4. Den Breejen, E., et al.: Autonomous forest fire detection, pp. 2003–2012. ADAI-Associacao para o Desenvolvimento da Aerodinamica Industrial, Coimbra (1998)

    Google Scholar 

  5. Merino, L., Caballero, F., Martinez-de Dios, J.R., Ollero, A.: Cooperative fire detection using unmanned aerial vehicles. In: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, pp. 1884–1889. IEEE (2005)

    Google Scholar 

  6. Sujit, P.B., Kingston, D., Beard, R.: Cooperative forest fire monitoring using multiple UAVs. In: 2007 46th IEEE Conference on Decision and Control, pp. 4875–4880. IEEE (2007)

    Google Scholar 

  7. Hrabia, C.E., Hessler, A., Xu, Y., Brehmer, J., Albayrak, S.: EffFeu project: efficient operation of unmanned aerial vehicles for industrial fire fighters. In: DroNet 2018 - Proceedings of the 2018 ACM International Conference on Mobile Systems, Applications and Services, pp. 33–38 (2018). https://doi.org/10.1145/3213526.3213533

  8. Seraj, E., Gombolay, M.: Coordinated control of UAVs for human-centered active sensing of wildfires. In: Proceedings of the American Control Conference, 2020-July, pp. 1845–1852 (2020). https://doi.org/10.23919/ACC45564.2020.9147613

  9. Luna, M.A., Refaat Ragab, A., Ale Isac, M.S., Flores Pena, P., Cervera, P.C.: A new algorithm using hybrid UAV swarm control system for firefighting dynamical task allocation. In: Conference Proceedings - IEEE International Conference on Systems, Man and Cybernetics, pp. 655–660 (2021). https://doi.org/10.1109/SMC52423.2021.9659275

  10. Yfantis, E.A.: A UAV with autonomy, pattern recognition for forest fire prevention, and AI for providing advice to firefighters fighting forest fires. In: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference, CCWC 2019, pp. 409–413 (2019). https://doi.org/10.1109/CCWC.2019.8666471

  11. Yuan, C., Liu, Z., Zhang, Y.: Aerial images-based forest fire detection for firefighting using optical remote sensing techniques and unmanned aerial vehicles. J. Intell. Rob. Syst. 88(2–4), 635–654 (2017). https://doi.org/10.1007/s10846-016-0464-7

    Article  Google Scholar 

  12. Toups, Z.O., Kerne, A.: Implicit coordination in firefighting practice: design implications for teaching fire emergency responders. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 707–716 (2007)

    Google Scholar 

  13. Seraj, E., Silva, A., Gombolay, M.: Safe coordination of human-robot firefighting teams. arXiv preprint https://arxiv.org/abs/1903.06847 (2019)

  14. Castro, D., New, J.: The promise of artificial intelligence. Center Data Innov. 115(10), 32–35 (2016)

    Google Scholar 

  15. Horowitz, M.C., Allen, G.C., Saravalle, E., Cho, A., Frederick, K., Scharre, P.: Artificial intelligence and international security. Center for a New American Security (2018)

    Google Scholar 

  16. Mallouhy, R.E., Guyeux, C., Abou Jaoude, C., Makhoul, A.: Machine learning for predicting firefighters’ interventions per type of mission. In: 2022 8th International Conference on Control, Decision and Information Technologies (CoDIT), vol. 1, pp. 1196–1200. IEEE (2022)

    Google Scholar 

  17. Raj, J.V., Sarath, T.V.: An IoT based real-time stress detection system for fire-fighters. In: 2019 International Conference on Intelligent Computing and Control Systems (ICCS), pp. 354–360. IEEE (2019)

    Google Scholar 

  18. Zadeh, N.R.N., Abdulwakil, A.H., Amar, M.J.R., Durante, B., Santos, C.V.N.R.: Fire-fighting UAV with shooting mechanism of fire extinguishing ball for smart city. Indones. J. Electr. Eng. Comput. Sci 22, 1320–1326 (2021)

    Google Scholar 

  19. Yfantis, E.A. A UAV with autonomy, pattern recognition for forest fire prevention, and AI for providing advice to firefighters fighting forest fires. In: 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC), pp. 0409–0413. IEEE (2019)

    Google Scholar 

  20. Sherstjuk, V., Zharikova, M., Sokol, I.: Forest fire-fighting monitoring system based on UAV team and remote sensing. In: 2018 IEEE 38th International Conference on Electronics and Nanotechnology (ELNANO), pp. 663–668. IEEE (2018)

    Google Scholar 

  21. Thakur, A., Kate, O., Malhotra, T.: Design and Development of smart UAV assistance for Firefighters (2021)

    Google Scholar 

  22. Bouguettaya, A., Zarzour, H., Taberkit, A.M., Kechida, A.: A review on early wildfire detection from unmanned aerial vehicles using deep learning-based computer vision algorithms. Signal Process. 190, 108309 (2022)

    Article  Google Scholar 

  23. Lugaresi, C., et al.: MediaPipe: A Framework for Building Perception Pipelines (2019). http://arxiv.org/abs/1906.08172

Download references

Author information

Authors and Affiliations

  1. School of Construction Machinery, Chang’an University, Xi’an, 710064, China

    Hong Wang

  2. Department of Mechanical and Aerospace, University of Missouri, Columbia, MO, 65203, USA

    Yuan Feng & Xu Huang

  3. Digital Worlds Institute, University of Florida, Gainesville, FL, 32611, USA

    Wenbin Guo

Authors
  1. Hong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenbin Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenbin Guo .

Editor information

Editors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Vincent G. Duffy

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, H., Feng, Y., Huang, X., Guo, W. (2023). An AI-Based Action Detection UAV System to Improve Firefighter Safety. In: Duffy, V.G. (eds) Digital Human Modeling and Applications in Health, Safety, Ergonomics and Risk Management. HCII 2023. Lecture Notes in Computer Science, vol 14028. Springer, Cham. https://doi.org/10.1007/978-3-031-35741-1_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35741-1_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35740-4

  • Online ISBN: 978-3-031-35741-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation