Skip to main content
SpringerLink
Log in

A deep neural network and rule-based technique for fire risk identification in video frames

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Automatically monitoring roadside fire risk plays a significant role in ensuring road safety by reducing potential hazards imposed to vehicle drivers and enabling effective roadside vegetation management. However, little work has been conducted in this field using video data collected by vehicle-mounted cameras. In this paper, a novel approach is proposed for roadside fire risk identification based on the biomass of grasses. Inspired by the biomass measurement method by human in grass curing, the proposed approach predicts the biomass and identifies high-risk regions using threshold-based rules based on two site-specific parameters of roadside grasses—brown grass coverage (BGC) and height (BGH). The BGC is calculated as the percentage of brown grass pixels in a sampling region, while the BGH is predicted based on the connectivity characteristics of grass stems along the vertical direction. To further reduce the false alarm rate of fire risk, we additionally incorporate and compare two deep learning techniques, including autoencoder and convolutional neural network, for refining the results. Our approach shows high performance of combining threshold-based rules with deep neural networks in classifying low and high fire risk on a roadside image dataset from video collected by the Department of Transport and Main Roads, Queensland, Australia.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Vazirabad YF, Karslioglu MO (2011) Lidar for biomass estimation, biomass Darko Matovic. Intech Open. https://doi.org/10.5772/16919. Available from: https://www.intechopen.com/books/biomass-detectionproduction-and-usage/lidar-for-biomass-estimation

  2. Bond WJ, Van Wilgen BW (2012) Fire and plants, vol 14. Springer, Berlin

    Google Scholar 

  3. Campbell NW, Thomas BT, Troscianko T (1997) Automatic segmentation and classification of outdoor images using neural networks. Int J Neural Syst 08(01):137–144. https://doi.org/10.1142/S0129065797000161

    Article  Google Scholar 

  4. Haibing Z, Shirong L, Chaoliang Z (2014) Outdoor scene understanding using SEVI-BOVW model. In: Neural networks (IJCNN), 2014 international joint conference on, 6–11 July 2014, pp 2986–2990. https://doi.org/10.1109/ijcnn.2014.6889778

  5. Harbas I, Subasic M (2014) Motion estimation aided detection of roadside vegetation. In: Image and signal processing (CISP), 2014 7th international congress on, 14–16 Oct. 2014, pp 420–425. https://doi.org/10.1109/cisp.2014.7003817

  6. Nguyen DV, Kuhnert L, Thamke S, Schlemper J, Kuhnert KD (2012) A novel approach for a double-check of passable vegetation detection in autonomous ground vehicles. In: Intelligent transportation systems (ITSC), 2012 15th international IEEE conference on, 16–19 Sept. 2012, pp 230–236. https://doi.org/10.1109/itsc.2012.6338752

  7. Blas MR, Agrawal M, Sundaresan A, Konolige K (2008) Fast color/texture segmentation for outdoor robots. In: Intelligent robots and systems (IROS), IEEE/RSJ international conference on, 22–26 Sept. 2008, pp 4078–4085. https://doi.org/10.1109/iros.2008.4651086

  8. Bosch A, Muñoz X, Freixenet J (2007) Segmentation and description of natural outdoor scenes. Image Vis Comput 25(5):727–740. https://doi.org/10.1016/j.imavis.2006.05.015

    Article  Google Scholar 

  9. Zhang L, Verma B, Stockwell D (2016) Spatial contextual superpixel model for natural roadside vegetation classification. Pattern Recognit 60:444–457. https://doi.org/10.1016/j.patcog.2016.05.013

    Article  Google Scholar 

  10. Zafarifar B, De With PHN (2008) Grass field detection for TV picture quality enhancement. In: Consumer electronics, 2008. ICCE 2008. Digest of technical papers. International conference on, 9–13 Jan 2008, pp 1–2. https://doi.org/10.1109/icce.2008.4587982

  11. Schepelmann A, Hudson RE, Merat FL, Quinn RD (2010) Visual segmentation of lawn grass for a mobile robotic lawnmower. In: Intelligent robots and systems (IROS), 2010 IEEE/RSJ international conference on, 18–22 Oct. 2010, pp 734–739. https://doi.org/10.1109/iros.2010.5650430

  12. Zhang L, Verma B, Stockwell D (2015) Roadside vegetation classification using color intensity and moments. In: The 11th international conference on natural computation, pp 1250–1255

  13. Chowdhury S, Verma B, Stockwell D (2015) A novel texture feature based multiple classifier technique for roadside vegetation classification. Expert Syst Appl 42(12):5047–5055. https://doi.org/10.1016/j.eswa.2015.02.047

    Article  Google Scholar 

  14. Bradley DM, Unnikrishnan R, Bagnell J (2007) Vegetation detection for driving in complex environments. In: Robotics and automation, 2007 IEEE international conference on, 10–14 April 2007, pp 503–508. https://doi.org/10.1109/robot.2007.363836

  15. Nguyen DV, Kuhnert L, Kuhnert KD (2012) Structure overview of vegetation detection. A novel approach for efficient vegetation detection using an active lighting system. Robot Auton Syst 60(4):498–508. https://doi.org/10.1016/j.robot.2011.11.012

    Article  Google Scholar 

  16. Nguyen DV, Kuhnert L, Jiang T, Thamke S, Kuhnert KD (2011) Vegetation detection for outdoor automobile guidance. In: Industrial technology (ICIT), 2011 IEEE international conference on, 14–16 March 2011, pp 358–364. https://doi.org/10.1109/icit.2011.5754402

  17. Nguyen DV, Kuhnert L, Kuhnert KD (2012) Spreading algorithm for efficient vegetation detection in cluttered outdoor environments. Robot Auton Syst 60(12):1498–1507. https://doi.org/10.1016/j.robot.2012.07.022

    Article  Google Scholar 

  18. Lu X, Wu H, Yuan Y (2014) Double constrained NMF for hyperspectral unmixing. IEEE Trans Geosci Remote Sens 52(5):2746–2758

    Article  Google Scholar 

  19. Yuan Y, Fu M, Lu X (2015) Substance dependence constrained sparse NMF for hyperspectral unmixing. IEEE Trans Geosci Remote Sens 53(6):2975–2986

    Article  Google Scholar 

  20. Royo C, Villegas D (2011) Field Measurements of canopy spectra for biomass assessment of small-grain cereals. In: Biomass—detection, production and usage. INTECH Open Access Publisher

  21. Sannier C, Taylor J, Plessis WD (2002) Real-time monitoring of vegetation biomass with NOAA-AVHRR in Etosha National Park, Namibia, for fire risk assessment. Int J Remote Sens 23(1):71–89

    Article  Google Scholar 

  22. Verbesselt J, Somers B, van Aardt J, Jonckheere I, Coppin P (2006) Monitoring herbaceous biomass and water content with SPOT VEGETATION time-series to improve fire risk assessment in savanna ecosystems. Remote Sens Environ 101(3):399–414. https://doi.org/10.1016/j.rse.2006.01.005

    Article  Google Scholar 

  23. Schneider P, Roberts DA, Kyriakidis PC (2008) A VARI-based relative greenness from MODIS data for computing the Fire Potential Index. Remote Sens Environ 112(3):1151–1167. https://doi.org/10.1016/j.rse.2007.07.010

    Article  Google Scholar 

  24. St-Onge B, Hu Y, Vega C (2008) Mapping the height and above-ground biomass of a mixed forest using lidar and stereo Ikonos images. Int J Remote Sens 29(5):1277–1294

    Article  Google Scholar 

  25. Ahamed T, Tian L, Zhang Y, Ting KC (2011) A review of remote sensing methods for biomass feedstock production. Biomass Bioenergy 35(7):2455–2469. https://doi.org/10.1016/j.biombioe.2011.02.028

    Article  Google Scholar 

  26. Sritarapipat T, Rakwatin P, Kasetkasem T (2014) Automatic rice crop height measurement using a field server and digital image processing. Sensors 14(1):900–926

    Article  Google Scholar 

  27. Juan Z, Xin-yuan H (2009) Measuring method of tree height based on digital image processing technology. In: Information science and engineering (ICISE), 2009 1st international conference on, 26–28 Dec. 2009, pp 1327–1331. https://doi.org/10.1109/icise.2009.732

  28. Leung T, Malik J (2001) Representing and recognizing the visual appearance of materials using three-dimensional textons. Int J Comput Vision 43(1):29–44. https://doi.org/10.1023/a:1011126920638

    Article  MATH  Google Scholar 

  29. Schmid C (2001) Constructing models for content-based image retrieval. In: IEEE computer society conference on computer vision and pattern recognition, 2001. IEEE, pp 39–45

  30. Geusebroek J-M, Smeulders AW, Van De Weijer J (2003) Fast anisotropic gauss filtering. IEEE Trans Image Process 12(8):938–943

    Article  MathSciNet  MATH  Google Scholar 

  31. Daugman JG (1980) Two-dimensional spectral analysis of cortical receptive field profiles. Vis Res 20(10):847–856. https://doi.org/10.1016/0042-6989(80)90065-6

    Article  Google Scholar 

  32. Chang T, Kuo C-C (1993) Texture analysis and classification with tree-structured wavelet transform. IEEE Trans Image Process 2(4):429–441

    Article  Google Scholar 

  33. Reyes-Aldasoro CC, Bhalerao A (2006) The Bhattacharyya space for feature selection and its application to texture segmentation. Pattern Recogn 39(5):812–826. https://doi.org/10.1016/j.patcog.2005.12.003

    Article  MATH  Google Scholar 

  34. Winn J, Criminisi A, Minka T Object categorization by learned universal visual dictionary. In: Computer vision (ICCV). Tenth IEEE international conference on, 17–21 Oct. 2005, pp 1800–1807. https://doi.org/10.1109/iccv.2005.171

  35. Kang Y, Kidono K, Naito T, Ninomiya Y (2008) Multiband image segmentation and object recognition using texture filter banks. In: 19th international conference on pattern recognition, 2008. IEEE, pp 1–4

  36. Shotton J, Winn J, Rother C, Criminisi A (2009) Texton boost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81(1):2–23. https://doi.org/10.1007/s11263-007-0109-1

    Article  Google Scholar 

  37. Kasson JM, Plouffe W (1992) An analysis of selected computer interchange color spaces. ACM Trans Graph 11(4):373–405. https://doi.org/10.1145/146443.146479

    Article  MATH  Google Scholar 

  38. Felzenszwalb P, Huttenlocher D (2004) Efficient graph-based image segmentation. Int J Comput Vision 59(2):167–181. https://doi.org/10.1023/B:VISI.0000022288.19776.77

    Article  Google Scholar 

  39. Verma B, Zhang L, Stockwell D (2017) Case study: roadside video data analysis for fire risk assessment. roadside video data analysis. Springer, Berlin, pp 159–183

    Chapter  Google Scholar 

  40. Rasmussen C (2004) Grouping dominant orientations for ill-structured road following. In: Computer vision and pattern recognition (CVPR). Proceedings of the 2004 IEEE computer society conference on, 27 June-2 July 2004, pp 470–477. https://doi.org/10.1109/cvpr.2004.1315069

  41. Zheng L, Zhao Y, Wang S, Wang J, Tian Q (2016) Good practice in CNN feature transfer. arXiv preprint arXiv:160400133

  42. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  43. Zhang L, Verma B, Stockwell D (2015) Class-semantic color-texture textons for vegetation classification. In: Arik S, Huang T, Lai WK, Liu Q (eds) Neural information processing (Lecture notes in computer science), vol 9489. Springer International Publishing, pp 354–362. https://doi.org/10.1007/978-3-319-26532-2_39

  44. Tighe J, Lazebnik S (2010) Superparsing: scalable nonparametric image parsing with superpixels. In: Daniilidis K, Maragos P, Paragios N (eds) Computer vision—ECCV 2010 (Lecture notes in computer science), vol 6315. Springer, Berlin, pp 352–365. https://doi.org/10.1007/978-3-642-15555-0_26

  45. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651. https://doi.org/10.1109/TPAMI.2016.2572683

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported under Australian Research Council's Linkage Projects funding scheme (project number LP140100939).

Author information

Authors and Affiliations

  1. School of Engineering and Technology, Central Queensland University, Brisbane, Australia

    Ligang Zhang & Brijesh Verma

Authors
  1. Ligang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brijesh Verma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ligang Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Verma, B. A deep neural network and rule-based technique for fire risk identification in video frames. Pattern Anal Applic 22, 187–203 (2019). https://doi.org/10.1007/s10044-018-0756-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-018-0756-6

Keywords

Search

Navigation