A high-precision forest fire smoke detection approach based on ARGNet
Introduction
Forest fires are a hazardous and destructive disaster that often causes economic and ecological damage. In recent years, forest fires have been frequent, and governments have incurred enormous management expenditures to cope with sudden forest fires (Stocks and Martell, 2016). Because forest burning leads to bare forestland, forests lose their function of water containment and soil conservation, which will further cause other natural disasters, such as internal floods, droughts, mudslides, landslides, and dust storms (Ren et al., 2011). After forest fires occur, local ecological conditions such as weather, water and soil are out of balance and often take decades to centuries to recover (Bakirci, 2010). At the same time, forest fires can lead to the burning of rare wildlife or loss of habitat, which can pose a challenge to the biodiversity of the Earth's ecology (Suthar and Bhavsar, 2021). In recent years, the prevention and monitoring of forest fires has become a research hotspot for forest fire prevention departments around the world. The detection methods for forest fires are divided into smoke detection and flame detection. Due to the many weeds and trees in the forest environment, flames are easily blocked in the early stages of a fire. When forest fires occur, they often produce a large amount of smoke. Smoke will spread over time, which makes it easier to detect. Therefore, smoke detection is an important tool for forest fire early warning and plays an important role in forest fire monitoring and prevention and control. Obtaining clearly visible images of smoke in the forest and automating the determination of where the smoke is located is important for strengthening forest fire prevention and control and speeding up the response to forest fire fighting. In order to obtain a clear and visible image of smoke in real time, it is essential to choose the right monitoring tool. Currently, the main tools that can acquire images of smoke in forest fires are observation towers, satellites and drones. The monitoring range of the observation towers is limited and their monitoring dead spots are numerous and coverage is inadequate. Satellite monitoring technology has a low spatial resolution and is susceptible to weather, cloud cover and orbital cycles, and its monitoring is poor in real time. In contrast, the simple structure, high flight flexibility and low acquisition costs of UAVs make them suitable as image acquisition tools in real-time forest fire monitoring.
However, there are still several pressing issues that need to be addressed in the task of detecting forest fire smoke using UAV. 1) Forest fires occur in a wide variety of scenes, and there could be many smoke-like objects, such as exposed grey-white rock and background of similar color, which have similar semantic information to smoke in the images. The conventional feature extraction network extracts less feature information, making it difficult to distinguish the feature differences among them. 2) Due to the need for actual aerial inspection by UAVs, for safety reasons, when capturing forest fire information, the UAV camera is often far from the fire source, and conventional methods can not perform well in smoke detection at a long distance. 3) The image captured by UAV is a top view, and when the UAV is at a high location, the captured image could have multiple forest fires. Conventional matrix NMS method filters each local location and has not yet combined global information, which might cause forest fire smoke images based on UAV capture to have a large deviation in the predicted location of the anchor frame. At the same time, it could also bring about missed and false detection.
In order to distinguish the feature differences between smoke and smoke-like objects, Y Cao et al. proposed an attention-enhanced bi-directional long and short-term memory network that explores the spatial and temporal features of image sequences to capture the feature differences between smoke and smoke-like objects (Cao et al., 2019). Y Hu et al. proposed the Value conversion-Attention mechanism module, which extracts deep colour and texture features of smoke by setting a joint horizontal and vertical weighting strategy to distinguish between smoke and smoke-like objects (Hu et al., 2022). While these methods are simple and effective, they are all designed for ground-based monitoring of smoke. For overhead views taken by UAVs, the characteristic relationship of smoke poses a difficulty for these methods. In order to adapt the characteristics of smoke under the conditions of overhead UAV photography and to distinguish smoke from smoke-like objects, we propose an Adjacent layer composite network (ALCN). It is proposed to parallelize two identical ResNet50-vd models in the feature extraction network. The first ResNet50-vd extracts features and transmits high-level semantic information to the second ResNet50-vd, which is fused with the low-level semantic information of the second ResNet50-vd to enhance the extraction of features with high transparency and no clear edges of smoke. In addition, the original MaxPool is replaced with SoftPool, thus preserving more feature information of the smoke image during downsampling.
In order to improve the detection of smoke at long distances, Z Jiao et al. proposed an improved UAV aerial forest fire detection method based on YOLOv3, which reduces the downsampling rate to reduce the feature loss of small objects (Jiao et al., 2019). Zhao et al. proposed small-sample smoke target detection methods based on target perception and deep convolution to improve the perception of long-range smoke. Although these methods were designed for long-range smoke and achieved certain results, their overall performance needs to be improved as they are limited to the convolution kernel angle, to some extent at the expense of close and medium-range smoke detection (Zhao et al., 2021). In order to avoid the degradation of smoke detection at close and medium distances caused by the improved detection of smoke at long distances, we propose recursive feature pyramid with deconvolution and dilated convolution(RDDFPN). First, add edges containing contextual information to the FPN module, and upsample using deconvolution to ensure the consistency of the original smoke feature information. Then, the dilated convolution is used to downsample and expand the perceptual field. Finally, the RDDFPN processed feature maps are re-entered into Backbone for recursive secondary processing to enhance feature fusion, and the lost return information from object detection is fed back more directly to adjust the parameters of Backbone to enhance the long-range smoke detection.
To reduce missed and false detections of multiple objects in an image, X Huang et al. proposed the R2NMS method, which effectively removes redundant frames without introducing large-scale false detections by using less occluded visible parts (Huang et al., 2020). H Yan et al. improved the NMS method of Faster RCNN, and the improved method raises the hard threshold of NMS by linear weighting to select the anchor frame in a weighted NMS (Yan et al., 2021). These methods improve the effectiveness of NMS to some extent when multiple objects are present in the image, but neither is designed for the characteristics of aerial photography of forest fire smoke from drones, which has a diffuse nature, and the design of both methods tends to result in diffuse smoke being incorrectly filtered out, leading to false detection. In order to reduce the missed and false detection of smoke in UAV aerial images, we propose global optimal nonmaximum suppression (GO-NMS) based on the smoke characteristics under UAV aerial photography. This method adopts the strategy of global selection of the anchor frame, sets the objective function, and iterates in such a way that the final result approaches the minimum value of the objective function to find the optimal solution. Such an image postprocessing method can adapt to the overhead view angle and improve the localization accuracy under forest fire smoke detection by UAV aerial photography, and thus, it can reduce missed and false detection.
The contributions of this paper are summarised as follows.
1) ALCN is proposed to highlight the high transparency and edge features of smoke to distinguish it from smoke-like objects. The SoftPool layer in it retains more feature information during feature extraction.
2) RDDFPN is proposed to enhance the feature extraction capability of the network, improve the long-range smoke detection capability and effectively fuse the rich feature information extracted by ALCN.
3) GO-NMS is proposed to set the objective function under the global perspective and select the optimal anchor frame through multiple rounds of iterations to improve the detection capability of multiple smoke locations under UAV aerial photography and effectively reduce missed and false detections.
Section snippets
Related work
When forest fires occur, a large amount of smoke is often produced, which is diffuse and has a wider area than a flame, therefore smoke detection is the main task in the detection of forest fires (Smith and Dragicevic, 2018). Among smoke detection methods, the main categories are: (1) manual detection methods or sensor detection methods. (2) Image processing methods: traditional image processing algorithms and deep learning methods. (3) Deep learning combined with UAV methods: UAVs in the sky
Dataset acquisition
The dataset collected in this paper consists of three major parts: a self-cropped ground dataset, a self-cropped UAV aerial photography dataset, and a self-collected UAV aerial photography dataset.
Part 1: For the task of forest fire smoke monitoring, there is no open source recognized standard dataset for smoke object detection at the pixel level. Currently, only a few datasets are publicly available, such as (1) the publicly available dataset from the Computer Vision and Pattern Recognition
Results
This section experimentally verifies the superiority of ARGNet for the UAV aerial photography forest fire smoke detection task. It consists of evaluation metrics, experimental environment and settings, ARGNet performance analysis, analysis of method effects, comparison between different models, ablation experiments, comparison of visualization results and practical application testing.
Discussion
We produced an object detection dataset for aerial photography of forest fire smoke images by UAVs, which covers a variety of features of smoke captured by UAVs at different distances when forest fires occur. Through the comparison and analysis of several sets of experiments, we verify the effectiveness of the proposed ARGNet for aerial forest fire smoke detection. In particular, it better solves the three major problems of confusing smoke with smoke-like objects, low accuracy of long-distance
Conclusion and outlook
With the increase in the value of forestry resources, preventing and controlling forest fires has become increasingly important and challenging. Strengthening forest fire smoke detection and fire prevention management is of great significance to enhance the security of forestry ecological safety. In the research of this paper, we build a UAV-IoT system to facilitate the transmission of forest fire scene images captured by UAVs to the server side for object detection. To improve the accuracy and
Funding
This work was supported by Changsha Municipal Natural Science Foundation (Grant No. kq2014160); in part by the National Natural Science Foundation in China (Grant No. 61703441); in part by the key projects of Department of Education Hunan Province (Grant No. 19A511); in part by Hunan Key Laboratory of Intelligent Logistics Technology (2019TP1015); and Natural Science Foundation of Hunan Provincial (Grant No. 2021JJ31164).
Data Availability Statement
The data presented in this study are available on request from the corresponding author. The data are not publicly available due to partial authors' disagreement.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
We are grateful to all members of the Forestry Information Research Centre for their advice and assistance in the course of this research. The language of our manuscript have been refined and polished by Elsevier Language Editing Services (Serial number: LE-221159-9DA3850979AC).
References (53)
- et al.
Fast forest fire smoke detection using MVMNet
Knowledge-Based Systems
(2022) - et al.
Smoke detection in video using wavelets and support vector machines
Fire Safety J.
(2009) - et al.
Aerial imagery pile burn detection using deep learning: the FLAME dataset
Comput. Networks
(2021) - et al.
Improved non-maximum suppression for object detection using harmony search algorithm
Appl. Soft Comput.
(2019) - et al.
Forest fire management expenditures in Canada: 1970–2013[J]
The Forestry Chronicle
(2016) - et al.
Modeling the mudslide aftermath of the 2007 Southern California Wildfires
Nat. Hazards
(2011) Negative impacts of forest fires on ecological balance and environmental sustainability: case of Turkey
Revija za geografijo
(2010)- Suthar, N., Bhavsar, A., 2021. WSN Based Prototype Architecture for Alerting the Effects of Forest Fire on Wild Animals...
- et al.
An attention enhanced bidirectional LSTM for early forest fire smoke recognition
IEEE Access
(2019) - Jiao, Z., Zhang, Y., Xin, J., Mu, L., Yi, Y., Liu, H., Liu, D., 2019. A deep learning based forest fire detection...
Fire smoke detection based on target-awareness and depthwise convolutions
Multimedia Tools Appl.
Nms by representative region: Towards crowded pedestrian detection by proposal pairing
A new face detection method based on Faster RCNN
J. Phys.: Conf. Ser.
October). A joint source channel adaptive communication system design for the fire environment. In 2017 Chinese Automation Congress (CAC)
Chemical sensor systems and associated algorithms for fire detection: A review
Sensors
Chemical sensor systems and associated algorithms for fire detection: a review
Sensors
Machine vision-based real-time early flame and smoke detection
Measur. Sci. Technol.
Xception: Deep learning with depthwise separable convolutions
Densely connected convolutional networks
Cited by (45)
SMWE-GFPNNet: A high-precision and robust method for forest fire smoke detection
2024, Knowledge-Based SystemsA triple interference removal network based on temporal and spatial attention interaction for forest smoke recognition in videos
2024, Computers and Electronics in AgricultureFireViTNet: A hybrid model integrating ViT and CNNs for forest fire segmentation
2024, Computers and Electronics in AgricultureA label-relevance multi-direction interaction network with enhanced deformable convolution for forest smoke recognition
2024, Expert Systems with ApplicationsLearning precise feature via self-attention and self-cooperation YOLOX for smoke detection
2023, Expert Systems with Applications
- 1
Jialei Zhan and Yaowen Hu contribute equally to this work.