A high-precision forest fire smoke detection approach based on ARGNet

Highlights

• A novel object detection network model is proposed for forest fire smoke detection.

• Build a UAV-IoT system to capture remote sensing images to monitor forest fires.

• The proposed method can achieve precise positioning of forest fire smoke.

Abstract

The occurrence of forest fires can lead to ecological damage, property loss, and human casualties. Current forest fire smoke detection methods do not sufficiently consider the characteristics of smoke with high transparency and no clear edges and have low detection accuracy, which cannot meet the needs of complex aerial forest fire smoke detection tasks. In this paper, we propose Adjacent layer composite network based on a recursive feature pyramid with deconvolution and dilated convolution and global optimal nonmaximum suppression (ARGNet) for high-accuracy detection of forest fire smoke. First, the Adjacent layer composite network is proposed to enhance the extraction of smoke features with high transparency and no clear edges, and SoftPool in it is used to retain more feature information of smoke. Then, a recursive feature pyramid with deconvolution and dilated convolution (RDDFPN) is proposed to fuse shallow visual features and deep semantic information in the channel dimension to improve the accuracy of long-range aerial smoke detection. Finally, global optimal nonmaximum suppression (GO-NMS) sets the objective function to globally optimize the selection of anchor frames to adapt to the aerial photography of multiple smoke locations in forest fire scenes. The experimental results show that the ARGNet parametric number on the UAV-IoT platform is as low as 53.48 M, mAP reaches 79.03%, mAP50 reaches 90.26%, mAP75 reaches 82.35%, FPS reaches 122.5, and GFLOPs reaches 55.78. Compared with other mainstream methods, it has the advantages of real-time detection and high accuracy.

Introduction

Forest fires are a hazardous and destructive disaster that often causes economic and ecological damage. In recent years, forest fires have been frequent, and governments have incurred enormous management expenditures to cope with sudden forest fires (Stocks and Martell, 2016). Because forest burning leads to bare forestland, forests lose their function of water containment and soil conservation, which will further cause other natural disasters, such as internal floods, droughts, mudslides, landslides, and dust storms (Ren et al., 2011). After forest fires occur, local ecological conditions such as weather, water and soil are out of balance and often take decades to centuries to recover (Bakirci, 2010). At the same time, forest fires can lead to the burning of rare wildlife or loss of habitat, which can pose a challenge to the biodiversity of the Earth's ecology (Suthar and Bhavsar, 2021). In recent years, the prevention and monitoring of forest fires has become a research hotspot for forest fire prevention departments around the world. The detection methods for forest fires are divided into smoke detection and flame detection. Due to the many weeds and trees in the forest environment, flames are easily blocked in the early stages of a fire. When forest fires occur, they often produce a large amount of smoke. Smoke will spread over time, which makes it easier to detect. Therefore, smoke detection is an important tool for forest fire early warning and plays an important role in forest fire monitoring and prevention and control. Obtaining clearly visible images of smoke in the forest and automating the determination of where the smoke is located is important for strengthening forest fire prevention and control and speeding up the response to forest fire fighting. In order to obtain a clear and visible image of smoke in real time, it is essential to choose the right monitoring tool. Currently, the main tools that can acquire images of smoke in forest fires are observation towers, satellites and drones. The monitoring range of the observation towers is limited and their monitoring dead spots are numerous and coverage is inadequate. Satellite monitoring technology has a low spatial resolution and is susceptible to weather, cloud cover and orbital cycles, and its monitoring is poor in real time. In contrast, the simple structure, high flight flexibility and low acquisition costs of UAVs make them suitable as image acquisition tools in real-time forest fire monitoring.

However, there are still several pressing issues that need to be addressed in the task of detecting forest fire smoke using UAV. 1) Forest fires occur in a wide variety of scenes, and there could be many smoke-like objects, such as exposed grey-white rock and background of similar color, which have similar semantic information to smoke in the images. The conventional feature extraction network extracts less feature information, making it difficult to distinguish the feature differences among them. 2) Due to the need for actual aerial inspection by UAVs, for safety reasons, when capturing forest fire information, the UAV camera is often far from the fire source, and conventional methods can not perform well in smoke detection at a long distance. 3) The image captured by UAV is a top view, and when the UAV is at a high location, the captured image could have multiple forest fires. Conventional matrix NMS method filters each local location and has not yet combined global information, which might cause forest fire smoke images based on UAV capture to have a large deviation in the predicted location of the anchor frame. At the same time, it could also bring about missed and false detection.

In order to distinguish the feature differences between smoke and smoke-like objects, Y Cao et al. proposed an attention-enhanced bi-directional long and short-term memory network that explores the spatial and temporal features of image sequences to capture the feature differences between smoke and smoke-like objects (Cao et al., 2019). Y Hu et al. proposed the Value conversion-Attention mechanism module, which extracts deep colour and texture features of smoke by setting a joint horizontal and vertical weighting strategy to distinguish between smoke and smoke-like objects (Hu et al., 2022). While these methods are simple and effective, they are all designed for ground-based monitoring of smoke. For overhead views taken by UAVs, the characteristic relationship of smoke poses a difficulty for these methods. In order to adapt the characteristics of smoke under the conditions of overhead UAV photography and to distinguish smoke from smoke-like objects, we propose an Adjacent layer composite network (ALCN). It is proposed to parallelize two identical ResNet50-vd models in the feature extraction network. The first ResNet50-vd extracts features and transmits high-level semantic information to the second ResNet50-vd, which is fused with the low-level semantic information of the second ResNet50-vd to enhance the extraction of features with high transparency and no clear edges of smoke. In addition, the original MaxPool is replaced with SoftPool, thus preserving more feature information of the smoke image during downsampling.

In order to improve the detection of smoke at long distances, Z Jiao et al. proposed an improved UAV aerial forest fire detection method based on YOLOv3, which reduces the downsampling rate to reduce the feature loss of small objects (Jiao et al., 2019). Zhao et al. proposed small-sample smoke target detection methods based on target perception and deep convolution to improve the perception of long-range smoke. Although these methods were designed for long-range smoke and achieved certain results, their overall performance needs to be improved as they are limited to the convolution kernel angle, to some extent at the expense of close and medium-range smoke detection (Zhao et al., 2021). In order to avoid the degradation of smoke detection at close and medium distances caused by the improved detection of smoke at long distances, we propose recursive feature pyramid with deconvolution and dilated convolution(RDDFPN). First, add edges containing contextual information to the FPN module, and upsample using deconvolution to ensure the consistency of the original smoke feature information. Then, the dilated convolution is used to downsample and expand the perceptual field. Finally, the RDDFPN processed feature maps are re-entered into Backbone for recursive secondary processing to enhance feature fusion, and the lost return information from object detection is fed back more directly to adjust the parameters of Backbone to enhance the long-range smoke detection.

To reduce missed and false detections of multiple objects in an image, X Huang et al. proposed the R2NMS method, which effectively removes redundant frames without introducing large-scale false detections by using less occluded visible parts (Huang et al., 2020). H Yan et al. improved the NMS method of Faster RCNN, and the improved method raises the hard threshold of NMS by linear weighting to select the anchor frame in a weighted NMS (Yan et al., 2021). These methods improve the effectiveness of NMS to some extent when multiple objects are present in the image, but neither is designed for the characteristics of aerial photography of forest fire smoke from drones, which has a diffuse nature, and the design of both methods tends to result in diffuse smoke being incorrectly filtered out, leading to false detection. In order to reduce the missed and false detection of smoke in UAV aerial images, we propose global optimal nonmaximum suppression (GO-NMS) based on the smoke characteristics under UAV aerial photography. This method adopts the strategy of global selection of the anchor frame, sets the objective function, and iterates in such a way that the final result approaches the minimum value of the objective function to find the optimal solution. Such an image postprocessing method can adapt to the overhead view angle and improve the localization accuracy under forest fire smoke detection by UAV aerial photography, and thus, it can reduce missed and false detection.

The contributions of this paper are summarised as follows.

1) ALCN is proposed to highlight the high transparency and edge features of smoke to distinguish it from smoke-like objects. The SoftPool layer in it retains more feature information during feature extraction.

2) RDDFPN is proposed to enhance the feature extraction capability of the network, improve the long-range smoke detection capability and effectively fuse the rich feature information extracted by ALCN.

3) GO-NMS is proposed to set the objective function under the global perspective and select the optimal anchor frame through multiple rounds of iterations to improve the detection capability of multiple smoke locations under UAV aerial photography and effectively reduce missed and false detections.