Abstract
Smoke detection plays an essential role in the wild video surveillance systems for abnormal events warning. In this paper, we introduced a dedicated neural network structure named Sniffer-Net to simultaneously extract smoke dynamic feature robustly and evaluate the smoke concentration accurately. Firstly, we utilize an improved LiteFlowNet to estimate the global optical flow from image sequence. Meanwhile, a Marr–Hildreth method is brought up and fused into this network to distinguish and eliminate occluded regions from global flow map. Then, an evaluation module based on Context-Encoder network is put forward specially to quantify smoke concentration levels. This network, following the improved LiteFlowNet, is modified through replacing the loss function and removing the multiscale scheme and trained to infer approximate smoke optical flow behind occlusion regions. Starting from the statistical view, the irregular RGB/HSV feature spaces are converted into a specific quantitative evaluation space. As a result, the whole evaluation system is responsible to transform the distribution of irregular smoke motion feature into a quantified form of representation. In turn, this transformation endows the system with a novel numerical standard for smoke concentration evaluation. Finally, an accuracy assessment method is applied to compare the results of detected smoke concentration with the human experience prior model, which feedback the accuracy and false detection rate of system algorithm. In the experiments of five smoke datasets, our proposed smoke detection approach is superior to other state-of-the-art methods, and concentration algorithm achieves the satisfactory performance of 97.3% accuracy on some specialized dataset.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Çetin AE, Dimitropoulos K, Gouverneur B et al (2013) Video fire detection—review. Digit Signal Process A Rev J. https://doi.org/10.1016/j.dsp.2013.07.003
Lee CY, Lin CT, Hong CT (2009) Spatio-temporal analysis in smoke detection. In: ICSIPA09—2009 IEEE international conference on signal and image processing applications, conference proceedings
Hui T-W, Tang X, Loy CC (2018) LiteFlowNet: a lightweight convolutional neural network for optical flow estimation, 4–6. https://doi.org/10.1109/CVPR.2018.00936
Ferrari RJ, Zhang H, Kube CR (2007) Real-time detection of steam in video images. Pattern Recognit. https://doi.org/10.1016/j.patcog.2006.07.007
Maruta H, Nakamura A, Kurokawa F (2010) A new approach for smoke detection with texture analysis and support vector machine. In: IEEE international symposium on industrial electronics
Gunay O, Toreyin BU, Kose K, Cetin AE (2012) Entropy-functional-based online adaptive decision fusion framework with application to wildfire detection in video. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2012.2183141
Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J. https://doi.org/10.1016/j.firesaf.2011.01.001
Zhang H, Sun Y, Liu L et al (2018) ClothingOut: a category-supervised GAN model for clothing segmentation and retrieval. Neural Comput Appl. https://doi.org/10.1007/s00521-018-3691-y
Tian H, Li W, Wang L, Ogunbona P (2012) A novel video-based smoke detection method using image separation. In: Proceedings—IEEE international conference on multimedia and expo
Chen TH, Yin YH, Huang SF, Ye YT (2006) The smoke detection for early fire-alarming system base on video processing. In: Proceedings—2006 international conference on intelligent information hiding and multimedia signal processing, IIH-MSP 2006
Ma L, Wu K, Zhu L (2010) Fire smoke detection in video images using Kalman filter and Gaussian mixture color model. In: Proceedings—international conference on artificial intelligence and computational intelligence, AICI 2010
Kopilovic I, Vagvolgyi B, Sziranyi T (2000) Application of panoramic annular lens for motion analysis tasks: surveillance and smoke detection. In: Proceedings of 15th International Conference on Pattern Recognition ICPR-2000, vol 4, pp 714–717. https://doi.org/10.1109/ICPR.2000.903017
Alamgir N, Nguyen K, Chandran V, Boles W (2018) Combining multi-channel color space with local binary co-occurrence feature descriptors for accurate smoke detection from surveillance videos. Fire Saf J. https://doi.org/10.1016/j.firesaf.2018.09.003
Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2008.01.013
Xiong Z, Caballero R, Wang H, et al (2007) Video-based smoke detection: possibilities, techniques, and challenges. In: Proceedings of suppression and detection research and applications
Feiniu Y, et al (2008) Video smoke detection based on accumulation and main motion orientation. J Image Graphics (in Chinese)
Lee CY, Lin CT, Hong CT, Su MT (2012) Smoke detection using spatial and temporal analyses. Int J Innov Comput Inf Control 8:4749–4770
Deldjoo Y, Nazary F, Fotouhi AM (2015) A novel fuzzy-based smoke detection system using dynamic and static smoke features. In: ICEE 2015—proceedings of the 23rd Iranian conference on electrical engineering
Toreyin B, Dedeoglu Y (2005) Wavelet based real-time smoke detection in video. In: 13th European signal processing conference
Surit S, Chatwiriya W (2011) Forest fire smoke detection in video based on digital image processing approach with static and dynamic characteristic analysis. In: Proceedings—1st ACIS/JNU international conference on computers, networks, systems, and industrial engineering, CNSI 2011
Toreyin BU, Dedeoglu Y, Cetin AE (2006) Contour based smoke detection in video using wavelets. In: European signal processing conference
Appana DK, Islam R, Khan SA, Kim JM (2017) A video-based smoke detection using smoke flow pattern and spatial-temporal energy analyses for alarm systems. Inf Sci (N Y). https://doi.org/10.1016/j.ins.2017.08.001
Kolesov I, Karasev P, Tannenbaum A, Haber E (2010) Fire and smoke detection in video with optimal mass transport based optical flow and neural networks. In: Proceedings—international conference on image processing, ICIP
Wang Z, Ben Salah M, Zhang H (2012) Discrete wavelet transform based steam detection with Adaboost. In: 2012 IEEE international conference on information and automation, ICIA 2012
Li X, Chen Z, Wu QMJ, Liu C (2018) 3D parallel fully convolutional networks for real-time video wildfire smoke detection. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2018.2889193
Xu G, Zhang Y, Zhang Q et al (2019) Video smoke detection based on deep saliency network. Fire Saf J 105:277–285. https://doi.org/10.1016/j.firesaf.2019.03.004
Ji Y, Zhang H, Jonathan Wu QM (2018) Saliency detection via conditional adversarial image-to-image network. Neurocomputing. https://doi.org/10.1016/j.neucom.2018.08.013
Yin Z, Shi J (2018) GeoNet: unsupervised learning of dense depth, optical flow and camera pose. https://doi.org/10.1109/CVPR.2018.00212
Tu Z, Xie W, Cao J et al (2017) Variational method for joint optical flow estimation and edge-aware image restoration. Pattern Recognit. https://doi.org/10.1016/j.patcog.2016.10.027
Tu Z, Poppe R, Veltkamp RC (2016) Adaptive guided image filter for warping in variational optical flow computation. Signal Process. https://doi.org/10.1016/j.sigpro.2016.02.018
Yang C, Lu X, Lin Z et al (2017) High-resolution image inpainting using multi-scale neural patch synthesis. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. Clin Toxicol. https://doi.org/10.2507/daaam.scibook.2010.27
Ilg E, Mayer N, Saikia T et al (2017) FlowNet 20: evolution of optical flow estimation with deep networks. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017
Song Y, Yang C, Lin Z et al (2018) Contextual-based image inpainting: infer, match, and translate. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Ko BC, Cheong KH, Nam JY (2009) Fire detection based on vision sensor and support vector machines. Fire Saf J. https://doi.org/10.1016/j.firesaf.2008.07.006
Grammalidis N, Dimitropoulos K, Cetin E (2017) FIRESENSE database of videos for flame and smoke detection. IEEE Trans Circuits Syst Video Technol 25:339–351. https://doi.org/10.5281/ZENODO.836749
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (No. 51805312); in part by Shanghai Sailing Program (No. 18YF1409400); in part by Training and Funding Program of Shanghai College Young Teachers (No. ZZGCD15102), and in part by Scientific Research Project of Shanghai University of Engineering Science (No. 2016-19).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interests regarding the publication of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Mi, Z., Zhang, W., Wu, X. et al. Sniffer-Net: quantitative evaluation of smoke in the wild based on spatial–temporal motion spectrum. Neural Comput & Applic 32, 9165–9180 (2020). https://doi.org/10.1007/s00521-019-04426-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-019-04426-z