Abstract
It is a challenging task to accurately recognize smoke from visual scenes due to large variations in smoke shape, color and texture. To improve recognition accuracy, we propose a framework mainly with a robust local feature extraction module based on Gabor convolutional networks. We first propose a Gabor convolutional network, each layer of which mainly consists of Gabor convolution and feature fusion. To fuse feature maps generated by Gabor convolution, we present three Basic Grouping Methods, which divide the feature maps into several groups along orientation axis, scale axis and both of them. To avoid exponential growth of feature maps and preserve discriminative information simultaneously, we propose three element-wise aggregation functions, which are mean, min and max, to combine feature maps in each group. To further improve efficiency, we use local binary pattern to encode hidden and output maps of Gabor convolutional layers. In addition, we use a weight vector to optimize concatenation of histograms for further improvement. Experiments show that our method achieves very outstanding results on smoke, texture and material image datasets. Although the feature extraction step of our method is training-free, our framework consistently outperforms state-of-the-art methods on small smoke datasets, even including deep learning-based methods.
Similar content being viewed by others
References
Hwang U, Jeong J (2016) Computer vision based smoke detection method by using colour and object tracking. In: Proceedings of 5th international conference on information technology convergence and services (ITCS), Sydney, pp 53–59
Jia Y, Yuan J, Wang J, Fang J, Zhang Q, Zhang Y (2016) A saliency-based method for early smoke detection in video sequences. Fire Technol 52(5):1–22
Yu C, Fang J, Wang J, Zhang Y (2012) Video fire smoke detection using motion and color features. Fire Technol 46(3):651–663
Hu Y, Lu X (2018) Real-time video fire smoke detection by utilizing spatial-temporal ConvNet features. Multimed Tools Appl 77(8):1–19
Yuan F (2008) A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit Lett 29(7):925–932
Yuan F (2012) A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection. Pattern Recognit 45(12):4326–4336
Maruta H, Nakamura A, Kurokawa F (2010) A new approach for smoke detection with texture analysis and support vector machine. In: Proceedings of IEEE international symposium industrial electronics, Bari, Italy, pp 1550–1555
Nguyen T, Kim J (2013) Multistage optical smoke detection approach for smoke alarm systems. Opt Eng 52(5):1–12
Yuan F (2011) Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf J 46(3):132–139
Yuan F, Shi J, Xia X, Fang Y, Fang Z, Mei T (2016) High-order local ternary patterns with locality preserving projection for smoke detection and image classification. Inf Sci 372:225–240
Yuan F, Shi J, Xia X, Yang Y, Fang Y, Wang R (2016) Sub oriented histograms of local binary patterns for smoke detection and texture classification. KSII Trans Internet Inf Syst 10(4):1807–1823
Yuan F, Xia X, Shi J, Li H, Li G (2017) Non-linear dimensionality reduction and Gaussian process based classification method for smoke detection. IEEE Access 5:6833–6841
Zhao Y, Zhou Z, Xu M (2015) Forest fire smoke video detection using spatiotemporal and dynamic texture features. J Electr Comput Eng 2015:1–7
Calderara S, Piccinini P, Cucchiara R (2011) Vision based smoke detection system using image energy and color information. Mach Vis Appl 22:705–719
Gonzalez-Gonzalez R, Alarcon-Aquino V, Rosas-Romero R, Starostenko O, Rodriguez-Asomoza J, Ramirez-Cortes J (2010) Wavelet-based smoke detection in outdoor video sequences. In: Proceedings of 53rd international midwest symposium on circuits and systems, midwest symposium on circuits and systems conference proceedings, IEEE, pp 383–387
Gubbi J, Marusica S, Palaniswamia M (2009) Smoke detection in video using wavelets and support vector machines. Fire Saf J 44(8):1110–1115
Maruta H, Kato Y, Nakamura A, Kurokawa F (2009) Smoke detection in open areas using its texture features and time series properties. In: Proceedings of IEEE international symposium on industrial electronics (ISIE), IEEE, pp 1904–1908
Toreyin BU, Dedeoglu Y, Cetin A (2005) Wavelet based real-time smoke detection in video. In: Proceedings of European signal processing conference, Antalya, pp 4–8
Ye S, Bai Z, Chen H, Bohush R, Ablameyko S (2017) An effective algorithm to detect both smoke and flame using color and wavelet analysis. Pattern Recognit Image Anal 27(1):131–138
Tung T, Kim J (2011) An effective four-stage smoke-detection algorithm using video images for early fire-alarm systems. Fire Saf J 46(5):276–282
Yin Z, Wan B, Yuan F, Xia X, Shi J (2017) A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5:18429–18438
Yuan F, Fang Z, Wu S, Yang Z, Fang Y (2015) A real-time video smoke detection using staircase searching based dual threshold AdaBoost and dynamic analysis. IET Image Proc 9(10):849–856
Dora L, Agrawal S, Panda R, Abraham A (2017) An evolutionary single Gabor kernel based filter approach to face recognition. Eng Appl Artif Intell 62:286–301
Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans Image Process 11:467–476
Su Y, Shan S, Chen X, Gao W (2009) Hierarchical ensemble of global and local classifiers for face recognition. IEEE Trans Image Process 18:1885–1896
Hwang W, Huang X, Li S, Kim J (2015) Face recognition using extended curvature Gabor classifier bunch. Pattern Recognit 48(4):1247–1260
Li C, Duan G, Zhong F (2015) Rotation invariant texture retrieval considering the scale dependence of Gabor wavelet. IEEE Trans Image Process 24(8):2344–2354
Jie Z, Qiang J, Nagy G (2007) A comparative study of local matching approach for face recognition. IEEE Trans Image Process 16:2617–2628
Cament L, Castillo L, Perez J, Galdames F, Perez C (2014) Fusion of local normalization and Gabor entropy weighted features for face identification. Pattern Recognit 47(2):568–577
Shen L, Bai L, Auer D (2008) 3D Gabor wavelets for evaluating SPM normalization algorithm. Med Image Anal 12(3):375–383
Zhan W, Shan S, Gao W, Chen X, Zhang H (2005) Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition. In: Proceedings of 10th IEEE international conference on computer vision (ICCV), pp 786–791
Elbakary M, Sundareshan M (2005) Accurate representation of local frequency using a computationally efficient Gabor filter fusion approach with application to image registration. Pattern Recognit Lett 26(14):2164–2173
Wang H, Qi H, Wang X (2013) A new Gabor based approach for wood recognition. Neurocomputing 116:192–200
Zhang B, Shan S, Chen X, Gao W (2007) Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. IEEE Trans Image Process 16:57–68
Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: Proceedings of advances in neural information processing systems, vol 25, pp 1097–1105
Zeiler M, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of European conference on computer vision (ECCV), pp 818–833
Balntas V, Riba E, Ponsa D, Mikolajczyk K (2016) Learning local feature descriptors with triplets and shallow convolutional neural networks. In: Proceedings of British machine vision conference (BMVC), vol 1, No. 2. pp 119.1–119.11
Hu Y, Li C, Hu D, Yu W (2016) Gabor feature based convolutional neural network for object recognition in natural scene. In: Proceedings of international conference on information science and control engineering, IEEE, pp 386–390
Wiskott L, Fellous JM, Kuiger N, Malsburg CVD (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779
Ojala T, Pietikäinen M, Mäenpää T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary pattern. Eur Conf Comput Vis 24(7):404–420
Caputo B, Hayman E, Mallikarjuna P (2005) Class-specific material categorisation. In: Proceedings of 10th IEEE international conference on computer vision, IEEE, vol 2, pp 1597–1604
Guo Z, Zhang L, Zhang D (2010) A completed modeling of local binary pattern operator for texture classification. IEEE Trans Image Process 19(6):1657–1663
Murala S, Maheshwari R, Balasubramanian R (2012) Local tetra patterns: A new feature descriptor for content-based image retrieval. IEEE Trans Image Process 21(5):2874–2886
Qian X, Hua X, Chen P, Ke L (2011) PLBP: An effective local binary patterns texture descriptor with pyramid representation. Pattern Recognit 44(10–11):2502–2515
Zhang B, Gao Y, Zhao S, Liu J (2013) Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans Image Process 19(2):533–544
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556
Qi X, Xiao R, Li C, Qiao Y, Guo J, Tang X (2014) Pairwise rotation invariant co-occurrence local binary pattern. IEEE Trans Pattern Anal Mach Intell 36(11):2199–2211
Ahonen T, Matas J, He C, Pietikäinen M (2009) Rotation invariant image description with local binary pattern histogram Fourier features. In: Proceedings of 16th image analysis, Scandinavian conference, Oslo, Norway, pp 61–70
Zhang J, Liang J, Zhao H (2013) Local energy pattern for texture classification using self-adaptive quantization thresholds. IEEE Trans Image Process 22(1):31–42
Crosier M, Griffin LD (2010) Using basic image features for texture classification. Int J Comput Vis 88(3):447–460
Sharma G, Hussain SU, Frédéric J (2012) Local higher-order statistics (LHS) for texture categorization and facial analysis. In: Proceedings of computer vision—European conference on computer vision. Springer, Berlin, pp 1–12
Hong X, Zhao G, Pietikäinen M, Chen X (2014) Combining LBP difference and feature correlation for texture description. IEEE Trans Image Process 23(6):2557–2568
Ryu J, Hong S, Yang H (2015) Sorted consecutive local binary pattern for texture classification. IEEE Trans Image Process 24(7):2254–2265
Mehta R, Egiazarian K (2016) Texture classification using dense micro-block difference. IEEE Trans Image Process 25(4):1604–1616
Chen J, Shan S, He C, Zhao G, Pietikainen M, Chen X (2010) WLD: a robust local image descriptor. IEEE Trans Pattern Anal Mach Intell 32(9):1705–1720
Acknowledgements
This work was partially supported by the National Natural Science Foundation of China (61862029), the Open Project of Hubei Key Laboratory of Intelligent Vision Based Monitoring for Hydroelectric Engineering (2018SDSJ01), the Natural Science Foundation of Jiangxi Province (20192BAB207011) and the Science Technology Projects of Jiangxi Education Department (GJJ170317, GJJ170892, GJJ180206).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Yuan, F., Li, G., Xia, X. et al. Encoding features from multi-layer Gabor filtering for visual smoke recognition. Pattern Anal Applic 23, 1117–1131 (2020). https://doi.org/10.1007/s10044-020-00864-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-020-00864-x