Skip to main content
SpringerLink
Log in

Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition

  • Original paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Traditional smoke recognition methods are mainly based on handcrafted features. However, it is difficult to design handcrafted features that are robust and discriminative for smoke recognition because of large variations in smoke color, shapes and textures. To solve this problem, we specifically design a basic block of convolutional neural networks (CNNs) and stack basic blocks to propose a novel deep multi-scale CNN (DMCNN) for smoke recognition. The basic block consists of several parallel convolutional layers with the same number of filters but different kernel sizes for scale invariance. Each convolutional layer is followed by a batch normalization to normalize the output of the convolutional layer. Then the basic block sums up all normalized outputs from multi-scale parallel layers and activates the sum as the final output of the block. To fully extract scale invariant features, we cascade eleven basic blocks, which is followed by a global average pooling and a 2D fully connected layer, to construct DMCNN. Experimental results show that our method achieves higher detection rates, higher accuracy rates and lower false alarm rates than existing methods. To further verify the efficiency of DMCNN, we also conducted face gender recognition experiments on the LFW database and our model also achieves obviously higher accuracy rates than other methods. Furthermore, our method is an efficient, lightweight CNN model with about 1 M parameters that are far less than other CNN methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Gubbi, J., Marusic, S., Palaniswami, M.: Smoke detection in video using wavelets and support vector machines. Fire Saf. J. 44(8), 1110–1115 (2009)

    Article  Google Scholar 

  2. Ferrari, R.J., Zhang, H., Kube, C.R.: Real-time detection of steam in video images. Pattern Recognit. 40(3), 1148–1159 (2007)

    Article  MATH  Google Scholar 

  3. Ye, S., Bai, Z., Chen, H., Bohush, R., Ablameyko, S.: An effective algorithm to detect both smoke and flame using color and wavelet analysis. Pattern Recognit. Image Anal. 27(1), 131–138 (2017)

    Article  Google Scholar 

  4. Yu, C., Fang, J., Wang, J., Zhang, Y.: Video fire smoke detection using motion and color features. Fire Technol. 46(3), 651–663 (2010)

    Article  Google Scholar 

  5. Yuan, F.: A fast accumulative motion orientation model based on integral image for video smoke detection. Pattern Recognit. Lett. 29(7), 925–932 (2008)

    Article  Google Scholar 

  6. Yuan, F.: Video-based smoke detection with histogram sequence of LBP and LBPV pyramids. Fire Saf. J. 46(3), 132–139 (2011)

    Article  Google Scholar 

  7. Yuan, F., Shi, J., Xia, X., Yang, Y., Fang, Y., Wang, R.: Sub oriented histograms of local binary patterns for smoke detection and texture classification. KSII Trans. Internet Inf. Syst. 10(4), 1807–1823 (2016)

    Google Scholar 

  8. Yuan, F.: A double mapping framework for extraction of shape-invariant features based on multi-scale partitions with AdaBoost for video smoke detection. Pattern Recognit. 45(12), 4326–4336 (2012)

    Article  Google Scholar 

  9. Yuan, F.: Rotation and scale invariant local binary pattern based on high order directional derivatives for texture classification. Digit. Signal Process. 26, 142–152 (2014)

    Article  Google Scholar 

  10. Yuan, F., Shi, J., Xia, X., Fang, Y., Fang, Z., Mei, T.: High-order local ternary patterns with locality preserving projection for smoke detection and image classification, information sciences. Inf. Sci. 372, 225–240 (2016)

    Article  Google Scholar 

  11. Yuan, F., Fang, Z., Wu, S., Yang, Y., Fang, Y.: A real-time video smoke detection using staircase searching based dual threshold AdaBoost and dynamic analysis. IET Image Process. 9(10), 849–856 (2015)

    Article  Google Scholar 

  12. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  13. Szeged, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.: Going deeper with convolutions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9. IEEE (2015)

  14. He, K., Zhang, X.: Deep residual learning for image recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778. IEEE (2016)

  15. Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. arXiv preprint arXiv:1608.06993 (2016)

  16. Yan, K., Huang, S., Song, Y., Liu, W., Fan, N.: Face recognition based on convolution neural network. In: 2017 36th Chinese Control Conference (CCC), pp. 4077–4081. IEEE (2017)

  17. Kang, B.N., Kim, Y., Kim, D.: Deep convolution neural network with stacks of multi-scale convolutional layer block using triplet of faces for face recognition in the wild. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 4460–4465. IEEE (2016)

  18. Chen, L., Guo, X., Geng, C.: Human face recognition based on adaptive deep convolution neural network. In: 2016 35th Chinese Control Conference (CCC), pp. 6967–6970. IEEE (2016)

  19. Chen, T., Lu, S., Fan, J.: S-CNN: subcategory-aware convolutional networks for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 99, 1 (2018). https://doi.org/10.1109/tpami.2017.2756936

    Article  Google Scholar 

  20. Li, X., Wang, S.: Object detection using convolutional neural networks in a coarse-to-fine manner. IEEE Geosci. Remote Sens. Lett. 14(11), 2037–2041 (2017)

    Article  MathSciNet  Google Scholar 

  21. Feng, J., Lim, F., Lu, S., Liu, J., Ma, D.: Injurious or noninjurious defect identification from MFL images in pipeline inspection using convolutional neural network. IEEE Trans. Instrum. Meas. 66(7), 1883–1892 (2017)

    Article  Google Scholar 

  22. Hubel, D.H., Wiesel, T.N.: Receptive fields binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106–154 (1962)

    Article  Google Scholar 

  23. Guo, Y., Liu, Y., Oerlemans, A.: Deep learning for visual understanding: a review. Neurocomputing 187, 27–48 (2016)

    Article  Google Scholar 

  24. Yin, Z., Wang, B., Yuan, F., Xia, X., Shi, J.: A deep normalization and convolutional neural network for image smoke detection. IEEE Access 5, 18429–18438 (2017)

    Article  Google Scholar 

  25. Ding, X., He, Q.: Energy-fluctuated multiscale feature learning with deep convnet for intelligent spindle bearing fault diagnosis. IEEE Trans. Instrum. Meas. 66(8), 1926–1935 (2017)

    Article  Google Scholar 

  26. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)

    Article  Google Scholar 

  27. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  28. Wang, Q., Wan, J., Yuan, Y.: Deep metric learning for crowdedness regression. IEEE Trans. Circuits Syst. Video Technol (T-CSVT) (2017). https://doi.org/10.1109/tcsvt.2017.2703920

    Article  Google Scholar 

  29. Wang, Q., Gao, J., Yuan, Y.: A joint convolutional neural networks and context transfer for street scenes labeling. IEEE Trans. Intell. Transp. Syst. 19(5), 1457–1470 (2018)

    Article  Google Scholar 

  30. Wang, Q., Yuan, Z., Li, X.: GETNET: a general end-to-end two-dimensional CNN framework for hyperspectral image change detection. IEEE Trans. Geosci. Remote Sens. (T-GRS) (2018). https://doi.org/10.1109/tgrs.2018.2849692

    Article  Google Scholar 

  31. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: European Conference on Computer Vision, pp. 818–833 (2014)

  32. Shi, J., Yuan, F., Xue, X.: Video smoke detection: a literature survey. J. Image Graph. 23(3), 303–322 (2018)

    Google Scholar 

  33. Vieira, D.A.G., Santos, A.L.D., Yehia, H.C., et al.: Smoke detection in environmental regions by means of computer vision. In: Proceedings of the 4th International Workshop, pp. 135–151. Springer, Cham (2016)

  34. Zhao, L., Luo, Y.M., Luo, X.Y.: Based on dynamic background update and dark channel prior of fire smoke detection algorithm. Appl. Res. Comput. 32(3), 957–960 (2017)

    Google Scholar 

  35. Prema, C.E., Vinsley, S.S., Suresh, S.: Multi feature analysis of smoke in YUV color space for early forest fire detection. Fire Technol. 52(5), 1319–1342 (2016)

    Article  Google Scholar 

  36. Zhou, Z.Q., Shi, Y.S., Gao, Z.F., et al.: Wildfire smoke detection based on local extremal region segmentation and surveillance. Fire Saf. J. 85, 50–58 (2016)

    Article  Google Scholar 

  37. Zhang, Q., Xu, J., Xu, L., Guo, H.: Deep convolutional neural networks for forest fire detection. In: Proceedings of the 2016 International Forum on Management, Education and Information Technology Application. Atlantis Press (2016)

  38. Frizzi, S., Kaabi, R., Bouchouicha, M., et al.: Convolutional neural network for video fire and smoke detection. In: IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society, pp. 877–882. IEEE (2016)

  39. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reduce internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)

  40. Zhang, Q., Lin, G., Zhang, Y., Xu, G., Wang, J.: Wildland forest fire smoke detection based on faster R-CNN using synthetic smoke images. Procedia Eng. 211, 441–446 (2018)

    Article  Google Scholar 

  41. Muhammad, K., Ahmad, J., Mehmood, I., Rho, S., Baik, S.W.: Convolutional neural networks based fire detection in surveillance videos. IEEE Access 6, 18174–18183 (2018)

    Article  Google Scholar 

  42. Gu, J., Wang, Z., Kuen, J.: Recent advance in convolutional neural networks. Pattern Recognit. 77, 354–377 (2018)

    Article  Google Scholar 

  43. Zhou, F., Jin, L., Dong, J.: Review of convolutional neural network. Chin. J. Comput. 40(6), 1229–1251 (2017)

    MathSciNet  Google Scholar 

  44. Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolution network. arXiv preprint arXiv:1505.00853 (2015)

  45. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807–814 (2010)

  46. Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net. arXiv preprint arXiv:1412.6806 (2014)

  47. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

  48. Chu, J.L., Krzyzak, A.: Analysis of feature maps selection in supervised learning using convolutional neural networks. In: Canadian Conference on Artificial Intelligence, pp. 59–70 (2014)

  49. Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, vol. 1, no. 2 (2007)

  50. Tian, Q., Arbel, T., Clark, J.J.: Deep LDA-pruned nets for efficient facial gender classification. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 512–521. IEEE (2017)

  51. Mahmood, S.F., Marhaban, M.H., Rokhani, F.Z., Samsudin, K., Arigbabu, O.A.: FASTA-ELM: a fast adaptive shrinkage/thresholding algorithm for extreme learning machine and its application to gender recognition. Neurocomputing 219, 312–322 (2017)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by Natural Science Foundation of China (61862029), Science Technology Application Project of Jiangxi Province (KJLD12066) and Science Technology Projects of Jiangxi Province (GJJ170317).

Author information

Authors and Affiliations

  1. School of Information Technology, Jiangxi University of Finance and Economics, Nanchang, 330032, Jiangxi, China

    Feiniu Yuan, Lin Zhang, Boyang Wan & Xue Xia

  2. College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai, 201418, China

    Feiniu Yuan

  3. School of Mathematics and Computer Science, Jiangxi Science and Technology Normal University, Nanchang, 330045, Jiangxi, China

    Lin Zhang

  4. Vocational School of Teachers and Technology, Jiangxi Agricultural University, Nanchang, 330045, Jiangxi, China

    Jinting Shi

Authors
  1. Feiniu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Boyang Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xue Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinting Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, F., Zhang, L., Wan, B. et al. Convolutional neural networks based on multi-scale additive merging layers for visual smoke recognition. Machine Vision and Applications 30, 345–358 (2019). https://doi.org/10.1007/s00138-018-0990-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-018-0990-3

Keywords

Search

Navigation