Skip to main content
SpringerLink
Log in

Hybrid nonlinear convolution filters for image recognition

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Typical convolutional filter only extract features linearly. Although nonlinearities are introduced into the feature extraction layer by using activation functions and pooling operations, they can only provide point-wise nonlinearity. In this paper, a Gaussian convolution for extracting nonlinear features is proposed, and a hybrid nonlinear convolution filter consisting of baseline convolution, Gaussian convolution and other nonlinear convolutions is designed. It can efficiently achieve the fusion of linear features and nonlinear features while preserving the advantages of traditional linear convolution filter in feature extraction. Extensive experiments on the benchmark datasets MNIST, CIFAR10, and CIFAR100 show that the hybrid nonlinear convolutional neural network has faster convergence and higher image recognition accuracy than the traditional baseline convolutional neural network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. LeCun Y, Boser B, Denker J, Henderson D (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput, pp 541–551

  2. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradientbased learning applied to document recognition. Proc IEEE, pp 2278–2324

  3. DiCarlo J, Zoccolan D, Rust N (2012) How does the brain solve visual object recognition. Neuron, pp 415–434

  4. Ren Shaoqing, He Kaiming, Girshick Ross, Sun Jian (2017) Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell, pp 1137–1149

  5. Dong C, Loy C, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proc Eur Conf Comput Vis, pp 184–199

  6. Wang N, Yeung D (2013) Learning a deep compact image representation for visual tracking. In: Proc Adv Neural Inf Process Syst, pp 809–817

  7. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 3431–3440

  8. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with Deep Convolutional Neural Networks. In: Conference on Neural Information Processing Systems, pp 1097–1105

  9. Russakovsky O, Deng J, Su H, Krause J (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3):211–252

    Article  MathSciNet  Google Scholar 

  10. Hubel D, Wiesel T (1968) Receptive fields and functional architecture of monkey striate cortex. J Physiol 195(1):215–243

    Article  Google Scholar 

  11. Rapela J, Mendel J, Grzywacz N (2006) Estimating nonlinear receptive fields from natural images. J Vis, 441–474

  12. Lin T-Y, Chowdhury AR, Maji S (2015) Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE international conference on computer vision,pp 1449–1457

  13. Niell C, Stryker M (2008) Highly selective receptive fields in mouse visual cortex. J Neurosci 28(30):7520–7536

    Article  Google Scholar 

  14. Cui Y, Zhou F, Wang J, Liu X, Lin Y, Belongie S (2017) Kernel pooling for convolutional neural networks. In: IEEE conference on computer vision and pattern RECOGNITION (CVPR), pp 3049–3058

  15. Zoumpourlis Georgios, Doumanoglou Alexandros, Vretos Nicholas, Daras Petros (2017) Non-linear convolution filters for CNN-based learning. IEEE International Conference on Computer Vision, pp 4771–4779

  16. Yuan C, Wu Y, Qin X (2019) An effective image classification method for shallow densely connected convolution networks through squeezing and splitting techniques. Appl Intell, 3570–3586

  17. Guyon I, Weston J, Barnhill S (2002) Gene selection for cancer classification using support vector machines. Mach. Learn, pp 389–422

  18. Szulborski R, Palmer L (1990) The two-dimensional spatial structure of nonlinear subunits in the receptive fields of complex cells. Vis Res 30(2):249–254

    Article  Google Scholar 

  19. Qian N. (1999) On the momentum term in gradient descent learning algorithms. Neural Networks : The Official Journal of the International Neural Network Society 12(1):145–151

    Article  Google Scholar 

  20. Kingma D, Ba J (2015) Adam: a method for stochastic optimization. International conference on learning representations, pp 1–13

  21. Hubel D, Wiesel T (1998) Early exploration of the visual cortex. Neuron 20(3):401–412

    Article  Google Scholar 

  22. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, pp 1–14 arXiv:1409.1556

  23. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions, pp 1–8, arXiv:1610.02357

  24. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, Springer, pp 630–645

  25. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition, pp 2261–2269

  26. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: IEEE conference on computer vision and pattern recognition, pp 7132–7141

  27. Yang Y, Zhong Z, Shen T, Lin Z (2018) Convolutional neural networks with alternately updated clique, pp 1–10, arXiv:1802.10419

  28. Yang B, Bender G, Ngiam J (2019) CondConv: Conditionally parameterized convolutions for efficient inference, pp 1–12, arXiv:1904.04971

  29. Ding X, Guo Y, Ding G, Han J (2019) ACNet: Strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks, pp 1–10, arXiv:1908.03930

  30. Lavin A, Gray S (2015) Fast algorithms for convolutional neural networks, pp 1–9, arXiv:1509.09308

  31. Zhang T, Qi G, Xiao B, Wang J (2017) Interleaved group convolutions for deep neural networks, pp 1–11, arXiv:1707.02725

  32. Howard A G, Zhu M, Chen B (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications, pp 1–9, arXiv:1704.04861

  33. Hui J, Shi L, Zhang P, He S (2016) The neural mechanism of visual consciousness in human brain. Progress Biochem Biophys 43(04):297–307

    Google Scholar 

  34. Li Q, Geng H (2008) Research progress of cognitive neuroscience of visual awareness. Progress in Natural Science (11):1211–1219

  35. Wen B, Dong W, Xie W (2018) Parameter optimization method for random forest based on improved grid search algorithm. Computer Engineering and Applications 50(10):154–157

    Google Scholar 

  36. Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281–305

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is supported by the Natural Science Foundation of Hebei Province (E2015203354), Science and Technology Research Key Project of High School of Hebei Province (ZD2016100) and Basic Research Special Breeding Project Supported by Yanshan University (16LGY015). We also thank MINST and CIFAR for their open-source datasets.

Author information

Authors and Affiliations

  1. Key Laboratory of Industrial Computer Control Engineering of Hebei Province, Yanshan University, Qinhuangdao, 066004, China

    Xiuling Zhang, Kailun Wei, Xuenan Kang & Jinxiang Li

Authors
  1. Xiuling Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kailun Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xuenan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinxiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiuling Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Wei, K., Kang, X. et al. Hybrid nonlinear convolution filters for image recognition. Appl Intell 51, 980–990 (2021). https://doi.org/10.1007/s10489-020-01845-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01845-7

Keywords

Search

Navigation