Skip to main content
SpringerLink
Log in

An effective image classification method for shallow densely connected convolution networks through squeezing and splitting techniques

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The main drawbacks of traditional densely connected convolution networks (DenseNet) lie in: complex network models, excessive parameters, a large amount of computational and storage resources, falling into the problem of over-fitting, resulting in low object recognition accuracy. In addition, in the field of fine-grained image classification, the recognition performance is insufficient due to the inadequate representation capability of extracting features. In order to cope with these problems, we propose a novel shallow densely connected convolution networks (called DenseNet-S), it works as: (1) we adopt a shallow network training strategy to degrade the computational complexity and reduce the parameters, in order to avoid excessive number of layers affecting the recognition accuracy; (2) we propose a novel squeeze method to further reduce the network parameters and effectively alleviate the over-fitting phenomena. In addition, we apply the fire module and add the squeeze layer and the expand layer to the convolution module in DenseNet; (3) we employ the factorization technique into small convolutions, which can partition a large two-dimensional convolution into two small one-dimensional convolutions, in order to improve the feature extraction capability and the recognition performance in fine-grained image classification. The effectiveness of DenseNet-S was evaluated by extensive experiments on three benchmark datasets including CIFAR-10, CIFAR-100 and SVHN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Lecun Y, Boser B, Denker J, Henderson D (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541–551

    Article  Google Scholar 

  2. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  3. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105

  4. Ren S, He K, Girshick R, Sun J (2015) Faster r-CNN: towards real-time object detection with region proposal networks. In: International conference on neural information processing systems, pp 91–99

  5. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 3431–3440

  6. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323

  7. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. pp 1–14, CoRR, arXiv:1409.1556

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. pp 770–778

  9. Huang G, Sun Y, Liu Z, Sedra D, Weinberger KQ (2016) Deep networks with stochastic depth. pp 646–661

  10. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, Springer, pp 630–645

  11. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: IEEE conference on computer vision and pattern recognition, pp 2261–2269

  12. Lin M, Chen Q, Yan S (2013) Network in network. pp 1–10, arXiv:1312.4400

  13. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D (2014) Vincent vanhoucke, and andrew rabinovich. Going deeper with convolutions, pp 1–9

  14. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: IEEE conference on computer vision and pattern recognition, pp 2818–2826

  15. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. arXiv preprint pp 1610–02357

  16. Sifre L, Mallat S (2014) Rigid-motion scattering for image classification. PhD thesis, Citeseer

  17. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications. pp 1–9, arXiv:1704.04861

  18. Zhang X, Zhou X, Lin M, Sun J (2017) ShuffleNet: An extremely efficient convolutional neural network for mobile devices. pp 1–9, arXiv:1707.01083

  19. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and 0.5mb model size. pp 1–13, arXiv:1602.07360

  20. Deng J, Dong W, Socher R, Li J (2009) ImageNet: A large-scale hierarchical image database. In: 2009. CVPR 2009. IEEE conference on computer vision and pattern recognition, pp 248–255

  21. Krizhevsky A, Hinton GE (2009) Learning multiple layers of features from tiny images. Technical report, Citeseer

  22. Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng AY (2011) Reading digits in natural images with unsupervised feature learning. In: NIPS workshop on deep learning and unsupervised feature learning, vol 2011, pp 5

  23. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456

  24. Gross S, Wilber M (2016) Training and investigating residual nets. Facebook AI Research, CA.[Online]. Avilable: http://torch.ch/blog/2016/02/04/resnets. html

  25. Sutskever I, Martens J, Dahl G, Hinton G (2013) On the importance of initialization and momentum in deep learning. In: International conference on machine learning, pp 1139–1147

  26. Malinowski M, Fritz M (2013) Learnable pooling regions for image classification. pp 1–10, arXiv:1301.3516

  27. Zeiler Matthew D, Fergus R (2013) Stochastic pooling for regularization of deep convolutional neural networks. pp 1–9, arXiv:1301.3557

  28. Srivastava N (2013) Improving neural networks with dropout. University of Toronto 182:566

    Google Scholar 

  29. Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp 2951–2959

  30. Goodfellow IJ, Warde-Farley D, Mirza M, Courville A, Bengio Y (2013) Maxout networks. pp 1–9, arXiv:1302.4389

  31. Srivastava N, Salakhutdinov RR (2013) Discriminative transfer learning with tree-based priors. In: Advances in neural information processing systems, pp 2094–2102

  32. Goodfellow IJ, Bulatov Y, Ibarz J, Arnoud S, Shet V (2013) Multi-digit number recognition from street view imagery using deep convolutional neural networks. pp 1–13, arXiv:1312.6082

  33. Larsson G, Maire M, Shakhnarovich G (2016) Fractalnet: Ultra-deep neural networks without residuals. pp 1–11, arXiv:1605.07648

  34. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: The all convolutional net. pp 1–14, arXiv:1412.6806

  35. Lee C-Y, Xie S, Gallagher P, Zhang Z, Tu Z (2015) Deeply-supervised nets. pp 562–570

  36. Yang Y, Zhong Z, Shen T, Lin Z (2018) Convolutional neural networks with alternately updated clique. In: IEEE conference on computer vision and pattern recognition, pp 2413–2422

  37. Hudson DA, Manning CD (2018) Compositional attention networks for machine reasoning. pp 1–20, arXiv:1803.03067

Download references

Author information

Authors and Affiliations

  1. Nanning Normal University, Nanning, 530299, China

    Changan Yuan, Yong Wu, Xiao Qin & Yonghua Pan

  2. Chengdu University of Information Technology, Chengdu, 610225, China

    Shaojie Qiao, Ping Huang, Dunhu Liu & Nan Han

Authors
  1. Changan Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaojie Qiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yonghua Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ping Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Dunhu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Nan Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiao Qin or Shaojie Qiao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is partially supported by the National Natural Science Foundation of China under Grant Nos. 61772091, 61802035; the Natural Science Foundation of Guangxi under Grant Nos. 2018GX NSFDA138005, 2016GXNSFAA380209, 2014GXNSFDA118037; the Project of Scientific Research and Technology Development in Guangxi under Grant Nos. AA18118047, AB16380272, AD18126015, 20175177; the Innovation Project of Guangxi Graduate Education under Grant No. YCSW2017187; the Sichuan Science and Technology Program under Grant Nos. 2018JY0448, 2019YFG0106, 2019YFS0067; the Innovative Research Team Construction Plan in Universities of Sichuan Province under Grant No. 18TD0027; the Scientific Research Foundation for Young Academic Leaders of Chengdu University of Information Technology under Grant No. J201701.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, C., Wu, Y., Qin, X. et al. An effective image classification method for shallow densely connected convolution networks through squeezing and splitting techniques. Appl Intell 49, 3570–3586 (2019). https://doi.org/10.1007/s10489-019-01468-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01468-7

Keywords

Search

Navigation