Skip to main content
SpringerLink
Log in

Deep ensemble network based on multi-path fusion

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Deep convolutional network is commonly stacked by vast number of nonlinear convolutional layers. Deep fused network can improve the training process of deep convolutional network due to its capability of learning multi-scale representations and of optimizing information flow. However, the depth in a deep fused network does not contribute to the overall performance significantly. Therefore, a deep ensemble network consisting of deep fused network and branch channel is proposed. First, two base networks are combined in a concatenation and fusion manner to generate a deep fused network architecture. Then, an ensemble block with embedded learning mechanisms is formed to improve feature representation power of the model. Finally, the computational efficiency is improved by introducing group convolution without loss of performance. Experimental results on the standard recognition tasks have shown that the proposed network achieves better classification performance and has superior generalization ability compared to the original residual networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Abdi M, Nahavandi S (2017) Multi-residual networks: improving the speed and accuracy of residual networks. arXiv preprint. arXiv: 1609.05672

  • Alom MZ, Hasan M, Yakopcic C et al (2017) Inception recurrent convolutional neural network for object recognition. arXiv preprint. arXiv: 1704.07709

  • Bell S, Zitnick CL, Bala K et al (2016) Inside-outside net: detecting objects in context with skip pooling and recurrent neural networks. In: Proceedings of the 2016 IEEE computer society conference on computer vision and pattern recognition, pp 2874–2883

  • Deng J, Dong W, Socher R et al (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of the 2009 IEEE computer society conference on computer vision and pattern recognition, pp 248–255

  • Girshick R, Donahue J, Darrell T et al (2016) Region-based convolutional networks for accurate object detection and segmentation. IEEE Trans Pattern Anal Mach Intell 38(1):142–158

    Article  Google Scholar 

  • Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. J Mach Learn Res 9:249–256

    Google Scholar 

  • Guo JM, Zhang SF, Li JM (2016) Hash learning with convolutional neural networks for semantic based image retrieval. Lecture Notes in Computer Science, vol 9561, pp 227–238

  • He KM, Zhang XY, Ren SQ et al (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the 2015 IEEE international conference on computer vision, pp 1026–1034

  • He KM, Zhang XY, Ren SQ et al (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE computer society conference on computer vision and pattern recognition, pp 770–778

  • He K, Zhang X, Ren SQ et al (2016) Identity mappings in deep residual networks. arXiv preprint. arXiv: 1603.05027

  • Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. arXiv preprint. arXiv: 1709.01507

  • Huang G, Liu S C, Maaten et al (2018) CondenseNet: an efficient DenseNet using learned group convolutions. arXiv preprint. arXiv: 1711.09224

  • Huang G, Liu Z, Weinberger KQ et al (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 2261–2269

  • Huang G, Sun Y, Liu Z et al (2016) Deep networks with stochastic depth. Leture Notes in Computer Science, vol 9908, pp 646–661

  • Ioannou Y, Robertson D, Cipolla R et al (2017) Deep roots: improving CNN efficiency with hierarchical filter groups. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 5977–5986

  • Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, vol 1, pp 448–456

  • Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  • Larsson G, Maire M, Shakhnarovich G (2017) FractalNet: ultra-deep neural networks without residuals. arXiv preprint. arXiv: 1605.07648

  • Lin M, Chen Q, Yan S (2014) Network in network. arXiv preprint. arXiv: 1312.4400

  • Liu XL, Deng ZD, Yang YH (2018) Recent progress in semantic image segmentation. Artif Intell Rev. https://doi.org/10.1007/s10462-018-9641-3

    Google Scholar 

  • Lv EH, Wang XS, Cheng YH et al (2018) Deep convolutional network based on pyramid architecture. IEEE Access 6:43125–43135

    Article  Google Scholar 

  • Patil H, Kothari A, Bhurchandi K (2015) 3-D face recognition: features, databases, algorithms and challenges. Artif Intell Rev 44(3):393–441

    Article  Google Scholar 

  • Romero A, Ballas N, Kahou SE et al (2015) Fitnets: hints for thin deep nets. arXiv preprint. arXiv: 1412.6550

  • Shang WL, Sohn K, Almeida D et al (2016) Understanding and improving convolutional neural networks via concatenated rectified linear units. In: Proceedings of the 33rd international conference on machine learning, vol 5, pp 3276–3284

  • Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  • Shen L, Lin ZC, Huang QM (2016) Relay backpropagation for effective learning of deep convolutional neural networks. In: Proceedings of the 14 European conference on computer vision, vol 9911, pp 467–482

  • Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. arXiv preprint. arXiv: 1409.1556

  • Srivastava RK, Greff K, Schmidhuber JG (2015) Training very deep networks. Advances in Neural Information Processing Systems 2377–2385:

  • Srivastava N, Hinton G, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  • Szegedy C, Liu W, Jia YQ et al (2015) Going deeper with convolutions. In: Proceedings of the 2015 IEEE computer society conference on computer vision and pattern recognition, pp 1-9

  • Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the 2016 IEEE computer society conference on computer vision and pattern recognition, pp 2818–2826

  • Szegedy C, Ioffe S, Vanhoucke V et al (2017) Inception-v4, inception-resNet and the impact of residual connections on learning. In: Proceedings of the 31st AAAI conference on artificial intelligence, pp 4278–4284

  • Targ S, Almeida D, Lyman K (2016) Resnet in resnet: generalizing residual architectures. arXiv preprint. arXiv: 1603.08029

  • Veit A, Wilber M, Belongie S (2016) Residual networks are exponential ensembles of relatively shallow networks. arXiv preprint. arXiv: 1605.06431

  • Wang JD, Zhen W, Zhang T et al (2016) Deeply-fused nets. arXiv preprint. arXiv: 1605.07716

  • Wang F, Jiang MQ, Qian C et al (2017) Residual attention network for image classification. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 6450–6458

  • Xie SN, Girshick R, Dollr P et al (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the 2017 IEEE computer society conference on computer vision and pattern recognition, pp 5987–5995

  • Zagoruyko S, Komodakis N (2017) Wide residual networks. arXiv preprint. arXiv: 1605.07146

  • Zhang T, Qi GJ, Xiao B et al (2017) Interleaved group convolutions for deep neural networks. arXiv preprint. arXiv: 1707.02725

  • Zhao LM, Wang JD, Li X et al (2016) On the connection of deep fusion to ensembling. arXiv preprint. arXiv: 1611.07718

Download references

Author information

Authors and Affiliations

  1. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, China

    Enhui Lv, Xuesong Wang, Yuhu Cheng & Qiang Yu

  2. Xuzhou Key Laboratory of Artificial Intelligence and Big Data, Xuzhou, 221116, China

    Enhui Lv, Xuesong Wang, Yuhu Cheng & Qiang Yu

Authors
  1. Enhui Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuesong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuhu Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuhu Cheng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61772532).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lv, E., Wang, X., Cheng, Y. et al. Deep ensemble network based on multi-path fusion. Artif Intell Rev 52, 151–168 (2019). https://doi.org/10.1007/s10462-019-09708-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-019-09708-5

Keywords

Search

Navigation