Skip to main content
Springer Nature Link
Log in

Cascaded hybrid residual U-Net for glioma segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Glioma segmentation is critical for making surgical plans. Recently, the traditional glioma segmentation method is less competitive with two deep learning segmentation strategies: the patch-based method which focuses more on the local feature for each pixel, and the image-based method which fully leverages the global feature and captures the overall shape, size and other characteristics of the lesion in a neighborhood of a pixel. In this study, we investigate and integrate the advantages of 2-D and 3-D image-based architectures, and propose a new convolutional neural network called the Cascaded Hybrid Residual U-Net (CHR-U-Net) for MRI glioma segmentation. The CHR-U-Net exploits both the 2D local features as well as the 3D global spatial contextual information simultaneously. In the first-level of CHR-U-Net, the R-2D-U-Net combines the 2D-U-Net and the residual unit for quick lesion area detecting without any miss. To prevent from missing false-positive pixels, the output of R-2D-U-Net is resampled by using the hard-mining to collect more possible false-positive samples. In the second-level of CHR-U-Net, the axial, coronal, and sagittal 3D-U-Nets are trained to predict whether pixels belong to the area of glioma. The results of three 3D-U-Nets are fused to improve the accuracy and reduce false positives. The database of 2017 BRATS challenge were used in our experiments for the verification. The Dices and Sensitivities of Enhancing, Whole, and Core areas were calculated. The Dices are 0.73, 0.90, and 0.83 and the Sensitivities are 0.83, 0.90, and 0.82, respectively, for the axial, coronal, and sagittal 3D-U-Nets. Experimental results show that the proposed model significantly improves the performance of glioma segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Anwar S, Hussain S, Majid M (2017) Brain tumor segmentation using cascaded deep convolutional neural network. Engineering in Medicine & Biology Society, pp 1998–2001

  2. Axel D, Mohammad H, David WF, Antoine B (2014) Brain tumor segmentation with deep neural networks. Proceedings of the MICCAI workshop on multimodal brain tumor segmentation challenge BRATS. pp 1–5

  3. Bauer S, Nolte LP, Reyes M (2011) Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. Med Image Comput Comput Assist Interv 14:354–361

    Google Scholar 

  4. Chen S, Ding C, Liu M (2019) Dual-force convolutional neural networks for accurate brain tumor segmentation. Pattern Recogn 88:90–100. https://doi.org/10.1016/j.patcog.2018.11.009

    Article  Google Scholar 

  5. Dvořák P, Menze B (2016) Local structure prediction with convolutional neural networks for multimodal brain tumor segmentation. In: Menze B, Langs G, Montillo A, Kelm M, Müller H, Zhang S, Cai W, Metaxas D (eds). Springer International Publishing, Cham, pp 59–71

  6. Farahani K, Menze B, Reyes M (2013) Multimodal brain tumor segmentation(BRATS 2013)

  7. Farahani K, Menze B Reyes M (2014) Brats 2014 challenge manuscripts

  8. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Conference on Computer Vision & Pattern Recognition, pp 580–587

  9. Havaei M, Davy A, Wardefarley D, Larochelle H (2015) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31

    Article  Google Scholar 

  10. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp 770–778

  11. Holland EC (2001) Progenitor cells and glioma formation. Curr Opin Neurol 14:683–688

    Article  Google Scholar 

  12. Hou L, Samaras D, Kurc TM, Saltz JH (2016) Patch-based convolutional neural network for whole slide tissue image classification. Computer Vision & Pattern Recognition, pp 2424–2433

  13. Hussain S, Anwar SM, Majid M (2017) Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing 282:S772479851

    Google Scholar 

  14. Kamnitsas K, Ledig C, Newcombe VFJ, Glocker B (2016) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  15. Kamnitsas K, Ferrante E, Parisot S (2016) Deep medic for brain tumor segmentation. International Workshop on Brainlesion: Glioma, multiple sclerosis, stroke and traumatic Brain. Springer, pp 138–149

  16. Kamnitsas K, Bai W, Ferrante E, Glocker B (2018) Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Crimi A, Bakas S, Kuijf H, Menze B, Reyes M (eds) Springer International Publishing, Cham, pp 450–462

  17. Kayalıbay B, Jensen G, van der Smagt P (2017) CNN-based segmentation of medical imaging data

  18. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. International Conference on Neural Information Processing Systems.

  19. Li X, Chen H, Qi X, Heng P (2017) H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging

  20. Li H, Li A, Wang M (2019) A novel end-to-end brain tumor segmentation method using improved fully convolutional networks. Comput Biol Med 108:150–160. https://doi.org/10.1016/j.compbiomed.2019.03.014

    Article  Google Scholar 

  21. Lin F, Wu Q, Liu J, Kong X (2020) Path aggregation U-net model for brain tumor segmentation. Multimedia Tools and Applications. https://doi.org/10.1007/s11042-020-08795-9

  22. Lindley DV, Smith AFM (1972) Bayes estimates for the linear model. J R Stat Soc Ser B 34:1–18. https://doi.org/10.1111/j.2517-6161.1972.tb00885.x

    Article  MathSciNet  MATH  Google Scholar 

  23. Logeswari T, Karnan M (2010) An improved implementation of brain tumor detection using soft computing. Cancer Res 4:6–14

    Google Scholar 

  24. Long J, Shelhamer E, Darrell T (2014) Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence 39:640–651

    Google Scholar 

  25. Meier R (2013) A hybrid model for multimodal brain tumor segmentation. NCI-MICCAI BRATS, pp 31–37

  26. Meier R, Bauer S, Slotboom J, Reyes M (2014) Appearance and context sensitive features for brain tumor segmentation. MICCAI Brain Tumor Segmentation Challenge, pp 20–26

  27. Mellitari F, Navab N, Amadi S (2016) V-Net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV), pp 565–571. https://doi.org/10.1109/3DV.2016.79

  28. Montgomery D, Peck E (1992) Introduction to linear regression analysis. J R Stat Soc Ser C 32:94. https://doi.org/10.2307/2348054

    Article  MATH  Google Scholar 

  29. Naceur MB, Saouli R, Akil M, Kachouri R (2018) Fully automatic brain tumor segmentation using end-to-end incremental deep neural networks in MRI images. Comput Methods Prog Biomed 166:39–49

    Article  Google Scholar 

  30. Noori M, Bahri A, Mohammadi K (2019) Attention-guided version of 2D UNet for automatic brain tumor segmentation. 2019 9th International Conference on Computer and Knowledge Engineering (ICCKE), pp 269–275. https://doi.org/10.1109/ICCKE48569.2019.8964956

  31. Ozgun C, Abdulkadir A, Lienkamp SS, Ronneberger O (2016) 3D U-net: Learning dense volumetric segmentation from sparse annotation. International conference on medical image computing and computer assisted intervention, pp 424–432

  32. Pan X, Li L, Yang H, Fan Y (2016) Accurate segmentation of nuclei in pathological images via sparse reconstruction and deep convolutional networks. Neurocomputing 229:S771474853

    Google Scholar 

  33. Pereira S, Pinto A, Alves V, Silva CA (2015) Deep convolutional neural networks for the segmentation of Gliomas in multi-sequence MRI. Proceedings of the MICCAI workshop on multimodal brain tumor segmentation challenge BRATS, pp 52–55

  34. Pinto A, Pereira S, Correia H, Silva CA (2015) Brain tumour segmentation based on extremely randomized Forest with high-level features. Engineering in Medicine & Biology Society, pp 3037–3040

  35. Qayyum A, Anwar SM, Awais M, Majid M (2017) Medical image retrieval using deep convolutional neural network. Neurocomputing. 266:8–20

    Article  Google Scholar 

  36. Richmond DL, Kainmueller D, Yang MY Rother C (2015) Relating cascaded random forests to deep convolutional neural networks for semantic segmentation. Computer Science

  37. Ruczinski I, Kooperberg C, LeBlanc M (2003) Logic Regression. J Comput Graph Stat 12:475–511. https://doi.org/10.1198/1061860032238

    Article  MathSciNet  MATH  Google Scholar 

  38. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536

    Article  Google Scholar 

  39. Sikka K, Sinha N, Singh PK, Mishra AK (2009) A fully automated algorithm under modified FCM framework for improved brain MR image segmentation. Magn Reson Imaging 27:994–1004

    Article  Google Scholar 

  40. Singh A (2011) Malignant brain tumor detection. Int J Comput Theory 4:1002

    Google Scholar 

  41. Szegedy C, Vanhoucke V, Ioffe S Wojna Z (2015) Rethinking the inception architecture for computer vision

  42. Urban G, Bendszus M, Hamprecht F, Kleesiek J (2014) Multi-modal brain tumor segmentatioin using deep convolutional neural networks. Proceedings MICCAI Bra TS (brain tumor segmentation challenge), pp 31–35

  43. Valverde S, Cabezas M, Roura E, Oliver A (2017) Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach. Neuroimage 155:159–168

    Article  Google Scholar 

  44. Wang G, Li W, Ourselin S, Vercauteren T (2018) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: Crimi A, Bakas S, Kuijf H, Menze B, Reyes M (eds) Springer International Publishing, Cham, pp 178–190

  45. Xue Y, Xu T, Zhang H, Huang X (2018) SegAN: adversarial network with multi-scale L1 loss for medical image segmentation. Neuroinformatics 16:383–392. https://doi.org/10.1007/s12021-018-9377-x

    Article  Google Scholar 

  46. Zikic D, Glocker B, Konukoglu E, Price SJ (2012) Decision forests for tissue-specific segmentation of high-grade Gliomas in multi-channel MR. International Conference on Medical Image Computing & Computer-assisted Intervention

  47. Zikic D, Ioannou Y, Brown M, Criminisi A (2014) Segmentation of brain tumor tissues with convolutional neural networks. MICCAI Bra TS (brain tumor segmentation challenge), pp 36–39

Download references

Acknowledgements

This research was financially supported by the National Key R&D program of China (Grant No. 2017YFC0112804) and the National Natural Science Foundation of China (No. 81671768).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China

    Jiaosong Long, Guangzhi Ma, Hong Liu, Enmin Song, Xiangyang Xu, Renchao Jin, Yuzhou Zhuang, DaiYang Liu, Guangzhi Ma & Enmin Song

  2. College of Computing and Software Engineering, Kennesaw State University, Atlanta, GA, USA

    Chih-Cheng Hung

Authors
  1. Jiaosong Long
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Guangzhi Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Hong Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Enmin Song
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Chih-Cheng Hung
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Xiangyang Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Renchao Jin
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Yuzhou Zhuang
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. DaiYang Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Guangzhi Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

  11. Enmin Song
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Guangzhi Ma or Enmin Song.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Long, J., Ma, G., Liu, H. et al. Cascaded hybrid residual U-Net for glioma segmentation. Multimed Tools Appl 79, 24929–24947 (2020). https://doi.org/10.1007/s11042-020-09210-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09210-z

Keywords