Skip to main content
SpringerLink
Log in

Glioma segmentation of optimized 3D U-net and prediction of multi-modal survival time

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

It is difficult to segment Glioma and its internal structure because the Glioma boundaries have edemas and complex internal structures. This paper proposes a new optimized, integrated 3D U-Net network to achieve accurate segmentation of Glioma and internal subareas. The contribution of this paper is twofold, it studies the clinical path of patients with Glioma and constructs an optimized 3D U-Net deep learning algorithm by combining them with the radiologic feature set. The proposed model was validated in the published Glioma operation data set of multi-modal MRI resonance images and clinicians manual segmentation data. The model can accurately segment the MRI multi-modality images of Glioma and intra-tumour nodes and achieve the multi-modality prediction of the overall survival period of patients. The experimental results further indicated that the segmentation accuracy of the proposed method was higher than other sophisticated methods. The Dice similarity coefficients of the whole tumor (WT) region, the core tumor (CT) region, and the augmentation / enhanced tumor (ET) region, were 0.9632, 0.8763, and 0.8421, respectively, which are better than the clinical experts’ manual segmentation results. Hence, this research can effectively promote the development of deep learning clinical precise diagnosis and medical technology for Glioma.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data citation

Our training data BraTs2017 is publicly available and downloaded from University of Pennsylvania Section Center for Biomedical Image Computing & Analytics (CBICA)’s Image Processing Portal: https://www.med.upenn.edu/sbia/brats2017/data.html. According to the Data Usage Agreement posted on the website, the following papers are cited: Menze et al. [28]; Bakas et al. [2]; Bakas et al. [3]; Bakas et al. [4]; Bakas et al. [5]. We claim that we did not use additional private data for data augmentation in this paper.

References

  1. Aziz M. J., Tehrani zade A. A., Farnia P, Alimohamadi M, Makkiabadi B, Ahmadian A, Alirezaie J (2021). Accurate automatic glioma segmentation in brain mri images based on capsnet. bioRxiv

  2. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby JS, Freymann J, Farahani K, Davatzikos C (2017a) Segmentation labels and radiomic features for the pre-operative scans of the tcga-gbm collection. The cancer imaging archive. Nat Sci Data 4:170117

    Article  Google Scholar 

  3. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J. S., Freymann J, Farahani K, Davatzikos C (2017b). Segmentation labels and radiomic features for the pre-operative scans of the tcga-lgg collection. The cancer imaging archive

  4. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby JS, Freymann JB, Farahani K, Davatzikos C (2017c) Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Nat Sci Data 4:170117

    Article  Google Scholar 

  5. Bakas S, Reyes M, Jakab A, Bauer S, Rempfler M, Crimi A, Shinohara R. T., Berger C, Ha S. M., Rozycki M, et al (2018). Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv preprint arXiv:1811.02629

  6. Bottou L, Bousquet O (2012) The tradeoffs of large scale learning. In: Suvrit S, Nowozin S, Wright SJ (eds) Optimization for machine learning. MIT Press, London, pp 351–368

    Google Scholar 

  7. Carter BS, Chiocca AE, Lonser R, Kaye AH, de Tribolet N (2015) Introduction: a focus on low-grade glioma. Neurosurg Focus 38(1):E2

    Article  Google Scholar 

  8. Çiçek,Ö, Abdulkadir A, Lienkamp S. S, Brox T, Ronneberger O (2016). 3d u-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention, pp. 424–432. Springer

  9. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. Adv Neural Inf Process Syst 25:2843–2851

    Google Scholar 

  10. Dandıl E (2017). Implementation and comparison of image segmentation methods for detection of brain tumors on mr images. In: IEEE 2017 International Conference on Computer Science and Engineering (UBMK), pp. 1025–1029

  11. Deng W, Xiao X, Deng H, Liu J (2010) MRI brain tumor segmentation with region growing method based on the gradients and variances along and inside of the boundary curve. In: 3rd International Conference on Biomedical Engineering and Informatics. pp 393–396

  12. Dice L (1945) Measures of the amount of ecologic association between species. J Ecol 26:297–302

    Article  Google Scholar 

  13. Diederik K, Ba J (2014). Adam: a method for stochastic optimization. arXiv:1412.6980

  14. Duchi J, Hazan E, Singer Y (2011) Adaptive subgradient methods for online learning and stochastic optimization. JMLR 12:2121–2159

    MathSciNet  MATH  Google Scholar 

  15. Goodfellow I, Bengio Y, Courville A, Bengio Y (2016) Deep learning, vol 1. MIT press, Cambridge

    MATH  Google Scholar 

  16. Guillemaud R, Marais P, Zisserman A, McDonald B, Crow T, Brady M (1996) A three dimensional mid sagittal plane for brain asymmetry measurement. Schizophrenia Res 2–3(18):183–184

    Article  Google Scholar 

  17. Hamed Y, Alzahrani AI, A’fza S, Mustaffa Z, Ismail MC, Eng KK (2020) Two steps hybrid calibration algorithm of support vector regression and k-nearest neighbors. Alexandria Eng J 59(3):1181–1190

    Article  Google Scholar 

  18. Hinton G. E., Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R. R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580

  19. Huynh T, Gao Y, Kang J, Wang L, Zhang P, Lian J, Shen D (2015) Estimating ct image from mri data using structured random forest and auto-context model. IEEE Trans Med Imaging 35(1):174–183

    Article  Google Scholar 

  20. Kaldera H, Gunasekara S, Dissanayake M (2019). Mri based glioma segmentation using deep learning algorithms. In: 2019 International Research Conference on Smart Computing and Systems Engineering (SCSE), pp. 51–56

  21. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  22. Kleesiek J, Urban G, Hubert A, Schwarz D, Maier-Hein K, Bendszus M, Biller A (2016) Deep mri brain extraction: a 3d convolutional neural network for skull stripping. NeuroImage 129:460–469

    Article  Google Scholar 

  23. LeCun Y, Bengio Y et al (1995) Convolutional networks for images, speech, and time series. Hand Brain Theory Neural Netw 3361(10):255–258

    Google Scholar 

  24. Li M, Zhang L, Xiang Z, Castillo E, Guerrero T (2016). An improved fuzzy c-means algorithm for brain mri image segmentation. In: IEEE 2016 International Conference on Progress in Informatics and Computing (PIC), pp. 336–339

  25. Li Z, Wang Y, Yu J, Guo Y, Cao W (2017) Deep learning based radiomics (dlr) and its usage in noninvasive idh1 prediction for low grade glioma. Sci Rep 7(1):5467

    Article  Google Scholar 

  26. Liaw A, Wiener M (2002) Classification and regression by randomforest. R News 2(3):18–22

    Google Scholar 

  27. Menze B, Isensee F, Wiest R, Wiestler B, Maier-Hein K, Reyes M, Bakas S (2020). Analyzing magnetic resonance imaging data from glioma patients using deep learning. In: Computerized Medical Imaging and Graphics, pp. 101828

  28. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby JS et al (2015) The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  29. Milletari F, Ahmadi S-A, Kroll C, Plate A, Rozanski V, Maiostre J, Levin J, Dietrich O, Ertl-Wagner B, Bötzel K et al (2017) Hough-cnn: deep learning for segmentation of deep brain regions in mri and ultrasound. Comput Vis Image Underst. 164:92–102

    Article  Google Scholar 

  30. Ostrom QT, Bauchet L, Davis FG, Deltour I, Fisher JL, Langer CE, Pekmezci M, Schwartzbaum JA, Turner MC, Walsh KM, Wrensch MR, Barnholtz-Sloan JS (2014) The epidemiology of glioma in adults: a “state of the science” review. Neuro-Oncol. 16(7):896–913

    Article  Google Scholar 

  31. Pouratian N, Schiff D (2010) Management of low-grade glioma. Curr Neurol Neurosci Rep 10(3):224–231

    Article  Google Scholar 

  32. Ronneberger O, Fischer P, Brox T (2015). U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Springer

  33. Rudie JD, Weiss DA, Saluja R, Rauschecker AM, Wang J, Sugrue L, Bakas S, Colby JB (2019) Multi-disease segmentation of gliomas and white matter hyperintensities in the brats data using a 3d convolutional neural network. Front Comput Neurosci 13:84

    Article  Google Scholar 

  34. Singh R, Mukhopadhyay K (2011) Survival analysis in clinical trials: basics and must know areas. Perspect Clin Res 2(4):145–148

    Article  Google Scholar 

  35. Stadlbauer A, Moser E, Gruber S, Buslei R, Nimsky C, Fahlbusch R, Ganslandt O (2004) Improved delineation of brain tumors: an automated method for segmentation based on pathologic changes of 1h-mrsi metabolites in gliomas. Neuroimage 23(2):454–461

    Article  Google Scholar 

  36. Wu S, Li H, Quang D, Guan Y (2020) Three-plane-assembled deep learning segmentation of gliomas. Radiol Artif Intell 2(2):e190011

    Article  Google Scholar 

  37. Wu Y, Zhao Z, Wu W, Lin Y, Wang M (2019) Automatic glioma segmentation based on adaptive superpixel. BMC Med Imaging 19:73

    Article  Google Scholar 

  38. Zhang S, Zong M, Sun K, Liu Y, Cheng D (2014). Efficient knn algorithm based on graph sparse reconstruction. In: International Conference on Advanced Data Mining and Applications, pp. 356–369. Springer

  39. Zhang W, Wang X, Li Z, Qu Y (2017) Brain tumor image segmentation based on c-v model optimized by watershed transformation. Comput Eng Appl 53(5):176–180

    Google Scholar 

  40. Zhang Y, Brady M, Smith S (2001) Segmentation of brain mr images through a hidden markov random field model and the expectation-maximization algorithm. IEEE Trans Med Imaging 20(1):45–57

    Article  Google Scholar 

  41. Zikic D, Ioannou Y, Brown M, Criminisi A (2014) Segmentation of brain tumor tissues with convolutional neural networks. In: Proceedings MICCAI-BRATS pp. 36–39

  42. Zou KH, Warfield SK, Bharatha AB, Tempany CM, Kaus MR, Haker SJ, Wells WMW, Jolesz FA, Kikinis R (2004) Statistical validation of image segmentation quality based on a spatial overlap index. Acad Radiol 11(2):178–189

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sichuan University, Chengdu, Sichuan, China

    Qihong Liu & Ling He

  2. University of Prince Edward Island, Charlottetown, PEI, Canada

    Kai Liu & Antonio Bolufé-Röhler

  3. McMaster University, Hamilton, Ontario, Canada

    Jing Cai

Authors
  1. Qihong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonio Bolufé-Röhler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ling He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai Liu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Q., Liu, K., Bolufé-Röhler, A. et al. Glioma segmentation of optimized 3D U-net and prediction of multi-modal survival time. Neural Comput & Applic 34, 211–225 (2022). https://doi.org/10.1007/s00521-021-06351-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06351-6

Keywords

Search

Navigation