Skip to main content
SpringerLink
Log in

Multi-view fusion segmentation for brain glioma on CT images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Computed tomography (CT) images of the brain aid the radiotherapy of glioma. The identification and contouring of the gross tumor volume (GTV) are important for radiotherapy. However, manual segmenting GTV is time-consuming, exhausting, and subjective, and automated methods are desired. To overcome these shortcomings, a novel neural network framework based on multi-view fusion is proposed to segment GTV in brain glioma automatically. The multi-view image that includes the previous image, current image, and following image is inputted in this framework to abstract extra spatial features and then aggregated to segment the GTV. Compared with the 2D segmentation framework, the proposed framework retains more spatial information due to the multi-view image. Meanwhile, compared with the 3D segmentation framework, the proposed framework considers fewer images, which means the model has fewer parameters and is easier to train while retaining much useful spatial information. Moreover, the GliomaCT dataset, a large CT dataset collected from West China Hospital, is used to train, validate, and test the proposed method. The performance of the proposed method and other state-of-the-art methods are compared on this dataset. The high dice similarity coefficient achieved in the experiments demonstrates the effectiveness of the proposed method for segmenting the GTV in brain glioma.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Altman DG, Bland JM (1983) Measurement in medicine: The analysis of method comparison studies. J R Stat Soc Ser D (The Statistician) 32(3):307–317. https://doi.org/10.2307/2987937

    Google Scholar 

  2. Bauer S, Nolte LP, Reyes M (2011) Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. In: International conference on medical image computing and computer-assisted intervention. Springer, pp. 354–361. https://doi.org/10.1007/978-3-642-23626-6_44

  3. Berger M, Weller M (2016) Gliomas (Volume 134) (Handbook of Clinical Neurology, Volume 134). Elsevier

  4. Cabria I, Gondra I (2017) Mri segmentation fusion for brain tumor detection. Inform Fusion 36:1–9. https://doi.org/10.1016/j.inffus.2016.10.003

    Article  Google Scholar 

  5. Chao KC, Low DA, Perez CA, Purdy JA (2000) Intensity-modulated radiation therapy in head and neck cancers: The mallinckrodt experience. Int J Cancer 90(2):92–103. https://doi.org/10.1002/(sici)1097-0215(20000420)90:2≺92::aid-ijc5≻3.0.co;2-9

    Article  Google Scholar 

  6. Chen C, Biffi C, Tarroni G, Petersen S, Bai W, Rueckert D (2019) Learning shape priors for robust cardiac MR segmentation from multi-view images. In: Medical image computing and computer assisted intervention. pp 523–531. https://doi.org/10.1007/978-3-030-32245-8_58

  7. Chen LC, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European conference on computer vision. pp 801–818. https://doi.org/10.1007/978-3-030-01234-2_49

  8. Corso JJ, Sharon E, Dube S, El-Saden S, Sinha U, Yuille A (2008) Efficient multilevel brain tumor segmentation with integrated bayesian model classification. IEEE Trans Med Imaging 27(5):629–640. https://doi.org/10.1109/TMI.2007.912817

    Article  Google Scholar 

  9. Deng J, Dong W, Socher R, Li L, Li K, Li F-F (2009) Imagenet: A large-scale hierarchical image database. In: Conference on computer vision and pattern recognition. pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848

  10. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: International conference on artificial intelligence and statistics. pp 315–323

  11. Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31. https://doi.org/10.1016/j.media.2016.05.004

    Article  Google Scholar 

  12. Havaei M, Larochelle H, Poulin P, Jodoin PM (2016) Within-brain classification for brain tumor segmentation. Int J Comput Assist Radiol Surgery 11(5):777–788. https://doi.org/10.1007/s11548-015-1311-1

    Article  Google Scholar 

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Conference on computer vision and pattern recognition. pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  14. Hu J, Chen Y, Zhong J, Ju R, Yi Z (2019) Automated analysis for retinopathy of prematurity by deep neural networks. IEEE Trans Med Imaging 38(1):269–279. https://doi.org/10.1109/TMI.2018.2863562

    Article  Google Scholar 

  15. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Conference on computer vision and pattern recognition. pp 4700–4708. https://doi.org/10.1109/CVPR.2017.243

  16. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning

  17. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnu-net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2):203–211. https://doi.org/10.1038/s41592-020-01008-z

    Article  Google Scholar 

  18. Islam A, Reza SM, Iftekharuddin KM (2013) Multifractal texture estimation for detection and segmentation of brain tumors. IEEE Trans Biomed Eng 60(11):3204–3215. https://doi.org/10.1109/TBME.2013.2271383

    Article  Google Scholar 

  19. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical Image Analysis 36:61–78. https://doi.org/10.1016/j.media.2016.10.004

    Article  Google Scholar 

  20. Kurata Y, Nishio M, Kido A, Fujimoto K, Yakami M, Isoda H, Togashi K (2019) Automatic segmentation of the uterus on MRI using a convolutional neural network. Comput Biol Med 114:103438. https://doi.org/10.1016/j.compbiomed.2019.103438

    Article  Google Scholar 

  21. Lin L, Dou Q, Jin YM, Zhou GQ, Tang YQ, Chen WL, Su BA, Liu F, Tao CJ, Jiang N et al (2019) Deep learning for automated contouring of primary tumor volumes by mri for nasopharyngeal carcinoma. Radiology 291(3):677–686. https://doi.org/10.1148/radiol.2019182012

    Article  Google Scholar 

  22. Loshchilov I, Hutter F (2016) Sgdr: Stochastic gradient descent with warm restarts. In: International conference on learning representations

  23. Men K, Dai J, Li Y (2017) Automatic segmentation of the clinical target volume and organs at risk in the planning ct for rectal cancer using deep dilated convolutional neural networks. Med Phys 44 (12):6377–6389. https://doi.org/10.1002/mp.12602

    Article  Google Scholar 

  24. Milletari F, Navab N, Ahmadi S (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: International conference on 3D vision, pp 565–571. https://doi.org/10.1109/3DV.2016.79

  25. Mortazi A, Karim R, Rhode K, Burt J, Bagci U (2017) CardiacNET: segmentation of left atrium and proximal pulmonary veins from MRI using multi-view CNN. In: Medical image computing and computer assisted intervention. pp 377–385. https://doi.org/10.1007/978-3-319-66185-8_43

  26. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L, Desmaison A, Köpf A., Yang E, DeVito Z, Raison M, Tejani A, Chilamkurthy S, Steiner B, Fang L, Bai J, Chintala S (2019) Pytorch: An imperative style, high-performance deep learning library. In: Advances in neural information processing systems. pp 8024– 8035

  27. Pi Y, Chen Y, Deng D, Qi X, Li J, Lv Q, Yi Z (2020) Automated diagnosis of multi-plane breast ultrasonography images using deep neural networks. Neurocomputing 403:371–382. https://doi.org/10.1016/j.neucom.2020.04.123

    Article  Google Scholar 

  28. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

  29. Shah M, Xiao Y, Subbanna N, Francis S, Arnold DL, Collins DL, Arbel T (2011) Evaluating intensity normalization on mris of human brain with multiple sclerosis. Med Image Anal 15(2):267–282. https://doi.org/10.1016/j.media.2010.12.003

    Article  Google Scholar 

  30. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J MachLearn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  31. Sutskever I, Vinyals O, Le QV (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems. pp 3104–3112

  32. Wang H, Huang H, Wang J, Wei M, Yi Z, Wang Z, Zhang H (2021) An intelligent system of pelvic lymph node detection. Int J Intell Syst. https://doi.org/10.1002/int.22452

  33. Wang H, Yang Y, Liu B, Fujita H (2019) A study of graph-based system for multi-view clustering. Knowl-Based Syst 163:1009–1019. https://doi.org/10.1016/j.knosys.2018.10.022

    Article  Google Scholar 

  34. Wang H, Zhang H, Hu J, Song Y, Bai S, Yi Z (2020) DeepEC: An error correction framework for dose prediction and organ segmentation using deep neural networks. Int J Intell Syst 35:1987–2008. https://doi.org/10.1002/int.22280

    Article  Google Scholar 

  35. Wang J, Ju R, Chen Y, Liu G, Yi Z (2020) Automated diagnosis of neonatal encephalopathy on aEEG using deep neural networks. Neurocomputing 398:95–107. https://doi.org/10.1016/j.neucom.2020.01.057

    Article  Google Scholar 

  36. Wu Y, Jiang X, Fang Z, Gao Y, Fujita H (2021) Multi-modal 3d object detection by 2d-guided precision anchor proposal and multi-layer fusion. Appl Soft Comput 108:107405. https://doi.org/10.1016/j.asoc.2021.107405

    Article  Google Scholar 

  37. Xiao Q, Dai J, Luo J, Fujita H (2019) Multi-view manifold regularized learning-based method for prioritizing candidate disease mirnas. Knowl-Based Syst 175:118–129. https://doi.org/10.1016/j.knosys.2019.03.023

    Article  Google Scholar 

  38. Yang M, Yu K, Zhang C, Li Z, Yang K (2018) Denseaspp for semantic segmentation in street scenes. In: Conference on computer vision and pattern recognition. pp 3684–3692. https://doi.org/10.1109/CVPR.2018.00388

  39. Zhang N, Ding S, Liao H, Jia W (2019) Multimodal correlation deep belief networks for multi-view classification. Appl Intell 49(5):1925–1936. https://doi.org/10.1007/s10489-018-1379-8

    Article  Google Scholar 

  40. Zhang X, Yang Y, Li T, Zhang Y, Wang H, Fujita H (2021) Cmc: A consensus multi-view clustering model for predicting alzheimer’s disease progression. Comput Methods Programs Biomed 199:105895. https://doi.org/10.1016/j.cmpb.2020.105895

    Article  Google Scholar 

  41. Zhang Y, Yang Y, Li T, Fujita H (2019) A multitask multiview clustering algorithm in heterogeneous situations based on lle and le. Knowl-Based Syst 163:776–786. https://doi.org/10.1016/j.knosys.2018.10.001

    Article  Google Scholar 

  42. Zhuang AH, Valentino DJ, Toga AW (2006) Skull-stripping magnetic resonance brain images using a model-based level set. NeuroImage 32(1):79–92. https://doi.org/10.1016/j.neuroimage.2006.03.019

    Article  Google Scholar 

  43. Zikic D, Glocker B, Konukoglu E, Criminisi A, Demiralp C, Shotton J, Thomas OM, Das T, Jena R, Price SJ (2012) Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel mr. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 369–376. https://doi.org/10.1007/978-3-642-33454-2_46

Download references

Acknowledgements

This work was supported by the National Key Research and Development Program of China (Grant No.2018AAA0100201).

Author information

Authors and Affiliations

  1. Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, 610065, People’s Republic of China

    Han Wang, Junjie Hu, Lei Zhang & Zhang Yi

  2. Department of Radiation Oncology, West China Hospital, Sichuan University, Chengdu, 610041, People’s Republic of China

    Ying Song & Sen Bai

Authors
  1. Han Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junjie Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sen Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhang Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Han Wang: Writing-original draft, Methodology. Junjie Hu: Writing-review & editing. Ying Song: Writing-review & editing. Lei Zhang: Supervision. Sen Bai: Supervision, Project administration. Zhang Yi: Supervision, Project administration, Funding acquisition, Writing-review & editing

Corresponding authors

Correspondence to Sen Bai or Zhang Yi.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Hu, J., Song, Y. et al. Multi-view fusion segmentation for brain glioma on CT images. Appl Intell 52, 7890–7904 (2022). https://doi.org/10.1007/s10489-021-02784-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02784-7

Keywords

Search

Navigation