Skip to main content
SpringerLink
Log in

Semantic segmentation of brain tumor with nested residual attention networks

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Brain tumors are one of the most serious brain diseases, which often result in a short life. However, in developing areas, medical resources are in shortage, which affect the diagnosis of brain tumors. With the development of computer science, many diseases can be diagnosed by telemedicine systems, which help physicians save much time and improve diagnostic accuracy. Therefore, we propose a semantic segmentation method for brain tumors based on nested residual attention networks. It can be deployed in social mx‘edia environment to work as a remote diagnosis system. The proposed method uses an improved residual attention block (RAB) as the basic unit. Based on the improved RAB, a nested RAB is designed to build the proposed method, which has better generalization. The proposed method includes an encoder part, a decoder part and skip connections. The encoder part learns multiple feature representations from inputs and the decoder part utilizes the learnt features to make segmentation predictions. In addition, high-level attention feature maps are exploited to induce low-level feature maps in skip connections to discard useless information. The proposed method is mainly validated on the dataset of Brain Tumor Segmentation challenge (BraTS) 2015. The proposed method achieves an average dice score of 0.87 (0.80, 0.72) for the whole tumor (core tumor, enhancing tumor) regions on BraTS 2015 dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abdel-Maksoud E, Elmogy M, Al-Awadi R (2015) Brain tumor segmentation based on a hybrid clustering technique. Egypt Inform J 16(1):71–81

    Article  Google Scholar 

  2. Bauer S, Nolte LP, Reyes M (2011) Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. In: International conference on medical image computing and computer-assisted intervention. Springer, New York, pp 354–361

  3. Chandra S, Vakalopoulou M, Fidon L, Battistella E, Estienne T, Sun R, Robert C, Deutsch E, Paragios N (2018) Context aware 3d cnns for brain tumor segmentation. In: International MICCAI brainlesion workshop. Springer, New York, pp 299–310

  4. Chen X, Hao Liew J, Xiong W, Chui CK, Ong SH (2018) Focus, segment and erase: an efficient network for multi-label brain tumor segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 654–669

  5. El-Melegy MT, Mokhtar HM (2014) Tumor segmentation in brain mri using a fuzzy approach with class center priors. EURASIP J Image Video Process 2014(1):21

    Article  Google Scholar 

  6. Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin PM, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31

    Article  Google Scholar 

  7. He K, Zhang X, Ren S, Jian S (2016) Identity mappings in deep residual networks. In: European conference on computer vision

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  9. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  10. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  11. Hung CJ, Ginzinger DG, Zarnegar R, Kanauchi H, Wong MG, Kebebew E, Clark OH, Duh QY (2003) Expression of vascular endothelial growth factor-c in benign and malignant thyroid tumors. 88(8), 3694

  12. Iqbal S, Ghani MU, Saba T, Rehman A (2018) Brain tumor segmentation in multi-spectral mri using convolutional neural networks (cnn). Microsc Res Techniq 81(4):419–427

    Article  Google Scholar 

  13. Isensee F, Kickingereder P, Wick W, Bendszus M, Maier-Hein KH (2017) Brain tumor segmentation and radiomics survival prediction: contribution to the brats 2017 challenge. In: International MICCAI brainlesion workshop. Springer, New York, pp 287–297

  14. Jaderberg M, Simonyan K, Zisserman A, et al. (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp 2017–2025

  15. Kamnitsas K, Ferrante E, Parisot S, Ledig C, Nori AV, Criminisi A, Rueckert D, Glocker B (2016) Deepmedic for brain tumor segmentation. In: International workshop on brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Springer, New York, pp 138–149

  16. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2016) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Med Image Anal 36:61

    Article  Google Scholar 

  17. Kermi A, Mahmoudi I, Khadir MT (2019) Deep convolutional neural networks using u-net for automatic brain tumor segmentation in multimodal mri volumes. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Springer International Publishing, Cham, pp 37–48

  18. Li J, Sun J, Liu L, Liu B, Xiao C, Wang F (2019) Improved maximum margin clustering via the bundle method. IEEE Access 7:63709–63721

    Article  Google Scholar 

  19. Li J, Wang F (2016) Semi-supervised learning via mean field methods. Neurocomputing 177:385–393

    Article  Google Scholar 

  20. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  21. McKinley R, Meier R, Wiest R (2018) Ensembles of densely-connected cnns with label-uncertainty for brain tumor segmentation International MICCAI brainlesion workshop. Springer, New York, pp 456–465

  22. Menon N, Ramakrishnan R (2015) Brain tumor segmentation in mri images using unsupervised artificial bee colony algorithm and fcm clustering. In: 2015 international conference on communications and signal processing (ICCSP), IEEE, pp 0006–0009

  23. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R (2015) The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans Med Imaging

  24. Milletari F, Navab N, Ahmadi S (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. pp 565–571

  25. Myronenko A (2018) 3d mri brain tumor segmentation using autoencoder regularization. In: International MICCAI brainlesion workshop. Springer, New York, pp 311–320

  26. Pereira S, Alves V, Silva CA (2018) Adaptive feature recombination and recalibration for semantic segmentation: application to brain tumor segmentation in mri. In: International conference on medical image computing and computer-assisted intervention. Springer, New York, pp 706–714

  27. Pereira S, Pinto A, Alves V, Silva CA (2016) Brain tumor segmentation using convolutional neural networks in mri images. IEEE Trans Med Imaging 35(5):1240–1251

    Article  Google Scholar 

  28. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, New York, pp 234–241

  29. Sharma M, Mukherjee S (2013) Fuzzy c-means, anfis and genetic algorithm for segmenting astrocytoma–a type of brain tumor International Journal 3(6)

  30. Shreyas V, Pankajakshan V (2017) A deep learning architecture for brain tumor segmentation in mri images. In: 2017 IEEE 19Th international workshop on multimedia signal processing (MMSP), IEEE, pp 1–6

  31. Sudre CH, Li W, Vercauteren T, Ourselin S, Cardoso MJ (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, New York, pp 240–248

  32. Tustison NJ, Shrinidhi K, Wintermark M, Durst CR, Kandel BM, Gee JC, Grossman MC, Avants BB (2015) Optimal symmetric multimodal templates and concatenated random forests for supervised brain tumor segmentation (simplified) with antsr. Neuroinformatics 13(2):209–225

    Article  Google Scholar 

  33. Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  34. Wang G, Li W, Ourselin S, Vercauteren T (2017) Automatic brain tumor segmentation using cascaded anisotropic convolutional neural networks. In: International MICCAI brainlesion workshop. Springer, New York, pp 178–190

  35. Wang L, Wang S, Chen R, Qu X, Chen Y, Huang S, Liu C (2019) Nested dilation networks for brain tumor segmentation based on magnetic resonance imaging. Front Neurosci pp 13

  36. Weninger L, Rippel O, Koppers S, Merhof D (2018) Segmentation of brain tumors and patient survival prediction: Methods for the brats 2018 challenge. In: International MICCAI brainlesion workshop. Springer, New York, pp 3–12

  37. Xu X, Zhang L, Li J, Guan Y, Zhang L (2019) A hybrid global-local representation cnn model for automatic cataract grading. IEEE Journal of Biomedical and Health Informatics

  38. Yang JJ, Li J, Mulder J, Wang Y, Chen S, Wu H, Wang Q, Pan H (2015) Emerging information technologies for enhanced healthcare. Comput Indust 69:3–11

    Article  Google Scholar 

  39. Yang JJ, Li J, Shen R, Zeng Y, He J, Bi J, Li Y, Zhang Q, Peng L, Wang Q (2016) Exploiting ensemble learning for automatic cataract detection and grading. Comput Meth Prog Biomed 124:45–57

    Article  Google Scholar 

  40. Zhang K, Sun M, Han TX, Yuan X, Guo L, Liu T (2017) Residual networks of residual networks: multilevel residual networks. IEEE Trans Circ Syst Video Technol 28(6):1303–1314

    Article  Google Scholar 

  41. Zhang N, Ruan S, Lebonvallet S, Liao Q, Zhu Y (2011) Kernel feature selection to fuse multi-spectral mri images for brain tumor segmentation. Comput Vis Image Underst 115(2):256–269

    Article  Google Scholar 

  42. Zhao X, Wu Y, Song G, Li Z, Zhang Y, Fan Y (2018) A deep learning model integrating fcnns and crfs for brain tumor segmentation. Med Image Anal 43:98–111

    Article  Google Scholar 

  43. Zhou C, Chen S, Ding C, Tao D (2018) Learning contextual and attentive information for brain tumor segmentation. In: International MICCAI brainlesion workshop. Springer, New York, pp 497–507

  44. Zhou C, Ding C, Wang X, Lu Z, Tao D (2019) One-pass multi-task networks with cross-task guided attention for brain tumor segmentation. arXiv:1906.01796

Download references

Acknowledgments

This study is supported by the National Key R&D Program of China with project No. 2017YFB1400803.

Author information

Authors and Affiliations

  1. School of Software Engineering, Beijing University of Technology, Beijing, China

    Jingchao Sun, Jianqiang Li & Lu Liu

Authors
  1. Jingchao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianqiang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by Jingchao Sun. The first draft of the manuscript was written by Jingchao Sun and Lu Liu commented on previous versions of the manuscript. Jianqiang Li read and approved the final manuscript.

Corresponding author

Correspondence to Jianqiang Li.

Ethics declarations

Conflict of interests

Author Jingchao Sun declares that he has no conflict of interest. Author Jianqiang Li declares that he has no conflict of interest. Author Lu Liu declares that she has no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Research involving Human Participants and/or Animals

This article does not contain any studies with human participants or animals performed by any of the authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sun, J., Li, J. & Liu, L. Semantic segmentation of brain tumor with nested residual attention networks. Multimed Tools Appl 80, 34203–34220 (2021). https://doi.org/10.1007/s11042-020-09840-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09840-3

Keywords

Search

Navigation