Skip to main content
SpringerLink
Log in

Fully Convolutional Deep Residual Neural Networks for Brain Tumor Segmentation

  • Conference paper
  • First Online:
Book cover Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (BrainLes 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10154))

Abstract

In this paper, a fully convolutional residual neural network (FCR-NN) based on linear identity mappings is implemented for medical image segmentation, employed here in the setting of brain tumors. Inspired by deep residual networks which won the ImageNet ILSVRC 2015 classification challenge, the FCR-NN combines optimization gains from residual identity mappings with a fully convolutional architecture for image segmentation that efficiently accounts for both low- and high-level image features. After training two separate networks, one for the task of whole tumor segmentation and a second for tissue sub-region segmentation, the serial FCR-NN architecture exceeds state-of-the art with complete tumor, core tumor and enhancing tumor validation Dice scores of 0.87, 0.81 and 0.72 respectively. Despite each FCR-NN comprising a complex 22 layer architecture, the fully convolutional design allows for complete segmentation of a tumor volume within 2 s.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks (2012)

    Google Scholar 

  2. Simonyan, K., Vedaldi, A., Zisserman, A.: Networks, deep inside convolutional: visualising image classification models and saliency maps. In: ICLR, p. 1 (2014)

    Google Scholar 

  3. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 07–12 June, 1–9 Sep 2015 (2015)

    Google Scholar 

  4. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 7(3), 171–180 (2015). Arxiv.Org

  5. Ciresan, D., Giusti, A.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in Neural Information Processing Systems, pp. 1–9 (2012)

    Google Scholar 

  6. Yang, S., Ramanan, D.: Multi-scale recognition with DAG-CNNs (2015)

    Google Scholar 

  7. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation (2014)

    Google Scholar 

  8. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440 (2014)

    Google Scholar 

  9. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). doi:10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  10. Havaei, M., Davy, A., Warde-Farley, D., Biard, A., Courville, A., Bengio, Y., Pal, C., Jodoin, P.-M., Larochelle, H.: Brain tumor segmentation with deep neural networks. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 29–33 (2015)

    Google Scholar 

  11. Kleesiek, J., Urban, G., Hubert, A., Schwarz, D., Maier-hein, K., Bendszus, M., Biller, A.: NeuroImage deep MRI brain extraction: a 3D convolutional neural network for skull stripping. NeuroImage 129, 460–469 (2016)

    Article  Google Scholar 

  12. Chen, H., Dou, Q., Yu, L., Heng, P.-A.: VoxResNet: deep voxelwise residual networks for volumetric brain segmentation, pp. 1–9 (2016). arXiv:1608.05895v1

  13. Sachdeva, J., Kumar, V., Gupta, I., Khandelwal, N., Ahuja, C.K.: A novel content-based active contour model for brain tumor segmentation. Magn. Reson. Imaging 30(5), 694–715 (2012)

    Article  Google Scholar 

  14. Harati, V., Khayati, R., Farzan, A.: Fully automated tumor segmentation based on improved fuzzy connectedness algorithm in brain MR images. Comput. Biol. Med. 41(7), 483–492 (2011)

    Article  Google Scholar 

  15. Prastawa, M., Bullitt, E., Gerig, G.: Simulation of brain tumors in MR images for evaluation of segmentation efficacy. Med. Image Anal. 13(2), 297–311 (2009)

    Article  Google Scholar 

  16. Menze, B.H., Leemput, K., Lashkari, D., Weber, M.-A., Ayache, N., Golland, P.: A generative model for brain tumor segmentation in multi-modal images. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6362, pp. 151–159. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15745-5_19

    Chapter  Google Scholar 

  17. Hamamci, A., Kucuk, N., Karaman, K., Engin, K., Unal, G.: Tumor - cut: segmentation of brain tumors on contrast enhanced MR images for radiosurgery applications. IEEE Trans. Med. Imaging 31(3), 790–804 (2012)

    Google Scholar 

  18. Zhu, Y., Young, G.S., Xue, Z., Huang, R.Y., You, H., Setayesh, K., Hatabu, H., Cao, F., Wong, S.T.: Semi-automatic segmentation software for quantitative clinical brain glioblastoma evaluation. Acad. Radiol. 19(8), 977–85 (2012)

    Google Scholar 

  19. Menze, B.H., Geremia, E., Ayache, N., Szekely, G.: Segmenting glioma in multi-modal images using a generative-discriminative model for brain lesion segmentation. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), p. 7 (2012)

    Google Scholar 

  20. Meier, R., Reyes, M., Bauer, S., Slotboom, J., Wiest, R.: A hybrid model for multimodal brain tumor segmentation. In: Proceedings of NCI-MICCAI BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 31–37 (2013)

    Google Scholar 

  21. Bakas, S., Zeng, K., Sotiras, A., Rathore, S., Akbari, H., Gaonkar, B., Rozycki, M., Pati, S., Davatzikos, C.: Segmentation of gliomas in multimodal magnetic resonance imaging volumes based on a hybrid generative - discriminative framework. In: Proceedings of MICCAI-BRATS (Multimodal Brain Tumor Segmentation Challenge), pp. 5–12 (2015)

    Google Scholar 

  22. Menze, B.H., Van Leemput, K., Lashkari, D., Riklin-Raviv, T., Geremia, E., Alberts, E., Gruber, P., Wegener, S., Weber, M.-A., Szekely, G., Ayache, N., Golland, P.: A generative probabilistic model and discriminative extensions for brain lesion segmentation with application to tumor and stroke. IEEE Trans. Med. Imaging 35(4), 933–946 (2016)

    Article  Google Scholar 

  23. Bakas, S., et al.: GLISTRboost: combining multimodal MRI segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 144–155. Springer, Cham (2016). doi:10.1007/978-3-319-30858-6_13

    Chapter  Google Scholar 

  24. Zeng, J., See, A.P., Phallen, J., Jackson, C.M., Belcaid, Z., Ruzevick, J., Durham, N., Meyer, C., Harris, T.J., Albesiano, E., Pradilla, G., Ford, E., Wong, J., Hammers, H.-J., Mathios, D., Tyler, B., Brem, H., Tran, P.T., Pardoll, D., Drake, C.G., Lim, M.: Anti-PD-1 blockade and stereotactic radiation produce long-term survival in mice with intracranial gliomas. Int. J. Rad. Oncol. Biol. Phys. 86(2), 343–349 (2013)

    Google Scholar 

  25. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  26. Pereira, S., Pinto, A., Alves, V., Silva, C.A.: Deep convolutional neural networks for the segmentation of gliomas in multi-sequence MRI. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 131–143. Springer, Cham (2016). doi:10.1007/978-3-319-30858-6_12

    Chapter  Google Scholar 

  27. Lecun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series (1995)

    Google Scholar 

  28. Nair, V., Hinton, G.E.: Rectified linear units improve restricted boltzmann machines (2010)

    Google Scholar 

  29. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, pp. 1–11 (2015). arXiv:1502.03167

  30. Bengio, Y., Boulanger-Lewandowski, N., Pascanu, R.: Advances in optimizing recurrent networks. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 8624–8628 (2013)

    Google Scholar 

  31. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: surpassing human-level performance on imagenet classification (2015)

    Google Scholar 

  32. Mandic, D.P.: A generalized normalized gradient descent algorithm. IEEE Signal Process. Lett. 11(2), 115–118 (2004)

    Article  Google Scholar 

  33. Vedaldi, A., Lenc, K.: MatConvNet. In: Proceedings of the 23rd ACM International Conference on Multimedia - MM 2015, pp. 689–692 (2015)

    Google Scholar 

  34. Agn, M., Puonti, O., Rosenschöld, P.M., Law, I., Leemput, K.: Brain tumor segmentation using a generative model with an RBM prior on tumor shape. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Handels, H. (eds.) BrainLes 2015. LNCS, vol. 9556, pp. 168–180. Springer, Cham (2016). doi:10.1007/978-3-319-30858-6_15

    Chapter  Google Scholar 

  35. Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., Lanczi, L., Gerstner, E., Weber, M.A., Arbel, T., Avants, B.B., Ayache, N., Buendia, P., Collins, D.L., Cordier, N., Corso, J.J., Criminisi, A., Das, T., Delingette, H., Demiralp, Ç., Durst, C.R., Dojat, M., Doyle, S., Festa, J., Forbes, F., Geremia, E., Glocker, B., Golland, P., Guo, X., Hamamci, A., Iftekharuddin, K.M., Jena, R., John, N.M., Konukoglu, E., Lashkari, D., Mariz, J.A., Meier, R., Pereira, S., Precup, D., Price, S.J., Raviv, T.R., Reza, S.M.S., Ryan, M., Sarikaya, D., Schwartz, L., Shin, H.C., Shotton, J., Silva, C.A., Sousa, N., Subbanna, N.K., Szekely, G., Taylor, T.J., Thomas, O.M., Tustison, N.J., Unal, G., Vasseur, F., Wintermark, M., Ye, D.H., Zhao, L., Zhao, B., Zikic, D., Prastawa, M., Reyes, M., Leemput, K.V.: The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34(10), 1993–2024 (2015)

    Google Scholar 

Download references

Acknowledgments

The author of this paper gratefully acknowledges the support of NVIDIA Corporation with the donation of GeForce GTX Titan X (12 GB) GPU used for this research.

Author information

Authors and Affiliations

  1. Department of Radiology, Columbia University Medical Center, New York City, New York, USA

    Peter D. Chang

Authors
  1. Peter D. Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter D. Chang .

Editor information

Editors and Affiliations

  1. University of Zurich, Istituto Italiano di Tecnologia, Genoa, Italy

    Alessandro Crimi

  2. TU München, Computer Science, Munich, Germany

    Bjoern Menze

  3. Medical Informatics, University of Lübeck, Lübeck, Germany

    Oskar Maier

  4. Surgical Technology and Biomechanics, Universität Bern, Bern, Switzerland

    Mauricio Reyes

  5. Department of Medicine, University of Cambridge, Cambridge, United Kingdom

    Stefan Winzeck

  6. Institute of Medical Informatics, University of Lübeck, Lübeck, Germany

    Heinz Handels

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Chang, P.D. (2016). Fully Convolutional Deep Residual Neural Networks for Brain Tumor Segmentation. In: Crimi, A., Menze, B., Maier, O., Reyes, M., Winzeck, S., Handels, H. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2016. Lecture Notes in Computer Science(), vol 10154. Springer, Cham. https://doi.org/10.1007/978-3-319-55524-9_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55524-9_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55523-2

  • Online ISBN: 978-3-319-55524-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation