Skip to main content
SpringerLink
Log in

Multi stream 3D hyper-densely connected network for multi modality isointense infant brain MRI segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Automatic accurate segmentation of medical images has significant role in computer-aided diagnosis and disease treatment. The segmentation of cerebrospinal fluid (CSF), gray matter (GM), and white matter (WM) tissues plays an important role in infant brain structure for studying early brain development. However, this task is very challenging due to low contrast between GM and WM in isointense phase (approximately 6-8 months of age). In this study, we develop a hyper-densely connected convolutional neural network (CNN) for segmentation of volumetric infant brain. The proposed model provides dense connection between layers to improve the performance of flow information in the network. It also allows the multiscale contextual information by concatenating the feature maps of early, intermediate, and later layers. This architecture employs MR-T1 and T2 as input, which are processed in two separate independent paths, and then their low, intermediate, and high layer features are fused for final segmentation. An important change relative to earlier densely connected networks is the application of direct layer connections from the same and different paths. In this scenario, each modality is processed in an independent path, and dense connections occur not only between layers within the same path, but also between layers in different paths. Adopting such dense connectivity leads to benefits of deep supervision and improved gradient flow. Furthermore, by combining the feature maps of early, intermediate, and late convolutional layers, our architecture injects multiscale information into the final segmentation. This suggested approach is examined in the MICCAI Grand Challenge iSEG and obtains significant advantages over existing approaches in terms of parameter efficiency and segmentation accuracy on 6-month infant brain MRI segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. http://babyconnectomeproject.org

  2. http://www.nitrc.org/projects/ibeat/

  3. http://iseg2017.web.unc.edu/rules/results/

References

  1. Anbeek P, Vincken KL, Groenendaal F, Koeman A, Van Osch MJ (2008) Probabilistic brain tissue segmentation in neonatal magnetic resonance imaging. Pediatr Res 2(6):158–163

    Article  Google Scholar 

  2. Bui TD, Shin J, Moon T (2017) 3D densely convolutional networks for volumetric segmentation. arXiv:1709.03199

  3. Cardoso M, Melbourne A, Kendall G, Modat M, Robertson N, Marlow N, Ourselin S (2013) Adapt: an adaptive preterm segmentation algorithm for neonatal brain MRI. NeuroImage 65:97–108

    Article  Google Scholar 

  4. Chen S-Q, Zhan R-H, Hu J-M, Zhang J (2017) Feature fusion based on convolutional neural network for SAR ATR. In: ITM Web of conferences, vol 12. EDP Sciences, p 05001

  5. Chen H, Dou Q, Yu L, Qin J, Heng P-A (2018) Voxresnet: Deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage 170:446–455. Segmenting the brain

    Article  Google Scholar 

  6. Chen H, Yu L, Dou Q, Shi L, Mok VCT, Heng PA (2015) Automatic detection of cerebral microbleeds via deep learning based 3D feature representation. In: 2015 IEEE 12th international symposium on biomedical imaging (ISBI), pp 764–767

  7. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In: Ourselin S, Joskowicz L, Sabuncu MR, Unal G, Wells W (eds) Medical image computing and computer-assisted intervention – MICCAI 2016. Springer International Publishing, Cham, pp 424–432

  8. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25. Curran Associates, Inc, pp 2843–2851

  9. Criminisi A, Shotton J, Zikic D (2014) The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans Med Imaging

  10. Dolz J, Desrosiers C, Ayed IB (2018) 3D fully convolutional networks for subcortical segmentation in MRI: a large-scale study. NeuroImage 170:456–470. Segmenting the brain

    Article  Google Scholar 

  11. Dolz J, Massoptier L, Vermandel M (2015) Segmentation algorithms of subcortical brain structures on MRI for radiotherapy and radiosurgery: a survey. IRBM 36(4):200–212

    Article  Google Scholar 

  12. Dou Q, Chen H, Yu L, Zhao L, Qin J, Wang D, Mok VC, Shi L, Heng PA (2016) Automatic detection of cerebral microbleeds from MR images via 3D convolutional neural networks. IEEE Trans Med Imaging 35(5):1182–1195

    Article  Google Scholar 

  13. Drozdzal M, Vorontsov E, Chartrand G, Kadoury S, Pal C (2016) The importance of skip connections in biomedical image segmentation. arXiv:1608.04117

  14. Fechter T, Adebahr S, Baltas D, Ayed IB, Desrosiers C, Dolz J (2017) Esophagus segmentation in CT via 3D fully convolutional neural network and random walk. Med Phys 44(12):6341–6352

    Article  Google Scholar 

  15. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: International conference on artificial intelligence and statistics, vol 15, pp 315–323

  16. Gui L, Lisowski R, Faundez T, Hüppi PS, Lazeyras F, Kocher M (2012) Morphology-driven automatic segmentation of MR images of the neonatal brain. Med Image Anal 16(8):1565–1579

    Article  Google Scholar 

  17. Havaei M, Davy A, Warde-Farley D, Biard A, Courville A, Bengio Y, Pal C, Jodoin P-M, Larochelle H (2017) Brain tumor segmentation with deep neural networks. Med Image Anal 35:18–31

    Article  Google Scholar 

  18. Havaei M, Guizard N, Larochelle H, Jodoin P-M (2016) Deep learning trends for focal brain pathology segmentation in MRI. Springer International Publishing, Cham, pp 125–148

    Google Scholar 

  19. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 770–778

  20. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: 2015 IEEE International conference on computer vision (ICCV), pp 1026–1034

  21. Huang G, Liu Z, van der Maaten L, Weinberger K (2017) Densely connected convolutional networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 2261–2269

  22. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456

  23. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia, MM ’14. ACM, New York, pp 675–678

  24. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  25. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1646–1654

  26. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  27. Kontschieder P, Bulo SR, Bischof H, Pelillo M (2011) And structured class-labels in random forests for semantic image labelling. In: Proceedings of the 2011 international conference on computer vision, ICCV ’11. IEEE Computer Society, Washington, pp 2190–2197

  28. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90

    Article  Google Scholar 

  29. Lan X, Ma AJ, Yuen PC (2014) Multi-cue visual tracking using robust feature-level fusion based on joint sparse representation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1194–1201

  30. Lan X, Ma AJ, Yuen PC, Chellappa R (2015) Joint sparse representation and robust feature-level fusion for multi-cue visual tracking. IEEE Trans Image Process 24(12):5826–5841

    Article  MathSciNet  MATH  Google Scholar 

  31. Lan X, Zhang S, Yuen PC, Chellappa R (2018) Learning common and feature-specific patterns: a novel multiple-sparse-representation-based tracker. IEEE Trans Image Process 27(4):2022–2037

    Article  MathSciNet  MATH  Google Scholar 

  32. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  33. Li W, Shi F, Li G, Gao Y, Lin W, Gilmore JH, Shen D (2014) Segmentation of neonatal brain MR images using patch-driven level sets. NeuroImage 84:141–158, 1

    Article  Google Scholar 

  34. Li W, Shi F, Lin W, Gilmore JH, Shen D (2011) Automatic segmentation of neonatal images using convex optimization and coupled level sets. NeuroImage 58(3):805–817

    Article  Google Scholar 

  35. Liu C, Wechsler H (2001) Shape-and texture-based enhanced fisher classifier for face recognition. IEEE Trans on Image Process 10(4):598–608

    Article  MATH  Google Scholar 

  36. Melbourne A, Cardoso MJ, Kendall GS, Robertson NJ, Marlow N, Ourselin S (2012) Neobrains12 challenge: adaptive neonatal MRI brain segmentation with myelinated white matter class and automated extraction of ventricles i-iv. MICCAI grand challenge: neonatal brain segmentation (NeoBrainSI2), pp 16–21

  37. Milletari F, Navab N, Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth international conference on 3D vision (3DV), pp 565–571

  38. Moeskops P, Benders Manon JNL, Chita SM, Kersbergen KJ, Groenendaal F, de Vries LS, Viergever MA, Isgum I (2015) Automatic segmentation of MR brain images of preterm infants using supervised classification. NeuroImage 118:628–641

    Article  Google Scholar 

  39. Moeskops P, Viergever MA, Mendrik AM, de Vries LS, Benders MJNL, Išgum I (2016) Automatic segmentation of MR brain images with a convolutional neural network. IEEE Trans Med Imaging 35(5):1252–1261

    Article  Google Scholar 

  40. Nie D, Wang L, Gao Y, Sken D (2016) Fully convolutional networks for multi-modality isointense infant brain image segmentation. In: 2016 IEEE 13Th international symposium on biomedical imaging (ISBI), pp 1342–1345

  41. Prasoon A, Petersen K, Igel C, Lauze F, Dam E, Nielsen M (2013) Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N (eds) Medical image computing and computer-assisted intervention – MICCAI 2013. Springer, Berlin, pp 246–253

  42. Prastawa M, Gilmore JH, Lin W, Gerig G (2005) Automatic segmentation of MR images of the developing newborn brain. Med Image Anal 9(5):457–66

    Article  Google Scholar 

  43. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF (eds) Medical image computing and computer-assisted intervention – MICCAI 2015. Springer International Publishing, Cham, pp 234–241

  44. Roth HR, Lu L, Seff A, Cherry KM, Hoffman J, Wang S, Liu J, Turkbey E, Summers RM (2014) A new 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In: Golland P, Hata N, Barillot C, Hornegger J, Howe R (eds) Medical image computing and computer-assisted intervention – MICCAI 2014. Springer International Publishing, Cham, pp 520–527

  45. Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  46. Shi F, Fan Y, Tang S, Gilmore JH, Lin W, Shen D (2010) Neonatal brain image segmentation in longitudinal MRI studies. NeuroImage 49(1):391–400

    Article  Google Scholar 

  47. Shi F, Yap P-T, Fan Y, Gilmore JH, Lin W, Shen D (2010) Construction of multi-region-multi-reference atlases for neonatal brain MRI segmentation. NeuroImage 51(2):684–693

    Article  Google Scholar 

  48. Shi F, Yap P-T, Guorong W u, Jia H, Gilmore JH, Lin W, Shen D (2011) Infant brain atlases from neonates to 1- and 2-year-olds. PloS one 6(4):e18746

    Article  Google Scholar 

  49. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  50. Wang L, Nie D, Li G, Puybareau E, Dolz J, Zhang Q, Wang F, Xia J, Wu Z, Chen J, Thung K-H, Bui TD, Shin J, Zeng G, Zheng G, Fonov VS, Doyle A, Xu Y, Moeskops P, Pluim JPW, Desrosiers C, Ayed IB, Sanroma G, Benkarim OM, Casamitjana A, Vilaplana V, Lin W, Li G, Shen D (2019) Benchmark on automatic 6-month-old infant brain segmentation algorithms: the iseg-2017 challenge. IEEE Trans Med Imaging

  51. Wang S, Kuklisova-Murgasova M, Schnabel JA (2012) An atlas-based method for neonatal MR brain tissue segmentation. In: MICCAI Grand challenge: Neonatal brain segmentation, pp 28–35

  52. Warfield SK, Zou KH, Wells WM (2004) Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging 23(7):903–921

    Article  Google Scholar 

  53. Weisenfeld NI, Mewes AUJ, Warfield SK (2006) Segmentation of newborn brain MRI. In: 3Rd IEEE international symposium on biomedical imaging: Nano to macro, 2006, pp 766–769

  54. Weisenfeld NI, Warfield SK (2009) Automatic segmentation of newborn brain MRI. NeuroImage 47(2):564–572

    Article  Google Scholar 

  55. Wu J, Avants B (2012) Automatic registration-based segmentation for neonatal brains using ANTs and Atropos. In: MICCAI Grand challenge: Neonatal brain segmentation (neobrains12), pp 36–43

  56. Xue H, Srinivasan L, Jiang S, Rutherford M, Edwards DA, Rueckert D, Hajnal JV (2007) Automatic segmentation and reconstruction of the cortex from neonatal MRI. NeuroImage 38(3):461–477

    Article  Google Scholar 

  57. Yang J, Yang J, Zhang D, Lu J (2003) Feature fusion: parallel strategy vs. serial strategy. Pattern Recognit 36(6):1369–1381

    Article  MATH  Google Scholar 

  58. Yu L, Cheng J-Z, Dou Q, Yang X, Chen H, Qin J (2017) Automatic 3D cardiovascular MR segmentation with densely-connected volumetric convnets. In: Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, Duchesne S (eds) International conference on medical image computing and computer assisted intervention - MICCAI. Springer International Publishing, Cham, pp 287–295

  59. Zhang W, Li R, Deng H, Li W, Lin W, Ji S, Shen D (2015) Deep convolutional neural networks for multi-modality isointense infant brain image segmentation. NeuroImage 108:214–224

    Article  Google Scholar 

  60. Zhuang S, Awate SP, Licht DJ, Gee JC (2007) Clinical neonatal brain MRI segmentation using adaptive nonparametric data models and intensity-based markov priors. In: Proceedings of the 10th international conference on medical image computing and computer-assisted intervention - volume part I, MICCAI’07. Springer, Berlin, pp 883–890

Download references

Acknowledgment

This research is supported by National Key Research and Development Program of China under grant 2018YFB1003500.

Author information

Authors and Affiliations

  1. Services Computing Technology and System Laboratory, School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Saqib Qamar, Hai Jin, Ran Zheng & Parvez Ahmad

  2. Cluster and Grid Computing Laboratory, School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Saqib Qamar, Hai Jin, Ran Zheng & Parvez Ahmad

  3. Big Data Technology and System Laboratory, School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China

    Saqib Qamar, Hai Jin, Ran Zheng & Parvez Ahmad

Authors
  1. Saqib Qamar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ran Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Parvez Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Saqib Qamar.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qamar, S., Jin, H., Zheng, R. et al. Multi stream 3D hyper-densely connected network for multi modality isointense infant brain MRI segmentation. Multimed Tools Appl 78, 25807–25828 (2019). https://doi.org/10.1007/s11042-019-07829-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-07829-1

Keywords

Search

Navigation