Skip to main content
SpringerLink
Log in

Path aggregation U-Net model for brain tumor segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The deep neural network has been widely used in semantic segmentation, especially in tumor image segmentation. The segmentation performance of traditional methods cannot meet the high standard of clinical application. In this paper, we propose a new neural network model called path aggregation U-Net (PAU-Net) model for brain tumor segmentation with multi-modality magnetic resonance imaging (MRI). Specifically, we shorten the distance between output layers and deep features by bottom-up path aggregation encoder (PA), reducing the introduction of noises. We present the enhanced decoder (ED) to reserve more intact information. The efficient feature pyramid (EFP) is used to improve mask prediction further, using fewer resources to complete the feature pyramid effect. Finally, experiments in BraTS2017 and BraTS2018 datasets are performed. The results show that the proposed method outperforms state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Adelson EH, Anderson CH, Bergen JR, Burt PJ, Ogden JM (1984) Pyramid methods in image processing. RCA engineer 29(6):33–41

    Google Scholar 

  2. Alex V, Safwan M, Krishnamurthi G (2017) Brain tumor segmentation from multi modal mr images using fully convolutional neural network. In: International MICCAI BraTS challenge, pp 1–8

  3. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby JS, Freymann JB, Farahani K, Davatzikos C (2017) Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features. Sci Data 4:170117

    Article  Google Scholar 

  4. Bauer S, Wiest R, Nolte L-P, Reyes M (2013) A survey of mri-based medical image analysis for brain tumor studies. Phys Med Bio 58(13):R97

    Article  Google Scholar 

  5. Fan J, Cao X, Yap PT, Birnet DS (2019) Brain image registration using dual-supervised fully convolutional networks. Med Image Anal 54:193–206

    Article  Google Scholar 

  6. Ghiasi G, Fowlkes CC (2016) Laplacian pyramid reconstruction and refinement for semantic segmentation. In: European conference on computer vision, pp 519–534. Springer

  7. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Computer Vision (ICCV), 2017 IEEE international conference on computer vision, pp 2980–2988 IEEE

  8. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  9. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In CVPR 1:3

    Google Scholar 

  10. Ioffe S, Szegedy C (2015) Batch normalization Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  11. Jégou S., Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In: Computer vision and pattern recognition workshops (CVPRW), 2017 IEEE conference on, pp 1175–1183. IEEE

  12. Kim G (2017) Brain tumor segmentation using deep u-net. In: International MICCAI BraTS challenge, pp 154–160

  13. Kong X, Sun G, Wu Q, Liu J, Lin F. (2018) Hybrid pyramid u-net model for brain tumor segmentation. In: International conference on intelligent information processing, pp 346–355. Springer

  14. Lin F, et al. (2019) FMNet: Feature Mining Networks for Brain Tumor Segmentation. In: 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), pp. 555–560. Portland, OR, USA,

  15. Lin G, Milan A, Shen C, Reid ID (2017) Refinenet multi-path refinement networks for high-resolution semantic segmentation. In Cvpr 1:5

    Google Scholar 

  16. Lin Tsung-Yi, Dollár P., Girshick R, He K, Hariharan B, Belongie SJ (2017) Feature pyramid networks for object detection. In CVPR 1:4

    Google Scholar 

  17. Liu S, Qi LU, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8759–8768

  18. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  19. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, et al. (2014) The multimodal brain tumor image segmentation benchmark brats. IEEE trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  20. Milletari F, Navab N, Ahmadi S (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: 3D Vision (3DV), 2016 fourth international conference on, pp 565–571. IEEE

  21. Paszke A, Chaurasia A, Kim S, Culurciello E (2016) Enet: A deep neural network architecture for real-time semantic segmentation. arXiv:1606.02147

  22. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional Networks for Biomedical Image Segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241. Springer

  23. Shaikh M, Anand G, Acharya G, Amrutkar A, Alex V, Krishnamurthi G (2017) Brain tumor segmentation using dense fully convolutional neural network. In: International MICCAI brainlesion workshop, pp 309–319. Springer

  24. Shen H, Zhang J, Zheng W (2017) Efficient symmetry-driven fully convolutional network for multimodal brain tumor segmentation. In: Image processing (ICIP), 2017 IEEE international conference on, pp 3864–3868. IEEE

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  26. Zhang X, Yao L, Wang X, Monaghan J, Mcalpine D (2019) A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv:1905.04149

  27. Zhang YU, Zhang H, Chen X, Liu M, Zhu X, Lee SW, Shen D (2019) Strength and similarity guided group-level brain functional network construction for mci diagnosis. Pattern Recogn 88:421–430

    Article  Google Scholar 

  28. Zhao H, Qi X, Shen X, shi J, jia j (2017) Icnet for real-time semantic segmentation on high-resolution images. arXiv:1704.08545

  29. Zhou T, Thung KH, Zhu X, Shen D (2019) Effective feature learning and fusion of multimodality data using stage-wise deep neural network for dementia diagnosis. Hum Brain Mapp 40(3):1001–1016

    Article  Google Scholar 

  30. Zhou T, Fu H, Shen J , Chen G, Shao L Hi-Net: Hybrid-fusion Network for Multi-modal MR Image Synthesis, IEEE transactions on medical imaging, 2020.

Download references

Acknowledgments

The work is supported by the Fundamental Research Funds of Shandong University (Grant No. 2017JC013), the Shandong Province Key Innovation Project (Grant No. 2017CXGC1504, 2017CXGC1502) and the Natural Science Fundation of Shandong Province (Grant No. ZR2019MH049).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong University, Qingdao, China

    Fengming Lin, Qiang Wu, Ju Liu & Xiangmao Kong

  2. Institute of Brain and Brain-Inspired Science, Shandong University, Qingdao, China

    Fengming Lin, Qiang Wu, Ju Liu & Xiangmao Kong

  3. Department of Radiology, Qilu Hospital, Shandong University, Jinan, China

    Dawei Wang

Authors
  1. Fengming Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ju Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dawei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangmao Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiang Wu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lin, F., Wu, Q., Liu, J. et al. Path aggregation U-Net model for brain tumor segmentation. Multimed Tools Appl 80, 22951–22964 (2021). https://doi.org/10.1007/s11042-020-08795-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-08795-9

Keywords

Search

Navigation