Skip to main content
SpringerLink
Log in

Iterative 3D feature enhancement network for pancreas segmentation from CT images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Automatic and accurate pancreas segmentation from 3D computed tomography volumes is a crucial prerequisite for computer-aided diagnosis, intraoperative planning and guidance. However, this is a challenging task because of the high inter-subject variability in the shape and location of the pancreas, as well as the existence of the surrounding organs. In order to address the above challenges, we propose a novel iterative 3D feature enhancement network to segment pancreas accurately. Specifically, the multi-level integrated features and the individual features at different levels can be progressively enhanced in an iterative manner by leveraging the complementary information encoded in different features. Therefore, the non-pancreas information at lower layers can be suppressed, and the fine details of pancreas at higher layers can be increased. In addition, because the pancreas region occupies only a small part of the scan, in order to prevent the final predictions from being biased toward the background class, we design the Dice similarity coefficients loss function in the training phase to mitigate this issue. Meanwhile, deep supervision with auxiliary classifier is incorporated in the intermediate layers at each iteration to guide the back-propagation of gradient flows and boost the discriminative capability at lower layers. Finally, in order to verify the effectiveness of the proposed method, we evaluated our approach on the publicly available NIH pancreas segmentation dataset. Extensive experiments illustrate that the proposed method achieves better performance than the state-of-the-art algorithms, and it can be easily applied to other volumetric image segmentation tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Cai J, Lu L, Xie Y, Xing F, Yang L (2017) Improving deep pancreas segmentation in CT and MRI images via recurrent neural contextual learning and direct loss function. arXiv preprint arXiv:170704912

  2. Cai J, Lu L, Xie Y, Xing F, Yang L (2017) Pancreas segmentation in MRI using graph-based decision fusion on convolutional neural networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 674–682

  3. Chen H, Dou Q, Wang X, Qin J, Heng PA (2016) Mitosis detection in breast cancer histology images via deep cascaded networks. In: Thirtieth AAAI conference on artificial intelligence

  4. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  5. Christ PF, Elshaer MEA, Ettlinger F, Tatavarty S, Bickel M, Bilic P, Rempfler M, Armbruster M, Hofmann F, D’Anastasi M et al (2016) Automatic liver and lesion segmentation in ct using cascaded fully convolutional neural networks and 3D conditional random fields. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 415–423

  6. Chu C, Oda M, Kitasaka T, Misawa K, Fujiwara M, Hayashi Y, Nimura Y, Rueckert D, Mori K (2013) Multi-organ segmentation based on spatially-divided probabilistic atlas from 3D abdominal CT images. Med Image Comput Comput Assist Interv 16(2):165–172

    Google Scholar 

  7. Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O (2016) 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 424–432

  8. Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. In: Medical image computing and computer-assisted intervention—MICCAI 2013. Springer, pp 411–418

  9. Dou Q, Chen H, Yu L, Zhao L, Qin J, Wang D, Mok VC, Shi L, Heng PA (2016) Automatic detection of cerebral microbleeds from mr images via 3D convolutional neural networks. IEEE Trans Med Imaging 35(5):1182–1195

    Article  Google Scholar 

  10. Dou Q, Chen H, Jin Y, Lin H, Qin J, Heng PA (2017) Automated pulmonary nodule detection via 3D convnets with online sample filtering and hybrid-loss residual learning. In: International conference on medical image computing and computer-assisted intervention, pp 630–638

  11. Farag A, Lu L, Roth HR, Liu J, Turkbey E, Summers RM (2017) A bottom-up approach for pancreas segmentation using cascaded superpixels and (deep) image patch labeling. IEEE Trans Image Process 26(1):386–399

    Article  MathSciNet  MATH  Google Scholar 

  12. Feng Y, Zhang L, Mo J (2018) Deep manifold preserving autoencoder for classifying breast cancer histopathological images. IEEE/ACM Trans Comput Biol Bioinform. https://doi.org/10.1109/TCBB.2018.2858763

    Article  Google Scholar 

  13. Feng Y, Zhang L, Yi Z (2018) Breast cancer cell nuclei classification in histopathology images using deep neural networks. Int J Comput Assist Radiol Surg 13(2):179–191

    Article  Google Scholar 

  14. Hammon M, Cavallaro A, Erdt M, Dankerl P, Kirschner M, Drechsler K, Wesarg S, Uder M, Janka R (2013) Model-based pancreas segmentation in portal venous phase contrast-enhanced CT images. J Digit Imaging 26(6):1082–90

    Article  Google Scholar 

  15. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  16. Hu X, Zhu L, Qin J, Fu CW, Heng PA (2018) Recurrently aggregating deep features for salient object detection. In: AAAI

  17. Ji Y, Zhang H, Wu QJ (2018) Salient object detection via multi-scale attention CNN. Neurocomputing 322:130–140

    Article  Google Scholar 

  18. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM international conference on multimedia. ACM, pp 675–678

  19. Jiang H, Wang X, Shi S (2013) Pancreas segmentation using level-set method based on statistical shape model. J Pure Appl Microbiol 7:433–440

    Google Scholar 

  20. Karasawa K, Kitasaka T, Oda M, Nimura Y, Hayashi Y, Fujiwara M, Misawa K, Rueckert D, Mori K (2015) Structure specific atlas generation and its application to pancreas segmentation from contrasted abdominal CT volumes. In: International MICCAI workshop on medical computer vision. Springer, pp 47–56

  21. Liu W, Rabinovich A, Berg AC (2015) Parsenet: looking wider to see better. arXiv preprint arXiv:150604579

  22. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  23. Mansoor A, Bagci U, Xu Z, Foster B, Olivier KN, Elinoff JM, Suffredini AF, Udupa JK, Mollura DJ (2015) A generic approach to pathological lung segmentation. IEEE Trans Med Imaging 34(1):354

    Article  Google Scholar 

  24. Milletari F, Navab N, Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: Fourth international conference on 3D vision, pp 565–571

  25. Mo J, Zhang L (2017) Multi-level deep supervised networks for retinal vessel segmentation. Int J Comput Assist Radiol Surg 12(12):2181–2193

    Article  Google Scholar 

  26. Mo J, Zhang L, Feng Y (2018) Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks. Neurocomputing 290:161–171

    Article  Google Scholar 

  27. Montufar GF, Pascanu R, Cho K, Bengio Y (2014) On the Number of Linear Regions of Deep Neural Networks. Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ (eds) Advances in neural information processing systems, vol 27. Curran Associates, Inc., pp 2924–2932. http://papers.nips.cc/paper/5422-on-the-number-of-linear-regions-of-deep-neural-networks.pdf

  28. Oda M, Shimizu N, Karasawa K, Nimura Y, Kitasaka T, Misawa K, Fujiwara M, Rueckert D, Mori K (2016) Regression forest-based atlas localization and direction specific atlas generation for pancreas segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 556–563

  29. Poynton CB, Chen KT, Chonde DB, Izquierdogarcia D, Gollub RL, Gerstner ER, Batchelor TT, Catana C (2014) Probabilistic atlas-based segmentation of combined T1-weighted and DUTE MRI for calculation of head attenuation maps in integrated PET/MRI scanners. Am J Nucl Med Mol Imaging 4(2):160–71

    Google Scholar 

  30. Roth H, Oda M, Shimizu N, Oda H, Hayashi Y, Kitasaka T, Fujiwara M, Misawa K, Mori K (2018) Towards dense volumetric pancreas segmentation in CT using 3D fully convolutional networks. In: Medical imaging 2018—image processing, vol 10574. International Society for Optics and Photonics, p 105740B

  31. Roth HR, Lu L, Farag A, Shin HC, Liu J, Turkbey EB, Summers RM (2015) Deeporgan: multi-level deep convolutional networks for automated pancreas segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 556–564

  32. Roth HR, Lu L, Farag A, Sohn A, Summers RM (2016) Spatial aggregation of holistically-nested networks for automated pancreas segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 451–459

  33. Roth HR, Lu L, Lay N, Harrison AP, Farag A, Sohn A, Summers RM (2018) Spatial aggregation of holistically-nested convolutional neural networks for automated pancreas localization and segmentation. Med Image Anal 45:94–107

    Article  Google Scholar 

  34. Saito A, Nawano S, Shimizu A (2016) Joint optimization of segmentation and shape prior from level-set-based statistical shape model, and its application to the automated segmentation of abdominal organs. Med Image Anal 28(33):46–65

    Article  Google Scholar 

  35. Shrivastava A, Sukthankar R, Malik J, Gupta A (2016) Beyond skip connections: top-down modulation for object detection. arXiv preprint arXiv:161206851

  36. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556

  37. Toshev A, Szegedy C (2014) Deeppose: human pose estimation via deep neural networks. In: Computer vision and pattern recognition, pp 1653–1660

  38. Wang J, Zhang L, Chen Y, Yi Z (2018) A new delay connection for long short-term memory networks. Int J Neural Syst 28(6):1750061

    Article  Google Scholar 

  39. Wang J, Ju R, Chen Y, Zhang L, Hu J, Wu Y, Dong W, Zhong J, Yi Z (2018) Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine 35:361–368

    Article  Google Scholar 

  40. Wang J, Zhang L, Guo Q, Yi Z (2018) Recurrent neural networks with auxiliary memory units. IEEE Trans Neural Netw Learn Syst 29(5):1652–1661

    Article  MathSciNet  Google Scholar 

  41. Wang L, Zhang L, Yi Z (2017) Trajectory predictor by using recurrent neural networks in visual tracking. IEEE Trans Cybern 47(10):3172–3183. https://doi.org/10.1109/TCYB.2017.2705345

    Article  Google Scholar 

  42. Wolz R, Chu C, Misawa K, Fujiwara M, Mori K, Rueckert D (2013) Automated abdominal multi-organ segmentation with subject-specific atlas generation. IEEE Trans Med Imaging 32(9):1723–1730

    Article  Google Scholar 

  43. Yang G, Gu J, Chen Y, Liu W, Tang L, Shu H, Toumoulin C (2014) Automatic kidney segmentation in CT images based on multi-atlas image registration. In: Engineering in medicine and biology society, pp 5538–5541

  44. Zhang H, Ji Y, Huang W, Liu L (2019) Sitcom-star-based clothing retrieval for video advertising: a deep learning framework. Neural Comput Appl 31(11):7361–7380

    Article  Google Scholar 

  45. Zhang L, Yi Z (2007) Dynamical properties of background neural networks with uniform firing rate and background input. Chaos Solitons Fractals 33(3):979–985. https://doi.org/10.1016/j.chaos.2006.01.061

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhang L, Yi Z, Amari S (2018) Theoretical study of oscillator neurons in recurrent neural networks. IEEE Trans Neural Netw Learn Syst 29(11):5242–5248

    Article  MathSciNet  Google Scholar 

  47. Zhou Y, Xie L, Shen W, Wang Y, Fishman EK, Yuille AL (2017) A fixed-point model for pancreas segmentation in abdominal CT scans. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 693–701

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant 61772353); Foundation for Youth Science and Technology Innovation Research Team of Sichuan Province (Grants 2016TD0018); Sichuan University Innovation Sparks Project (Grant 2018SCUH0040); and Natural Science Foundation of Inner Mongolia Autonomous Region of China (Grant 2018MS06002).

Author information

Authors and Affiliations

  1. College of Computer Science, Sichuan University, Chengdu, 610065, China

    Juan Mo, Lei Zhang & Yan Wang

  2. School of Science, Inner Mongolia University of Science and Technology, Baotou, 014010, China

    Juan Mo

  3. West China Second University Hospital, Sichuan University, Chengdu, 610065, China

    Haiying Huang

Authors
  1. Juan Mo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haiying Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal rights

No animal or human experiments were conducted as part of this research.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mo, J., Zhang, L., Wang, Y. et al. Iterative 3D feature enhancement network for pancreas segmentation from CT images. Neural Comput & Applic 32, 12535–12546 (2020). https://doi.org/10.1007/s00521-020-04710-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-04710-3

Keywords

Search

Navigation