Skip to main content

Advertisement

Springer Nature Link
Log in

Seg-CapNet: A Capsule-Based Neural Network for the Segmentation of Left Ventricle from Cardiac Magnetic Resonance Imaging

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Deep neural networks (DNNs) have been extensively studied in medical image segmentation. However, existing DNNs often need to train shape models for each object to be segmented, which may yield results that violate cardiac anatomical structure when segmenting cardiac magnetic resonance imaging (MRI). In this paper, we propose a capsule-based neural network, named Seg-CapNet, to model multiple regions simultaneously within a single training process. The Seg-CapNet model consists of the encoder and the decoder. The encoder transforms the input image into feature vectors that represent objects to be segmented by convolutional layers, capsule layers, and fully-connected layers. And the decoder transforms the feature vectors into segmentation masks by up-sampling. Feature maps of each down-sampling layer in the encoder are connected to the corresponding up-sampling layers, which are conducive to the backpropagation of the model. The output vectors of Seg-CapNet contain low-level image features such as grayscale and texture, as well as semantic features including the position and size of the objects, which is beneficial for improving the segmentation accuracy. The proposed model is validated on the open dataset of the Automated Cardiac Diagnosis Challenge 2017 (ACDC 2017) and the Sunnybrook Cardiac Magnetic Resonance Imaging (MRI) segmentation challenge. Experimental results show that the mean Dice coefficient of Seg-CapNet is increased by 4.7% and the average Hausdorff distance is reduced by 22%. The proposed model also reduces the model parameters and improves the training speed while obtaining the accurate segmentation of multiple regions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Cootes T F, Taylor C J, Cooper D H et al. Active shape models—Their training and application. Computer Vision and Image Understanding, 1995, 61(1): 38-59. https://doi.org/10.1006/cviu.1995.1004.

    Article  Google Scholar 

  2. Soliman A, Khalifa F, Elnakib A et al. Accurate lungs segmentation on CT chest images by adaptive appearance-guided shape modeling. IEEE Transactions on Medical Imaging, 2016, 36(1): 263-276. https://doi.org/10.1109/TMI.2016.2606370.

    Article  Google Scholar 

  3. Cootes T F, Edwards G J, Taylor C J. Active appearance models. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2001, 23(6): 681-685. https://doi.org/10.1109/34.927467.

  4. Matthews l, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60: 135-164. https://doi.org/10.1023/B:VISI.0000029666.37597.d3

  5. Wachinger C, Fritscher K, Sharp G et al. Contour-driven atlas-based segmentation. IEEE Transactions on Medical Imaging, 2015, 34(12): 2492-2505. https://doi.org/10.1109/TMI.2015.2442753.

    Article  Google Scholar 

  6. Maintz J B, Viergever M A. A survey of medical image registration. Medical Image Analysis, 1998, 2(1): 1-36. https://doi.org/10.1016/S1361-8415(01)80026-8.

    Article  Google Scholar 

  7. Litjens G, Kooi T, Bejnordi B E et al. A survey on deep learning in medical image analysis. Medical Image Analysis, 2017, 42: 60-88. https://doi.org/10.1016/j.media.2017.07.005.

    Article  Google Scholar 

  8. LeCun Y, Bengio Y, Hinton G E. Deep learning. Nature, 2015, 521(7553): 436-444. https://doi.org/10.1038/nature14539.

  9. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In Proc. the 26th Int. Conference on Neural Information Processing Systems, December 2012, pp.1097-1105. https://doi.org/10.5555/2999134.2999257.

  10. Badrinarayanan V, Handa V, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv:1505.07293, 2015. https://arxiv.org/pdf/1505.07293.pdf, March, 2020.

  11. Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615.

    Article  Google Scholar 

  12. Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1520-1528. https://doi.org/10.1109/ICCV.2015.178.

  13. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In Proc. the 2015 IEEE International Conference on Computer Vision and Pattern Recognition, June 2015, pp.3431-3440. https://doi.org/10.1109/CVPR.2015.7298965.

  14. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, October 2015, pp.234-241. https://doi.org/10.1007/978-3-319-24574-4_28.

  15. Gu Z W, Cheng J, Fu H Z et al. CE-Net: Context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, 2019, 38(10): 2281-2292. https://doi.org/10.1109/TMI.2019.2903562.

    Article  Google Scholar 

  16. Wang G, Liu X, Li C et al. A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images. IEEE Transactions on Medical Imaging, 2020, 39(8): 2653-2663. https://doi.org/10.1109/TMI.2020.3000314.

    Article  Google Scholar 

  17. Hu X, Li F, Samaras D et al. Topology-preserving deep image segmentation. In Proc. the 33rd Annual Conference of Neural Information Processing Systems, December 2019, pp.5658-5669.

  18. Karimi D, Salcudean S E. Reducing the Hausdorff Distance in medical image segmentation with convolutional neural networks. IEEE Transactions on Medical Imaging, 2020, 39(2): 499-513. https://doi.org/10.1109/TMI.2019.2930068.

    Article  Google Scholar 

  19. Moltz J H, Hänsch A, Lassen-Schmidt B et al. Learning a loss function for segmentation: A feasibility study. In Proc. the 17th IEEE Int. Biomedical Imaging Symp., April 2020, pp.357-360. https://doi.org/10.1109/ISBI45749.2020.9098557.

  20. Hinton G E, Alex K, Wang S D. Transforming auto-encoders. In Proc. the 21st Int. Conference on Artificial Neural Networks, June 2011, pp.44-51. https://doi.org/10.1007/978-3-642-21735-7_6.

  21. Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules. In Proc. the 31 st Int. Conference on Neural Information Processing Systems, December 2017, pp.3856-3866.

  22. LaLonde R, Bagci U. Capsules for object segmentation. arXiv:1804.04241, 2018. https://arxiv.org/pdf/1804.0424-1v1.pdf, March, 2020.

  23. Kromm C, Rohr K. Inception capsule network for retinal blood vessel segmentation and centerline extraction. In Proc. the 17th IEEE Int. Biomedical Imaging Symp., April 2020, pp.1223-1226. https://doi.org/10.1109/ISBI45749.2020.9098538.

  24. He Y, Qin W, Wu Y et al. Automatic left ventricle segmentation from cardiac magnetic resonance images using a capsule network. Journal of X-Ray Science and Technology, 2020, 28(3):541-553. https://doi.org/10.3233/XST-190621.

    Article  Google Scholar 

  25. Hara K, Saito D, Shouno H. Analysis of function of rectified linear unit used in deep learning. In Proc. the 2015 International Joint Conference on Neural Networks, July 2015. https://doi.org/10.1109/IJCNN.2015.7280578.

  26. Loffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167, 2015. https://arxiv.org/pdf/1502.03167.pdf, March 2020.

  27. Bernard O, Lalande A, Zotti C et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE Transactions on Medical Imaging, 2018, 37(11): 2514-2525. https://doi.org/10.1109/TMI.2018.2837502.

    Article  Google Scholar 

  28. Chen L, Papandreou G, Schroff F et al. Rethinking atrous convolution for semantic image segmentation. arXiv: 1706.05587, 2017. https://arxiv.org/abs/1706.05587, June 2020.

  29. Zhou Z W, Siddiquee M, Tajbakhsh N et al. UNet++: A nested U-Net architecture for medical image segmentation. In Proc. the 4th International Workshop on Deep Learning in Medical Image Analysis, September 2018, pp.3-11. https://doi.org/10.1007/978-3-030-00889-5_1.

Download references

Author information

Authors and Affiliations

  1. School of Software, Zhengzhou University, Zhengzhou, 450000, China

    Yang-Jie Cao, Shuang Wu, Chang Liu, Nan Lin, Cong Yang & Jie Li

  2. Center of Modern Analysis and Gene Sequencing, Zhengzhou University, Zhengzhou, 450000, China

    Yuan Wang

  3. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200000, China

    Jie Li

Authors
  1. Yang-Jie Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Shuang Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Chang Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Nan Lin
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yuan Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Cong Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Jie Li
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Cong Yang.

Supplementary Information

ESM 1

(PDF 344 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, YJ., Wu, S., Liu, C. et al. Seg-CapNet: A Capsule-Based Neural Network for the Segmentation of Left Ventricle from Cardiac Magnetic Resonance Imaging. J. Comput. Sci. Technol. 36, 323–333 (2021). https://doi.org/10.1007/s11390-021-0782-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-021-0782-5

Keywords