Skip to main content
SpringerLink
Log in

MMNet: A multi-scale deep learning network for the left ventricular segmentation of cardiac MRI images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the development of deep learning network models, the automatic segmentation of medical images is becoming increasingly popular. Left ventricular cavity segmentation is an important step in the diagnosis of cardiac disease, but post-processing segmentation is a time-consuming and challenging task. That is why a fully automated segmentation method can assist specialists in increasing their efficiency. Inspired by the power of deep neural networks, a multi-scale multi-skip connection network (MMNet) model is proposed to fully automate the left ventricular segmentation of cardiac magnetic resonance imaging (MRI) images; this model is simple and efficient and has high segmentation accuracy without pre-detecting left ventricular localization. MMNet redesigns the classic encoder and decoder to take advantage of multi-scale feature information, effectively solving the problem of difficult segmentation due to blurred left ventricular edge information and the low accuracy of end-systolic segmentation of the cardiac area. In the model encoding stage, a multi-scale feature fusion module applying dilated convolution is proposed to obtain richer semantic information from different perceptual fields. The decoding stage reconstructs the full-size skip connection structure to make full use of the feature information obtained from different layers for contextual semantic information fusion. At the same time, a pre-activation module is used before each weighting layer to prevent overfitting phenomena from arising. The experimental results demonstrate that the proposed model has better segmentation performance than advanced benchmark models. Ablation experiments show that the proposed modules are effective at improving segmentation results. Therefore, MMNet is a promising approach for the left ventricular fully automated segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Code Availability

Some of the codes generated or used during the study are is available from the corresponding author by request.

References

  1. Salerno M, Sharif B, Arheden H, Kumar A, Axel L, Li D, Neubauer S (2017) Recent advances in cardiovascular magnetic resonance: techniques and applications. Circulation: Cardiovascular Imaging 10(6):e003951

    Google Scholar 

  2. Xue W, Brahm G, Pandey S, Leung S, Li S (2018) Full left ventricle quantification via deep multitask relationships learning. Medical Image Analysis 43:54–65

    Article  Google Scholar 

  3. Bernard O, Lalande A, Zotti C, Cervenansky F, Yang X, Heng PA, Jodoin PM (2018) Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Transactions On Medical Imaging 37(11):2514–2525

    Article  Google Scholar 

  4. Duan J, Bello G, Schlemper J, Bai W, Dawes TJ, Biffi C, Rueckert D (2019) Automatic 3D bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach. IEEE Transactions on Medical Imaging 38(9):2151–2164

    Article  Google Scholar 

  5. Budai A, Suhai FI, Csorba K, Toth A, Szabo L, Vago H, Merkely B (2020) Fully automatic segmentation of right and left ventricle on short-axis cardiac MRI images. Comput Med Imaging Graph 85:101786

    Article  Google Scholar 

  6. Leclerc S, Smistad E, ØStvik A, Cervenansky F, Espinosa F, Espeland T, Bernard O (2020) LU-Net: A Multistage Attention Network to Improve the Robustness of Segmentation of Left Ventricular Structures in 2-D Echocardiography. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 67 (12):2519–2530

    Article  Google Scholar 

  7. Penso M, Moccia S, Scafuri S, Muscogiuri G, Pontone G, Pepi M, Caiani EG (2021) Automated left and right ventricular chamber segmentation in cardiac magnetic resonance images using dense fully convolutional neural network. Comput Methods Prog Biomed 204:106059

    Article  Google Scholar 

  8. Mahapatra D (2013) Cardiac image segmentation from cine cardiac MRI using graph cuts and shape priors. Journal of Digital Imaging 26(4):721–730

    Article  Google Scholar 

  9. Auger DA, Zhong X, Epstein FH, Meintjes EM, Spottiswoode BS (2014) Semi-automated left ventricular segmentation based on a guide point model approach for 3D cine DENSE cardiovascular magnetic resonance. J Cardiovasc Magn Reson 16:1–12

    Article  Google Scholar 

  10. Wang L, Pei M, Codella NC, Kochar M, Weinsaft JW, Li J, Wang Y (2015) Left ventricle: fully automated segmentation based on spatiotemporal continuity and myocardium information in cine cardiac magnetic resonance imaging (LV-FAST). BioMed Research International

  11. Liu X, Zhu X, Li M, Wang L, Zhu E, Liu T, Gao W (2019) Multiple kernel k k-means with incomplete kernels. IEEE Transactions on Pattern Analysis and Machine Intelligence 42 (5):1191–1204

    Google Scholar 

  12. Luo Y, Yang B, Xu L, Hao L, Liu J, Yao Y, van de Vosse F (2018) Segmentation of the left ventricle in cardiac MRI using a hierarchical extreme learning machine model. International Journal of Machine Learning and Cybernetics 9(10):1741– 1751

    Article  Google Scholar 

  13. Tsang W, Wang B, Ghaziani ZN, Sun J, Chan R, Rakowski H (2020) Machine learning for left ventricular segmentation and scar quantification in hypertrophic cardiomyopathy patients. Can J Cardiol 36(10):S81–S82

    Article  Google Scholar 

  14. Yu X, Ye X, Gao Q (2020) Infrared handprint image restoration algorithm based on apoptotic mechanism. IEEE Access 8:47334–47343

    Article  Google Scholar 

  15. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  16. Sun J, Peng Y, Guo Y, Li D (2021) Segmentation of the multimodal brain tumor image used the multi-pathway architecture method based on 3D FCN. Neurocomputing 423:34–45

    Article  Google Scholar 

  17. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, Cham, pp 234–241

  18. Isensee F, Jaeger PF, Full PM, Wolf I, Engelhardt S, Maier-Hein KH (2017) Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. In: International workshop on statistical atlases and computational models of the heart. Springer, Cham, pp 120–129

  19. Zotti C, Luo Z, Lalande A, Jodoin PM (2018) Convolutional neural network with shape prior applied to cardiac MRI segmentation. IEEE Journal of Biomedical and Health Informatics 23(3):1119–1128

    Article  Google Scholar 

  20. Khened M, Kollerathu VA, Krishnamurthi G (2019) Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers. Medical Image Analysis 51:21–45

    Article  Google Scholar 

  21. Yang X, Zhang Y, Lo B, Wu D, Liao H, Zhang Y (2020), DBAN: Adversarial Network with Multi-Scale Features for Cardiac MRI Segmentation. IEEE Journal of Biomedical and Health Informatics

  22. Cui H, Yuwen C, Jiang L, Xia Y, Zhang Y (2021) Multiscale attention guided U-Net architecture for cardiac segmentation in short-axis MRI images. Comput Methods Prog Biomed 106142

  23. Bernard O, Lalande A, Zotti C, Cervenansky F, Yang X, Heng PA, Sanroma G (2018) Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Transactions on Medical Imaging 37(11):2514–2525

    Article  Google Scholar 

  24. Radau P, Lu Y, Connelly K, Paul G, Dick AJWG, Wright G (2009) Evaluation framework for algorithms segmenting short axis cardiac MRI. The MIDAS Journal-Cardiac MR Left Ventricle Segmentation Challenge 49

  25. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  26. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  27. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  28. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv:1511.07122

  29. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  30. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision. Springer, Cham, pp 630–645

  31. Jégou S, Drozdzal M, Vazquez D, Romero A, Bengio Y (2017) The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 11–19

  32. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, pp 3–11

  33. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980

  34. Painchaud N, Skandarani Y, Judge T, Bernard O, Lalande A, Jodoin PM (2020) Cardiac segmentation with strong anatomical guarantees. IEEE Trans Med Imaging 39(11):3703–3713

    Article  Google Scholar 

  35. Pop M, Sermesant M, Zhao J, Li S, McLeod K, Young A, ..., Mansi T. (eds) (2019) Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges, vol 11395. Springer, Berlin

    Google Scholar 

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China[Grant No. 61976126], Shandong Nature Science Foundation of China [Grant No. ZR2017MF054, ZR2019MF003, ZR2020MF044].

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, 266590, Shandong, China

    Ziyue Wang, Yanjun Peng, Dapeng Li, Yanfei Guo & Bin Zhang

  2. Shandong Province Key Laboratory of Wisdom Mining Information Technology, Shandong University of Science and Technology, Qingdao, 266590, Shandong, China

    Yanjun Peng

Authors
  1. Ziyue Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanjun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dapeng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanfei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Ziyue Wang proposed the method and conducted the experiments, analysed the data and wrote the manuscript. Yanjun Peng supervised the project and participated in manuscript revisions. Dapeng Li and Yanfei Guo provided critical reviews that helped improve the manuscript.

Corresponding author

Correspondence to Yanjun Peng.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflicts of interest.

Additional information

Availability of data and materials

Data related to the current study are available from the corresponding author on reasonable request.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Peng, Y., Li, D. et al. MMNet: A multi-scale deep learning network for the left ventricular segmentation of cardiac MRI images. Appl Intell 52, 5225–5240 (2022). https://doi.org/10.1007/s10489-021-02720-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02720-9

Keywords

Search

Navigation