Skip to main content

Advertisement

Springer Nature Link
Log in

Echocardiographic segmentation based on semi-supervised deep learning with attention mechanism

  • 1230 : Sentient Multimedia Systems and Visual Intelligence
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Echocardiographic examination is one of the main methods for clinical diagnosis, management and follow-up of heart diseases. Echocardiographic segmentation is an essential step for obtaining precise measurements and accurate diagnosis. However, the current methods are mostly time-consuming, relatively subjective, and produce inconsistent results due to varying ultrasound image quality. To solve these problems, we propose an automatic 2D echocardiographic segmentation method, which is objective and robust for the change of image quality. Specifically, our method first constructs an echocardiographic motion estimation network to extract the heart motion features for the echocardiographic segmentation network. Then, based on semi-supervised learning, the echocardiographic segmentation network is trained by labeled images’ ground truth and unlabeled images’ pseudo labels, which are derived from the motion features. In addition, we introduce attention mechanism to observe its impact on segmentation performance. Experimental results show that the peak signal-to-noise ratio and the structural similarity index between the target images and the images reconstructed by the motion features are over 30dB and 92%, respectively. The echocardiographic segmentation network achieves 95.93% accuracy and 90.94% dice similarity coefficient in the segmentation of cardiac end-diastolic, and achieves 96.06% accuracy and 91.51% dice similarity coefficient in the segmentation of cardiac end-systolic. These results prove that the motion features and segmentation results obtained from our method are effective and accurate. Our code is publicly available at: https://github.com/cherish-fere/motion_net

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Albishri AA, Shah SJH, Lee Y (2019) CU-net: cascaded U-net model for automated liver and lesion segmentation and summarization[C]//2019 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE:1416–1423

  2. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  3. Belaid A, Boukerroui D (2018) Local maximum likelihood segmentation of echocardiographic images with Rayleigh distribution[J]. SIViP 12(6):1087–1096

    Article  Google Scholar 

  4. Belaid A, Boukerroui D, Maingourd Y et al (2010) Phase-based level set segmentation of ultrasound images[J]. IEEE Trans Inf Technol Biomed 15(1):138–147

    Article  Google Scholar 

  5. Bernard O, Bosch JG, Heyde B et al (2015) Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography[J]. IEEE Trans Med Imaging 35(4):967–977

    Article  Google Scholar 

  6. Bonifazi G, Lista L, Menasce D et al (2021) A study on the possible merits of using symptomatic cases to trace the development of the COVID-19 pandemic[J]. European Phys J Plus 136(5):481

    Article  Google Scholar 

  7. Carneiro G, Nascimento JC, Freitas A (2011) The segmentation of the left ventricle of the heart from ultrasound data using deep learning architectures and derivative-based search methods[J]. IEEE Trans Image Process 21(3):968–982

    Article  MathSciNet  Google Scholar 

  8. Chen S, Ding C, Tao D (2020) Boundary-assisted region proposal networks for nucleus segmentation[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd international conference, Lima, Peru, October 4–8, 2020, proceedings, part V 23. Springer International Publishing, 2020: 279-288.

  9. Cikes M, Tong L, Sutherland GR et al (2014) Ultrafast cardiac ultrasound imaging: technical principles, applications, and clinical benefits[J]. JACC Cardiovasc Imaging 7(8):812–823

    Article  Google Scholar 

  10. Dash S, Acharya BR et al (2020) Deep learning techniques for biomedical and health informatics[M]. Springer International Publishing, Cham, Switzerland

    Book  Google Scholar 

  11. Datar M, Gionis A, Indyk P et al (2002) Maintaining stream statistics over sliding windows[J]. SIAM J Comput 31(6):1794–1813

    Article  MathSciNet  Google Scholar 

  12. Guo W, Zhao X, Zhang S et al (2022) Learning inter-class optical flow difference using generative adversarial networks for facial expression recognition[J]. Multimed Tools Appl:1–18

  13. Hamarneh G, Gustavsson T (2000) Combining snakes and active shape models for segmenting the human left ventricle in echocardiographic images[C]//Computers in Cardiology . Vol. 27 (cat. 00CH37163). IEEE, 2000: 115-118.

  14. He K, Zhang X, Ren S, et al. (2016) Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778

  15. Horimoto M, Takenaka T, Igarashi K et al (2000) Invasive examination of cardiovascular disease[J]. Rinsho byori. The Japanese. J Clin Pathol 48(2):128–137

    Google Scholar 

  16. Huang G, Liu Z, Van Der Maaten L, et al. (2017) Densely connected convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708.

  17. Jiang F, Grigorev A, Rho S et al (2018) Medical image semantic segmentation based on deep learning[J]. Neural Comput & Applic 29:1257–1265

    Article  Google Scholar 

  18. Kamranian Z, Naghsh Nilchi AR, Sadeghian H et al (2020) Joint motion boundary detection and CNN-based feature visualization for video object segmentation[J]. Neural Comput & Applic 32:4073–4091

    Article  Google Scholar 

  19. Keeney T, Fox AB, Jette DU et al (2019) Functional trajectories of persons with cardiovascular disease in late life[J]. J Am Geriatr Soc 67(1):37–42

    Article  Google Scholar 

  20. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980

  21. Leclerc S, Grenier T, Espinosa F et al (2017) A fully automatic and multi-structural segmentation of the left ventricle and the myocardium on highly heterogeneous 2D echocardiographic data[C]//2017 IEEE international Ultrasonics symposium (IUS). IEEE:1–4

  22. Leclerc S, Smistad E, Pedrosa J et al (2019) Deep learning for segmentation using an open large-scale dataset in 2D echocardiography[J]. IEEE Trans Med Imaging 38(9):2198–2210

    Article  Google Scholar 

  23. Lee DH (2013) Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks[C]//workshop on challenges in representation learning. ICML 3(2):896

    Google Scholar 

  24. Lei B, Xia Z, Jiang F et al (2020) Skin lesion segmentation via generative adversarial networks with dual discriminators[J]. Med Image Anal 64:101716

    Article  Google Scholar 

  25. Li H, He X, Zhou F et al (2018) Dense deconvolutional network for skin lesion segmentation[J]. IEEE J Biomed Health Inform 23(2):527–537

    Article  Google Scholar 

  26. Lin Z, Li S, Ni D et al (2019) Multi-task learning for quality assessment of fetal head ultrasound images[J]. Med Image Anal 58:101548

    Article  Google Scholar 

  27. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 3431-3440.

  28. Manson JAE, Cook NR, Lee IM et al (2019) Marine n− 3 fatty acids and prevention of cardiovascular disease and cancer[J]. N Engl J Med 380(1):23–32

    Article  Google Scholar 

  29. Oktay O, Schlemper J, Folgoc LL, et al. (2018)Attention u-net: Learning where to look for the pancreas[C]//Computer Vision and Pattern Recognition (CVPR). 1-14

  30. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer International Publishing, 2015: 234-241.

  31. Shengshou H et al (2019) Summary of the 2018 report on cardiovascular diseases in China [J] Chinese. Circ J 34(3):209–220

    Google Scholar 

  32. Smistad E, Østvik A (2017) 2D left ventricle segmentation using deep learning[C]//2017 IEEE international ultrasonics symposium (IUS). IEEE:1–4

  33. Swain D, Parmar B, Shah H, Gandhi A, Pradhan M, Kaur H et al (2022) Cardiovascular disease prediction using various machine learning algorithms. Journal of computer science [J]. J Comput Sci 18:993–1004. https://doi.org/10.3844/jcssp.2022.993.1004

    Article  Google Scholar 

  34. Ullah H, Ullah M, Uzair M (2019) A hybrid social influence model for pedestrian motion segmentation[J]. Neural Comput & Applic 31:7317–7333

    Article  Google Scholar 

  35. Woo S, Park J, Lee J Y, et al. (2018) Cbam: Convolutional block attention module[C]//Proceedings of the European conference on computer vision (ECCV). 3-19

  36. Ye M, Shen J (2020) Probabilistic structural latent representation for unsupervised embedding[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5457-5466

  37. Yurtkulu SC, Şahin YH, Unal G (2019) Semantic segmentation with extended DeepLabv3 architecture[C]//2019 27th signal processing and communications applications conference (SIU). IEEE:1–4

  38. Zhang L, Sheng Z, Li Y et al (2020) Image object detection and semantic segmentation based on convolutional neural network[J]. Neural Comput & Applic 32:1949–1958

    Article  Google Scholar 

  39. Zhao H, Shi J, Qi X, et al. (2017) Pyramid scene parsing network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2881-2890.

  40. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N et al (2018) Unet++: a nested u-net architecture for medical image segmentation[C]//deep learning in medical image analysis and multimodal learning for clinical decision support: 4th international workshop, DLMIA 2018, and 8th international workshop, ML-CDS 2018, held in conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, proceedings 4. Springer International Publishing:3–11

Download references

Acknowledgements

This work was supported partly by National Natural Science Foundation of China (Nos. 62271328, U22A2024, 62101338, 62201360 and U1902209), National Natural Science Foundation of Guangdong Province (Nos. 2019B1515120029, 2020B121201001 and 2021A1515110746), Shenzhen Key Basic Research Project (Nos. JCYJ20220818095809021, KCXFZ2020 1221173213036, and SGDX20201103095802007).

Author information

Author notes

  1. Jiajun Liang and Huijuan Pan contributed equally to this work.

Authors and Affiliations

  1. School of Biomedical Engineering, Health Science Center, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Shenzhen University, Shenzhen, 518060, Guangdong, China

    Jiajun Liang, Zhuo Xiang, Yali Qiu, Libao Guo, Tianfu Wang & Baiying Lei

  2. Department of Urology, Southern University of Science and Technology Hospital, Shenzhen, China

    Huijuan Pan

  3. Centre for Smart Health, School of Nursing, The Hong Kong Polytechnic University, Hong Kong, China

    Jing Qin

  4. Department of Ultrasound, Huazhong University of Science and Technology Union Shenzhen Hospital, Shenzhen, China

    Wei Jiang

Authors
  1. Jiajun Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huijuan Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhuo Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Qin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yali Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Libao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tianfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Wei Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Baiying Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Wei Jiang or Baiying Lei.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest to this work.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, J., Pan, H., Xiang, Z. et al. Echocardiographic segmentation based on semi-supervised deep learning with attention mechanism. Multimed Tools Appl 83, 36953–36973 (2024). https://doi.org/10.1007/s11042-023-16044-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16044-y

Keywords