Abstract
Segmentation of the left ventricle in 2D echocardiography is essential for cardiac function measures, such as ejection fraction. Fully-supervised algorithms have been used in the past to segment the left ventricle and then offline estimate ejection fraction, but it is costly and time-consuming to obtain annotated segmentation ground truths for training. To solve this issue, we propose a weakly/semi-supervised framework with multi-level geometric regularization of vertex (boundary points), boundary and region predictions. The framework benefits from learning discriminative features and neglecting boundary uncertainty via the proposed contrastive learning. Firstly, we propose a multi-level regularized semi-supervised paradigm, where the regional and boundary regularization is favourable owing to the intrinsic geometric coherence of vertex, boundary and region predictions. Secondly, we propose an uncertain region-aware contrastive learning mechanism along the boundary via a hard negative sampling strategy for labeled and unlabeled data at the pixel level. The uncertain region along the boundary is omitted, enabling generalized semi-supervised learning with reliable boundary prediction. Thirdly, for the first time, we proposed a differentiable ejection fraction estimation module along with 2D echocardiographic left ventricle segmentation for end-to-end training without offline post-processing. Supervision on ejection fraction can also serve as weak supervision for the left ventricular vertex, region and boundary predictions without further annotations. Experiments on the EchoNet-Dynamic datasets demonstrate that our method outperforms state-of-the-art semi-supervised approaches for segmenting left ventricle and estimating ejection fraction.
Y. Meng and Y. Zhang—co-first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
PNPOLY- point inclusion in polygon test. https://wrf.ecse.rpi.edu/Research/Short_Notes/ pnpoly.html. Accessed 01 Jun 2022
Cheng, B., Girshick, R., Dollár, P., Berg, A.C., Kirillov, A.: Boundary IoU: improving object-centric image segmentation evaluation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2021)
Deng, K., et al.: TransBridge: a lightweight transformer for left ventricle segmentation in echocardiography. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 63–72. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_7
Dodge, H.T., Sandler, H., Ballew, D.W., Lord, J.D., Jr.: The use of biplane angiocardiography for the measurement of left ventricular volume in man. Am. Heart J. 60(5), 762–776 (1960)
Folland, E., Parisi, A., Moynihan, P., Jones, D.R., Feldman, C.L., Tow, D.: Assessment of left ventricular ejection fraction and volumes by real-time, two- dimensional echocardiography. A comparison of cineangiographic and radionuclide techniques. Circulation 60(4), 760–766 (1979)
Gao, S., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.H.: Res2Net: a new multi-scale backbone architecture. IEEE Trans. Pattern Anal. Mach. Intell. 42, 652–662 (2019)
Gu, R., et al.: Contrastive semi-supervised learning for domain adaptive segmentation across similar anatomical structures. IEEE Trans. Med. Imaging 42(1), 245–256 (2022)
Guo, L., et al.: Dual attention enhancement feature fusion network for segmentation and quantitative analysis of paediatric echocardiography. Med. Image Anal. 71, 102042 (2021)
Hu, H., Cui, J., Wang, L.: Region-aware contrastive learning for semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16291–16301 (2021)
Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. In: International Conference on Learning Representations (ICLR) (2017)
Lazarow, J., Xu, W., Tu, Z.: Instance segmentation with mask-supervised polygonal boundary transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4382–4391 (2022)
Li, H., Wang, Y., Qu, M., Cao, P., Feng, C., Yang, J.: EchoEFNet: multi-task deep learning network for automatic calculation of left ventricular ejection fraction in 2D echocardiography. Comput. Biol. Med. 156, 106705 (2023)
Li, Y., Luo, L., Lin, H., Chen, H., Heng, P.-A.: Dual-consistency semi-supervised learning with uncertainty quantification for COVID-19 lesion segmentation from CT images. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 199–209. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_19
Luo, X., Chen, J., Song, T., Wang, G.: Semi-supervised medical image segmentation through dual-task consistency. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 8801–8809 (2021)
Luo, X., et al.: Scribble-supervised medical image segmentation via dual-branch network and dynamically mixed pseudo labels supervision. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds.) Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022, Part I, Singapore, 18–22 September 2022, pp. 528–538. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16431-6_50
Luo, X., et al.: Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency. Med. Image Anal. 80, 102517 (2022)
Meng, Y., et al.: Diagnosis of diabetic neuropathy by artificial intelligence using corneal confocal microscopy. Eur. J. Ophthalmol. 32, 11–12 (2022)
Meng, Y., et al.: Bilateral adaptive graph convolutional network on CT based COVID-19 diagnosis with uncertainty-aware consensus-assisted multiple instance learning. Med. Image Anal. 84, 102722 (2023)
Meng, Y., et al.: Transportation object counting with graph-based adaptive auxiliary learning. IEEE Trans. Intell. Transp. Syst. 24(3), 3422–3437 (2022)
Meng, Y., et al.: Shape-aware weakly/semi-supervised optic disc and cup segmentation with regional/marginal consistency. In: Proceedings of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention, MICCAI 2022, Part IV, Singapore, 18–22 September 2022. pp. 524–534. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16440-8_50
Meng, Y., et al.: Regression of instance boundary by aggregated CNN and GCN. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12353, pp. 190–207. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58598-3_12
Meng, Y., et al.: Artificial intelligence based analysis of corneal confocal microscopy images for diagnosing peripheral neuropathy: a binary classification model. J. Clin. Med. 12(4), 1284 (2023)
Meng, Y., et al.: CNN-GCN aggregation enabled boundary regression for biomedical image segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12264, pp. 352–362. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59719-1_35
Meng, Y., et al.: Bi-GCN: boundary-aware input-dependent graph convolution network for biomedical image segmentation. In: 32nd British Machine Vision Conference, BMVC 2021. British Machine Vision Association (2021)
Meng, Y., et al.: Dual consistency enabled weakly and semi-supervised optic disc and cup segmentation with dual adaptive graph convolutional networks. IEEE Trans. Med. Imaging 42, 416–429 (2022)
Meng, Y., et al.: Spatial uncertainty-aware semi-supervised crowd counting. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15549–15559 (2021)
Meng, Y., et al.: Graph-based region and boundary aggregation for biomedical image segmentation. IEEE Trans. Med. Imaging 41, 690–701 (2021)
Mukaka, M.M.: A guide to appropriate use of correlation coefficient in medical research. Malawi Med. J. 24(3), 69–71 (2012)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Ouyang, D., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)
Patefield, A., et al.: Deep learning using preoperative AS-OCT predicts graft detachment in DMEK. Trans. Vis. Sci. Technol. 12(5), 14 (2023)
Preston, F.G., et al.: Artificial intelligence utilising corneal confocal microscopy for the diagnosis of peripheral neuropathy in diabetes mellitus and prediabetes. Diabetologia 65, 457–466 (2022). https://doi.org/10.1007/s00125-021-05617-x
Robinson, J., Chuang, C.Y., Sra, S., Jegelka, S.: Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592 (2020)
Shi, Y., et al.: Inconsistency-aware uncertainty estimation for semi-supervised medical image segmentation. IEEE Trans. Med. Imaging 41(3), 608–620 (2021)
Yu, L., Wang, S., Li, X., Fu, C.-W., Heng, P.-A.: Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 605–613. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_67
Zeng, Y., et al.: MAEF-Net: multi-attention efficient feature fusion network for left ventricular segmentation and quantitative analysis in two-dimensional echocardiography. Ultrasonics 127, 106855 (2023)
Zhang, Y., Meng, Y., Zheng, Y.: Automatically segment the left atrium and scars from LGE-MRIs using a boundary-focused nnU-Net. In: Zhuang, X., Li, L., Wang, S., Wu, F. (eds.) Left Atrial and Scar Quantification and Segmentation, LAScarQS 2022. LNCS, vol. 13586, pp. 49–59. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-31778-1_5
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Meng, Y., Zhang, Y., Xie, J., Duan, J., Zhao, Y., Zheng, Y. (2024). Weakly/Semi-supervised Left Ventricle Segmentation in 2D Echocardiography with Uncertain Region-Aware Contrastive Learning. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14437. Springer, Singapore. https://doi.org/10.1007/978-981-99-8558-6_9
Download citation
DOI: https://doi.org/10.1007/978-981-99-8558-6_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8557-9
Online ISBN: 978-981-99-8558-6
eBook Packages: Computer ScienceComputer Science (R0)