Abstract
The left ventricle ejection fraction is an important index for assessing cardiac function and diagnosing cardiac diseases. At present, EchoNet-Dynamic dataset is the unique large-scale resource for studying ejection fraction estimation by echocardiography. Through segmentation of the end-systolic and end-diastolic frames, the ejection fraction can be calculated based on the volumes at these phases. However, existing segmentation methods either mostly focus on single-frame segmentation and rarely consider information across consecutive frames, or they fail to effectively exploit temporal information between consecutive frames, resulting in suboptimal segmentation performance. In our study, we constructed a dual-branch spatial-temporal feature extraction model for achieving echocardiogram video segmentation. One branch was dedicated to extracting semantic features of frames under supervision, while the other branch learned the optical flows between frames in an unsupervised manner. Subsequently, we jointly trained these two branches using a temporal consistency mechanism to acquire spatial-temporal features of the frames. This approach enhances both video segmentation performance and the consistency of transition frame segmentation. Experimental results demonstrate that our proposed model achieves promising segmentation performance compared to existing methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Spencer, K.T., Kimura, B.J., Korcarz, C.E., Pellikka, P.A., Rahko, P.S., Siegel, R.J.: Focused cardiac ultrasound: recommendations from the american society of echocardiography. J. Am. Soc. Echocardiogr. 26(6), 567–581 (2013)
Ali, Y., Janabi-Sharifi, F., Beheshti, S.: Echocardiographic image segmentation using deep res-u network. Biomed. Signal Process. Control 64, 102248 (2021)
Puyol-Antón, E., et al.: Ai-enabled assessment of cardiac systolic and diastolic function from echocardiography. arXiv preprint arXiv:2203.11726 (2022)
Li, M., et al.: Unified model for interpreting multi-view echocardiographic sequences without temporal information. Appl. Soft Comput. 88, 106049 (2020)
Deng, K., Meng, Y., Gao, D., Bridge, J., Shen, Y., Lip, G., Zhao, Y., Zheng, Y.: TransBridge: a lightweight transformer for left ventricle segmentation in echocardiography. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 63–72. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_7
Shi, S., Alimu, P., Mahemuti, P., Chen, Q., Wu, H.: The study of echocardiography of left-ventricle segmentation combining transformer and CNN. SSRN 4184447 (2022)
Liu, F., Wang, K., Liu, D., Yang, X., Tian, J.: Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography. Med. Image Anal. 67, 101873 (2021)
Ye, Z., Kumar, Y.J., Song, F., Li, G., Zhang, S.: Bi-DCNet: bilateral network with dilated convolutions for left ventricle segmentation. Life 13(4), 1040 (2023)
Wei, H., Cao, H., Cao, Y., Zhou, Y., Xue, W., Ni, D., Li, S.: Temporal-consistent segmentation of echocardiography with co-learning from appearance and shape. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 623–632. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_60
Chen, Y., Zhang, X., Haggerty, C.M., Stough, J.V.: Assessing the generalizability of temporally coherent echocardiography video segmentation. In: Medical Imaging 2021: Image Processing, vol. 11596, pp. 463–469. International Society for Optics and Photonics (2021)
Li, M., Wang, C., Zhang, H., Yang, G.: MV-RAN: multiview recurrent aggregation network for echocardiographic sequences segmentation and full cardiac cycle analysis. Comput. Biol. Med. 120, 103728 (2020)
Sirjani, N., et al.: Automatic cardiac evaluations using a deep video object segmentation network. Insights Imaging 13(1), 1–14 (2022)
Painchaud, N., Duchateau, N., Bernard, O., Jodoin, P.-M.: Echocardiography segmentation with enforced temporal consistency. IEEE Trans. Med. Imaging 41(10), 2867–2878 (2022)
Wu, H., Liu, J., Xiao, F., Wen, Z., Cheng, L., Qin, J.: Semi-supervised segmentation of echocardiography videos via noise-resilient spatiotemporal semantic calibration and fusion. Med. Image Anal. 78, 102397 (2022)
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)
Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 270–279 (2017)
Yin, Z., Shi, J.: GeoNet: unsupervised learning of dense depth, optical flow and camera pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1983–1992 (2018)
Ding, M., Wang, Z., Zhou, B., Shi, J., Lu, Z., Luo, P.: Every frame counts: joint learning of video segmentation and optical flow. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10713–10720 (2020)
Ouyang, D., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)
Ta, K., Ahn, S.S., Stendahl, J.C., Sinusas, A.J., Duncan, J.S.: A semi-supervised joint network for simultaneous left ventricular motion tracking and segmentation in 4D echocardiography. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 468–477. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_45
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lyu, J., Meng, J., Zhang, Y., Ling, S.H., Sun, L. (2024). Joint Semantic Feature and Optical Flow Learning for Automatic Echocardiography Segmentation. In: Huang, DS., Zhang, C., Zhang, Q. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14868. Springer, Singapore. https://doi.org/10.1007/978-981-97-5600-1_14
Download citation
DOI: https://doi.org/10.1007/978-981-97-5600-1_14
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-5599-8
Online ISBN: 978-981-97-5600-1
eBook Packages: Computer ScienceComputer Science (R0)