Skip to main content

Joint Semantic Feature and Optical Flow Learning for Automatic Echocardiography Segmentation

  • Conference paper
  • First Online:
Advanced Intelligent Computing Technology and Applications (ICIC 2024)

Abstract

The left ventricle ejection fraction is an important index for assessing cardiac function and diagnosing cardiac diseases. At present, EchoNet-Dynamic dataset is the unique large-scale resource for studying ejection fraction estimation by echocardiography. Through segmentation of the end-systolic and end-diastolic frames, the ejection fraction can be calculated based on the volumes at these phases. However, existing segmentation methods either mostly focus on single-frame segmentation and rarely consider information across consecutive frames, or they fail to effectively exploit temporal information between consecutive frames, resulting in suboptimal segmentation performance. In our study, we constructed a dual-branch spatial-temporal feature extraction model for achieving echocardiogram video segmentation. One branch was dedicated to extracting semantic features of frames under supervision, while the other branch learned the optical flows between frames in an unsupervised manner. Subsequently, we jointly trained these two branches using a temporal consistency mechanism to acquire spatial-temporal features of the frames. This approach enhances both video segmentation performance and the consistency of transition frame segmentation. Experimental results demonstrate that our proposed model achieves promising segmentation performance compared to existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Spencer, K.T., Kimura, B.J., Korcarz, C.E., Pellikka, P.A., Rahko, P.S., Siegel, R.J.: Focused cardiac ultrasound: recommendations from the american society of echocardiography. J. Am. Soc. Echocardiogr. 26(6), 567–581 (2013)

    Article  Google Scholar 

  2. Ali, Y., Janabi-Sharifi, F., Beheshti, S.: Echocardiographic image segmentation using deep res-u network. Biomed. Signal Process. Control 64, 102248 (2021)

    Article  Google Scholar 

  3. Puyol-Antón, E., et al.: Ai-enabled assessment of cardiac systolic and diastolic function from echocardiography. arXiv preprint arXiv:2203.11726 (2022)

  4. Li, M., et al.: Unified model for interpreting multi-view echocardiographic sequences without temporal information. Appl. Soft Comput. 88, 106049 (2020)

    Article  Google Scholar 

  5. Deng, K., Meng, Y., Gao, D., Bridge, J., Shen, Y., Lip, G., Zhao, Y., Zheng, Y.: TransBridge: a lightweight transformer for left ventricle segmentation in echocardiography. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 63–72. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_7

    Chapter  Google Scholar 

  6. Shi, S., Alimu, P., Mahemuti, P., Chen, Q., Wu, H.: The study of echocardiography of left-ventricle segmentation combining transformer and CNN. SSRN 4184447 (2022)

    Google Scholar 

  7. Liu, F., Wang, K., Liu, D., Yang, X., Tian, J.: Deep pyramid local attention neural network for cardiac structure segmentation in two-dimensional echocardiography. Med. Image Anal. 67, 101873 (2021)

    Article  Google Scholar 

  8. Ye, Z., Kumar, Y.J., Song, F., Li, G., Zhang, S.: Bi-DCNet: bilateral network with dilated convolutions for left ventricle segmentation. Life 13(4), 1040 (2023)

    Article  Google Scholar 

  9. Wei, H., Cao, H., Cao, Y., Zhou, Y., Xue, W., Ni, D., Li, S.: Temporal-consistent segmentation of echocardiography with co-learning from appearance and shape. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 623–632. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_60

    Chapter  Google Scholar 

  10. Chen, Y., Zhang, X., Haggerty, C.M., Stough, J.V.: Assessing the generalizability of temporally coherent echocardiography video segmentation. In: Medical Imaging 2021: Image Processing, vol. 11596, pp. 463–469. International Society for Optics and Photonics (2021)

    Google Scholar 

  11. Li, M., Wang, C., Zhang, H., Yang, G.: MV-RAN: multiview recurrent aggregation network for echocardiographic sequences segmentation and full cardiac cycle analysis. Comput. Biol. Med. 120, 103728 (2020)

    Article  Google Scholar 

  12. Sirjani, N., et al.: Automatic cardiac evaluations using a deep video object segmentation network. Insights Imaging 13(1), 1–14 (2022)

    Article  Google Scholar 

  13. Painchaud, N., Duchateau, N., Bernard, O., Jodoin, P.-M.: Echocardiography segmentation with enforced temporal consistency. IEEE Trans. Med. Imaging 41(10), 2867–2878 (2022)

    Article  Google Scholar 

  14. Wu, H., Liu, J., Xiao, F., Wen, Z., Cheng, L., Qin, J.: Semi-supervised segmentation of echocardiography videos via noise-resilient spatiotemporal semantic calibration and fusion. Med. Image Anal. 78, 102397 (2022)

    Article  Google Scholar 

  15. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  16. Dosovitskiy, A., et al.: FlowNet: learning optical flow with convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2758–2766 (2015)

    Google Scholar 

  17. Godard, C., Mac Aodha, O., Brostow, G.J.: Unsupervised monocular depth estimation with left-right consistency. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 270–279 (2017)

    Google Scholar 

  18. Yin, Z., Shi, J.: GeoNet: unsupervised learning of dense depth, optical flow and camera pose. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1983–1992 (2018)

    Google Scholar 

  19. Ding, M., Wang, Z., Zhou, B., Shi, J., Lu, Z., Luo, P.: Every frame counts: joint learning of video segmentation and optical flow. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 10713–10720 (2020)

    Google Scholar 

  20. Ouyang, D., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)

    Article  Google Scholar 

  21. Ta, K., Ahn, S.S., Stendahl, J.C., Sinusas, A.J., Duncan, J.S.: A semi-supervised joint network for simultaneous left ventricular motion tracking and segmentation in 4D echocardiography. In: Martel, A.L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga, M.A., Zhou, S.K., Racoceanu, D., Joskowicz, L. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 468–477. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_45

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yu Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lyu, J., Meng, J., Zhang, Y., Ling, S.H., Sun, L. (2024). Joint Semantic Feature and Optical Flow Learning for Automatic Echocardiography Segmentation. In: Huang, DS., Zhang, C., Zhang, Q. (eds) Advanced Intelligent Computing Technology and Applications. ICIC 2024. Lecture Notes in Computer Science, vol 14868. Springer, Singapore. https://doi.org/10.1007/978-981-97-5600-1_14

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5600-1_14

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5599-8

  • Online ISBN: 978-981-97-5600-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics