Abstract
Automated segmentation of anatomical structures in fetal ultrasound video is challenging due to the highly diverse appearance of anatomies and image quality. In this paper, we propose an ultrasound video anatomy segmentation approach to iteratively memorise and segment incoming video frames, which is suitable for online segmentation. This is achieved by a spatio-temporal model that utilizes an adaptive memory bank to store the segmentation history of preceding frames to assist the current frame segmentation. The memory is updated adaptively using a skip gate mechanism based on segmentation confidence, preserving only high-confidence predictions for future use. We evaluate our approach and related state-of-the-art methods on a clinical dataset. The experimental results demonstrate that our method achieves superior performance with an F1 score of 84.83%. Visually, the use of adaptive temporal memory also aids in reducing error accumulation during video segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Caelles, S., Maninis, K.K., Pont-Tuset, J., Leal-Taixé, L., Cremers, D., Van Gool, L.: One-shot video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 221–230 (2017)
Campos, V., Jou, B., Giró-i Nieto, X., Torres, J., Chang, S.F.: Skip RNN: learning to skip state updates in recurrent neural networks. In: International Conference on Learning Representations (2018)
Chang, R., Wang, D., Guo, H., Ding, J., Wang, L.: Weakly-supervised ultrasound video segmentation with minimal annotations. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12908, pp. 648–658. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87237-3_62
Chen, J., et al.: Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306 (2021)
Gleed, A.D., et al.: Automatic image guidance for assessment of placenta location in ultrasound video sweeps. Ultrasound Med. Biol. 49(1), 106–121 (2023)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, Q., Huang, Y., Luo, Y., Yuan, F., Li, X.: Segmentation of breast ultrasound image with semantic classification of superpixels. Med. Image Anal. 61, 101657 (2020)
Huang, Z., Huang, L., Gong, Y., Huang, C., Wang, X.: Mask scoring R-CNN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6409–6418 (2019)
Jauniaux, E., Collins, S., Burton, G.J.: Placenta accreta spectrum: pathophysiology and evidence-based anatomy for prenatal ultrasound imaging. Am. J. Obstet. Gynecol. 218(1), 75–87 (2018)
Oh, S.W., Lee, J.Y., Xu, N., Kim, S.J.: Video object segmentation using space-time memory networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9226–9235 (2019)
Oppenheimer, L., et al.: Diagnosis and management of placenta previa. J. Obstet. Gynaecol. Can. 29(3), 261–266 (2007)
Qi, H., Collins, S., Noble, A.: Weakly supervised learning of placental ultrasound images with residual networks. In: Valdés Hernández, M., González-Castro, V. (eds.) MIUA 2017. CCIS, vol. 723, pp. 98–108. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-60964-5_9
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schilpzand, M., et al.: Automatic placenta localization from ultrasound imaging in a resource-limited setting using a predefined ultrasound acquisition protocol and deep learning. Ultrasound Med. Biol. 48(4), 663–674 (2022)
Self, A., Gleed, A., Bhatnagar, S., Noble, A., Papageorghiou, A.: Vp18. 01: machine learning applied to the standardised six-step approach for placental localisation in basic obstetric ultrasound. Ultrasound Obstetr. Gynecol. 58, 172–172 (2021)
Self, A., et al.: Developing clinical artificial intelligence for obstetric ultrasound to improve access in underserved regions: protocol for a computer-assisted low-cost point-of-care ultrasound (calopus) study. JMIR Res. Protocols 11(9), e37374 (2022)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Yang, X., et al.: Towards automated semantic segmentation in prenatal volumetric ultrasound. IEEE Trans. Med. Imaging 38(1), 180–193 (2018)
Zhou, T., Li, L., Bredell, G., Li, J., Unkelbach, J., Konukoglu, E.: Volumetric memory network for interactive medical image segmentation. Med. Image Anal. 83, 102599 (2023)
Zimmer, V.A., et al.: A multi-task approach using positional information for ultrasound placenta segmentation. In: Hu, Y., et al. (eds.) ASMUS/PIPPI -2020. LNCS, vol. 12437, pp. 264–273. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60334-2_26
Zimmer, V.A., et al.: Towards whole placenta segmentation at late gestation using multi-view ultrasound images. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11768, pp. 628–636. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32254-0_70
Acknowledgments
We acknowledge the ERC (ERC-ADG-2015 694581, project PULSE), the Global Challenges Research Fund (EP/R013853/1, project CALOPUS), EPSRC Programme Grant (EP/T028572/1, project VisualAI), and the InnoHK-funded Hong Kong Centre for Cerebro-cardiovascular Health Engineering (COCHE) Project 2.1 (Cardiovascular risks in early life and fetal echocardiography).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, H., Men, Q., Gleed, A., Papageorghiou, A.T., Noble, J.A. (2023). Ultrasound Video Segmentation with Adaptive Temporal Memory. In: Kainz, B., Noble, A., Schnabel, J., Khanal, B., Müller, J.P., Day, T. (eds) Simplifying Medical Ultrasound. ASMUS 2023. Lecture Notes in Computer Science, vol 14337. Springer, Cham. https://doi.org/10.1007/978-3-031-44521-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-44521-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44520-0
Online ISBN: 978-3-031-44521-7
eBook Packages: Computer ScienceComputer Science (R0)
