Abstract
Echocardiography is the only technique capable of real-time imaging of the heart and is vital for diagnosing the majority of cardiac diseases. However, there is a severe shortage of experienced cardiac sonographers, due to the heart’s complex structure and significant operational challenges. To mitigate this situation, we present a Cardiac Copilot system capable of providing real-time probe movement guidance to assist less experienced sonographers in conducting freehand echocardiography. This system can enable non-experts, especially in primary departments and medically underserved areas, to perform cardiac ultrasound examinations, potentially improving global healthcare delivery. The core innovation lies in proposing a data-driven world model, named Cardiac Dreamer, for representing cardiac spatial structures. This world model can provide structure features of any cardiac planes around the current probe position in the latent space, serving as an precise navigation map for autonomous plane localization. We train our model with real-world ultrasound data and corresponding probe motion from 110 routine clinical scans with 151K sample pairs by three certified sonographers. Evaluations on three standard planes with 37K sample pairs demonstrate that the world model can reduce navigation errors by up to 33% and exhibit more stable performance.
H. Jiang and Z. Sun—These authors contributed equally to this work. This work was done while Haojun Jiang was an intern at Beijing Academy of Artificial Intelligence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Droste, R., Drukker, L., Papageorghiou, A.T., Noble, J.A.: Automatic probe movement guidance for freehand obstetric ultrasound. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12263, pp. 583–592. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59716-0_56
Duan, Y., et al.: One-shot imitation learning. Adv. Neural Inf. Process. Syst. 30 (2017)
Ehler, D., et al.: Guidelines for cardiac sonographer education: recommendations of the American society of echocardiography sonographer training and education committee. J. Am. Soc. Echocardiogr. 14(1), 77–84 (2001)
Gardner, C.J., et al.: Guidelines for cardiac sonographer education: report of the American society of echocardiography sonographer education and training committee. J. Am. Soc. Echocardiogr. 5(6), 635–639 (1992)
Ha, D., Schmidhuber, J.: World models. arXiv preprint arXiv:1803.10122 (2018)
Hafner, D., Lillicrap, T., Ba, J., Norouzi, M.: Dream to control: learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603 (2019)
Hafner, D., et al.: Learning latent dynamics for planning from pixels. In: International Conference on Machine Learning, pp. 2555–2565. PMLR (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ho, J., Ermon, S.: Generative adversarial imitation learning. Adv. Neural Inf. Process. Syst. 29 (2016)
Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. (CSUR) 50(2), 1–35 (2017)
Jiang, H., Lin, Y., Han, D., Song, S., Huang, G.: Pseudo-q: generating pseudo language queries for visual grounding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15513–15523 (2022)
Jiang, H., et al.: Cross-modal adapter for text-video retrieval. arXiv preprint arXiv:2211.09623 (2022)
Narang, A., et al.: Utility of a deep-learning algorithm to guide novices to acquire echocardiograms for limited diagnostic use. JAMA Cardiol. 6(6), 624–632 (2021)
Roth, G.A., et al.: Global, regional, and national burden of cardiovascular diseases for 10 causes, 1990 to 2015. J. Am. Coll. Cardiol. 70(1), 1–25 (2017)
Shida, Y., Kumagai, S., Tsumura, R., Iwata, H.: Automated image acquisition of parasternal long-axis view with robotic echocardiography. IEEE Rob. Autom. Lett. (2023)
Shida, Y., Sugawara, M., Tsumura, R., Chiba, H., Uejima, T., Iwata, H.: Diagnostic posture control system for seated-style echocardiography robot. Int. J. Comput. Assist. Radiol. Surg. 18(5), 887–897 (2023)
Song, P., et al.: Global and regional prevalence, burden, and risk factors for carotid atherosclerosis: a systematic review, meta-analysis, and modelling study. Lancet Glob. Health 8(5), e721–e729 (2020)
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 6000–6010. Curran Associates Inc. (2017)
Yang, L., et al.: Condensenet v2: sparse feature reactivation for deep networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3569–3578 (2021)
Acknowledgement
This work was supported in part by the National Key R&D Program of China (2021ZD0140407), the NSFC (62321005) and the Deng Feng Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Conflicts of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Jiang, H. et al. (2024). Cardiac Copilot: Automatic Probe Guidance for Echocardiography with World Model. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15001. Springer, Cham. https://doi.org/10.1007/978-3-031-72378-0_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-72378-0_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-72377-3
Online ISBN: 978-3-031-72378-0
eBook Packages: Computer ScienceComputer Science (R0)
