Abstract
Echocardiogram video plays a crucial role in analysing cardiac function and diagnosing cardiac diseases. Current deep neural network methods primarily aim to enhance diagnosis accuracy by incorporating prior knowledge, such as segmenting cardiac structures or lesions annotated by human experts. However, diagnosing the inconsistent behaviours of the heart, which exist across both spatial and temporal dimensions, remains extremely challenging. For instance, the analysis of cardiac motion acquires both spatial and temporal information from the heartbeat cycle. To address this issue, we propose a novel reconstruction-based approach named CardiacNet to learn a better representation of local cardiac structures and motion abnormalities through echocardiogram videos. CardiacNet accompanied by the Consistency Deformation Codebook (CDC) and the Consistency Deformed-Discriminator (CDD) to learn the commonalities across abnormal and normal samples by incorporating cardiac prior knowledge. In addition, we propose benchmark datasets named CardiacNet-PAH and CardiacNet-ASD for evaluating the effectiveness of cardiac disease assessment. In experiments, our CardiacNet can achieve state-of-the-art results in three different cardiac disease assessment tasks on public datasets CAMUS, EchoNet, and our datasets. The code and dataset are available at: https://github.com/xmed-lab/CardiacNet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
One step TFlops of denoising. A total of 1000 steps are used in inference.
References
Cuturi, M.: Sinkhorn distances: lightspeed computation of optimal transport. NIPS 26 (2013)
Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: CVPR, pp. 12873–12883 (2021)
Ganame, J., Mertens, L., et al.: Regional myocardial deformation in children with hypertrophic cardiomyopathy: morphological and clinical correlations. Eur. Heart J. 28(23), 2886–2894 (2007)
Geske, J.B., Bos, J.M., Gersh, B.J., Ommen, S.R., Eidem, B.W., Ackerman, M.J.: Deformation patterns in genotyped patients with hypertrophic cardiomyopathy. Eur. Heart J. Cardiovascu. Imaging 15(4), 456–465 (2014)
Ghorbani, A., et al.: Deep learning interpretation of echocardiograms. NPJ digit. med. 3(1), 10 (2020)
Gong, D., et al.: Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection. In: ICCV (2019)
Hara, K., Kataoka, H., Satoh, Y.: Can spatiotemporal 3D CNNs retrace the history of 2D cnns and imagenet? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6546–6555 (2018)
Hörmander, F., Totaro, N., Waldschmidt, A.V.M.: Grundlehren der mathematischen wissenschaften 332, vol. 5. Springer (2006)
Huo, X., et al.: HiFuse: hierarchical multi-scale feature fusion network for medical image classification. Biomed. Signal Process. Control 87, 105534 (2024)
Kamran, S.A., Hossain, K.F., Tavakkoli, A., Zuckerbrod, S.L., Baker, S.A.: VTGAN: semi-supervised retinal image synthesis and disease prediction using vision transformers. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pp. 3228–3238 (2021). https://doi.org/10.1109/ICCVW54120.2021.00362
Kascenas, A., Pugeault, N., O’Neil, A.Q.: Denoising autoencoders for unsupervised anomaly detection in brain MRI. In: International Conference on Medical Imaging with Deep Learning, pp. 653–664. PMLR (2022)
Lai, W.W., Mertens, L.L., Cohen, M.S., Geva, T.: Echocardiography in Pediatric and Congenital Heart Disease: From Fetus to Adult. John Wiley & Sons (2015)
Leclerc, S., et al.: Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans. Med. Imaging 38(9), 2198–2210 (2019)
Lin, X., et al.: Echocardiography-based AI detection of regional wall motion abnormalities and quantification of cardiac function in myocardial infarction. Front. Cardiovasc. Med. 9, 903660 (2022)
Lin, Y., Luo, Z., Zhao, W., Li, X.: Learning deep intensity field for extremely sparse-view CBCT reconstruction. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023, pp. 13–23. Springer Nature Switzerland (2023). https://doi.org/10.1007/978-3-031-43999-5_2
Lin, Y., Wang, H., Chen, J., Li, X.: Learning 3D gaussians for extremely sparse-view cone-beam CT reconstruction (2024). https://arxiv.org/abs/2407.01090
Lin, Y., Yang, J., Wang, H., Ding, X., Zhao, W., Li, X.: C⌃2rv: cross-regional and cross-view learning for sparse-view CBCT reconstruction. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11205–11214 (June 2024)
Liu, B., et al.: A deep learning framework assisted echocardiography with diagnosis, lesion localization, phenogrouping heterogeneous disease, and anomaly detection. Sci. Rep. 13(1), 3 (2023)
Lu, Y., Li, K., Pu, B., Tan, Y., Zhu, N.: A YOLOX-based deep instance segmentation neural network for cardiac anatomical structures in fetal ultrasound images. IEEE/ACM Trans. Comput. Biol. Bioinform. (2022)
Mallya, M., Hamarneh, G.: Deep multimodal guidance for medical image classification. In: MICCAI. Springer (2022). https://doi.org/10.1007/978-3-031-16449-1_29
McDonagh, T.A., et al.: 2021 esc guidelines for the diagnosis and treatment of acute and chronic heart failure: Developed by the task force for the diagnosis and treatment of acute and chronic heart failure of the european society of cardiology (esc) with the special contribution of the heart failure association (hfa) of the esc. Eur. Heart J. 42(36), 3599–3726 (2021)
Mcleod, G., et al.: Echocardiography in congenital heart disease. Prog. Cardiovasc. Dis. 61(5–6), 468–475 (2018)
Meena, T., Kabiraj, A., Reddy, P.B., Roy, S.: Weakly supervised confidence aware probabilistic cam multi-thorax anomaly localization network. In: 2023 IEEE 24th International Conference on Information Reuse and Integration for Data Science (IRI), pp. 309–314. IEEE (2023)
Niemann, M., et al.: Echocardiographic quantification of regional deformation helps to distinguish isolated left ventricular non-compaction from dilated cardiomyopathy. Eur. J. Heart Fail. 14(2), 155–161 (2012)
Ouyang, D., et al.: Video-based AI for beat-to-beat assessment of cardiac function. Nature 580(7802), 252–256 (2020)
Oxborough, D., et al.: The right ventricle of the endurance athlete: the relationship between morphology and deformation. J. Am. Soc. Echocardiogr. 25(3), 263–271 (2012)
Popp, R.L.: Echocardiographic assessment of cardiac disease. Circulation 54(4), 538–552 (1976)
Pu, B., et al.: HFSCCD: a hybrid neural network for fetal standard cardiac cycle detection in ultrasound videos. IEEE J. Biomed. Health Inform. (2024)
Pu, B., et al.: MobileUNet-FPN: a semantic segmentation model for fetal ultrasound four-chamber segmentation in edge computing environments. IEEE J. Biomed. Health Inform. 26(11), 5540–5550 (2022)
Pu, B., et al.: Unsupervised domain adaptation for anatomical structure detection in ultrasound images. In: Forty-first International Conference on Machine Learning
Pu, B., et al.: M3-UDA: a new benchmark for unsupervised domain adaptive fetal cardiac structure detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11621–11630 (2024)
Pu, B., Zhu, N., Li, K., Li, S.: Fetal cardiac cycle detection in multi-resource echocardiograms using hybrid classification framework. Futur. Gener. Comput. Syst. 115, 825–836 (2021)
Ristea, N.C., et al.: CyTran: cycle-consistent transformers for non-contrast to contrast CT translation. Neurocomputing (2023). https://doi.org/10.1016/j.neucom.2023.03.072
Ryser, A., Manduchi, L., Laumer, F., Michel, H., Wellmann, S., Vogt, J.E.: Anomaly detection in echocardiograms with dynamic variational trajectory models. In: Machine Learning for Healthcare Conference, pp. 425–458. PMLR (2022)
Sanchez, P., Kascenas, A., Liu, X., O’Neil, A.Q., Tsaftaris, S.A.: What is healthy? generative counterfactual diffusion for lesion localization. In: MICCAI Workshop on Deep Generative Models, pp. 34–44. Springer (2022). https://doi.org/10.1007/978-3-031-18576-2_4
Sanjeevi, G., Gopalakrishnan, U., Pathinarupothi, R.K., Madathil, T.: Automatic diagnostic tool for detection of regional wall motion abnormality from echocardiogram. J. Med. Syst. 47(1), 13 (2023)
Schäfer, M., et al.: Myocardial strain-curve deformation patterns after fontan operation. Sci. Rep. 13(1), 11912 (2023)
Schlegl, T., Seeböck, P., Waldstein, S.M., Langs, G., Schmidt-Erfurth, U.: f-AnoGAN: fast unsupervised anomaly detection with generative adversarial networks. Med. Image Anal. 54, 30–44 (2019)
Schlemper, J., et al.: Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019)
Silva-Rodríguez, J., Naranjo, V., Dolz, J.: Constrained unsupervised anomaly segmentation. Med. Image Anal. 80, 102526 (2022)
Sun, D., et al.: Chamber attention network (CAN): towards interpretable diagnosis of pulmonary artery hypertension using echocardiography. J. Adv. Res (2023)
Tseng, C.H., Chien, S.J., Wang, P.S., Lee, S.J., Pu, B., Zeng, X.J.: Real-time automatic m-mode echocardiography measurement with panel attention. IEEE J. Biomed. Health Inform. (2024)
Upton, M., Gibson, D., Brown, D.: Echocardiographic assessment of abnormal left ventricular relaxation in man. Heart 38(10), 1001–1009 (1976)
Van Den Oord, A., Vinyals, O., et al.: Neural discrete representation learning. NIPS 30 (2017)
Wolleb, J., Bieder, F., Sandkühler, R., Cattin, P.C.: Diffusion models for medical anomaly detection. In: MICCAI, pp. 35–45. Springer (2022). https://doi.org/10.1007/978-3-031-16452-1_4
Wolleb, J., Sandkühler, R., Cattin, P.C.: DescarGAN: disease-specific anomaly detection with weak supervision. In: MICCAI, pp. 14–24. Springer (2020). https://doi.org/10.1007/978-3-030-59719-1_2
Yang, J., Ding, X., Zheng, Z., Xu, X., Li, X.: GraphECHO: graph-driven unsupervised domain adaptation for echocardiogram video segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11878–11887 (2023)
Yu, K., Ghosh, S., Liu, Z., Deible, C., Batmanghelich, K.: Anatomy-guided weakly-supervised abnormality localization in chest x-rays. In: MICCAI, pp. 658–668. Springer (2022). https://doi.org/10.1007/978-3-031-16443-9_63
Zaman, F., et al.: Spatio-temporal hybrid neural networks reduce erroneous human “judgement calls” in the diagnosis of takotsubo syndrome. EClinicalMedicine 40 (2021)
Zhang, J., et al.: Viral pneumonia screening on chest x-rays using confidence-aware anomaly detection. IEEE Trans. Med. Imaging 40(3), 879–890 (2020)
Zheng, Z., Yang, J., Ding, X., Xu, X., Li, X.: GL-fusion: Global-local fusion network for multi-view echocardiogram video segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 78–88. Springer (2023). https://doi.org/10.1007/978-3-031-43901-8_8
Zimmerer, D., Kohl, S.A., Petersen, J., Isensee, F., Maier-Hein, K.H.: Context-encoding variational autoencoder for unsupervised anomaly detection. arXiv preprint arXiv:1812.05941 (2018)
Acknowledgement
This work was supported by a research grant from the Beijing Institute of Collaborative Innovation (BICI) under collaboration with the Hong Kong University of Science and Technology under Grant HCIC-004 and a project of Hetao Shenzhen-Hong Kong Science and Technology Innovation Cooperation Zone (HZQB-KCZYB-2020083).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, J., Lin, Y., Pu, B., Guo, J., Xu, X., Li, X. (2025). CardiacNet: Learning to Reconstruct Abnormalities for Cardiac Disease Assessment from Echocardiogram Videos. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15081. Springer, Cham. https://doi.org/10.1007/978-3-031-73337-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-73337-6_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-73336-9
Online ISBN: 978-3-031-73337-6
eBook Packages: Computer ScienceComputer Science (R0)