Abstract
High-quality virtual populations of human hearts are of significant importance for a variety of applications, such as in silico simulations of cardiac physiology, data augmentation, and medical device development. However, their creation is a challenging endeavor since the synthesized hearts not only need to exhibit plausible shapes on an individual level but also accurately capture the considerable variability across the true underlying population. In this work, we present Snowflake-Net as a novel approach to automatically generate arbitrarily-sized and realistic populations of 3D heart anatomies in the form of high-resolution point clouds. Our proposed method combines transformer components with point cloud-based deep learning to effectively and directly process 3D heart anatomies reconstructed from cine magnetic resonance images. We develop our approach on a large UK Biobank dataset of about 1000 subjects. We find that the Snowflake-Net achieves average reconstruction errors of 0.90 mm in terms of mean Chamfer Distances, which is considerably below the pixel resolution of the underlying MRI acquisition, and outperforms a prior state-of-the-art approach by \(\sim \)20%. Furthermore, we show the Snowflake-Net’s ability to create new 3D cardiac anatomies with a high degree of realism on both an individual and population level and observe the generated virtual and the true underlying populations to be highly similar in terms of multiple generation quality metrics. Finally, we investigate how the captured 3D shape variability is encoded in the low-dimensional latent space and its effect on model interpretability.
J. Peng and M. Beetz—Authors contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bai, W., et al.: A bi-ventricular cardiac atlas built from 1000+ high resolution MR images of healthy subjects and an analysis of shape and motion. Med. Image Anal. 26(1), 133–145 (2015)
Banerjee, A., et al.: A completely automated pipeline for 3D reconstruction of human heart from 2D cine magnetic resonance slices. Philos. Trans. A Math. Phys. Eng. Sci. 379(2212), 20200257 (2021)
Beetz, M., Banerjee, A., Grau, V.: Biventricular surface reconstruction from cine MRI contours using point completion networks. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 105–109. IEEE (2021)
Beetz, M., Banerjee, A., Grau, V.: Generating subpopulation-specific biventricular anatomy models using conditional point cloud variational autoencoders. In: Puyol Antón, E., et al. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 75–83. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93722-5_9
Beetz, M., Banerjee, A., Grau, V.: Multi-domain variational autoencoders for combined modeling of MRI-based biventricular anatomy and ECG-based cardiac electrophysiology. Front. Physiol. 13, 991 (2022)
Beetz, M., Banerjee, A., Grau, V.: Point2Mesh-net: combining point cloud and mesh-based deep learning for cardiac shape reconstruction. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers. STACOM 2022. Lecture Notes in Computer Science, vol. 13593, pp. 280–290. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-23443-9_26
Beetz, M., Banerjee, A., Grau, V.: Modeling 3D cardiac contraction and relaxation with point cloud deformation networks. arXiv preprint arXiv:2307.10927 (2023)
Beetz, M., Banerjee, A., Grau, V.: Multi-objective point cloud autoencoders for explainable myocardial infarction prediction. arXiv preprint arXiv:2307.11017 (2023)
Beetz, M., Ossenberg-Engels, J., Banerjee, A., Grau, V.: Predicting 3D cardiac deformations with point cloud autoencoders. In: Puyol Antón, E., et al. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 219–228. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93722-5_24
Beetz, M., et al.: Combined generation of electrocardiogram and cardiac anatomy models using multi-modal variational autoencoders. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–4 (2022)
Beetz, M., et al.: Interpretable cardiac anatomy modeling using variational mesh autoencoders. Front. Cardiovasc. Med. 9, 3258 (2022)
Beetz, M., et al.: 3D shape-based myocardial infarction prediction using point cloud classification networks. arXiv preprint arXiv:2307.07298 (2023)
Beetz, M., et al.: Mesh U-Nets for 3D cardiac deformation modeling. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers. STACOM 2022. Lecture Notes in Computer Science, vol. 13593, pp. 245–257. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-23443-9_23
Beetz, M., et al.: Multi-class point cloud completion networks for 3D cardiac anatomy reconstruction from cine magnetic resonance images. arXiv preprint arXiv:2307.08535 (2023)
Bertrand, A., et al.: Deep learning-based emulation of human cardiac activation sequences. In: Bernard, O., Clarysse, P., Duchateau, N., Ohayon, J., Viallon, M. (eds.) Functional Imaging and Modeling of the Heart. FIMH 2023. Lecture Notes in Computer Science, vol. 13958, pp. 213–222. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-35302-4_22
Biffi, C., et al.: Explainable anatomical shape analysis through deep hierarchical generative models. IEEE Trans. Med. Imaging 39(6), 2088–2099 (2020)
Chang, Y., Jung, C.: Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds. Neurocomputing 418, 270–279 (2020)
Corral-Acero, J., et al.: The ‘digital twin’ to enable the vision of precision cardiology. Eur. Heart J. 41(48), 4556–4564 (2020)
Dou, H., Ravikumar, N., Frangi, A.F.: A conditional flow variational autoencoder for controllable synthesis of virtual populations of anatomy. arXiv preprint arXiv:2306.14680 (2023)
Dou, H., et al.: A generative shape compositional framework: towards representative populations of virtual heart chimaeras. arXiv preprint arXiv:2210.01607 (2022)
Gilbert, K., et al.: Artificial intelligence in cardiac imaging with statistical atlases of cardiac anatomy. Front. Cardiovasc. Med. 7, 102 (2020)
Gooya, A., Davatzikos, C., Frangi, A.F.: A bayesian approach to sparse model selection in statistical shape models. SIAM J. Imag. Sci. 8(2), 858–887 (2015)
Li, L., et al.: Deep computational model for the inference of ventricular activation properties. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers. STACOM 2022. Lecture Notes in Computer Science, vol. 13593, pp. 369–380. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-23443-9_34
Petersen, S.E., et al.: UK Biobank’s cardiovascular magnetic resonance protocol. J. Cardiovasc. Magn. Reson. 18(1), 1–7 (2015)
Piazzese, C., et al.: Statistical shape models of the heart: applications to cardiac imaging. In: Statistical Shape and Deformation Analysis, pp. 445–480. Elsevier (2017)
Qi, C.R., et al.: PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 652–660 (2017)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Viceconti, M., et al.: In silico trials: verification, validation and uncertainty quantification of predictive models used in the regulatory evaluation of biomedical products. Methods 185, 120–127 (2021)
Xiang, P., et al.: Snowflake point deconvolution for point cloud completion and generation with skip-transformer. IEEE Trans. Pattern Anal. Mach. Intell. 45(5), 6320–6338 (2022)
Xiong, Z., et al.: Automatic 3D surface reconstruction of the left atrium from clinically mapped point clouds using convolutional neural networks. Front. Physiol. 13, 880260–880260 (2022)
Yang, G., et al.: PointFlow: 3D point cloud generation with continuous normalizing flows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4541–4550 (2019)
Ye, M., et al.: PC-U Net: learning to jointly reconstruct and segment the cardiac walls in 3D from CT data. In: Puyol Anton, E., et al. (eds.) STACOM 2020. LNCS, vol. 12592, pp. 117–126. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68107-4_12
Zakeri, A., et al.: A probabilistic deep motion model for unsupervised cardiac shape anomaly assessment. Med. Image Anal. 75, 102276 (2022)
Acknowledgments
This research has been conducted using the UK Biobank Resource under Application Number ‘40161’. The authors express no conflict of interest. The work of M. Beetz was supported by the Stiftung der Deutschen Wirtschaft (Foundation of German Business). A. Banerjee is a Royal Society University Research Fellow and is supported by the Royal Society Grant No. URF\(\backslash \)R1\(\backslash \)221314. The work of A. Banerjee was partially supported by the British Heart Foundation (BHF) Project under Grant PG/20/21/35082. The work of V. Grau was supported by the CompBioMed 2 Centre of Excellence in Computational Biomedicine (European Commission Horizon 2020 research and innovation programme, grant agreement No. 823712).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Peng, J., Beetz, M., Banerjee, A., Chen, M., Grau, V. (2024). Generating Virtual Populations of 3D Cardiac Anatomies with Snowflake-Net. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxRecon Challenge Papers. STACOM 2023. Lecture Notes in Computer Science, vol 14507. Springer, Cham. https://doi.org/10.1007/978-3-031-52448-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-52448-6_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-52447-9
Online ISBN: 978-3-031-52448-6
eBook Packages: Computer ScienceComputer Science (R0)