Abstract
Myocardial infarction (MI) is one of the most common causes of death in the world. Image-based biomarkers commonly used in the clinic, such as ejection fraction, fail to capture more complex patterns in the heart’s 3D anatomy and thus limit diagnostic accuracy. In this work, we present the multi-objective point cloud autoencoder as a novel geometric deep learning approach for explainable infarction prediction, based on multi-class 3D point cloud representations of cardiac anatomy and function. Its architecture consists of multiple task-specific branches connected by a low-dimensional latent space to allow for effective multi-objective learning of both reconstruction and MI prediction, while capturing pathology-specific 3D shape information in an interpretable latent space. Furthermore, its hierarchical branch design with point cloud-based deep learning operations enables efficient multi-scale feature learning directly on high-resolution anatomy point clouds. In our experiments on a large UK Biobank dataset, the multi-objective point cloud autoencoder is able to accurately reconstruct multi-temporal 3D shapes with Chamfer distances between predicted and input anatomies below the underlying images’ pixel resolution. Our method outperforms multiple machine learning and deep learning benchmarks for the task of incident MI prediction by 19% in terms of Area Under the Receiver Operating Characteristic curve. In addition, its task-specific compact latent space exhibits easily separable control and MI clusters with clinically plausible associations between subject encodings and corresponding 3D shapes, thus demonstrating the explainability of the prediction.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Avard, E., et al.: Non-contrast cine cardiac magnetic resonance image radiomics features and machine learning algorithms for myocardial infarction detection. Comput. Biol. Med. 141, 105145 (2022)
Bai, W., et al.: A population-based phenome-wide association study of cardiac and aortic structure and function. Nat. Med. 26(10), 1654–1662 (2020)
Banerjee, A., et al.: A completely automated pipeline for 3D reconstruction of human heart from 2D cine magnetic resonance slices. Philosophical Trans. Royal Soc. A: Math. Phys. Eng. Sci. 379(2212), 20200257 (2021)
Beetz, M., Banerjee, A., Grau, V.: Biventricular surface reconstruction from cine MRI contours using point completion networks. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 105–109. IEEE (2021)
Beetz, M., Banerjee, A., Grau, V.: Generating subpopulation-specific biventricular anatomy models using conditional point cloud variational autoencoders. In: Puyol Antón, E., et al. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 75–83. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93722-5_9
Beetz, M., Banerjee, A., Grau, V.: Multi-domain variational autoencoders for combined modeling of MRI-based biventricular anatomy and ECG-based cardiac electrophysiology. In: Frontiers in Physiology, p. 991 (2022)
Beetz, M., Banerjee, A., Grau, V.: Point2Mesh-Net: combining point cloud and mesh-based deep learning for cardiac shape reconstruction. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 280–290. Springer (2023). https://doi.org/10.1007/978-3-031-23443-9_26
Beetz, M., Ossenberg-Engels, J., Banerjee, A., Grau, V.: Predicting 3D cardiac deformations with point cloud autoencoders. In: Puyol Antón, E., et al. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 219–228. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93722-5_24
Beetz, M., et al.: Combined generation of electrocardiogram and cardiac anatomy models using multi-modal variational autoencoders. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–4 (2022)
Beetz, M., et al.: Interpretable cardiac anatomy modeling using variational mesh autoencoders. In: Frontiers in Cardiovascular Medicine, p. 3258 (2022)
Beetz, M., et al.: 3D shape-based myocardial infarction prediction using point cloud classification networks. arXiv preprint arXiv:2307.07298 (2023)
Beetz, M., et al.: Mesh U-Nets for 3D cardiac deformation modeling. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 245–257. Springer (2023). https://doi.org/10.1007/978-3-031-23443-9_23
Beetz, M., et al.: Multi-class point cloud completion networks for 3D cardiac anatomy reconstruction from cine magnetic resonance images. arXiv preprint arXiv:2307.08535 (2023)
Beetz, M., et al.: Post-infarction risk prediction with mesh classification networks. In: International Workshop on Statistical Atlases and Computational Models of the Heart. pp. 291–301. Springer (2023). https://doi.org/10.1007/978-3-031-23443-9_27
Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems 14 (2001)
Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37(11), 2514–2525 (2018)
Cetin, I., et al.: A radiomics approach to computer-aided diagnosis with cardiac cine-MRI. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 82–90. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_9
Chang, Y., Jung, C.: Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds. Neurocomputing 418, 270–279 (2020)
Chen, X., et al.: Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds. Med. Image Anal. 74, 102228 (2021)
Corral Acero, J., et al.: Understanding and improving risk assessment after myocardial infarction using automated left ventricular shape analysis. JACC: Cardiovascular Imaging (2022)
Isensee, F., Jaeger, P.F., Full, P.M., Wolf, I., Engelhardt, S., Maier-Hein, K.H.: Automatic cardiac disease assessment on cine-mri via time-series segmentation and domain specific features. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 120–129. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_13
Khan, M.A., et al.: Global epidemiology of ischemic heart disease: results from the global burden of disease study. Cureus 12(7) (2020)
Khened, M., Alex, V., Krishnamurthi, G.: Densely connected fully convolutional network for short-axis cardiac cine MR image segmentation and heart diagnosis using random forest. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 140–151. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_15
Petersen, S.E., et al.: Imaging in population science: cardiovascular magnetic resonance in 100,000 participants of UK Biobank - rationale, challenges and approaches. J. Cardiovasc. Magn. Reson. 15(46), 1–10 (2013)
Petersen, S.E., et al.: UK Biobank’s cardiovascular magnetic resonance protocol. J. Cardiovasc. Magn. Reson. 18(8), 1–7 (2016)
Qi, C.R., et al.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)
Qi, C.R., et al.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)
Reindl, M., et al.: Role of cardiac magnetic resonance to improve risk prediction following acute ST-elevation myocardial infarction. J. Clin. Med. 9(4), 1041 (2020)
Suinesiaputra, A., et al.: Statistical shape modeling of the left ventricle: myocardial infarct classification challenge. IEEE J. Biomed. Health Inform. 22(2), 503–515 (2017)
Wolterink, J.M., Leiner, T., Viergever, M.A., Išgum, I.: Automatic segmentation and disease classification using cardiac cine MR images. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 101–110. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_11
Yang, Y., et al.: Foldingnet: interpretable unsupervised learning on 3D point clouds. arXiv preprint arXiv:1712.07262 (2017)
Ye, M., et al.: PC-U net: learning to jointly reconstruct and segment the cardiac walls in 3D from CT data. In: Puyol Anton, E., et al. (eds.) STACOM 2020. LNCS, vol. 12592, pp. 117–126. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68107-4_12
Yuan, W., et al.: PCN: point completion network. In: 2018 International Conference on 3D Vision (3DV), pp. 728–737 (2018)
Zhang, N., et al.: Deep learning for diagnosis of chronic myocardial infarction on nonenhanced cardiac cine MRI. Radiology 291(3), 606–617 (2019)
Zhou, X.-Y., Wang, Z.-Y., Li, P., Zheng, J.-Q., Yang, G.-Z.: One-stage shape instantiation from a single 2D image to 3D point cloud. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 30–38. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_4
Acknowledgment
This research has been conducted using the UK Biobank Resource under Application Number ‘40161’. The authors express no conflict of interest. The work of M. Beetz was supported by the Stiftung der Deutschen Wirtschaft (Foundation of German Business). A. Banerjee is a Royal Society University Research Fellow and is supported by the Royal Society Grant No. . The work of A. Banerjee was partially supported by the British Heart Foundation (BHF) Project under Grant PG/20/21/35082. The work of V. Grau was supported by the CompBioMed 2 Centre of Excellence in Computational Biomedicine (European Commission Horizon 2020 research and innovation programme, grant agreement No. 823712).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Beetz, M., Banerjee, A., Grau, V. (2023). Multi-objective Point Cloud Autoencoders for Explainable Myocardial Infarction Prediction. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14221. Springer, Cham. https://doi.org/10.1007/978-3-031-43895-0_50
Download citation
DOI: https://doi.org/10.1007/978-3-031-43895-0_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43894-3
Online ISBN: 978-3-031-43895-0
eBook Packages: Computer ScienceComputer Science (R0)