Skip to main content

Multi-objective Point Cloud Autoencoders for Explainable Myocardial Infarction Prediction

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023 (MICCAI 2023)

Abstract

Myocardial infarction (MI) is one of the most common causes of death in the world. Image-based biomarkers commonly used in the clinic, such as ejection fraction, fail to capture more complex patterns in the heart’s 3D anatomy and thus limit diagnostic accuracy. In this work, we present the multi-objective point cloud autoencoder as a novel geometric deep learning approach for explainable infarction prediction, based on multi-class 3D point cloud representations of cardiac anatomy and function. Its architecture consists of multiple task-specific branches connected by a low-dimensional latent space to allow for effective multi-objective learning of both reconstruction and MI prediction, while capturing pathology-specific 3D shape information in an interpretable latent space. Furthermore, its hierarchical branch design with point cloud-based deep learning operations enables efficient multi-scale feature learning directly on high-resolution anatomy point clouds. In our experiments on a large UK Biobank dataset, the multi-objective point cloud autoencoder is able to accurately reconstruct multi-temporal 3D shapes with Chamfer distances between predicted and input anatomies below the underlying images’ pixel resolution. Our method outperforms multiple machine learning and deep learning benchmarks for the task of incident MI prediction by 19% in terms of Area Under the Receiver Operating Characteristic curve. In addition, its task-specific compact latent space exhibits easily separable control and MI clusters with clinically plausible associations between subject encodings and corresponding 3D shapes, thus demonstrating the explainability of the prediction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Avard, E., et al.: Non-contrast cine cardiac magnetic resonance image radiomics features and machine learning algorithms for myocardial infarction detection. Comput. Biol. Med. 141, 105145 (2022)

    Article  Google Scholar 

  2. Bai, W., et al.: A population-based phenome-wide association study of cardiac and aortic structure and function. Nat. Med. 26(10), 1654–1662 (2020)

    Article  Google Scholar 

  3. Banerjee, A., et al.: A completely automated pipeline for 3D reconstruction of human heart from 2D cine magnetic resonance slices. Philosophical Trans. Royal Soc. A: Math. Phys. Eng. Sci. 379(2212), 20200257 (2021)

    Article  Google Scholar 

  4. Beetz, M., Banerjee, A., Grau, V.: Biventricular surface reconstruction from cine MRI contours using point completion networks. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 105–109. IEEE (2021)

    Google Scholar 

  5. Beetz, M., Banerjee, A., Grau, V.: Generating subpopulation-specific biventricular anatomy models using conditional point cloud variational autoencoders. In: Puyol Antón, E., et al. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 75–83. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93722-5_9

    Chapter  Google Scholar 

  6. Beetz, M., Banerjee, A., Grau, V.: Multi-domain variational autoencoders for combined modeling of MRI-based biventricular anatomy and ECG-based cardiac electrophysiology. In: Frontiers in Physiology, p. 991 (2022)

    Google Scholar 

  7. Beetz, M., Banerjee, A., Grau, V.: Point2Mesh-Net: combining point cloud and mesh-based deep learning for cardiac shape reconstruction. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 280–290. Springer (2023). https://doi.org/10.1007/978-3-031-23443-9_26

  8. Beetz, M., Ossenberg-Engels, J., Banerjee, A., Grau, V.: Predicting 3D cardiac deformations with point cloud autoencoders. In: Puyol Antón, E., et al. (eds.) STACOM 2021. LNCS, vol. 13131, pp. 219–228. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-93722-5_24

    Chapter  Google Scholar 

  9. Beetz, M., et al.: Combined generation of electrocardiogram and cardiac anatomy models using multi-modal variational autoencoders. In: 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), pp. 1–4 (2022)

    Google Scholar 

  10. Beetz, M., et al.: Interpretable cardiac anatomy modeling using variational mesh autoencoders. In: Frontiers in Cardiovascular Medicine, p. 3258 (2022)

    Google Scholar 

  11. Beetz, M., et al.: 3D shape-based myocardial infarction prediction using point cloud classification networks. arXiv preprint arXiv:2307.07298 (2023)

  12. Beetz, M., et al.: Mesh U-Nets for 3D cardiac deformation modeling. In: International Workshop on Statistical Atlases and Computational Models of the Heart, pp. 245–257. Springer (2023). https://doi.org/10.1007/978-3-031-23443-9_23

  13. Beetz, M., et al.: Multi-class point cloud completion networks for 3D cardiac anatomy reconstruction from cine magnetic resonance images. arXiv preprint arXiv:2307.08535 (2023)

  14. Beetz, M., et al.: Post-infarction risk prediction with mesh classification networks. In: International Workshop on Statistical Atlases and Computational Models of the Heart. pp. 291–301. Springer (2023). https://doi.org/10.1007/978-3-031-23443-9_27

  15. Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: Advances in Neural Information Processing Systems 14 (2001)

    Google Scholar 

  16. Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37(11), 2514–2525 (2018)

    Article  Google Scholar 

  17. Cetin, I., et al.: A radiomics approach to computer-aided diagnosis with cardiac cine-MRI. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 82–90. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_9

    Chapter  Google Scholar 

  18. Chang, Y., Jung, C.: Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds. Neurocomputing 418, 270–279 (2020)

    Article  Google Scholar 

  19. Chen, X., et al.: Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds. Med. Image Anal. 74, 102228 (2021)

    Article  Google Scholar 

  20. Corral Acero, J., et al.: Understanding and improving risk assessment after myocardial infarction using automated left ventricular shape analysis. JACC: Cardiovascular Imaging (2022)

    Google Scholar 

  21. Isensee, F., Jaeger, P.F., Full, P.M., Wolf, I., Engelhardt, S., Maier-Hein, K.H.: Automatic cardiac disease assessment on cine-mri via time-series segmentation and domain specific features. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 120–129. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_13

    Chapter  Google Scholar 

  22. Khan, M.A., et al.: Global epidemiology of ischemic heart disease: results from the global burden of disease study. Cureus 12(7) (2020)

    Google Scholar 

  23. Khened, M., Alex, V., Krishnamurthi, G.: Densely connected fully convolutional network for short-axis cardiac cine MR image segmentation and heart diagnosis using random forest. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 140–151. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_15

    Chapter  Google Scholar 

  24. Petersen, S.E., et al.: Imaging in population science: cardiovascular magnetic resonance in 100,000 participants of UK Biobank - rationale, challenges and approaches. J. Cardiovasc. Magn. Reson. 15(46), 1–10 (2013)

    Google Scholar 

  25. Petersen, S.E., et al.: UK Biobank’s cardiovascular magnetic resonance protocol. J. Cardiovasc. Magn. Reson. 18(8), 1–7 (2016)

    Google Scholar 

  26. Qi, C.R., et al.: Pointnet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652–660 (2017)

    Google Scholar 

  27. Qi, C.R., et al.: Pointnet++: deep hierarchical feature learning on point sets in a metric space. In: Advances in Neural Information Processing Systems, pp. 5099–5108 (2017)

    Google Scholar 

  28. Reindl, M., et al.: Role of cardiac magnetic resonance to improve risk prediction following acute ST-elevation myocardial infarction. J. Clin. Med. 9(4), 1041 (2020)

    Article  MathSciNet  Google Scholar 

  29. Suinesiaputra, A., et al.: Statistical shape modeling of the left ventricle: myocardial infarct classification challenge. IEEE J. Biomed. Health Inform. 22(2), 503–515 (2017)

    Google Scholar 

  30. Wolterink, J.M., Leiner, T., Viergever, M.A., Išgum, I.: Automatic segmentation and disease classification using cardiac cine MR images. In: Pop, M., et al. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 101–110. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_11

    Chapter  Google Scholar 

  31. Yang, Y., et al.: Foldingnet: interpretable unsupervised learning on 3D point clouds. arXiv preprint arXiv:1712.07262 (2017)

  32. Ye, M., et al.: PC-U net: learning to jointly reconstruct and segment the cardiac walls in 3D from CT data. In: Puyol Anton, E., et al. (eds.) STACOM 2020. LNCS, vol. 12592, pp. 117–126. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-68107-4_12

    Chapter  Google Scholar 

  33. Yuan, W., et al.: PCN: point completion network. In: 2018 International Conference on 3D Vision (3DV), pp. 728–737 (2018)

    Google Scholar 

  34. Zhang, N., et al.: Deep learning for diagnosis of chronic myocardial infarction on nonenhanced cardiac cine MRI. Radiology 291(3), 606–617 (2019)

    Article  Google Scholar 

  35. Zhou, X.-Y., Wang, Z.-Y., Li, P., Zheng, J.-Q., Yang, G.-Z.: One-stage shape instantiation from a single 2D image to 3D point cloud. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11767, pp. 30–38. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32251-9_4

    Chapter  Google Scholar 

Download references

Acknowledgment

This research has been conducted using the UK Biobank Resource under Application Number ‘40161’. The authors express no conflict of interest. The work of M. Beetz was supported by the Stiftung der Deutschen Wirtschaft (Foundation of German Business). A. Banerjee is a Royal Society University Research Fellow and is supported by the Royal Society Grant No. . The work of A. Banerjee was partially supported by the British Heart Foundation (BHF) Project under Grant PG/20/21/35082. The work of V. Grau was supported by the CompBioMed 2 Centre of Excellence in Computational Biomedicine (European Commission Horizon 2020 research and innovation programme, grant agreement No. 823712).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Marcel Beetz .

Editor information

Editors and Affiliations

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 101 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Beetz, M., Banerjee, A., Grau, V. (2023). Multi-objective Point Cloud Autoencoders for Explainable Myocardial Infarction Prediction. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14221. Springer, Cham. https://doi.org/10.1007/978-3-031-43895-0_50

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43895-0_50

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43894-3

  • Online ISBN: 978-3-031-43895-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics