Abstract
Visual recognition of cardiac images is important for cardiac pathology diagnosis and treatment. Due to the limited availability of annotated datasets, traditional methods usually extract features directly from two-dimensional slices of three-dimensional (3D) heart images, followed by pathological classification. This process may not ensure the overall anatomical consistency in 3D heart. A new method for classification of cardiac pathology is therefore proposed based on 3D parametric model reconstruction. First, 3D heart models are reconstructed based on multiple 3D volumes of cardiac imaging data at the end-systole (ES) and end-diastole (ED) phases. Next, based on these reconstructed 3D hearts, 3D parametric models are constructed through the statistical shape model (SSM), and then the heart data are augmented via the variation in shape parameters of one 3D parametric model with visual knowledge constraints. Finally, shape and motion features of 3D heart models across two phases are extracted to classify cardiac pathology. Comprehensive experiments on the automated cardiac diagnosis challenge (ACDC) dataset of the Statistical Atlases and Computational Modelling of the Heart (STACOM) workshop confirm the superior performance and efficiency of this proposed approach.
摘要
心脏图像的视觉识别对于心脏病理诊断和治疗具有重要意义. 由于可用标注数据集有限, 传统方法通常基于三维心脏图像的二维切片对病理分类特征进行提取, 难以确保心脏解剖结构的整体一致性. 为此, 本文提出一种基于三维参数模型重建的心脏病理分类方法. 首先, 基于收缩末期和舒张末期时相心脏图像的多个三维心脏成像数据重建三维心脏模型. 其次, 基于重建的三维心脏模型, 通过统计形状模型方法构建三维参数模型. 然后, 基于三维统计形状模型及其视觉知识约束对心脏数据进行增强. 最后, 提取不同时相的三维心脏模型的形状和运动特征, 对心脏病理进行分类. 在STACOM公开挑战赛的ACDC数据集上的实验验证了所提方法的优越性和有效性.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ammar A, Bouattane O, Youssfi M, 2021. Automatic cardiac cine MRI segmentation and heart disease classification. Comput Med Imag Graph, 88:101864. https://doi.org/10.1016/j.compmedimag.2021.101864
Attar R, Pereañez M, Bowles C, et al., 2019. 3D cardiac shape prediction with deep neural networks: simultaneous use of images and patient metadata. Proc 22nd Int Conf on Medical Image Computing and Computer-Assisted Intervention, p.586–594. https://doi.org/10.1007/978-3-030-32245-8_65
Bai WJ, Oktay O, Rueckert D, 2016. Classification of myocardial infarcted patients by combining shape and motion features. Proc 6th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.140–145. https://doi.org/10.1007/978-3-319-28712-6_15
Bernard O, Lalande A, Zotti C, et al., 2018. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imag, 37(11):2514–2525. https://doi.org/10.1109/TMI.2018.2837502
Bernardino G, Benkarim O, Sanz-de la Garza M, et al., 2020. Handling confounding variables in statistical shape analysis-application to cardiac remodelling. Med Image Anal, 65:101792. https://doi.org/10.1016/j.media.2020.101792
Besl PJ, McKay ND, 1992. A method for registration of 3-D shapes. IEEE Trans Patt Anal Mach Intell, 14(2):239–256. https://doi.org/10.1109/34.121791
Biffi C, Cerrolaza JJ, Tarroni G, et al., 2019. 3D high-resolution cardiac segmentation reconstruction from 2D views using conditional variational autoencoders. Proc IEEE 16th Int Symp on Biomedical Imaging, p.1643–1646. https://doi.org/10.1109/ISBI.2019.8759328
Brock A, Donahue J, Simonyan K, 2019. Large scale GAN training for high fidelity natural image synthesis. https://arxiv.org/abs/1809.11096
Cerqueira MD, Weissman NJ, Dilsizian V, et al., 2002. Standardized myocardial segmentation and nomenclature for tomographic imaging of the heart—a statement for healthcare professionals from the Cardiac Imaging Committee of the Council on Clinical Cardiology of the American Heart Association. Circulation, 105(4):539–542. https://doi.org/10.1161/hc0402.102975
Cetin I, Sanroma G, Petersen SE, et al., 2017. A radiomics approach to computer-aided diagnosis with cardiac cine-MRI. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.82–90. https://doi.org/10.1007/978-3-319-75541-0_9
Chang YK, Jung C, 2020. Automatic cardiac MRI segmentation and permutation-invariant pathology classification using deep neural networks and point clouds. Neurocomputing, 418:270–279. https://doi.org/10.1016/j.neucom.2020.08.030
Charles RQ, Su H, Mo KC, et al., 2017. PointNet: deep learning on point sets for 3D classification and segmentation. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.77–85. https://doi.org/10.1109/CVPR.2017.16
Chen X, Ravikumar N, Xia Y, et al., 2021. Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds. Med Imag Anal, 74:102228. https://doi.org/10.1016/j.media.2021.102228
Cootes TF, Taylor CJ, Cooper DH, et al., 1995. Active shape models—their training and application. Comput Vis Imag Underst, 61(1):38–59. https://doi.org/10.1006/cviu.1995.1004
Dryden IL, Mardia KV, 1998. Statistical Shape Analysis. John Wiley & Sons, London, UK, p.663–669.
Frangi AF, Rueckert D, Schnabel JA, et al., 2002. Automatic construction of multiple-object three-dimensional statistical shape models: application to cardiac modeling. IEEE Trans Med Imag, 21(9):1151–1166. https://doi.org/10.1109/TMI.2002.804426
Gao ZF, Zhang HY, Wang DF, et al., 2018. Robust recovery of myocardial kinematics using dual ℋ∞ criteria. Multim Tools Appl, 77(17):23043–23071. https://doi.org/10.1007/s11042-017-5395-1
Gjesdal O, Bluemke DA, Lima JA, 2011. Cardiac remodeling at the population level—risk factors, screening, and outcomes. Nat Rev Cardiol, 8(12):673–685. https://doi.org/10.1038/nrcardio.2011.154
Isensee F, Jaeger PF, Full PM, et al., 2018. Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.120–129. https://doi.org/10.1007/978-3-319-75541-0_13
Jolliffe IT, 2002. Principal component analysis. J Mark Res, 87(4):513.
Karras T, Laine S, Aila T, 2019. A style-based generator architecture for generative adversarial networks. Proc IEEE/CVF Conf on Computer Vision and Pattern Recognition, p.4396–4405. https://doi.org/10.1109/CVPR.2019.00453
Khened M, Alex V, Krishnamurthi G, 2018. Densely connected fully convolutional network for short-axis cardiac cine MR image segmentation and heart diagnosis using random forest. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.140–151. https://doi.org/10.1007/978-3-319-75541-0_15
Liu HF, Shi PC, 2009. Maximum a posteriori strategy for the simultaneous motion and material property estimation of the heart. IEEE Trans Biomed Eng, 56(2):378–389. https://doi.org/10.1109/TBME.2008.2006012
Pan YH, 1996. The synthesis reasoning. Patt Recogn Artif Intell, 9(3):201–208 (in Chinese).
Pan YH, 2019. On visual knowledge. Front Inform Technol Electron Eng, 20(8):1021–1025. https://doi.org/10.1631/FITEE.1910001
Pan YH, 2020. Multiple knowledge representation of artificial intelligence. Engineering, 6(3):216–217. https://doi.org/10.1016/j.eng.2019.12.011
Pan YH, 2021a. Miniaturized five fundamental issues about visual knowledge. Front Inform Technol Electron Eng, 22(5):615–618. https://doi.org/10.1631/FITEE.2040000
Pan YH, 2021b. On visual understanding. Front Inform Technol Electron Eng, early access. https://doi.org/10.1631/FITEE.2130000
Parajuli N, Lu A, Duncan JS, 2016. Left ventricle classification using active shape model and support vector machine. Proc 6th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.154–161. https://doi.org/10.1007/978-3-319-28712-6_17
Rodero C, Strocchi M, Marciniak M, et al., 2021. Linking statistical shape models and simulated function in the healthy adult human heart. PLoS Comput Biol, 17(4):e1008851. https://doi.org/10.1371/journal.pcbi.1008851
Suinesiaputra A, Ablin P, Albà X, et al., 2018. Statistical shape modeling of the left ventricle: myocardial infarct classification challenge. IEEE J Biomed Health Inform, 22(2):503–515. https://doi.org/10.1109/JBHI.2017.2652449
Thermos S, Liu X, O’Neil A, et al., 2021. Controllable cardiac synthesis via disentangled anatomy arithmetic. Proc 24th Int Conf on Medical Image Computing and Computer-Assisted Intervention, p.160–170. https://doi.org/10.1007/978-3-030-87199-4_15
Van Dijck C, Wirix-Speetjens R, Jonkers I, et al., 2018. Statistical shape model-based prediction of tibiofemoral cartilage. Comput Methods Biomech Biomed Eng, 21(9):568–578. https://doi.org/10.1080/10255842.2018.1495711
Wolterink JM, Leiner T, Viergever MA, et al., 2018. Automatic segmentation and disease classification using cardiac cine MR images. Proc 8th Int Workshop on Statistical Atlases and Computational Models of the Heart, p.101–110. https://doi.org/10.1007/978-3-319-75541-0_11
Zheng Q, Delingette H, Ayache N, 2019. Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow. Med Imag Anal, 56:80–95. https://doi.org/10.1016/j.media.2019.06.001
Zhou QY, Park J, Koltun V, 2016. Fast global registration. Proc 14th European Conf on Computer Vision, p.766–782. https://doi.org/10.1007/978-3-319-46475-6_47
Author information
Authors and Affiliations
Contributions
Yun TIAN and Yunhe PAN designed the research. Jinxiao XIAO and Yansong LI conducted the experiments and drafted the paper. Penghui LI and Shifeng ZHAO helped organize the paper. Dongrong XU revised and finalized the paper.
Corresponding author
Ethics declarations
Jinxiao XIAO, Yansong LI, Yun TIAN, Dongrong XU, Penghui LI, Shifeng ZHAO, and Yunhe PAN declare that they have no conflict of interest.
Additional information
Project supported by the National Natural Science Foundation of China (Nos. 72091511, 62172047, and 61802020)
Rights and permissions
About this article
Cite this article
Xiao, J., Li, Y., Tian, Y. et al. Visual recognition of cardiac pathology based on 3D parametric model reconstruction. Front Inform Technol Electron Eng 23, 1324–1337 (2022). https://doi.org/10.1631/FITEE.2200102
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2200102