Abstract
The accumulation of carotid plaque leads to carotid artery stenosis, which in turn increases the risk of cerebrovascular disease. Non-invasive diagnosis of carotid stenosis using fundus images offers a promising approach. However, the challenge lies in extracting relevant features from these images, as convolutional neural networks(CNNs) or Transformers, which focus solely on individual images, fail to consider the interdependencies between them, leading to limited diagnostic accuracy. To address this issue, we propose a novel and effective network by combining CNNs and multi-relational graph convolutional neural networks(M-GCNs). Firstly, we feed the input images into four distinct branches, which consist of CNNs or Transformers, with each branch associated with a particular relation. This process generates unique feature vectors for each branch. Secondly, we construct a multi-graph for the four kinds of clinical data, such as gender, age, sex and pid, to obtain four adjacency matrices. Finally, the feature vectors and the corresponding four adjacency matrices are input into the graph convolutional network layer respectively to obtain the prediction features, and then the prediction results are obtained through the fully connected layer. Experiments are carried out on a private dataset and the results demonstrate that the accuracy of the proposed algorithm is 10%–20% higher than that of the comparison model. Our code is available at https://github.com/momoyrz/Carotid-stenosis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Belotti, F., et al.: Ophthalmic artery originating from the anterior cerebral artery: anatomo-radiological study, histological analysis, and literature review. Neurosurg. Rev. 39(3), 483–493 (2016)
Bruna, J., Zaremba, W., Szlam, A., LeCun, Y.: Spectral networks and locally connected networks on graphs (2014)
Chaikijurajai, T., Ehlers, J.P., Tang, W.H.W.: Retinal microvasculature: a potential window into heart failure prevention. JACC Heart Fail. 10(11), 785–791 (2022)
Chen, C.F.R., Fan, Q., Panda, R.: Crossvit: cross-attention multi-scale vision transformer for image classification. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 357–366 (2021)
Debrey, S.M., et al.: Diagnostic accuracy of magnetic resonance angiography for internal carotid artery disease: a systematic review and meta-analysis. Stroke 39(8), 2237–2248 (2008)
Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering (2017)
Fu, H., Cheng, J., Xu, Y., Wong, D.W.K., Liu, J., Cao, X.: Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. IEEE Trans. Medical Imaging 37(7), 1597–1605 (2018). https://doi.org/10.1109/TMI.2018.2791488
Gulshan, V., et al.: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22), 2402–2410 (2016)
Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. Adv. Neural Inf. Process. Syst. 30, 1–11 (2017)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. IEEE, Las Vegas (2016)
Jiang, B., Zhang, Z., Lin, D., Tang, J., Luo, B.: Semi-supervised learning with graph learning-convolutional networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11305–11312 (2019)
Kohane, I.S.: Ten things we have to do to achieve precision medicine. Science 349(6243), 37–38 (2015). https://doi.org/10.1126/science.aab1328
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. Commun. ACM 60(6), 84–90 (2017)
Lee, J.I., et al.: Stroke in patients with occlusion of the internal carotid artery: options for treatment. Expert Rev. Neurotherapeut. 14(10), 1153–1167 (2014)
Li, F., et al.: Deep learning-based automated detection for diabetic retinopathy and diabetic macular oedema in retinal fundus photographs. Eye 36(7), 1433–1441 (2022)
Li, X., Ng, M.K., Xu, G., Yip, A.: Multi-relational graph convolutional networks: Generalization guarantees and experiments. Neural Netw. 161, 343–358 (2023). https://www.sciencedirect.com/science/article/abs/pii/S0893608023000576
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9992–10002. IEEE, Montreal (2021)
Lu, W., Tong, Y., Yu, Y., Xing, Y., Chen, C., Shen, Y.: Deep learning-based automated classification of multi-categorical abnormalities from optical coherence tomography images. Transl. Vision Sci. Technol. 7(6), 41–41 (2018)
van der Maaten, L., Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9(86), 2579–2605 (2008). https://www.jmlr.org/papers/v9/vandermaaten08a.html
Mao, C., Yao, L., Luo, Y.: ImageGCN: multi-relational image graph convolutional networks for disease identification with chest x-rays. IEEE Trans. Med. Imaging 41(8), 1990–2003 (2022)
Momjian-Mayor, I., Baron, J.C.: The pathophysiology of watershed infarction in internal carotid artery disease: review of cerebral perfusion studies. Stroke 36(3), 567–577 (2005)
Schlichtkrull, M., Kipf, T.N., Bloem, P., van den Berg, R., Titov, I., Welling, M.: Modeling relational data with graph convolutional networks. In: Gangemi, A., et al. (eds.) ESWC 2018. LNCS, vol. 10843, pp. 593–607. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93417-4_38
Shaban, M., Awan, R., Fraz, M.M., Azam, A., Snead, D., Rajpoot, N.M.: Context-aware convolutional neural network for grading of colorectal cancer histology images (2019)
Shamshad, F., et al.: Transformers in medical imaging: a survey. Med. Image Anal. 88, 102802 (2023)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114. PMLR (2019)
Wiebers, D.O., et al.: Pathogenesis, natural history, and treatment of unruptured intracranial aneurysms. In: Mayo Clinic Proceedings, vol. 79, pp. 1572–1583. Elsevier (2004)
Zitnik, M., Agrawal, M., Leskovec, J.: Modeling polypharmacy side effects with graph convolutional networks. Bioinformatics 34(13), i457–i466 (2018)
Acknowledgements
This work is supported in part by the National Natural Science Foundation of China(No.U22A2024, 62106153, 82271103), Guangdong Basic and Applied Basic Research Foundation(No.2020A1515110605, 2022A15150 12326) and Natural Science Foundation of Shenzhen(No.JCYJ20220818095809021).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Qu, J. et al. (2023). Multi-relational Graph Convolutional Neural Networks for Carotid Artery Stenosis Diagnosis via Fundus Images. In: Antony, B., Chen, H., Fang, H., Fu, H., Lee, C.S., Zheng, Y. (eds) Ophthalmic Medical Image Analysis. OMIA 2023. Lecture Notes in Computer Science, vol 14096. Springer, Cham. https://doi.org/10.1007/978-3-031-44013-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-44013-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44012-0
Online ISBN: 978-3-031-44013-7
eBook Packages: Computer ScienceComputer Science (R0)
