Abstract
We present a new framework for retinal artery/vein classification from fundus images and corresponding fluorescein angiography (FA) images. While FA seem to provide the most relevant information, it is often insufficient depending on the acquisition conditions. As fundus images are often acquired by default, we combine the fundus image and FA within a parallel convolutional neural network to extract the maximum information in the generated features. Furthermore, we use these features as the input to a hierarchical graph neural network to ensure that the connectivity of vessels plays a part in the classification. We provide investigative evidence through ablative and comparative quantitative evaluations to better determine the optimal configuration in combining the fundus image and FA in a deep learning framework and demonstrate the enhancement in performance compared to previous methods.
This work was supported by the National Research Foundation of Korea (NRF) grants funded by the Korean government (MSIT) (NRF-2018R1D1A1A09083241 and NRF-2020R1F1A1051847).
This work was done while K. Noh was with Seoul National University Bundang Hospital.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alam, M., Toslak, D., Lim, J.I., Yao, X.: Color fundus image guided artery-vein differentiation in optical coherence tomography angiography. Invest. Ophthal. Vis. Sci. 59(12), 4953–4962 (2018). https://doi.org/10.1167/iovs.18-24831
Badawi, S.A., Fraz, M.M.: Multiloss function based deep convolutional neural network for segmentation of retinal vasculature into arterioles and venules. BioMed Res. Int. 2019, 1–17 (2019). https://doi.org/10.1155/2019/4747230
Cunha-Vaz, J.G.: Pathophysiology of diabetic retinopathy. Br. J. Ophthalmol. 62(6), 351–355 (1978). https://doi.org/10.1136/bjo.62.6.351
Eppenhof, K., Bekkers, E., Berendschot, T.T., Pluim, J., ter Haar Romeny, B.: Retinal artery/vein classification via graph cut optimization. In: Proceedings of the Ophthalmic Medical Image Analysis Second International Workshop, OMIA 2015, Held in Conjunction with MICCAI 2015, pp. 121–128 (2015). https://doi.org/10.17077/omia.1035
Estrada, R., Allingham, M.J., Mettu, P.S., Cousins, S.W., Tomasi, C., Farsiu, S.: Retinal artery-vein classification via topology estimation. IEEE Trans. Med. Imaging 34(12), 2518–2534 (2015). https://doi.org/10.1109/TMI.2015.2443117
Gao, H., Ji, S.: Graph u-nets. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of International Conference on Machine Learning (ICML). Proceedings of Machine Learning Research, Long Beach, California, USA, 09–15 June 2019, vol. 97, pp. 2083–2092. PMLR (2019). http://proceedings.mlr.press/v97/gao19a.html
Hemelings, R., Elen, B., Stalmans, I., Keer, K.V., Boever, P.D., Blaschko, M.B.: Artery-vein segmentation in fundus images using a fully convolutional network. Comput. Med. Imaging Graph. 76, 101636 (2019). https://doi.org/10.1016/j.compmedimag.2019.05.004
Hu, Q., Abràmoff, M.D., Garvin, M.K.: Automated separation of binary overlapping trees in low-contrast color retinal images. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 436–443. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40763-5_54
Ikram, M.K., et al.: Retinal vessel diameters and cerebral small vessel disease: the Rotterdam Scan Study. Brain 129(1), 182–188 (2005). https://doi.org/10.1093/brain/awh688
Joshi, V.S., Reinhardt, J.M., Garvin, M.K., Abramoff, M.D.: Automated method for identification and artery-venous classification of vessel trees in retinal vessel networks. PLoS ONE 9(2), 1–12 (2014). https://doi.org/10.1371/journal.pone.0088061
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: Proceedings of International Conference on Learning Representations (ICLR) (2017)
Ma, W., Yu, S., Ma, K., Wang, J., Ding, X., Zheng, Y.: Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 769–778. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_85
Meyer, M.I., Galdran, A., Costa, P., Mendonça, A.M., Campilho, A.: Deep convolutional artery/vein classification of retinal vessels. In: Campilho, A., Karray, F., ter Haar Romeny, B. (eds.) ICIAR 2018. LNCS, vol. 10882, pp. 622–630. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93000-8_71
Noh, K.J., Kim, J., Park, S.J., Lee, S.: Multimodal registration of fundus images with fluorescein angiography for fine-scale vessel segmentation. IEEE Access 8, 63757–63769 (2020)
Noh, K.J., Park, S.J., Lee, S.: Fine-scale vessel extraction in fundus images by registration with fluorescein angiography. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 779–787. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_86
Noh, K.J., Park, S.J., Lee, S.: Scale-space approximated convolutional neural networks for retinal vessel segmentation. Comput. Methods Programs Biomed. (2019). https://doi.org/10.1016/j.cmpb.2019.06.030
Relan, D., MacGillivray, T., Ballerini, L., Trucco, E.: Retinal vessel classification: sorting arteries and veins. In: Proceedings of 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7396–7399 (2013). https://doi.org/10.1109/EMBC.2013.6611267
Ritt, M., Schmieder, R.E.: Wall-to-lumen ratio of retinal arterioles as a tool to assess vascular changes. Hypertension 54(2), 384–387 (2009). https://doi.org/10.1161/HYPERTENSIONAHA.109.133025
Shin, S.Y., Lee, S., Yun, I.D., Lee, K.M.: Deep vessel segmentation by learning graphical connectivity. Med. Image Anal. 58, 101556 (2019). https://doi.org/10.1016/j.media.2019.101556
Simó, A., de Ves, E.: Segmentation of macular fluorescein angiographies. A statistical approach. Pattern Recogn. 34(4), 795–809 (2001). https://doi.org/10.1016/S0031-3203(00)00032-7
Staal, J., Abramoff, M.D., Niemeijer, M., Viergever, M.A., van Ginneken, B.: Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 23(4), 501–509 (2004). https://doi.org/10.1109/TMI.2004.825627
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Li, P., Bengio, Y.: Graph attention networks. In: Proceedings of International Conference on Learning Representations (ICLR) (2018). https://openreview.net/forum?id=rJXMpikCZ
Welikala, R., et al.: Automated arteriole and venule classification using deep learning for retinal images from the UK biobank cohort. Comput. Biol. Med. 90, 23–32 (2017). https://doi.org/10.1016/j.compbiomed.2017.09.005
Xu, X., Ding, W., Abrmoff, M.D., Cao, R.: An improved arteriovenous classification method for the early diagnostics of various diseases in retinal image. Comput. Methods Programs Biomed. 141, 3–9 (2017). https://doi.org/10.1016/j.cmpb.2017.01.007
Zhai, Z., et al.: Linking convolutional neural networks with graph convolutional networks: application in pulmonary artery-vein separation. In: Zhang, D., Zhou, L., Jie, B., Liu, M. (eds.) GLMI 2019. LNCS, vol. 11849, pp. 36–43. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35817-4_5
Zhao, Y., et al.: Retinal vascular network topology reconstruction and artery/vein classification via dominant set clustering. IEEE Trans. Med. Imaging 39(2), 341–356 (2020). https://doi.org/10.1109/TMI.2019.2926492
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Noh, K.J., Park, S.J., Lee, S. (2020). Combining Fundus Images and Fluorescein Angiography for Artery/Vein Classification Using the Hierarchical Vessel Graph Network. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12265. Springer, Cham. https://doi.org/10.1007/978-3-030-59722-1_57
Download citation
DOI: https://doi.org/10.1007/978-3-030-59722-1_57
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59721-4
Online ISBN: 978-3-030-59722-1
eBook Packages: Computer ScienceComputer Science (R0)
