Abstract
The precise alignment of retina images from different modalities allows ophthalmologists not only to track morphological/pathological changes over time but also to combine different modalities to approach the diagnosis, prognostication, management and monitoring of a retinal disease. We propose an image registration algorithm to trace changes in the retina structure across modalities using vessel segmentation and automatic landmark detection. The segmentation of the vessels is done using a U-Net and the detection of the vessel junctions is achieved with Mask R-CNN. We evaluated the results of our approach using manual grading by expert readers. In the largest dataset (FA-to-SLO/OCT) containing 1130 pairs we achieve an average error rate of 13.12%. We compared our method with intensity based affine registration methods using original and vessel segmentation images.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. CoRR abs/1703.06870 (2017). http://arxiv.org/abs/1703.06870
Hervella, Á.S., Rouco, J., Novo, J., Ortega, M.: Multimodal registration of retinal images using domain-specific landmarks and vessel enhancement. CoRR abs/1803.00951 (2018). http://arxiv.org/abs/1803.00951
Khojasteh, P., Aliahmad, B., Kumar, D.K.: Fundus images analysis using deep features for detection of exudates, hemorrhages and microaneurysms. In: BMC Ophthalmology (2018)
Klein, S., Staring, M., Murphy, K., Viergever, M.A., Pluim, J.P.W.: elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imag. 29(1), 196–205 (2010). https://doi.org/10.1109/TMI.2009.2035616
Lam, C., Yi, D., Guo, M., Lindsey, T.: Automated detection of diabetic retinopathy using deep learning. AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science 2017, 147–155 (2018)
Li, Y., Gregori, G., Knighton, R.W., Lujan, B., Rosenfeld, P.: Registration of OCT fundus images with color fundus photographs based on blood vessel ridges. Opt. Express 19, 7–16 (2011). https://doi.org/10.1364/OE.19.000007
Li, Z., Huang, F., Zhang, J., Dashtbozorg, B., Abbasi-Sureshjani, S., Sun, Y., Long, X., Yu, Q., ter Haar Romeny, B., Tan, T.: Multi-modal and multi-vendor retina image registration. Biomed. Opt. Express 9(2), 410–422 (2018)
Liskowski, P., Krawiec, K.: Segmenting retinal blood vessels with deep neural networks. IEEE Trans. Med. Imag. 35, 1–1 (2016)
Mattes, D., Haynor, D.R., Vesselle, H., Lewellyn, T.K., Eubank, W.: Nonrigid multimodality image registration. Proc. SPIE - Int. Soc. Opt. Eng. 4322, 1609–1620 (2001). https://doi.org/10.1117/12.431046
Miri, M.S., Abramoff, M., Kwon, Y.H., Garvin, M.K.: Multimodal registration of SD-OCT volumes and fundus photographs using histograms of oriented gradients. Biomed. Opt. Express 7, 5252–5267 (2016)
Novais, E., Baumal, C., Sarraf, D., Freund, K., Duker, J.: Multimodal imaging in retinal disease: a consensus definition. Ophthalmic Surg. Lasers & Imag. Retina 47, 201–205 (2016). https://doi.org/10.3928/23258160-20160229-01
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
Palén, A.: Advanced algorithms for manipulating 2D objects on touch screens. Master’s thesis, Tampere University of Technology (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Schlegl, T., et al.: Fully automated detection and quantification of macular fluid in OCT using deep learning. Ophthalmology 125(4), 549–558 (2018)
Schmitz-Valckenberg, S., Holz, F., Bird, A., Spaide, R.F.: Fundus autofluorescence imaging: review and perspectives. Retina (Philadelphia, Pa.) 28, 385–409 (2008). https://doi.org/10.1097/IAE.0b013e318164a907
Zana, F., Klein, J.C.: A multimodal registration algorithm of eye fundus images using vessels detection and hough transform. IEEE Trans. Med. Imag. 18(5), 419–428 (1999). https://doi.org/10.1109/42.774169
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Arikan, M., Sadeghipour, A., Gerendas, B., Told, R., Schmidt-Erfurt, U. (2019). Deep Learning Based Multi-modal Registration for Retinal Imaging. In: Suzuki, K., et al. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support. ML-CDS IMIMIC 2019 2019. Lecture Notes in Computer Science(), vol 11797. Springer, Cham. https://doi.org/10.1007/978-3-030-33850-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-33850-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33849-7
Online ISBN: 978-3-030-33850-3
eBook Packages: Computer ScienceComputer Science (R0)
