Abstract
Unpaired image-to-image translation of retinal images can efficiently increase the training dataset for deep-learning-based multi-modal retinal registration methods. Our method integrates a vessel segmentation network into the image-to-image translation task by extending the CycleGAN framework. The segmentation network is inserted prior to aUNet vision transformer generator network and serves as a shared representation between both domains. We reformulate the original identity loss to learn the direct mapping between the vessel segmentation and the real image. Additionally, we add a segmentation loss termto ensure shared vessel locations between fake and real images. In the experiments, our method shows a visually realistic look and preserves the vessel structures, which is a prerequisite for generating multi-modal training data for image registration.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Isola P, Zhu JY, Zhou T, Efros AA. Image-to-image translation with conditional adversarial networks. Proc IEEE CVPR 2017. 2017.
Kamran SA, Hossain KF, Tavakkoli A, Zuckerbrod SL, Baker SA. VTGAN: semi-supervised retinal image synthesis and disease prediction using vision transformers. 2021 IEEE/CVF ICCVW. 2021:3228–38.
Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation using cycleconsistent adversarial networks. Proc IEEE ICCV 2017. 2017:2242–51.
Li K, Yu L, Wang S, Heng PA. Unsupervised retina image synthesis via disentangled representation learning. SASHIMI 2019. 2019:32–41.
Cai Z, Xin J, Wu J, Liu S, Zuo W, Zheng N. Triple multi-scale adversarial learning with self-attention and quality loss for unpaired fundus fluorescein angiography synthesis. IEEE EMBC 2020. 2020:1592–5.
Liang N, Yuan L,Wen X, Xu H,Wang J. End-to-end retina image synthesis based on CGAN using class feature loss and improved retinal detail loss. IEEE Access. 2022;10:83125–37.
Niu Y, Gu L, Zhao Y, Lu F. Explainable diabetic retinopathy detection and retinal image generation. IEEE J Biomed Health Inform. 2022;26(1):44–55.
Torbunov D, Huang Y,Yu H, Huang J,Yoo S, LinMet al.UVCGAN:UNet vision transformer cycle-consistentGANfor unpaired image-to-image translation. Proc IEEE/CVFWACV2023. 2023:702–12.
Hajeb Mohammad Alipour S, Rabbani H, Akhlaghi MR. Diabetic retinopathy grading by digital curvelet transform. Comput Math Methods Med. 2012;2021:761901.
Sindel A, Hohberger B, Maier A, Christlein V. Multi-modal retinal image registration using a keypoint-based vessel structure aligning network. MICCAI 2022. 2022:108–18.
Budai A, Bock R, Maier A, Hornegger J, Michelson G. Robust vessel segmentation in fundus images. Int J Biomed Imaging. 2013.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Der/die Autor(en), exklusiv lizenziert an Springer Fachmedien Wiesbaden GmbH, ein Teil von Springer Nature
About this paper
Cite this paper
Sindel, A., Maier, A., Christlein, V. (2023). A Vesselsegmentation-based CycleGAN for Unpaired Multi-modal Retinal Image Synthesis. In: Deserno, T.M., Handels, H., Maier, A., Maier-Hein, K., Palm, C., Tolxdorff, T. (eds) Bildverarbeitung für die Medizin 2023. BVM 2023. Informatik aktuell. Springer Vieweg, Wiesbaden. https://doi.org/10.1007/978-3-658-41657-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-658-41657-7_11
Published:
Publisher Name: Springer Vieweg, Wiesbaden
Print ISBN: 978-3-658-41656-0
Online ISBN: 978-3-658-41657-7
eBook Packages: Computer Science and Engineering (German Language)