Data Augmentation with Generative Methods for Inherited Retinal Diseases: A Systematic Review
Abstract
:1. Introduction
2. Materials and Methods
2.1. Related Work
2.2. Search Question
- Can generative methods be used to generate ocular fundus autofluorescence data in inherited retinal diseases?
- Which generative models are most efficiently used to generate synthetic ocular fundus autofluorescence data in inherited retinal diseases?
- How should synthetic ocular fundus autofluorescence data generated by generative models be evaluated, so that it can be used to train classifiers in inherited retinal diseases?
2.3. Search Strategy
2.4. Data Extraction and Synthesis
3. Results
Characteristics of the Included Studies
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
IRD | Inherited Retinal Diseases |
FAF | Autofluorescence Ocular Fundus |
OCT | Optical Coherence Tomography |
AI | and optical coherence tomography |
GAN | Generative Adversarial Network |
PRISMA | Preferred Reporting Items for Systematic reviews and Meta-Analyses |
MeSH | Medical Subject Headings |
DME | Diabetic Macular Edema |
DR | Diabetic Retinopathy |
AMD | Age-related Macular Degeneration |
Appendix A
Architectures Analyses
Architectures | Diseases | Strengths | Weaknesses |
---|---|---|---|
DCGAN [37] | DR | The DCGAN architecture is successful in generating synthetic images, including medical images such as FAF images. DCGAN allows processing to be focused on specific regions of the images, which is useful for tasks such as analyzing small areas or areas of interest in the images. | DCGAN has difficulties with unbalanced datasets, especially when the images of different severities of the condition are poorly represented. To generate realistic images, DCGAN usually needs a large amount of data to learn the probability of distribution of real data. DCGAN also suffers from instability problems, such as non-convergence and mode collapse. While the images generated by the DCGAN are structurally similar to real images, they can present notable distortions in the frequency domain. |
StyleGAN2 [14] | IRD | StyleGAN2 generates synthetic images of hereditary retinal diseases with high visual quality, which experts classify as realistic. The images generated have similar diversity to the real ones and are not exact copies, making them suitable for further analysis. Models trained with synthetic data offer similar classification performance to real data. StyleGAN2 is chosen for its image quality, short training time, and relatively low computational cost. | The quality of synthetic images can be compromised by problems such as low exposure and background leakage. The qualitative evaluation of images is subjective, with great variation in scores and possible confusion caused by overlapping and atypical phenotypes. Disease classes overlap in the feature space, which can cause confusion when generating synthetic images. GAN can memorize images or capture subtle attributes, especially if the dataset is small or the training long. The quality of synthetic images is lower than that of real data. |
CycleGAN [26] | IRD | Effective in generalizing OCT images of rare diseases with few examples, avoiding overfitting. Maintains the structures of the choroid and peripheral retina when translating normal images into pathological ones. Creates new samples from normal OCT images to increase variation in rare disease classes. Extensible technique for segmentation with small datasets. Generates synthetic images with transformation of morphological characteristics. | Synthetic images can have artifacts, requiring careful selection to build deep learning models. OCT images generated have a resolution of 256×256, which can impact classification. High computational cost to train high-resolution models. Some rare diseases, such as Stargardt and retinitis pigmentosa, have high rejection rates. |
CGAN [53] | Retinal Diseases | Generation of images conditional on labels or other auxiliary information, making it possible to create specific synthetic datasets. High quality and realism of the images generated, which suggests that synthetic images closely resemble real images. The ability to generate a large number of synthetic images demonstrates CGAN’s potential to augment existing medical datasets. | CGAN is an opaque model, which means that it can be difficult to fully understand the internal process of generating the images. The images generated by CGAN may contain artifacts or be of variable quality. CGAN’s performance depends heavily on the quality and diversity of the actual training data. Training GANs, including CGANs, can be challenging and unstable, requiring careful selection of hyperparameters and training strategies to avoid problems such as model collapse. |
WGAN [36] | DR | WGAN is effective at generating high-quality synthetic images with great diversity and realism. The architecture is robust and successful in challenging imaging tasks. WGAN solves one of the main problems in traditional GANs (such as DCGAN), mode collapse, allowing a wider variety of samples to be generated. Using the Wasserstei distance improves the convergence and quality of the images, helping the generator to produce more realistic samples. The images generated have a high degree of realism, with variation in the characteristics of real images, such as the optic disc and veins. The synthetic images generated are almost indistinguishable from the real ones, with a slight difference in noise. | Training can be unstable, with oscillations in the error. When examining the images, you can see that small artefacts remain, and the image is slightly more pixilated than the original images. The WGAN discriminator estimates the parameter w of the continuous K-Lipschitz function, rather than the probability of the image being real, making it unsuitable for inpainting. The automatic inpainting methodology is limited by the size of the image produced by the WGAN. |
StyleGAN [29] | AMD | StyleGAN is effective at creating high-resolution images, such as retinal images that are almost indistinguishable from the real thing. StyleGAN can generate retinal images with specific diseases, even with little training data, and can balance unbalanced datasets, improving the performance of diagnostic models. The intermediate latent space allows visual attributes to be modified in the images, which provides more flexibility compared to other GANs. StyleGAN successfully maintains vascular structures in retinal images, which was not a priority in previous studies. StyleGAN has great potential in medical areas, such as data augmentation and protection of patient privacy. | Training StyleGAN to generate high-quality synthetic images is still challenging due to the need for a lot of data. The images generated need improvements in microstructures, such as the representation of the optic disc and vascularization. Some atypical features, such as abnormal structures in the optic disc, irregular vessels, and atypical reflections, were observed in the images generated. |
StyleGAN-ADA [27] | AMD | Experts were unable to accurately differentiate between real and synthetic images, demonstrating the realism of the images generated. The model trained on three public datasets and performed well on an external dataset, demonstrating its ability to generalize in the detection of AMD. It allows synthetic images to be created where macular degeneration is evident or absent. The use of ADA improves the quality and diversity of the images, making the model more efficient, especially when there is little training data. | If the synthetic data is not representative or diverse enough, it can introduce noise into the training of deep learning models. Validation with a small dataset may not be enough to guarantee robust generalization. |
LDM [45] | Retinal Blood Vessel | LDM is effective in data augmentation. It combines diffusion models with autoencoders to generate smaller latent representations, making processing more efficient. LDM uses L1 loss, perceptual loss, and patch-based adversarial objectives to optimize autoencoder training. LDM outperforms other GAN architectures in terms of efficiency, with better results in FID (lower value, higher quality) and IS (higher value, more diversity in the images). LDM generated high-quality data, with a high inception score, and low FID, indicating that the augmented data is very similar to the original dataset. | The LDM still faces difficulties in generating high-resolution images during the data augmentation process. |
VAE [41] | DME | VAE networks are naturally stable, giving greater stability to the GAN network when used together. The integration of the VAE improves the stability of the GAN, providing faster convergence and avoiding mode collapse. The structure of the pre-trained VAE decoder is transferred to the generator, improving initialization and performance. | VAEs, in general, can generate less sharp and realistic images than GANs, often resulting in blurrier images. The article focuses more on how VAE improves GAN, without detailing other specific limitations. |
Appendix B
Research Specification
Database | Search Sentence | Filters | Number of Studies |
---|---|---|---|
PubMed | (Generative Model* [Text word] OR Generative adversarial network [Text word] OR GAN [Text word] OR Data Augmentation [Text word] OR Synthetic Data Generation [Text word] OR Augmentation Techniques [Text word]) AND (Retinal Diseases [MESH] OR Retinal Degeneration [MESH] OR Retinitis Pigmentosa [MESH] OR Inherited retinal disease* [Text word]) | No filters | 85 |
IEEE Xplore | (Generative Model* OR Generative adversarial network OR GAN OR Data Augmentation OR Synthetic Data Generation OR Augmentation Techniques) AND (Retinal Diseases OR Retinal Degeneration OR Retinitis Pigmentosa OR Inherited retinal disease*) | Search by: “All Metadata e Mesh_Terms (Retinal Diseases OR Retinal Degeneration OR Retinitis Pigmentosa)” | 187 |
Web of Science | (Generative Model* OR Generative adversarial network OR GAN OR Data Augmentation OR Synthetic Data Generation OR Augmentation Techniques) AND (Retinal Diseases OR Retinal Degeneration OR Retinitis Pigmentosa OR Inherited retinal disease*) | Search by: “Topic”, no filters | 356 |
Scopus | (“Generative Model*” OR “Generative adversarial network” OR “GAN” OR “Data Augmentation” OR “Synthetic Data Generation” OR “Augmentation Techniques”) AND (“Retinal Diseases” OR “Retinal Degeneration” OR “Retinitis Pigmentosa” OR “Inherited retinal disease*”) | Search by: Article title, Abstract, keywords, no filters | 114 |
Appendix C
Results of Included Studies
Articles | Authors | Publication Date | Architectures | Evaluation Metrics | Diseases | Results |
---|---|---|---|---|---|---|
Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration | Burlina, Philippe M, et al. | 10 January 2019 | ProGAN | Retinal specialists | Age-related Macular Degeneration | Greater equality in the results. |
Geographic atrophy segmentation in SD-OCT images using synthesized fundus autofluorescence imaging | Wu, et al. | 27 October 2019 | RA-CGAN | PSNR, SSIM | Geographic Atrophy | It generated high-quality images. Some with better quality than the real images. |
CGAN-based Synthetic Medical Image Augmentation between Retinal Fundus Images and Vessel Segmented Images | HaoQi, et al. | 1 January 2020 | DCGAN | ROC/AUC, PR, Dice Coefficient, Sensitivity, Specificity | Retinal Vessel Analysis | It generates realistic images that maintain the statistical distribution of the original dataset. |
Retinal optical coherence tomography image classification with label smoothing generative adversarial network | He, et al. | 12 May 2020 | DCGAN, WGAN-GP | Precision, Sensitivity, Specificity, F1 | Age-related Macular Degeneration | Generated high quality images. Improved the performance of the classifier by using the data generated in training it. |
Study on the Method of Fundus Image Generation Based on Improved GAN | Guo, et al. | 12 June 2020 | WGAN, CGAN | SSIM, SD, IS, FID | Hard Exudate | They generate high-quality images with great diversity. |
Feasibility study to improve deep learning in OCT diagnosis of rare retinal diseases with few-shot classification | Yoo, et al. | 25 January 2021 | CycleGAN | Retinal specialists | Inherited Retinal Disease | They improved the performance of the models by using these data. |
Addressing Artificial Intelligence Bias in Retinal Diagnostics | Burlina, Philippe, et al. | 1 February 2021 | StyleGAN | ACC, ROC/AUC | Diabetic Retinopathy | Use of generated data to balance dataset; Improved the values of the metrics obtained. Greater equality in the results. |
RV-GAN: Segmenting Retinal Vascular Structure in Fundus Photographs Using a Novel Multi-scale Generative Adversarial Network | Kamran, et al. | 14 May 2021 | RV-GAN | ROC/AUC, ACC, Sensitivity, Specificity, F1, Mean-IoU, SSIM | Degenerative Retinal Diseases | Generate new data with higher quality. |
Deepfakes in Ophthalmology Applications and Realism of Synthetic Retinal Images from Generative Adversarial Networks | Chen, et al. | 29 October 2021 | Pix2Pix HD | ACC, Retinal specialists | Retinopathy of Prematurity | It generates realistic images. Experts found it difficult to distinguish between real and synthetic images. |
Generative Image Inpainting for Retinal Images using Generative Adversarial Networks | Magister, et al. | 4 November 2021 | WGAN | ACC, SNR, Evaluation of the Coherence of the Inpainted Image | Diabetic Retinopathy | Generated realistic images with great variety. |
RF-GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Adversarial Network | Chen, et al. | 10 November 2021 | CGAN | FID, SWD | Diabetic Retinopathy | Generated images with high fidelity and appearance. Improved the performance of Diabetic Retinopathy classifiers using the data generated. Improved the generalization capacity of the model as the generated data added diversity. |
Synthesizing realistic high-resolution retina image by style-based generative adversarial network and its utilization | Kim, et al. | 1 January 2022 | StyleGAN | Retinal specialists, SNR, ROC/AUC, Sensitivity, Specificity, ACC | Age-related Macular Degeneration | It generates highly realistic images. Experts found it difficult to distinguish between real and synthetic images. |
DR-GAN: Conditional Generative Adversarial Network for Fine-Grained Lesion Synthesis on Diabetic Retinopathy Images | Yi Zhou, et al. | 1 January 2022 | DR-GAN | Retinal specialists, FID, SWD, ACC | Diabetic Retinopathy | Images generated for data augmentation. They improved the performance of the models by using these data. |
An innovative medical image synthesis based on dual GAN deep neural networks for improved segmentation quality | Beji, Ahmed, et al. | 30 May 2022 | DCGAN | SSIM, MSE, PSNR, SIFT, Oriented FAST and Rotated BRIEF (ORB) | Retinal Diseases | Improved the values of the metrics obtained. |
A Dual-Discriminator Fourier Acquisitive GAN for Generating Retinal Optical Coherence Tomography Images | Tajmirriahi, Mahnoosh, et al. | 11 July 2022 | VAE | Euclidean Distance, FID, MS-SSIM | Diabetic Macular Edema | Generated more realistic, high-resolution images than other GAN architectures. Increased the dataset. Improved the efficiency of the classifier by using synthetic images to train it. |
LAC-GAN: Lesion attention conditional GAN for Ultra-widefield image synthesis | Lei, et al. | 11 November 2022 | DCGAN | Retinal specialists, AACC | Retinal Diseases | It generates images with reasonable detail. Using these images improved the model’s performance. It added diversity, helping to improve the model’s generalization performance. |
Generative adversarial network-based deep learning approach in classification of retinal conditions with optical coherence tomography images | Sun, et al. | 22 November 2022 | StyleGAN2-ADA | ROC)/AUC, Sensitivity, Specificity, Precision, ACC, F1, MCC | Retinal Diseases | Accuracy improved when the models were trained with synthetic data to balance the dataset. |
Synthesizing multi-frame high-resolution fluorescein angiography images from retinal fundus images using generative adversarial networks | Li, et al. | 21 February 2023 | CycleGAN | SSIM, PSNR, NCC | Retinal Diseases | They generate low-quality images. They synthesize these images to generate high-quality, high-resolution images. |
Fundus Image-Label Pairs Synthesis and Retinopathy Screening via GANs With Class-Imbalanced Semi-Supervised Learning | Xie, et al. | 27 March 2023 | CISSL-GAN | FID, Precision, Recall, Similarity | Retinopathy | Used to simultaneously improve the generation of class conditions of the fundus image and the classification performance in a typical scenario of insufficient and unbalanced labels. |
SynthEye: Investigating the Impact of Synthetic Data on Artificial Intelligence-assisted Gene Diagnosis of Inherited Retinal Disease | Yoga Advaith Veturi, et al. | 1 June 2023 | StyleGAN2 | Euclidean Distance, ROC/AUC, BRISQUE, LPIPS | Inherited Retinal Disease | It generates realistic images that have led to some of them being misjudged by clinical experts. The performance of the classifiers did not worsen with the data generated. Performance was like using only real data. |
Synthetic artificial intelligence using generative adversarial network for retinal imaging in detection of age-related macular degeneration | Wang, et al. | 22 June 2023 | StyleGAN2 | SSIM, ROC/AUC, k score, ACC, Sensitivity, Specificity | Age-related Macular Degeneration | Generates images with robust AMD lesions despite the Dataset having few images for initial training. They would easily confuse experts in distinguishing the generated data from the real thing. |
Generating OCT B-Scan DME images using optimized Generative Adversarial Networks (GANs) | Tripathi, et al. | 2 August 2023 | StyleGAN2 | FID, MSE | Diabetic Macular Edema | It generated realistic images, very similar to real ones. |
ROP-GAN: an image synthesis method for retinopathy of prematurity based on generative adversarial network. | Hou, et al. | 6 October 2023 | ROP-GAN | FID, IS, Classification Task with deep learning models | Retinopathy of Prematurity | Generates synthetic images of classes with little data. Generates fundus images in several stages. Improved the accuracy of the classifiers by adding the generated data to their training. |
Multi-Layer Preprocessing and U-Net with Residual Attention Block for Retinal Blood Vessel Segmentation | Alsayat, et al. | 30 October 2023 | LDM | PSNR, SSIM, FID, IS | Retinal Blood Vessel | It generates high-quality, diverse data to add to the training data set. |
Automated detection of crystalline retinopathy via fundus photography using multistage generative adversarial networks | Choi, et al. | 1 December 2023 | CycleGAN | ROC/AUC, Sensitivity, Specificity | Crystalline Retinopathy | It generates realistic images of the pathology. The accuracy of the model improved with the use of these synthetic data in training. |
Robust Deep Learning for Eye Fundus Images: Bridging Real and Synthetic Data for Enhancing Generalization | Oliveira, et al. | 1 January 2024 | StyleGAN2-ADA | FID, SSIM, PSNR, Retinal specialists | Age-related Macular Degeneration | Generates synthetic images similar to real ones with high resolution from a few images. Specialists found it difficult to distinguish the generated images from the real ones. Using the generated images with the real ones to train the models improved their accuracy. |
The Role of Fundus Imaging and GAN in Diabetic Retinopathy Classification using VGG19 | Kabilan, et al. | 1 January 2024 | DCGAN | SSIM, FID | Diabetic Retinopathy (DR) | Generates images to overcome data scarcity. |
Development of a generative deep learning model to improve epiretinal membrane detection in fundus photography | Choi, et al. | 1 January 2024 | StyleGAN2 | ROC/AUC, Sensitivity, Specificity, NPV, PPV | Epiretinal Membrane | It generates more realistic images (compared to other methods), with better quality. Using these data to train models improves the performance of disease detection. |
Transfer Learning and Interpretable Analysis-Based Quality Assessment of Synthetic Optical Coherence Tomography Images by CGAN Model for Retinal Diseases | Han, et al. | 13 January 2024 | CGAN | ACC, Precision, F1, Grad-CAM, Occlusion sensitivity, LIME | Retinal Diseases | They generate images that are very similar to real images. They are comparable to images of real retinal diseases, according to the metrics obtained. |
Digital ray: enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection | Lixue Liu, et al. | 5 June 2024 | CycleGAN | FID, ROC/AUC, KID | Retinopathy | Generated images with realistic features and better quality. Improved classifier accuracy with the use of synthetic data in training. |
Revolutionizing diabetic retinopathy diagnosis through advanced deep learning techniques: Harnessing the power of GAN model with transfer learning and the DiaGAN-CNN model | Shoaib, et al. | 29 August 2024 | DiaGAN | ROC/AUC | Diabetic Retinopathy | It generated realistic images, similar but not the same as the originals. It improved training results in metrics such as accuracy and precision. |
VSG-GAN: A high-fidelity image synthesis method with semantic manipulation in retinal fundus image | Liu, et al. | 3 September 2024 | VSG-GAN | KID, FID, IS, SSIM | Diabetic Retinopathy | Generates images with high fidelity. Improved the accuracy of classifiers when using real images and those generated during training. Expands the dataset through efficient data augmentation. |
References
- Ben-Yosef, T. Inherited Retinal Diseases. Int. J. Mol. Sci. 2022, 23, 13467. [Google Scholar] [CrossRef] [PubMed]
- Hanany, M.; Rivolta, C.; Sharon, D. Worldwide carrier frequency and genetic prevalence of autosomal recessive inherited retinal diseases. Proc. Natl. Acad. Sci. USA 2020, 117, 2710–2716. [Google Scholar] [CrossRef] [PubMed]
- Bessant, D. Molecular genetics and prospects for therapy of the inherited retinal dystrophies. Curr. Opin. Genet. Dev. 2001, 11, 307–316. [Google Scholar] [CrossRef] [PubMed]
- Miere, A.; Le Meur, T.; Bitton, K.; Pallone, C.; Semoun, O.; Capuano, V.; Capuano, V.; Colantuono, D.; Taibouni, K.; Chenoune, Y.; et al. Deep Learning-Based Classification of Inherited Retinal Diseases Using Fundus Autofluorescence. J. Clin. Med. 2020, 9, 3303. [Google Scholar] [CrossRef]
- Sahel, J.-A.; Marazova, K.; Audo, I. Clinical Characteristics and Current Therapies for Inherited Retinal Degener-ations. Cold Spring Harb. Perspect. Med. 2015, 5, a017111. [Google Scholar] [CrossRef]
- Schmidt-Erfurth, U.; Sadeghipour, A.; Gerendas, B.S.; Waldstein, S.M.; Bogunović, H. Artificial intelligence in retina. Prog. Retin. Eye Res. 2018, 67, 1–29. [Google Scholar] [CrossRef]
- Jiang, F.; Jiang, Y.; Zhi, H.; Dong, Y.; Li, H.; Ma, S.; Wang, Y.; Dong, Q.; Shen, H.; Wang, Y. Artificial intelligence in healthcare: Past, present and fu-ture. Stroke Vasc. Neurol. 2017, 2, 230–243. [Google Scholar] [CrossRef]
- Guo, C.; Yu, M.; Li, J. Prediction of different eye diseases based on fundus photography via deep transfer learning. J. Clin. Med. 2021, 10, 5481. [Google Scholar] [CrossRef]
- Liu, T.Y.A.; Ling, C.; Hahn, L.; Jones, C.K.; Boon, C.J.; Singh, M.S. Prediction of visual impairment in retinitis pigmentosa using deep learning and multimodal fundus images. Br. J. Ophthalmol. 2023, 107, 1484–1489. [Google Scholar] [CrossRef]
- Xie, Y.; Wan, Q.; Xie, H.; Xu, Y.; Wang, T.; Wang, S.; Lei, B. Fundus Image-Label Pairs Synthesis and Retinopathy Screening via GANs With Class-Imbalanced Semi-Supervised Learning. IEEE Trans. Med. Imaging 2023, 42, 2714–2725. [Google Scholar] [CrossRef]
- Perez, L.; Wang, J. The Effectiveness of Data Augmentation in Image Classification using Deep Learning. arXiv 2017, arXiv:1712.04621. [Google Scholar]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. arXiv 2014, arXiv:1406.2661. [Google Scholar] [CrossRef]
- Kupas, D.; Harangi, B. Solving the problem of imbalanced dataset with synthetic image generation for cell classification using deep learning. In Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Online, 1–5 November 2021; IEEE: Piscataway, NJ, USA, 2021; pp. 2981–2984. [Google Scholar] [CrossRef]
- Veturi, Y.A.; Woof, W.; Lazebnik, T.; Moghul, I.; Woodward-Court, P.; Wagner, S.K.; de Guimarães, T.A.C.; Varela, M.D.; Liefers, B.; Patel, P.J.; et al. SynthEye: Investigating the Impact of Synthetic Data on Artificial Intelligence-assisted Gene Diagnosis of Inherited Retinal Disease. Ophthalmol. Sci. 2023, 3, 100258. [Google Scholar] [CrossRef]
- Chen, Y.; Long, J.; Guo, J. RF-GANs: A Method to Synthesize Retinal Fundus Images Based on Generative Ad-versarial Network. Comput. Intell. Neurosci. 2021, 2021, 3812865. [Google Scholar] [CrossRef]
- Hou, N.; Shi, J.; Ding, X.; Nie, C.; Wang, C.; Wan, J. ROP-GAN: An image synthesis method for retinopathy of prematurity based on generative adversarial network. Phys. Med. Biol. 2023, 68, 205016. [Google Scholar] [CrossRef]
- Ruthotto, L.; Haber, E. An introduction to deep generative modeling. GAMM-Mitteilungen 2021, 44, e202100008. [Google Scholar] [CrossRef]
- Menti, E.; Bonaldi, L.; Ballerini, L.; Ruggeri, A.; Trucco, E. Automatic generation of synthetic retinal fundus images: Vascular network. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Berlin/Heidelberg, Germany, 2016; Volume 9968 LNCS, pp. 167–176. [Google Scholar] [CrossRef]
- Costa, P.; Galdran, A.; Meyer, M.I.; Abràmoff, M.D.; Niemeijer, M.; Mendonça, A.M.; Campilho, A. Towards Adversarial Retinal Image Synthesis. arXiv 2017, arXiv:1701.08974. [Google Scholar]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, n71. [Google Scholar] [CrossRef]
- PROSPERO. International Prospective Register of Systematic Reviews. Centre for Reviews and Dissemination, University of York. Available online: https://www.crd.york.ac.uk/prospero/ (accessed on 2 March 2025).
- Arsalan, M.; Baek, N.R.; Owais, M.; Mahmood, T.; Park, K.R. Deep learning-based detection of pigment signs for analysis and diagnosis of retinitis pigmentosa. Sensors 2020, 20, 3454. [Google Scholar] [CrossRef]
- Masumoto, H.; Tabuchi, H.; Nakakura, S.; Ohsugi, H.; Enno, H.; Ishitobi, N.; Ohsugi, E.; Mitamura, Y. Accuracy of a deep convolu-tional neural network in detection of retinitis pigmentosa on ultrawide-field images. PeerJ 2019, 7, e6900. [Google Scholar] [CrossRef]
- Mou, L.; Zhao, Y.; Fu, H.; Liu, Y.; Cheng, J.; Zheng, Y.; Su, P.; Yang, J.; Chen, L.; Frangi, A.F.; et al. CS2-Net: Deep learning segmentation of curvilinear structures in medical imaging. Med. Image Anal. 2021, 67, 101874. [Google Scholar] [CrossRef]
- JBI. JBI Manual for Evidence Synthesis; JBI: Adelaide, Australia, 2024. [Google Scholar] [CrossRef]
- Yoo, T.K.; Choi, J.Y.; Kim, H.K. Feasibility study to improve deep learning in OCT diagnosis of rare retinal diseases with few-shot classification. Med. Biol. Eng. Comput. 2021, 59, 401–415. [Google Scholar] [CrossRef] [PubMed]
- Oliveira, G.C.; Rosa, G.H.; Pedronette, D.C.G.; Papa, J.P.; Kumar, H.; Passos, L.A.; Kumar, D. Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization. Biomed. Signal Process. Control 2024, 94, 106263. [Google Scholar] [CrossRef]
- Wang, Z.; Lim, G.; Ng, W.Y.; Tan, T.E.; Lim, J.; Lim, S.H.; Foo, V.; Lim, J.; Sinisterra, L.G.; Zheng, F.; et al. Synthetic artificial intelligence using generative adversarial network for retinal imaging in detection of age-related macular degeneration. Front. Med. 2023, 10, 1184892. [Google Scholar] [CrossRef]
- Kim, M.; Kim, Y.N.; Jang, M.; Hwang, J.; Kim, H.K.; Yoon, S.C.; Kim, Y.J.; Kim, N. Synthesizing realistic high-resolution reti-na image by style-based generative adversarial network and its utilization. Sci. Rep. 2022, 12, 17307. [Google Scholar] [CrossRef]
- Burlina, P.M.; Joshi, N.; Pacheco, K.D.; Liu, T.Y.A.; Bressler, N.M. Assessment of Deep Generative Models for High-Resolution Synthetic Retinal Image Generation of Age-Related Macular Degeneration. JAMA Ophthalmol. 2019, 137, 258. [Google Scholar] [CrossRef]
- He, X.; Fang, L.; Rabbani, H.; Chen, X.; Liu, Z. Retinal optical coherence tomography image classification with label smoothing generative adversarial network. Neurocomputing 2020, 405, 37–47. [Google Scholar] [CrossRef]
- Wu, M.; Cai, X.; Chen, Q.; Ji, Z.; Niu, S.; Leng, T.; Rubin, D.L.; Park, H. Geographic atrophy segmentation in SD-OCT images using synthesized fundus autofluorescence imaging. Comput. Methods Programs Biomed. 2019, 182, 105101. [Google Scholar] [CrossRef]
- Choi, E.Y.; Han, S.H.; Ryu, I.H.; Kim, J.K.; Lee, I.S.; Han, E.; Kim, H.; Choi, J.Y.; Yoo, T.K. Automated detection of crystalline retinopathy via fundus photography using multistage generative adversarial networks. Biocybern. Biomed. Eng. 2023, 43, 725–735. [Google Scholar] [CrossRef]
- Kamran, S.A.; Hossain, K.F.; Tavakkoli, A.; Zuckerbrod, S.L.; Sanders, K.M.; Baker, S.A. RV-GAN: Segmenting Retinal Vascular Structure in Fundus Photographs using a Novel Multi-scale Generative Adversarial Network. arXiv 2021, arXiv:2101.00535. [Google Scholar] [CrossRef]
- Burlina, P.; Joshi, N.; Paul, W.; Pacheco, K.D.; Bressler, N.M. Addressing artificial intelligence bias in retinal diagnostics. Transl. Vis. Sci. Technol. 2021, 10, 1–13. [Google Scholar] [CrossRef] [PubMed]
- Magister, L.C.; Arandjelovic, O. Generative Image Inpainting for Retinal Images using Generative Adversarial Networks. In Proceedings of the 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Online, 1–5 November 2021; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2021; pp. 2835–2838. [Google Scholar] [CrossRef]
- Kabilan, C.; Madhesh Kumar, S.; Latha Selvi, G. The Role of Fundus Imaging and GAN in Diabetic Retinopathy Classification using VGG19. In Proceedings of the 3rd International Conference on Advances in Computing, Communication and Applied Informatics, ACCAI 2024, Chennai, India, 9–10 May 2024; Institute of Electrical and Electronics Engineers Inc.: Piscataway, NJ, USA, 2024. [Google Scholar] [CrossRef]
- Shoaib, M.R.; Emara, H.M.; Mubarak, A.S.; Omer, O.A.; Abd El-Samie, F.E.; Esmaiel, H. Revolutionizing dia-betic retinopathy diagnosis through advanced deep learning techniques: Harnessing the power of GAN model with transfer learning and the DiaGAN-CNN model. Biomed. Signal Process. Control. 2025, 99, 106790. [Google Scholar] [CrossRef]
- Liu, J.; Xu, S.; He, P.; Wu, S.; Luo, X.; Deng, Y.; Huang, H. VSG-GAN: A High Fidelity Image Synthesis Method with Seman-tic Manipulation in Retinal Fundus Image. Biophys. J. 2024, 123, 2815–2829. [Google Scholar] [CrossRef]
- Zhou, Y.; Wang, B.; He, X.; Cui, S.; Shao, L. DR-GAN: Conditional Generative Adversarial Network for Fi-ne-Grained Lesion Synthesis on Diabetic Retinopathy Images. IEEE J. Biomed. Health Inform. 2022, 26, 56–66. [Google Scholar] [CrossRef]
- Tajmirriahi, M.; Kafieh, R.; Amini, Z.; Lakshminarayanan, V. A Dual-Discriminator Fourier Acquisitive GAN for Generating Retinal Optical Coherence Tomography Images. IEEE Trans. Instrum. Meas. 2022, 71, 1–8. [Google Scholar] [CrossRef]
- Tripathi, A.; Kumar, P.; Mayya, V.; Tulsani, A. Generating OCT B-Scan DME images using optimized Generative Adversarial Networks (GANs). Heliyon 2023, 9, e18773. [Google Scholar] [CrossRef]
- Choi, J.Y.; Ryu, I.H.; Kim, J.K.; Lee, I.S.; Yoo, T.K. Development of a generative deep learning model to improve epiretinal membrane detection in fundus photography. BMC Med. Inform. Decis. Mak. 2024, 24, 25. [Google Scholar] [CrossRef]
- Guo, J.; Pang, Z.; Yang, F.; Shen, J.; Zhang, J. Study on the Method of Fundus Image Generation Based on Im-proved GAN. Math. Probl. Eng. 2020, 2020, 6309596. [Google Scholar] [CrossRef]
- Alsayat, A.; Elmezain, M.; Alanazi, S.; Alruily, M.; Mostafa, A.M.; Said, W. Multi-Layer Preprocessing and U-Net with Residual Attention Block for Retinal Blood Vessel Segmentation. Diagnostics 2023, 13, 3364. [Google Scholar] [CrossRef]
- HaoQi, G.; Ogawara, K. CGAN-based Synthetic Medical Image Augmentation between Retinal Fundus Images and Vessel Segmented Images. In Proceedings of the 2020 5th International Conference on Control and Robotics Engineering (ICCRE), Osaka, Japan, 24–26 April 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 218–223. [Google Scholar] [CrossRef]
- Liu, L.; Hong, J.; Wu, Y.; Liu, S.; Wang, K.; Li, M.; Zhao, L.; Liu, Z.; Li, L.; Cui, T.; et al. Digital ray: Enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection. Br. J. Ophthalmol. 2024, 108, 1423–1429. [Google Scholar] [CrossRef]
- Chen, J.S.; Coyner, A.S.; Chan, R.V.P.; Hartnett, M.E.; Moshfeghi, D.M.; Owen, L.A.; Kalpathy-Cramer, J.; Chiang, M.F.; Campbell, J.P. Deepfakes in Oph-thalmology: Applications and Realism of Synthetic Retinal Images from Generative Adversarial Networks. Ophthalmol. Sci. 2021, 1, 100079. [Google Scholar] [CrossRef] [PubMed]
- Beji, A.; Blaiech, A.G.; Said, M.; Abdallah, A.B.; Bedoui, M.H. An innovative medical image synthesis based on dual GAN deep neural networks for improved segmentation quality. Appl. Intell. 2023, 53, 3381–3397. [Google Scholar] [CrossRef]
- Sun, L.C.; Pao, S.I.; Huang, K.H.; Wei, C.Y.; Lin, K.F.; Chen, P.N. Generative adversarial network-based deep learning approach in classification of retinal conditions with optical coherence tomography images. Graefe’s Arch. Clin. Exp. Ophthalmol. 2023, 261, 1399–1412. [Google Scholar] [CrossRef] [PubMed]
- Lei, H.; Tian, Z.; Xie, H.; Zhao, B.; Zeng, X.; Cao, J.; Liu, W.; Wang, J.; Zhang, G.; Wang, S.; et al. LAC-GAN: Lesion attention conditional GAN for Ul-tra-widefield image synthesis. Neural Netw. 2023, 158, 89–98. [Google Scholar] [CrossRef]
- Li, P.; He, Y.; Wang, P.; Wang, J.; Shi, G.; Chen, Y. Synthesizing multi-frame high-resolution fluorescein angi-ography images from retinal fundus images using generative adversarial networks. BioMedical Eng. Online 2023, 22, 16. [Google Scholar] [CrossRef]
- Han, K.; Yu, Y.; Lu, T. Transfer Learning and Interpretable Analysis-Based Quality Assessment of Synthetic Op-tical Coherence Tomography Images by CGAN Model for Retinal Diseases. Processes 2024, 12, 182. [Google Scholar] [CrossRef]
- Jayasumana, S.; Ramalingam, S.; Veit, A.; Glasner, D.; Chakrabarti, A.; Kumar, S. Rethinking FID: Towards a Better Evaluation Metric for Image Generation. arXiv 2023, arXiv:2401.09603. [Google Scholar]
- Nilsson, J.; Akenine-Möller, T. Understanding SSIM. arXiv 2020, arXiv:2006.13846. [Google Scholar]
Identifier | Inclusion Criteria |
---|---|
PubMed | It is focused on biomedicine and health, which meets the context of inherited retinal diseases. |
IEEE Xplore | It is a database specializing in technology and engineering, covering the area of artificial intelligence. |
Web of Science and Scopus | They are databases with a multidisciplinary scope and scientific impact, which can encompass studies covering both areas. |
Identifier | Inclusion Criteria |
---|---|
IC1 | Journal and conference articles |
IC2 | Articles dealing with retinal diseases |
IC3 | Using generative models for data augmentation |
IC4 | Articles in English language |
IC5 | Articles published after 2019 |
Identifier | Inclusion Criteria |
---|---|
EC1 | Articles only use traditional data augmentation techniques |
EC2 | Articles use generative models but not for data augmentation |
EC3 | Performing data augmentation but not in ophthalmic diseases |
EC4 | Articles that are not possible to obtain |
EC5 | Articles do not cover the topic |
Year | Publications | Percentage (%) |
---|---|---|
2019 | 2 | 6.25 |
2020 | 3 | 9.38 |
2021 | 6 | 18.75 |
2022 | 6 | 18.75 |
2023 | 8 | 25.00 |
2024 | 7 | 21.88 |
Total | 32 | 100 |
Architectures | Acronym | Frequency | Percentage (%) |
---|---|---|---|
Deep Convolutional Generative Adversarial Network | DCGAN | 5 | 13.89 |
Style-Based Generator Architecture for Generative Adversarial Networks 2 | StyleGAN2 | 4 | 11.11 |
Cycle-Consistent Generative Adversarial Network | CycleGAN | 4 | 11.11 |
Conditional Generative Adversarial Network | CGAN | 3 | 8.33 |
Wasserstein Generative Adversarial Network | WGAN | 3 | 8.33 |
Style-Based Generator Architecture for Generative Adversarial Networks | StyleGAN | 2 | 5.56 |
Style-Based Generative Adversarial Network 2 with Adaptive Discriminator Augmentation | StyleGAN-ADA | 2 | 5.56 |
Auxiliary Classifier Generative Adversarial Network | ACGAN | 1 | 2.78 |
Class-imbalanced semi-supervised learning—Generative Adversarial Network | CISSL-GAN | 1 | 2.78 |
Dimension Augmenter Generative Adversarial Network | DiAGAN | 1 | 2.78 |
Diabetic Retinopathy Generative Adversarial Network | DR-GAN | 1 | 2.78 |
Latent Diffusion Models | LDM | 1 | 2.78 |
High-Definition Image-to-Image Translation with Conditional Generative Adversarial Networks | Pix2Pix HD | 1 | 2.78 |
Progressive Growing of Generative Adversarial Network | ProGAN | 1 | 2.78 |
Residual Attention Conditional Generative Adversarial Network | RA-CGAN | 1 | 2.78 |
Retinopathy of Prematurity Generative Adversarial Network | ROP-GAN | 1 | 2.78 |
RV-Generative Adversarial Network | RV-GAN | 1 | 2.78 |
Variational Autoencoder | VAE | 1 | 2.78 |
vessel and style guided generative adversarial network | VSG-GAN | 1 | 2.78 |
Wasserstein Generative Adversarial Network with Gradient Penalty | WGAN-GP | 1 | 2.78 |
Total | 36 | 100 |
Metrics | Acronym | Frequency | Percentage (%) |
---|---|---|---|
Fréchet Inception Distance | FID | 12 | 11.21 |
Receiver Operating Characteristic/Area Under the Curve | ROC/AUC | 11 | 10.28 |
Accuracy | ACC | 10 | 9.35 |
Structural Similarity Index Measure | SSIM | 10 | 9.35 |
Retinal Specialists | - | 8 | 7.48 |
Sensitivity, Specificity | - | 8 | 7.48 |
Peak Signal-to-Noise Ratio | PSNR | 5 | 4.67 |
F1-Score | F1 | 4 | 3.74 |
Inception Score | IS | 4 | 3.74 |
Precision | - | 4 | 3.74 |
Euclidean Distance | - | 2 | 1.87 |
Kernel Inception Distance | KID | 2 | 1.87 |
Mean Squared Error | MSE | 2 | 1.87 |
Signal-to-Noise Ratios | SNR | 2 | 1.87 |
Sliced Wasserstein Distance | SWD | 2 | 1.87 |
Precision and Recall Curve | PR | 2 | 1.87 |
Blind/Referenceless Image Spatial Quality Evaluator | BRISQUE | 1 | 0.93 |
Classification Task with Deep Learning Models | - | 1 | 0.93 |
Cohen’s Kappa Score | K Score | 1 | 0.93 |
Gradient-weighted Class Activation Mapping | Grad-CAM | 1 | 0.93 |
Learned Perceptual Image Patch Similarity | LPIPS | 1 | 0.93 |
Local Interpretable Model-Agnostic Explanations | LIME | 1 | 0.93 |
Mean Intersection Over Union | Mean-IoU | 1 | 0.93 |
Multiscale Structural Similarity Index Measure | MS-SSIM | 1 | 0.93 |
Normalized Cross-Correlation | NCC | 1 | 0.93 |
Occlusion Sensitivity | - | 1 | 0.93 |
Recall | - | 1 | 0.93 |
Scale Invariant Feature Transform | SIFT | 1 | 0.93 |
Sharpness Difference | SD | 1 | 0.93 |
Similarity | - | 1 | 0.93 |
Dice Coefficient | - | 1 | 0.93 |
Negative Predictive Value | NPV | 1 | 0.93 |
Positive Predictive Value | PPV | 1 | 0.93 |
Oriented FAST and Rotated BRIEF | ORB | 1 | 0.93 |
Oriented FAST and Rotated BRIEF | ORB | 1 | 0.93 |
Matthews Correlation Coefficient | MCC | 1 | 0.93 |
Evaluation of the Coherence of the Inpainted Image | - | 1 | 0.93 |
Total | 107 | 100 |
Study Context | Frequency | Percentage (%) |
---|---|---|
Diabetic Retinopathy | 9 | 28.13 |
Age-Related Macular Degeneration | 6 | 18.75 |
Retinal Diseases | 5 | 15.63 |
Inherited Retinal Disease | 2 | 6.25 |
Retinopathy | 2 | 6.25 |
Retinopathy of Prematurity | 2 | 6.25 |
Retinal Blood Vessel | 2 | 6.25 |
Crystalline Retinopathy | 1 | 3.13 |
Degenerative Retinal Diseases | 1 | 3.13 |
Epiretinal Membrane | 1 | 3.13 |
Hard Exudate | 1 | 3.13 |
Total | 32 | 100 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Machado, J.; Marta, A.; Mestre, P.; Beirão, J.M.; Cunha, A. Data Augmentation with Generative Methods for Inherited Retinal Diseases: A Systematic Review. Appl. Sci. 2025, 15, 3084. https://doi.org/10.3390/app15063084
Machado J, Marta A, Mestre P, Beirão JM, Cunha A. Data Augmentation with Generative Methods for Inherited Retinal Diseases: A Systematic Review. Applied Sciences. 2025; 15(6):3084. https://doi.org/10.3390/app15063084
Chicago/Turabian StyleMachado, Jorge, Ana Marta, Pedro Mestre, João Melo Beirão, and António Cunha. 2025. "Data Augmentation with Generative Methods for Inherited Retinal Diseases: A Systematic Review" Applied Sciences 15, no. 6: 3084. https://doi.org/10.3390/app15063084
APA StyleMachado, J., Marta, A., Mestre, P., Beirão, J. M., & Cunha, A. (2025). Data Augmentation with Generative Methods for Inherited Retinal Diseases: A Systematic Review. Applied Sciences, 15(6), 3084. https://doi.org/10.3390/app15063084