Abstract
Skin cancer is one of the primary causes of death in the world. Timely diagnosis of skin cancer can reduce the number of deaths. Skin cancer can be diagnosed early using deep learning-based systems. The performance of deep learning-based systems suffers from overfitting if we don’t have enough data to train them. Acquiring a large amount of skin lesion images for training a deep learning-based system is a difficult task. Overfitting can be avoided using data augmentation. Generative adversarial networks (GANs) are very popular in skin lesion tasks because of their ability to generate high-quality synthetic skin lesion images. GANs are used for the classification and segmentation of skin-lesion images. We review the most relevant papers discussing the use of GANs for augmenting skin lesion datasets in this work. We gave an overview of the most commonly used GAN architectures in skin lesion analysis.
Similar content being viewed by others
Data Availability
Not Applicable.
References
Abdelhalim ISA, Mohamed MF, Mahdy YB (2021) Data augmentation for skin lesion using self-attention based progressive generative adversarial network. Expert Syst Appl 165:113922
Abhishek K, Hamarneh G (2019) Mask2lesion: mask-constrained adversarial skin lesion image synthesis. In: International workshop on simulation and synthesis in medical imaging, pp 71–80. Springer
Adegun A, Viriri S (2021) Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif Intell Rev 54(2):811–841
Annala L, Neittaanmäki N, Paoli J, Zaar O, Pölönen I (2020) Generating hyperspectral skin cancer imagery using generative adversarial neural network. In: 2020 42nd Annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp 1600–1603. IEEE
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, pp 214–223. PMLR
Augusto D, Oliveira B (2020) Controllable skin lesion synthesis using texture patches, Bézier Curves and Conditional GANs. In: 2020 IEEE 17Th international symposium on biomedical imaging (ISBI), pp 1798–1802. IEEE
Ballerini L, Fisher RB, Aldridge B, Rees J (2013) A color and texture based hierarchical k-NN approach to the classification of non-melanoma skin lesions. In: Color medical image analysis, pp 63–86. Springer
Baur C, Albarqouni S, Navab N (2018a) MelanoGANs: high resolution skin lesion synthesis with GANs. arXiv:1804.04338
Baur C, Albarqouni S, Navab N (2018b) Generating highly realistic images of skin lesions with GANs. In: OR 2.0 Context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp. 260–267. Springer
Bi L, Feng D, Fulham M, Kim J (2019) Improving skin lesion segmentation via stacked adversarial learning. In: 2019 IEEE 16Th International symposium on biomedical imaging (ISBI 2019), pp 1100–1103. IEEE
Bisla D, Choromanska A, Berman RS, Stein JA, Polsky D (2019) Towards automated melanoma detection with deep learning: data purification and augmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0
Bissoto A, Avila S (2020) Improving Skin Lesion Analysis with Generative Adversarial Networks. In: Anais Estendidos do XXXIII conference on graphics, patterns and images, pp 70–76. SBC
Bissoto A, Perez Fábio, Valle E, Avila S (2018) Skin lesion synthesis with generative adversarial networks. In: OR 2.0 Context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp 294–302. Springer
Bissoto A, Valle E, Avila S (2021) GAN-Based Data Augmentation and Anonymization for Skin-Lesion Analysis: A critical review. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1847–1856
Chen Liang-Chieh, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818
Chi Y, Bi L, Kim J, Feng D, Kumar A (2018) Controlled synthesis of dermoscopic images via a new color labeled generative style transfer network to enhance melanoma segmentation. In: 2018 40Th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 2591–2594. IEEE
Codella Noel CF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H et al (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 168–172. IEEE
Codella Noel CF, Nguyen Q -B, Pankanti S, Gutman DA, Helba B, Halpern AC, Smith JR (2017) Deep learning ensembles for melanoma recognition in dermoscopy images. IBM J Res Dev 61(4/5):5–1
Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo Aadi , Liopyris K, Marchetti M et al (2019) Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). arXiv:1902.03368
Combalia M, Codella NCF, Rotemberg V, Helba B, Vilaplana V, Reiter O, Carrera C, Barreiro A, Halpern AC, Puig S et al (2019) BCN20000: dermoscopic lesions in the wild. arXiv:1908.02288
Denton E, Chintala S, Szlam A, Fergus R (2015) Deep generative image models using a laplacian pyramid of adversarial networks. arXiv:1506.05751
Dildar M, Akram S, Irfan M, Khan HU, Ramzan M, Mahmood AR, Alsaiari SA, Saeed AHM, Alraddadi MO, Mahnashi MH (2021) Skin cancer detection: a Review using deep learning techniques. Int J Environ Res Public Health 18(10):5479
Ding S, Zheng J, Liu Z, Zheng Y, Chen Y, Xiaomin X, Jia L, Xie J (2224) High-resolution dermoscopy image synthesis with conditional generative adversarial networks. Biomed Signal Process Control 64(10):2021
Díaz IG (2017) Incorporating the knowledge of dermatologists to convolutional neural networks for the diagnosis of skin lesions. arXiv:1703.01976
Emre Celebi M, Wen QUAN, Iyatomi HITOSHI, Shimizu KOUHEI, Zhou H, Schaefer G (2015) A state-of-the-art survey on lesion border detection in dermoscopy images. Dermoscopy Image Nalysis 10:97–129
Erhan D, Courville A, Bengio Y, Vincent P (2010) Why does unsupervised pre-training help deep learning?. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 201–208. JMLR Workshop and Conference Proceedings
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Sebastian T (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118
Fan H, Xie F, Li Y, Jiang Z, Liu J (2017) Automatic segmentation of dermoscopy images using saliency combined with Otsu threshold. Comput Biol Med 85:75–85
Fornaciali M, Carvalho M, Bittencourt FV, Avila S, Valle E (2016) Towards automated melanoma screening: Proper computer vision & reliable results. arXiv:1604.04024
Fossen-Romsaas S, Storm-Johannessen A, Lundervold AS (2020) Synthesizing skin lesion images using cycleGANs–a case study
Furger F, Amruthalingam L, Navarini A, Pouly M (2020) Applications of generative adversarial networks to dermatologic imaging. In: IAPR Workshop on artificial neural networks in pattern recognition, pp 187–199. Springer
Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style. arXiv:1508.06576
Ghorbani A, Natarajan V, Coz D, Liu Y (2020) Dermgan: synthetic generation of clinical skin images with pathology. In: Machine learning for health workshop, pp. 155–170. PMLR
Goodfellow I, Pouget-Abadie J, Mirza M, Bing X u, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Advances in Neural Information Processing Systems, 27
Gutman D, Codella NCF, Celebi E, Helba B, Marchetti M, Mishra N, Halpern A (2016) Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). arXiv:1605.01397
Hasan MK, Elahi MTE, Alam MA, Jawad MT (2021) Dermoexpert skin lesion classification using a hybrid convolutional neural network through segmentation, transfer learning, and augmentation medRxiv
Hasan MdK, Dahal L, Samarakoon PN, Tushar FI, Martí R (2020) DSNet: automatic dermoscopic skin lesion segmentation. Comput Biol Med 120:103738
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) GANS trained by a two time-scale update rule converge to a local nash equilibrium. Adv Neural Inf Process Syst, 30
Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708
Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456. PMLR
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134
Izadi S, Mirikharaji Z, Kawahara J, Hamarneh G (2018) Generative adversarial networks to segment skin lesions. In: 2018 IEEE 15Th international symposium on biomedical imaging (ISBI 2018), pp 881–884. IEEE
Jalalian A, Mashohor S, Mahmud R, Karasfi B, Iqbal B, Saripan M, Ramli ARB (2017) Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection. EXCLI J 16:113
Jie H, Li S, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141
Jun F u, Liu J, Tian H, Li Y, Bao Y, Fang Z, Hanqing L (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3146–3154
Kang Y, Gao S, Roth RE (2019) Transferring multiscale map styles using generative adversarial networks. International Journal of Cartography 5 (2-3):115–141
Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of GANs for improved quality, stability, and variation. arXiv:1710.10196
Kawahara J, Daneshvar S, Argenziano G, Hamarneh G (2018) Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J Biomed Health Inf 23(2):538–546
Kittler H, Pehamberger H, Wolff K, Binder MJTIO (2002) Diagnostic accuracy of dermoscopy. Lancet Oncol 3(3):159–165
Korotkov K, Garcia R (2012) Computerized analysis of pigmented skin lesions: a review. Artificial Intell Med 56(2):69–90
Lazo C (2021) Segmentation of skin lesions and their attributes using generative adversarial networks. arXiv:2102.00169
Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690
Lei B, Xia Z, Jiang F, Jiang X, Ge Z, Yanwu X, Qin J, Chen S, Wang T, Wang S (1716) Skin lesion segmentation via generative adversarial networks with dual discriminators. Med Image Anal 64(10):2020
Lucieri A, Dengel A, Ahmed S (2021) Deep learning based decision support for medicine–a case study on skin cancer diagnosis. arXiv:2103.05112
Matsunaga K, Hamada A, Minagawa A, Koga H (2017) Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. arXiv:1703.03108
Mendonça T, Ferreira PM, Marques JS, Marcal ARS, Rozeira J (2013) PH 2-A dermoscopic image database for research and benchmarking. In: 2013 35Th Annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 5437–5440. IEEE
Menegola A, Fornaciali M, Pires R, Bittencourt Flávia V., Avila S, Valle E (2017a) Knowledge transfer for melanoma screening with deep learning. In: 2017 IEEE 14Th International symposium on biomedical imaging (ISBI 2017), pp 297–300. IEEE
Menegola A, Tavares J, Fornaciali M, Li LT, Avila S, Valle E (2017b) RECOD titans at ISIC challenge 2017. arXiv:1703.04819
Mikołajczyk A, Grochowski M (2018) Data augmentation for improving deep learning in image classification problem. In: 2018 International interdisciplinary phd workshop (IIPhDW), pp 117–122. IEEE
Mikołajczyk A, Grochowski M (2019) Style transfer-based image synthesis as an efficient regularization technique in deep learning. In: 2019 24Th International conference on methods and models in automation and robotics (MMAR), pp 42–47. IEEE
Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv:1411.1784
Mishra NK, Celebi ME M (2016) An overview of melanoma detection in dermoscopy images using image processing and machine learning. arXiv:1601.07843
Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier GANs
Osuala R, Kushibar K, Garrucho L, Linardos A, Szafranowska Z, Klein S, Glocker B, Diaz O, Lekadir K (2021) A review of generative adversarial networks in cancer imaging new applications. New Solutions. arXiv:2107.09543
Pacheco Andre GC, Krohling RA (2019) Recent advances in deep learning applied to skin cancer detection. arXiv:1912.03280
Palatucci MM, Pomerleau DA, Hinton G, Mitchell T (2009) Zero-shot learning with semantic output codes
Park T, Liu M-Y, Wang T-C, Zhu J-Y (2019) Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2337–2346
Perez F, Vasconcelos C, Avila S, Valle E (2018) Data augmentation for skin lesion analysis. In: OR 2.0 Context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp. 303–311. Springer
Pham Tri-Cong, Luong Chi-Mai, Visani M, Hoang Van-Dung (2018) Deep CNN and data augmentation for skin lesion classification. In: Asian Conference on Intelligent Information and Database Systems, pp 573–582. Springer
Pollastri F, Bolelli F, Palacios RP, Grana C (2018) Improving skin lesion segmentation with generative adversarial networks. In: 2018 IEEE 31St International symposium on computer-based medical systems (CBMS), pp 442–443. IEEE
Pollastri F, Bolelli F, Paredes R, Grana C (2020) Augmenting data with GANs to segment melanoma skin lesions. Multimed Tools Appl 79 (21):15575–15592
Qasim AB, Ezhov I, Shit S, Schoppe O, Paetzold JC, Sekuboyina A, Kofler F, Lipkova J, Li H, Menze B (2020) Red-GAN: attacking class imbalance via conditioned generation. Yet another medical imaging perspective. In: Medical imaging with deep learning, pp 655–668. PMLR
Qin Z, Liu Z, Zhu P, Xue Y (2020) A GAN-based image synthesis method for skin lesion classification. Comput Methods Prog Biomed 195:105568
Radford A, Luke M, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434
Rashid H, Asjid Tanveer M, Khan HA (2019) Skin lesion classification using GAN based data augmentation. In: 2019 41St annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 916–919. IEEE
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp 234–241. Springer
Rotemberg V, Kurtansky N, Betz-Stablein B, Caffery L, Chousakos E, Codella N, Combalia M, Dusza S, Guitera P, Gutman D et al (2021) A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scient Data 8(1):1–8
Sarker MMK, Rashwan HA, Akram F, Singh VK, Banu SF, Chowdhury FUH, Choudhury KA, Chambon S, Radeva P, Puig D et al (2021) SLSNEt: skin lesion segmentation using a lightweight generative adversarial network. Expert Systems with Applications, 115433
Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. Journal of Big Data 6(1):1–48
Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958
Trivedi A, Muppalla S, Pathak S, Mobasher A, Janowski P, Dodhia R, Ferres JML (2020) Improving Lesion Detection by exploring bias on Skin Lesion dataset. arXiv:2010.01485
Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scient Data 5(1):1–9
Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8798–8807
Wei J, Suriawinata A, Vaickus L, Ren B, Liu X, Wei J, Hassanpour S (2019) Generative image translation for data augmentation in colorectal histopathology images. Proc Mach Learn Res 116:10
Wenli T u, Liu X, Wei H u, Pan Z (2019) Dense-residual network with adversarial learning for skin lesion segmentation. IEEE Access 7:77037–77051
Xian Y, Lampert CH, Schiele B, Akata Z (2018) Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell 41(9):2251–2265
Xue Y, Tao X u, Huang X (2018) Adversarial learning with multi-scale loss for skin lesion segmentation. In: 2018 IEEE 15Th International symposium on biomedical imaging (ISBI 2018), pp 859–863. IEEE
Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890
Zhao C, Shuai R, Li M, Liu W, Die H, Menglin W (2021) Dermoscopy image classification based on StyleGAN and DenseNet201. IEEE Access 9:8659–8679
Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using Cycle-Consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232
Zunair H, Hamza AB (2020) Melanoma detection using adversarial training and deep transfer learning. Physics in Medicine & Biology 65(13):135005
Acknowledgements
I want to acknowledge Dr. Hazrat Ali’s valuable input during the review process.
Funding
Not Applicable
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interests
The authors declare that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
A list of abbreviations used in this paper is given in Table 6.
1.1 Datasets used for skin lesion analysis
GANs were used for synthesizing lesion images due to the difficulties in collecting large datasets for skin lesion analysis. Skin lesion analysis is mostly performed using open-source datasets. Some of the available open-source datasets are presented in Table 7.
1.2 Github repositories
Codes are available for synthesizing skin lesion images using different architectures of GANs. Github repositories for implementing some of the GAN architectures discussed in Sections 3 and 4 are presented in Table 8.
A comparison of the computational cost of different synthesis networks given in Ding et al., Sarker et al. is presented in Table 9. Table 9 compares the number of parameters in millions.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gilani, S.Q., Marques, O. Skin lesion analysis using generative adversarial networks: a review. Multimed Tools Appl 82, 30065–30106 (2023). https://doi.org/10.1007/s11042-022-14267-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-14267-z