Skip to main content
Log in

Skin lesion analysis using generative adversarial networks: a review

  • Track 2: Medical Applications of Multimedia
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Skin cancer is one of the primary causes of death in the world. Timely diagnosis of skin cancer can reduce the number of deaths. Skin cancer can be diagnosed early using deep learning-based systems. The performance of deep learning-based systems suffers from overfitting if we don’t have enough data to train them. Acquiring a large amount of skin lesion images for training a deep learning-based system is a difficult task. Overfitting can be avoided using data augmentation. Generative adversarial networks (GANs) are very popular in skin lesion tasks because of their ability to generate high-quality synthetic skin lesion images. GANs are used for the classification and segmentation of skin-lesion images. We review the most relevant papers discussing the use of GANs for augmenting skin lesion datasets in this work. We gave an overview of the most commonly used GAN architectures in skin lesion analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30
Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38
Fig. 39
Fig. 40
Fig. 41
Fig. 42
Fig. 43
Fig. 44

Similar content being viewed by others

Data Availability

Not Applicable.

References

  1. Abdelhalim ISA, Mohamed MF, Mahdy YB (2021) Data augmentation for skin lesion using self-attention based progressive generative adversarial network. Expert Syst Appl 165:113922

    Article  Google Scholar 

  2. Abhishek K, Hamarneh G (2019) Mask2lesion: mask-constrained adversarial skin lesion image synthesis. In: International workshop on simulation and synthesis in medical imaging, pp 71–80. Springer

  3. Adegun A, Viriri S (2021) Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif Intell Rev 54(2):811–841

    Article  Google Scholar 

  4. Annala L, Neittaanmäki N, Paoli J, Zaar O, Pölönen I (2020) Generating hyperspectral skin cancer imagery using generative adversarial neural network. In: 2020 42nd Annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp 1600–1603. IEEE

  5. Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: International conference on machine learning, pp 214–223. PMLR

  6. Augusto D, Oliveira B (2020) Controllable skin lesion synthesis using texture patches, Bézier Curves and Conditional GANs. In: 2020 IEEE 17Th international symposium on biomedical imaging (ISBI), pp 1798–1802. IEEE

  7. Ballerini L, Fisher RB, Aldridge B, Rees J (2013) A color and texture based hierarchical k-NN approach to the classification of non-melanoma skin lesions. In: Color medical image analysis, pp 63–86. Springer

  8. Baur C, Albarqouni S, Navab N (2018a) MelanoGANs: high resolution skin lesion synthesis with GANs. arXiv:1804.04338

  9. Baur C, Albarqouni S, Navab N (2018b) Generating highly realistic images of skin lesions with GANs. In: OR 2.0 Context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp. 260–267. Springer

  10. Bi L, Feng D, Fulham M, Kim J (2019) Improving skin lesion segmentation via stacked adversarial learning. In: 2019 IEEE 16Th International symposium on biomedical imaging (ISBI 2019), pp 1100–1103. IEEE

  11. Bisla D, Choromanska A, Berman RS, Stein JA, Polsky D (2019) Towards automated melanoma detection with deep learning: data purification and augmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 0–0

  12. Bissoto A, Avila S (2020) Improving Skin Lesion Analysis with Generative Adversarial Networks. In: Anais Estendidos do XXXIII conference on graphics, patterns and images, pp 70–76. SBC

  13. Bissoto A, Perez Fábio, Valle E, Avila S (2018) Skin lesion synthesis with generative adversarial networks. In: OR 2.0 Context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp 294–302. Springer

  14. Bissoto A, Valle E, Avila S (2021) GAN-Based Data Augmentation and Anonymization for Skin-Lesion Analysis: A critical review. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 1847–1856

  15. Chen Liang-Chieh, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  16. Chi Y, Bi L, Kim J, Feng D, Kumar A (2018) Controlled synthesis of dermoscopic images via a new color labeled generative style transfer network to enhance melanoma segmentation. In: 2018 40Th annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 2591–2594. IEEE

  17. Codella Noel CF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H et al (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 168–172. IEEE

  18. Codella Noel CF, Nguyen Q -B, Pankanti S, Gutman DA, Helba B, Halpern AC, Smith JR (2017) Deep learning ensembles for melanoma recognition in dermoscopy images. IBM J Res Dev 61(4/5):5–1

    Article  Google Scholar 

  19. Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo Aadi , Liopyris K, Marchetti M et al (2019) Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). arXiv:1902.03368

  20. Combalia M, Codella NCF, Rotemberg V, Helba B, Vilaplana V, Reiter O, Carrera C, Barreiro A, Halpern AC, Puig S et al (2019) BCN20000: dermoscopic lesions in the wild. arXiv:1908.02288

  21. Denton E, Chintala S, Szlam A, Fergus R (2015) Deep generative image models using a laplacian pyramid of adversarial networks. arXiv:1506.05751

  22. Dildar M, Akram S, Irfan M, Khan HU, Ramzan M, Mahmood AR, Alsaiari SA, Saeed AHM, Alraddadi MO, Mahnashi MH (2021) Skin cancer detection: a Review using deep learning techniques. Int J Environ Res Public Health 18(10):5479

    Article  Google Scholar 

  23. Ding S, Zheng J, Liu Z, Zheng Y, Chen Y, Xiaomin X, Jia L, Xie J (2224) High-resolution dermoscopy image synthesis with conditional generative adversarial networks. Biomed Signal Process Control 64(10):2021

    Google Scholar 

  24. Díaz IG (2017) Incorporating the knowledge of dermatologists to convolutional neural networks for the diagnosis of skin lesions. arXiv:1703.01976

  25. Emre Celebi M, Wen QUAN, Iyatomi HITOSHI, Shimizu KOUHEI, Zhou H, Schaefer G (2015) A state-of-the-art survey on lesion border detection in dermoscopy images. Dermoscopy Image Nalysis 10:97–129

    Google Scholar 

  26. Erhan D, Courville A, Bengio Y, Vincent P (2010) Why does unsupervised pre-training help deep learning?. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp 201–208. JMLR Workshop and Conference Proceedings

  27. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Sebastian T (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118

    Article  Google Scholar 

  28. Fan H, Xie F, Li Y, Jiang Z, Liu J (2017) Automatic segmentation of dermoscopy images using saliency combined with Otsu threshold. Comput Biol Med 85:75–85

    Article  Google Scholar 

  29. Fornaciali M, Carvalho M, Bittencourt FV, Avila S, Valle E (2016) Towards automated melanoma screening: Proper computer vision & reliable results. arXiv:1604.04024

  30. Fossen-Romsaas S, Storm-Johannessen A, Lundervold AS (2020) Synthesizing skin lesion images using cycleGANs–a case study

  31. Furger F, Amruthalingam L, Navarini A, Pouly M (2020) Applications of generative adversarial networks to dermatologic imaging. In: IAPR Workshop on artificial neural networks in pattern recognition, pp 187–199. Springer

  32. Gatys LA, Ecker AS, Bethge M (2015) A neural algorithm of artistic style. arXiv:1508.06576

  33. Ghorbani A, Natarajan V, Coz D, Liu Y (2020) Dermgan: synthetic generation of clinical skin images with pathology. In: Machine learning for health workshop, pp. 155–170. PMLR

  34. Goodfellow I, Pouget-Abadie J, Mirza M, Bing X u, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Advances in Neural Information Processing Systems, 27

  35. Gutman D, Codella NCF, Celebi E, Helba B, Marchetti M, Mishra N, Halpern A (2016) Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (ISBI) 2016, hosted by the international skin imaging collaboration (ISIC). arXiv:1605.01397

  36. Hasan MK, Elahi MTE, Alam MA, Jawad MT (2021) Dermoexpert skin lesion classification using a hybrid convolutional neural network through segmentation, transfer learning, and augmentation medRxiv

  37. Hasan MdK, Dahal L, Samarakoon PN, Tushar FI, Martí R (2020) DSNet: automatic dermoscopic skin lesion segmentation. Comput Biol Med 120:103738

    Article  Google Scholar 

  38. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  39. Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) GANS trained by a two time-scale update rule converge to a local nash equilibrium. Adv Neural Inf Process Syst, 30

  40. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  41. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning, pp 448–456. PMLR

  42. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134

  43. Izadi S, Mirikharaji Z, Kawahara J, Hamarneh G (2018) Generative adversarial networks to segment skin lesions. In: 2018 IEEE 15Th international symposium on biomedical imaging (ISBI 2018), pp 881–884. IEEE

  44. Jalalian A, Mashohor S, Mahmud R, Karasfi B, Iqbal B, Saripan M, Ramli ARB (2017) Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection. EXCLI J 16:113

    Google Scholar 

  45. Jie H, Li S, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  46. Jun F u, Liu J, Tian H, Li Y, Bao Y, Fang Z, Hanqing L (2019) Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3146–3154

  47. Kang Y, Gao S, Roth RE (2019) Transferring multiscale map styles using generative adversarial networks. International Journal of Cartography 5 (2-3):115–141

    Article  Google Scholar 

  48. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of GANs for improved quality, stability, and variation. arXiv:1710.10196

  49. Kawahara J, Daneshvar S, Argenziano G, Hamarneh G (2018) Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J Biomed Health Inf 23(2):538–546

    Article  Google Scholar 

  50. Kittler H, Pehamberger H, Wolff K, Binder MJTIO (2002) Diagnostic accuracy of dermoscopy. Lancet Oncol 3(3):159–165

    Article  Google Scholar 

  51. Korotkov K, Garcia R (2012) Computerized analysis of pigmented skin lesions: a review. Artificial Intell Med 56(2):69–90

    Article  Google Scholar 

  52. Lazo C (2021) Segmentation of skin lesions and their attributes using generative adversarial networks. arXiv:2102.00169

  53. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  54. Lei B, Xia Z, Jiang F, Jiang X, Ge Z, Yanwu X, Qin J, Chen S, Wang T, Wang S (1716) Skin lesion segmentation via generative adversarial networks with dual discriminators. Med Image Anal 64(10):2020

    Google Scholar 

  55. Lucieri A, Dengel A, Ahmed S (2021) Deep learning based decision support for medicine–a case study on skin cancer diagnosis. arXiv:2103.05112

  56. Matsunaga K, Hamada A, Minagawa A, Koga H (2017) Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. arXiv:1703.03108

  57. Mendonça T, Ferreira PM, Marques JS, Marcal ARS, Rozeira J (2013) PH 2-A dermoscopic image database for research and benchmarking. In: 2013 35Th Annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 5437–5440. IEEE

  58. Menegola A, Fornaciali M, Pires R, Bittencourt Flávia V., Avila S, Valle E (2017a) Knowledge transfer for melanoma screening with deep learning. In: 2017 IEEE 14Th International symposium on biomedical imaging (ISBI 2017), pp 297–300. IEEE

  59. Menegola A, Tavares J, Fornaciali M, Li LT, Avila S, Valle E (2017b) RECOD titans at ISIC challenge 2017. arXiv:1703.04819

  60. Mikołajczyk A, Grochowski M (2018) Data augmentation for improving deep learning in image classification problem. In: 2018 International interdisciplinary phd workshop (IIPhDW), pp 117–122. IEEE

  61. Mikołajczyk A, Grochowski M (2019) Style transfer-based image synthesis as an efficient regularization technique in deep learning. In: 2019 24Th International conference on methods and models in automation and robotics (MMAR), pp 42–47. IEEE

  62. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv:1411.1784

  63. Mishra NK, Celebi ME M (2016) An overview of melanoma detection in dermoscopy images using image processing and machine learning. arXiv:1601.07843

  64. Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier GANs

  65. Osuala R, Kushibar K, Garrucho L, Linardos A, Szafranowska Z, Klein S, Glocker B, Diaz O, Lekadir K (2021) A review of generative adversarial networks in cancer imaging new applications. New Solutions. arXiv:2107.09543

  66. Pacheco Andre GC, Krohling RA (2019) Recent advances in deep learning applied to skin cancer detection. arXiv:1912.03280

  67. Palatucci MM, Pomerleau DA, Hinton G, Mitchell T (2009) Zero-shot learning with semantic output codes

  68. Park T, Liu M-Y, Wang T-C, Zhu J-Y (2019) Semantic image synthesis with spatially-adaptive normalization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2337–2346

  69. Perez F, Vasconcelos C, Avila S, Valle E (2018) Data augmentation for skin lesion analysis. In: OR 2.0 Context-aware operating theaters, computer assisted robotic endoscopy, clinical image-based procedures, and skin image analysis, pp. 303–311. Springer

  70. Pham Tri-Cong, Luong Chi-Mai, Visani M, Hoang Van-Dung (2018) Deep CNN and data augmentation for skin lesion classification. In: Asian Conference on Intelligent Information and Database Systems, pp 573–582. Springer

  71. Pollastri F, Bolelli F, Palacios RP, Grana C (2018) Improving skin lesion segmentation with generative adversarial networks. In: 2018 IEEE 31St International symposium on computer-based medical systems (CBMS), pp 442–443. IEEE

  72. Pollastri F, Bolelli F, Paredes R, Grana C (2020) Augmenting data with GANs to segment melanoma skin lesions. Multimed Tools Appl 79 (21):15575–15592

    Article  Google Scholar 

  73. Qasim AB, Ezhov I, Shit S, Schoppe O, Paetzold JC, Sekuboyina A, Kofler F, Lipkova J, Li H, Menze B (2020) Red-GAN: attacking class imbalance via conditioned generation. Yet another medical imaging perspective. In: Medical imaging with deep learning, pp 655–668. PMLR

  74. Qin Z, Liu Z, Zhu P, Xue Y (2020) A GAN-based image synthesis method for skin lesion classification. Comput Methods Prog Biomed 195:105568

    Article  Google Scholar 

  75. Radford A, Luke M, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434

  76. Rashid H, Asjid Tanveer M, Khan HA (2019) Skin lesion classification using GAN based data augmentation. In: 2019 41St annual international conference of the IEEE engineering in medicine and biology society (EMBC), pp 916–919. IEEE

  77. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention, pp 234–241. Springer

  78. Rotemberg V, Kurtansky N, Betz-Stablein B, Caffery L, Chousakos E, Codella N, Combalia M, Dusza S, Guitera P, Gutman D et al (2021) A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scient Data 8(1):1–8

    Google Scholar 

  79. Sarker MMK, Rashwan HA, Akram F, Singh VK, Banu SF, Chowdhury FUH, Choudhury KA, Chambon S, Radeva P, Puig D et al (2021) SLSNEt: skin lesion segmentation using a lightweight generative adversarial network. Expert Systems with Applications, 115433

  80. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. Journal of Big Data 6(1):1–48

    Article  Google Scholar 

  81. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  82. Trivedi A, Muppalla S, Pathak S, Mobasher A, Janowski P, Dodhia R, Ferres JML (2020) Improving Lesion Detection by exploring bias on Skin Lesion dataset. arXiv:2010.01485

  83. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scient Data 5(1):1–9

    Article  Google Scholar 

  84. Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8798–8807

  85. Wei J, Suriawinata A, Vaickus L, Ren B, Liu X, Wei J, Hassanpour S (2019) Generative image translation for data augmentation in colorectal histopathology images. Proc Mach Learn Res 116:10

    Google Scholar 

  86. Wenli T u, Liu X, Wei H u, Pan Z (2019) Dense-residual network with adversarial learning for skin lesion segmentation. IEEE Access 7:77037–77051

    Article  Google Scholar 

  87. Xian Y, Lampert CH, Schiele B, Akata Z (2018) Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell 41(9):2251–2265

    Article  Google Scholar 

  88. Xue Y, Tao X u, Huang X (2018) Adversarial learning with multi-scale loss for skin lesion segmentation. In: 2018 IEEE 15Th International symposium on biomedical imaging (ISBI 2018), pp 859–863. IEEE

  89. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  90. Zhao C, Shuai R, Li M, Liu W, Die H, Menglin W (2021) Dermoscopy image classification based on StyleGAN and DenseNet201. IEEE Access 9:8659–8679

    Article  Google Scholar 

  91. Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using Cycle-Consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

  92. Zunair H, Hamza AB (2020) Melanoma detection using adversarial training and deep transfer learning. Physics in Medicine & Biology 65(13):135005

    Article  Google Scholar 

Download references

Acknowledgements

I want to acknowledge Dr. Hazrat Ali’s valuable input during the review process.

Funding

Not Applicable

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Syed Qasim Gilani.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

A list of abbreviations used in this paper is given in Table 6.

Table 6 List of abbreviations used in this paper

1.1 Datasets used for skin lesion analysis

GANs were used for synthesizing lesion images due to the difficulties in collecting large datasets for skin lesion analysis. Skin lesion analysis is mostly performed using open-source datasets. Some of the available open-source datasets are presented in Table 7.

Table 7 Datasets for skin lesion analysis

1.2 Github repositories

Codes are available for synthesizing skin lesion images using different architectures of GANs. Github repositories for implementing some of the GAN architectures discussed in Sections 3 and 4 are presented in Table 8.

Table 8 GitHub repositories for skin lesion analysis

A comparison of the computational cost of different synthesis networks given in Ding et al., Sarker et al. is presented in Table 9. Table 9 compares the number of parameters in millions.

Table 9 Computational cost of different semantic synthesis method

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gilani, S.Q., Marques, O. Skin lesion analysis using generative adversarial networks: a review. Multimed Tools Appl 82, 30065–30106 (2023). https://doi.org/10.1007/s11042-022-14267-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14267-z

Keywords

Navigation