Skip to main content

From Majority to Minority: A Diffusion-Based Augmentation for Underrepresented Groups in Skin Lesion Analysis

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2024 Workshops (MICCAI 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15274))

  • 69 Accesses

Abstract

AI-based diagnoses have demonstrated dermatologist-level performance in classifying skin cancer. However, such systems are prone to under-performing when tested on data from minority groups that lack sufficient representation in the training sets. Although data collection and annotation offer the best means for promoting minority groups, these processes are costly and time-consuming. Prior works have suggested that data from majority groups may serve as a valuable information source to supplement the training of diagnostic tools for minority groups. In this work, we propose an effective diffusion-based augmentation framework that maximizes the use of rich information from majority groups to benefit minority groups. Using groups with different skin types as a case study, our results show that the proposed framework can generate synthetic images that improve diagnostic results for the minority groups, even when there is little or no reference data from these target groups. The practical value of our work is evident in medical imaging analysis, where under-diagnosis persists as a problem for certain groups due to insufficient representation. Our implementation detail is available at https://github.com/janet-sw/skin-diff.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Akrout, M., et al.: Diffusion-based data augmentation for skin disease classification: impact across original medical datasets to fully synthetic images (2023)

    Google Scholar 

  2. Brinker, T., et al.: A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur. J. Cancer 111, 148–154 (2019). https://doi.org/10.1016/j.ejca.2019.02.005

  3. Coustasse, A., Sarkar, R., Abodunde, B., Metzger, B.J., Slater, C.M.: Use of teledermatology to improve dermatological access in rural areas. Telemedicine Journal and e-Health: The Official Journal of the American Telemedicine Association (2019)

    Google Scholar 

  4. Daneshjou, R., et al.: Disparities in dermatology AI performance on a diverse, curated clinical image set. Science Advances (2022)

    Google Scholar 

  5. Dhariwal, P., Nichol, A.: Diffusion models beat GANs on image synthesis. Adv. Neural. Inf. Process. Syst. 34, 8780–8794 (2021)

    MATH  Google Scholar 

  6. Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017)

    Article  MATH  Google Scholar 

  7. Fitzpatrick, T.B.: The validity and practicality of sun-reactive skin types I through VI. Arch. Dermatol. 124(6), 869–871 (1988)

    Article  MATH  Google Scholar 

  8. Gal, R., et al.: An image is worth one word: personalizing text-to-image generation using textual inversion. In: The Eleventh International Conference on Learning Representations (2023). https://openreview.net/forum?id=NAQvF08TcyG

  9. Ghorbani, A., Natarajan, V., Coz, D., Liu, Y.: DermGAN: synthetic generation of clinical skin images with pathology. In: Dalca, A.V., et al. (eds.) Proceedings of the Machine Learning for Health NeurIPS Workshop. Proceedings of Machine Learning Research, vol. 116, pp. 155–170. PMLR (2020). https://proceedings.mlr.press/v116/ghorbani20a.html

  10. Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  11. Groh, M., et al.: Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1820–1828 (2021)

    Google Scholar 

  12. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  13. Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Adv. Neural. Inf. Process. Syst. 33, 6840–6851 (2020)

    Google Scholar 

  14. Hu, E.J., et al.: LoRA: low-rank adaptation of large language models. In: International Conference on Learning Representations (2022). https://openreview.net/forum?id=nZeVKeeFYf9

  15. Ktena, I., et al.: Generative models improve fairness of medical classifiers under distribution shifts (2023)

    Google Scholar 

  16. Liu, Y., et al.: A deep learning system for differential diagnosis of skin diseases. Nat. Med. 26(6), 900–908 (2020)

    Article  MATH  Google Scholar 

  17. von Platen, P., et al.: Diffusers: state-of-the-art diffusion models. https://github.com/huggingface/diffusers (2022)

  18. Qin, Z., Liu, Z., Zhu, P., Xue, Y.: A GAN-based image synthesis method for skin lesion classification. Comput. Methods Programs Biomed. 195, 105568 (2020)

    Article  MATH  Google Scholar 

  19. Rezk, E., Eltorki, M., El-Dakhakhni, W., et al.: Improving skin color diversity in cancer detection: deep learning approach. JMIR Dermatol. 5(3), e39143 (2022)

    Article  MATH  Google Scholar 

  20. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10684–10695 (2022)

    Google Scholar 

  21. Sagers, L.W., et al.: Augmenting medical image classifiers with synthetic data from latent diffusion models (2023)

    Google Scholar 

  22. Sagers, L.W., Diao, J.A., Groh, M., Rajpurkar, P., Adamson, A., Manrai, A.K.: Improving dermatology classifiers across populations using images generated by large diffusion models. In: NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research (2022). https://openreview.net/forum?id=Vzdbjtz6Tys

  23. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  24. Wang, J., Zhang, Y., Ding, Z., Hamm, J.: Achieving reliable and fair skin lesion diagnosis via unsupervised domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5157–5166 (2024)

    Google Scholar 

  25. Wu, B., et al.: Visual transformers: token-based image representation and processing for computer vision (2020)

    Google Scholar 

Download references

Acknowledgments

This work was partly supported by the NSF EPSCoR-Louisiana Materials Design Alliance (LAMDA) program #OIA-1946231 and partly by the Harold L. and Heather E. Jurist Center of Excellence for Artificial Intelligence at Tulane University.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Janet Wang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J., Chung, Y., Ding, Z., Hamm, J. (2025). From Majority to Minority: A Diffusion-Based Augmentation for Underrepresented Groups in Skin Lesion Analysis. In: Celebi, M.E., Reyes, M., Chen, Z., Li, X. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2024 Workshops. MICCAI 2024. Lecture Notes in Computer Science, vol 15274. Springer, Cham. https://doi.org/10.1007/978-3-031-77610-6_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-77610-6_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-77609-0

  • Online ISBN: 978-3-031-77610-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics