Skip to main content
SpringerLink
Log in

Exploring strategies to generate Fitzpatrick skin type metadata for dermoscopic images using individual typology angle techniques

  • Track 2: Medical Applications of Multimedia
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

When building machine learning models for dermatology one should strive to ensure that the resulting solutions do not exhibit any bias, which could cause a misdiagnosis of some patients – for example, due to the patient’s skin type being under-represented in the datasets used to build those models. Public dermatology datasets usually lack representativeness for certain skin tones, which are typically encoded using the Fitzpatrick skin type (a range of 1-6, where higher numbers mean darker skin tones). Moreover, those datasets usually lack metadata containing skin type information. To address the lack of such metadata, the Fitzpatrick skin type can be estimated based on pixel contents by computing the image’s Individual Typology Angle (ITA) and subsequently mapping ranges of ITA values to the corresponding Fitzpatrick skin type. This work focuses on computing the ITA for skin lesion images without binary masks indicating the location of the lesions. Four different approaches are proposed, experimentally evaluated using three datasets from the International Skin Imaging Collaboration (ISIC) and compared. The approaches were evaluated based on the estimated ITA value (essentially a regression task) and the resulting Fitzpatrick skin type (a classification task). Experimental results showed that the Structured Patches and Center Cropped methods performed best overall. The proposed algorithms can be used as a proxy for generating the Fitzpatrick skin type metadata for datasets that lack that metadata and without the need for segmentation masks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://pypi.org/project/derm-ita/

  2. Examples of challenges include https://grand-challenge.org/challenges/ and https://www.kaggle.com/

  3. https://pypi.org/project/derm-ita/

  4. https://github.com/AdamCorbinFAUPhD/derm_ita

  5. https://colab.research.google.com/

References

  1. Bakhshali MA, Gholizadeh M, Layegh P, Nahidi Y, Memarzadeh Z, Tayyebi meibodi N, Eslami S (2021) Evaluation of high-efficiency image coding algorithm for dermatology images in teledermatology. Skin Res Technol, pp 1–7. https://doi.org/10.1111/srt.13081

  2. Berseth M (2017) ISIC 2017 - Skin lesion analysis towards melanoma detection. arXiv:1703.00523 [cs]. Accessed 2021-11-04

  3. Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W (2020) Machine learning in dermatology: current applications, opportunities, and limitations. Dermatology and Therapy 10(3):365–386. https://doi.org/10.1007/s13555-020-00372-0. Accessed 2021-10-01

    Article  Google Scholar 

  4. Chardon A, Cretois I, Hourseau C (1991) Skin colour typology and suntanning pathways, Accessed 2021-10-28. Int J Cosmet Sci 13(4):191–208. https://doi.org/10.1111/j.1467-2494.1991.tb00561.x

    Article  Google Scholar 

  5. Codella NCF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, Halpern A (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017, international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). arXiv:1710.05006 [cs]. Accessed 2021-11-03

  6. Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo A, Liopyris K, Marchetti M, Kittler H, Halpern A (2019) Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (ISIC). arXiv:1902.03368 [cs]. Accessed 2021-11-03

  7. Combalia M, Codella NCF, Rotemberg V, Helba B, Vilaplana V, Reiter O, Carrera C, Barreiro A, Halpern A.C, Puig S, Malvehy J (2019) BCN20000: Dermoscopic lesions in the wild. arXiv:1908.02288 [cs, eess]. Accessed 2021-11-03

  8. Daneshjou R, Smith MP, Sun MD, Rotemberg V, Zou J (2021) Lack of transparency and potential bias in artificial intelligence data sets and algorithms: a scoping review, Accessed 2021-12-16. JAMA Dermatol 157(11):1362. https://doi.org/10.1001/jamadermatol.2021.3129

    Article  Google Scholar 

  9. Daneshjou R, Vodrahalli K, Liang W, Novoa RA, Jenkins M, Rotemberg V, Ko J, Swetter SM, Bailey EE, Gevaert O, Mukherjee P, Phung M, Yekrang K, Fong B, Sahasrabudhe R, Zou J, Chiou A (2021) Disparities in Dermatology AI: assessments using diverse clinical images. arXiv:2111.08006 [cs, eess], Accessed 2022-01-10

  10. Das S (2021) Automated Bias reduction in deep learning based melanoma diagnosis using a semi-supervised algorithm, vol 15

  11. Fitzpatrick TB (1988) The validity and practicality of sun-reactive skin types I Through VI, Accessed 2021-09-14. Arch Dermatol 124(6):869. https://doi.org/10.1001/archderm.1988.01670060015008

    Article  Google Scholar 

  12. Giotis I, Molders N, Land S, Biehl M, Jonkman MF, Petkov N (2015) MED-NODE: A computer-assisted melanoma diagnosis system using non-dermoscopic images. Expert Syst Appl 42(19):6578–6585. https://doi.org/10.1016/j.eswa.2015.04.034

    Article  Google Scholar 

  13. Groh M, Harris C, Soenksen L, Lau F, Han R, Kim A, Koochek A, Badri O (2021) Evaluating deep neural networks trained on clinical images in dermatology with the Fitzpatrick 17k Dataset. arXiv:2104.09957 [cs]. Accessed 2021-09-14

  14. Gutman D, Codella NCF, Celebi E, Helba B, Marchetti M, Mishra N, Halpern A (2016) Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (ISBI) 2016 ,hosted by the international skin imaging collaboration (ISIC). arXiv:1605.01397 [cs]. Accessed 2021-11-04

  15. Jafari MH, Karimi N, Nasr-Esfahani E, Samavi S, Soroushmehr SMR, Ward K, Najarian K (2016) Skin lesion segmentation in clinical images using deep learning. In: 2016 23rd International conference on pattern recognition (ICPR), pp 337–342. IEEE. https://doi.org/10.1109/ICPR.2016.7899656. http://ieeexplore.ieee.org/document/7899656/. Accessed 2022-05-25

  16. Kamulegeya LH, Okello M, Bwanika JM, Musinguzi D, Lubega W, Rusoke D, Nassiwa F, Börve A (2019) Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning. Technical report, Bioinformatics, Accessed 2022-05-25. https://doi.org/10.1101/826057

  17. Kawahara J, Daneshvar S, Argenziano G, Hamarneh G (2019) Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE J Biomedi Health Inf 23(2):538–546. https://doi.org/10.1109/JBHI.2018.2824327

    Article  Google Scholar 

  18. Kim H, Tadesse GA, Cintas C, Speakman S, Varshney K (2021) Out-of-Distribution detection in dermatology using input perturbation and subset scanning. arXiv:2105.11160 [cs], Accessed 2021-12-20

  19. Kinyanjui NM, Odonga T, Cintas C, Codella NCF, Panda R, Sattigeri P, Varshney KR (2019) Estimating skin tone and effects on classification performance in dermatology Datasets. arXiv:1910.13268 [cs, stat]. Accessed 2021-10-01

  20. Kinyanjui NM, Odonga T, Cintas C, Codella NCF, Panda R, Sattigeri P, Varshney KR (2020) Fairness of classifiers across skin tones in dermatology. In: Martel AL, Abolmaesumi P, Stoyanov D, Mateus D, Zuluaga MA, Zhou SK, Racoceanu D, Joskowicz L (eds) Medical image computing and computer assisted intervention – MICCAI 2020. Lecture Notes in Computer Science, pp 320–329, Springer. https://doi.org/10.1007/978-3-030-59725-2_31

  21. Koklu M, Ozkan IA (2017) Skin lesion classification using machine learning algorithms. Int J Intell Syst Appl Eng 4(5):285–289. https://doi.org/10.18201/ijisae.2017534420. Accessed 2021-10-01

    Article  Google Scholar 

  22. Mendonca T, Ferreira PM, Marques JS, Marcal ARS, Rozeira J (2013) PH2 - A dermoscopic image database for research and benchmarking. In: 2013 35th Annual international conference of the ieee engineering in medicine and biology society (EMBC), pp 5437–5440. IEEE. https://doi.org/10.1109/EMBC.2013.6610779. http://ieeexplore.ieee.org/document/6610779/ Accessed 2021-11-03

  23. Merler M, Ratha N, Feris RS, Smith JR (2019) Diversity in faces, Accessed 2021-09-29. arXiv:1901.10436

  24. Pacheco A, Lima GR, Salomão AS, Krohling B, Biral IP, Giorisatto De Angelo G, Alves Jr FCR, Esgario JGM, Simora AC, Castro PBC, Rodrigues FB, Frasson PHL, Krohling RA, Knidel H, Santos MCS, Espírito Santo RB, Macedo TLSG, Canuto TRP, de Barros LFS (2020) PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones. Elsevier Inc vol 1. https://doi.org/10.17632/zr7vgbcyr2.1. Publisher: Mendeley Data. Accessed 2021-11-02

  25. Raumanns R, Schouten G, Joosten M, Pluim JPW, Cheplygina V (2021) ENHANCE (ENriching Health data by ANnotations of Crowd and Experts): A case study for skin lesion classification. arXiv:2107.12734 [cs]. Accessed 2021-09-30

  26. Rotemberg V, Kurtansky N, Betz-Stablein B, Caffery L, Chousakos E, Codella N, Combalia M, Dusza S, Guitera P, Gutman D, Halpern A, Helba B, Kittler H, Kose K, Langer S, Lioprys K, Malvehy J, Musthaq S, Nanda J, Reiter O, Shih G, Stratigos A, Tschandl P, Weber J, Soyer HP (2021) A patient-centric dataset of images and metadata for identifying melanomas using clinical context. Scientific Data 8(1):34. https://doi.org/10.1038/s41597-021-00815-z. Accessed 2021-11-03

    Article  Google Scholar 

  27. Sun X, Yang J, Sun M, Wang K (2016) A benchmark for automatic visual classification of clinical skin disease images. In: European conference on computer vision, pp 206–222

  28. Tadesse GA, Kim H, Daneshjou R, Cintas C, Varshney KR, Adelekun A, Lipoff JB, Onyekaba G, Rottemberg V, Zou J (2021) Automated evaluation of representation in dermatology educational materials, vol 5

  29. Thi Hai van N, Tat Thang N, Di J, Guo M (2018) Predicting color change in wood during heat treatment using an artificial neural network model. Bioresources 3:6250–6264. https://doi.org/110.15376/biores.13.3.6250-6264

    Google Scholar 

  30. Tschandl P, Codella N, Akay BN, Argenziano G, Braun RP, Cabo H, Gutman D, Halpern A, Helba B, Hofmann-Wellenhof R, Lallas A, Lapins J, Longo C, Malvehy J, Marchetti MA, Marghoob A, Menzies S, Oakley A, Paoli J, Puig S, Rinner C, Rosendahl C, Scope A, Sinz C, Soyer HP, Thomas L, Zalaudek I, Kittler H (2019) Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study. Lancet Oncol 20 (7):938–947. https://doi.org/10.1016/S1470-2045(19)30333-X. Accessed 2021-10-01

    Article  Google Scholar 

  31. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5 (1):180161. https://doi.org/10.1038/sdata.2018.161. Accessed 2021-11-03

    Article  Google Scholar 

  32. Wilkes M, Wright CY, du Plessis JL, Reeder A (2015) Fitzpatrick skin type, individual typology angle, and melanin index in an african population: steps toward universally applicable skin photosensitivity assessments, Accessed 2022-05-25. JAMA Dermatol 151(8):902–903. https://doi.org/10.1001/jamadermatol.2015.0351

    Article  Google Scholar 

  33. Wu Y, Tanaka T, Akimoto M (2020) Utilization of individual typology angle (ITA) and Hue Angle in the measurement of skin color on images. Bioimaging Society 28:1–8

    Google Scholar 

  34. Yang J, Wu X, Liang J, Sun X, Cheng M-M, Rosin PL, Wang L (2019) Self-paced balance learning for clinical skin disease recognition. IEEE Trans Neural Netw Learn Syst

  35. Yuan Y (2019) Automatic skin lesion segmentation with fully convolutional-deconvolutional networks. IEEE J Biomed Health Inf 23 (2):519–526. https://doi.org/10.1109/JBHI.2017.2787487. arXiv:1703.05165. Accessed 2021-09-30

    Article  Google Scholar 

  36. Yuan Y, Chao M, Lo Y-C (2017) Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging 36(9):1876–1886. https://doi.org/10.1109/TMI.2017.2695227. Conference Name: IEEE Transactions on Medical Imaging

    Article  Google Scholar 

  37. of Edinburgh TU (2019) Edinburgh innovations: Dermofit Image library. https://licensing.edinburgh-innovations.ed.ac.uk/i/software/dermofit-image-library.html Accessed 2021-11-02

Download references

Author information

Authors and Affiliations

  1. Florida Atlantic University, Boca Raton, FL, USA

    Adam Corbin & Oge Marques

Authors
  1. Adam Corbin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oge Marques
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Corbin.

Ethics declarations

The authors did not receive support from any organization for the submitted work.

Competing interests

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Corbin, A., Marques, O. Exploring strategies to generate Fitzpatrick skin type metadata for dermoscopic images using individual typology angle techniques. Multimed Tools Appl 82, 23771–23795 (2023). https://doi.org/10.1007/s11042-022-14211-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14211-1

Keywords

Search

Navigation