Abstract
Recent advances in computer vision have led to breakthroughs in the development of automated skin image analysis. However, no attempt has been made to evaluate the consistency in performance across populations with varying skin tones. In this paper, we present an approach to estimate skin tone in skin disease benchmark datasets and investigate whether model performance is dependent on this measure. Specifically, we use individual typology angle (ITA) to approximate skin tone in dermatology datasets. We look at the distribution of ITA values to better understand skin color representation in two benchmark datasets: 1) the ISIC 2018 Challenge dataset, a collection of dermoscopic images of skin lesions for the detection of skin cancer, and 2) the SD-198 dataset, a collection of clinical images capturing a wide variety of skin diseases. To estimate ITA, we first develop segmentation models to isolate non-diseased areas of skin. We find that the majority of the data in the two datasets have ITA values between 34.5\(^\circ \) and 48\(^\circ \), which are associated with lighter skin, and is consistent with under-representation of darker skinned populations in these datasets. We also find no measurable correlation between accuracy of machine learning models and ITA values, though more comprehensive data is needed for further validation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abedini, M., et al.: Accurate and scalable system for automatic detection of malignant melanoma. In: Celebi, M.E., Mendonca, T., Marques, J.S. (eds.) Dermoscopy Image Analysis. CRC Press (2015)
Adamson, A.S., Smith, A.: Machine learning and health care disparities in dermatology. JAMA Dermatol. 154(11), 1247–1248 (2018)
Barocas, S., Selbst, A.D.: Big data’s disparate impact. Calif. Law Rev. 104(3), 671–732 (2016)
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R.L., Torre, L.A., Jemal, A.: Global cancer statistics 2018. CA-Cancer J. Clin. 68(6), 394–424 (2018)
Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: Proceedings of the Conference on Fairness, Accountability and Transparency, pp. 77–91 (2018)
Casale, G.R., Siani, A.M., Diémoz, H., Agnesod, G., Parisi, A.V., Colosimo, A.: Extreme UV index and solar exposures at Plateau Rosà (3500 m a.s.l.) in Valle d’Aosta Region, Italy. Sci. Total Environ. 512–513, 622–630 (2015)
Celebi, M.E., Codella, N., Halpern, A.: Dermoscopy image analysis: overview and future directions. IEEE J. Biomed. Health 23(2), 474–478 (2019)
Celebi, M.E., Codella, N., Halpern, A., Shen, D.: Guest editorial: skin lesion image analysis for melanoma detection. IEEE J. Biomed. Health 23(2), 479–480 (2019)
Chaturvedi, S.S., Gupta, K., Prasad, P.: Skin lesion analyser: an efficient seven-way multi-class skin cancer classification using MobileNet. arXiv:1907.03220 (2019)
Codella, N., et al.: Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the International Skin Imaging Collaboration (ISIC). arXiv:1902.03368 (2019)
Codella, N.C.F., et al.: Deep learning ensembles for melanoma recognition in dermoscopy images. IBM J. Res. Dev. 61(4/5), 5 (2016)
Eilers, S., et al.: Accuracy of self-report in assessing Fitzpatrick skin phototypes I through VI. JAMA Dermatol. 149(11), 1289–1294 (2013)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)
Gohara, M.: Skin cancer: an African perspective. Brit. J. Dermatol. 173(Suppl. 2), 17–21 (2015)
Haenssle, H.A., et al.: Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann. Oncol. 29(8), 1836–1842 (2018)
He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. arXiv:1703.06870 (2018)
International Skin Imaging Collaboration: ISIC 2018: Skin lesion analysis towards melanoma detection (2018). https://challenge2018.isic-archive.com/
Johnson, J.W.: Automatic nucleus segmentation with mask-RCNN. In: Arai, K., Kapoor, S. (eds.) CVC 2019. AISC, vol. 944, pp. 399–407. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-17798-0_32
Karimkhani, C., et al.: Global skin disease morbidity and mortality: an update from the global burden of disease study 2013. JAMA Dermatol. 153(5), 406–412 (2017)
Korotkov, K., Garcia, R.: Computerized analysis of pigmented skin lesions: a review. Artif. Intell. Med. 56(2), 69–90 (2012)
Kundu, R.V., Patterson, S.: Dermatologic conditions in skin of color: Part I. Special considerations for common skin disorders. Am. Fam. Phys. 87(12), 850–856 (2013)
Mahendraraj, K., Sidhu, K., Lau, C.S.M., McRoy, G.J., Chamberlain, R.S., Smith, F.O.: Malignant melanoma in African–Americans: a population-based clinical outcomes study involving 1106 African–American patients from the surveillance, epidemiology, and end result (SEER) database (1988–2011). Medicine 96(15), e6258 (2017)
Marchetti, M.A., Chung, E., Halpern, A.C.: Screening for acral lentiginous melanoma in dark-skinned individuals. JAMA Dermatol. 151(10), 1055–1056 (2015)
Merler, M., Ratha, N., Feris, R.S., Smith, J.R.: Diversity in faces. arXiv:1901.10436 (2019)
Muthukumar, V.: Color-theoretic experiments to understand unequal gender classification accuracy from face images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2019)
Raji, I.D., Buolamwini, J.: Actionable auditing: investigating the impact of publicly naming biased performance results of commercial AI products. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 429–435 (2019)
Rotemberg, V., Halpern, A., Dusza, S.W., Codella, N.C.F.: The role of public challenges and data sets towards algorithm development, trust, and use in clinical practice. Semin. Cutan. Med. Surg. 38(1), E38–E42 (2019)
Stoecker, W.V., Moss, R.H.: Editorial: digital imaging in dermatology. Comput. Med. Imag. Grap. 16(3), 145–150 (1992)
Sun, X., Yang, J., Sun, M., Wang, K.: A benchmark for automatic visual classification of clinical skin disease images. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 206–222. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_13
Tschandl, P., Rosendahl, C., Kittler, H.: Data descriptor: the HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 (2018)
Varshney, K.R.: Trustworthy machine learning and artificial intelligence. ACM XRDS 26(3), 26–29 (2019)
Wilkes, M., Wright, C.Y., du Plessis, J.L., Reeder, A.: Fitzpatrick skin type, individual typology angle, and melanin index in an African population. JAMA Dermatol. 151(8), 902–903 (2015)
Wilson, B., Hoffman, J., Morgenstern, J.: Predictive inequity in object detection. arXiv:1902.11097 (2019)
Wu, X.C., et al.: Racial and ethnic variations in incidence and survival of cutaneous melanoma in the United States, 1999–2006. J. Am. Acad. Dermatol. 65(5), S26.e1–S26.e13 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Kinyanjui, N.M. et al. (2020). Fairness of Classifiers Across Skin Tones in Dermatology. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12266. Springer, Cham. https://doi.org/10.1007/978-3-030-59725-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-59725-2_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59724-5
Online ISBN: 978-3-030-59725-2
eBook Packages: Computer ScienceComputer Science (R0)
