Abstract
Deep learning models have achieved great success in automating skin lesion diagnosis. However, the ethnic disparity in these models’ predictions, where lesions on darker skin types are usually underrepresented and have lower diagnosis accuracy, receives little attention. In this paper, we propose FairDisCo, a disentanglement deep learning framework with contrastive learning that utilizes an additional network branch to remove sensitive attributes, i.e. skin-type information from representations for fairness and another contrastive branch to enhance feature extraction. We compare FairDisCo to three fairness methods, namely, resampling, reweighting, and attribute-aware, on two newly released skin lesion datasets with different skin types: Fitzpatrick17k and Diverse Dermatology Images (DDI). We adapt two fairness-based metrics DPM and EOM for our multiple classes and sensitive attributes task, highlighting the skin-type bias in skin lesion classification. Extensive experimental evaluation demonstrates the effectiveness of FairDisCo, with fairer and superior performance on skin lesion classification tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adegun, A., Viriri, S.: Deep learning techniques for skin lesion analysis and melanoma cancer detection: a survey of state-of-the-art. Artif. Intell. Rev. 54(2), 811–841 (2021)
AlKattash, J.A.: Dermaamin. https://www.dermaamin.com/site/ (2022)
Alvi, Mohsan, Zisserman, Andrew, Nellåker, Christoffer: Turning a blind eye: explicit removal of biases and variation from deep neural network embeddings. In: Leal-Taixé, Laura, Roth, Stefan (eds.) ECCV 2018. LNCS, vol. 11129, pp. 556–572. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11009-3_34
Balch, C.M., et al.: Final version of 2009 AJCC melanoma staging and classification. J. Clin. Oncol. 27(36), 6199 (2009)
Barata, C., Celebi, M.E., Marques, J.S.: A survey of feature extraction in dermoscopy image analysis of skin cancer. IEEE J. Biomed. Health Inform. 23(3), 1096–1109 (2018)
Bellamy, R.K., et al.: Ai fairness 360: an extensible toolkit for detecting and mitigating algorithmic bias. IBM J. Res. Dev. 63(4/5), 1–4 (2019)
Bendekgey, H., Sudderth, E.: Scalable and stable surrogates for flexible classifiers with fairness constraints. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
Beutel, A., Chen, J., Zhao, Z., Chi, E.H.: Data decisions and theoretical implications when adversarially learning fair representations. arXiv preprint arXiv:1707.00075 (2017)
Bevan, P.J., Atapour-Abarghouei, A.: Detecting melanoma fairly: skin tone detection and debiasing for skin lesion classification. arXiv preprint arXiv:2202.02832 (2022)
Bhardwaj, Aditya, Rege, Priti P..: Skin lesion classification using deep learning. In: Merchant, S.. N.., Warhade, Krishna, Adhikari, Debashis (eds.) Advances in Signal and Data Processing. LNEE, vol. 703, pp. 575–589. Springer, Singapore (2021). https://doi.org/10.1007/978-981-15-8391-9_42
Chabi Adjobo, E., Sanda Mahama, A.T., Gouton, P., Tossa, J.: Towards accurate skin lesion classification across all skin categories using a PCNN fusion-based data augmentation approach. Computers 11(3), 44 (2022)
Daneshjou, R., et al.: Disparities in dermatology AI: assessments using diverse clinical images. arXiv preprint arXiv:2111.08006 (2021)
Du, M., Yang, F., Zou, N., Hu, X.: Fairness in deep learning: a computational perspective. IEEE Intell. Syst. 36(4), 25–34 (2020)
El-Khatib, H., Popescu, D., Ichim, L.: Deep learning-based methods for automatic diagnosis of skin lesions. Sensors 20(6), 1753 (2020)
Elazar, Y., Goldberg, Y.: Adversarial removal of demographic attributes from text data. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 11–21 (2018)
Esteva, A., et al.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017)
Fitzpatrick, T.B.: The validity and practicality of sun-reactive skin types I through VI. Arch. Dermatol. 124(6), 869–871 (1988)
Gessert, N., Nielsen, M., Shaikh, M., Werner, R., Schlaefer, A.: Skin lesion classification using ensembles of multi-resolution efficientnets with meta data. MethodsX 7, 100864 (2020)
Gessert, N., et al.: Skin lesion classification using CNNs with patch-based attention and diagnosis-guided loss weighting. IEEE Trans. Biomed. Eng. 67(2), 495–503 (2019)
Groh, M., Harris, C., Daneshjou, R., Badri, O., Koochek, A.: Towards transparency in dermatology image datasets with skin tone annotations by experts, crowds, and an algorithm. arXiv preprint arXiv:2207.02942 (2022)
Groh, M., et al.: Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1820–1828 (2021)
Harangi, B.: Skin lesion classification with ensembles of deep convolutional neural networks. J. Biomed. Inform. 86, 25–32 (2018)
Hazirbas, C., Bitton, J., Dolhansky, B., Pan, J., Gordo, A., Ferrer, C.C.: Casual conversations: A dataset for measuring fairness in AI. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2289–2293 (2021)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9729–9738 (2020)
Healsmith, M., Bourke, J., Osborne, J., Graham-Brown, R.: An evaluation of the revised seven-point checklist for the early diagnosis of cutaneous malignant melanoma. Br. J. Dermatol. 130(1), 48–50 (1994)
Henning, J.S., et al.: The cash (color, architecture, symmetry, and homogeneity) algorithm for dermoscopy. J. Am. Acad. Dermatol. 56(1), 45–52 (2007)
Jamil, U., Khalid, S.: Comparative study of classification techniques used in skin lesion detection systems. In: 17th IEEE International Multi Topic Conference 2014, pp. 266–271. IEEE (2014)
Kamiran, F., Calders, T.: Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33(1), 1–33 (2012)
Kawahara, J., BenTaieb, A., Hamarneh, G.: Deep features to classify skin lesions. In: 2016 IEEE 13th international symposium on biomedical imaging (ISBI), pp. 1397–1400. IEEE (2016)
Khosla, P., et al.: Supervised contrastive learning. Adv. Neural. Inf. Process. Syst. 33, 18661–18673 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Kinyanjui, Newton M.., et al.: Fairness of classifiers across skin tones in dermatology. In: Martel, Anne L.., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 320–329. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_31
Li, X., Cui, Z., Wu, Y., Gu, L., Harada, T.: Estimating and improving fairness with adversarial learning. arXiv preprint arXiv:2103.04243 (2021)
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput Surv. (CSUR) 54(6), 1–35 (2021)
Nachbar, F., et al.: The ABCD rule of dermatoscopy: high prospective value in the diagnosis of doubtful melanocytic skin lesions. J. Am. Acad. Dermatol. 30(4), 551–559 (1994)
Petersen, F., Mukherjee, D., Sun, Y., Yurochkin, M.: Post-processing for individual fairness. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
Puyol-Antón, Esther, et al.: Fairness in cardiac MR image analysis: an investigation of bias due to data imbalance in deep learning based segmentation. In: de Bruijne, Marleen, et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 413–423. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_39
Samuel, F.D.S.: Atlas dermatologico. http://atlasdermatologico.com.br/index.jsf (2022)
Sarhan, Mhd Hasan, Navab, Nassir, Eslami, Abouzar, Albarqouni, Shadi: Fairness by learning orthogonal disentangled representations. In: Vedaldi, Andrea, Bischof, Horst, Brox, Thomas, Frahm, Jan-Michael. (eds.) ECCV 2020. LNCS, vol. 12374, pp. 746–761. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58526-6_44
Sung, H., et al.: Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: Can. J. Clin. 71(3), 209–249 (2021)
Thota, M., Leontidis, G.: Contrastive domain adaptation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2209–2218 (2021)
Wadsworth, C., Vera, F., Piech, C.: Achieving fairness through adversarial learning: an application to recidivism prediction (2018)
Wang, P., Han, K., Wei, X.S., Zhang, L., Wang, L.: Contrastive learning based hybrid networks for long-tailed image classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 943–952 (2021)
Wang, T., Zhao, J., Yatskar, M., Chang, K.W., Ordonez, V.: Balanced datasets are not enough: estimating and mitigating gender bias in deep image representations. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 5310–5319 (2019)
Wang, Z., et al.: Towards fairness in visual recognition: effective strategies for bias mitigation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8919–8928 (2020)
Wang, Z., et al.: Fairness-aware adversarial perturbation towards bias mitigation for deployed deep models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10379–10388 (2022)
WHO: Cancer (2022). https://www.who.int/news-room/fact-sheets/detail/cancer
Wu, Y., Zeng, D., Xu, X., Shi, Y., Hu, J.: Fairprune: achieving fairness through pruning for dermatological disease diagnosis. arXiv preprint arXiv:2203.02110 (2022)
Xu, Tian, White, Jennifer, Kalkan, Sinan, Gunes, Hatice: Investigating bias and fairness in facial expression recognition. In: Bartoli, Adrien, Fusiello, Andrea (eds.) ECCV 2020. LNCS, vol. 12540, pp. 506–523. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-65414-6_35
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Du, S., Hers, B., Bayasi, N., Hamarneh, G., Garbi, R. (2023). FairDisCo: Fairer AI in Dermatology via Disentanglement Contrastive Learning. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13804. Springer, Cham. https://doi.org/10.1007/978-3-031-25069-9_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-25069-9_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25068-2
Online ISBN: 978-3-031-25069-9
eBook Packages: Computer ScienceComputer Science (R0)