Skip to main content
SpringerLink
Log in

Classification of large-scale image database of various skin diseases using deep learning

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

This article has been updated

Abstract

Purpose

The purpose of this study was to develop a deep learning-based computer-aided diagnosis system for skin disease classification using photographic images of patients. The targets are 59 skin diseases, including localized and diffuse diseases captured by photographic cameras, resulting in highly diverse images in terms of the appearance of the diseases or photographic conditions.

Methods

ResNet-18 is used as a baseline model for classification and is reinforced by metric learning to boost generalization in classification by avoiding the overfitting of the training data and increasing the reliability of CADx for dermatologists. Patient-wise classification is performed by aggregating the inference vectors of all the input patient images.

Results

The experiment using 70,196 images of 13,038 patients demonstrated that classification accuracy was significantly improved by both metric learning and aggregation, resulting in patient accuracies of 0.579 for Top-1, 0.793 for Top-3, and 0.863 for Top-5. The McNemar test showed that the improvements achieved by the proposed method were statistically significant.

Conclusion

This study presents a deep learning-based classification of 59 skin diseases using multiple photographic images of a patient. The experimental results demonstrated that the proposed classification reinforced by metric learning and aggregation of multiple input images was effective in the classification of patients with diverse skin diseases and imaging conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Change history

  • 05 August 2021

    Red words and red numerals were present in table 1 and these have been changed to black text.

References

  1. Griffiths C, Barker J, Bleiker T, Chalmers R, Creamer D (2016) Rook's textbook of dermatology. Wiley-Blackwell

  2. Codella NCF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, Halpern A (2018) Skin lesion analysis toward melanoma detection: a challenge. In: The 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), pp 168–172. https://doi.org/10.1109/ISBI.2018.8363547

  3. Codella N, Rotemberg V, Tschandl P, Celebi ME, Dusza S, Gutman D, Helba B, Kalloo A, Liopyris K, Marchetti M, Kittler H, Halpern A (2019) Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC)

  4. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118. https://doi.org/10.1038/nature21056

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Fujisawa Y, Otomo Y, Ogata Y, Nakamura Y, Fujita R, Ishitsuka Y, Watanabe R, Okiyama N, Ohara K, Fujimoto M (2019) Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images surpasses board-certified dermatologists in skin tumour diagnosis. Br J Dermatol 180(2):373–381. https://doi.org/10.1111/bjd.16924

    Article  CAS  PubMed  Google Scholar 

  6. Tschandl P, Rosendahl C, Akay BN, Argenziano G, Blum A, Braun RP, Cabo H, Gourhant JY, Kreusch J, Lallas A, Lapins J, Marghoob A, Menzies S, Neuber NM, Paoli J, Rabinovitz HS, Rinner C, Scope A, Soyer PH, Sinz C, Thomas L, Zalaudek I, Kittler H (2019) Expert-Level Diagnosis of nonpigmented skin cancer by combined convolutional neural networks. JAMA Dermatol 155(1):58–65. https://doi.org/10.1001/jamadermatol.2018.4378

    Article  PubMed  Google Scholar 

  7. Dorj UO, Lee KK, Choi JY, Lee M (2018) The skin cancer classification using deep convolutional neural network. Multimedia Tools Appl 77:9909–9924. https://doi.org/10.1007/s11042-018-5714-1

    Article  Google Scholar 

  8. Han SS, Kim MS, Lim W, Park GH, Park I, Chang SE (2018) Classification of the clinical images for benign and malignant cutaneous tumors using a deep learning algorithm. J Investig Dermatol 138(7):1529–1538. https://doi.org/10.1016/j.jid.2018.01.028

    Article  CAS  PubMed  Google Scholar 

  9. Harangi B (2018) Skin lesion classification with ensembles of deep convolutional neural networks. J Biomed Inform 86:25–32. https://doi.org/10.1016/j.jbi.2018.08.006

    Article  PubMed  Google Scholar 

  10. Wei D, Cao S, Ma K, Zheng Y (2020) Learning and exploiting interclass visual correlations for medical image classification. Med Image Comput Comput Assist Interv 12261:106–115. https://doi.org/10.1007/978-3-030-59710-8_11

    Article  Google Scholar 

  11. Yap J, Yolland W, Tschandl P (2018) Multimodal skin lesion classification using deep learning. Exp Dermatol 27(11):1261–1267. https://doi.org/10.1111/exd.13777

    Article  PubMed  Google Scholar 

  12. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161. https://doi.org/10.1038/sdata.2018.161

    Article  PubMed  PubMed Central  Google Scholar 

  13. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. IEEE Conf Comput Vis Pattern Recognit (CVPR) 2016:2818–2826. https://doi.org/10.1109/CVPR.2016.308

    Article  Google Scholar 

  14. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. IEEE Conf Comput Vis Pattern Recognit (CVPR) 2015:1–9. https://doi.org/10.1109/CVPR.2015.7298594

    Article  Google Scholar 

  15. Schroff F, Kalenichenko D, Philbin J (2015) FaceNet: a unified embedding for face recognition and clustering. IEEE Conf Comput Vis Pattern Recognit (CVPR) 2015:815–823. https://doi.org/10.1109/CVPR.2015.7298682

    Article  Google Scholar 

  16. Deng J, Guo J, Xue N, Zafeiriou S (2019) ArcFace: additive angular margin loss for deep face recognition. IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR) 2019:4685–4694. https://doi.org/10.1109/CVPR.2019.00482

    Article  Google Scholar 

  17. Santi P, Irina S, Matt R (2019) Few-shot learning with deep triplet networks for brain imaging modality recognition. Int Worksh Med Image Learn Less Labels Imperfect Data 11795:181–189. https://doi.org/10.1007/978-3-030-33391-1_21

    Article  Google Scholar 

  18. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conf Comput Vis Pattern Recognit (CVPR) 2016:770–778. https://doi.org/10.1109/CVPR.2016.90

    Article  Google Scholar 

  19. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning, vol 37, pp 448–456

  20. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vis 115:211–225. https://doi.org/10.1007/s11263-015-0816-y

    Article  Google Scholar 

  21. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 1026–1034

  22. Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: 3rd International conference on learning representations (ICLR)

  23. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929

Download references

Acknowledgements

This research is supported in part by the ICT infrastructure establishment and implementation of artificial intelligence for clinical and medical research from the Japan Agency for Medical Research and Development, AMED, JP20lk1010036.

Author information

Authors and Affiliations

  1. Institute of Engineering, Tokyo University of Agriculture and Technology, Koganei, Tokyo, Japan

    Masaya Tanaka, Atsushi Saito & Akinobu Shimizu

  2. Department of Dermatology, Tohoku University Graduate School of Medicine, Sendai, Miyagi, Japan

    Kosuke Shido & Kenshi Yamasaki

  3. Department of Dermatology, Faculty of Medicine, University of Tsukuba, Tsukuba, Ibanaki, Japan

    Yasuhiro Fujisawa

  4. Department of Dermatology, Osaka University Graduate School of Medicine, Suita, Osaka, Japan

    Manabu Fujimoto

  5. Research Centre for Medical Bigdata, National Institute of Informatics, Chiyoda-ku, Tokyo, Japan

    Kohei Murao, Youichirou Ninomiya & Shin’ichi Satoh

Authors
  1. Masaya Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Atsushi Saito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kosuke Shido
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yasuhiro Fujisawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kenshi Yamasaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Manabu Fujimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kohei Murao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Youichirou Ninomiya
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Shin’ichi Satoh
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Akinobu Shimizu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akinobu Shimizu.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Ethical approval

This manuscript has not been published elsewhere and is not under consideration for publication in any other journal. All authors have approved the manuscript and agree with its submission to IJCARS. All procedures in this study involving human participants were performed in accordance with the ethical standards of the institutional research committees and the 1975 Helsinki Declaration (as revised in 2008(5)).

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Confusion matrices of the image classification results at Levels 1, 2, and 3 of the taxonomy-tree. The darker the color of the cell, the higher the number of image classifications (Fig. 

Fig. 8
figure 8

Confusion matrices of the baseline model and the proposed model (\(\gamma = 4\))

Full size image

8).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tanaka, M., Saito, A., Shido, K. et al. Classification of large-scale image database of various skin diseases using deep learning. Int J CARS 16, 1875–1887 (2021). https://doi.org/10.1007/s11548-021-02440-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02440-y

Keywords

Search

Navigation