Skip to main content
SpringerLink
Log in

Skin lesion detection using an ensemble of deep models: SLDED

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Skin cancer is a major public health concern and the most common type of cancer among the other types. Reliable automated classification systems will provide clinicians with great help to detect malignant skin lesions as quickly as possible. Recently, deep learning-based approaches have efficiently outperformed other conventional machine learning models in medical image classification tasks. In this study, a novel computer-aided approach is designed for Skin Lesion Detection by creating an Ensemble of Deep (SLDED) models. More specifically, we initially performed a modified faster R-CNN using VGGNet feature extractor on ISIC archive database, including 4668 skin lesion images for lesion localization, and we obtained a mean average precision (mAP) of 0.96. Then we fused four different convolutional neural networks (CNNs) into one framework to obtain high classification accuracy. Moreover, a weighted majority voting method is proposed to aggregate the final decision of each individual voter. We evaluate our experimental classification results on 934 and 200 images from ISIC and PH2 test data. We achieved the average accuracy of 97.1% and 96%, Area under receiver operating characteristics curve (AUC) of 98.6% and 98.1%, precision of 87.1% and 90.2%, recall of 86.7% and 85.4% for ISIC and PH2 test data, respectively. As another objective evaluation, we have tested our proposed procedure on official test set of 2016 and 2017 International Symposium on Biomedical Imaging (ISIB) challenges. It outperforms the results of other proposed frameworks that have been published in those challenges. The results demonstrate that our proposed SLDED method is a meaningful approach to classify four different skin lesions with a high accuracy despite the lack of access to expensive computational equipment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

The datasets analyzed during the current study are available in the International Skin Imaging Collaboration (ISIC) Archive repository, https://challenge.isic-archive.com/data/.

References

  1. Abbes W, Sellami D (2017) Automatic skin lesions classification using ontology-based semantic analysis of optical standard images. Procedia Comput Sci 112:2096–2105

    Article  Google Scholar 

  2. Agarwal M, Damaraju N, Chaieb S Skin lesion analysis toward melanoma detection

  3. Argenziano G, Soyer HP (2001) Dermoscopy of pigmented skin lesions–a valuable tool for early. Lancet Oncol 2(7):443–449

    Article  Google Scholar 

  4. Argenziano G et al (2006) Dermoscopy improves accuracy of primary care physicians to triage lesions suggestive of skin cancer. J Clin Oncol 24(12):1877–1882

    Article  Google Scholar 

  5. Attia M et al (2017) Skin melanoma segmentation using recurrent and convolutional neural networks. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE

  6. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  7. Barata C, Celebi ME, Marques JS (2017) Development of a clinically oriented system for melanoma diagnosis. Pattern Recogn 69:270–285

    Article  Google Scholar 

  8. Baumann LS et al (2018) Safety and efficacy of hydrogen peroxide topical solution, 40%(w/w), in patients with seborrheic keratoses: results from 2 identical, randomized, double-blind, placebo-controlled, phase 3 studies (A-101-SEBK-301/302). J Am Acad Dermatol 79(5):869–877

  9. Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MathSciNet  MATH  Google Scholar 

  10. Bi L et al (2017) Semi-automatic skin lesion segmentation via fully convolutional networks. In: 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017). IEEE

  11. Brinker TJ et al (2019) A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur J Cancer 111:148–154

    Article  Google Scholar 

  12. Brinker TJ et al (2019) Comparing artificial intelligence algorithms to 157 German dermatologists: the melanoma classification benchmark. Eur J Cancer 111:30–37

    Article  Google Scholar 

  13. Brinker TJ et al (2019) Deep neural networks are superior to dermatologists in melanoma image classification. Eur J Cancer 119:11–17

    Article  Google Scholar 

  14. Burdick J et al (2017) The impact of segmentation on the accuracy and sensitivity of a melanoma classifier based on skin lesion images. In: SIIM 2017 scientific program: Pittsburgh, PA, June 1-June 3, 2017, David L. Lawrence Convention Center

  15. Carli P et al (2000) Preoperative assessment of melanoma thickness by ABCD score of dermatoscopy. J Am Acad Dermatol 43(3):459–466

    Article  Google Scholar 

  16. Celebi ME et al (2007) A methodological approach to the classification of dermoscopy images. Comput Med Imaging Graph 31(6):362–373

    Article  Google Scholar 

  17. center, c.s. Estimated new cases, 2019. Available from: https://cancerstatisticscenter.cancer.org/#!/

  18. Chang W-Y et al (2013) Computer-aided diagnosis of skin lesions using conventional digital photography: a reliability and feasibility study. PLoS ONE 8(11):e76212

    Article  Google Scholar 

  19. Codella NC et al (2017) Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). IEEE

  20. Collaboration, I.S.I. (2020) ISIC archive. Available from: https://www.isic-archive.com/#!/topWithHeader/wideContentTop/main

  21. Conn AR, Gould NI, Toint P (1991) A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J Numer Anal 28(2):545–572

    Article  MathSciNet  MATH  Google Scholar 

  22. Dascalu A, David E (2019) Skin cancer detection by deep learning and sound analysis algorithms: A prospective clinical study of an elementary dermoscope. EBioMedicine 43:107–113

    Article  Google Scholar 

  23. Díaz IG (2017) Incorporating the knowledge of dermatologists to convolutional neural networks for the diagnosis of skin lesions. arXiv preprint arXiv:1703.01976

  24. Esteva A et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115

    Article  Google Scholar 

  25. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press

  26. Grichnik JM, Rhodes AR, Sober AJ (2008) Benign neoplasias and hyperplasias of melanocytes. Fitzpatrick’s dermatology in general medicine, 7th edn, pp 1099–103

  27. Haenssle HA et al (2018) Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 29(8):1836–1842

    Article  Google Scholar 

  28. Hahnloser RH et al (2000) Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature 405(6789):947

    Article  Google Scholar 

  29. Harangi B (2018) Skin lesion classification with ensembles of deep convolutional neural networks. J Biomed Inform 86:25–32

    Article  Google Scholar 

  30. He K et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  31. He K et al (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision

  32. Hekler A et al (2019) Pathologist-level classification of histopathological melanoma images with deep neural networks. Eur J Cancer 115:79–83

    Article  Google Scholar 

  33. Hekler A et al (2019) Superior skin cancer classification by the combination of human and artificial intelligence. Eur J Cancer 120:114–121

    Article  Google Scholar 

  34. Hu H et al (2018) CNNAuth: continuous authentication via two-stream convolutional neural networks. In: 2018 IEEE international conference on networking, architecture and storage (NAS). IEEE

  35. Isasi AG, Zapirain BG, Zorrilla AM (2011) Melanomas non-invasive diagnosis application based on the ABCD rule and pattern recognition image processing algorithms. Comput Biol Med 41(9):742–755

    Article  Google Scholar 

  36. Jain S, Pise N (2015) Computer aided melanoma skin cancer detection using image processing. Procedia Comput Sci 48:735–740

    Article  Google Scholar 

  37. Jaleel JA, Salim S, Aswin R (2013) Computer aided detection of skin cancer. In: 2013 International Conference on Circuits, Power and Computing Technologies (ICCPCT). IEEE

  38. Kallenberg M et al (2016) Unsupervised deep learning applied to breast density segmentation and mammographic risk scoring. IEEE Trans Med Imaging 35(5):1322–1331

    Article  Google Scholar 

  39. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  40. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems

  41. Levine AB et al (2019) Rise of the machines: advances in deep learning for cancer diagnosis. Trends Cancer 5:157–169

    Article  Google Scholar 

  42. Li Y, Hu H, Zhou G (2018) Using data augmentation in continuous authentication on smartphones. IEEE Internet Things J 6(1):628–640

    Article  Google Scholar 

  43. Li Y et al (2020) Using feature fusion strategies in continuous authentication on smartphones. IEEE Internet Comput 24(2):49–56

    Article  Google Scholar 

  44. Lopez AR et al (2017) Skin lesion classification from dermoscopic images using deep learning techniques. In: 2017 13th IASTED international conference on biomedical engineering (BioMed). IEEE

  45. Majtner T, Yildirim-Yayilgan S, Hardeberg JY (2016) Combining deep learning and hand-crafted features for skin lesion classification. In: 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA). IEEE

  46. Manning C, Raghavan P, Schütze H (2010) Introduction to information retrieval. Nat Lang Eng 16(1):100–103

    MATH  Google Scholar 

  47. Mar VJ, Scolyer RA, Long GV (2017) Computer-assisted diagnosis for skin cancer: have we been outsmarted? Lancet 389(10083):1962–1964

    Article  Google Scholar 

  48. Marchetti MA et al (2018) Results of the 2016 international skin imaging collaboration international symposium on biomedical imaging challenge: comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J Am Acad Dermatol 78(2):270-277. e1

  49. Maron RC et al (2019) Systematic outperformance of 112 dermatologists in multiclass skin cancer image classification by convolutional neural networks. Eur J Cancer 119:57–65

    Article  Google Scholar 

  50. Matsunaga K et al (2017) Image classification of melanoma, nevus and seborrheic keratosis by deep neural network ensemble. arXiv preprint arXiv:1703.03108

  51. Megahed M et al (2002) Reliability of diagnosis of melanoma in situ. Lancet 359(9321):1921–1922

    Article  Google Scholar 

  52. Mendonca T et al (2015) PH2: A public database for the analysis of dermoscopic images. In: Dermoscopy image analysis. CRC Press

  53. Menegola A et al (2017) RECOD titans at ISIC challenge 2017. arXiv preprint arXiv:1703.04819

  54. Mirzaalian-Dastjerdi H et al (2018) Detecting and measuring surface area of skin lesions, in Bildverarbeitung für die Medizin 2018. Springer, pp 29–34

  55. Moss RH et al (1989) Skin cancer recognition by computer vision. Comput Med Imaging Graph 13(1):31–36

    Article  Google Scholar 

  56. Mueller SA et al (2019) Mutational patterns in metastatic cutaneous squamous cell carcinoma. J Invest Dermatol 139(7):1449-1458.e1

    Article  Google Scholar 

  57. Nida N et al (2019) Melanoma lesion detection and segmentation using deep region based convolutional neural network and fuzzy C-means clustering. Int J Med Informatics 124:37–48

    Article  Google Scholar 

  58. Okur E, Turkan M (2018) A survey on automated melanoma detection. Eng Appl Artif Intell 73:50–67

    Article  Google Scholar 

  59. Renzi M et al (2019) Management of skin cancer in the elderly. Dermatol Clin 37(3):279–286

  60. Sarıgül M, Avci BMOM (2019) Differential convolutional neural network. Neural Netw 116:279–287

    Article  Google Scholar 

  61. Schaefer G et al (2014) An ensemble classification approach for melanoma diagnosis. Memetic Comput 6(4):233–240

    Article  Google Scholar 

  62. Soudani A, Barhoumi W (2019) An image-based segmentation recommender using crowdsourcing and transfer learning for skin lesion extraction. Expert Syst Appl 118:400–410

    Article  Google Scholar 

  63. Sreekantaswamy S et al (2019) Aging and the treatment of basal cell carcinoma. Clin Dermatol 37:373–378

    Article  Google Scholar 

  64. Stoecker WV, Moss RH (1992) Digital imaging in dermatology. Elsevier

  65. Stoecker WV et al (2005) Detection of asymmetric blotches (asymmetric structureless areas) in dermoscopy images of malignant melanoma using relative color. Skin Res Technol 11(3):179–184

    Article  Google Scholar 

  66. Szegedy C et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  67. Uijlings JR et al (2013) Selective search for object recognition. Int J Comput Vision 104(2):154–171

    Article  Google Scholar 

  68. Vasconcelos CN, Vasconcelos BN (2017) Experiments using deep learning for dermoscopy image analysis. Pattern Recognit Lett 139:95–103

    Article  Google Scholar 

  69. Xie S et al (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition

  70. Yu L et al (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Industrial and Systems Engineering, Tarbiat Modares University, Tehran, Iran

    Ali Shahsavari & Toktam Khatibi

  2. Department of Computer Science, Wayne State University, Detroit, MI, 48201, USA

    Sima Ranjbari

Authors
  1. Ali Shahsavari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toktam Khatibi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sima Ranjbari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Toktam Khatibi.

Ethics declarations

Competing interests

The authors have no competing interests in this study.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shahsavari, A., Khatibi, T. & Ranjbari, S. Skin lesion detection using an ensemble of deep models: SLDED. Multimed Tools Appl 82, 10575–10594 (2023). https://doi.org/10.1007/s11042-022-13666-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13666-6

Keywords

Search

Navigation