Skip to main content
SpringerLink
Log in

DermoDeep-A classification of melanoma-nevus skin lesions using multi-feature fusion of visual features and deep neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The Scientific community has been developing computer-aided detection systems (CADs) for automatic diagnosis of pigmented skin lesions (PSLs) for nearly 30 years. Several works have addressed this issue and obtained encouraging results, however, there has not been much focus on the pre-processing step, determining the relevance of the features considered and how they may be important indicators of a lesion’s malignancy. To differentiate between nevus and melanoma skin lesions, the development of CAD system is a challenging task due to the use of inaccurate image processing techniques. In this paper, a new classification system is developed for PSLs known as DermoDeep through a fusion of multiple visual features and deep-neural-network approach. A new aggregation of visual features along with descriptors are extracted in a perceptual-oriented color space. Moreover, a new five-layer architecture based DermoDeep system is proposed. This DermoDeep system applied on 2800 region-of-interest (ROI) PSLs including 1400 nevus and 1400 malignant lesions. The classification accuracy of DermoDeep system is compared with the state-of-the-art methods and evaluated by the sensitivity (SE), specificity (SP) and area under the receiver operating characteristics (AUC) curve. The difference between AUC of DermoDeep is statistically significant compared to other techniques with AUC: 0.96 (p < 0.001), SE of 93% and SP of 95%. The obtained results demonstrate that the DermatDeep can be used to assist dermatologists during a screening process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Abbas Q et al (2013) Pattern classification of dermoscopy images: a perceptually uniform model. Pattern Recogn 46(1):86–97

    Article  Google Scholar 

  2. Abbas Q et al (2013) Melanoma recognition framework based on expert definition of ABCD for dermoscopic images. Skin Res Technol 19:93–102

    Article  Google Scholar 

  3. Abbas Q, Sadaf M, Akram A (2016) Prediction of dermoscopy patterns for recognition of both melanocytic and non-melanocytic skin lesions. Computers 5(13):1–16

    Google Scholar 

  4. Aghbari ZA, Al-Haj R (2006) Hill-manipulation: an effective algorithm for color image segmentation. Image Vis Comput 24:894–903

    Article  Google Scholar 

  5. Argenziano G et al (2000) Interactive atlas of dermoscopy CD. EDRA medical publishing and new media. Milan, Italy

    Google Scholar 

  6. Barata C, Marques J, Rozeira J (2012) A system for the detection of pigment network in Dermoscopy images using directional filters. IEEE Transactions on Biomedical Imaging 59(10):2744–2754

    Article  Google Scholar 

  7. Barata C et al (2014) Two systems for the detection of melanomas in dermoscopy images using texture and color features. IEEE Syst J 8:965–979

    Article  Google Scholar 

  8. Barata C et al (2016) Clinically inspired analysis of dermoscopy images using a generative model. Computer Vision and Image Understanding151: 124–137.

  9. Barata C, Celebi ME, Marques JS (2017) Development of a clinically oriented system for melanoma diagnosis. Pattern Recogn 69:270–285

    Article  Google Scholar 

  10. Bay H et al (2006) SURF: speeded up robust features. Computer Vision, Lecture Notes in Computer Science 3951:404–417

    Article  Google Scholar 

  11. Bengio Y (2009) Learning deep architectures for AI found. Trends Mach Learn:1–127

  12. Blum A et al (2004) Digital image analysis for diagnosis of cutaneous melanoma. Development of a highly effective computer algorithm based on analysis of 837 melanocytic lesions. Br J Dermatol 151:1029–1038

    Article  Google Scholar 

  13. Bradley AP (1997) The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recogn 30(7):1145–1159

    Article  Google Scholar 

  14. Celebi ME, Zornberg A (2014) Automated quantification of clinically significant colors in Dermoscopy images and its application to skin lesion classification. IEEE Syst J 8(3):980–984

    Article  Google Scholar 

  15. Celebi ME et al (2007) A methodological approach to the classification of dermoscopy images. Comput Med Imaging Graph 31:362–373

    Article  Google Scholar 

  16. Celebi ME, Kingravi H, Vela PA (2013) A comparative study of efficient initialization methods for the K-means clustering algorithm. Expert Syst Appl 40(1):200–210

    Article  Google Scholar 

  17. Codella N et al (2015) Deep learning, sparse coding, and SVM for melanoma recognition in dermoscopy images. Machine learning in medical imaging. In: Proceeding of the 6th international workshop, MLMI 2015, Munich, Germany, lecture notes in computer science, springer, Berlin/Heidelberg, Germany, vol 9352, pp 118–126

    Google Scholar 

  18. Demyanov S et al (2016) Classification of dermoscopy patterns using deep convolutional neural networks. Proceeding of13thIEEE International Symposium on Biomedical Imaging (ISBI), San Francisco, Canada:364–366

  19. Deng L (2014) A tutorial survey of architectures, algorithms, and applications for deep learning. APSIPA Trans Signal Inf Process:1–29

  20. Dreiseitl S, Binder M (2005) Do physicians value decision support? A look at the effect of decision support systems on physician opinion. Artif Intell Med 33(1):25–30

    Article  Google Scholar 

  21. Fornaciali M et al (2016) Towards automated melanoma screening: proper computer vision &reliable results. https://arxiv.org/abs/1604.04024. Accessed 5 April 2017

  22. Garnavi R, Aldeen M, Bailey J (2012) Computer-aided diagnosis of melanoma using border- and wavelet-based texture analysis. IEEE Trans Inf Technol Biomed 16(6):1239–1252

    Article  Google Scholar 

  23. GravesA MAR, Hinton G (2013) Speech recognition with deep recurrent neural networks. Proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC:6645–6649

  24. Hinton G et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97

    Article  Google Scholar 

  25. Hu H, Li Y, Liu M, Liang W (2014) Classification of defects in steel strip surface based on multiclass support vector machine. Multimed Tools Appl 69:199–216

    Article  Google Scholar 

  26. Hu H, Liu Y, Liu M, Nie L (2016) Surface defect classification in large-scale strip steel image collection via hybrid chromosome genetic algorithm. Neurocomputing 181:86–95

    Article  Google Scholar 

  27. International Skin Imaging Collaboration (2017) ISIC 2017: skin lesion analysis towards melanoma detection. Computer vision and pattern recognition, http://isdis.net/isic-project/. Accessed 1 January 2017

  28. Ishihara Y et al (2006) Early acral melanoma in situ: correlation between the parallel ridge pattern on dermoscopy and microscopic features. Am J Dermatopathol 28:21–27

    Article  Google Scholar 

  29. Iyatomi H et al ( 2008) Computer-based classification of dermoscopy images of melanocytic lesions on acral volar skin. J Investig Dermatol 128: 2049–2054.

  30. Iyatomi H et al (2008) An improved internet-based melanoma screening system with dermatologist-like tumor area extraction algorithm. Comput Med Imaging Graph 32(7):566–579

    Article  Google Scholar 

  31. Jaworek-Korjakowska J, Kłeczek P (2016) Automatic classification of specific melanocytic lesions. Biomed Res Int 2016:1–17

    Google Scholar 

  32. Kawahara J, BenTaieb A, Hamarneh G (2016) Deep features to classify skin lesions. Proceeding of 13thIEEE International Symposium on Biomedical Imaging (ISBI), San Francisco, Canada:1397–1400

  33. Keren G, Schuller B (2016) Convolutional RNN: an enhanced model for extracting features from sequential data. In Neural Networks (IJCNN), the IEEE 2016 International Joint Conference:3412–3419

  34. Laura K et al (2015) Computer-aided classification of melanocytic lesions using dermoscopic images. J Am Acad Dermatol 73(5):769–776

    Article  Google Scholar 

  35. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  36. Lie M, Zhang L, Hu H, Nie Liqiang DJ (2016) A classification model for semantic entailment recognition with feature combination. Neurocomputing 208:127–135

    Article  Google Scholar 

  37. Lingala M et al (2014) Fuzzy logic color detection: blue areas in melanoma dermoscopy images. Comput Med Imaging Graph 38:403–410

    Article  Google Scholar 

  38. Liu Z et al (2012) Distribution quantification on dermoscopy images for computer-assisted diagnosis of cutaneous melanomas. Medical & Biological Engineering & Computing 50(5):503–513

    Article  Google Scholar 

  39. Liu W, Mei T, Zhang Y, Che C, Luo J (2015) Multi-task deep visual-semantic embedding for video thumbnail selection. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA 2015:3707–3715

    Google Scholar 

  40. Liu L, Wiliem A, Chen S, Lovell BC (2017) What is the best way for extracting meaningful attributes from pictures? Pattern Recogn 64:314–326

    Article  Google Scholar 

  41. Liu L, Nie F, Wiliem A, Li Z, Zhang T, Lovell BC (2018) Multi-modal joint clustering with application for unsupervised attribute discovery. IEEE Trans Image Process 27(9):4345–4356

    Article  MathSciNet  Google Scholar 

  42. Liu D, Liu L, Tie Y, Qi L (2018) Multi-task image set classification via joint representation with class-level sparsity and intra-task low rankness. Pattern Recognition Letter in press

  43. Luo MR, Cui G, Rigg B (2001) The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Research & Application 26(5):340–350

    Article  Google Scholar 

  44. Ma Z, Tavares JMRS (2017) Effective features to classify skin lesions in dermoscopic images. Expert Syst Appl 84:92–101

    Article  Google Scholar 

  45. Ma Z, Tavares JMRS (2017) Effective features to classify skin lesions in dermoscopic images. Expert Syst Appl 84:92–101

    Article  Google Scholar 

  46. McDonald R, Smith KJ (1995) CIE94 - anew colour-difference formula. J Soc Dyers 111:376–379

    Article  Google Scholar 

  47. Md F (2005) Color appearance models, vol 82. Wiley-IS&T

  48. Melgosa M, Huertas R, Berns RS (2004) Relative significance of the terms in the CIEDE2000 and CIE94 color difference formulas. J OpImage Sci Vis 21(12):2269–2275

    Google Scholar 

  49. Mendonca T et al (2013) PH2 - a dermoscopic image database for research and benchmarking. Proceeding of IEEE conference on Eng Med BiolSoc, Piscataway NJ USA, In, pp 5437–5440

    Google Scholar 

  50. Mirzaalian H, Lee TK, Hamarneh G (2016) Skin lesion tracking using structured graphical models. Med Image Anal 27:84–92

    Article  Google Scholar 

  51. Mishra NK, Celebi ME (2016) An Overview of Melanoma Detection in Dermoscopy Images Using Image Processing and Machine Learning. https://arxiv.org/abs/1601.07843. Accessed 6 August 2018

  52. Nie L, Zhang L, Yan Y et al (2017) Multiview physician-specific attributes fusion for health seeking. IEEE Transactions on cybernetics 47(11):3680–3691

    Article  Google Scholar 

  53. Pathan S, Prabhu KG, Siddalingaswamy PC (2018) Techniques and algorithms for computer aided diagnosis of pigmented skin lesions—a review. Biomedical Signal Processing and Control 39:237–262

    Article  Google Scholar 

  54. Premaladha J, Ravichandran KS (2016) Novel approaches for diagnosing melanoma skin lesions through supervised and deep learning algorithms. J Med Syst 40(96):1–12

    Google Scholar 

  55. Roberta B et al (2016) A computational approach for detecting pigmented skin lesions in macroscopic images. Expert Syst Appl 61:53–63

    Article  Google Scholar 

  56. Rosendahl C et al (2011) Diagnostic accuracy of dermatoscopy for melanocytic and nonmelanocytic pigmented lesions. J Am Acad Dermatol 64:1068–1073

    Article  Google Scholar 

  57. Ruiz D et al (2011) A decision support system for the diagnosis of melanoma: a comparative approach. Expert Syst Appl 38(12):15217–15223

    Article  Google Scholar 

  58. Sadeghi M et al (2013) Detection and analysis of irregular streaks in dermoscopic images of skin lesions. IEEE Trans Med Imaging 32:849–861

    Article  Google Scholar 

  59. Sadri AR et al (2017) WN based approach to melanoma diagnosis from Dermoscopy images. IET Image Process 11(7):475–482

    Article  Google Scholar 

  60. Sáez A, Serrano C, Acha B (2014) Model-based classification methods of global patterns in dermoscopic images. IEEE Trans Med Imaging 33(5):1137–1147

    Article  Google Scholar 

  61. Schanda J (2007) International commission on illumination. Colorimetry: understanding the CIE system. John Wiley Sons, New Jersey

    Google Scholar 

  62. Sharma G, Wu W, Dalal EN (2005) The CIEDE2000 color-difference formula: implementation notes, supplementary test data, and mathematical observations. Color Res Appl 30(1):21–30

    Article  Google Scholar 

  63. Shimizu K (2015) Four-class classification of skin lesions with task decomposition strategy. IEEE Trans Biomed Eng 62(1):274–283

    Article  Google Scholar 

  64. Stoecker WV et al (2011) Detection of granularity in dermoscopy images of malignant melanoma using color and texture features. Comput Med Imaging Graph 35:144–147

    Article  Google Scholar 

  65. Thomas S et al ( 2013) Deep neural network features and semi-supervised training for low resource speech recognition. In: proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, pp. 6704–6708.

  66. University of Auckland, (2016) New Zealand. Dermatologic image, database. http://dermnetnz.org/doctors/dermoscopy-course/. Accessed 15 March 2016

  67. Unlu E, Akay BN, Erdem C (2014) Comparison of dermatoscopic diagnostic algorithms based on calculation: the ABCD rule of dermatoscopy, the seven-point checklist, the three-point checklist and the CASH algorithm in dermatoscopic evaluation of melanocytic lesions. J Dermatol 41(7):598–603

    Article  Google Scholar 

  68. Wang B, Qi G, Tang S, Zhang L, Deng L, Zhang Y (2018) Automated pulmonary nodule detection: high sensitivity with few candidates. MICCAI 2:759–767

    Google Scholar 

  69. Weifeng L, Xinghao Y, Dapeng T, Jun C, Yuanyan T (2018) Multiview dimension reduction via hessian multiset canonical correlations. Information Fusion 41:119–128

    Article  Google Scholar 

  70. Ye L, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action2Activity: Recognizing Complex Activities from Sensor Data. IJCAI'15 Proceedings of the 24th International Conference on Artificial Intelligence Pages 1617–1623, Buenos Aires, Argentina — July 25–31.

  71. Yoshida T et al (2016) Simple and effective pre-processing for automated melanoma discrimination based on cytological findings. In: Proceedings of the 2016 IEEE international conference on big data, pp 3439–3442

    Chapter  Google Scholar 

  72. Zortea M et al (2014) Performance of a dermoscopy-based computer vision system for the diagnosis of pigmented skin lesions compared with visual evaluation by experienced dermatologists. Artif Intell Med 60:13–26

    Article  Google Scholar 

Download references

Acknowledgements

This study was funded by Al Imam Mohammad Ibn Saud Islamic University (IMSIU) (grant number 360905).

Author information

Authors and Affiliations

  1. FR-85, College of Computer and Information Sciences, Al Imam Mohammad Ibn Saud Islamic University (IMSIU), P.O. 5701, Riyadh, 11432, Saudi Arabia

    Qaisar Abbas

  2. Department of Computer Science, University of Central Arkansas, Conway, AR, USA

    M. Emre Celebi

Authors
  1. Qaisar Abbas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Emre Celebi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qaisar Abbas.

Ethics declarations

Conflict of interest

Authors declare that they do not have conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abbas, Q., Celebi, M.E. DermoDeep-A classification of melanoma-nevus skin lesions using multi-feature fusion of visual features and deep neural network. Multimed Tools Appl 78, 23559–23580 (2019). https://doi.org/10.1007/s11042-019-7652-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7652-y

Keywords

Search

Navigation