Abstract
Skin cancer is the most prevalent genre of all cancers. Melanoma, being the deadliest of all skin cancers, calls for the requirement of an automated Artificial Intelligence-based skin diagnosis system to assist physicians with early diagnosis. We propose a fusion of conventional therapeutic approaches and deep learning frameworks to identify skin lesions. The work explores the scope of employing image data, handcrafted lesion features, and patient-centric metadata together to diagnose skin cancers effectively. We combined the image features transfer-learned from EfficientNets, colour and texture information extracted from the images, and patients’ preprocessed metadata to produce the final hybrid model. They were fed to a multi-input single-output (MISO) model to fine-tune an artificial neural network classifier. Multiple MISO models were trained with their backbones substituted with EfficientNets B4 through B7. The predicted labels from these, along with a separate set of models trained with only image data and metadata were ensembled using majority soft voting. We experimented with weighing the models based on their contribution to ensemble accuracy and ensemble sensitivity. Each model was trained and evaluated using the well-known ISIC2018 and ISIC2019 datasets. The extreme imbalance in the datasets necessitates the use of appropriate evaluation metrics. ISIC2018 tested 90.49% sensitive and 97.76% specific, whereas the larger and more divergent dataset ISIC2019 rated 85.58% sensitive and 98.29% specific. The network is by far the finest compared to most other research in the field.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Almaraz-Damian J. -A., Ponomaryov V., Sadovnychiy S., Castillejos-Fernandez H. (2020) Melanoma and nevus skin lesion classification using handcraft and deep learning feature fusion via mutual information measures. Entropy 22(4):484
Anand H. S., Vinod Chandra S. S. (2016) Association rule mining using treap. International Journal of Machine Learning and Cybernetics 9(4):589–597
Aswathy A. L., Anand H. S., Vinod Chandra S. S. (2021) Covid-19 diagnosis and severity detection from ct-images using transfer learning and back propagation neural network. Journal of Infection and Public Health 14(10):1435–1445
Codella N.C., Gutman D., Celebi M.E., Helba B., Marchetti M.A., Dusza S.W., Kalloo A., Liopyris K., Mishra N., Kittler H., et al. (2018) Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic). In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 168–172. IEEE
Combalia M., Codella N. C., Rotemberg V., Helba B., Vilaplana V., Reiter O., Carrera C., Barreiro A., Halpern A. C., Puig S., et al. (2019) Bcn20000: Dermoscopic lesions in the wild. arXiv:1908.02288
Dugonik B., Dugonik A., Marovt M., Golob M. (2020) Image quality assessment of digital image capturing devices for melanoma detection. Appl. Sci. 10(8):2876
Gessert N., Nielsen M., Shaikh M., Werner R., Schlaefer A. (2020) Skin lesion classification using ensembles of multi-resolution efficientnets with meta data. MethodsX 7:100864
Gessert N., Sentker T., Madesta F., Schmitz R., Kniep H., Baltruschat I., Werner R., Schlaefer A. (2018) Skin lesion diagnosis using ensembles, unscaled multi-crop evaluation and loss weighting. arXiv:1808.01694
Ghalejoogh G. S., Kordy H. M., Ebrahimi F. (2020) A hierarchical structure based on stacking approach for skin lesion classification. Expert Syst. Appl. 145:113127
Gong A., Yao X., Lin W. (2020) Classification for dermoscopy images using convolutional neural networks based on the ensemble of individual advantage and group decision. IEEE Access 8:155337–155351
Guissous A. E. (2019) Skin lesion classification using deep neural network. arXiv:1911.07817
Ha Q., Liu B., Liu F. (2020) Identifying melanoma images using efficientnet ensemble: Winning solution to the siim-isic melanoma classification challenge. arXiv:2010.05351
Hameed N., Shabut A. M., Ghosh M. K., Hossain M. A. (2020) Multi-class multi-level classification algorithm for skin lesions classification using machine learning techniques. Expert Syst. Appl. 141:112961
Haralick R. M., Shanmugam K., Dinstein I. H. (1973) Textural features for image classification. IEEE Transactions on systems, man, and cybernetics (6), pp. 610–621
Harangi B. (2018) Skin lesion classification with ensembles of deep convolutional neural networks. Journal of biomedical informatics 86:25–32
ISIC Challenge. https://challenge.isic-archive.com/
ISIC Challenge Datasets. https://challenge.isic-archive.com/data/
Kassem M. A., Hosny K. M., Fouad M. M. (2020) Skin lesions classification into eight classes for isic 2019 using deep convolutional neural network and transfer learning. IEEE Access 8:114822–114832
Milton M. A. A. (2019) Automated skin lesion classification using ensemble of deep neural networks in isic 2018: Skin lesion analysis towards melanoma detection challenge. arXiv:1901.10802
Monika M. K., Vignesh N. A., Kumari C. U., Kumar M., Lydia E. L. (2020) Skin cancer detection and classification using machine learning. Materials Today: Proceedings 33:4266–4270
Nahata H., Singh S. P. (2020) Deep learning solutions for skin cancer detection and diagnosis. Machine Learning with Health Care Perspective, pp. 159–182
Shahin A.H., Kamal A., Elattar M.A. (2018) Deep ensemble learning for skin lesion classification from dermoscopic images. In: 2018 9th Cairo International Biomedical Engineering Conference (CIBEC), pp. 150–153 . IEEE
Sondermann W., Zimmer L., Schadendorf D., Roesch A., Klode J., Dissemond J. (2016) Initial misdiagnosis of melanoma located on the foot is associated with poorer prognosis. Medicine 95(29)
Steppan J., Hanke S. (2021) Analysis of skin lesion images with deep learning. arXiv:2101.03814
Tan M., Le Q. (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 . PMLR
Tschandl P., Rosendahl C., Kittler H. (2018) The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific data 5(1):1–9
Valiuddin M. (2019) Using the efficientnet convolutional neural network architecture for skin lesion analysis and melanoma detection a submission for the ISIC2019 challenge
Zghal N. S., Derbel N. (2020) Melanoma skin cancer detection based on image processing. Current Medical Imaging 16(1):50–58
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
This article does not contain any studies with human participants or animals performed by the author.
Conflict of interests
The authors declare that they have no conflict of interest.
Consent for publication
Yes
Additional information
Availability of data and materials
The datasets are downloaded from the ISIC repository https://challenge.isic-archive.com/data/
Code availability
http://mirworks.in/downloads.php
Consent to participate
Not applicable
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Sharafudeen, M., S., V.C.S. Detecting skin lesions fusing handcrafted features in image network ensembles. Multimed Tools Appl 82, 3155–3175 (2023). https://doi.org/10.1007/s11042-022-13046-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-13046-0