Skip to main content

Advertisement

Springer Nature Link
Log in

Segmentation of skin lesions image based on U-Net + +

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the medical field, melanoma is one of the most dangerous skin cancers. However, the accuracy rate of doctors’ identification of melanoma is only 60%. Diagnosis requires higher technical experience and low fault tolerance for doctors who identify melanoma and other skin lesions. Therefore, the accurate segmentation of melanoma is of vital importance for clinical diagnosis and treatment. The current segmentation of melanoma is mainly based on fully connected networks (FCNs) and U-Net. Nevertheless, these two kinds of neural networks are prone to parameter redundancy, and the gradient disappears when depth increases, which reduces the Jaccard index of the skin lesion image segmentation model. To solve the above problems and improve the survival rate of melanoma patients, this paper proposes an improved skin lesion segmentation model based on U-Net++. In particular, we introduce a new loss function, which improves the Jaccard index of skin lesion image segmentation. The experiments show that our model has excellent performance on the segmentation of the ISIC2018 Task I dataset, and achieves a Jaccard index of 84.73%. The proposed method improves the Jaccard index of segmentation of skin lesion images and could also assist dermatological doctors in determining and diagnosing the types of skin lesions and the boundary between lesions and normal skin.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

References

  1. Abdeltawab H et al. (2019) Automatic Segmentation and Functional Assessment of the Left Ventricle using U-net Fully Convolutional Network, in 2019 IEEE International Conference on Imaging Systems and Techniques (IST): IEEE, pp. 1–6, doi: https://doi.org/10.1109/IST48021.2019.9010123.

  2. Al Nazi Z, Abir TA (2020) Automatic Skin Lesion Segmentation and Melanoma Detection: Transfer Learning Approach with U-Net and DCNN-SVM, in Proceedings of International Joint Conference on Computational Intelligence: Springer, pp. 371–381, doi: https://doi.org/10.1007/978-981-13-7564-4_32.

  3. Ali A-R, Li J, O’Shea SJ, Yang G, Trappenberg T, Ye X (2019) A Deep Learning Based Approach to Skin Lesion Border Extraction With a Novel Edge Detector in Dermoscopy Images," in 2019 International Joint Conference on Neural Networks (IJCNN): IEEE, pp. 1–7, doi: https://doi.org/10.1109/IJCNN.2019.8852134.

  4. Alom MZ, Hasan M, Yakopcic C, Taha TM, Asari VK (2018) Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation, arXiv preprint arXiv:1802.06955. [Online]. Available: https://arxiv.org/abs/1802.06955

  5. Amin J, Sharif A, Gul N, Anjum MA, Nisar MW, Azam F, Bukhari SAC (2020) Integrated design of deep features fusion for localization and classification of skin cancer. Pattern Recogn Lett 131:63–70. https://doi.org/10.1016/j.patrec.2019.11.042

    Article  Google Scholar 

  6. Ballerini L, Fisher RB, Aldridge B, Rees J (2013) A color and texture based hierarchical K-NN approach to the classification of non-melanoma skin lesions, in Color Medical Image Analysis: Springer, pp. 63–86.

  7. Bertels J et al. (2019) Optimizing the Dice score and Jaccard index for medical image segmentation: Theory and practice, in International Conference on Medical Image Computing and Computer-Assisted Intervention: Springer, pp. 92–100, doi: https://doi.org/10.1007/978-3-030-32245-8_11

  8. Celebi ME, Kingravi HA, Uddin B, Iyatomi H, Aslandogan YA, Stoecker WV, Moss RH (2007) A methodological approach to the classification of dermoscopy images. Comput Med Imaging Graph 31(6):362–373. https://doi.org/10.1016/j.compmedimag.2007.01.003

    Article  Google Scholar 

  9. Chan S, Reddy V, Myers B, Thibodeaux Q, Brownstone N, Liao W (2020) Machine learning in dermatology: current applications, opportunities, and limitations. Dermatol Ther 10:1–22. https://doi.org/10.1007/s13555-020-00372-0

    Article  Google Scholar 

  10. Cheng Y et al (2008) Skin lesion classification using relative color features. Skin Res Technol 14(1):53–64. https://doi.org/10.1111/j.1600-0846.2007.00261.x

    Article  Google Scholar 

  11. Codella N et al. (2019) Skin lesion analysis toward melanoma detection 2018: A challenge hosted by the international skin imaging collaboration (isic)," arXiv preprint arXiv:1902.03368. [Online]. Available: https://arxiv.org/abs/1902.03368

  12. Duan Y, Wang J, Hu M, Zhou M, Li Q, Sun L, Qiu S, Wang Y (2019) Leukocyte classification based on spatial and spectral features of microscopic hyperspectral images. Opt Laser Technol 112:530–538. https://doi.org/10.1016/j.optlastec.2018.11.057

    Article  Google Scholar 

  13. Duarte AF, Sousa-Pinto B, Freitas A, Delgado L, Costa-Pereira A, Correia O (2018) Skin cancer healthcare impact: a nation-wide assessment of an administrative database. Cancer Epidemiol 56:154–160. https://doi.org/10.1016/j.canep.2018.08.004

    Article  Google Scholar 

  14. Falk T, Mai D, Bensch R, Çiçek Ö, Abdulkadir A, Marrakchi Y, Böhm A, Deubner J, Jäckel Z, Seiwald K, Dovzhenko A, Tietz O, Dal Bosco C, Walsh S, Saltukoglu D, Tay TL, Prinz M, Palme K, Simons M, … Ronneberger O (2019) U-net: deep learning for cell counting, detection, and morphometry. Nat Methods 16(1):67–70

    Article  Google Scholar 

  15. Feigelson HS, Powers JD, Kumar M, Carroll NM, Pathy A, Ritzwoller DP (2019) Melanoma incidence, recurrence, and mortality in an integrated healthcare system: a retrospective cohort study. Cancer Med 8(9):4508–4516. https://doi.org/10.1002/cam4.2252

    Article  Google Scholar 

  16. Fong ZV, Tanabe KK (2014) Comparison of melanoma guidelines in the USA, Canada, Europe, Australia and New Zealand: a critical appraisal and comprehensive review. Br J Dermatol 170(1):20–30. https://doi.org/10.1111/bjd.12687

    Article  Google Scholar 

  17. Gandini S, Masala G, Palli D, Cavicchi B, Saieva C, Ermini I, Baldini F, Gnagnarella P, Caini S (2018) Alcohol, alcoholic beverages, and melanoma risk: a systematic literature review and dose-response meta-analysis. Eur J Nutr 57(7):2323–2332. https://doi.org/10.1007/s00394-018-1613-5

    Article  Google Scholar 

  18. Garbe C, Keim U, Eigentler TK, Amaral T, Katalinic A, Holleczek B, Martus P, Leiter U (2019) Time trends in incidence and mortality of cutaneous melanoma in Germany. J Eur Acad Dermatol Venereol 33(7):1272–1280. https://doi.org/10.1111/jdv.15322

    Article  Google Scholar 

  19. Gardner LJ, Strunck JL, Wu YP, Grossman D (2019) Current controversies in early-stage melanoma questions on incidence, screening, and histologic regression. J Am Acad Dermatol 80(1):1–12. https://doi.org/10.1016/j.jaad.2018.03.053

    Article  Google Scholar 

  20. L. Hu, S. F. Jin, L. Chen, and Y. L. Wang, "Trends in the incidence and mortality of cutaneous melanoma in Hong Kong between 1983 and 2015," Int J Clin Exp Med, vol. 11, no. 8, pp. 8259–8266, 2018. [Online]. Available: http://ijcem.com/files/ijcem0071882.pdf.

  21. Hua K-L, Hsu C-H, Hidayati SC, Cheng W-H, Chen Y-J (2015) Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets ther 8, doi: https://doi.org/10.2147/OTT.S80733

  22. Ibtehaz N, Rahman MS (2020) MultiResUNet: rethinking the U-net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87. https://doi.org/10.1016/j.neunet.2019.08.025

    Article  Google Scholar 

  23. Isaksson K et al Survival in 31 670 patients with thin melanomas: a Swedish population-based study. British J Dermatol. https://doi.org/10.1111/bjd.19015

  24. Jafari M, Auer D, Francis S, Garibaldi J, Chen X (2020) DRU-Net: An Efficient Deep Convolutional Neural Network for Medical Image Segmentation," in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI): IEEE, pp. 1144–1148, doi: https://doi.org/10.1109/ISBI45749.2020.9098391.

  25. Jain S, Pise N (2015) Computer aided melanoma skin cancer detection using image processing. Procedia Comput Sci 48:735–740. https://doi.org/10.1016/j.procs.2015.04.209

    Article  Google Scholar 

  26. Janka EA, Kékedi K, Várvölgyi T, Gellén E, Kiss B, Remenyik É, Emri G (Nov 2019) Increasing melanoma incidence in the elderly in north-East Hungary: is this a more serious problem than we thought? Eur J Cancer Prev 28(6):544–550. https://doi.org/10.1097/cej.0000000000000489

    Article  Google Scholar 

  27. Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD (2020) DoubleU-Net: A Deep Convolutional Neural Network for Medical Image Segmentation, arXiv preprint arXiv:2006.04868. [Online]. Available: https://arxiv.org/abs/2006.04868

  28. Kasmi R, Mokrani K (2016) Classification of malignant melanoma and benign skin lesions: implementation of automatic ABCD rule. IET Image Process 10(6):448–455. https://doi.org/10.1049/iet-ipr.2015.0385

    Article  Google Scholar 

  29. Kohl SA et al. (2018) A probabilistic u-net for segmentation of ambiguous images, arXiv preprint arXiv:1806.05034

  30. Leydesdorff L (2008) On the normalization and visualization of author co-citation data: Salton's cosine versus the Jaccard index. J Am Soc Inf Sci Technol 59(1):77–85. https://doi.org/10.1002/asi.20732

    Article  Google Scholar 

  31. Lucas RM, Yazar S, Young AR, Norval M, de Gruijl FR, Takizawa Y, Rhodes LE, Sinclair CA, Neale RE (2019) Human health in relation to exposure to solar ultraviolet radiation under changing stratospheric ozone and climate. Photochem Photobiol Sci 18(3):641–680. https://doi.org/10.1039/c8pp90060d

    Article  Google Scholar 

  32. Ma L, Shuai R, Ran X, Liu W, Ye C (2020) Combining DC-GAN with ResNet for blood cell image classification. Med Biol Eng Comput 58:1–14. https://doi.org/10.1007/s11517-020-02163-3

    Article  Google Scholar 

  33. Milletari F, Navab N, Ahmadi A (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation, in 2016 Fourth International Conference on 3D Vision (3DV): IEEE, pp. 565–571, doi: https://doi.org/10.1109/3DV.2016.79.

  34. Mishra NK, Celebi ME (2016) An overview of melanoma detection in dermoscopy images using image processing and machine learning, arXiv preprint arXiv:1601.07843. [Online]. Available: https://arxiv.org/abs/1601.07843.

  35. Oh CM, Cho H, Won YJ, Kong HJ, Roh YH, Jeong KH, Jung KW (2018) Nationwide trends in the incidence of melanoma and non-melanoma skin cancers from 1999 to 2014 in South Korea. Cancer Res Treat 50(3):729–737. https://doi.org/10.4143/crt.2017.166

    Article  Google Scholar 

  36. Oktay O et al. (2018) Attention u-net: Learning where to look for the pancreas, arXiv preprint arXiv:1804.03999

  37. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359. https://doi.org/10.1109/TKDE.2009.191

    Article  Google Scholar 

  38. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation, in International Conference on Medical image computing and computer-assisted intervention: Springer, pp. 234–241, doi: https://doi.org/10.1007/978-3-319-24574-4_28.

  39. Rustam F, Mehmood A, Ahmad M, Ullah S, Khan DM, Choi GS (2020) Classification of shopify app user reviews using novel multi text features. IEEE Access 8:30234–30244

    Article  Google Scholar 

  40. Sacchetto L, Zanetti R, Comber H, Bouchardy C, Brewster DH, Broganelli P, Chirlaque MD, Coza D, Galceran J, Gavin A, Hackl M, Katalinic A, Larønningen S, Louwman MWJ, Morgan E, Robsahm TE, Sanchez MJ, Tryggvadóttir L, Tumino R, … Rosso S (2018) Trends in incidence of thick, thin and in situ melanoma in Europe. Eur J Cancer 92:108–118. https://doi.org/10.1016/j.ejca.2017.12.024

    Article  Google Scholar 

  41. Saini S, Gupta D, Tiwari AK (2020) Detector-SegMentor Network for Skin Lesion Localization and Segmentation, arXiv preprint arXiv:2005.06550. [Online]. Available: https://arxiv.org/abs/2005.06550.

  42. Schadendorf D, van Akkooi ACJ, Berking C, Griewank KG, Gutzmer R, Hauschild A, Stang A, Roesch A, Ugurel S (2018) Melanoma. Lancet 392(10151):971–984. https://doi.org/10.1016/s0140-6736(18)31559-9

    Article  Google Scholar 

  43. Shen C et al. (2018) On the influence of Dice loss function in multi-class organ segmentation of abdominal CT using 3D fully convolutional networks," arXiv preprint arXiv:1801.05912. [Online]. Available: https://arxiv.org/abs/1801.05912

  44. Siegel RL, Miller KD, Jemal A (2019) Cancer statistics, 2019. CA Cancer J Clin 69(1):7–34. https://doi.org/10.3322/caac.21551

    Article  Google Scholar 

  45. Stromberg U et al (2016) Cutaneous malignant melanoma show geographic and socioeconomic disparities in stage at diagnosis and excess mortality. Acta Oncol 55(8):993–1000. https://doi.org/10.3109/0284186x.2016.1144934

    Article  Google Scholar 

  46. Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: A survey. J Machine Learn Res 10, no. Jul, pp. 1633–1685. [Online]. Available: http://www.jmlr.org/papers/volume10/taylor09a/taylor09a.pdf

  47. Telea A (2004) An image inpainting technique based on the fast marching method. J Graphics Tools 9(1):23–34. https://doi.org/10.1080/10867651.2004.10487596

    Article  Google Scholar 

  48. Thapar S, Garg S (2012) Study and implementation of various morphology based image contrast enhancement techniques," CSE Department, IT Department, pp. 2–5. [Online]. Available: https://pdfs.semanticscholar.org/aead/8e0c897e855b44f9a113fb4d6bb3acc3de8d.pdf?_ga=2.57754025.284154416.1594946891-935382206.1594946891

  49. Torrey L, Shavlik J (2010) Transfer learning, in Handbook of research on machine learning applications and trends: algorithms, methods, and techniques: IGI Global, pp. 242–264

  50. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions," Scientific data, vol. 5, p. 180161, doi: https://doi.org/10.1038/sdata.2018.161.

  51. Wang G et al (2014) Morphological background detection and illumination normalization of text image with poor lighting. PLoS One 9(11):e110991. https://doi.org/10.1371/journal.pone.0110991

    Article  Google Scholar 

  52. Wang L, Lee C-Y, Tu Z, Lazebnik S (2015) Training deeper convolutional networks with deep supervision, arXiv preprint arXiv:1505.02496. [Online]. Available: https://arxiv.org/abs/1505.02496.

  53. Wu J, Chen EZ, Rong R, Li X, Xu D, Jiang H (2019) Skin Lesion Segmentation with C-UNet," in 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC): IEEE, pp. 2785–2788, doi: https://doi.org/10.1109/EMBC.2019.8857773.

  54. Yang D, Salciccioli J, Marshall D, Sheri A, Shalhoub J (2020) Trends in malignant melanoma mortality in 31 countries from 1985 to 2015. Br J Dermatol 183:1056–1064. https://doi.org/10.1111/bjd.19010

    Article  Google Scholar 

  55. Yang S, Yu S, Zhao B, Wang Y (2020) Reducing the feature divergence of RGB and near-infrared images using Switchable Normalization," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 46–47. [Online]. Available: https://openaccess.thecvf.com/content_CVPRW_2020/papers/w5/Yang_Reducing_the_Feature_Divergence_of_RGB_and_Near-Infrared_Images_Using_CVPRW_2020_paper.pdf. [Online]. Available: https://openaccess.thecvf.com/content_CVPRW_2020/papers/w5/Yang_Reducing_the_Feature_Divergence_of_RGB_and_Near-Infrared_Images_Using_CVPRW_2020_paper.pdf

  56. Young K, Booth G, Simpson B, Dutton R, Shrapnel S (2019) Deep neural network or dermatologist?," in Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support: Springer, pp. 48–55

  57. Yu J, Blaschko M (2015) Learning submodular losses with the Lovász hinge, in International Conference on Machine Learning, pp. 1623–1631. [Online]. Available: https://hal.inria.fr/hal-01151823/file/YU_ICML2015.pdf. [Online]. Available: https://hal.inria.fr/hal-01151823/file/YU_ICML2015.pdf

  58. Yu J, Blaschko MB (2018) The lovász hinge: a novel convex surrogate for submodular losses. IEEE Trans Pattern Anal Mach Intell 42:735–748. https://doi.org/10.1109/TPAMI.2018.2883039

    Article  Google Scholar 

  59. Yu L, Chen H, Dou Q, Qin J, Heng P-A (2016) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004. https://doi.org/10.1109/TMI.2016.2642839

    Article  Google Scholar 

  60. Zhang L, Yang G, Ye X (2019) Automatic skin lesion segmentation by coupling deep fully convolutional networks and shallow network with textons. J Med Imaging 6(2):024001. https://doi.org/10.1117/1.JMI.6.2.024001

    Article  Google Scholar 

  61. Zhang N, Cai Y-X, Wang Y-Y, Tian Y-T, Wang X-L, Badami B (2020) Skin cancer diagnosis based on optimized convolutional neural network. Artif Intell Med 102:101756. https://doi.org/10.1016/j.artmed.2019.101756

    Article  Google Scholar 

  62. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation, in Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: Springer, pp. 3–11

  63. Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2019) UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867. https://doi.org/10.1109/TMI.2019.2959609

    Article  Google Scholar 

  64. Zhu L, Feng S, Zhu W, Chen X (2020) ASNet: An adaptive scale network for skin lesion segmentation in dermoscopy images, in Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging, vol. 11317: International Society for Optics and Photonics, p. 113170W, doi: https://doi.org/10.1117/12.2549178.

Download references

Acknowledgments

Chen Zhao contributed to the writing and editing of the paper and the operation and editing of the code. Renjun Shuai (corresponding author) contributed to technological guidance and provided experimental equipment and major financial support. Li Ma contributed technical support and guidance for the paper concept. Wenjia Liu contributed to the technical guidance, and as a consultant in the medical consultant field, Die Hu contributed part of the paper correction, and Menglin Wu contributed to the direction of the paper and the funding of support and related work.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Nanjing Tech University, Nanjing, 211816, China

    Chen Zhao, Renjun Shuai & Menglin Wu

  2. Nanjing Health Information Center, Nanjing, 210003, China

    Li Ma

  3. Changzhou No. 2 People’s Hospital affiliated to Nanjing Medical University, Changzhou, 213003, China

    Wenjia Liu

Authors
  1. Chen Zhao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Renjun Shuai
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Li Ma
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Wenjia Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Menglin Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Renjun Shuai.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported in part by The National Natural Science Foundation of China NO.61701222.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, C., Shuai, R., Ma, L. et al. Segmentation of skin lesions image based on U-Net + +. Multimed Tools Appl 81, 8691–8717 (2022). https://doi.org/10.1007/s11042-022-12067-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12067-z

Keywords