Skip to main content

Advertisement

SpringerLink
Log in

Nested segmentation and multi-level classification of diabetic foot ulcer based on mask R-CNN

  • Track 2: Medical Applications of Multimedia
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A diabetic foot ulcer(DFU) is a common chronic complication of diabetes because of the dysfunction of islets or receptors of insulin, and it has a high disability and mortality rate. Measuring diabetic foot ulcers is also one of the popular application areas where computer vision combines with deep learning techniques. However, some remaining defects in these studies prevent them from accurately visualizing the wound of different severity. Based on this, we used a multi-classification model to mark the wounds into five grades according to the Wagner diabetic foot grading method. It segmented the different grades in each different level wound using colorfully nested ring shapes to reflect the gradual change of wound grades. We collected 1426 DFU images, of which 967 had nested labels and 459 were single-level labels, with images marked with colored rings to show different degrees of wounds. And then, we constructed a deep learning model of diabetes foot ulcer wounds for semantic segmentation based on Mask Region-based convolutional neural networks (Mask R-CNN), and obtain different levels of diabetes nested segmentation results to reflect the different severity in one wound. Finally, we test and evaluate the performance data of the model. Compared with the state-of- the-art results concerning segmentation and classification and diagnosis of diabetic foot wounds, our model has achieved better performance data (specificity = 99.50%, sensitivity = 70.62%, precision = 84.56%, Mean Average Precision = 85.70%). It shows the effectiveness of our nested segmentation and multi-level classification method. It provides some suggestions and directions for the subsequent evaluation and diagnosis and treatment of diabetic foot ulcers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abdissa D, Adugna T, Gerema U, Dereje D (2020) Prevalence of diabetic foot ulcer and associated factors among adult diabetic patients on follow-up clinic at jimma medical center, southwest ethiopia, 2019: an institutional-based cross-sectional study. J Diabetes Res 2020:1–6. https://doi.org/10.1155/2020/4106383

    Article  Google Scholar 

  2. Al-Rubeaan K, Almashouq MK, Youssef AM, Al- Qumaidi H, Al Derwish M, Ouizi S, Al-Shehri K, Masoodi SN (2017) All-cause mortality among diabetic foot patients and related risk factors in saudi arabia. PLoS ONE 12(11):e0188097. https://doi.org/10.1371/journal.pone.0188097

    Article  Google Scholar 

  3. Alsabek MB, Abdul AR, Aziz (2022) Diabetic foot ulcer, the effect of resource-poor environments on healing time and direct cost: a cohort study during syrian crisis. Int Wound J 19(3):531–537. https://doi.org/10.1111/iwj.13651

  4. Cassidy B, Reeves ND, Pappachan JM, Gillespie D, OShea C, Rajbhandari S, Maiya AG, Frank E, Boulton AJ, Armstrong DG, Najafi B, Wu J, Moi HY (2021) The DFUC 2020 dataset: analysis towards diabetic foot ulcer detection. touchREV Endocrinol 17(1):5–11. https://doi.org/10.17925/EE.2021.17.1.5

    Article  Google Scholar 

  5. Chiao J-Y, Chen K-Y, Liao KY-K, Hsieh P-H, Zhang G, Huang T-C (2019) Detection and classification the breast tumors using Mask R-CNN on sonograms. Medicine 98(19):e15200. https://doi.org/10.1097/MD.0000000000015200

    Article  Google Scholar 

  6. Cui C, Thurnhofer-Hemsi K, Soroushmehr R, Mishra A, Gryak J, Domínguez E, Najarian K, López-Rubio E (2019) Diabetic wound segmentation using convolutional neural networks. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 1002–1005. https://doi.org/10.1109/EMBC.2019.8856665

  7. Dmitriyeva M, Kozhakhmetova Z, Urazova S, Kozhakhmetov S, Turebayev D, Toleubayev M (2022) Inflammatory biomarkers as predictors of infected diabetic foot ulcer. Curr Diabetes Rev 18(6):4–11. https://doi.org/10.2174/1573399817666210928144706

    Article  Google Scholar 

  8. Girshick R (2015) Fast R-CNN. In: Proceedings of the IEEE inter- national conference on computer vision, pp 1440–1448. https://doi.org/10.1109/ICCV.2015.169

  9. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 580–587. https://doi.org/10.1109/CVPR.2014.81

  10. Gourishetti K, Keni R, Nayak PG, Jitta SR, Bhaskaran NA, Kumar L, Kumar N, Krishnadas N, Shenoy RR (2020) Sesamol-loaded plga nanosuspension for accelerating wound healing in diabetic foot ulcer in rats. Int J Nanomed 15:9265–9282. https://doi.org/10.2147/IJN.S268941

    Article  Google Scholar 

  11. Goyal M, Yap MH, Reeves ND, Rajbhandari S, Spragg J (2017) Fully convolutional networks for diabetic foot ulcer segmentation. In: 2017 IEEE international conference on systems, man, and cybernetics (SMC), pp 618–623. https://doi.org/10.1109/SMC.2017.8122675

  12. Goyal M, Reeves ND, Rajbhandari S, Yap MH (2018) Robust methods for real-time diabetic foot ulcer detection and localization on mobile devices. IEEE J Biomed Health Inf 23(4):1730–1741. https://doi.org/10.1109/JBHI.2018.2868656

    Article  Google Scholar 

  13. Goyal M, Reeves ND, Davison AK, Rajbhandari S, Spragg J, Yap MH (2018) DFUnet: convolutional neural networks for diabetic foot ulcer classification. IEEE Trans Emerg Topics Comput Intell 4(5):728–739. https://doi.org/10.1109/TETCI.2018.2866254

  14. Goyal M, Reeves ND, Rajbhandari S, Ahmad N, Wang C, Yap MH (2020) Recognition of ischaemia and infection in diabetic foot ulcers: dataset and techniques. Comput Biol Med 117:103616. https://doi.org/10.1016/j.compbiomed.2020.103616

  15. Gribbon KT, Bailey DG (2004) A novel approach to real-time bilinear interpolation. In: Proceedings. DELTA 2004. Second IEEE international workshop on electronic design, test and applications, pp 126–131. https://doi.org/10.1109/DELTA.2004.10055

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  17. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. In: Proceedings of the IEEE international conference on computer vision, pp 2961–2969. https://doi.org/10.1109/ICCV.2017.322

  18. Joulin A, Cissé M, Grangier D, Jégou H et al (2017) Efficient softmax approximation for GPUs. In: Proceedings of international conference on machine learning, pp 1302–1310. https://doi.org/10.48550/arXiv.1609.04309

  19. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. Proceedings of international conference for learning representations, pp 1–15. https://doi.org/10.48550/arXiv.1412.6980

  20. Kouitcheu Mabeku LB, Noundjeu Ngamga ML, Leundji H (2020) Helicobacter pylori infection, a risk factor for type 2 diabetes mellitus: a hospital-based cross-sectional study among dyspeptic patients in doualacameroon. Sci Rep 10(1):1–11. https://doi.org/10.1038/s41598-020-69208-3

    Article  Google Scholar 

  21. Lavery LA, Armstrong DG, Harkless LB (1996) Classification of diabetic foot wounds. J Foot Ankle Surg 35(6):528–531. https://doi.org/10.1016/s1067-2516(96)80125-6

    Article  Google Scholar 

  22. LeCun Y, Boser B, Denker J, Henderson D, Howard R, Hubbard W, Jackel L (1989) Handwritten digit recognition with a back-propagation network. Adv Neural Inf Process Syst 2:396–404. https://doi.org/10.5555/2969830.2969879

    Article  Google Scholar 

  23. Lin T-Y, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125. https://doi.org/10.1109/CVPR.2017.106

  24. Lu C, Kong Y, Guan Z (2020) A Mask R-CNN model for reidentifying extratropical cyclones based on quasi-supervised thought. Sci Rep 10(1):1–9. https://doi.org/10.1038/s41598-020-71831-z

    Article  Google Scholar 

  25. Malerbi FK, Mendes G, Barboza N, Morales PH, Montargil R, Andrade RE (2022) Diabetic macular edema screened by handheld smartphone-based retinal camera and artificial intelligence. J Med Syst 46(1):1–6. https://doi.org/10.1007/s10916-021-01795-8

    Article  Google Scholar 

  26. Ohura N, Mitsuno R, Sakisaka M, Terabe Y, Morishige Y, Uchiyama A, Okoshi T, Shinji I, Takushima A (2019) Convolutional neural networks for wound detection: the role of artificial intelligence in wound care. J Wound Care 28(10):13–24. https://doi.org/10.12968/jowc.2019.28.Sup10.S13

    Article  Google Scholar 

  27. Rastogi A, Goyal G, Kesavan R, Bal A, Kumar H, Kamath P, Jude EB, Armstrong DG, Bhansali A et al (2020) Long term outcomes after incident diabetic foot ulcer: Multicenter large cohort prospective study (edi-focus investigators) epidemiology of diabetic foot complications study: epidemiology of diabetic foot complications study. Diabetes Res Clin Pract 162:108–113. https://doi.org/10.1016/j.diabres.2020.108113

  28. Rebecca R, Dominic S, Boyoung K, James M, Williams D, Leslie A (2020) The diabetic foot ulcer. Australian J Gen Practitioners 49(5):250–255. https://doi.org/10.31128/AJGP-11-19-5161

    Article  Google Scholar 

  29. Ren S, He K, Girshick R, Sun J, Faster R-CNN (2015) Towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28:91–99. https://doi.org/10.1109/TPAMI.2016.2577031

    Article  Google Scholar 

  30. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Proceedings of International Conference on Medical image computing and computer-assisted intervention, pp 234–241. https://doi.org/10.1007/978-3-319-24574-428

  31. Rümenapf G, Morbach S, Rother U, Uhl C, Görtz H, Böckler D, Behrendt C-A, Hochlenert D, Engels G, Sigl M et al (2021) Diabetisches fußsyndrom–teil 1: definition, pathophysiologie, diagnostik und klassifikation, Der Chirurg. Z Fur Alle Gebiete Der Operativen Medizen 92(1):81–94. https://doi.org/10.1007/s00104-020-01301-9

  32. Syed MH, Salata K, Hussain MA, Zamzam A, de Mestral C, Wheatcroft M, Harlock J, Awartani D, Aljabri B, Verma A et al (2020) The economic burden of inpatient diabetic foot ulcers in Toronto, Canada. Vascular 28(5):520–529. https://doi.org/10.1177/1708538120923420

  33. Wagner FW Jr (1981) The dysvascular foot: a system for diagnosis and treatment. Foot Ankle 2(2):64–122. https://doi.org/10.1177/107110078100200202

    Article  Google Scholar 

  34. Wang P, Ge R, Xiao X, Cai Y, Wang G, Zhou F (2017) Rectified- linear-unit-based deep learning for biomedical multi-label data. Interdiscip Sci Comput Life Sci 9(3):419–422. https://doi.org/10.1007/s12539-016-0196-1

    Article  Google Scholar 

  35. Wang T, Zhong L, Yuan J, Wang T, Yin S, Sun Y, Li- u X, Liu X, Ling S (2020) Quantitative analysis of functional filtering bleb size using Mask R-CNN. Ann Transl Med 8(11):709–717. https://doi.org/10.21037/atm.2020.03.135

    Article  Google Scholar 

  36. Woldemariam GT, Atnafu NT, Radie YT, Wolde GT, Gebreagziabher TT, Gebrehiwot TG, Teka YH, Hagos MG, Hagezom HM, Yigzaw HB et al (2020) Determinants of diabetic foot ulcer among adult patients with diabetes attending the diabetic clinic in tikur anbessa specialized hospital, addis ababa, ethiopia: Unmatched case–control study. Diabetes Metab Syndr Obes: Targets Ther 13:3739–3747. https://doi.org/10.2147/DMSO.S265988

    Article  Google Scholar 

  37. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1492–1500. https://doi.org/10.1109/CVPR.2017.634

  38. Yap MH, Hachiuma R, Alavi A, Brüngel R, Cassidy B, Goyal M, Zhu H, Rückert J, Olshansky M, Huang X et al (2021) Deep learning in diabetic foot ulcers detection: a comprehensive evaluation. Comput Biol Med 135:104596. https://doi.org/10.1016/j.compbiomed.2021.104596

    Article  Google Scholar 

  39. Yap MH, Cassidy B, Pappachan JM, O’Shea C, Gillespie D, Reeves N (2021) Analysis towards classification of infection and ischaemia of diabetic foot ulcers. In: 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), pp 1–4. https://doi.org/10.1109/BHI50953.2021.9508563

Download references

Acknowledgements

We are grateful to all patients involved in all trials and investigators for their previous work that enabled the present study. This study was supported by the Hunan Province Natural Science Foundation (grant number 2022JJ30673), Scientific Research Fund of Hunan Provincial Education Department (grant number 20C0402), Hunan First Normal University (grant number XYS16N03), the Projects of the National Natural Science Foundation of China (grant number 82073018), the Shenzhen Science and Technology Innovation Commission (Natural Science Foundation of Shenzhen, grant number JCYJ20210324113001005), Management Research Fund of Xiangya Hospital of Central South University (grant number 2021GL11).

Author information

Author notes

  1. Cong Cao and Yue Qiu contributed equally to this work.

Authors and Affiliations

  1. School of Mathematics and Statistics, Central South University, Changsha, 410083, China

    Cong Cao, Zheng Wang, Jiaoju Wang, Alphonse Houssou Hounye & Muzhou Hou

  2. Department of Dermatology of Xiangya Hospital, Central South University, Changsha, 410008, China

    Yue Qiu & Jiarui Ou

  3. Science and Engineering School, Hunan First Normal University, Changsha, 410205, China

    Zheng Wang

  4. Teaching and Research Section of Clinical Nursing, Xiangya Hospital of Central South University, Changsha, 410008, China

    Qiuhong Zhou

  5. Department of Detmatology, Shenzhen Peoples Hospital, The Second Clinical Medical College, Jinan University, The First Affiliated Hospital, Southern University of Science and Technology, Shenzhen, 518020, Guangdong, China

    Jianglin Zhang

Authors
  1. Cong Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yue Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiarui Ou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiaoju Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alphonse Houssou Hounye
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Muzhou Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qiuhong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jianglin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Muzhou Hou, Qiuhong Zhou or Jianglin Zhang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

I confirm I have included a data availability statement in my main manuscript file.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, C., Qiu, Y., Wang, Z. et al. Nested segmentation and multi-level classification of diabetic foot ulcer based on mask R-CNN. Multimed Tools Appl 82, 18887–18906 (2023). https://doi.org/10.1007/s11042-022-14101-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14101-6

Keywords

Search

Navigation