Skip to main content

Advertisement

Springer Nature Link
Log in

An efficient U-shaped network combined with edge attention module and context pyramid fusion for skin lesion segmentation

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Skin lesion segmentation is an important process in skin diagnosis, but still a challenging problem due to the variety of shapes, colours, and boundaries of melanoma. In this paper, we propose a novel and efficient U-shaped network named EAM-CPFNet, which combines with edge attention module (EAM) and context pyramid fusion (CPF) to improve the performance of the skin lesion segmentation. First, we design a plug-and-play module named edge attention module (EAM), which is used to highlight the edge information learned in the encoder. Secondly, we integrate two pyramid modules collectively named context pyramid fusion (CPF) for context information fusion. One is multiple global pyramid guidance (GPG) modules, which replace the skip connections between the encoder and the decoder to capture global context information, and the other is scale-aware pyramid fusion (SAPF) module, which is designed to dynamically fuse multi-scale context information in high-level features by utilizing spatial and channel attention mechanisms. Furthermore, we introduce full-scale skip connections to enhance different levels of global context information. We evaluate the proposed method on the publicly available ISIC2018 dataset, and the experimental results demonstrate that our proposed method is very competitive compared with other state-of-the-art methods for the skin lesion segmentation.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Siegel RL, Miller KD (2018) Jemal A (2018) Cancer statistics. CA Cancer J Clin 60(Suppl 12):277–300

    Google Scholar 

  2. Zanddizari H, Nguyen N, Zeinali B, Chang JM (2021) A new preprocessing approach to improve the performance of CNN-based skin lesion classification. Med Biol Eng Compu 59(5):1123–1131

    Article  Google Scholar 

  3. Balch CM, Gershenwald JE, Soong SJ, Thompson JF et al (2009) Final version of 2009 AJCC melanoma staging and classification. J Clin Oncol 27(36):6199–6206

    Article  Google Scholar 

  4. Huang S, Lee FF, Miao R, Si Q, Lu CW, Chen Q (2020) A deep convolutional neural network architecture for interstitial lung disease pattern classification. Med Biol Eng Compu 58(4):725–737

    Article  Google Scholar 

  5. Wu H, Pan J, Li Z, Wen Z, Qin J (2021) Automated skin lesion segmentation via an adaptive dual attention module. IEEE Trans Med Imag 40(1):357–370

    Article  Google Scholar 

  6. Sez A, Serrano C, Acha B (2014) Model-based classification methods of global patterns in dermoscopic images. IEEE Trans Med Imag 33(5):1137–1147

    Article  Google Scholar 

  7. Hu K, Liu S, Zhang Y, Cao C, Xiao F, Huang W, Gao X (2020) Automatic segmentation of dermoscopy images using saliency combined with adaptive thresholding based on wavelet transform. Multimed Tools Appl 79(21–22):14625–145642

    Article  Google Scholar 

  8. Khadidos A Sanchez V Li C (2014) Active contours based on weighted gradient vector flow and balloon forces for medical image segmentation. In: IEEE International Conference on Image Processing, pp 902–906

  9. Long J Shelhamer E Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 3431–3440

  10. Ronneberger O Fischer P Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 234–241

  11. Dash M, Londhe ND, Ghosh S, Semwal A, Sonawane RS (2019) Pslsnet: Automated psoriasis skin lesion segmentation using modified u-net-based fully convolutional network. Biomed Signal Process Control 52:226–237

    Article  Google Scholar 

  12. Lbtehaz N, Rahman MS (2020) MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    Article  Google Scholar 

  13. Szegedy C Liu W Jia Y Sermanet P et al. (2015) Going deeper with convolutions. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1–9

  14. Oktay O Schlemper J Folgoc LL Lee MCH et al. (2018) Attention U-Net: learning where to look for the pancreas. arXiv:1804.03999

  15. Arora R, Raman B, Nayyar K, Awasthi R (2021) Automated skin lesion segmentation using attention-based deep convolutional neural network. Biomed Signal Process Control 65:102358

    Article  Google Scholar 

  16. Zhang J, Xie Y, Xia Y, Shen C (2019) Attention residual learning for skin lesion classification. IEEE Trans Med Imag 38(9):2092–2103

    Article  Google Scholar 

  17. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  18. Pohlen T Hermans A Mathias M Leibe B (2017) Full-resolution residual networks for semantic segmentation in street scenes. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 4151–4160

  19. Milletari F Navab N Ahmadi SA (2016) V-net: Fully convolutional neural networks for volumetric medical image segmentation. In: Fourth International Conference on 3D Vision, pp 565–571

  20. Çiçek Ö Abdulkadir A Lienkamp SS Brox T Ronneberger O (2016) 3D u-net: learning dense volumetric segmentation from sparse annotation. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 424–432

  21. Alom MZ Hasan M Yakopcic C Taha TM Asari VK (2018) Recurrent residual convolutional neural network based on u-net (r2u-net) for medical image segmentation. arXiv:1804.03999

  22. Pawan SJ, Sankar R, Jain A et al (2021) Capsule network–based architectures for the segmentation of sub-retinal serous fluid in optical coherence tomography images of central serous chorioretinopathy. Med Biol Eng Compu 59(6):1245–1259

    Article  CAS  Google Scholar 

  23. Zhou Z Siddiquee MMR Tajbakhsh N Liang J (2018) Unet++: A nested u-net architecture for medical image segmentation. In Deep learning in medical image analysis and multimodal learning for clinical decision support, Springer, 3–11

  24. Geirhos R Rubisch P Michaelis C Bethge M Wichmann FA Brendel W (2019) ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In: International Conference on Learning Representations

  25. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP et al (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Imag Anal 36:61–78

    Article  Google Scholar 

  26. Hatamizadeh A Terzopoulos D Myronenko A (2020) Edge-gated CNNs for volumetric semantic segmentation of medical images. arXiv:1804.03999

  27. Sun J Darbehani F Zaidi M Wang B (2020) SAUNet: shape attentive U-Net for interpretable medical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, 797–806

  28. Zhang Z Fu H Dai H Shen J Pang Y Shao L (2019) ET-Net: A generic edge-attention guidance network for medical image segmentation. In: International conference on medical image computing and computer-assisted intervention, Springer, 442–450

  29. Vaswani A Shazeer N Parmar N Uszkoreit J et al. (2017) Attention is all you need. In: Proc Adv Neural Inf Process Syst, 6000–6010

  30. Huang G, Zhu J, Li J, Wang Z et al (2020) Channel-attention U-Net: channel attention mechanism for semantic segmentation of esophagus and esophageal cancer. IEEE Access 8:122798–122810

    Article  Google Scholar 

  31. He K Zhang X Ren S Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE Conf Comput Vis Pattern Recognit, 770–778

  32. Huang G Liu Z Laurens VDM Weinberger KQ (2017) Densely connected convolutional networks. In: Proc IEEE Conf Comput Vis Pattern Recognit, 4700–4708

  33. Guan S, Khan A, Sikdar S, Chitnis PV (2020) Fully dense UNet for 2D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Informat 24(2):568–576

    Article  Google Scholar 

  34. Zhu Z Xu M Bai S Huang T Bai X (2019) Asymmetric non-local neural networks for semantic segmentation. In Proc IEEE Int Conf Comput Vis, 593–602

  35. Feng S, Zhao H, Shi F, Cheng X et al (2020) CPFNet: Context pyramid fusion network for medical image segmentation. IEEE Trans Med Imag 39(10):3008–3018

    Article  Google Scholar 

  36. Woo S Park J Lee JY Kweon IS (2018) Cbam: convolutional block attention module. In Proc Eur Conf Comput Vis, 3–19

  37. Paszke A Gross S Chintala S Chanan G Yang E DeVito Z Lin Z Desmaison A Antiga L Lerer A (2017) Automatic differentiation in pytorch. In NIPS Workshop

  38. Codella NCF Gutman D Celebi ME Helba B et al. (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15th Int Symp Biomed Imag, 168–172

  39. Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161

    Article  Google Scholar 

  40. Codella N Rotemberg V Tschandl P Celebi ME et al. (2019) Skin lesion analysis toward melanoma detection 2018: a challenge hosted by the international skin imaging collaboration (ISIC). arXiv:1902.03368

  41. Wu H Zhang J Huang K Liang K Yu Y (2019) FastFCN: rethinking dilated convolution in the backbone for semantic segmentation. arXiv:1902.03368

  42. Gu Z, Cheng J, Fu H, Zhou K et al (2019) CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans Med Imag 38(10):2281–2292

    Article  Google Scholar 

  43. Peng C Zhang X Yu G Luo G Sun J (2017) Large kernel matters- Improve semantic segmentation by global convolutional network. In: Proc IEEE Conf Comput Vis Pattern Recognit, 4353–4361

  44. Yu C Wang J Peng C Gao C Yu G Sang N (2018) Learning a discriminative feature network for semantic segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, 1857–1866

  45. Altaf F, Islam SMS, Akhtar N, Janjua NK (2019) Going deep in medical image analysis: concepts, methods, challenges and future directions. IEEE Access 7:99540–99572

    Article  Google Scholar 

  46. Jha D Riegler MA Johansen D Halvorsen P Johansen HD (2020) DoubleU-Net: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd International Symposium on Computer-Based Medical System, 558–564

  47. Sarker MMK Rashwan HA Akram F Banu SF et al. (2018) Slsdeep: skin lesion segmentation based on dilated residual and pyramid pooling networks. In: International conference on medical image computing and computer-assisted intervention, Springer, 21–29

  48. Chen LC Zhu Y Papandreou G Schroff F Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmenstation. In Proc Eur Conf Comput Vis, 801–818

  49. Park J Woo S Lee JY Kweon IS (2018) Bam: bottleneck attention module. arXiv:1807.06514

  50. Fu J Liu J Tian H Li Y Bao Y Fang Z Lu H (2019) Dual attention network for scene segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, 3146–3154

  51. Liu L, Ying YT, Mrinal M (2021) Skin lesion segmentation using deep learning with auxiliary task. J Imaging 7(4):67

    Article  Google Scholar 

  52. Yuan Y (2017) Automatic skin lesion segmentation with fully convolutional-deconvolutional networks. arXiv:1703.05165

  53. Al-Masni MA, Al-Antari MA, Choi MT et al (2018) Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks. Comput Methods Programs Biomed 162:221–231

    Article  Google Scholar 

  54. Zhao H Shi J Qi X Wang X Jia J. (2017) Pyramid scene parsing network. In: Proc IEEE Conf Comput Vis Pattern Recognit, 2881–2890.

  55. Jha D Smedsrud PH Riegler MA Halvorsen P et al. (2020) Kvasir-seg: a segmented polyp dataset. In: International Conference on Multimedia Modeling, Springer, 451–462

  56. Bernal J, Sanchez FJ, Fernandez-Esparrach G, Gil D, Rodrıguez C, Vilarino F (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput Med Imag Graph 43:99–111

    Article  Google Scholar 

  57. Jha D Smedsrud PH Riegler MA Johansen D Lange TD Halvorsen P Johansen HD (2019) Resunet++: an advanced architecture for medical image segmentation. In: IEEE International Symposium on Multimedia, 225–2255

  58. Zhang Z, Liu Q, Wang Y (2018) Road extraction by deep residual u-net. IEEE Geosci Remote Sens Lett 15(5):749–753

    Article  CAS  Google Scholar 

Download references

Author information

Author notes

  1. Bin Zuo and Feifei Lee contributed equally.

Authors and Affiliations

  1. Shanghai Engineering Research Center of Assistive Devices, School of Medical Instrument and Food Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Bin Zuo & Feifei Lee

  2. Rehabilitation Engineering and Technology Institute, University of Shanghai for Science and Technology, Shanghai, 200093, China

    Bin Zuo & Feifei Lee

  3. Major of Electrical Engineering and Electronics, Graduate School of Engineering, Kogakuin University, Tokyo, 163-8677, Japan

    Qiu Chen

Authors
  1. Bin Zuo
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Feifei Lee
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Qiu Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Feifei Lee or Qiu Chen.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zuo, B., Lee, F. & Chen, Q. An efficient U-shaped network combined with edge attention module and context pyramid fusion for skin lesion segmentation. Med Biol Eng Comput 60, 1987–2000 (2022). https://doi.org/10.1007/s11517-022-02581-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02581-5

Keywords