Skip to main content
SpringerLink
Log in

Fully convolutional neural network with attention gate and fuzzy active contour model for skin lesion segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This study proposes an approach for segmentation of skin lesions from dermoscopic images based on fully convolutional neural network and active contour model (ACM). The architecture of fully convolutional neural network (FCN) is adapted from the SegNet neural network. Particularly, the paper proposes to use the skip connection architecture and integrate the additive attention gate (AG) into the SegNet architecture. So that the model can better handle the variation in shapes and sizes of desired objects and produce more accurate segmentation. In addition, the fuzzy energy-based shape distance is introduced to the loss function for minimizing the dissimilarity between the prediction and reference masks. Moreover, the fuzzy energy-based ACM, with contours initialized from the network predicted masks, is employed to further evolve the contour toward desired object boundary. The proposed model therefore can take the advantages of the neural network and the fuzzy ACM to build a fully automatic and robust approach for segmentation of skin lesions. The proposed approach is evaluated on the ISIC 2017 and PH2 challenge databases. Comparative results on the two databases show desired performances of the approach while compared to other state-of-the-arts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Abraham N, Khan N (2019) A novel focal tversky loss function with improved attention U-net for lesion segmentation. Proc. 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019):683–687

  2. Arora M, Kumar M (2021) AutoFER: PCA and PSO based automatic facial emotion recognition. Multimed Tools Appl 80:3039–3049

    Article  Google Scholar 

  3. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  4. Balakrishna C, Dadashzadeh S, Soltaninejad S (2018) Automatic detection of lumen and media in the IVUS images using U-net with VGG16 encoder. arXiv:180607554

  5. Bansal M, Kumar M, Kumar M XGBoost: 2D-Object Recognition Using Shape Descriptors and Extreme Gradient Boosting Classifier. Comput Methods Data Eng 2020. pp. 207–222

  6. Bi L, Kim J, Ahn E, Kumar A, Fulham M, Feng D (2017) Dermoscopic image segmentation via multi-stage fully convolutional networks. IEEE Trans Biomed Eng 64(9):2065–2074

    Article  Google Scholar 

  7. Bi L, Kim J, Ahn E, Feng D (2017) Automatic skin lesion analysis using large-scale dermoscopy images and deep residual networks. In arXiv:170304197:available: https://arxiv.org/abs/1703.04197

  8. Bi L, Kim J, Ahn E, Kumar A, Feng D, Fulham M (2019) Step-wise integration of deep class-specific learning for dermoscopic image segmentation. Pattern Recogn 85:78–89

    Article  Google Scholar 

  9. Bozorgtabar B, Abedini M, Garnavi R (2016) Sparse coding based skin lesion segmentation using dynamic rule-based refinement. In: Proc. Int. Workshop Mach. Learn. Med. Imag., Cham, Switzerland:, Springer, pp 254–261

  10. Bresson X, Esedoḡlu S, Vandergheynst P, Thiran J, Osher S (2007) Fast global minimization of the active contour/Snake model. J Math Imaging Vis 28(2):151–167

    Article  MathSciNet  Google Scholar 

  11. Cao Y, Liu S, Peng Y, Li J (2020) DenseUNet: densely connected UNet for Electron microscopy image segmentation. IET Image Process 14(12):2682–2689

    Article  Google Scholar 

  12. Celebi ME, Wen Q, Iyatomi H, Shimizu K, Zhou H, Schaefer G (2015) A state-of-the-art survey on lesion border detection in dermoscopy images. In: Dermoscopy Image Analysis. pp. 97–129

  13. Chan T, Vese L (2001) Active contours without edges. IEEE Trans Image Process 10(2):266–277

    Article  Google Scholar 

  14. Chan T, Sandberg B, Vese L (2000) Active contours without edges for vector-valued images. J Vis Commun Image Represent 11(2):130–141

    Article  Google Scholar 

  15. Chen XM, Williams BR, Vallabhaneni S, Czanner G, Williams R, Zheng Y (2019) Learning Active Contour Models for Medical Image Segmentation. Proc IEEE Conf Comput Vision Patt Recog (CVPR): 11623–11640

  16. Codella NC, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra N, Kittler H, Halpern A (2018) Skin lesion analysis toward melanoma detection: A challenge at the 2017international symposium on biomedical imaging (ISBI). Proc. IEEE 15th Int.Symp. Biomed. Imag.:168–172

  17. Galdran A, Alvarez-Gila A, Meyer ML, Saratxaga CL, Araújo T, Garrote E, Aresta G, Costa P, Mendonça AM, Campilho AC (2017) Data-driven color augmentation techniques for deep skin image analysis. In arXiv:170303702:https://arxiv.org/abs/1703.03702

  18. Ge Z, Demyanov S, Chakravorty R, Bowling A, Garnavi R Skin disease recognition using deep saliency features and multimodal learning of dermoscopy and clinical images. In: Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2017. pp. 250–258

  19. Ibtehaz N, Rahman MS (2019) Multiresunet: Rethinking the U-Net architecture for multimodalbiomedical image segmentation. Available: https://arxiv.org/abs/1703.03702

  20. Ibtehaza N, Rahman M (2020) MultiResUNet : rethinking the U-net architecture for multimodal biomedical image segmentation. Neural Netw 121:74–87

    Article  Google Scholar 

  21. Jahanifar M, Tajeddin NZ, Asl BM, Gooya A (2019) Supervised saliency map driven segmentation of lesions in dermoscopic images. IEEE J Biomed Health Inform 23(2):509–518

    Article  Google Scholar 

  22. Kaur A, Kaur A, Kaur M (2019) Face detection techniques: a review. Artif Intell Rev 52(2):927–948

    Article  MathSciNet  Google Scholar 

  23. Krinidis S, Chatzis V (2009) Fuzzy energy-based active contours. IEEE Trans Image Process 18(12):2747–2755

    Article  MathSciNet  Google Scholar 

  24. Kumar K, Kumar M, Kaur A (2021) Face detection in still images under occlusion and non-uniform illumination. Multimed Tools Appl 80:14565–14590

    Article  Google Scholar 

  25. Kumar K, Kumar M, Bansal M (2021) 2D object recognition: a comparative analysis of SIFT, SURF and ORB feature descriptors. Multimed Tools Appl 80:18839–18857

    Article  Google Scholar 

  26. Le Cun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444

    Article  Google Scholar 

  27. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proc IEEE Conf Comp Vision Patt Recog (CVPR):3431–3440

  28. Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks. Conf Comput Vision Patt Recogn (CVPR), 2019. pp. 3618–3627

  29. Lu X, Wang W, Danelljan M, Zhou T, Shen J (2020) Gool L video object segmentation with episodic graph memory networks. Eur Conf Comput Vision:661–679

  30. Lu X, Ma C, Shen J, Yang X, Reid I, Yang MH (2020) Deep object tracking with shrinkage loss. IEEE Trans Pattern Anal Mach Intell PP:1. https://doi.org/10.1109/TPAMI.2020.3041332

    Article  Google Scholar 

  31. Lu X, Wang W, Shen J, Crandall D, Luo J (2020) Zero-shot video object segmentation with co-attention Siamese networks. IEEE Trans Patt Anal Mach Intell PP:1. https://doi.org/10.1109/TPAMI.2020.3040258

    Article  Google Scholar 

  32. Lu X, Ma C, Ni B, Yang X (2021) Adaptive region proposal with channel regularization for robust object tracking. IEEE Trans Circ Syst Video Technol 31(4):1268–1282

    Article  Google Scholar 

  33. Mendonça T, Ferreira PM, Marques JS, Marcal ARS, Rozeira J (2013) PH2-a dermoscopic image database for research and benchmarking. In: Proc. 35th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC). pp 5437–5440

  34. Mishraa NK, Celebi ME (2016) An overview of melanoma detection in dermoscopy images using image processing and machine learning. arXiv:160107843

  35. Mondal A (2020) Fuzzy energy based active contour model for multi-region image segmentation. Multimed Tools and Appl 79:1535–1554

    Article  Google Scholar 

  36. Ninh QC, Tran TT, Tran TT, Tran TAX, Pham VT (2020) Skin Lesion Segmentation Based on Modification of SegNet Neural Networks. Proc. 6th NAFOSTED Conf Inform Comput Sci (NICS), Hanoi:575–578

  37. Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. Proc. IEEE Int Conf Comput Vision:1520–1528

  38. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, Mc-Donagh S, Hammerla NY, Kainz B, Glocker B, Rueckert D Attention U-Net: Learning Where to Look for the Pancreas. Proc. 1st Conf. Med. Imaging with Deep Learn., 2018. p Available: https://arxiv.org/abs/1804.03999

  39. Pellacani G, Seidenari S (2002) Comparison between morphological parameters in pigmented skin lesion images acquired by means of epiluminescence surface microscopy and polarized-light videomicroscopy. Clin Dermatol 20(3):222–227

    Article  Google Scholar 

  40. Ronneberger O, Fischer P (2015) Brox T U-net: convolutional networks for biomedical image segmentation. Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent.:234–241

  41. Sadegh SM, Erdogmus D (2017) Gholipour a Tversky loss function for image segmentation using 3D fully convolutional deep networks. Proc. Int. Workshop Mach. Learn. Med. Imag.:379–387

  42. Salimi SB, Bozorgtabar S, Schmid-Saugeon P, Ekenel HK, Rad MS, Thiran J.-P. (2019) DermoNet: densely linked convolutional neural networkfor efficient skin lesion segmentation. EURASIP J Image and Video Process 71:1–10

  43. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556,

  44. Singh S, Ahuja U, Mm K, Kumar K, Sachdeva M (2021) Face mask detection using YOLOv3 and faster R-CNN models: COVID-19 environment. Multimed Tools Appl 80:19753–19768

    Article  Google Scholar 

  45. Tang Y, Yang F, Yuan S, Zhan CA (2019) A multi-stage framework with context information fusion structure for skin lesion segmentation. Proc. IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019)

  46. Tran PV (2016) A fully convolutional neural network for cardiac segmentation in short-axis MRI. Available: https://arxiv.org/pdf/1604.00494.pdf

  47. Tran TT, Pham VT, Shyu KK (2014) Image segmentation using fuzzy energy-based active contour with shape prior. J Vis Commun Image Represent 25(7):1732–1745

    Article  Google Scholar 

  48. Ünver H, Ayan E (2019) Skin lesion segmentation in Dermoscopic images with combination of YOLO and GrabCut algorithm. Diagnostics (Basel) 9(3):E72. https://doi.org/10.3390/diagnostics9030072

    Article  Google Scholar 

  49. Wenli T, Xiaoming L, Wei H, Zhifang P (2019) Dense-residual network with adversarial learning for skin lesion segmentation. IEEE Access 7:77037–77051

    Article  Google Scholar 

  50. Wu Y, Ma W, Gong M, Li H, Jiao L (2015) Novel fuzzy active contour model with kernel metric for image segmentation. Appl Soft Comput 34:301–311

    Article  Google Scholar 

  51. Xue Y, Xu T, Zhang H, Long LR, Huang X (2018) SegAN: adversarial network with multi-scale L 1 loss for medical image segmentation. Neuroinformatic 16(383–392):383–392

    Article  Google Scholar 

  52. Yu L, Chen H, Dou Q, Qin J, Heng PA (2017) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004

    Article  Google Scholar 

  53. Yuan Y, Lo Y-C (2019) Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks. IEEE J Biomed Health Inform 23(2):519–526

    Article  MathSciNet  Google Scholar 

  54. Yuan Y, Chao M, Lo YC (2017) Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging 36(9):1876–1886

    Article  Google Scholar 

  55. Zhou Z, Siddiquee M, Tajbakhsh N, Liang J (2018) UNet++: a nested U-net architecture for medical image segmentation. Int Workshop Deep Learn Med Image Anal. 3–11

Download references

Acknowledgements

This research is funded by Vietnam National Foundation for Science and Technology Development (NAFOSTED) under grant number 102.05-2018.302, and partly supported by the Hanoi University of Science and Technology (HUST) under project number T2021-PC-005.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Hanoi University of Science and Technology, Hanoi, Vietnam

    Thi-Thao Tran & Van-Truong Pham

Authors
  1. Thi-Thao Tran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Van-Truong Pham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Van-Truong Pham.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tran, TT., Pham, VT. Fully convolutional neural network with attention gate and fuzzy active contour model for skin lesion segmentation. Multimed Tools Appl 81, 13979–13999 (2022). https://doi.org/10.1007/s11042-022-12413-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12413-1

Keywords

Search

Navigation