Skip to main content
Springer Nature Link
Log in

Serial attention network for skin lesion segmentation

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Due to the influence of color, boundaries, and shapes of melanoma, the segmentation of the lesion area is still a challenging problem. In this paper, we propose a method to connect two attention modules (channel attention and spatial attention) serially and embed them into the skip connection of the encoder–decoder network. Different from previous work on image segmentation through attention mechanism, we have made a different combination of the existing popular spatial attention module and channel attention module. The experimental results show that the sequence combinations with channel attention in front and spatial attention in the rear are more likely to aggregate global and local information as well as information between channels than other combinations. Our method achieved an average Jaccard Index of 0.7692 on the ISIC2017 dataset. At the same time, we also compared with some advanced methods of image segmentation, the experimental results show our proposed method has a competitive performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39:2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  • Bi L, Kim J, Ahn E, Kumar A, Fulham M, Feng D (2017) Dermoscopic image segmentation via multistage fully convolutional networks. IEEE Trans Biomed Eng 64:2065–2074. https://doi.org/10.1109/TBME.2017.2712771

    Article  Google Scholar 

  • Chaki J, Dey N, Rajinikanth V, Ashour AS, Shi F (2021) Recognition of skin diseases using curvelet transforms and law’s texture energy measures. Soft computing applications. Springer International Publishing, Cham, pp 51–61

    Chapter  Google Scholar 

  • Chen L-C, Papandreou G, Schroff F, Adam HJA (2017) Rethinking atrous convolution for semantic image segmentation abs/1706.05587

  • Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018a) Encoder-decoder with atrous separable convolution for semantic image segmentation. Computer vision—ECCV 2018. Springer International Publishing, Cham, pp 833–851

    Chapter  Google Scholar 

  • Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018b) DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40:834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  • Codella NCF et al. (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 International symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), 4–7 April 2018, pp 168–172. https://doi.org/10.1109/ISBI.2018.8363547

  • Dey N, Rajinikanth V, Ashour AS, Tavares JMRS (2018) Social group optimization supported segmentation and evaluation of skin melanoma images. Symmetry. https://doi.org/10.3390/sym10020051

    Article  MATH  Google Scholar 

  • Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, Lu H (2019) Dual attention network for scene segmentation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 15–20 June 2019, pp 3141–3149. https://doi.org/10.1109/CVPR.2019.00326

  • Garcia-Arroyo JL, Garcia-Zapirain B (2019) Segmentation of skin lesions in dermoscopy images using fuzzy classification of pixels and histogram thresholding. Comput Methods Programs Biomed 168:11–19. https://doi.org/10.1016/j.cmpb.2018.11.001

    Article  Google Scholar 

  • Gu Z et al (2019) CE-Net: context encoder network for 2D medical image segmentation. IEEE Trans Med Imaging 38:2281–2292. https://doi.org/10.1109/TMI.2019.2903562

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37:1904–1916. https://doi.org/10.1109/TPAMI.2015.2389824

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), 27–30 June 2016, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  • Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 18–23 June 2018, pp 7132–7141. https://doi.org/10.1109/CVPR.2018.00745

  • Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell 42:2011–2023. https://doi.org/10.1109/TPAMI.2019.2913372

    Article  Google Scholar 

  • Jha D, Riegler MA, Johansen D, Halvorsen P, Johansen HD (2020) DoubleU-Net: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd International symposium on computer-based medical systems (CBMS), 28–30 July 2020, pp 558–564. https://doi.org/10.1109/CBMS49503.2020.00111

  • Kaul C, Manandhar S, Pears N (2019) Focusnet: an attention-based fully convolutional network for medical image segmentation. In: 2019 IEEE 16th international symposium on biomedical imaging (ISBI 2019), 8–11 April 2019, pp 455–458. https://doi.org/10.1109/ISBI.2019.8759477

  • Li Y, Shen L (2018) Skin lesion analysis towards melanoma detection using deep learning network. Sensors. https://doi.org/10.3390/s18020556

    Article  Google Scholar 

  • Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), 7–12 June 2015, pp 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965

  • Md Zahangir A, Chris Y, Mahmudul H, Tarek MT, Vijayan KA (2019) Recurrent residual U-Net for medical image segmentation. J Med Imaging 6:1–16. https://doi.org/10.1117/1.JMI.6.1.014006

    Article  Google Scholar 

  • Peng C, Zhang X, Yu G, Luo G, Sun J (2017) Large Kernel matters—improve semantic segmentation by global convolutional network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 21–26 July 2017, pp 1743–1751. https://doi.org/10.1109/CVPR.2017.189

  • Rahman M, Alpaslan N, Bhattacharya P (2016) Developing a retrieval based diagnostic aid for automated melanoma recognition of dermoscopic images. In: 2016 IEEE applied imagery pattern recognition workshop (AIPR), 18–20 Oct. 2016, pp 1–7. https://doi.org/10.1109/AIPR.2016.8010594

  • Rajinikanth V, Satapathy SC, Dey N, Fernandes SL, Manic KS (2019) Skin melanoma assessment using Kapur’s entropy and level set—a study with bat algorithm. Smart intelligent computing and applications. Springer Singapore, Singapore, pp 193–202

    Google Scholar 

  • Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Medical image computing and computer-assisted intervention—MICCAI 2015. Springer International Publishing, Cham, pp 234–241

    Chapter  Google Scholar 

  • Roy AG, Navab N, Wachinger C (2018) Concurrent spatial and channel ‘squeeze & excitation’ in fully convolutional networks. Medical image computing and computer assisted intervention—MICCAI 2018. Springer International Publishing, Cham, pp 421–429

    Chapter  Google Scholar 

  • Silveira M et al (2009) Comparison of segmentation methods for melanoma diagnosis in dermoscopy images. IEEE J Sel Top Signal Process 3:35–45. https://doi.org/10.1109/JSTSP.2008.2011119

    Article  Google Scholar 

  • Society AC (2020) Key statistics for melanoma skin cancer. https://www.cancer.org/cancer/melanoma-skin-cancer/about/keystatistics.html. Accessed 4 Jul 2020

  • Sun K, Xiao B, Liu D, Wang J (2019) Deep high-resolution representation learning for human pose estimation. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 15–20 June 2019, pp 5686–5696. https://doi.org/10.1109/CVPR.2019.00584

  • Thao LT, Quang NH (2017) Automatic skin lesion analysis towards melanoma detection. In: 2017 21st Asia Pacific symposium on intelligent and evolutionary systems (IES), 15–17 Nov. 2017, pp 106–111. https://doi.org/10.1109/IESYS.2017.8233570

  • Vaswani A et al. (2017) Attention is all you need. In: Paper presented at the proceedings of the 31st international conference on neural information processing systems, Long Beach, California, USA

  • Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 18–23 June 2018, pp 7794–7803. https://doi.org/10.1109/CVPR.2018.00813

  • Wang Q, Wu B, Zhu P, Li P, Zuo W, Hu Q (2020) ECA-Net: efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), 13–19 June 2020, pp 11531–11539. https://doi.org/10.1109/CVPR42600.2020.01155

  • Wen HJA (2017) II-FCN for skin lesion analysis towards melanoma detection. abs/1702.08699

  • Wong A, Scharcanski J, Fieguth P (2011) Automatic skin lesion segmentation via iterative stochastic region merging. IEEE Trans Inf Technol Biomed 15:929–936. https://doi.org/10.1109/TITB.2011.2157829

    Article  Google Scholar 

  • Yang M, Yu K, Zhang C, Li Z, Yang K (2018) DenseASPP for semantic segmentation in street scenes. In: 2018 IEEE/CVF conference on computer vision and pattern recognition, 18–23 June 2018. pp 3684–3692. https://doi.org/10.1109/CVPR.2018.00388

  • Yu L, Chen H, Dou Q, Qin J, Heng P (2017b) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36:994–1004. https://doi.org/10.1109/TMI.2016.2642839

    Article  Google Scholar 

  • Yu F, Koltun V, Funkhouser T (2017a) Dilated residual networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 21–26 July 2017, pp 636–644. https://doi.org/10.1109/CVPR.2017.75

  • Yuan Y, Lo YC (2019) Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks. IEEE J Biomed Health Inform 23:519–526. https://doi.org/10.1109/JBHI.2017.2787487

    Article  Google Scholar 

  • Yuan Y, Chao M, Lo Y (2017) Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging 36:1876–1886. https://doi.org/10.1109/TMI.2017.2695227

    Article  Google Scholar 

  • Zhang H, Goodfellow I, Metaxas D, Odena A (2019) Self-attention generative adversarial networks. In: Paper presented at the proceedings of the 36th international conference on machine learning, proceedings of machine learning research

  • Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), 21–26 July 2017, pp 6230–6239. https://doi.org/10.1109/CVPR.2017.660

  • Zhou H, Li X, Schaefer G, Celebi ME, Miller P (2013) Mean shift based gradient vector flow for image segmentation. Comput Vis Image Underst 117:1004–1016. https://doi.org/10.1016/j.cviu.2012.11.015

    Article  Google Scholar 

  • Zhou Z, Siddiquee MMR, Tajbakhsh N, Liang J (2020) UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans Med Imaging 39:1856–1867. https://doi.org/10.1109/TMI.2019.2959609

    Article  Google Scholar 

Download references

Acknowledgements

This research is partially supported by Science and Technology Department of Xinjiang Uyghur Autonomous Region Fund Project (2020E0234), and Department of Education, Xinjiang Uygur Autonomous Region (CN) Postgraduate Research and Innovation Project (XJ2020G072). We would also like to thank our tutor for the careful guidance and all the participants for their insightful comments.

Author information

Authors and Affiliations

  1. College of Software, Xinjiang University, Urumqi, 830000, China

    Yuan Ren, Shengwei Tian, Zhiqi Guo & Yanhan Zhang

  2. Key Laboratory of Software Engineering Technology, Xinjiang University, Urumqi, 830000, China

    Yuan Ren, Shengwei Tian, Junlong Cheng, Zhiqi Guo & Yanhan Zhang

  3. Network Center, Xinjiang University, Urumqi, 830046, China

    Long Yu

  4. College of Information Science and Engineering, Xinjiang University, Urumqi, 830000, China

    Junlong Cheng

Authors
  1. Yuan Ren
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Long Yu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Shengwei Tian
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Junlong Cheng
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Zhiqi Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Yanhan Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Long Yu.

Ethics declarations

Conflict of interest

The authors declare no conflicts of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, Y., Yu, L., Tian, S. et al. Serial attention network for skin lesion segmentation. J Ambient Intell Human Comput 13, 799–810 (2022). https://doi.org/10.1007/s12652-021-02933-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-02933-3

Keywords