Abstract
Because of the large variation in appearance, the existence of artifacts, the low contrast, skin lesion segmentation remains a challenging task. In this paper, we propose a novel Scale Attention based Atrous Spatial Pyramid Pooling (Scale-Att-ASPP) module for skin lesion segmentation with attentive boundary aware. Our network is based on the Generative Adversarial Network (GAN), which includes the segmentation network and the critic network. In the segmentation network, we design the Scale-Att-ASPP module to automatically select the optimal scale of the skin lesion feature of the intermediate convolution layer (Inter-CL) in the encoding path, meanwhile, the irrelevant artifacts features are automatically diminished without using complex pre-processing. After introducing the output of the Scale-Att-ASPP module to the same level layer in the decoding path through skip connection in pixel-wise addition way, the more meaningful semantic segmentation is gained. The Jaccard distance loss is employed to solve the problem of label imbalance in skin lesion segmentation. Our network is adversarially trained on ISBI 2017 dataset by the multi-scale L1 loss introduced by the critic network, which guides the Scale-Att-ASPP module learning to focus on the optimal scale of the skin lesion feature. Finally, our network significantly improves the segmentation performance compared with other state-of-the-art methods, especially for the JAC and SEN scores. Besides, our proposed network works efficiently and shows robustness for different datasets.









Similar content being viewed by others
References
Ahn E, Bi L, Jung Y, Kim J, Li C, Fulham MJ, Feng DD (2015) Automated saliency-based lesion segmentation in dermoscopic images. In: EMBC 2015, Milan, Italy, 25–29 Aug. 2015, pp 3009–3012
Ahn E, Kim J, Bi L, Kumar A, Li C, Fulham MJ, Feng DD (2017) Saliency-based lesion segmentation via background detection in dermoscopic images. IEEE J Biomed Health Inform 21(6):1685–1693
Al-masni MA, Al-antari MA, Choi M-T, Han S-M, Kim T-S (2018) Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks. Comput Methods Prog Biomed 162:221–231
Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for scene segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495
Berseth M (2017) Isic 2017-skin lesion analysis towards melanoma detection. arXiv:1703.00523
Bi L, Kim J, Ahn E, Feng D, Fulham M (2016) Automated skin lesion segmentation via image-wise supervised learning and multi-scale superpixel based cellular automata. In: ISBI 2016, Prague, Czech Republic, 13–16 April 2016, pp 1059–1062
Bi L, Jinman K, Ahn E, Feng D (2017) Automatic skin lesion analysis using large-scale dermoscopy images and deep residual networks. arXiv:1703.04197
Bissoto A, Perez F, Valle E, Avila S (2018) Skin lesion synthesis with generative adversarial networks. arXiv:1902.03253
Celebi ME, Kingravi HA, Iyatomi H, Aslandogan YA, Stoecker WV, Moss RH, Malters JM, Grichnik JM, Marghoob AA, Rabinovitz HS, Menzies SW (2008) Border detection in dermoscopy images using statistical region merging. Skin Res Technol 14(3):347–353
Chen L-C, Yang Y, Wang J, Xu W, Yuille AL (2016) Attention to scale: Scale-aware semantic image segmentation. In: CVPR 2016, Las Vegas, NV, USA, 27–30 June, 2016, pp 3640–3649
Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587
Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: ECCV 2018, 8–14 September 2018, Munich, Germany, pp.833–851
Codella NCF, Gutman D, Celebi ME, Helba B, Marchetti MA, Dusza SW, Kalloo A, Liopyris K, Mishra NK, Kittler H, Halpern A (2018) Skin lesion analysis toward melanoma detection: a challenge at the 2017 international symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). ISBI 2018:168–172
Dakhia A, Wang T, Lu H (2019) Multi-scale pyramid pooling network for salient object detection. Neurocomputing 333:211–220
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database, in Proc. IEEE Conf Comput Vis Pattern Recognit 2009:248–255
Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchical features for scene labeling. IEEE Trans Pattern Anal Mach Intell 35(8):1915–1929
Gutman D, Codella NC, Celebi E, Helba B, Marchetti M, Mishra N, Halpern A (2016) Skin lesion analysis toward melanoma detection: a challenge at the international symposium on biomedical imaging (isbi) 2016″, hosted by the international skin imaging collaboration (ISIC). arXiv:1605.01397
Hariharan B, Arbeláez P, Girshick R, Malik J (2015) Hypercolumns for object segmentation and fine-grained localization. In: CVPR 2015, 7–12 June 2015, Boston, USA, pp 447–456
He K, Zhang X, Ren S, Sun J (2016) Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, pp. 770–778, https://doi.org/10.1109/CVPR.2016.90.
Hu J, Shen L, Albanie S, Sun G, Wu E (2018) Squeeze-and-Excitation Networks. In: CVPR 2018, Salt Lake City, USA, 19–21 June, 2018, pp 7132–7141
Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61–78
Lin BS, Michael K, Kalra S, Tizhoosh HR (2017) Skin lesion segmentation: U-Nets versus clustering. In: SSCI 2017, November 2017, pp 1–7
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: CVPR 2015, 7–12 June 2015, Boston, USA, pp 3431–3440
Ma Z, Tavares JMRS (2016) A novel approach to segment skin lesions in dermoscopic images based on a deformable model. IEEE J Biomed Health Inform 20(2):615–623
Mendonça T, Ferreira PM, Marques JS, Marcal AR, Rozeira J (2013) PH2-A dermoscopic image database for research and benchmarking. In: EMBC 2013, 3–7 July 2013, pp. 5437–5440
Mete M, Sirakov NM (2010) Lesion detection in demoscopy images with novel density-based and active contour approaches. BMC Bioinformatics 11(Suppl 6):S23
Noh H, Hong S, Han B (2015) Learning deconvolution network for semantic segmentation. In: ICCV 2015, 7–13 December 2015, Santiago, Chile, pp 1520–1528
Oktay O, Schlemper J, Le Folgoc L (2018) Attention U-Net: Learning where to look for the pancreas. In: MIDL 2018, 4–6 July, 2018, pp 1–10
Oliveira RB, Papa JP, Pereira AS, Tavares JMRS (2018) Computational methods for pigmented skin lesion classification in images: review and future trends. Neural Comput Applic 29(3):613–636
Rahman M, Alpaslan N, Bhattacharya P (2016) Developing a retrieval based diagnostic aid for automated melanoma recognition of dermoscopic images. In: IEEE Applied Imagery Pattern Recognition Workshop, Washington, DC, USA, 18–20 Oct. 2016, pp 1–7
Rogers HW, Weinstock MA, Feldman SR, Coldiron BM (2015) Incidence estimate of nonmelanoma skin cancer (keratinocyte carcinomas) in the U.S. population, 2012. JAMA Dermatol 151(10):1081–1086
Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: MICCAI 2015, 5–9 October 2015, Munich, Germany, pp 234–241
Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X (2016) Improved techniques for training GANs. In: NIPS 2016, Barcelona, Spain, June 2016
Sarker Md. MK, Rashwan HA, Akram F (2018) SLSDeep: Skin Lesion Segmentation Based on Dilated Residual and Pyramid Pooling Networks. In: MICCAI 2018, Granada, Spain, 16–20 September 2018
Shelhamer E, Long J, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):1–1
Siegel RL, Miller KD, Jemal A (2016) Cancer statistics, 2016. CA Cancer J Clin 66:7–30
Tang J, Hou X, Yang C, Li Y, Xin Y, Guo W, Wei Z, Liu Y, Jiang G (2017) Recent developments in nanomedicine for melanoma treatment. Int J Cancer 141(4):646–653
Wang F, Jiang M, Qian C, Yang S, Li C, Zhang H, Wang X, Tang X (2017) Residual Attention Network for Image Classification. In: CVPR 2017, 21–26 July, 2017, pp 6450–6458
Wang TC, Liu MY, Zhu JY, Tao A, Kautz J, Catanzaro B (2018) High resolution image synthesis and semantic manipulation with conditional GANs. In: CVPR 2018, Salt Lake City, 19-21 June, 2018, pp 8798-8807
Xue Y, Xu T, Han Z, Rodney Long L, Huang X (2018) SegAN: adversarial network with multi-scale L1 loss for medical image segmentation. Neuroinformatics 16(6):383–392
Yu L, Chen H, Dou Q, Qin J, Heng PA (2017) Automated melanoma recognition in dermoscopy images via very deep residual networks. IEEE Trans Med Imaging 36(4):994–1004
Yuan Y, Lo Y-C (2019) Improving Dermoscopic image segmentation with enhanced convolutional-Deconvolutional networks. IEEE J Biomed Health Inform 23(2):519–526
Yuan Y, Chao M, Lo Y-C (2017) Automatic skin lesion segmentation using deep fully convolutional networks with Jaccard distance. IEEE Trans Med Imaging 36(9):1876–1886
Yüksel ME, Borlu M (2009) Accurate segmentation of dermoscopic images by image thresholding based on type-2 fuzzy logic. IEEE Trans Fuzzy Syst 17(4):976–982
Zhang J, Xie Y, Xia Y, Shen C (2019) Attention residual learning for skin lesion classification. IEEE Trans Med Imaging 38(9):2092–2103
Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: CVPR 2017, Honolulu, HI, USA, 21–26 July, 2017, pp 2881–2890
Acknowledgements
This work was supported by National Natural Science Foundation of China (61771056), National Key R&D Program of China (2017YFC0110700).
Author information
Authors and Affiliations
Corresponding authors
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Wei, Z., Shi, F., Song, H. et al. Attentive boundary aware network for multi-scale skin lesion segmentation with adversarial training. Multimed Tools Appl 79, 27115–27136 (2020). https://doi.org/10.1007/s11042-020-09334-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09334-2