Skip to main content

Advertisement

SpringerLink
Log in

MAMC-Net: an effective deep learning framework for whole-slide image tumor segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Segmenting histopathological image automatically is an important task in computer-aided pathology analysis. However, it is challenging to segment and analyze digitalized histopathology images due to the large size of WSI, diversity and complexity of features. In this paper, we propose a multi-resolution attention and multi-scale convolution network (MAMC-Net) for the automatic tumor segmentation of WSI. First, the proposed MAMC-Net design the multi-resolution attention module that utilizes multi-resolution images as the pyramid inputs to generate a wider range feature information and richer details. Specifically, we employ an attention mechanism at each level to capture discriminative features related with the segmentation task. Furthermore, a multi-scale convolution module is designed to multi-scale feature representation by aggregating intact semantic information from the deep layer of encoder and high-resolution details from the final layer of decoder. To further obtain the accurate segmentation results, we adopt a fully connected Conditional Random Field (CRF) to splice the overlapping maps to avoid discontinuities and inconsistencies of cancer boundaries. Finally, we demonstrate the effectiveness of our framework on open-source datasets, including CAME-LYON17 (breast cancer metastases) and BOT (gastric cancer) datasets. The experimental results show that our proposed MAMC-Net obtains superior performance compared with other state-of-the-art methods, such as a Dice coefficient (DSC) of 0.929, an IOU score of 0.867, recall of 0.933 on the breast cancer dataset, a Dice coefficient (DSC) of 0.89, an IOU score of 0.802, recall of 0.903 on the gastric cancer dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

All the data generated or analyzed during this study is included in this published article. The datasets used or analyzed during the current study are available from the official website or the corresponding author on reasonable request.

Notes

  1. https://camelyon17.grand-challenge.org/Data

  2. http://www.datadreams.org/raceDetail/raceData

References

  1. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  2. Bejnordi BE, Veta M, Van Diest PJ, Van Ginneken B, Karssemeijer N, Litjens G (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with Breast Cancer. JAMA 318 (22):2199–2210

    Article  Google Scholar 

  3. Bullock J, Cuesta-Lázaro C, Quera-Bofarull A (2019) XNet: a convolutional neural network (CNN) implementation for medical X-ray image segmentation suitable for small datasets. In: Medical imaging 2019: biomedical applications in molecular, structural, and functional imaging, vol 10953, p 109531Z

  4. Chen P, Liang Y, Shi X, Yang L, Gader P (2021) Automatic whole slide pathology image diagnosis framework via unit stochastic selection and attention fusion. Neurocomputing 453:312–325

    Article  Google Scholar 

  5. Chen LC, Yang Y, Wang J, Xu W, Yuille AL (2016) Attention to scale: scale-aware semantic image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3640–3649

  6. Cho S, Jang H, Tan JW, Jeong WK (2021) DeepScribble: interactive pathology image segmentation using deep neural networks with scribbles. In: 2021 IEEE 18th international symposium on biomedical imaging (ISBI), pp 761–765

  7. Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, Tsirigos A (2018) Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med 24 (10):1559–1567

    Article  Google Scholar 

  8. Das A, Nair MS, Peter SD (2020) Computer-aided histopathological image analysis techniques for automated nuclear atypia scoring of breast cancer: a review. J Digit Imaging 33(5):1091–1121

    Article  Google Scholar 

  9. Feng R, Liu X, Chen J, Chen DZ, Gao H, Wu J (2020) A deep learning approach for colonoscopy pathology WSI analysis: accurate segmentation and classification. IEEE Journal of Biomedical and Health Informatics

  10. Gu F, Burlutskiy N, Andersson M, Wilén LK (2018) Multi-resolution networks for semantic segmentation in whole slide images. In: Computational pathology and ophthalmic medical image analysis, pp 11–18

  11. Guo Z, Liu H, Ni H, Wang X, Su M, Guo W, Qian Y (2019) A fast and refined cancer regions segmentation framework in whole-slide breast pathological images. Sci Rep 9(1):1–10

    Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  13. Khened M, Kori A, Rajkumar H, Krishnamurthi G, Srinivasan B (2021) A generalized deep learning framework for whole-slide image segmentation and analysis. Sci Rep 11(1):1–14

    Article  Google Scholar 

  14. Krähenbühl P, Koltun V (2011) Efficient inference in fully connected CRFs with gaussian edge potentials. Adv Neural Inf Process Syst 24:109–117

    Google Scholar 

  15. Lee H, Park J, Hwang JY (2020) Channel attention module with multiscale grid average pooling for breast cancer segmentation in an ultrasound image. IEEE Trans Ultrason Ferroelectr Freq Control 67(7):1344–1353

    Google Scholar 

  16. Li C, Li X, Rahaman M, Li X, Sun H, Zhang H, Grzegorzek M (2021) A comprehensive review of computer-aided whole-slide image analysis: from datasets to feature extraction, segmentation, classification, and detection approaches. arXiv:2102.10553

  17. Li Z, Tao R, Wu Q, Li B (2021) DA-RefineNet: dual-inputs attention refinenet for whole slide image segmentation. In: 2020 25th international conference on pattern recognition (ICPR), pp 1918–1925

  18. Lin H, Chen H, Dou Q, Wang L, Qin J, Heng PA (2018) ScanNet: a fast and dense scanning framework for metastastic breast cancer detection from whole-slide image. In: 2018 IEEE winter conference on applications of computer vision (WACV), pp 539–546

  19. Liu J, Desrosiers C, Zhou Y (2020) Att-MoE: attention-based Mixture of Experts for nuclear and cytoplasmic segmentation. Neurocomputing 411:139–148

    Article  Google Scholar 

  20. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  21. Mehta S, Mercan E, Bartlett J, Weaver D, Elmore JG, Shapiro L (2018) Y-Net: joint segmentation and classification for diagnosis of breast biopsy images. In: International conference on medical image computing and computer-assisted intervention, pp 893–901

  22. Mehta S, Mercan E, Bartlett J, Weaver D, Elmore J, Shapiro L (2018) Learning to segment breast biopsy whole slide images. In: 2018 IEEE winter conference on applications of computer vision (WACV), pp 663–672

  23. Mehta S, Rastegari M, Caspi A, Shapiro L, Hajishirzi H (2018) EspNet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), pp 552–568

  24. Mills S (2019) Histology for pathologists. Lippincott Williams & Wilkins

  25. Nguyen C, Asad Z, Huo Y (2021) Evaluating transformer based semantic segmentation networks for pathological image segmentation. arXiv:2108.11993

  26. Pan Y, Sun Z, Wang W, Yang Z, Jia J, Feng X, Zou S (2020) Automatic detection of squamous cell carcinoma metastasis in esophageal lymph nodes using semantic segmentation. Clin Transl Med 10(3):e129

    Article  Google Scholar 

  27. Rastogi P, Khanna K, Singh V (2021) Gland segmentation in colorectal cancer histopathological images using U-net inspired convolutional network. Neural Comput Appl :1–13

  28. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241

  29. Schmitz R, Madesta F, Nielsen M, Krause J, Steurer S, Werner R, Rösch T (2021) Multi-scale fully convolutional neural networks for histopathology image segmentation: from nuclear aberrations to the global tissue architecture. Med Image Anal 70:101996

    Article  Google Scholar 

  30. Sinha A, Dolz J (2020) Multi-scale self-guided attention for medical image segmentation. IEEE J Biomed Health Inform 25(1):121–130

    Article  Google Scholar 

  31. Spanhol FA, Oliveira LS, Petitjean C, Heutte L (2016) Breast cancer histopathological image classification using convolutional neural networks. In: 2016 international joint conference on neural networks (IJCNN), pp 2560–2567

  32. Stoean R (2020) Analysis on the potential of an EA–surrogate modelling tandem for deep learning parametrization: an example for cancer classification from medical images. Neural Comput Appl 32(2):313–322

    Article  Google Scholar 

  33. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1–9

  34. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  35. Tao S, Guo Y, Zhu C, Chen H, Zhang Y, Yang J, Liu J (2019) Highly efficient follicular segmentation in thyroid cytopathological whole slide image. In: International workshop on health intelligence, pp 149–157

  36. Teng L, Li H, Karim S (2019) DMCNN: a deep multiscale convolutional neural network model for medical image segmentation. Journal of Healthcare Engineering

  37. Tokunaga H, Teramoto Y, Yoshizawa A, Bise R (2019) Adaptive weighting multi-field-of-view CNN for semantic segmentation in pathology. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12597–12606

  38. Veta M, Pluim JP, Van Diest JP, Viergever MA (2014) Breast cancer histopathology image analysis: a review. IEEE Trans Biomed Eng 61 (5):1400–1411

    Article  Google Scholar 

  39. Vidyarthi A, Patel A (2021) Deep assisted dense model based classification of invasive ductal breast histology images. Neural Comput Appl :1–11

  40. Wang Y, Deng Z, Hu X, Zhu L, Yang X, Xu X, Ni D (2018) Deep attentional features for prostate segmentation in ultrasound. In: International conference on medical image computing and computer-assisted intervention, pp 523–530

  41. Wang D, Khosla A, Gargeya R, Irshad H, Beck AH (2016) Deep learning for identifying metastatic breast cancer. arXiv:1606.05718

  42. Zhang J, Jin Y, Xu J, Xu X, Zhang Y (2018) MDU-Net: multi-scale densely connected u-net for biomedical image segmentation. arXiv:1812.00352

  43. Zhang J, Xie Y, Wu Q, Xia Y (2019) Medical image classification using synergic deep learning. Med Image Anal 54:10–19

    Article  Google Scholar 

  44. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  45. Zheng Y, Jiang Z, Xie F, Shi J, Zhang H, Huai J, Yang X (2020) Diagnostic regions attention network (DRA-Net) for histopathology WSI recommendation and retrieval. IEEE Trans Med Imaging 40(3):1090–1103

    Article  Google Scholar 

  46. van Rijthoven M, Balkenhol M, Siliņa K, van der Laak J, Ciompi F (2021) HookNet: multi-resolution convolutional neural networks for semantic segmentation in histopathology whole-slide images. Med Image Anal 68:101890

    Article  Google Scholar 

Download references

Acknowledgements

This article was supported by Natural Science Foundation of Hunan Province in China (2020JJ4588, 2020JJ4090), Joint Fund for Regional Innovation and Development of National Natural Science Foundation in China (U19A2083), and Open Project of Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education, Xiangtan University (2020ICIP06). We would especially like to thank Associate Professor Chaoyang Ai for his contributions to the English revision of this manuscript.

Author information

Author notes

    Authors and Affiliations

    1. College of Automation and Electronic Information, Xiangtan University, Xiangtan, 411105, Hunan, China

      Li Zeng, Hongzhong Tang, Wei Wang & Mingjian Xie

    2. Key Laboratory of Intelligent Computing and Information Processing of Ministry of Education, Xiangtan University, Xiangtan, 411105, Hunan, China

      Hongzhong Tang

    3. College of Foreign Languages; Inter-disciplinary Research Center of Language Intelligence and Cultural Heritages, Hunan University, Changsha, 410082, Hunan, China

      Zhaoyang Ai

    4. School of Information and Electrical Engineering, Hunan University of Science and Technology, Xiangtan, 411201, Hunan, China

      Lei Chen

    5. The First People’s Hospital of Xiangtan City, Xiangtan, 411101, Hunan, China

      Yongjun Wu

    Authors
    1. Li Zeng
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Hongzhong Tang
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Wei Wang
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Mingjian Xie
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Zhaoyang Ai
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Lei Chen
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Yongjun Wu
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Hongzhong Tang.

    Ethics declarations

    Conflict of Interests

    The authors declare that they have no conflict of interest.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Li Zeng and Wei Wang are contributed equally to this work.

    Rights and permissions

    Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Zeng, L., Tang, H., Wang, W. et al. MAMC-Net: an effective deep learning framework for whole-slide image tumor segmentation. Multimed Tools Appl 82, 39349–39369 (2023). https://doi.org/10.1007/s11042-023-15065-x

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-023-15065-x

    Keywords

    Search

    Navigation