Skip to main content
SpringerLink
Log in

SGC-ARANet: scale-wise global contextual axile reverse attention network for automatic brain tumor segmentation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Automated brain tumour segmentation using magnetic resonance imaging (MRI) is essential for clinical decision-making and surgical planning. Numerous studies have demonstrated the feasibility of segmenting brain tumours using deep learning models such as U-shaped architectures. Unfortunately, due to the diversity of tumors and complex boundaries, it is insufficient to obtain contextual data on tumor cells and their surroundings from a single stage. To overcome this limitation, we proposed a Scale-wise Global Contextual Axile Reverse Attention Network (SGC-ARANet) consisting of four modules that improve segmentation performance. We begin by creating three global multi-level guidance (GMLG) modules to provide various levels of global contextual data. Additionally, we develop a scale-wise multi-level blend module (SWMB) that dynamically blends multi-scale contextual data with high-level features. Following that, we demonstrated how a partial decoder (PD) connected in parallel to the encoder is utilized to aggregate high-level and SWMB feature maps to create a global map. The axile reverse attention (ARA) module is then presented to simulate multi-modality tumor regions and boundaries using global and GMLG feature maps. We evaluate our model using the publicly available BraTS 2019 and 2020 brain tumor segmentation datasets. The results indicate that our SGC-ARANet is competitive or outperforms numerous State-of-the-art (SOTA) algorithms for several segmentation measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber= 9049412

  2. https://github.com/ShuhanChen/RAS_ECCV18

  3. https://www.researchgate.net/publication/267510459

  4. https://github.com/lxtGH/dfn_seg [25]

  5. https://github.com/maryamhashemi1995/MS-SEGMENTATION

  6. https://github.com/shalabh147/Brain-Tumor

  7. https://github.com/wuhuikai/FastFCN

  8. https://github.com/guanfuchen/semseg/issues/51

References

  1. Kampffmeyer M, Dong N, Liang X, Zhang Y, Xing EP (2018) Connnet: a long-range relation-aware pixel-connectivity network for salient segmentation. IEEE Trans Image Process 28(5):2518–2529

    Article  MathSciNet  Google Scholar 

  2. Fu J, Liu J, Wang Y, Zhou J, Wang C, Lu H (2019) Stacked deconvolutional network for semantic segmentation. IEEE Trans Image Process

  3. Fang Y, Ding G, Li J, Fang Z (2018) Deep3dsaliency: Deep stereoscopic video saliency detection model by 3d convolutional networks. IEEE Trans Image Process 28(5):2305–2318

    Article  MathSciNet  Google Scholar 

  4. Sabokrou M, Fayyaz M, Fathy M, Klette R (2017) Deep-cascade: Cascading 3d deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Trans Image Process 26(4):1992–2004

    Article  MathSciNet  MATH  Google Scholar 

  5. Saleem H, Shahid AR, Raza B (2021) Visual interpretability in 3d brain tumor segmentation network. Comput Biol Med 133:104410

    Article  Google Scholar 

  6. Zhou Z, He Z, Shi M, Du J, Chen D (2020) 3d dense connectivity network with atrous convolutional feature pyramid for brain tumor segmentation in magnetic resonance imaging of human heads. Comput Biol Med 121:103766

    Article  Google Scholar 

  7. Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2014) Appearance-and context-sensitive features for brain tumor segmentation. Proc MICCAI BRATS Challenge:020–026

  8. Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2013) A hybrid model for multimodal brain tumor segmentation. Multimodal Brain Tumor Segment 31:31–37

    Google Scholar 

  9. Tustison NJ, Shrinidhi K, Wintermark M, Durst CR, Kandel BM, Gee JC, Grossman MC, Avants BB (2015) Optimal symmetric multimodal templates and concatenated random forests for supervised brain tumor segmentation (simplified) with antsr. Neuroinformatics 13(2):209–225

    Article  Google Scholar 

  10. Karri M, Annavarapu CSR, Mallik S, Zhao Z, Acharya UR (2022) Multi-class nucleus detection and classification using deep convolutional neural network with enhanced high dimensional dissimilarity translation model on cervical cells. Biocybern Biomed Eng 42(3):797–814

    Article  Google Scholar 

  11. Bauer S, Nolte L-P, Reyes M (2011) Fully automatic segmentation of brain tumor images using support vector machine classification in combination with hierarchical conditional random field regularization. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 354–361

  12. Pinto A, Pereira S, Correia H, Oliveira J, Rasteiro DM, Silva CA (2015) Brain tumour segmentation based on extremely randomized forest with high-level features. In: 2015 37th Annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 3037–3040

  13. Geremia E, Menze BH, Ayache N (2013) Spatially adaptive random forests. In: 2013 IEEE 10th international symposium on biomedical imaging. IEEE, pp 1344–1347

  14. Lee C-H, Wang S, Murtha A, Brown MR, Greiner R (2008) Segmenting brain tumors using pseudo–conditional random fields. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 359– 366

  15. Meier R, Bauer S, Slotboom J, Wiest R, Reyes M (2014) Patient-specific semi-supervised learning for postoperative brain tumor segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 714–721

  16. Li W, Wang G, Fidon L, Ourselin S, Cardoso MJ, Vercauteren T (2017) On the compactness, efficiency, and representation of 3d convolutional networks: brain parcellation as a pretext task. In: International conference on information processing in medical imaging. Springer, pp 348–360

  17. Zikic D, Glocker B, Konukoglu E, Criminisi A, Demiralp C, Shotton J, Thomas OM, Das T, Jena R, Price SJ (2012) Decision forests for tissue-specific segmentation of high-grade gliomas in multi-channel mr. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 369–376

  18. LeCun Y, Bengio Y, Hinton G et al (2015) Deep learning. Nature 521(7553):436–444. Google scholar google scholar cross ref cross ref

    Article  Google Scholar 

  19. Shaikh M, Anand G, Acharya G, Amrutkar A, Alex V, Krishnamurthi G (2017) Brain tumor segmentation using dense fully convolutional neural network. In: International MICCAI brainlesion workshop. Springer, pp 309–319

  20. Islam M, Ren H (2017) Fully convolutional network with hypercolumn features for brain tumor segmentation. In: Proceedings of MICCAI workshop on multimodal brain tumor segmentation challenge (BRATS)

  21. Moreno Lopez M, Ventura J (2017) Dilated convolutions for brain tumor segmentation in mri scans. In: International MICCAI brainlesion workshop. Springer, pp 253–262

  22. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Med Image Anal 36:61–78

    Article  Google Scholar 

  23. Castillo LS, Daza LA, Rivera LC, Arbeláez P (2017) Volumetric multimodality neural network for brain tumor segmentation. In: 13th International conference on medical information processing and analysis, vol 10572, p 105720. International society for optics and photonics

  24. Wu H, Zhang J, Huang K, Liang K, Yu Y (2019) Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation. arXiv:1903.11816

  25. Yu C, Wang J, Peng C, Gao C, Yu G, Sang N (2018) Learning a discriminative feature network for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1857–1866

  26. Peng C, Zhang X, Yu G, Luo G, Sun J (2017) Large kernel matters–improve semantic segmentation by global convolutional network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4353–4361

  27. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, Mori K, McDonagh S, Hammerla NY, Kainz B et al (2018) Attention u-net: learning where to look for the pancreas. arXiv:1804.03999

  28. Qin Y, Kamnitsas K, Ancha S, Nanavati J, Cottrell G, Criminisi A, Nori A (2018) Autofocus layer for semantic segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 603–611

  29. Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  30. Gu Z, Cheng J, Fu H, Zhou K, Hao H, Zhao Y, Zhang T, Gao S, Liu J (2019) Ce-net: context encoder network for 2d medical image segmentation. IEEE Trans Med Imaging 38(10):2281–2292

    Article  Google Scholar 

  31. Chen L-C, Yang Y, Wang J, Xu W, Yuille AL (2016) Attention to scale: scale-aware semantic image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3640–3649

  32. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 510–519

  33. Sun W, Wu T (2019) Learning spatial pyramid attentive pooling in image synthesis and image-to-image translation. arXiv:1901.06322

  34. Li H, Xiong P, An J, Wang L (2018) Pyramid attention network for semantic segmentation. arXiv:1805.10180

  35. Feng S, Zhao H, Shi F, Cheng X, Wang M, Ma Y, Xiang D, Zhu W, Chen X (2020) Cpfnet: context pyramid fusion network for medical image segmentation. IEEE Trans Med Imaging 39 (10):3008–3018

    Article  Google Scholar 

  36. Zhou Z, Rahman Siddiquee MM, Tajbakhsh N, Liang J (2018) Unet++: a nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, pp 3–11

  37. Fu H, Cheng J, Xu Y, Wong DWK, Liu J, Cao X (2018) Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. IEEE Trans Med Imaging 37(7):1597–1605

    Article  Google Scholar 

  38. Wu Z, Su L, Huang Q (2019) Cascaded partial decoder for fast and accurate salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3907–3916

  39. Wei Y, Feng J, Liang X, Cheng M-M, Zhao Y, Yan S (2017) Object region mining with adversarial erasing: a simple classification to semantic segmentation approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1568–1576

  40. Chen S, Tan X, Wang B, Hu X (2018) Reverse attention for salient object detection. In: Proceedings of the european conference on computer vision (ECCV), pp 234–250

  41. Zhang Z, Lin Z, Xu J, Jin W-D, Lu S-P, Fan D-P (2021) Bilateral attention network for rgb-d salient object detection. IEEE Trans Image Process 30:1949–1961

    Article  Google Scholar 

  42. Qin X, Zhang Z, Huang C, Gao C, Dehghan M, Jagersand M (2019) Basnet: boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7479–7489

  43. Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R et al (2014) The multimodal brain tumor image segmentation benchmark (brats). IEEE Trans Med Imaging 34(10):1993–2024

    Article  Google Scholar 

  44. Bakas S, Reyes M, Jakab A, Bauer S, Rempfler M, Crimi A, Shinohara RT, Berger C, Ha SM, Rozycki M et al (2018) Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge. arXiv:1811.02629

  45. Hashemi M, Akhbari M, Jutten C (2022) Delve into multiple sclerosis (ms) lesion exploration: a modified attention u-net for ms lesion segmentation in brain mri. Comput Biol Med 145:105402

    Article  Google Scholar 

  46. Agravat RR, Raval MS (2019) Brain tumor segmentation and survival prediction. In: International MICCAI brainlesion workshop. Springer, pp 338–348

  47. Amian M, Soltaninejad M (2019) Multi-resolution 3d cnn for mri brain tumor segmentation and survival prediction. In: International MICCAI brainlesion workshop. Springer, pp 221–230

  48. Kim S, Luna M, Chikontwe P, Park SH (2019) Two-step u-nets for brain tumor segmentation and random forest with radiomics for survival time prediction. In: International MICCAI brainlesion workshop. Springer, pp 200–209

  49. Liu Z, Tong L, Chen L, Zhou F, Jiang Z, Zhang Q, Wang Y, Shan C, Li L, Zhou H (2021) Canet: context aware network for brain glioma segmentation. IEEE Trans Med Imaging 40 (7):1763–1777

    Article  Google Scholar 

  50. Zhou T, Canu S, Vera P, Ruan S (2021) Latent correlation representation learning for brain tumor segmentation with missing mri modalities. IEEE Trans Image Process 30:4263–4274

    Article  Google Scholar 

  51. Jiang Z, Ding C, Liu M, Tao D (2019) Two-stage cascaded u-net: 1st place solution to brats challenge 2019 segmentation task. In: International MICCAI brainlesion workshop. Springer, pp 231–241

  52. Wang F, Jiang R, Zheng L, Meng C, Biswal B (2019) 3d u-net based brain tumor segmentation and survival days prediction. In: International MICCAI brainlesion workshop. Springer, pp 131–141

  53. Zhao Y-X, Zhang Y-M, Liu C-L (2019) Bag of tricks for 3d mri brain tumor segmentation. In: International MICCAI brainlesion workshop. Springer, pp 210–220

  54. Myronenko A, Hatamizadeh A (2019) Robust semantic segmentation of brain tumor regions from 3d mris. In: International MICCAI brainlesion workshop. Springer, pp 82–89

  55. Li X, Luo G, Wang K (2019) Multi-step cascaded networks for brain tumor segmentation. In: International MICCAI brainlesion workshop. Springer, pp 163–173

  56. Clark K, Vendt B, Smith K, Freymann J, Kirby J, Koppel P, Moore S, Phillips S, Maffitt D, Pringle M et al (2013) The cancer imaging archive (tcia): maintaining and operating a public information repository. J Digital Imaging 26(6):1045–1057

    Article  Google Scholar 

  57. Scarpace L, Flanders AE, Jain R, Mikkelsen T, Andrews DW (2015) Data from REMBRANDT the cancer imaging archive

  58. Bakas S, Akbari H, Sotiras A, Bilello M, Rozycki M, Kirby J, Freymann J, Farahani K, Davatzikos C (2017) Segmentation labels and radiomic features for the pre-operative scans of the tcga-lgg collection. Cancer Imaging Archive, vol 286

  59. Erickson B, Akkus Z, Sedlar J, Kofiatis P (2017) Data from lgg-1p19qdeletion. Cancer Imaging Archive, vol 76

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Indian Institute of Technology (ISM), Dhanbad, 826004, Jharkhand, India

    Meghana Karri & Chandra Sekhara Rao Annvarapu

  2. Department of Biomedical Informatics and Medical Engineering, Asia University, Taichung, Taiwan

    U Rajendra Acharya

  3. Department of Electronics and Computer Engineering, Ngee Ann Polytechnic, Singapore, Clementi, 599489, Singapore

    U Rajendra Acharya

  4. Department of Biomedical Engineering, SUSS University, School of Science and Technology, Clementi, Singapore

    U Rajendra Acharya

Authors
  1. Meghana Karri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chandra Sekhara Rao Annvarapu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. U Rajendra Acharya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Meghana Karri or Chandra Sekhara Rao Annvarapu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Chandra Sekhara Rao Annvarapu and U Rajendra Acharya contributed equally to this work.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karri, M., Annvarapu, C.S.R. & Acharya, U.R. SGC-ARANet: scale-wise global contextual axile reverse attention network for automatic brain tumor segmentation. Appl Intell 53, 15407–15423 (2023). https://doi.org/10.1007/s10489-022-04209-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04209-5

Keywords

Search

Navigation