Skip to main content
Log in

Transformers for colorectal cancer segmentation in CT imaging

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Most recently transformer models became the state of the art in various medical image segmentation tasks and challenges, outperforming most of the conventional deep learning approaches. Picking up on that trend, this study aims at applying various transformer models to the highly challenging task of colorectal cancer (CRC) segmentation in CT imaging and assessing how they hold up to the current state-of-the-art convolutional neural network (CNN), the nnUnet. Furthermore, we wanted to investigate the impact of the network size on the resulting accuracies, since transformer models tend to be significantly larger than conventional network architectures.

Methods

For this purpose, six different transformer models, with specific architectural advancements and network sizes were implemented alongside the aforementioned nnUnet and were applied to the CRC segmentation task of the medical segmentation decathlon.

Results

The best results were achieved with the Swin-UNETR, D-Former, and VT-Unet, each transformer models, with a Dice similarity coefficient (DSC) of 0.60, 0.59 and 0.59, respectively. Therefore, the current state-of-the-art CNN, the nnUnet could be outperformed by transformer architectures regarding this task. Furthermore, a comparison with the inter-observer variability (IOV) of approx. 0.64 DSC indicates almost expert-level accuracy. The comparatively low IOV emphasizes the complexity and challenge of CRC segmentation, as well as indicating limitations regarding the achievable segmentation accuracy.

Conclusion

As a result of this study, transformer models underline their current upward trend in producing state-of-the-art results also for the challenging task of CRC segmentation. However, with ever smaller advances in total accuracies, as demonstrated in this study by the on par performances of multiple network variants, other advantages like efficiency, low computation demands, or ease of adaption to new tasks become more and more relevant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F (2021) Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: Cancer J Clin 71(3):209–249

    PubMed  Google Scholar 

  2. Yao L, Xia Y, Zhang H, Yao J, Jin D, Qiu B, Zhang Y, Li S, Liang Y, Hua XS, Lu L, Chen X, Liu Z, Zhang L (2022) DeepCRC: colorectum and colorectal cancer segmentation in CT scans via deep colorectal coordinate transform. In: International conference on medical image computing and computer-assisted intervention. Springer Nature, Switzerland, pp 564–573

  3. Mainenti PP, Stanzione A, Guarino S, Romeo V, Ugga L, Romano F, Storto G, Maurea S, Brunetti A (2019) Colorectal cancer: Parametric evaluation of morphological, functional and molecular tomographic imaging. World J Gastroenterol 25(35):5233

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Ahmad HM, Khan MJ, Yousaf A, Ghuffar S, Khurshid K (2020) Deep learning: a breakthrough in medical imaging. Curr Med Imaging 16(8):946–956

    Article  PubMed  Google Scholar 

  5. Antonelli M, Reinke A, Bakas S, Farahani K, Kopp-Schneider A, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, van Ginneken B, Bilello M, Bilic P, Christ PF, Do RKG, Gollub MJ, Heckers SH, Huisman H, Jarnagin WR, McHugo MK et al (2022) The medical segmentation decathlon. Nat Commun 13(1):4128

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Jia X, Mai X, Cui Y, Yuan Y, Xing X, Seo H, Xing L, Meng MQH (2020) Automatic polyp recognition in colonoscopy images using deep learning and two-stage pyramidal feature prediction. IEEE Trans Autom Sci Eng 17(3):1570–1584

    Google Scholar 

  7. Duc NT, Oanh NT, Thuy NT, Triet TM, Dinh VS (2022) Colonformer: an efficient transformer-based method for colon polyp segmentation. IEEE Access 10:80575–80586

    Article  Google Scholar 

  8. Sui D, Zhang K, Liu W, Chen J, Ma X, Tian Z (2021) CST: A multitask learning framework for colorectal cancer region mining based on transformer. BioMed Res Int 2021:6207964

    Article  PubMed  PubMed Central  Google Scholar 

  9. Soomro MH, De Cola G, Conforto S, Schmid M, Giunta G, Guidi E, Neri E, Caruso D, Ciolina M, Laghi A (2018) Automatic segmentation of colorectal cancer in 3D MRI by combining deep learning and 3D level-set algorithm—a preliminary study. In: IEEE middle east conference on biomedical engineering (MECBME), pp 198–203

  10. Pei Y, Mu L, Fu Y, He K, Li H, Guo S, Liu X, Li M, Zhang H, Li X (2020) Colorectal tumor segmentation of CT scans based on a convolutional neural network with an attention mechanism. IEEE Access 8:64131–64138

    Article  Google Scholar 

  11. Tang Y, Yang D, Li W, Roth HR, Landman B, Xu D, Nath V, Hatamizadeh A (2022) Self-supervised pre-training of swin transformers for 3d medical image analysis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 20730–20740

  12. Zhang R, Bai Z, Yu R, Pang W, Wang L, Zhu L, Zhang X, Zhang H, Hu W (2023) AG-CRC: anatomy-guided colorectal cancer segmentation in CT with imperfect anatomical knowledge. arXiv preprint arXiv:2310.04677

  13. Liu X, Guo S, Zhang H, He K, Mu S, Guo Y, Li X (2019) Accurate colorectal tumor segmentation for CT scans based on the label assignment generative adversarial network. Med Phys 46(8):3532–3542

    Article  PubMed  Google Scholar 

  14. Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18(2):203–211

    Article  CAS  PubMed  Google Scholar 

  15. Hatamizadeh A, Tang Y, Nath V, Yang D, Myronenko A, Landman B, Roth HR, Xu D (2022) UNETR: Transformers for 3D medical image segmentation. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 574–584

  16. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, 2015, proceedings, Part III 18. Springer International Publishing, pp 234–241

  17. Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2021) Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images. International MICCAI Brainlesion Workshop. Springer International Publishing, Cham, pp 272–284

    Google Scholar 

  18. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 10012–10022

  19. Peiris H, Hayat M, Chen Z, Egan G, Harandi M (2022) A robust volumetric transformer for accurate 3D tumor segmentation. In: international conference on medical image computing and computer-assisted intervention. Springer Nature, Switzerland, pp 162–172

  20. Zhou HY, Guo J, Zhang Y, Han X, Yu L, Wang L, Yu Y (2023) nnformer: Volumetric medical image segmentation via a 3D transformer. In: IEEE transactions on image processing

  21. Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Zhou Y (2021) Transunet: Transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:2102.04306

  22. Chen J, Mei J, Li X, Lu Y, Yu Q, Wei Q, Zhou Y (2023) 3D transunet: advancing medical image segmentation through vision transformers. arXiv preprint arXiv:2310.07781

  23. Wu Y, Liao K, Chen J, Wang J, Chen DZ, Gao H, Wu J (2023) D-former: a U-shaped dilated transformer for 3d medical image segmentation. Neural Comput Appl 35(2):1931–1944

    Article  Google Scholar 

  24. Tummala P, Hille G, Saalfeld S (2023) Automatic lung nodule segmentation in CT imaging using an improved 3D-Res2Unet. BVM Workshop. Springer Fachmedien Wiesbaden, Wiesbaden, pp 165–170

    Google Scholar 

  25. Hille G, Agrawal S, Tummala P, Wybranski C, Pech M, Surov A, Saalfeld S (2023) Joint liver and hepatic lesion segmentation in MRI using a hybrid CNN with transformer layers. Comput Methods Programs Biomed 240:107647

    Article  PubMed  Google Scholar 

Download references

Funding

This work is funded by the Federal Ministry of Education and Research within the research campus STIMULATE under the grant number ‘13GW0473A’.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Georg Hille.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study, formal consent is not required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hille, G., Tummala, P., Spitz, L. et al. Transformers for colorectal cancer segmentation in CT imaging. Int J CARS 19, 2079–2087 (2024). https://doi.org/10.1007/s11548-024-03217-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-024-03217-9

Keywords

Navigation