Skip to main content
SpringerLink
Log in

A multiple organ segmentation system for CT image series using Attention-LSTM fused U-Net

  • 1195: Deep Learning for Multimedia Signal Processing and Applications
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The multi-organ contouring (MOC) task is required when a radiotherapy is performed to eradicate the cancerous tissue while minimizing the dosage delivered to the surrounding healthy organs. Currently most of the task is done manually with enormous labor and time cost. To reduce the scheduling waiting time from increasing cancer population, it is beneficial to both the patients and therapeutists to have an automatic contouring tool. In this work an Attention-LSTM fused U-Net model is proposed to perform the multiple organ segmentation from a CT image series. The organs to be delineated include lung, liver, stomach, esophagus, heart, and kidneys. To train and evaluate our model, the CT image series of 146 patients was acquired from a local hospital with IRB approval. The segmentation accuracy of the six organs in terms of Dice Similarity Coefficient (DSC) were 99.27%, 95.48%, 88.53%, 80.81%, 93.8%, and 93.46%, respectively. To make the AI-embedded MOC system readily applicable in clinical environments, a data processing workflow and the corresponding GUI were also implemented and published on Github. The doctors can download the CT image data from the PACS server, use our system to perform MOC tasks, and output the contouring results in DICOM-RT format so they can be uploaded back to the treatment planning system for further fine-tuning and dosage/path calculation. To our best knowledge the work might be the first non-commercial model-integrated system compatible with the commercial treatment planning systems and ready to be used by the doctors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

No.

Code availability

https://github.com/chenpin627/Organ-Segmentation-UI

References

  1. Bilic P, Christ PF et al (2019) The Liver Tumor Segmentation Benchmark (LiTS). arXiv:1901.04056

  2. Chen PH et al (2020) Attention-LSTM fused U-Net architecture for organ segmentation in CT images. International Symposium on Computer, Consumer and Control (IS3C)

  3. Huang H et al (2020) UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. ICASSP 2020–2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, pp 1055–1059

  4. Liu Y, Lei Y, Fu Y, Wang T, Tang X, Jiang X, Curran WJ, Liu T, Patel P, Yang X (2020) CT-based multi-organ segmentation using a 3D self-attention U-Net network for pancreatic radiotherapy. Med Phys 47:4316–4324. https://doi.org/10.1002/mp.14386

    Article  Google Scholar 

  5. Macchia ML et al (2012) Systematic evaluation of three different commercial software solutions for automatic segmentation for adaptive therapy in head-and-neck, prostate and pleural cancer. Radiat Oncol 7:160

    Article  Google Scholar 

  6. Maitre AL et al (2012) Impact of the accuracy of automatic tumour functional volume delineation on radiotherapy treatment planning. Phys Med Biol 57(17):5381–5397

    Article  Google Scholar 

  7. Mattiucci GC, Boldrini L, Chiloiro G, D’Agostino GR, Chiesa S, de Rose F, Azario L, Pasini D, Gambacorta MA, Balducci M, Valentini V (2013) Automatic delineation for replanning in nasopharynx radiotherapy: what is the agreement among experts to be considered as benchmark? Acta Oncol 52(7):1417–1422

    Article  Google Scholar 

  8. Novikov AA et al (2019) Deep sequential segmentation of organs in volumetric medical scans. IEEE Trans Med Imaging 38(5):1207–1215

    Article  Google Scholar 

  9. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K et al ( 2018) Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:180403999

  10. Ronneberger O et al (2015) U-net: convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, Cham

    Google Scholar 

  11. Shen C, Milletari F et al (2019) Improving V-Nets for multi-class abdominal organ segmentation. Proc. SPIE 10949, Medical Imaging 2019: Image Processing, 109490B

  12. Shi X et al (2015) Convolutional LSTM Network: a machine learning approach for precipitation nowcasting. In: Proceedings of the 28th International Conference on Neural Information Processing Systems—volume 1 (NIPS’15). MIT Press, Cambridge

    Google Scholar 

  13. Sudre CH et al (2017) Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, Cham, pp 240–248

    Chapter  Google Scholar 

  14. Taha AA, Hanbury A (2015) Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15(1):29

    Article  Google Scholar 

  15. Tang H et al 2021 Spatial context-aware self-attention model for multi-organ segmentation. arXiv:2012.09279, accepted by WACV

  16. Tang Y et al (2021) High-resolution 3D abdominal segmentation with random patch network fusion. Med Image Anal 69:101894

    Article  Google Scholar 

  17. Thomson D et al (2014) Evaluation of an automatic segmentation algorithm for definition of head and neck organs at risk. Radiat Oncol 9:173

    Article  Google Scholar 

  18. Tian Z et al (2020) A multiscale-based adjustable convolutional neural network for multiple organ segmentation. Wirel Commun Mob Comput 2020:Article ID 9595687 13 pages

    Google Scholar 

  19. Wang Y, Zhou Y, Shen W, Park S, Fishman EK, Yuille AL (2019) Abdominal multi-organ segmentation with organ attention networks and statistical fusion. Med Image Anal 55:88–102

    Article  Google Scholar 

  20. Yang J, Beadle BM, Garden AS, Schwartz DL, Aristophanous M (2015) A multimodality segmentation framework for automatic target delineation in head and neck radiotherapy. Med Phys 42(9):5310–5320

    Article  Google Scholar 

  21. Yu L et al (2017) Volumetric ConvNets with mixed residual connections for automated prostate segmentation from 3D MR images. Thirty-first AAAI conference on artificial intelligence

Download references

Acknowledgments

The work is supported by the grants from Ministry of Science and Technology, Taiwan (No.: MOST 108-2221-E-194 -042, MOST 110-2221-E-194-011).

Funding

Supported by the grant from Ministry of Science and Technology, Taiwan (No.: MOST 108–2221-E-194 -042).

Author information

Authors and Affiliations

  1. Department of Computer Science and Information Engineering, National Chung Cheng University, Chiayi, Taiwan

    Pin-Hsiu Chen, Cheng-Hsien Huang, Wen-Tse Chiu, Chen-Mao Liao, Yu-Ruei Lin & Wei-Min Liu

  2. Department of Radiation Oncology, Dalin Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, Chiayi, Taiwan

    Shih-Kai Hung, Liang-Cheng Chen, Hui-Ling Hsieh, Wen-Yen Chiou, Moon-Sing Lee & Hon-Yi Lin

Authors
  1. Pin-Hsiu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cheng-Hsien Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wen-Tse Chiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen-Mao Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yu-Ruei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shih-Kai Hung
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Liang-Cheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hui-Ling Hsieh
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Wen-Yen Chiou
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Moon-Sing Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Hon-Yi Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Wei-Min Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei-Min Liu.

Ethics declarations

Conflicts of interest/competing interests

No conflict of interest regarding to the manuscript.

Ethics approval

IRB number B10804010-1 approved by Tzu Chi Hospital in Chiayi, Taiwan.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, PH., Huang, CH., Chiu, WT. et al. A multiple organ segmentation system for CT image series using Attention-LSTM fused U-Net. Multimed Tools Appl 81, 11881–11895 (2022). https://doi.org/10.1007/s11042-021-11889-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11889-7

Keywords

Search

Navigation