Skip to main content

Advertisement

Log in

Patient-specific and global convolutional neural networks for robust automatic liver tumor delineation in follow-up CT studies

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Radiological longitudinal follow-up of tumors in CT scans is essential for disease assessment and liver tumor therapy. Currently, most tumor size measurements follow the RECIST guidelines, which can be off by as much as 50%. True volumetric measurements are more accurate but require manual delineation, which is time-consuming and user-dependent. We present a convolutional neural networks (CNN) based method for robust automatic liver tumor delineation in longitudinal CT studies that uses both global and patient specific CNNs trained on a small database of delineated images. The inputs are the baseline scan and the tumor delineation, a follow-up scan, and a liver tumor global CNN voxel classifier built from radiologist-validated liver tumor delineations. The outputs are the tumor delineations in the follow-up CT scan. The baseline scan tumor delineation serves as a high-quality prior for the tumor characterization in the follow-up scans. It is used to evaluate the global CNN performance on the new case and to reliably predict failures of the global CNN on the follow-up scan. High-scoring cases are segmented with a global CNN; low-scoring cases, which are predicted to be failures of the global CNN, are segmented with a patient-specific CNN built from the baseline scan. Our experimental results on 222 tumors from 31 patients yield an average overlap error of 17% (std = 11.2) and surface distance of 2.1 mm (std = 1.8), far better than stand-alone segmentation. Importantly, the robustness of our method improved from 67% for stand-alone global CNN segmentation to 100%. Unlike other medical imaging deep learning approaches, which require large annotated training datasets, our method exploits the follow-up framework to yield accurate tumor tracking and failure detection and correction with a small training dataset.

Flow diagram of the proposed method. In the offline mode (orange), a global CNN is trained as a voxel classifier to segment liver tumor as in [31]. The online mode (blue) is used for each new case. The input is baseline scan with delineation and the follow-up CT scan to be segmented. The main novelty is the ability to predict failures by trying the system on the baseline scan and the ability to correct them using the patient-specific CNN

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bilello M, Gokturk SB, Desser T, Napel S, Jeffrey RB, Beaulieu CF (2004) Automatic detection and classification of hypo-dense hepatic lesion on contrast-enhanced venous-phase CT. Med Phys 31(9):2584–2593

    Article  PubMed  Google Scholar 

  2. Bourquain H, Schenk A, Link F, Preim B, Peitgen OH (2002) Hepavision2a software assistant for preoperative planning in living related liver transplantation and oncologic liver surgery. Proc 16th Conf Comp Assist Radiol Surg, 341–6

  3. Chen EL (1998) An automatic diagnostic system for CT liver image classification. IEEE Trans Biomed Eng 45(6):783–794

    Article  PubMed  CAS  Google Scholar 

  4. Coghlin C, Murray GI (2010) Current and emerging concepts in tumor metastasis. J Pathol 222(1):1–15

    Article  PubMed  CAS  Google Scholar 

  5. Cohen AB, Diamant I, Klang E, Amitai M, Greenspan H (2014) Automatic detection and segmentation of liver metastatic lesions on serial CT examinations. Proc SPIE Med Imag Conf, 903519-27

  6. Deng X, Du G (2008) Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentation

  7. Eisenhauer E, Therasse P et al (2009) New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eu J Cancer 45(2):228–247

    Article  CAS  Google Scholar 

  8. Freiman M, Eliassaf O, Taieb Y, Joskowicz L, Azraq Y, Sosna J (2008) An iterative Bayesian approach for nearly automatic liver segmentation: algorithm and validation. Int J Comp Aided Radiol Surg 3:439–446

    Article  Google Scholar 

  9. Freiman M, Cooper O, Lischinski D, Joskowicz L (2010) Liver tumors segmentation from CTA images using voxels classification and affinity constraint propagation. Int J Comput Assist Radiol Surg 6(2):247–255

    Article  PubMed  Google Scholar 

  10. Greenspan H, van Ginneken B, Summers RM (2016) Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging 35(5):1153–1159

    Article  Google Scholar 

  11. Hassouna MS, Farag AA (2007) Multistencils fast marching methods: a highly accurate solution to the eikonal equation on cartesian domains. IEEE Trans Pattern Analy Mach Intell 29(9):1563–1574

    Article  Google Scholar 

  12. Hong JS, Kaneto T, Sekiguchi R, Park KH (2001) Automatic liver tumor detection from CT. IEEE Trans Inf Syst 84(6):741–748

    Google Scholar 

  13. Jia Y (2013) Caffe: An open source convolutional architecture for fast feature embedding. http://caffe.berkeleyvision.org

  14. Klein S, Staring M, Murphy K, Viergever MA, Pluim JP (2010) Elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205

    Article  PubMed  Google Scholar 

  15. Lewis RL (2007) Liver and biliary tract tumors. Cecil medicine, 23rd edn. Saunders Elsevier, Philadelphia (Ch. 206)

    Google Scholar 

  16. Li Y, Hara S, Shimura K (2006) A machine learning approach for locating boundaries of liver tumors in ct images. Proc 18th Int Conf Pattern Recogn, 400–3

  17. Li W, Jia F, Hu Q (2015) Automatic segmentation of liver tumor in CT images with deep convolutional neural networks. J. of Comput Commun 3(11):146

    Article  Google Scholar 

  18. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proc IEEE Conf Comput Vis Pattern Recog :3431–40

  19. Mala K, Sadasivam V, Avlagappan S (2007) Neural network based texture analysis of liver tumors from computed tomography images. Int J Biomed Sci 2:33–40

    Google Scholar 

  20. Masuda Y, Tateyama T, Xiong W, Zhou J, Wakamiya M, Kanasaki S, Furukawa A, Chen YW (2011) Liver tumor detection in CT images by adaptive contrast enhancement and the EM/MPM algorithm. 18th IEEE Int Conf Image Process :1421–1424

  21. Militzer A, Tietjen C, Hornegger J (2013) Learning a prior model for automatic liver lesion segmentation in follow-up CT images. Dept Informatik technical reports, CS-2013-03, ISSN 2191-5008

  22. Miller AB, Hoogstraten B, Staquet M, Winkler A (2011) Reporting results of cancer treatment. Cancer 47(1):207–214

    Article  Google Scholar 

  23. Moltz J, Bornemann L, Dicken V, Peitgen H (2008) Segmentation of liver metastases in CT scans by adaptive thresholding and morphological processing. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentation

  24. Moltz JH, Schwier M, Peitgen HO (2009) A general framework for automatic detection of matching lesions in follow-up CT. Proc IEEE Int Symp Biomed Imag, :843–6

  25. Pescia D, Paragios N, Chemouny S (2008) Automatic detection of liver tumors. Prof. 5th IEEE Int Symp Biomed Imaging :672–67

  26. Schmidt G, Binnig G, Kietzmann M, Kim J (2008) Cognition network technology for a fully automated 3D segmentation of liver tumors. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentation

  27. Shimizu A, Narihira T, Furukawa D, Kobatake H, Nawano S, Shinozaki K (2008) Ensemble segmentation using AdaBoost with application to liver lesion extraction from a CT volume. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentation

  28. Smeets D, Stijnen B, Loeckx D, Dobbelaer B, Suetens P (2008) Segmentation of liver metastases using a level set method with spiral-scanning technique and supervised fuzzy pixel classification. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentation

  29. Stawiaski J, Decenciere E, Bidault F (2008) Interactive liver tumor segmentation using graph-cuts and watershed. Proc. MICCAI workshop on 3D segmentation in the clinic: a grand challenge II—liver tumor segmentation

  30. Vivanti R, Joskowicz L, Karaaslan OA, Sosna J (2015a) Automatic lung tumor segmentation with leaks removal in follow-up CT studies. Int J Comp Assist Radiol Surg. https://doi.org/10.1007/s11548-015-1150-0 to appear

  31. Vivanti R, Ephrat A, Joskowicz L, Lev-Cohain N, Sosna J, (2015b) Automatic liver tumor segmentation in follow-up CT studies using convolutional neural networks. 1st International Workshop on Patch-based Techniques in Medical Imaging, MICCAI workshop Springer International Publishing, 49

  32. Weizman L, Ben-Sira L, Joskowicz L, Precel R, Constantini S, Ben-Bashat D (2010) Automatic segmentation and components classification of optic pathway gliomas in MRI. Med Image Comput Comput Assist Interv 103–110

  33. Wong D, Liu J, Fengshou Y, Tian Q, Xiong W, Zhou J, Wang SC (2008) A semi-automated method for liver tumor segmentation based on 2D region growing with knowledge-based constraints. Proc MICCAI Work 41(43):159

    Google Scholar 

  34. Zhou J, Xiong W, Tian Q, Qi Y, Liu J, Leow WK, Han T, Venkatesh SK, Wang SC (2008) Semi-automatic segmentation of 3D liver tumors from CT scans using voxel classification and propagational learning. Proc MICCAI Work 41:43–51

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by Grant 53681 from the Israel Ministry of Science, Technology and Space entitled: METASEG: a new medical image segmentation paradigm for clinical decision support and big data radiology.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Refael Vivanti.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Protection of human and animal rights statement

No animals or humans were involved in this research. All scans were anonymized before delivery to the researchers.

Electronic supplementary material

ESM 1

(DOCX 14.9 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vivanti, R., Joskowicz, L., Lev-Cohain, N. et al. Patient-specific and global convolutional neural networks for robust automatic liver tumor delineation in follow-up CT studies. Med Biol Eng Comput 56, 1699–1713 (2018). https://doi.org/10.1007/s11517-018-1803-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-018-1803-6

Keywords

Navigation