Skip to main content
SpringerLink
Log in

Synthetic CT images for semi-sequential detection and segmentation of lung nodules

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Accurately detecting and segmenting lung nodules from CT images play a critical role in the earlier diagnosis of lung cancer and thus have attracted much interest from the research community. However, due to the irregular shapes of nodules, and the low-intensity contrast between the nodules and other lung areas, precisely segmenting nodules from lung CT images is a very challenging task. In this paper, we propose a highly effective and robust solution to this problem by innovatively utilizing the changes of nodule shapes over continuous slices (inter-slice changes) and develop a deep learning based end-to-end system. Different from the existing 2.5D or 3D methods that attempt to explore the inter-slice features, we propose to create a novel synthetic image to depict the unique changing pattern of nodules between slices in distinctive colour patterns. Based on the new synthetic images, we then adopt the deep learning based image segmentation techniques and develop a modified U-Net architecture to learn the unique color patterns formed by nodules. With our proposed approach, the detection and segmentation of nodules can be achieved simultaneously with an accuracy significantly higher than the state of the arts by 10% without introducing high computation cost. By taking advantage of inter-slice information and form the proposed synthetic image, the task of lung nodule segmentation is done more accurately and effectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Aresta G, Jacobs C, Araújo T, Cunha A, Ramos I, van Ginneken B, Campilho A (2019) iw-net: an automatic and minimalistic interactive lung nodule segmentation deep network. Sci Rep 9 (1):1–9

    Article  Google Scholar 

  2. Armato SG III, McLennan G, Bidaut L, McNitt-Gray MF, Meyer CR, Reeves AP (2011) The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on ct scans. Med Phys 38(2):915–931

    Article  Google Scholar 

  3. Arnab A, Torr PH (2017) Pixelwise instance segmentation with a dynamically instantiated network. In: CVPR, vol 1, p 5

  4. Ben-Cohen A, Diamant I, Klang E, Amitai M, Greenspan H (2016) Fully convolutional network for liver segmentation and lesions detection. In: Deep learning and data labeling for medical applications. Springer, pp 77–85

  5. Cai J, Lu L, Xie Y, Xing F, Yang L (2017) Improving deep pancreas segmentation in ct and mri images via recurrent neural contextual learning and direct loss function. arXiv:1707.04912

  6. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018) Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  7. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in neural information processing systems, pp 2843–2851

  8. Dehmeshki J, Amin H, Valdivieso M, Ye X (2008) Segmentation of pulmonary nodules in thoracic CT scans: a region growing approach. IEEE Trans Med Imaging 27(4):467–480

    Article  Google Scholar 

  9. Faust O, Hagiwara Y, Hong TJ, Lih OS, Acharya UR (2018) Deep learning for healthcare applications based on physiological signals: a review. Comput Methods Progr Biomed 161:1–13

    Article  Google Scholar 

  10. Fedorov A, Hancock M, Clunie D, Brochhausen M, Bona J, Kirby J, Freymann J, Pieper S, Aerts H, Kikinis R (2018) Standardized representation of the LIDC annotations using dicom. Tech. rep. PeerJ Preprints

  11. Gao XW, Hui R, Tian Z (2017) Classification of ct brain images based on deep learning networks. Comput Methods Progr Biomed 138:49–56

    Article  Google Scholar 

  12. Hesamian MH, Jia W, He X, Kennedy PJ (2019) Atrous convolution for binary semantic segmentation of lung nodule. In: ICASSP 2019-2019 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1015–1019

  13. Hu S, Hoffman EA, Reinhardt JM (2001) Automatic lung segmentation for accurate quantitation of volumetric X-ray CT images. IEEE Trans Med Imaging 20(6):490–498

    Article  Google Scholar 

  14. Jiang J, Hu YC, Liu CJ, Halpenny D, Hellmann MD, Deasy JO, Mageras G, Veeraraghavan H (2018) Multiple resolution residually connected feature streams for automatic lung tumor segmentation from ct images. IEEE Trans Med Imaging 38(1):134–144

    Article  Google Scholar 

  15. Jin D, Xu Z, Tang Y, Harrison AP, Mollura DJ (2018) Ct-realistic lung nodule simulation from 3d conditional generative adversarial networks for robust lung segmentation. arXiv:1806.04051

  16. John J, Mini M (2016) Multilevel thresholding based segmentation and feature extraction for pulmonary nodule detection. Procedia Technol 24:957–963

    Article  Google Scholar 

  17. Kamnitsas K, Ledig C, Newcombe VF, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2017) Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation. Medical image analysis 36:61–78

    Article  Google Scholar 

  18. Keshani M, Azimifar Z, Tajeripour F, Boostani R (2013) Lung nodule segmentation and recognition using SVM classifier and active contour modeling: a complete intelligent system. Comput Biol Med 43 (4):287–300

    Article  Google Scholar 

  19. Li X, Chen H, Qi X, Dou Q, Fu CW, Heng PA (2018) H-denseunet: hybrid densely connected unet for liver and tumor segmentation from ct volumes. IEEE Trans Med Imaging 37(12):2663–2674

    Article  Google Scholar 

  20. Liu X, Hou F, Qin H, Hao A (2018) Multi-view multi-scale cnns for lung nodule type classification from ct images. Pattern Recognit 77:262–275

    Article  Google Scholar 

  21. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  22. Milletari F, Navab N, Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth international conference on 3D vision (3DV). IEEE, pp 565–571

  23. Moccia S, De Momi E, El Hadji S, Mattos LS (2018) Blood vessel segmentation algorithms—review of methods, datasets and evaluation metrics. Comput Methods Progr Biomed 158 :71–91

    Article  Google Scholar 

  24. Nam CM, Kim J, Lee KJ (2018) “Lung nodule segmentation with convolutional neural network trained by simple diameter information,” in Proc. 1st Conf. Med. Imag. Deep Learn. (MIDL), 2018

  25. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  26. Roth HR, Lu L, Seff A, Cherry KM, Hoffman J, Wang S, Liu J, Turkbey E, Summers RM (2014) A new 2.5 D representation for lymph node detection using random sets of deep convolutional neural network observations. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 520–527

  27. Shakibapour E, Cunha A, Aresta G, Mendonça AM, Campilho A (2019) An unsupervised metaheuristic search approach for segmentation and volume measurement of pulmonary nodules in lung CT scans. Expert Syst Appl 119:415–428

    Article  Google Scholar 

  28. Shen W, Zhou M, Yang F, Yang C, Tian J (2015) Multi-scale convolutional neural networks for lung nodule classification. In: International conference on information processing in medical imaging. Springer, pp 588–599

  29. Siegel RL, Miller KD, Jemal A (2017) Cancer statistics, 2017. CA: A Cancer J Clin 67 (1):7–30

    Google Scholar 

  30. Sivakumar S, Chandrasekar C (2013) Lung nodule detection using fuzzy clustering and support vector machines. Int J Eng Technol 5(1):179–185

    Google Scholar 

  31. Sun W, Huang X, Tseng TLB, Qian W (2017) Automatic lung nodule graph cuts segmentation with deep learning false positive reduction. In: Medical imaging 2017: computer-aided diagnosis, vol 10134. International Society for Optics and Photonics, p 101343M

  32. Tseng KL, Lin YL, Hsu W, Huang CY (2017) Joint sequence learning and cross-modality convolution for 3d biomedical segmentation. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 3739–3746

  33. Valente IRS, Cortez PC, Neto EC, Soares JM, de Albuquerque VHC, Tavares JMR (2016) Automatic 3d pulmonary nodule detection in ct images: a survey. Comput Methods Progr Biomed 124:91–107

    Article  Google Scholar 

  34. Wang S, Zhou M, Liu Z, Liu Z, Gu D, Zang Y, Dong D, Gevaert O, Tian J (2017) Central focused convolutional neural networks: developing a data-driven model for lung nodule segmentation. Med Image Anal 40:172–183

    Article  Google Scholar 

  35. Wang W, Lu Y, Wu B, Chen T, Chen DZ, Wu J (2018) Deep active self-paced learning for accurate pulmonary nodule segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 723–731

  36. Xiao Y, Wu J, Lin Z, Zhao X (2018) A deep learning-based multi-model ensemble method for cancer prediction. Comput Methods Progr Biomed 153:1–9

    Article  Google Scholar 

Download references

Funding

No funding was received.

Author information

Authors and Affiliations

  1. School of Electrical and Data Engineering, University of Technology Sydney, Sydney, Australia

    Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He & Qingqing Wang

  2. School of Computer Science, Australian Artificial Intelligence Institute, University of Technology Sydney, Sydney, Australia

    Paul J. Kennedy

Authors
  1. Mohammad Hesam Hesamian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenjing Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiangjian He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qingqing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul J. Kennedy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Hesam Hesamian.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Informed consent

For this type of study a formal consent is not required.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hesamian, M.H., Jia, W., He, X. et al. Synthetic CT images for semi-sequential detection and segmentation of lung nodules. Appl Intell 51, 1616–1628 (2021). https://doi.org/10.1007/s10489-020-01914-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-020-01914-x

Keywords

Search

Navigation