Skip to main content

Advertisement

Log in

Synthetic CT image generation of shape-controlled lung cancer using semi-conditional InfoGAN and its applicability for type classification

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

In recent years, convolutional neural network (CNN), an artificial intelligence technology with superior image recognition, has become increasingly popular and frequently used for classification tasks in medical imaging. However, the amount of labelled data available for classifying medical images is often significantly less than that of natural images, and the handling of rare diseases is often challenging. To overcome these problems, data augmentation has been performed using generative adversarial networks (GANs). However, conventional GAN cannot effectively handle the various shapes of tumours because it randomly generates images. In this study, we introduced semi-conditional InfoGAN, which enables some labels to be added to InfoGAN, for the generation of shape-controlled tumour images. InfoGAN is a derived model of GAN, and it can represent object features in images without any label.

Methods

Chest computed tomography images of 66 patients diagnosed with three histological types of lung cancer (adenocarcinoma, squamous cell carcinoma, and small cell lung cancer) were used for analysis. To investigate the applicability of the generated images, we classified the histological types of lung cancer using a CNN that was pre-trained with the generated images.

Results

As a result of the training, InfoGAN was possible to generate images that controlled the diameters of each lesion and the presence or absence of the chest wall. The classification accuracy of the pre-trained CNN was 57.7%, which was higher than that of the CNN trained only with real images (34.2%), thereby suggesting the potential of image generation.

Conclusion

The applicability of semi-conditional InfoGAN for feature learning and representation in medical images was demonstrated in this study. InfoGAN can perform constant feature learning and generate images with a variety of shapes using a small dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of data and materials

The datasets analysed during this study are not publicly available due to included patient information.

Code availability

Source codes used in this study are available from the corresponding author upon request.

References

  1. Fujita H (2020) AI-based computer-aided diagnosis (AI-CAD): the latest review to read first. Radiol Phys Technol 13:6–19

    Article  Google Scholar 

  2. Gao XW, Hui R, Tian Z (2017) Classification of CT brain images based on deep learning networks. Comput Methods Programs Biomed 138:49–56

    Article  Google Scholar 

  3. Araújo T, Aresta G, Castro E, Rouco J, Aguiar P, Eloy C, Polónia A, Campilho A (2017) Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 12(6):e0177544

    Article  Google Scholar 

  4. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) ImageNet: A large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248–255

  5. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional Neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  6. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the 28th annual conference on neural information processing systems, pp 2672–2680

  7. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A (2018) Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. Cancer J Clin 68(6):394–424

    Article  Google Scholar 

  8. Hirsch FR, Franklin WA, Gazdar AF, Bunn PA Jr (2001) Early detection of lung cancer: clinical perspectives of recent advances in biology and radiology. Clin Cancer Res 7:5–22

    CAS  PubMed  Google Scholar 

  9. Henschke CI, Yankelevitz DF, Libby DM, Pasmantier MW, Miettinen OS (2006) Survival of patients with stage I lung cancer detected on CT screening. New Engl J Med 355(17):1763–1771

    Article  Google Scholar 

  10. Alakwaa W, Nassef M, Badr A (2017) Lung cancer detection and classification with 3D convolutional neural network (3D-CNN). Int J Adv Comput Sci Appl 8(8):409–417

    Google Scholar 

  11. Qin R, Wang Z, Jiang L, Qiao K, Hai J, Chen J, Xu J, Shi D, Yan B (2020) Fine-grained lung cancer classification from PET and CT images based on multidimensional attention mechanism. BioMed Res Int 2020:6153657

    Google Scholar 

  12. Teramoto A, Tsukamoto T, Yamada A, Kiriyama Y, Imaizumi K, Saito K, Fujita H (2020) Deep learning approach to classification of lung cytological images: two-step training using actual and synthesized images by progressive growing of generative adversarial networks. PLoS ONE 15(3):e0229951

    Article  CAS  Google Scholar 

  13. Travis WD, Asamura H, Bankier AA, Beasley MB, Detterbeck F, Flieder DB, Goo JM, MacMahon H, Naidich D, Nicholson AG, Powell CA, Prokop M, Rami-Porta R, Rusch V, Schil P, Yatabe Y (2016) The IASLC Lung cancer staging project: proposals for coding T categories for subsolid nodules and assessment of tumour size in part-solid tumours in the Forthcoming Eighth Edition of the TNM Classification of Lung Cancer. J Thorac Oncol 11(8):1204–1223

    Article  Google Scholar 

  14. Aberle DR, DeMello S, Berg CD, Black WC, Brewer B, Church TR, Clingan KL, Duan F, Fagerstrom RM, Gareen IF, Gatsonis CA, Gierada DS (2013) Results of the two incidence screenings in the national lung screening trial. N Engl J Med 369(10):920–931

    Article  CAS  Google Scholar 

  15. Wiener RS, Schwartz LM, Woloshin S, Welch HG (2011) Population-based risk for complications after transthoracic needle lung biopsy of a pulmonary nodule: an analysis of discharge records. Ann Int Med 155(3):137–144

    Article  Google Scholar 

  16. Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: A conference paper at ICLR

  17. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784.

  18. Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) Interpretable representation learning by information maximizing generative adversarial nets. Adv Neural Inf Process Syst 29:2180–2188

    Google Scholar 

  19. Iqbal T, Ali H (2018) Generative adversarial network for medical images (MI-GAN). J Med Syst 42(11):231

    Article  Google Scholar 

  20. Wolterink JM, Dinkla AM, Savenije MH, Seevinck PR, Berg CA, Isgum I (2017) Deep MR to CT synthesis using unpaired data. In: Simulation and synthesis in medical imaging, pp 14–23

  21. Dar SU, Yurt M, Karacan L, Erdem A, Erdem E, Çukur T (2019) Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans Med Imaging 38(10):2375–2388

    Article  Google Scholar 

  22. Han C, Hayashi H, Rundo L, Araki R, Shimoda W, Muramatsu S, Furukawa Y, Mauri G, Nakayama H (2018) GAN-based synthetic brain MR image generation. In Proceedings of IEEE 15th international symposium on biomedical imaging, pp 734–738

  23. Gatys L, Ecker A, Bethge M (2016) Image style transfer using convolutional neural networks. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 2414–2423

  24. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of IEEE international conference on computer vision (ICCV), pp 2242–2251

  25. Onishi Y, Teramoto A, Tsujimoto M, Tsukamoto T, Saito K, Toyama H, Imaizumi K, Fujita H (2019) Automated pulmonary nodule classification in computed tomography images using a deep convolutional neural network trained by generative adversarial networks. BioMed Res Int 2019:6051939

    PubMed  PubMed Central  Google Scholar 

  26. Frid-Adar M, Klang E, Amitai M, Goldberger J, Greenspan H (2018) Synthetic data augmentation using GAN for improved liver lesion classification. In: proceedings of IEEE 15th international symposium on biomedical imaging, pp 289–293

  27. Salehinejad H, Valaee S, Dowdell T, Colak E, Barfett J (2018) Generalization of deep neural networks for chest pathology classification in X-rays using generative adversarial networks. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp 990–994

  28. Srivastava A, Valkov L, Russell C, Gutmann MU, Sutton C (2017) VEEGAN: Reducing mode collapse in GANs using implicit variational learning. Adv Neural Inf Process Syst 30:3308–3318

    Google Scholar 

  29. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on machine learning vol 37, pp. 448–456

  30. Heusel M, Ramsauer H, Unterthiner T, Nessler B (2017) GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Adv Neural Inf Process Syst 30:6629–6640

    Google Scholar 

  31. McInnes L, Healy J, Melville J (2018) UMAP: uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426

  32. Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM international conference on multimedia, pp 675–678

Download references

Acknowledgements

This research was partially supported by a Grant-in-Aid for Scientific Research on Innovative Areas (Multidisciplinary Computational Anatomy, No. 26108005) and a Grant-in-Aid for Scientific Research (No. 17K09070), MEXT, Japan.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Atsushi Teramoto.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethics approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Toda, R., Teramoto, A., Tsujimoto, M. et al. Synthetic CT image generation of shape-controlled lung cancer using semi-conditional InfoGAN and its applicability for type classification. Int J CARS 16, 241–251 (2021). https://doi.org/10.1007/s11548-021-02308-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02308-1

Keywords

Navigation