Abstract
Chest X-ray (CXR) is a common imaging modality for examination of pneumonia. However, some pneumonia signs which are visible in CT may not be clearly identifiable in CXR. It is challenging to create a good ground truth for positive pneumonia cases based on CXR images especially for cases with small pneumonia lesions. In this paper, we propose a novel CT-based CXR synthesis framework, called ct2cxr, to perform data augmentation for pneumonia classification. Generative Adversarial Networks (GANs) were exploited and a customized loss function was proposed for model training to preserve the target pathology and maintain high image fidelity. Our results show that CXR images generated through style mixing can enhance the performance of general pneumonia classification models. Testing the models on a Covid-19 dataset shows similar improvements over the baseline models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apostolopoulos, I.D., Mpesiana, T.A.: COVID-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Phys. Eng. Sci. Med. 43(2), 635–640 (2020)
Armato, S.G., III., et al.: The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38(2), 915–931 (2011)
Bińkowski, M., Sutherland, D.J., Arbel, M., Gretton, A.: Demystifying mmd GANs. arXiv preprint arXiv:1801.01401 (2018)
Borghesi, A., Maroldi, R.: COVID-19 outbreak in Italy: experimental chest x-ray scoring system for quantifying and monitoring disease progression. Radiol. Med. (Torino) 125, 509–513 (2020)
Chowdhury, M.E., et al.: Can AI help in screening viral and COVID-19 pneumonia? IEEE Access 8, 132665–132676 (2020)
Goodfellow, I., et al.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27, 1–8 (2014)
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Adv. Neural Inf. Process. Syst. 30, 1–9 (2017)
Irvin, J., Rajpurkar, P., Ko, M., et al.: CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 590–597 (2019)
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of StyleGAN. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8110–8119 (2020)
Khan, A.I., Shah, J.L., Bhat, M.M.: CoroNet: a deep neural network for detection and diagnosis of COVID-19 from chest x-ray images. Comput. Methods Programs Biomed. 196, 105581 (2020)
Li, X., Yang, J., Zhu, Y.: Digitally reconstructed radiograph generation by an adaptive monte Carlo method. Phys. Med. Biol. 51(11), 2745 (2006)
Liu, S., Dowling, J.A., Engstrom, C., Greer, P.B., Crozier, S., Chandra, S.S.: Manipulating medical image translation with manifold disentanglement. arXiv preprint arXiv:2011.13615 (2020)
Loey, M., Smarandache, F., M Khalifa, N.E.: Within the lack of chest COVID-19 x-ray dataset: a novel detection model based on GAN and deep transfer learning. Symmetry. 12(4), 651 (2020)
Maghdid, H.S., Asaad, A.T., Ghafoor, K.Z., Sadiq, A.S., Mirjalili, S., Khan, M.K.: Diagnosing COVID-19 pneumonia from x-ray and CT images using deep learning and transfer learning algorithms. In: Multimodal Image Exploitation and Learning 2021, vol. 11734, p. 117340E. International Society for Optics and Photonics (2021)
Mostafavi, S.M.: COVID19-CT-Dataset: An Open-Access Chest CT Image Repository of 1000+ Patients with Confirmed COVID-19 Diagnosis (2021). https://doi.org/10.7910/DVN/6ACUZJ
Nguyen, D., et al.: Deep learning-based COVID-19 pneumonia classification using chest CT images: model generalizability. Front. Artif. Intell. 4, 694875 (2021)
Ren, Z., Yu, S.X., Whitney, D.: Controllable medical image generation via generative adversarial networks. Electron. Imaging 2021(11), 112–121 (2021)
Richardson, E., et al.: Encoding in style: a StyleGAN encoder for image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2287–2296 (2021)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. Adv. Neural Inf. Process. Syst. 29, 1–9 (2016)
Sherouse, G.W., Novins, K., Chaney, E.L.: Computation of digitally reconstructed radiographs for use in radiotherapy treatment design. Int. J. Rad. Oncol. * Biol.* Phys. 18(3), 651–658 (1990)
Signoroni, A., et al.: BS-Net: learning COVID-19 pneumonia severity on a large chest x-ray dataset. Medical Image Analysis p. 102046 (2021). https://doi.org/10.1016/j.media.2021.102046, https://www.sciencedirect.com/science/article/pii/S136184152100092X
Sim, J.Z.T., Ting, Y.H., Tang, Y., et al.: Diagnostic performance of a deep learning model deployed at a national COVID-19 screening facility for detection of pneumonia on frontal chest radiographs. In: Healthcare, vol. 10, p. 175. MDPI (2022)
Staub, D., Murphy, M.J.: A digitally reconstructed radiograph algorithm calculated from first principles. Med. Phys. 40(1), 011902 (2013)
Tov, O., Alaluf, Y., Nitzan, Y., Patashnik, O., Cohen-Or, D.: Designing an encoder for StyleGAN image manipulation. ACM Trans. Graph. (TOG) 40(4), 1–14 (2021)
Unberath, M., et al.: DeepDRR – a catalyst for machine learning in fluoroscopy-guided procedures. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 98–106. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_12
Waheed, A., Goyal, M., Gupta, D., Khanna, A., Al-Turjman, F., Pinheiro, P.R.: COVIDGAN: data augmentation using auxiliary classifier GAN for improved COVID-19 detection. IEEE Access 8, 91916–91923 (2020)
Wang, L., Chen, W., Yang, W., Bi, F., Yu, F.R.: A state-of-the-art review on image synthesis with generative adversarial networks. IEEE Access 8, 63514–63537 (2020)
Yan, T., Wong, P.K., Ren, H., Wang, H., Wang, J., Li, Y.: Automatic distinction between COVID-19 and common pneumonia using multi-scale convolutional neural network on chest ct scans. Chaos, Solitons Fractals 140, 110153 (2020)
Yousefzadeh, M., et al.: AI-corona: radiologist-assistant deep learning framework for COVID-19 diagnosis in chest CT scans. PLOS ONE. 16(5), e0250952 (2021)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 586–595 (2018)
Zhou, L., et al.: Lung swapping autoencoder: Learning a disentangled structure-texture representation of chest radiographs. arXiv preprint arXiv:2201.07344 (2022)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Acknowledgement
The work is partially funded by a GAP project ACCL/19-GAP012-R20H.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Yuen, P.H.H. et al. (2022). CT2CXR: CT-based CXR Synthesis for Covid-19 Pneumonia Classification. In: Lian, C., Cao, X., Rekik, I., Xu, X., Cui, Z. (eds) Machine Learning in Medical Imaging. MLMI 2022. Lecture Notes in Computer Science, vol 13583. Springer, Cham. https://doi.org/10.1007/978-3-031-21014-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-21014-3_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21013-6
Online ISBN: 978-3-031-21014-3
eBook Packages: Computer ScienceComputer Science (R0)