Abstract
Synthesis of computed tomography (CT) images from magnetic resonance (MR) images plays an important role in radiotherapy treatment planning. CycleGANs have achieved promising performance in unsupervised MR-to-CT synthesis. However, the inter-modality gap between the two modalities and the loss of high-frequency information in the synthetic CT images are still not well addressed. In this paper, we propose a spatially invariant and frequency-aware CycleGAN (SF-CycleGAN) to improve the performance of unsupervised MR-to-CT synthesis. Specifically, we introduce a translation-invariant generator to generate CT from MR images, while maintaining the invariance of spatial feature during translation for those positions having similar characteristics. Furthermore, we define a frequency-consistent loss to promote the consistency of the frequency between real and synthesized images and adaptively guide the model to pay more attention to synthesizing the harder-frequency (e.g., higher-frequency) parts. Intensive results in unpaired brain MR-to-CT image synthesis demonstrate that our method provides both quantitatively and qualitatively superior performance as compared to the baseline (CycleGAN) and other state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, L., et al.: MRI-based treatment planning for radiotherapy dosimetric verification for prostate IMRT. Int. J.. Radiat. Oncol. * Biol.* Phys. 60(2), 636–647 (2004)
Khoo, V., Joon, D.: New developments in MRI for target volume delineation in radiotherapy. Br. J. Radiol. 79(special_issue_1), S2–S15 (2006)
Wang, T., et al.: A review on medical imaging synthesis using deep learning and its clinical applications. J. Appl. Clin. Med. Phys. 22(1), 11–36 (2021)
Guerreiro, F., et al.: Evaluation of a multi-atlas CT synthesis approach for MRI-only radiotherapy treatment planning. Physica Med. 35, 7–17 (2017)
Xu, R., et al.: SGDA: towards 3D universal pulmonary nodule detection via slice grouped domain attention. IEEE/ACM Trans. Comput. Biol. Bioinform. (2023)
Xu, R., Luo, Y., Du, B., Kuang, K., Yang, J.: LSSANet: a long short slice-aware network for pulmonary nodule detection. In: Wang, L., et al. (eds.) MICCAI 2022. Lecture Notes in Computer Science, vol. 13431, pp. 664–674. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16431-6_63
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)
Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)
Wolterink, J.M., Dinkla, A.M., Savenije, M.H.F., Seevinck, P.R., van den Berg, C.A.T., Išgum, I.: Deep MR to CT synthesis using unpaired data. In: Tsaftaris, S.A., Gooya, A., Frangi, A.F., Prince, J.L. (eds.) SASHIMI 2017. LNCS, vol. 10557, pp. 14–23. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68127-6_2
Hiasa, Y., et al.: Cross-modality image synthesis from unpaired data using CycleGAN. In: Gooya, A., Goksel, O., Oguz, I., Burgos, N. (eds.) SASHIMI 2018. LNCS, vol. 11037, pp. 31–41. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00536-8_4
Yang, H., et al.: Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN. IEEE Trans. Med. Imaging 39(12), 4249–4261 (2020)
Xiang, L., et al.: Deep embedding convolutional neural network for synthesizing CT image from T1-weighted MR image. Med. Image Anal. 47, 31–44 (2018)
Shi, Z., Mettes, P., Zheng, G., Snoek, C.: Frequency-supervised MR-to-CT image synthesis. In: Engelhardt, S., et al. (eds.) DGM4MICCAI/DALI -2021. LNCS, vol. 13003, pp. 3–13. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88210-5_1
Mathieu, M., Couprie, C., LeCun, Y.: Deep multi-scale video prediction beyond mean square error. arXiv preprint: arXiv:1511.05440 (2015)
Xu, Z.J., Zhou, H.: Deep frequency principle towards understanding why deeper learning is faster. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 10541–10550 (2021)
Chen, J., Wang, X., Guo, Z., Zhang, X., Sun, J.: Dynamic region-aware convolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8064–8073 (2021)
Jiang, L., Dai, B., Wu, W., Loy, C.C.: Focal frequency loss for image reconstruction and synthesis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13919–13929 (2021)
Miyato, T., Kataoka, T., Koyama, M., Yoshida, Y.: Spectral normalization for generative adversarial networks. arXiv preprint: arXiv:1802.05957 (2018)
Han, X.: MR-based synthetic CT generation using a deep convolutional neural network method. Med. Phys. 44(4), 1408–1419 (2017)
Cusumano, D., et al.: A deep learning approach to generate synthetic CT in low field MR-guided adaptive radiotherapy for abdominal and pelvic cases. Radiother. Oncol. 153, 205–212 (2020)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Porter, E., et al.: Gamma knife MR/CT/RTSTRUCT sets with hippocampal contours. Cancer Imaging Archive (2022)
Clark, K., et al.: The cancer imaging archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26(6), 1045–1057 (2013)
Tang, H., Liu, H., Xu, D., Torr, P.H., Sebe, N.: AttentionGAN: unpaired image-to-image translation using attention-guided generative adversarial networks. IEEE Trans. Neural Netw. Learn. Syst. (2021)
Park, T., Efros, A.A., Zhang, R., Zhu, J.-Y.: Contrastive learning for unpaired image-to-image translation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12354, pp. 319–345. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58545-7_19
Fu, H., Gong, M., Wang, C., Batmanghelich, K., Zhang, K., Tao, D.: Geometry-consistent generative adversarial networks for one-sided unsupervised domain mapping. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2427–2436 (2019)
Han, J., Shoeiby, M., Petersson, L., Armin, M.A.: Dual contrastive learning for unsupervised image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 746–755 (2021)
Guo, J., Li, J., Fu, H., Gong, M., Zhang, K., Tao, D.: Alleviating semantics distortion in unsupervised low-level image-to-image translation via structure consistency constraint. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18249–18259 (2022)
Acknowledgements.
This work is supported by the Science and Technology Innovation Cultivation Foundation of Zhongnan Hospital of Wuhan University (ZNPY2019095), the Medical Science and Technology Innovation Platform Project of Zhongnan Hospital of Wuhan University (PTXM2022033), the National Natural Science Foundation of China (No.62262026), the project of Jiangxi Education Department (No.GJJ211111), and the Fundamental Research Funds for the Central Universities (No. 2042023kf1033).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Song, S., Zhang, J., Hu, W., Luo, Y., Zhou, X. (2023). Spatially Invariant and Frequency-Aware CycleGAN for Unsupervised MR-to-CT Synthesis. In: Iliadis, L., Papaleonidas, A., Angelov, P., Jayne, C. (eds) Artificial Neural Networks and Machine Learning – ICANN 2023. ICANN 2023. Lecture Notes in Computer Science, vol 14262. Springer, Cham. https://doi.org/10.1007/978-3-031-44201-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-44201-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44200-1
Online ISBN: 978-3-031-44201-8
eBook Packages: Computer ScienceComputer Science (R0)