Abstract
The missing data issue is a common problem in multi-modal neuroimage (e.g., MRI and PET) based diagnosis of neurodegenerative disorders. Although various generative adversarial networks (GANs) have been developed to impute the missing data, most current solutions treat the image imputation and disease diagnosis as two standalone tasks without considering the impact of diagnosis on image synthesis, leading to less competent synthetic images to the diagnosis task. In this paper, we propose the collaborative diagnosis-synthesis framework (CDSF) for joint missing neuroimage imputation and multi-modal diagnosis of neurodegenerative disorders. Under the CDSF framework, there is an image synthesis module (ISM) and a multi-modal diagnosis module (MDM), which are trained in a collaborative manner. Specifically, ISM is trained under the supervision of MDM, which poses the feature-consistent constraint to the cross-modality image synthesis, while MDM learns the disease-related multi-modal information from both real and synthetic multi-modal neuroimages. We evaluated our CDSF model against five image synthesis methods and three multi-modal diagnosis models on an ADNI datasets with 1464 subjects. Our results suggest that the proposed CDSF model not only generates neuroimages with higher quality, but also achieves the state-of-the-art performance in AD identification and MCI-to-AD conversion prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Zhou, J., Yuan, L., Liu, J., Ye, J.: A multi-task learning formulation for predicting disease progression. In: ACM SIGKDD, pp. 814–822. ACM (2011)
Pan, Y., Liu, M., Lian, C., Xia, Y., Shen, D.: Spatially-constrained fisher representation for brain disease identification with incomplete multi-modal neuroimages. IEEE Trans. Med. Imag. 39(9), 2965–2975 (2020)
Baraldi, A.N., Enders, C.K.: An introduction to modern missing data analyses. J. Sch. Psychol. 48(1), 5–37 (2010)
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference Computer Vision Pattern Recognition, pp. 1125–1134 (2017)
Choi, Y., Choi, M., Kim, M., Ha, J., Kim, S., Choo, J.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE/CVF Conference Computer Vision Pattern Recognition, pp. 8789–8797 (2018)
Lee, D., Kim, J., Moon, W., Ye, J.C.: Collagan: Collaborative gan for missing image data imputation. In: Proceedings of the IEEE/CVF Conference Computer Vision Pattern Recognition, pp. 2482–2491(2019)
Huang, X., Liu, M.Y., Belongie, S., Kautz, J.: Multimodal unsupervised image-to-image translation. In: Proceedings of the European Conference Computer Vision, pp. 179–196 (2018)
Pan, Y., Liu, M., Lian, C., Zhou, T., Xia, Y., Shen, D.: Synthesizing missing PET from MRI with cycle-consistent generative adversarial networks for Alzheimer’s disease diagnosis. In: Proceedings of the International Conference Medical Image Comput. Computer Assisted Intervention, pp. 455–463 (2018)
Cohen, J.P., Luck, M., Honari, S.: Distribution matching losses can hallucinate features in medical image translation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 529–536. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_60
Pan, Y., Liu, M., Lian, C., Xia, Y., Shen, D.: Disease-image specific generative adversarial network for brain disease diagnosis with incomplete multi-modal neuroimages. In: Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.-T., Khan, A. (eds.) MICCAI 2019. LNCS, vol. 11766, pp. 137–145. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32248-9_16
Liu, Y., Pan, Y., Yang, W., Ning, Z., Yue, L., Liu, M.: Joint neuroimage synthesis and representation learning for conversion prediction of subjective cognitive decline. In: Proceedings of the International Conference Medical Image Comput. Computer Assisted Intervention, pp. 583–592, September 2020
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE/CVF Conference Computer Vision Pattern Recognition, pp. 770–778 (2016)
Jack, C., Bernstein, M., Fox, N., et al.: The Alzheimer’s disease neuroimaging initiative (ADNI): MRI methods. J. Magn. Reson. Imaging 27(4), 685–691 (2008)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Hore, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: Proceedings of the International Conference Pattern Recognition, pp. 2366–2369(2010)
Koyejo, O.O., Natarajan, N., Ravikumar, P.K., Dhillon, I.S.: Consistent binary classification with generalized performance metrics. In: Conference Neural Information Processing System, pp. 2744–2752 (2014)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grants 61771397, in part by the CAAI-Huawei MindSpore Open Fund under Grants CAAIXSJLJJ-2020-005B, and in part by the China Postdoctoral Science Foundation under Grants BX2021333.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pan, Y., Chen, Y., Shen, D., Xia, Y. (2021). Collaborative Image Synthesis and Disease Diagnosis for Classification of Neurodegenerative Disorders with Incomplete Multi-modal Neuroimages. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12905. Springer, Cham. https://doi.org/10.1007/978-3-030-87240-3_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-87240-3_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87239-7
Online ISBN: 978-3-030-87240-3
eBook Packages: Computer ScienceComputer Science (R0)
