Qing Guo
ABSTRACT
Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) can facilitate the diagnosis of Alzheimer's disease (AD). But PET data are not always available due to high financial costs and some other reasons. A common research approach to this problem is to filter out incomplete data, which will reduce the sample size. In order to solve the problem of missing PET data, we propose a new attention GAN model AMIGAN, which learns an end-to-end mapping function to generate PET scans from MRI and disease clinical information. We design a 3D attention multi-scale convolutional U-Net generator architecture and implement a pyramidal convolutional attention module PCAM for the generator skip connection process. PCAM can focus on image detail and improve the visual quality of the synthesized PET scans. Moreover, two discriminators are built for image authenticity discrimination and disease information category discrimination respectively, and disease information is integrated into the generated image. In addition, the standard adversarial loss is supplemented with structural similarity loss, reconstruction loss and disease information category loss components, which reduce the reconstruction error and enhance structural consistency across different angles. Experiments show that AMIGAN has better image generation performance than previously proposed methods.
- C. R. Mcdonald, L. Gharapetian, L. K. Mcevoy, C. Fennema-Notestine, and A. M. Dale., “Relationship between regional atrophy rates and cognitive decline in mild cognitive impairment,” Neurobiology of Aging 33(2), 242–253 (2012).Google Scholar
Cross Ref
- Y. Huang, J. Xu, Y. Zhou, T. Tong, X. Zhuang, and Adn Initiative., “Diagnosis of alzheimer's disease via multi-modality 3d convolutional neural network,” Frontiers in neuroscience 13 (2019).Google Scholar
- H. Kang, J. Park , K. Cho , D.-Y. Kang ., “Visual and quantitative evaluation of amyloid brain pet image synthesis with generative adversarial network,” Applied Sciences 10 (2020).Google Scholar
- A.M. Rossetto, W. Zhou, “Gandalf: peptide generation for drug design using sequential and structural generative adversarial networks,” Proceedings of the 11th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics (2020).Google ScholarDigital Library
- Y. Pan, M. Liu, C. Lian, Y. Xia, D. Shen, “Spatially-constrained fisher representation for brain disease identification with incomplete multi-modal neuroimages”, IEEE Trans Med Imaging 39 (9), 2965–2975 (2020).Google ScholarCross Ref
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, .et al, “Generative adversarial nets,” , Neural Information Processing Systems,(2014).Google Scholar
- Kenji Ono, Yutaro Iwamoto, Yen-Wei Chen, and Masahiro Nonaka, "Automatic Segmentation of Infant Brain Ventricles with Hydrocephalus in MRI Based on 2.5D U-Net and Transfer Learning," Journal of Image and Graphics, Vol. 8, No. 2, pp. 42-46, June 2020. doi: 10.18178/joig.8.2.42-46Google ScholarCross Ref
- Yuki Enokiya, Yutaro Iwamoto, Yen-Wei Chen, and Xian-Hua Han, "Automatic Liver Segmentation Using U-Net with Wasserstein GANs," Journal of Image and Graphics, Vol. 6, No. 2, pp. 152-159, December 2018. doi: 10.18178/joig.6.2.152-159Google ScholarCross Ref
- Yusuke Ikeda, Keisuke Doman, Yoshito Mekada, and Shigeru Nawano, "Lesion Image Generation Using Conditional GAN for Metastatic Liver Cancer Detection," Journal of Image and Graphics, Vol. 9, No. 1, pp. 27-30, March 2021. doi: 10.18178/joig.9.1.27-30Google ScholarCross Ref
- Z. Zha, B. Wen, X. Yuan, J. Zhou, J. Zhou, and C. Zhu., “Triply complementary priors for image restoration,” IEEE transactions on image processing : a publication of the IEEE Signal Processing Society 30, 5819–5834, (2021).Google ScholarCross Ref
- Z. Zha, X. Yuan, Bihan Wen, Jiantao Zhou, Jiachao Zhang, and Ce Zhu. “From rank estimation to rank approximation,” Rank residual constraint for image restoration (2018).Google Scholar
- Y. Wang, L. Zhou, B. Yu, L. Wang, C. Zu, D. S. Lalush, W. Lin, X. Wu, J. Zhou, and D. Shen. 3d auto-context-based locality adaptive multi-modality gans for pet synthesis. IEEE transactions on medical imaging, 38(6),1328, (2019).Google ScholarCross Ref
- Yongsheng Pan, Mingxia Liu, Chunfeng Lian, Tao Zhou, Yong Xia, and Dinggang Shen., “Synthesizing missing pet from mri with cycle-consistent generative adversarial networks for alzheimer's disease diagnosis,” Springer, Cham (2018).Google Scholar
- Yongsheng Pan, Mingxia Liu, Chunfeng Lian, Yong Xia, and Dinggang Shen., “Disease-image specific generative adversarial network for brain disease diagnosis with incomplete multi-modal neuroimages,” Springer, Cham (2019).Google ScholarDigital Library
- Xingyu Gao, Feng Shi, Dinggang Shen, and Manhua Liu., “Task-induced pyramid and attention gan for multimodal brain image imputation and classification in alzheimer's disease,” IEEE Journal of Biomedical and Health Informatics, 26(1), 36–43 (2022).Google ScholarCross Ref
- A. Sikka, Skand, J. S. Virk, and D. R. Bathula., “Mri to pet cross-modality translation using globally and locally aware gan (gla-gan) for multi-modal diagnosis of alzheimer's disease,” arXiv preprint arXiv:2108.02160 (2021).Google Scholar
- G. Huang, Z. Liu, Vdm Laurens, and K. Q. Weinberger., “Densely connected convolutional networks.” IEEE Computer Society (2017).Google ScholarCross Ref
- Sanghyun Woo, Jongchan Park, Joon Young Lee, and In So Kweon., “Cbam: Convolutional block attention module,” Springer, Cham, (2018).Google Scholar
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox., “U-net: Convolutional networks for biomedical image segmentation,” Springer, Cham (2015).Google Scholar
- P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros., “Image-to-image translation with conditional adversarial networks.” IEEE (2016).Google Scholar
- Euijin Jung, Miguel Luna, and Sang Hyun Park., “Conditional gan with 3d discriminator for mri generation of alzheimer's disease progression,” Pattern Recognition, 133:109061 (2023).Google ScholarDigital Library
- Li, Hanzhi, Jianping Jia, and Zhiqiang Yang., "Mini-mental state examination in elderly Chinese: a population-based normative study," Journal of Alzheimer's disease 53(2) 487-496 (2016).Google ScholarCross Ref
- S. Hu, J. Yuan, and S. Wang., “Cross-modality synthesis from mri to pet using adversarial u-net with different normalization.” In 2019 International Conference on Medical Imaging Physics and Engineering (ICMIPE) (2019).Google ScholarCross Ref
- J. C. Morris., “The clinical dementia rating (cdr): Current version and scoring rules.” Neurology, 43(11), 2412–2414 (1993).Google ScholarCross Ref
- C. R. Jack, M. A. Bernstein, N. C. Fox, P. Thompson, and M. W. Weiner., “The alzheimer's disease neuroimaging initiative (adni): Mri methods,” Journal of Magnetic Resonance Imaging, 27(4),685–691, (2010).Google ScholarCross Ref
- A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. Devito, et. al, “Automatic differentiation in pytorch,” (2017).Google Scholar
Index Terms
- Synthesizing PET Scans Using Attention GAN Based on MRI Scans and Alzheimer's Disease Information
Recommendations
Generation of Amyloid PET Images via Conditional Adversarial Training for Predicting Progression to Alzheimer’s Disease
PRedictive Intelligence in MEdicineAbstractNew positron emission tomography (PET) tracers could have a substantial impact on early diagnosis of Alzheimer’s disease (AD) and mild cognitive impairment (MCI) progression, particularly if they are accompanied by optimised deep learning methods. ...
Deformation field analysis of magnetic resonance images in Alzheimer's disease
AIKED'09: Proceedings of the 8th WSEAS international conference on Artificial intelligence, knowledge engineering and data basesMedical imaging has gone through a revolution since the advent of X-ray computed tomography imaging in early seventies and the introduction of other imaging modalities such as magnetic resonance imaging, positron emission tomography, etc in the later ...
A novel skull stripping technique for T1-weighted MRI human head scans
ICVGIP '12: Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image ProcessingAutomatic segmentation of brain tissues from magnetic resonance images still remains as a challenge due to variations in shape and size, use of different pulse sequences, overlapping signal intensities and imaging artifacts. In this paper, an automatic ...
Comments