Abstract
Multiple Sclerosis (MS) is a chronic progressive neurological disease characterized by the development of lesions in the white matter of the brain. T2-fluid-attenuated inversion recovery (FLAIR) brain magnetic resonance imaging (MRI) provides superior visualization and characterization of MS lesions, relative to other MRI modalities. Longitudinal brain FLAIR MRI in MS, involving repetitively imaging a patient over time, provides helpful information for clinicians towards monitoring disease progression. Predicting future whole brain MRI examinations with variable time lag has only been attempted in limited applications, such as healthy aging and structural degeneration in Alzheimer’s Disease. In this article, we present novel modifications to deep learning architectures for MS FLAIR image synthesis / estimation, in order to support prediction of longitudinal images in a flexible continuous way. This is achieved with learned transposed convolutions, which support modelling time as a spatially distributed array with variable temporal properties at different spatial locations. Thus, this approach can theoretically model spatially-specific time-dependent brain development, supporting the modelling of more rapid growth at appropriate physical locations, such as the site of an MS brain lesion. This approach also supports the clinician user to define how far into the future a predicted examination should target. Accurate prediction of future rounds of imaging can inform clinicians of potentially poor patient outcomes, which may be able to contribute to earlier treatment and better prognoses. Four distinct deep learning architectures have been developed. The ISBI2015 longitudinal MS dataset was used to validate and compare our proposed approaches. Results demonstrate that a modified ACGAN achieves the best performance and reduces variability in model accuracy. Public domain code is made available at https://github.com/stfxecutables/Temporally-Adjustable-Longitudinal-MRI-Synthesis.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We cannot report the metrics only based on the lesion area, since no lesion labels were provided to 14 participants in this ISBI2015 dataset.
References
McGinley, M.P., Goldschmidt, C.H., Rae-Grant, A.D.: Diagnosis and treatment of multiple sclerosis: a review. JAMA 325, 765–779 (2021)
Wei, W., et al.: Fluid-attenuated inversion recovery MRI synthesis from multisequence MRI using three-dimensional fully convolutional networks for multiple sclerosis. J Med Imaging (Bellingham). 6, 14005 (2019). https://doi.org/10.1117/1.JMI.6.1.014005
Salem, M., et al.: Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-NET. IEEE Access. 7, 25171–25184 (2019). https://doi.org/10.1109/ACCESS.2019.2900198
Wegmayr, V., Hörold, M., Buhmann, J.M.: Generative aging of brain MRI for early prediction of MCI-AD conversion. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1042–1046 (2019). https://doi.org/10.1109/ISBI.2019.8759394
Ravi, D., Blumberg, S.B., Ingala, S., Barkhof, F., Alexander, D.C., Oxtoby, N.P.: Degenerative adversarial neuroimage nets for brain scan simulations: application in ageing and dementia. Med. Image Anal. 75, 102257 (2022). https://doi.org/10.1016/j.media.2021.102257
Xia, T., Chartsias, A., Wang, C., Tsaftaris, S.A.: Learning to synthesise the ageing brain without longitudinal data. Med. Image Anal. 73, 102169 (2021). https://doi.org/10.1016/J.MEDIA.2021.102169
Wang, J., Berger, D., Mattie, D., Levman, J.: Multichannel input pixelwise regression 3D U-Nets for medical image estimation with 3 applications in brain MRI. In: International Conference on Medical Imaging with Deep Learning (2021)
Doyle, A., Precup, D., Arnold, D.L., Arbel, T.: Predicting future disease activity and treatment responders for multiple sclerosis patients using a bag-of-lesions brain representation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 186–194. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_22
Tousignant, A., et al. (eds.): Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning, pp. 483–492. PMLR (2019)
Sepahvand, N.M., Hassner, T., Arnold, D.L., Arbel, T.: CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 57–69. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_6
Durso-Finley, J., Falet, J.-P.R., Nichyporuk, B., Arnold, D.L., Arbel, T.: Personalized prediction of future lesion activity and treatment effect in multiple sclerosis from baseline MRI. In: International Conference on Medical Imaging with Deep Learning, pp. 1–20 (2022)
Zhang, Z., Song, Y., Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5810–5818 (2017)
Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976 (2017)
Goodfellow, I.J.: NIPS 2016 Tutorial: Generative Adversarial Networks (2016). arXiv preprint arXiv:1701.00160
Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: ICML’17 Proceedings of the 34th International Conference on Machine Learning – 70, pp. 2642–2651 (2017)
Carass, A., et al.: Longitudinal Multiple Sclerosis Lesion Segmentation: Resource and Challenge. Neuroimage. 148, 77–102 (2017)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861
Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: ICLR 2015: International Conference on Learning Representations 2015 (2015)
Yu, B., Zhou, L., Wang, L., Shi, Y., Fripp, J., Bourgeat, P.: Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Trans. Med. Imaging 38, 1750–1762 (2019). https://doi.org/10.1109/TMI.2019.2895894
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: NIPS’16 Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 2234–2242 (2016)
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)
Fu, J., Tzortzakakis, A., Barroso, J., Westman, E., Ferreira, D., Moreno, R.: Generative Aging of Brain Images with Diffeomorphic Registration (2022). arXiv preprint arXiv:2205.15607
Bowles, C., Gunn, R., Hammers, A., Rueckert, D.: Modelling the progression of Alzheimer’s disease in MRI using generative adversarial networks. SPIE (2018)
Kim, S.T., Küçükaslan, U., Navab, N.: Longitudinal brain MR image modeling using personalized memory for alzheimer’s disease. IEEE Access 9, 143212–143221 (2021)
Kumar, A., et al.: Counterfactual image synthesis for discovery of personalized predictive image markers. In: Medical Image Assisted Biomarkers’ Discovery (2022)
Acknowledgements
This work was supported by an NSERC Discovery Grant to JL. Funding was also provided by a Nova Scotia Graduate Scholarship and a StFX Graduate Scholarship to JW. Computational resources were provided by Compute Canada.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, J., Berger, D., Mazerolle, E., Soufan, O., Levman, J. (2023). Temporally Adjustable Longitudinal Fluid-Attenuated Inversion Recovery MRI Estimation / Synthesis for Multiple Sclerosis. In: Bakas, S., et al. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2022. Lecture Notes in Computer Science, vol 13769. Springer, Cham. https://doi.org/10.1007/978-3-031-33842-7_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-33842-7_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33841-0
Online ISBN: 978-3-031-33842-7
eBook Packages: Computer ScienceComputer Science (R0)