Skip to main content

Temporally Adjustable Longitudinal Fluid-Attenuated Inversion Recovery MRI Estimation / Synthesis for Multiple Sclerosis

  • Conference paper
  • First Online:
Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (BrainLes 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13769))

Included in the following conference series:

  • 522 Accesses

Abstract

Multiple Sclerosis (MS) is a chronic progressive neurological disease characterized by the development of lesions in the white matter of the brain. T2-fluid-attenuated inversion recovery (FLAIR) brain magnetic resonance imaging (MRI) provides superior visualization and characterization of MS lesions, relative to other MRI modalities. Longitudinal brain FLAIR MRI in MS, involving repetitively imaging a patient over time, provides helpful information for clinicians towards monitoring disease progression. Predicting future whole brain MRI examinations with variable time lag has only been attempted in limited applications, such as healthy aging and structural degeneration in Alzheimer’s Disease. In this article, we present novel modifications to deep learning architectures for MS FLAIR image synthesis / estimation, in order to support prediction of longitudinal images in a flexible continuous way. This is achieved with learned transposed convolutions, which support modelling time as a spatially distributed array with variable temporal properties at different spatial locations. Thus, this approach can theoretically model spatially-specific time-dependent brain development, supporting the modelling of more rapid growth at appropriate physical locations, such as the site of an MS brain lesion. This approach also supports the clinician user to define how far into the future a predicted examination should target. Accurate prediction of future rounds of imaging can inform clinicians of potentially poor patient outcomes, which may be able to contribute to earlier treatment and better prognoses. Four distinct deep learning architectures have been developed. The ISBI2015 longitudinal MS dataset was used to validate and compare our proposed approaches. Results demonstrate that a modified ACGAN achieves the best performance and reduces variability in model accuracy. Public domain code is made available at https://github.com/stfxecutables/Temporally-Adjustable-Longitudinal-MRI-Synthesis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We cannot report the metrics only based on the lesion area, since no lesion labels were provided to 14 participants in this ISBI2015 dataset.

References

  1. McGinley, M.P., Goldschmidt, C.H., Rae-Grant, A.D.: Diagnosis and treatment of multiple sclerosis: a review. JAMA 325, 765–779 (2021)

    Article  Google Scholar 

  2. Wei, W., et al.: Fluid-attenuated inversion recovery MRI synthesis from multisequence MRI using three-dimensional fully convolutional networks for multiple sclerosis. J Med Imaging (Bellingham). 6, 14005 (2019). https://doi.org/10.1117/1.JMI.6.1.014005

  3. Salem, M., et al.: Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-NET. IEEE Access. 7, 25171–25184 (2019). https://doi.org/10.1109/ACCESS.2019.2900198

    Article  Google Scholar 

  4. Wegmayr, V., Hörold, M., Buhmann, J.M.: Generative aging of brain MRI for early prediction of MCI-AD conversion. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1042–1046 (2019). https://doi.org/10.1109/ISBI.2019.8759394

  5. Ravi, D., Blumberg, S.B., Ingala, S., Barkhof, F., Alexander, D.C., Oxtoby, N.P.: Degenerative adversarial neuroimage nets for brain scan simulations: application in ageing and dementia. Med. Image Anal. 75, 102257 (2022). https://doi.org/10.1016/j.media.2021.102257

  6. Xia, T., Chartsias, A., Wang, C., Tsaftaris, S.A.: Learning to synthesise the ageing brain without longitudinal data. Med. Image Anal. 73, 102169 (2021). https://doi.org/10.1016/J.MEDIA.2021.102169

    Article  Google Scholar 

  7. Wang, J., Berger, D., Mattie, D., Levman, J.: Multichannel input pixelwise regression 3D U-Nets for medical image estimation with 3 applications in brain MRI. In: International Conference on Medical Imaging with Deep Learning (2021)

    Google Scholar 

  8. Doyle, A., Precup, D., Arnold, D.L., Arbel, T.: Predicting future disease activity and treatment responders for multiple sclerosis patients using a bag-of-lesions brain representation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 186–194. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_22

    Chapter  Google Scholar 

  9. Tousignant, A., et al. (eds.): Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning, pp. 483–492. PMLR (2019)

    Google Scholar 

  10. Sepahvand, N.M., Hassner, T., Arnold, D.L., Arbel, T.: CNN prediction of future disease activity for multiple sclerosis patients from baseline MRI and lesion labels. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 57–69. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_6

    Chapter  Google Scholar 

  11. Durso-Finley, J., Falet, J.-P.R., Nichyporuk, B., Arnold, D.L., Arbel, T.: Personalized prediction of future lesion activity and treatment effect in multiple sclerosis from baseline MRI. In: International Conference on Medical Imaging with Deep Learning, pp. 1–20 (2022)

    Google Scholar 

  12. Zhang, Z., Song, Y., Qi, H.: Age progression/regression by conditional adversarial autoencoder. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5810–5818 (2017)

    Google Scholar 

  13. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967–5976 (2017)

    Google Scholar 

  14. Goodfellow, I.J.: NIPS 2016 Tutorial: Generative Adversarial Networks (2016). arXiv preprint arXiv:1701.00160

  15. Odena, A., Olah, C., Shlens, J.: Conditional image synthesis with auxiliary classifier GANs. In: ICML’17 Proceedings of the 34th International Conference on Machine Learning – 70, pp. 2642–2651 (2017)

    Google Scholar 

  16. Carass, A., et al.: Longitudinal Multiple Sclerosis Lesion Segmentation: Resource and Challenge. Neuroimage. 148, 77–102 (2017)

    Google Scholar 

  17. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004). https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  18. Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: ICLR 2015: International Conference on Learning Representations 2015 (2015)

    Google Scholar 

  19. Yu, B., Zhou, L., Wang, L., Shi, Y., Fripp, J., Bourgeat, P.: Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Trans. Med. Imaging 38, 1750–1762 (2019). https://doi.org/10.1109/TMI.2019.2895894

    Article  Google Scholar 

  20. Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training GANs. In: NIPS’16 Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 2234–2242 (2016)

    Google Scholar 

  21. Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J.: Stargan: Unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)

    Google Scholar 

  22. Fu, J., Tzortzakakis, A., Barroso, J., Westman, E., Ferreira, D., Moreno, R.: Generative Aging of Brain Images with Diffeomorphic Registration (2022). arXiv preprint arXiv:2205.15607

  23. Bowles, C., Gunn, R., Hammers, A., Rueckert, D.: Modelling the progression of Alzheimer’s disease in MRI using generative adversarial networks. SPIE (2018)

    Google Scholar 

  24. Kim, S.T., Küçükaslan, U., Navab, N.: Longitudinal brain MR image modeling using personalized memory for alzheimer’s disease. IEEE Access 9, 143212–143221 (2021)

    Article  Google Scholar 

  25. Kumar, A., et al.: Counterfactual image synthesis for discovery of personalized predictive image markers. In: Medical Image Assisted Biomarkers’ Discovery (2022)

    Google Scholar 

Download references

Acknowledgements

This work was supported by an NSERC Discovery Grant to JL. Funding was also provided by a Nova Scotia Graduate Scholarship and a StFX Graduate Scholarship to JW. Computational resources were provided by Compute Canada.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jacob Levman .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J., Berger, D., Mazerolle, E., Soufan, O., Levman, J. (2023). Temporally Adjustable Longitudinal Fluid-Attenuated Inversion Recovery MRI Estimation / Synthesis for Multiple Sclerosis. In: Bakas, S., et al. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2022. Lecture Notes in Computer Science, vol 13769. Springer, Cham. https://doi.org/10.1007/978-3-031-33842-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33842-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33841-0

  • Online ISBN: 978-3-031-33842-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics