Skip to main content
SpringerLink
Log in

Multiple Sclerosis Lesion Segmentation Using Longitudinal Normalization and Convolutional Recurrent Neural Networks

  • Conference paper
  • First Online:
Machine Learning in Clinical Neuroimaging and Radiogenomics in Neuro-oncology (MLCN 2020, RNO-AI 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12449))

Abstract

Magnetic resonance imaging (MRI) is the primary clinical tool to examine inflammatory brain lesions in Multiple Sclerosis (MS). Disease progression and inflammatory activities are examined by longitudinal image analysis to support diagnosis and treatment decision. Automated lesion segmentation methods based on deep convolutional neural networks (CNN) have been proposed, but are not yet applied in the clinical setting. Typical CNNs working on cross-sectional single time-point data have several limitations: changes to the image characteristics between single examinations due to scanner and protocol variations have an impact on the segmentation output, while at the same time the additional temporal correlation using pre-examinations is disregarded.

In this work, we investigate approaches to overcome these limitations. Within a CNN architectural design, we propose convolutional Long Short-Term Memory (C-LSTM) networks to incorporate the temporal dimension. To reduce scanner- and protocol dependent variations between single MRI exams, we propose a histogram normalization technique as pre-processing step. The ISBI 2015 challenge data was used for network training and cross-validation.

We demonstrate that the combination of the longitudinal normalization and CNN architecture increases the performance and the inter-time-point stability of the lesion segmentation. In the combined solution, the dice coefficient was increased and made more consistent for each subject. The proposed methods can therefore be used to increase the performance and stability of fully automated lesion segmentation applications in the clinical routine or in clinical trials.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arnon, R., Miller, A.: Translational NeuroImmunology in Multiple Sclerosis, 1st edn. Academic Press Inc., London (2016)

    Google Scholar 

  2. Aslani, S., Dayan, M., Murino, V., Sona, D.: Deep 2D encoder-decoder convolutional neural network for multiple sclerosis lesion segmentation in brain MRI. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 132–141. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_13

    Chapter  Google Scholar 

  3. Aslani, S., et al.: Multi-branch convolutional neural network for multiple sclerosis lesion segmentation. NeuroImage 196, 1–15 (2019). https://doi.org/10.1016/j.neuroimage.2019.03.068

    Article  Google Scholar 

  4. Balakrishnan, G., Zhao, A., Sabuncu, M.R., Guttag, J., Dalca, A.V.: VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans. Med. Imaging 38(8), 1788–1800 (2019)

    Article  Google Scholar 

  5. Baur, C., Wiestler, B., Albarqouni, S., Navab, N.: Deep autoencoding models for unsupervised anomaly segmentation in brain MR images. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11383, pp. 161–169. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11723-8_16

    Chapter  Google Scholar 

  6. Birenbaum, A., Greenspan, H.: Longitudinal multiple sclerosis lesion segmentation using multi-view convolutional neural networks 10008, 58–67 (2016). https://doi.org/10.1007/978-3-319-46976-8_7

  7. Brosch, T., Tang, L.Y., Yoo, Y., Li, D.K., Traboulsee, A., Tam, R.: Deep 3D convolutional encoder networks with shortcuts for multiscale feature integration applied to multiple sclerosis lesion segmentation. IEEE Trans. Med. Imaging 35(5), 1229–1239 (2016). https://doi.org/10.1109/TMI.2016.2528821

    Article  Google Scholar 

  8. Carass, A., et al.: Longitudinal multiple sclerosis lesion segmentation: resource and challenge. NeuroImage 148, 77–102 (2017). https://doi.org/10.1016/j.neuroimage.2016.12.064

    Article  Google Scholar 

  9. Cohen, J.A., Rae-Grant, A.: Handbook of Multiple Sclerosis, 1st edn. Springer Healthcare, London (2012). https://doi.org/10.1007/978-1-907673-50-4

    Book  Google Scholar 

  10. Compston, A., et al.: McAlpine’s Multiple Sclerosis, 4th edn. Churchill Livingstone. Elsevier Inc. (2005)

    Google Scholar 

  11. Dennis Jr., J.E., Woods, D.J.: Optimization on microcomputers. The Nelder-Mead simplex algorithm. Technical report (1985)

    Google Scholar 

  12. Fartaria, M.J., et al.: Partial volume-aware assessment of multiple sclerosis lesions. NeuroImage Clin. 18, 245–253 (2018)

    Article  Google Scholar 

  13. Gaser, C., Dahnke, R.: Cat-a computational anatomy toolbox for the analysis of structural MRI data. HBM 2016, 336–348 (2016)

    Google Scholar 

  14. Ghafoorian, M., Bram, P.: Convolutional neural networks for MS lesion segmentation, method description of DIAG team. In: Proceedings of the 2015 Longitudinal Multiple Sclerosis Lesion Segmentation Challenge, pp. 1–2 (2015)

    Google Scholar 

  15. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  16. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017 2017, vol. 2017 pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR.2017.243

  17. IACL: The 2015 longitudinal MS lesion segmentation challenge (2018). http://iacl.ece.jhu.edu/MSChallenge. Accessed 12 May 2020

  18. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  19. McKinley, R., et al.: Automatic detection of lesion load change in Multiple Sclerosis using convolutional neural networks with segmentation confidence. NeuroImage Clin. 25, 102104 (2020). https://doi.org/10.1016/j.nicl.2019.102104

    Article  Google Scholar 

  20. Novikov, A.A., Major, D., Wimmer, M., Lenis, D., Buehler, K.: Deep sequential segmentation of organs in volumetric medical scans. IEEE Trans. Med. Imaging 38(5), 1207–1215 (2018)

    Article  Google Scholar 

  21. Nyul, L.G., Udupa, J.K., Zhang, X.: New variants of a method of MRI scale standardization. IEEE Trans. Med. Imaging 19(2), 143–150 (2000)

    Article  Google Scholar 

  22. Penny, W.D., Friston, K.J., Ashburner, J.T., Kiebel, S.J., Nichols, T.E.: Statistical Parametric Mapping: the Analysis of Functional Brain Images. Elsevier (2011)

    Google Scholar 

  23. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer Assisted Intervention, vol. 9351, pp. 234–241 (2015). https://doi.org/10.1007/978-3-319-24574-4_28

  24. Roy, S., Butman, J.A., Reich, D.S., Calabresi, P.A., Pham, D.L.: Multiple sclerosis lesion segmentation from brain MRI via fully convolutional neural networks (2013) (2018). http://arxiv.org/abs/1803.09172

  25. Roy, S., et al.: Longitudinal intensity normalization in the presence of multiple sclerosis lesions. In: 2013 IEEE 10th International Symposium on Biomedical Imaging, pp. 1384–1387. IEEE (2013)

    Google Scholar 

  26. Salem, M., et al.: Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-NET. IEEE Access 7, 25171–25184 (2019). https://doi.org/10.1109/ACCESS.2019.2900198

    Article  Google Scholar 

  27. Salem, M., et al.: A fully convolutional neural network for new T2-w lesion detection in multiple sclerosis. NeuroImage Clin. 25, 102149 (2020). https://doi.org/10.1016/j.nicl.2019.102149

    Article  Google Scholar 

  28. Shinohara, R.T., et al.: Statistical normalization techniques for magnetic resonance imaging. NeuroImage Clin. 6, 9–19 (2014)

    Article  Google Scholar 

  29. Valverde, S., et al.: Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach. NeuroImage 155, 159–168 (2017). https://doi.org/10.1016/j.neuroimage.2017.04.034

    Article  Google Scholar 

  30. Weaver, K.F., Morales, V.C., Dunn, S.L., Godde, K., Weaver, P.F.: An Introduction to Statistical Analysis in Research: with Applications in the Biological and Life Sciences. Wiley, Hoboken (2017)

    Book  Google Scholar 

Download references

Acknowledgment

Sergio Tascon-Morales was supported by the Education, Audiovisual and Culture Executive Agency (EACEA) as part of the Erasmus Mundus Joint Master degree in Medical Imaging and Applications (MAIA).

Author information

Authors and Affiliations

  1. mediri GmbH, Heidelberg, Germany

    Sergio Tascon-Morales, Stefan Hoffmann, Martin Treiber, Daniel Mensing, Matthias Guenther & Johannes Gregori

  2. University of Girona, Girona, Spain

    Sergio Tascon-Morales & Arnau Oliver

  3. University of Burgundy, Le Creusot, France

    Sergio Tascon-Morales

  4. University of Cassino and Southern Lazio, Cassino, Italy

    Sergio Tascon-Morales

  5. Fraunhofer MEVIS, Bremen, Germany

    Matthias Guenther

  6. University of Bremen, Bremen, Germany

    Matthias Guenther

Authors
  1. Sergio Tascon-Morales
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stefan Hoffmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Treiber
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Mensing
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Arnau Oliver
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthias Guenther
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Johannes Gregori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Sergio Tascon-Morales , Stefan Hoffmann , Martin Treiber , Daniel Mensing , Arnau Oliver , Matthias Guenther or Johannes Gregori .

Editor information

Editors and Affiliations

  1. Donders Institute, Nijmegen, The Netherlands

    Seyed Mostafa Kia

  2. Lahore University of Management Sciences, Lahore, Pakistan

    Hassan Mohy-ud-Din

  3. University of Pennsylvania, Philadelphia, PA, USA

    Ahmed Abdulkadir

  4. King’s College London, London, UK

    Cher Bass

  5. The University of Texas Health Science Center at San Antonio, San Antonio, TX, USA

    Mohamad Habes

  6. University College London, London, UK

    Jane Maryam Rondina

  7. CUBRIC, Cardiff, UK

    Chantal Tax

  8. IBM Almaden Research Center, San Jose, CA, USA

    Hongzhi Wang

  9. University of Oslo, Oslo, Norway

    Thomas Wolfers

  10. Eli Lilly Pharmaceutical Company, Philadelphia, PA, USA

    Saima Rathore

  11. Symbiosis Institute of Technology, Pune, India

    Madhura Ingalhalikar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tascon-Morales, S. et al. (2020). Multiple Sclerosis Lesion Segmentation Using Longitudinal Normalization and Convolutional Recurrent Neural Networks. In: Kia, S.M., et al. Machine Learning in Clinical Neuroimaging and Radiogenomics in Neuro-oncology. MLCN RNO-AI 2020 2020. Lecture Notes in Computer Science(), vol 12449. Springer, Cham. https://doi.org/10.1007/978-3-030-66843-3_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-66843-3_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-66842-6

  • Online ISBN: 978-3-030-66843-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation