Abstract
Multiple sclerosis (MS) is an immune-mediated neurodegenerative disease that results in progressive damage to the brain and spinal cord. Volumetric analysis of the brain tissues with Magnetic Resonance Imaging (MRI) is essential to monitor the progression of the disease. However, the presence of focal brain pathology leads to tissue misclassifications, and has been traditionally addressed by “inpainting” MS lesions with voxel intensities sampled from surrounding normal-appearing white matter. Based on the characteristics of brain MRIs and MS lesions, we propose a Lesion Gate Network (LG-Net) for MS lesion inpainting with a learnable dynamic gate mask integrated with the convolution blocks to dynamically select the features for a lesion area defined by a noisy lesion mask. We also introduce a lesion gate consistency loss to support the training of the gated lesion convolution by minimizing the differences between the features selected from the brain with and without lesions. We evaluated the proposed model on both public and in-house data and our method demonstrated a faster and superior performance than the state-of-the-art inpainting techniques developed for MS lesion and general image inpainting tasks.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ashburner, J., Friston, K.J.: Unified segmentation. Neuroimage 26(3), 839–851 (2005)
Barnes, C., Goldman, D.B., Shechtman, E., Finkelstein, A.: The PatchMatch randomized matching algorithm for image manipulation. Commun. ACM 54(11), 103–110 (2011)
Battaglini, M., Jenkinson, M., Stefano, N.D.: Evaluating and reducing the impact of white matter lesions on brain volume measurements. Hum. Brain Mapp. 33(9), 2062–2071 (2012)
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Chau, W., McIntosh, A.R.: The Talairach coordinate of a point in the MNI space: how to interpret it. Neuroimage 25(2), 408–416 (2005)
IXI dataset. https://brain-development.org/ixi-dataset/. Accessed 15 Sept 2021
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Liu, G., Reda, F.A., Shih, K.J., Wang, T.-C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11215, pp. 89–105. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01252-6_6
Liu, H., Jiang, B., Song, Y., Huang, W., Yang, C.: Rethinking image inpainting via a mutual encoder-decoder with feature equalizations. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12347, pp. 725–741. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58536-5_43
Ma, Y., et al.: Multiple sclerosis lesion analysis in brain magnetic resonance images: techniques and clinical applications. arXiv preprint arXiv:2104.10029 (2021)
Magon, S., et al.: White matter lesion filling improves the accuracy of cortical thickness measurements in multiple sclerosis patients: a longitudinal study. BMC Neurosci. 15, 106 (2014). https://doi.org/10.1186/1471-2202-15-106
Prados, F., et al.: A multi-time-point modality-agnostic patch-based method for lesion filling in multiple sclerosis. Neuroimage 139, 376–384 (2016)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sastre-Garriga, J., et al.: MAGNIMS consensus recommendations on the use of brain and spinal cord atrophy measures in clinical practice. Nat. Rev. Neurol. 16, 171–182 (2020)
De Stefano, N.: Clinical relevance of brain volume measures in multiple sclerosis. CNS Drugs 28(2), 147–156 (2014). https://doi.org/10.1007/s40263-014-0140-z
Tustison, N.J., et al.: N4ITK: improved N3 bias correction. IEEE Trans. Med. Imaging 29(6), 1310–1320 (2010)
Xiong, H., Wang, C., Barnett, M., Wang, C.: Multiple sclerosis lesion filling using a non-lesion attention based convolutional network. In: Yang, H., Pasupa, K., Leung, A.C.-S., Kwok, J.T., Chan, J.H., King, I. (eds.) ICONIP 2020. LNCS, vol. 12532, pp. 448–460. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63830-6_38
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. In: CVPR, pp. 5505–5514 (2018)
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Free-form image inpainting with gated convolution. In: ICCV, pp. 4471–4480 (2019)
Zhang, Y., Brady, J.M., Smith, S.: Hidden Markov random field model for segmentation of brain MR image. In: Medical Imaging 2000: Image Processing, vol. 3979, pp. 1126–1137 (2000)
Acknowledgments
The authors acknowledge the support of Australian Government Cooperative Research Centres Project grant (CRCPFIVE000141) and Research Training Program (RTP) Scholarship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Compliance with Ethical Standards The study was approved by the University of Sydney Human Research and Ethics Committee and all procedures adhered the tenets of the Declaration of Helsinki; we also appreciate the efforts devoted to collect and share the IXI brain dataset for open access [6].
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Tang, Z., Cabezas, M., Liu, D., Barnett, M., Cai, W., Wang, C. (2021). LG-Net: Lesion Gate Network for Multiple Sclerosis Lesion Inpainting. In: de Bruijne, M., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science(), vol 12907. Springer, Cham. https://doi.org/10.1007/978-3-030-87234-2_62
Download citation
DOI: https://doi.org/10.1007/978-3-030-87234-2_62
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87233-5
Online ISBN: 978-3-030-87234-2
eBook Packages: Computer ScienceComputer Science (R0)
