Abstract
Medial temporal lobe atrophy (MTA) score is a key feature for Alzheimer’s disease (AD) diagnosis. Diagnosis of MTA from images acquired using magnetic resonance imaging (MRI) technology suffers from high inter- and intra-observer discrepancies. The recently-developed Vision Transformer (ViT) can be trained on MRI images to classify MTA scores, but is a “black-box” model whose internal working is unknown. Further, a fully-trained classifier is also susceptible to inconsistent predictions by nature of its labels used for training. Augmenting imaging data with tabular features could potentially rectify this issue, but ViTs are designed to process imaging data as its name suggests. This work aims to develop an accurate and explainable MTA classifier. We introduce a multi-modality training scheme to simultaneously handle tabular and image data. Our proposed method processes multi-modality data consisting of T1-weighted brain MRI and tabular data encompassing brain region volumes, cortical thickness, and radiomics features. Our method outperforms various baselines considered, and its attention map on input images and feature importance scores on tabular data explains its reasoning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arık, S.O., Pfister, T.: Tabnet: attentive interpretable tabular learning. In: AAAI, vol. 35, pp. 6679–6687 (2021)
Caliendo, M., Kopeinig, S.: Some practical guidance for the implementation of propensity score matching. J. Econ.Surv. 22(1), 31–72 (2008)
Chefer, H., Gur, S., Wolf, L.: Transformer interpretability beyond attention visualization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 782–791 (2021)
Cheng, D., Liu, M.: Cnns based multi-modality classification for ad diagnosis. In: 2017 10th International Congress on Image and Signal Processing, Biomedical Engineering and Informatics (CISP-BMEI), pp. 1–5. IEEE (2017)
Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Duara, R., et al.: Medial temporal lobe atrophy on mri scans and the diagnosis of Alzheimer disease. Neurology 71(24), 1986–1992 (2008)
Fischl, B.: Freesurfer. Neuroimage 62(2), 774–781 (2012)
Foret, P., Kleiner, A., Mobahi, H., Neyshabur, B.: Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412 (2020)
Lahat, D., Adali, T., Jutten, C.: Multimodal data fusion: an overview of methods, challenges, and prospects. Proc. IEEE 103(9), 1449–1477 (2015)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Mårtensson, G., et al.: Avra: automatic visual ratings of atrophy from mri images using recurrent convolutional neural networks. NeuroImage Clin. 23, 101872 (2019)
Martins, A., Astudillo, R.: From softmax to sparsemax: a sparse model of attention and multi-label classification. In: International Conference on Machine Learning, pp. 1614–1623. PMLR (2016)
Mobadersany, P., et al.: Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl. Acad. Sci. 115(13), E2970–E2979 (2018)
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R.: Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recogn. 65, 211–222 (2017)
Park, H.Y., Park, C.R., Suh, C.H., Shim, W.H., Kim, S.J.: Diagnostic performance of the medial temporal lobe atrophy scale in patients with Alzheimer’s disease: a systematic review and meta-analysis. Eur. Radiol. 31(12), 9060–9072 (2021)
Park, Y.W., et al.: Radiomics features of hippocampal regions in magnetic resonance imaging can differentiate medial temporal lobe epilepsy patients from healthy controls. Sci. Rep. 10(1), 1–8 (2020)
Pölsterl, S., Sarasua, I., Gutiérrez-Becker, B., Wachinger, C.: A wide and deep neural network for survival analysis from anatomical shape and tabular clinical data. In: Cellier, P., Driessens, K. (eds.) ECML PKDD 2019. CCIS, vol. 1167, pp. 453–464. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43823-4_37
Pölsterl, S., Wolf, T.N., Wachinger, C.: Combining 3D image and tabular data via the dynamic affine feature map transform. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 688–698. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_66
Scheltens, P., Launer, L.J., Barkhof, F., Weinstein, H.C., van Gool, W.A.: Visual assessment of medial temporal lobe atrophy on magnetic resonance imaging: interobserver reliability. J. Neurol. 242(9), 557–560 (1995)
Scheltens, P., et al.: Atrophy of medial temporal lobes on MRI in" probable" Alzheimer’s disease and normal ageing: diagnostic value and neuropsychological correlates. J. Neurol. Neurosurg. Psych. 55(10), 967–972 (1992)
Smilkov, D., Thorat, N., Kim, B., Viégas, F., Wattenberg, M.: Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017)
Spasov, S., et al.: A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer’s Disease. Neuroimage 189, 276–287 (2019)
Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., Jégou, H.: Training data-efficient image transformers & distillation through attention. In: International Conference on Machine Learning, pp. 10347–10357. PMLR (2021)
Van Griethuysen, J.J., et al.: Computational radiomics system to decode the radiographic phenotype. Can. Res. 77(21), e104–e107 (2017)
Wang, W., Tran, D., Feiszli, M.: What makes training multi-modal classification networks hard? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12695–12705 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lee, D. et al. (2022). Augmenting Magnetic Resonance Imaging with Tabular Features for Enhanced and Interpretable Medial Temporal Lobe Atrophy Prediction. In: Abdulkadir, A., et al. Machine Learning in Clinical Neuroimaging. MLCN 2022. Lecture Notes in Computer Science, vol 13596. Springer, Cham. https://doi.org/10.1007/978-3-031-17899-3_13
Download citation
DOI: https://doi.org/10.1007/978-3-031-17899-3_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17898-6
Online ISBN: 978-3-031-17899-3
eBook Packages: Computer ScienceComputer Science (R0)
