Abstract
Mammography is commonly used as an imaging technique in breast cancer screening but comes with the disadvantage of a high overdiagnosis rate and low sensitivity in dense tissue. dynamic contrast enhanced (DCE)-magnetic resonance imaging (MRI) features higher sensitivity but requires time consuming dynamic imaging and injection of contrast media, limiting the capability of the technique as a widespread screening method. In this work, we extend the masked autoencoder (MAE) approach to perform anomaly detection on volumetric, multispectral MRI. This new model, coined masked autoencoder for medical imaging (MAEMI), is trained on two non-contrast enhanced breast MRI sequences, aiming at lesion detection without the need for intravenous injection of contrast media and temporal image acquisition, paving the way for more widespread use of MRI in breast cancer diagnosis. During training, only non-cancerous images are presented to the model, with the purpose of localizing anomalous tumor regions during test time. We use a public dataset for model development. Performance of the architecture is evaluated in reference to subtraction images created from DCE-MRI. Code has been made publicly available: https://github.com/LangDaniel/MAEMI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ayatollahi, F., Shokouhi, S.B., Mann, R.M., Teuwen, J.: Automatic breast lesion detection in ultrafast DCE-MRI using deep learning. Med. Phys. 48(10), 5897–5907 (2021)
Baur, C., Denner, S., Wiestler, B., Navab, N., Albarqouni, S.: Autoencoders for unsupervised anomaly segmentation in brain MR images: a comparative study. Med. Image Anal. 69, 101952 (2021)
Bercea, C.I., Wiestler, B., Rueckert, D., Schnabel, J.A.: Generalizing unsupervised anomaly detection: towards unbiased pathology screening. In: Medical Imaging with Deep Learning (2023)
Bône, A., et al.: Contrast-enhanced brain MRI synthesis with deep learning: key input modalities and asymptotic performance. In: 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1159–1163. IEEE (2021)
Chow, J.K., Su, Z., Wu, J., Tan, P.S., Mao, X., Wang, Y.H.: Anomaly detection of defects on concrete structures with the convolutional autoencoder. Adv. Eng. Inf. 45, 101105 (2020)
Clark, K., et al.: The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Dig. Imaging 26, 1045–1057 (2013)
Crosby, D., et al.: Early detection of cancer. Science 375(6586), eaay9040 (2022)
Dibden, A., Offman, J., Duffy, S.W., Gabe, R.: Worldwide review and meta-analysis of cohort studies measuring the effect of mammography screening programmes on incidence-based breast cancer mortality. Cancers 12(4), 976 (2020)
Dosovitskiy, A., et al.: An image is worth 16\(\times \)16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Fraum, T.J., Ludwig, D.R., Bashir, M.R., Fowler, K.J.: Gadolinium-based contrast agents: a comprehensive risk assessment. J. Magn. Reson. Imaging 46(2), 338–353 (2017)
Han, L., et al.: Synthesis-based imaging-differentiation representation learning for multi-sequence 3D/4D MRI. arXiv preprint arXiv:2302.00517 (2023)
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.: Masked autoencoders are scalable vision learners. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16000–16009 (2022)
Herent, P., et al.: Detection and characterization of MRI breast lesions using deep learning. Diagn. Interv. Imaging 100(4), 219–225 (2019)
Kascenas, A., Pugeault, N., O’Neil, A.Q.: Denoising autoencoders for unsupervised anomaly detection in brain MRI. In: International Conference on Medical Imaging with Deep Learning, pp. 653–664. PMLR (2022)
Lei, S., et al.: Global patterns of breast cancer incidence and mortality: a population-based cancer registry data analysis from 2000 to 2020. Cancer Commun. 41(11), 1183–1194 (2021)
Leithner, D., Moy, L., Morris, E.A., Marino, M.A., Helbich, T.H., Pinker, K.: Abbreviated MRI of the breast: does it provide value? J. Magn. Reson. Imaging 49(7), e85–e100 (2019)
Løberg, M., Lousdal, M.L., Bretthauer, M., Kalager, M.: Benefits and harms of mammography screening. Breast Cancer Res. 17(1), 1–12 (2015)
Maicas, G., Carneiro, G., Bradley, A.P., Nascimento, J.C., Reid, I.: Deep reinforcement learning for active breast lesion detection from DCE-MRI. In: Descoteaux, M., et al. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 665–673. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66179-7_76
Müller-Franzes, G., et al.: Using machine learning to reduce the need for contrast agents in breast MRI through synthetic images. Radiology 307(3), e222211 (2023)
Prabhakar, C., Li, H.B., Yang, J., Shit, S., Wiestler, B., Menze, B.: ViT-AE++: improving vision transformer autoencoder for self-supervised medical image representations. arXiv preprint arXiv:2301.07382 (2023)
Saha, A., et al.: Dynamic contrast-enhanced magnetic resonance images of breast cancer patients with tumor locations. Cancer Imaging Arch. (2021)
Saha, A., et al.: A machine learning approach to radiogenomics of breast cancer: a study of 922 subjects and 529 DCE-MRI features. Brit. J. cancer 119(4), 508–516 (2018)
Sardanelli, F., et al.: Sensitivity of MRI versus mammography for detecting foci of multifocal, multicentric breast cancer in fatty and dense breasts using the whole-breast pathologic examination as a gold standard. Am. J. Roentgenol. (2012)
Schwartz, E., et al.: MAEDAY: MAE for few and zero shot AnomalY-Detection. arXiv preprint arXiv:2211.14307 (2022)
Tian, Y., et al.: Unsupervised anomaly detection in medical images with a memory-augmented multi-level cross-attentional masked autoencoder. arXiv preprint arXiv:2203.11725 (2022)
Turnbull, L.W.: Dynamic contrast-enhanced MRI in the diagnosis and management of breast cancer. NMR Biomed. Int. J. Dev. Dev. Appl. Magn. Reson. Vivo 22(1), 28–39 (2009)
Wanders, J.O., et al.: Volumetric breast density affects performance of digital screening mammography. Breast Cancer Res. Treat. 162, 95–103 (2017)
Xu, Z., et al.: Swin MAE: masked autoencoders for small datasets. arXiv preprint arXiv:2212.13805 (2022)
Zavrtanik, V., Kristan, M., Skočaj, D.: Reconstruction by inpainting for visual anomaly detection. Pattern Recogn. 112, 107706 (2021)
Acknowledgements
DML was in part financed by the Helmholtz Information and Data Science Academy (HIDA) under the “Israel Exchange Program".
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary Material
Supplementary Material
Single reconstruction examples. The left block shows axial slices of T1 non-fat saturated MRI-patches and the right block T1 fat saturated slices. The first column shows unaltered MRI-patches, the second column the masked model input and the third column the MRI-patches recovered by MAEMI. Examples represent a masking ratio of 90% (for the whole 3D patch) and a ViT-patch size of \(8\times 8\times 2\). During inference, multiple masks are generated to maximize likelihood of anomalies to be removed.
Reconstruction examples for different masking ratios and a fixed ViT-patch size of \(8\times 8\times 2\). Very high ratios (>90%) lead blurry images, while moderate ratios allow for reconstruction of detailed structures. On the one hand, reconstructions need to be detailed enough to reduce the amount of false positive findings. On the other, high masking ratios increase the likelihood of the anomaly to be removed, leading to a better true positive rate.
Subtraction images and anomaly maps were multiplied with segmentation masks to remove anomalies lying obviously outside of the breast tissue. This is mainly needed as contrast agent is also taken up in organs outside of the breast. The left column shows the raw subtraction/anomaly map, and the right column the raw maps multiplied with the segmentation mask of the image in the upper left corner. Performance metrics were only calculated for voxels lying inside the segmentation mask, limiting the influence of trivial predictions, i.e. voxels that represent air do not containing any anomalies.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lang, D.M., Schwartz, E., Bercea, C.I., Giryes, R., Schnabel, J.A. (2023). Multispectral 3D Masked Autoencoders for Anomaly Detection in Non-Contrast Enhanced Breast MRI. In: Ali, S., van der Sommen, F., van Eijnatten, M., Papież, B.W., Jin, Y., Kolenbrander, I. (eds) Cancer Prevention Through Early Detection. CaPTion 2023. Lecture Notes in Computer Science, vol 14295. Springer, Cham. https://doi.org/10.1007/978-3-031-45350-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-031-45350-2_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45349-6
Online ISBN: 978-3-031-45350-2
eBook Packages: Computer ScienceComputer Science (R0)
