Abstract
Improved imaging protocols and rapid scanning are making whole-body imaging increasingly popular in medical applications, where accurate object segmentation often plays an important role. Nowadays, convolutional neural networks (CNN) are the go-to method for automated image segmentation. Unfortunately, with improved image resolution, whole-body scans can grow large in size, preventing CNN training on entire images due to GPU memory limitations. An alternative to full-image training is patch-based training, which however suffers from context loss. We propose a way of reintroducing spatial context with marginal increase in memory requirements by employing landmark-normalized distance maps that carry relative position information for the patches. As a proof-of-concept we evaluate its effect with multiple networks with and without built-in context integration, using a multi-class whole-body MRI dataset. The results show that our proposed approach yields significant improvement over the baseline networks, even when used out-of-the box. We provide a discussion on its regularization properties and usability cases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=YicbFdNTTy
Estrada, S., et al.: FatSegNet: a fully automated deep learning pipeline for adipose tissue segmentation on abdominal dixon MRI. Magn. Reson. Med. 83(4), 1471–1483 (2020). https://doi.org/10.1002/mrm.28022
Eustace, S.J., Nelson, E.: Whole body magnetic resonance imaging. BMJ 328(7453), 1387 (2004). https://doi.org/10.1136/bmj.328.7453.1387
Ghafoorian, M., Karssemeijer, N., Heskes, T., et al.: Location sensitive deep convolutional neural networks for segmentation of white matter hyperintensities. Sci. Rep. 7, 5110 (2017). https://doi.org/10.1038/s41598-017-05300-5
Ghafoorian, M., et al.: Deep multi-scale location-aware 3d convolutional neural networks for automated detection of Lacunes of presumed vascular origin. NeuroImage: Clin. 14, 391–399 (2017). https://doi.org/10.1016/j.nicl.2017.01.033
Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Patch-based convolutional neural network for whole slide tissue image classification. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2424–2433 (2016). https://doi.org/10.1109/CVPR.2016.266
Kamnitsas, K., et al.: Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 36, 61–78 (2017). https://doi.org/10.1016/j.media.2016.10.004
Kao, P.Y., et al.: Improving patch-based convolutional neural networks for MRI brain tumor segmentation by leveraging location information. Front. Neurosci. 13, 01449 (2020). https://doi.org/10.3389/fnins.2019.01449
Lavdas, I., et al.: Fully automatic, multiorgan segmentation in normal whole body magnetic resonance imaging (mri), using classification forests (cfs), convolutional neural networks (cnns), and a multi-atlas (ma) approach. Med. Phys. 44(10), 5210–5220 (2017). https://doi.org/10.1002/mp.12492
Lind, L.: Relationships between three different tests to evaluate endothelium-dependent vasodilation and cardiovascular risk in a middle-aged sample. J. Hypertens. 31, 1570–1574 (2013). https://doi.org/10.1097/HJH.0b013e3283619d50
Liu, W., Rabinovich, A., Berg, A.C.: ParseNet: looking wider to see better (2015). https://doi.org/10.48550/arxiv.1506.04579
Luo, W., Li, Y., Urtasun, R., Zemel, R.: Understanding the effective receptive field in deep convolutional neural networks. In: Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 29. Curran Associates, Inc. (2016)
Pezzano, G., Ribas Ripoll, V., Radeva, P.: CoLe-CNN: context-learning convolutional neural network with adaptive loss function for lung nodule segmentation. Comput. Methods Programs Biomed. 198, 105792 (2021). https://doi.org/10.1016/j.cmpb.2020.105792
Rachmadi, M.F., del C. Valdés-Hernández, M., Agan, M.L.F., Di Perri, C., Komura, T.: Segmentation of white matter hyperintensities using convolutional neural networks with global spatial information in routine clinical brain MRI with none or mild vascular pathology. Computer. Med. Imag. Graph. 66, 28–43 (2018). https://doi.org/10.1016/j.compmedimag.2018.02.002
Rickmann, A.M., Senapati, J., Kovalenko, O., Peters, A., Bamberg, F., Wachinger, C.: AbdomenNet: deep neural network for abdominal organ segmentation in epidemiologic imaging studies. BMC Med. Imag. 22, 168 (2022). https://doi.org/10.1186/s12880-022-00893-4
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Strand, R., Malmberg, F., Johansson, L., Lind, L., Sundbom, M., Ahlström, H., Kullberg, J.: A concept for holistic whole body MRI data analysis, Imiomics. PLOS ONE 12(2), 1–17 (2017). https://doi.org/10.1371/journal.pone.0169966
Summers, P., et al.: Whole-body magnetic resonance imaging: technique, guidelines and key applications. Ecancermedicalscience 15, 1164 (2021). https://doi.org/10.3332/ecancer.2021.1164
Valindria, V.V., et al.: Small organ segmentation in whole-body MRI using a two-stage FCN and weighting schemes. In: Shi, Y., Suk, H.-I., Liu, M. (eds.) MLMI 2018. LNCS, vol. 11046, pp. 346–354. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00919-9_40
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6230–6239 (2017). https://doi.org/10.1109/CVPR.2017.660
Acknowledgments
E.B. is partially funded by the Centre for Interdiciplinary Mathematics (CIM), Uppsala University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Breznik, E., Kullberg, J., Ahlström, H., Strand, R. (2023). Introducing Spatial Context in Patch-Based Deep Learning for Semantic Segmentation in Whole Body MRI. In: Gade, R., Felsberg, M., Kämäräinen, JK. (eds) Image Analysis. SCIA 2023. Lecture Notes in Computer Science, vol 13886. Springer, Cham. https://doi.org/10.1007/978-3-031-31438-4_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-31438-4_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-31437-7
Online ISBN: 978-3-031-31438-4
eBook Packages: Computer ScienceComputer Science (R0)
