Abstract
Training medical image segmentation models usually requires a large amount of labeled data. By contrast, humans can quickly learn to accurately recognise anatomy of interest from medical (e.g. MRI and CT) images with some limited guidance. Such recognition ability can easily generalise to new images from different clinical centres. This rapid and generalisable learning ability is mostly due to the compositional structure of image patterns in the human brain, which is less incorporated in medical image segmentation. In this paper, we model the compositional components (i.e. patterns) of human anatomy as learnable von-Mises-Fisher (vMF) kernels, which are robust to images collected from different domains (e.g. clinical centres). The image features can be decomposed to (or composed by) the components with the composing operations, i.e. the vMF likelihoods. The vMF likelihoods tell how likely each anatomical part is at each position of the image. Hence, the segmentation mask can be predicted based on the vMF likelihoods. Moreover, with a reconstruction module, unlabeled data can also be used to learn the vMF kernels and likelihoods by recombining them to reconstruct the input image. Extensive experiments show that the proposed vMFNet achieves improved generalisation performance on two benchmarks, especially when annotations are limited. Code is publicly available at: https://github.com/vios-s/vMFNet.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
vMF kernels are similar to prototypes in [35]. However, prototypes are usually calculated as the mean of the feature vectors for each class using the ground-truth masks. vMF kernels are learnt as the cluster centres of the feature vectors.
References
Achille, A., Soatto, S.: Emergence of invariance and disentanglement in deep representations. JMLR 19(1), 1947–1980 (2018)
Arad Hudson, D., Zitnick, L.: Compositional transformers for scene generation. In: NeurIPS (2021)
Bernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang, X., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE TMI 37(11), 2514–2525 (2018)
Campello, V.M., et al.: Multi-centre, multi-vendor and multi-disease cardiac segmentation: the M &Ms challenge. IEEE TMI 40(12), 3543–3554 (2021)
Carlucci, F.M., D’Innocente, A., Bucci, S., Caputo, B., Tommasi, T.: Domain generalisation by solving jigsaw puzzles. In: CVPR, pp. 2229–2238 (2019)
Chartsias, A., Joyce, T., et al.: Disentangled representation learning in cardiac image analysis. Media 58, 101535 (2019)
Chen, C., Hammernik, K., Ouyang, C., Qin, C., Bai, W., Rueckert, D.: Cooperative training and latent space data augmentation for robust medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 149–159. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_14
Chen, C., Qin, C., Qiu, H., et al.: Deep learning for cardiac image segmentation: a review. Front. Cardiovasc. Med. 7(25), 1–33 (2020)
Dou, Q., Castro, D.C., Kamnitsas, K., Glocker, B.: Domain generalisation via model-agnostic learning of semantic features. In: NeurIPS (2019)
Dubuisson, M.P., Jain, A.K.: A modified hausdorff distance for object matching. In: ICPR, vol. 1, pp. 566–568. IEEE (1994)
Gu, R., Zhang, J., Huang, R., Lei, W., Wang, G., Zhang, S.: Domain composition and attention for unseen-domain generalizable medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 241–250. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_23
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
He, Y., Carass, A., Zuo, L., et al.: Autoencoder based self-supervised test-time adaptation for medical image analysis. Media 72, 102136 (2021)
Hu, M., et al.: Fully test-time adaptation for image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 251–260. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_24
Huang, J., Guan, D., Xiao, A., Lu, S.: FSDR: frequency space domain randomization for domain generalization. In: CVPR (2021)
Huynh, D., Elhamifar, E.: Compositional zero-shot learning via fine-grained dense feature composition. In: NeurIPS, vol. 33, pp. 19849–19860 (2020)
Isensee, F., Jaeger, P.F., et al.: nnUNet: a self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 18(2), 203–211 (2021)
Iwasawa, Y., Matsuo, Y.: Test-time classifier adjustment module for model-agnostic domain generalization. In: NeurIPS, vol. 34 (2021)
Karani, N., Erdil, E., Chaitanya, K., Konukoglu, E.: Test-time adaptable neural networks for robust medical image segmentation. Media 68, 101907 (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: ICLR (2015)
Kortylewski, A., He, J., Liu, Q., Yuille, A.L.: Compositional convolutional neural networks: a deep architecture with innate robustness to partial occlusion. In: CVPR, pp. 8940–8949 (2020)
Li, D., Yang, Y., Song, Y.Z., Hospedales, T.: Learning to generalise: meta-learning for domain generalisation. In: AAAI (2018)
Li, H., Wang, Y., Wan, R., Wang, S., et al.: Domain generalisation for medical imaging classification with linear-dependency regularization. In: NeurIPS (2020)
Li, L., Zimmer, V.A., Schnabel, J.A., Zhuang, X.: AtrialGeneral: domain generalization for left atrial segmentation of multi-center LGE MRIs. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12906, pp. 557–566. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87231-1_54
Liu, N., Li, S., Du, Y., Tenenbaum, J., Torralba, A.: Learning to compose visual relations. In: NeurIPS, vol. 34 (2021)
Liu, Q., Chen, C., Qin, J., Dou, Q., Heng, P.A.: FedDG: Federated domain generalization on medical image segmentation via episodic learning in continuous frequency space. In: CVPR, pp. 1013–1023 (2021)
Liu, Q., Dou, Q., Heng, P.-A.: Shape-aware meta-learning for generalizing prostate MRI segmentation to unseen domains. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12262, pp. 475–485. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59713-9_46
Liu, X., Thermos, S., Chartsias, A., O’Neil, A., Tsaftaris, S.A.: Disentangled representations for domain-generalised cardiac segmentation. In: STACOM Workshop (2020)
Liu, X., Thermos, S., O’Neil, A., Tsaftaris, S.A.: Semi-supervised meta-learning with disentanglement for domain-generalised medical image segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 307–317. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_29
Liu, X., Thermos, S., Valvano, G., Chartsias, A., O’Neil, A., Tsaftaris, S.A.: Measuring the biases and effectiveness of content-style disentanglement. In: BMVC (2021)
Milletari, F., Navab, N., Ahmadi, S.A.: VNet: fully convolutional neural networks for volumetric medical image segmentation. In: 3DV, pp. 565–571. IEEE (2016)
Paszke, A., Gross, S., Massa, F., Lerer, A., et. al: PyTorch: an imperative style, high-performance deep learning library. In: NeurIPS, pp. 8026–8037 (2019)
Prados, F., Ashburner, J., Blaiotta, C., Brosch, T., et al.: Spinal cord grey matter segmentation challenge. Neuroimage 152, 312–329 (2017)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: NeurIPS, pp. 4080–4090 (2017)
Tokmakov, P., Wang, Y.X., Hebert, M.: Learning compositional representations for few-shot recognition. In: CVPR, pp. 6372–6381 (2019)
Valvano, G., Leo, A., Tsaftaris, S.A.: Re-using adversarial mask discriminators for test-time training under distribution shifts. arXiv preprint arXiv:2108.11926 (2021)
Valvano, G., Leo, A., Tsaftaris, S.A.: Stop throwing away discriminators! Re-using adversaries for test-time training. In: Albarqouni, S., et al. (eds.) DART/FAIR -2021. LNCS, vol. 12968, pp. 68–78. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87722-4_7
Yao, H., Hu, X., Li, X.: Enhancing pseudo label quality for semi-superviseddomain-generalized medical image segmentation. arXiv preprint arXiv:2201.08657 (2022)
Yuan, X., Kortylewski, A., et al.: Robust instance segmentation through reasoning about multi-object occlusion. In: CVPR, pp. 11141–11150 (2021)
Zakazov, I., Shirokikh, B., Chernyavskiy, A., Belyaev, M.: Anatomy of domain shift impact on U-Net layers in MRI segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 211–220. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_20
Zhang, L., Wang, X., Yang, D., Sanford, T., et al.: Generalising deep learning for medical image segmentation to unseen domains via deep stacked transformation. IEEE TMI 39(7), 2531–2540 (2020)
Zhang, Y., Kortylewski, A., Liu, Q., et al.: A light-weight interpretable compositionalnetwork for nuclei detection and weakly-supervised segmentation. arXiv preprint arXiv:2110.13846 (2021)
Acknowledgement
This work was supported by the University of Edinburgh, the Royal Academy of Engineering and Canon Medical Research Europe by a PhD studentship to Xiao Liu. This work was partially supported by the Alan Turing Institute under the EPSRC grant EP/N510129/1. S.A. Tsaftaris acknowledges the support of Canon Medical and the Royal Academy of Engineering and the Research Chairs and Senior Research Fellowships scheme (grant RCSRF1819
8
25).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, X., Thermos, S., Sanchez, P., O’Neil, A.Q., Tsaftaris, S.A. (2022). vMFNet: Compositionality Meets Domain-Generalised Segmentation. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13437. Springer, Cham. https://doi.org/10.1007/978-3-031-16449-1_67
Download citation
DOI: https://doi.org/10.1007/978-3-031-16449-1_67
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16448-4
Online ISBN: 978-3-031-16449-1
eBook Packages: Computer ScienceComputer Science (R0)
