Abstract
In recent years, deep learning models have considerably advanced the performance of segmentation tasks on Brain Magnetic Resonance Imaging (MRI). However, these models show a considerable performance drop when they are evaluated on unseen data from a different distribution. Since annotation is often a hard and costly task requiring expert supervision, it is necessary to develop ways in which existing models can be adapted to the unseen domains without any additional labelled information. In this work, we explore one such technique which extends the CycleGAN [2] architecture to generate label-preserving data in the target domain. The synthetic target domain data is used to train the nn-UNet [3] framework for the task of multi-label segmentation. The experiments are conducted and evaluated on the dataset [1] provided in the ‘Cross-Modality Domain Adaptation for Medical Image Segmentation’ challenge [23] for segmentation of vestibular schwannoma (VS) tumour and cochlea on contrast enhanced (ceT1) and high resolution (hrT2) MRI scans. In the proposed approach, our model obtains dice scores (DSC) 0.73 and 0.49 for tumour and cochlea respectively on the validation set of the dataset. This indicates the applicability of the proposed technique to real-world problems where data may be obtained by different acquisition protocols as in [1] where hrT2 images are more reliable, safer, and lower-cost alternative to ceT1.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Code available at: https://github.com/aitorzip/PyTorch-CycleGAN.
- 2.
Code available at: https://github.com/MIC-DKFZ/nnUNet.
References
Shapey, J., et al.: Segmentation of vestibular schwannoma from MRI, an open annotated dataset and baseline algorithm. Sci. Data 8(1), 286 (2021). https://doi.org/10.1038/s41597-021-01064-w
Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. CoRR, abs/1703.10593 (2017)
Isensee, F., et al.: nn-UNet: self-adapting framework for U-Net-based medical image segmentation. CoRR, abs/1809.10486 (2018)
Isensee, F., Jaeger, P.F., Full, P.M., Vollmuth, P., Maier-Hein, K.H.: nn-UNet for brain tumor segmentation. CoRR, abs/2011.00848 (2020)
Gretton, A., Borgwardt, K.M., Rasch, M.J., Schölkopf, B., Smola, A.J.: A kernel method for the two-sample-problem. In: Schölkopf, B., Platt, J.C., Hofmann, T. (eds.) NIPS, pp. 513–520. MIT Press (2006). ISBN: 0-262-19568-2
Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. CoRR, abs/1511.05547 (2015)
Damodaran, B.B., Kellenberger, B., Flamary, R., Tuia, D., Courty, N.: DeepJDOT: deep joint distribution optimal transport for unsupervised domain adaptation. CoRR, abs/1803.10081 (2018)
Kang, G., Jiang, L., Yang, Y., Hauptmann, A.G.: Contrastive adaptation network for unsupervised domain adaptation. CoRR, abs/1901.00976 (2019)
Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation (2014)
Goodfellow, I., et al.: Generative Adversarial Nets. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc. (2014). https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
Shen, J., Qu, Y., Zhang, W., Yu, Y.: Wasserstein distance guided representation learning for domain adaptation. In: McIlraith, S.A., Weinberger, K.Q. (eds.) AAAI, pp. 4058–4065. AAAI Press (2018)
Chen, C., Dou, Q., Chen, H., Qin, J., Heng, P.-A.: Synergistic image and feature adaptation: towards cross-modality domain adaptation for medical image segmentation. In: AAAI, pp. 865–872. AAAI Press (2019). ISBN: 978-1-57735-809-1
Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting batch normalization for practical domain adaptation. In: ICLR Workshop. OpenReview.net (2017)
Carlucci, F.M., Porzi, L., Caputo, B., Ricci, E., Bulò, S.R.: AutoDIAL: automatic domain alignment layers. In: ICCV, pp. 5077–5085. IEEE Computer Society (2017). ISBN: 978-1-5386-1032-9
Saito, K., Ushiku, Y., Harada, T.: Asymmetric tri-training for unsupervised domain adaptation. In: Precup, D., Teh, Y.W. (eds) ICML, pp. 2988–2997. PMLR (2017)
Michieli, U., Biasetton, M., Agresti, G., Zanuttigh, P.: Adversarial learning and self-teaching techniques for domain adaptation in semantic segmentation. IEEE Trans. Intell. Veh. 5, 508–518 (2020)
Chang, W.L., Wang, H.P., Peng, W.H., Chiu, W.C.: All about structure: adapting structural information across domains for boosting semantic segmentation. In: CVPR, pp. 1900–1909. Computer Vision Foundation/IEEE (2019)
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., Hardt, M.: Test-time training with self-supervision for generalization under distribution shifts. In: Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 9229–9248 (2020). https://proceedings.mlr.press/v119/sun20b.html
Karani, N., Chaitanya, K., Konukoglu, E.: Test-time adaptable neural networks for robust medical image segmentation. CoRR, abs/2004.04668 (2020)
Varsavsky, T., Orbes-Arteaga, M., Sudre, C.H., Graham, M.S., Nachev, P., Cardoso, M.J.: Test-time unsupervised domain adaptation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 428–436. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_42 ISBN: 978-3-030-59710-8
Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: Fully Test-Time Adaptation by Entropy Minimization. In: International Conference on Learning Representations (2021). https://openreview.net/forum?id=uXl3bZLkr3c
Campello, V.M., et al.: Multi-centre, multi-vendor and multi-disease cardiac segmentation: the M&Ms challenge. IEEE Trans. Med. Imaging 40(12), 3543–3554 (2021). https://doi.org/10.1109/TMI.2021.3090082
Dorent, R., et al.: CrossMoDA 2021 challenge: Benchmark of Cross-Modality Domain Adaptation techniques for Vestibular Schwannoma and Cochlea Segmentation (2022). arXiv: 2201.02831. https://doi.org/10.48550/arxiv.2201.02831
Yi, Z., Zhang, H.(Richard), Tan, P., Gong, M.: DualGAN: unsupervised dual learning for image-to-image translation. Paper presented at the meeting of the ICCV (2017)
Royer, A., et al.: XGAN: unsupervised image-to-image translation for many-to-many mappings. In: Singh, R., Vatsa, M., Patel, V.M., Ratha, N. (eds.) Domain Adaptation for Visual Understanding, pp. 33–49. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-30671-7_3
Acknowledgements
This work has been partially supported by the Spanish project PID2019-105093GB-I00 and by ICREA under the ICREA Academia programme. Additionally, this work has also been supported in part by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 825903 and No. 952103.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Joshi, S. et al. (2022). nn-UNet Training on CycleGAN-Translated Images for Cross-modal Domain Adaptation in Biomedical Imaging. In: Crimi, A., Bakas, S. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2021. Lecture Notes in Computer Science, vol 12963. Springer, Cham. https://doi.org/10.1007/978-3-031-09002-8_47
Download citation
DOI: https://doi.org/10.1007/978-3-031-09002-8_47
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09001-1
Online ISBN: 978-3-031-09002-8
eBook Packages: Computer ScienceComputer Science (R0)