Abstract
Purpose
Well-established segmentation models will suffer performance degradation when deployed on data with heterogeneous features, especially in the field of medical image analysis. Although researchers have proposed many approaches to address this problem in recent years, most of them are feature-adaptation-based adversarial networks, the problems such as training instability often arise in adversarial training. To ameliorate this challenge and improve the robustness of processing data with different distributions, we propose a novel unsupervised domain adaptation framework for cross-domain medical image segmentation.
Methods
In our proposed approach, Fourier transform guided images translation and multi-model ensemble self-training are integrated into a unified framework. First, after Fourier transform, the amplitude spectrum of source image is replaced with that of target image, and reconstructed by the inverse Fourier transform. Second, we augment target dataset with the synthetic cross-domain images, performing supervised learning using the original source set labels while implementing regularization by entropy minimization on predictions of unlabeled target data. We employ several segmentation networks with different hyperparameters simultaneously, pseudo-labels are generated by averaging their outputs and comparing to confidence threshold, and gradually optimize the quality of pseudo-labels through multiple rounds self-training.
Results
We employed our framework to two liver CT datasets for bidirectional adaptation experiments. In both experiments, compared to the segmentation network without domain alignment, dice similarity coefficient (DSC) increased by nearly 34% and average symmetric surface distance (ASSD) decreased by about 10. The DSC values were also improved by 10.8% and 6.7%, respectively, compared to the existing model.
Conclusion
We propose a Fourier transform-based UDA framework, the experimental results and comparisons demonstrate that the proposed method can effectively diminish the performance degradation caused by domain shift and performs best on the cross-domain segmentation tasks. Our proposed multi-model ensemble training strategy can also improve the robustness of the segmentation system.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Rekka M, Nawres K, Henda N, Saoussen HZ (2021) A bilinear convolutional neural network for lung nodules classification on CT images. Int J CARS 16:91–101. https://doi.org/10.1007/s11548-020-02283-z
NiroomandFam B, Nikravanshalmani A, Khalilian M (2021) Automatic breast mass detection in mammograms using density of wavelet coefficients and a patch-based CNN. Int J CARS 16:1805–1815. https://doi.org/10.1007/s11548-021-02443-9
Tan T, Wang Z, Du H, Xu J, Qiu B (2021) Lightweight pyramid network with spatial attention mechanism for accurate retinal vessel segmentation. Int J CARS 16:673–682. https://doi.org/10.1007/s11548-021-02344-x
Matthew P, Neumann M, Iyyer M, Gardner M, Zettlemoyer L (2018) Deep Contextualized Word Representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp 2227–2237. New Orleans, Louisiana. Association for Computational Linguistics. https://aclanthology.org/N18-1202
Yang YC, Soatto S (2020) FDA: Fourier domain adaptation for semantic segmentation. In: 2020 IEEE conference on computer vision and pattern recognition (CVPR). pp 4085–4095. https://doi.org/10.1109/CVPR42600.2020.00414
Yang YC, Lao D, Sundaramoorthi G, Soatto S (2020) Phase consistent ecological domain adaptation. In: 2020 IEEE conference on computer vision and pattern recognition (CVPR). pp 9008–9017.https://doi.org/10.1109/CVPR42600.2020.00903
Piotrowski LN, Campbell FW (1982) A demonstration of the visual importance and flexibility of spatial-frequency amplitude and phase. Perception 11(3):337–346
Oppenheim AV, Lim JS (1981) The importance of phase in signals. Proc IEEE 69(5):529–541
Hansen BC, Hess RF (2007) Structural sparseness and spatial phase alignment in natural scenes. J Opt Soc Am A 24(7):1873–1885. https://doi.org/10.1364/JOSAA.24.001873
Tarvainen A, Valpola H (2017) Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results. In: 2020 conference and workshop on neural information processing systems (NeurIPS). https://doi.org/10.48550/arXiv.1703.01780
Laine S, Aila T (2017) Temporal Ensembling for semi-supervised learning. In: Proceedings of the 5th international conference on learning representations (ICLR). https://doi.org/10.48550/arXiv.1610.02242
Bilic P, Christ PF, Vorontsov E, Chlebus G, Chen H, Dou Q, Fu CW, Han X, Heng PA, Hesser J, Kadoury S, Kopczyski T, Le M, Li CM, Li XM, Lipkova J, Lowengrub J, Meine H, Moltz JH, Pal C, Piraud M, Qi XJ, Qi J, Rempfler M, Roth K, Schenk A, Sekuboyina A, Zhou P, Hulsemeyer C, Beetz M, Ettlinger F, Gruen F, Kaissis G, Lohfer F, Braren R, Holch J, Hofmann F, Sommer W, Heinemann V, Jacobs C, Mamani GEH, Ginneken B, Chartrand G, Tang A, Drozdzal M, Kadoury S, Ben-Cohen A, Klang E, Amitai M, Konen E, Greenspan H, Moreau J, Hostettler A, Soler L, Vivanti R, Szeskin A, Cohain Naama, Sosna J, Joskowicz L, Kumar A, Kore A, Wang CL, Feng DG, Li F, Krishnamurthi G, He J, Wu JR, Kim J, Zhou JY, Ma J, Li JB, Maninis KK, Kaluva CK, Bi L, Khened M, Beliver M, Lin QZ, Yang XP, Yuan YD, Chen YN, Li YQ, Qiu YD, Wu YL, Menze B (2019) The liver tumor segmentation benchmark (LiTS). https://arxiv.org/abs/1901.04056v1
Kavur AE, Gezer NS, Baris M, Aslan S, Conze PH, Groza V, Pham D, Chatterjee S, Ernst P, Ozkan S, Baydar B, Lachinov D, Han S, Pauli J, Isensee F, Perkonigg M, Sathish R, Rajan R, Sheet D, Dovletov G, Speck O, Nurnberger A, Hein M, Akar GB, Unal G, Dicle O, Selver MA (2021) CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation. Med Image Anal. https://doi.org/10.1016/j.media.2020.101950
Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). pp 234–241.https://doi.org/10.1007/978-3-319-24574-4_28
Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE international conference on computer vision (ICCV). pp 2242–2251. https://doi.org/10.1109/ICCV.2017.244
Vu TH, Jain H, Bucher M, Cord M, Perez P (2019). ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation. In: 2019 IEEE conference on computer vision and pattern recognition (CVPR). pp 2517–2526 https://doi.org/10.1109/CVPR.2019.00262
Acknowledgements
This research work is supported by the grants from National Natural Science Foundation of China (61673007). We sincerely thank reviewers for their good advice.
Funding
This research work is supported by the grants from National Natural Science Foundation of China (61673007).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Consent to participate
This article does not contain patient data.
Consent for publication
This article does not contain patient data.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Jiang, K., Gong, T. & Quan, L. A medical unsupervised domain adaptation framework based on Fourier transform image translation and multi-model ensemble self-training strategy. Int J CARS 18, 1885–1894 (2023). https://doi.org/10.1007/s11548-023-02867-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-023-02867-5