Skip to main content
SpringerLink
Log in

M-MSSEU: source-free domain adaptation for multi-modal stroke lesion segmentation using shadowed sets and evidential uncertainty

  • Research
  • Published:
Health Information Science and Systems Aims and scope Submit manuscript

Abstract

Due to the unavailability of source domain data encountered in unsupervised domain adaptation, there has been an increasing number of studies on source-free domain adaptation (SFDA) in recent years. To better solve the SFDA problem and effectively leverage the multi-modal information in medical images, this paper presents a novel SFDA method for multi-modal stroke lesion segmentation in which evidential deep learning instead of convolutional neural network. Specifically, for multi-modal stroke images, we design a multi-modal opinion fusion module which uses Dempster-Shafer evidence theory for decision fusion of different modalities. Besides, for the SFDA problem, we use the pseudo label learning method, which obtains pseudo labels from the pre-trained source model to perform the adaptation process. To solve the unreliability of pseudo label caused by domain shift, we propose a pseudo label filtering scheme using shadowed sets theory and a pseudo label refining scheme using evidential uncertainty. These two schemes can automatically extract unreliable parts in pseudo labels and jointly improve the quality of pseudo labels with low computational costs. Experiments on two multi-modal stroke lesion datasets demonstrate the superiority of our method over other state-of-the-art SFDA methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

The data and materials of this study will be made available by the corresponding author upon a reasonable request.

References

  1. El-Hariri H, Neto LASM, Cimflova P, Bala F. Evaluating nnu-uet for early ischemic change segmentation on non-contrast computed tomography in patients with acute ischemic stroke. Comput Biol Med. 2022;141: 105033.

    Article  Google Scholar 

  2. Khezrpour S, Seyedarabi H, Razavi SN, Farhoudi M. Automatic segmentation of the brain stroke lesions from MR flair scans using improved U-net framework. Biomed Signal Process Control. 2022;78:103978.

    Article  Google Scholar 

  3. Wilson G, Cook DJ. A survey of unsupervised deep domain adaptation. ACM Trans Intell Syst Technol. 2020;11:1–46.

    Article  Google Scholar 

  4. Fang Y, Yap P-T, Lin W, Zhu H, Liu M. Source-free unsupervised domain adaptation: a survey 2022. arXiv preprint. Available from: arXiv:2301.00265

  5. Gawlikowski J, Tassi CRN, Ali M, Lee J, Humt M, Feng J, Kruspe A, Triebel R, Jung P, Roscher R, et al. A survey of uncertainty in deep neural networks 2021. arXiv preprint. Available from:arXiv:2107.03342

  6. Lai Y, Shi Y, Han Y, Shao Y, Qi M, Li B. Exploring uncertainty in deep learning for construction of prediction intervals 2021. arXiv preprint. Available from: arXiv:2104.12953

  7. Boehm KM, Khosravi P, Vanguri R, Gao J, Shah SP. Harnessing multimodal data integration to advance precision oncology. Nat Rev Cancer. 2022;22(2):114–26.

    Article  Google Scholar 

  8. Cui C, Yang H, Wang Y, Zhao S, Asad Z, Coburn LA, Wilson KT, Landman B, Huo Y. Deep multi-modal fusion of image and non-image data in disease diagnosis and prognosis: a review. Prog Biomed Eng. 2023;5(13):52–65.

    Google Scholar 

  9. Guan H, Liu M. Domain adaptation for medical image analysis: a survey. IEEE Trans Biomed Eng. 2022;69(3):1173–85.

    Article  Google Scholar 

  10. Yang C, Guo X, Chen Z, Yuan Y. Source free domain adaptation for medical image segmentation with fourier style mining. Med Image Anal. 2022;79: 102457.

    Article  Google Scholar 

  11. Bateson M, Kervadec H, Dolz J, Lombaert H, Ayed IB. Source-free domain adaptation for image segmentation. Med Image Anal. 2022;82:102617.

    Article  Google Scholar 

  12. Liu X, Yuan Y. A source-free domain adaptive polyp detection framework with style diversification flow. IEEE Trans Med Imaging. 2022;41(7):1897–908.

    Article  Google Scholar 

  13. Kondo S. Source-free unsupervised domain adaptation with norm and shape constraints for medical image segmentation 2022. arXiv preprint. Available from: arXiv:2209.01300

  14. Bateson M, Kervadec H, Dolz J, Lombaert H, Ben Ayed I. Source-relaxed domain adaptation for image segmentation. In: Medical image computing and computer assisted intervention. Cham: Springer; 2020. p. 490–9.

    Google Scholar 

  15. VS V, Valanarasu JMJ, Patel VM. Target and task specific source-free domain adaptive image segmentation 2022. arXiv preprint. Available from: arXiv:2203.15792

  16. Xu Z, Lu D, Wang Y, Luo J, Wei D, Zheng Y, Tong RKY. Denoising for relaxing: unsupervised domain adaptive fundus image segmentation without sourcedata. In: Medical image computing and computer assisted intervention. Cham: Springer; 2022. p. 214–24.

    Google Scholar 

  17. Liu L, Kurgan L, Wu F-X, Wang J. Attention convolutional neural network for accurate segmentation and quantification of lesions in ischemic stroke disease. Med Image Anal. 2020;65:101791.

    Article  Google Scholar 

  18. Dolz J, Desrosiers C, Ben Ayed I. IVD-Net: intervertebral disc localization and segmentation in MRI with a multi-modal UNet. In: Medical image computing and computer assisted intervention. Cham: Springer; 2019. p. 130–43.

    Google Scholar 

  19. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017; pp. 4700–4708.

  20. Kamnitsas K, Bai W, Ferrante E, McDonagh S, Sinclair M, Pawlowski N, Rajchl M, Lee M, Kainz B, Rueckert D, Glocker B. Ensembles of multiple models and architectures for robust brain tumour segmentation. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries. Berlin: Springer; 2018. p. 450–65.

    Chapter  Google Scholar 

  21. Han Z, Zhang C, Fu H, Zhou JT. Trusted multi-view classification with dynamic evidential fusion. IEEE Trans Pattern Anal Mach Intell. 2023;45(2):2551–66.

    Article  Google Scholar 

  22. Xu S, Chen Y, Ma C, Yue X. Deep evidential fusion network for medical image classification. Int J Approx Reason. 2022;150:188–98.

    Article  MathSciNet  Google Scholar 

  23. Rizve MN, Duarte K, Rawat YS, Shah M. In: Defense of pseudo-labeling: an uncertainty-aware pseudo-label selection framework for semi-supervised learning 2021. arXiv preprint. Available from: arXiv:2101.06329

  24. Chen C, Liu Q, Jin Y, Dou Q, Heng P-A. Source-free domain adaptive fundus image segmentation with denoised pseudo-labeling. In: Medical Image Computing and Computer Assisted Intervention, 2021;pp. 225–35.

  25. Gal Y, Ghahramani Z. Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International conference on machine learning, 2016; pp. 1050–1059.

  26. Zheng Z, Yang Y. Rectifying pseudo label learning via uncertainty estimation for domain adaptive semantic segmentation. Int J Comput Vis. 2021;129(4):1106–20.

    Article  Google Scholar 

  27. Van Amersfoort J, Smith L, Teh YW, Gal Y. Uncertainty estimation using a single deep deterministic neural network. Int Conf Mach Learn. 2020;119:9690–700.

    Google Scholar 

  28. Zheng H, Chen Y, Yue X, Ma C, Liu X, Yang P, Lu J. Deep pancreas segmentation with uncertain regions of shadowed sets. Magn Reson Imaging. 2020;68:45–52.

    Article  Google Scholar 

  29. Tong Z, Xu P, Denœux T. An evidential classifier based on Dempster-Shafer theory and deep learning. Neurocomputing. 2021;450:275–93.

    Article  Google Scholar 

  30. Sensoy M, Kaplan L, Kandemir M. Evidential deep learning to quantify classification uncertainty. In: Proceedings of the 32nd international conference on neural information processing system, 2018; pp. 3183–3193.

  31. Dempster AP. Upper and lower probability inferences based on a sample from a finite univariate population. Biometrika. 1967;54(3–4):515–28.

    Article  MathSciNet  Google Scholar 

  32. Jsang A. Subjective Logic: a formalism for reasoning under uncertainty. Berlin: Springer; 2018.

    Google Scholar 

  33. Ghesu FC, Georgescu B, Gibson E, Guendel S, Kalra MK, Singh R, Digumarthy SR, Grbic S, Comaniciu D. Quantifying and leveraging classification uncertainty for chest radiograph assessment. In: Medical image computing and computer assisted intervention. Berlin: Springer; 2019. p. 676–84.

    Google Scholar 

  34. Shafer G. A mathematical theory of evidence. Priceton: Princeton University Press; 1976. p. 42.

    Book  Google Scholar 

  35. Li H, Nan Y, Del Ser J, Yang G. Region-based evidential deep learning to quantify uncertainty and improve robustness of brain tumor segmentation. Neural Comput Appl. 2022;1:1–15.

    Google Scholar 

  36. Pedrycz W. Shadowed sets: representing and processing fuzzy sets. IEEE Trans Syst Man Cybern. 1998;28(1):103–9.

    Article  Google Scholar 

  37. Hernandez Petzsche MR, Rosa E, Hanning U, Wiest R, Valenzuela W, Reyes M, Meyer M, Liew S-L, Kofler F, Ezhov I, et al. ISLES 2022: a multi-center magnetic resonance imaging stroke lesion segmentation dataset. Sci Data. 2022;9(1):762.

    Article  Google Scholar 

  38. Maier O, Menze BH, Gablentz J, Häni L, Heinrich MP, et al. ISLES 2015 - a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. Med Image Anal. 2017;35:250–69.

    Article  Google Scholar 

  39. Marstal K, Berendsen F, Staring M, Klein S. Simpleelastix: a user-friendly, multi-lingual library for medical image registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2016; pp. 134–142.

  40. Karthik R, Gupta U, Jha A, Menaka R. A deep supervised approach for ischemic lesion segmentation from multimodal MRI using fully convolutional network. App Soft Comput. 2019;84: 105685.

    Article  Google Scholar 

  41. Wang S, Yu L, Li K, Yang X, Fu C-W, Heng P-A. Boundary and entropy-driven adversarial learning for fundus image segmentation. In: Medical image computing and computer assisted intervention. Berlin: Springer; 2019. p. 102–10.

    Google Scholar 

  42. Wang D, Shelhamer E, Liu S, Olshausen B, Darrell T. Tent: fully test-time adaptation by entropy minimization 2020. arXiv preprint. Available from: arXiv:2006.10726

  43. Yushkevich PA, Piven J, Cody Hazlett H, Gimpel Smith R, Ho S, Gee JC, Gerig G. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage. 2006;31(3):1116–28.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers and the associate editor for their insightful comments that significantly improved the quality of this paper. This work was supported by the National Nature Science Foundation of China under Grant Nos. 61872143, 82372029 and 61771196. Discipline Construction of Pudong New Area Health Commission (PWGw2020-01). Joint Research Project of Pudong New Area Health and Family Planning Commission (PW2021D-14). Discipline Construction of Pudong New Area Health Commission (PWZxk2022-03).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, East China University of Science and Technology, No.130 Meilong Road, Shanghai, 200237, China

    Zhicheng Wang, Hongqing Zhu, Ziying Wang & Ning Chen

  2. Department of Radiology, Gongli Hospital of Shanghai Pudong New Area, Shanghai, 200135, China

    Bingcang Huang & Weiping Lu

  3. Shanghai Health Commission Key Lab of Artificial Intelligence (AI)-Based Management of Inflammation and Chronic Diseases, Sino-French Cooperative Central Lab, Gongli Hospital of Shanghai Pudong New Area, Shanghai, 200135, China

    Ying Wang

Authors
  1. Zhicheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongqing Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bingcang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weiping Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ning Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongqing Zhu.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Zhu, H., Huang, B. et al. M-MSSEU: source-free domain adaptation for multi-modal stroke lesion segmentation using shadowed sets and evidential uncertainty. Health Inf Sci Syst 11, 46 (2023). https://doi.org/10.1007/s13755-023-00247-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13755-023-00247-6

Keywords

Search

Navigation