skip to main content
10.1145/3604078.3604098acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicdipConference Proceedingsconference-collections
research-article

Multi-decoder Networks for Semi-supervised Medical Image Segmentation

Published: 26 October 2023 Publication History

Abstract

To improve the performance of semi-supervised image segmentation, it is important to effectively generate pseudo-labels from unlabeled images. However, the impact of pseudo-label confidence on segmentation performance is often overlooked. Low-confidence pseudo-labels can misguide the model and lead to overfitting, making it challenging to use them effectively. To address this issue, we propose a consistency constraint-based network that employs one encoder and three decoders () to generate distinct pseudo-labels. To assess the confidence of the generated pseudo-labels, we introduce a critic network that learns relevant features and effectively regularizes the confidence of -generated pseudo-labels. For evaluating the unlabeled images, we define a loss function that minimizes entropy, consisting of three sets of losses. We compare the performance of our model with two other semi-supervised segmentation algorithms using Dice, MAE, and F1 indicators. Our results demonstrate that the model outperforms the comparison models on all three metrics. In summary, our proposed consistency constraint-based network with a critic network and entropy-based loss function can effectively generate high-confidence pseudo-labels for semi-supervised image segmentation and improve the overall performance of the model.

References

[1]
Ronneberger, O., Fischer, P., and Brox, T., “U-net: Convolutional networks for biomedical image segmentation,” in [Medical Image Computing and Computer-Assisted Intervention (MICCAI)], 234–241 (2015).
[2]
Zhou, Z., Rahman Siddiquee, M. M., Tajbakhsh, N., and Liang, J., “Unet++: A nested u-net architecture for medical image segmentation,” in [Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support (DLMIA)], 3–11 (2018).
[3]
Choi and Won, J., “Knowledge distillation from cross teaching teachers for efficient semi-supervised abdominal organ segmentation in ct,” in [Fast and Low-Resource Semi-supervised Abdominal Organ Segmentation (MICCAI)], 13816, 101–115 (2022).
[4]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N., “An image is worth 16x16 words: Transformers for image recognition at scale,” in [Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)], 10687–10698 (2021).
[5]
Wang, Z., Zheng, J.-Q., and Voiculescu, I., “An uncertainty-aware transformer for mri cardiac semantic segmentation via mean teachers,” in [Annual Conference on Medical Image Understanding and Analysis (MIUA)], 13413,494–507(2022).
[6]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N., “An image is worth 16x16 words: Transformers for image recognition at scale,” in [Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)], 10687–10698 (2021).
[7]
Wang, L., Wang, J., Zhu, L., Fu, H., Li, P., Cheng, G., Feng, Z., Li, S., and Heng, P.-A., “Dual multi-scale mean teacher network for semi-supervised infection segmentation in chest ct volume for covid-19,” arXiv preprint arXiv:2211.05548 (2022).
[8]
Huang, H., Chen, Z., Chen,C., Lu,M.,and Zou,Y.,“Complementary consistency semi-supervised learning for 3d left atrial image segmentation,” arXiv preprint arXiv:2210.01438 (2022).
[9]
Gu, R., Zhang, J., Wang, G., Lei, W., Song, T., Zhang, X., Li, K., and Zhang, S., “Contrastive semisupervised learning for domain adaptive segmentation across similar anatomical structures,” IEEE Transactions on Medical Imaging 42(1),245–256(2023).
[10]
Wang, Z. and Voiculescu, I., “Triple-view feature learning for medical image segmentation,” in [ResourceEfficient Medical Image Analysis (REMIA)],42–54(2022).
[11]
Hu, S., Ye, Y., Liao, Z., and Xia, Y., “Boundary-aware network for kidney parsing,” arXiv preprint arXiv:2208.13338 (2022).
[12]
Bukhari, S. T. and Mohy-ud Din, H., “Ed u-net for brain tumor segmentation: Submission to the rsna-asnrmiccai brats 2021 challenge,” in [Medical Image Computing and Computer Assisted Intervention (MICCAI)], 276–288 (2022).
[13]
Guo, M.-H., Lu, C.-Z., Hou, Q., Liu, Z., Cheng, M.-M., and Hu, S.-M., “Segnext: Rethinking convolutional attention design for semantic segmentation,” arXiv preprint arXiv:2209.08575 (2022).
[14]
Wang, Y., Wang, H., Shen, Y., Fei, J., Li, W., Jin, G., Wu, L., Zhao, R., and Le, X., “Semi-supervised semantic segmentation using unreliable pseudo-labels,” in [Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)],4248–4257(June2022).
[15]
Liu, Y., Tian, Y., Chen, Y., Liu, F., Belagiannis, V., and Carneiro, G., “Perturbed and strict mean teachers for semi-supervised semantic segmentation,” in [Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)], 4258–4267 (June 2022).
[16]
Abuduweili, A., Li, X., Shi, H., Xu, C.-Z., and Dou, D., “Adaptive consistency regularization for semisupervised transfer learning,” in [Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)], 6923–6932 (June 2021).
[17]
You, C., Dai, W., Min, Y., Liu, F., Zhang, X., Feng, C., Clifton, D. A., Zhou, S. K., Hamilton Staib, L., and Duncan, J. S., “Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective,” arXiv e-prints (Feb.2023).
[18]
Mendel, R., Rauber, D., de Souza, L. A., Papa, J. P., and Palm, C., “Error-correcting mean-teacher: Corrections instead of consistency-targets applied to semi-supervised medical image segmentation,” Computers in Biology and Medicine 154, 106585 (2023).
[19]
Pan, M.-Z., Liao, X.-L., Li, Z., Deng, Y.-W., Chen, Y., and Bian, G.-B., “Semi-supervised medical image segmentation guided by bi-directional constrained dual-task consistency,” Bioengineering 10(2) (2023).
[20]
Lu, L., Yin, M., Fu, L., and Yang, F., “Uncertainty-aware pseudo-label and consistency for semi-supervised medical image segmentation,” Biomedical Signal Processing and Control 79, 104203 (2023)
[21]
Zhang, S., Zhang, J., Tian, B., Lukasiewicz, T., and Xu, Z., “Multi-modal contrastive mutual learning and pseudo-label re-learning for semi-supervised medical image segmentation,” Medical Image Analysis 83, 102656 (2023).
[22]
Zhao, X., Qi, Z., Wang, S., Wang, Q., Wu, X., Mao, Y., and Zhang, L., “RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentation,” arXiv e-prints (Jan. 2023).
[23]
Zhao, X., Qi, Z., Wang, S., Wang, Q., Wu, X., Mao, Y., and Zhang, L., “RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentation,” arXiv e-prints (Jan. 2023).
[24]
Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N. Y., Kainz, B., “Attention u-net: Learning where to look for the pancreas,” arXiv preprint arXiv:1804.03999 (2018).
[25]
Tomar, N. K., Jha, D., Bagci, U., and Ali, S., “Tganet: Text-guided attention for improved polyp segmentation,” in [arXiv preprint arXiv:2205.04280], (2022).
[26]
Liu, X., Thermos, S., Sanchez, P., O'Neil, A., and Tsaftaris, S. A., “vmfnet: Compositionality meets domaingeneralised segmentation,” in [International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI)], (2022).
[27]
Shen, Q., Li, Y., Jin, J., and Liu, B., “Q-net: Query-informed few-shot medical image segmentation,” arXiv preprint arXiv:2208.11451 (2022).
[28]
Wang, H., Cao, P., Wang, J., and Zaiane, O. R., “Uctransnet: Rethinking the skip connections in u-net from a channel-wise perspective with transformer,” arXiv preprint arXiv:2109.04335 (2021).
[29]
Gribbon, K. T. and Bailey, D. G., “A novel approach to real-time bilinear interpolation,” in [Proceedings. Second IEEE International Workshop on Electronic Design, Test and Applications (DELTA)], 126–131, IEEE(2004).
[30]
Rukundo, O. and Cao, H., “Nearest neighbor value interpolation,” arXiv preprint arXiv:1211.1768 (2012).
[31]
Pang, S., Pang, C., Zhao, L., Chen, Y., Su, Z., Zhou, Y., Huang, M., Yang, W., Lu, H., and Feng, Q., “Spineparsenet: Spine parsing for volumetric mr image by a two-stage segmentation framework with semantic image representation,” IEEE Transactions on Medical Imaging 40(1), 262–273 (2021).
[32]
Caicedo, J. C., Goodman, A., Karhohs, K. W., Cimini, B. A., Ackerman, J., Haghighi, M., Heng, C., Becker, T., Doan, M., McQuin, C., “Nucleus segmentation across imaging experiments: the 2018 data science bowl,” Nature methods 16(12), 1247–1253 (2019).
[33]
Simpson, A. L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B. A., Litjens, G., Menze, B., “A large annotated medical image dataset for the development and evaluation of segmentation algorithms,” arXiv preprint arXiv:1902.09063 (2019).
[34]
Pollehn, L., “Bacteria detection with darkfield microscopy.” https://www.kaggle.com/longnguyen2306/ bacteria-detection-with-darkfield-microscopy/. Accessed January 3,2022.
[35]
“CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation,” Medical Image Analysis 69,101950(Apr.2021).
[36]
Jha, D., Smedsrud, P. H., Riegler, M., Halvorsen, P., de Lange, T., Johansen, D., and Johansen, H. D., “Kvasir-seg: A segmented polyp dataset,” ArXiv abs/1911.07069 (2020).
[37]
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., “Pytorch: An imperative style, high-performance deep learning library,” Advances in neural information processing systems 32, 8026–8037 (2019).
[38]
Kingma, D. P. and Ba, J., “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).
[39]
Mukkamala, M. C. and Hein, M., “Variants of rmsprop and adagrad with logarithmic regret bounds,” in [International Conference on Machine Learning (ICML)],2545–2553,PMLR(2017).
[40]
Peiris, H., Chen, Z., Egan, G., and Harandi, M., “Duo-segnet: Adversarial dual-views for semi-supervised medical image segmentation,” in [International Conference on Medical Image Computing and ComputerAssisted Intervention (MICCAI)],428–438(2021).
[41]
Tarvainen, A. and Valpola, H., “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” arXiv preprint arXiv:1703.01780 (2017).

Index Terms

  1. Multi-decoder Networks for Semi-supervised Medical Image Segmentation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICDIP '23: Proceedings of the 15th International Conference on Digital Image Processing
    May 2023
    711 pages
    ISBN:9798400708237
    DOI:10.1145/3604078
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 26 October 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Consistency Constraints
    2. Medical Image Segmentation
    3. Pseudo-label Learning
    4. Semi-supervised Learning

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICDIP 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 45
      Total Downloads
    • Downloads (Last 12 months)29
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 06 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media