Abstract
Pseudo-labeling based methods are commonly employed for effectively utilizing unlabeled data in semi-supervised semantic image segmentation. It tends to select high-confidence pixels in images as pseudo labels and discard most of the pixels predicted with low confidence, but each pixel is valuable for accurate segmentation. Therefore, we propose a semi-supervised semantic image segmentation algorithm based on complementary reconfirmation mechanism (CR-Seg) to constrain the low-confidence pixels. Firstly, the predictions are divided into high-confidence and low-confidence pixels by a dynamic threshold. The High-confidence pixels supervise the predictions of the student model and guide the classifier to learn what they are. While the low-confidence pixels are equally important for model training, they are used to generate complementary reconfirmation loss for providing additional information as complementary labels. Our method achieves mIoU of 69.12, 73.84, 74.03, and 76.91% under 1/16, 1/8, 1/4, and 1/2 partitions of the classic PASCAL VOC 2012. The experimental results demonstrate that low-confidence pixels can provide more information to the model as complementary labels, thereby improving the model’s segmentation performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X.: Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 43(10), 3349–3364 (2020)
French, G., Laine, S., Aila, T., Mackiewicz, M., Finlayson, G.: Semi-supervised semantic segmentation needs strong, varied perturbations. In: British Machine Vision Conference, pp. 1–21 (2019)
Ouali, Y., Hudelot, C., Tami, M.: Semi-supervised semantic segmentation with cross-consistency training. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12,674–12,684 (2020)
Wang, Y., Wang, H., Shen, Y., Fei, J., Li, W., Jin, G., Wu, L., Zhao, R., Le, X.: Semi-supervised semantic segmentation using unreliable pseudo-labels. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4248–4257 (2022)
Chen, X., Yuan, Y., Zeng, G., Wang, J.: Semi-supervised semantic segmentation with cross pseudo supervision. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2613–2622 (2021)
Sohn, K., Berthelot, D., Carlini, N., Zhang, Z., Zhang, H., Raffel, C.A., Cubuk, E.D., Kurakin, A., Li, C.L.: Fixmatch: simplifying semi-supervised learning with consistency and confidence. Adv. Neural. Inf. Process. Syst. 33, 596–608 (2020)
Liu, S., Zhi, S., Johns, E., Davison, A.J.: Bootstrapping semantic segmentation with regional contrast. In: International Conference on Learning Representations, pp. 774–791 (2021)
Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural. Inf. Process. Syst. 30, 774–790 (2017)
Duan, Y., Zhao, Z., Qi, L., Wang, L., Zhou, L., Shi, Y., Gao, Y.: Mutexmatch: semi-supervised learning with mutex-based consistency regularization. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Zhang, B., Wang, Y., Hou, W., Wu, H., Wang, J., Okumura, M., Shinozaki, T.: Flexmatch: boosting semi-supervised learning with curriculum pseudo labeling. Adv. Neural. Inf. Process. Syst. 34, 18408–18419 (2021)
Souly, N., Spampinato, C., Shah, M.: Semi supervised semantic segmentation using generative adversarial network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5688–5696 (2017)
Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: Cutmix: Regularization strategy to train strong classifiers with localizable features. In: The IEEE/CVF International Conference on Computer Vision, pp. 6023–6032 (2019)
Everingham, M., Eslami, S.A., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vision 111, 98–136 (2015)
Cordts, M., Omran, M., Ramos, S. et al.: The cityscapes dataset for semantic urban scene understanding. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 3213–3223 (2016)
Hariharan, B., Arbeláez, P., Bourdev, L., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: 2011 International Conference on Computer Vision, pp. 991–998. IEEE (2011)
Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: The European Conference on Computer Vision (ECCV), pp. 801–818 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Acknowledgment
This work is supported in part by the Key Program of NSFC (No. U1908214), 111 Project (No. D23006), the Scientific Research Fundation of Education Department of Liaoning Province (No. LJKMZ20221839), the Science and Technology Innovation Fund of Dalian (No. 2020JJ25CY001), Program for Innovative Research Team in University of Liaoning Province (LT2020015), Support Plan for Key Field Innovation Team of Dalian (2021RT06).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xiao, Y., Dong, J., Zhang, Q., Yi, P., Liu, R., Wei, X. (2024). Semi-supervised Semantic Segmentation with Complementary Reconfirmation Mechanism. In: Naik, N., Jenkins, P., Grace, P., Yang, L., Prajapat, S. (eds) Advances in Computational Intelligence Systems. UKCI 2023. Advances in Intelligent Systems and Computing, vol 1453. Springer, Cham. https://doi.org/10.1007/978-3-031-47508-5_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-47508-5_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47507-8
Online ISBN: 978-3-031-47508-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)