Abstract
Prototype learning has been widely explored for few-shot segmentation. Existing methods typically learn the prototype from the foreground features of all support images, which rarely consider the background similarities between the query images and the support images. This unbalanced prototype learning strategy limits its capability to mutually correct the segmentation errors between the foreground and background. In this paper, we propose a Complementary Prototype Learning and Cascaded Refinement (CPLCR) network for few-shot segmentation. Firstly, both the foreground and background features of the support images are used to learn our complementary prototypes. Then, the foreground and background similarity maps are jointly derived between the query image feature and our complementary prototypes, which capture more comprehensive prior information. Finally, we fuse the query image feature, foreground prototype and the foreground/background similarity maps together, and feed them to a cascaded refinement module, which recursively reuses the output of previous iteration to refine the segmentation result. Extensive experimental results show that the proposed CPLCR model outperforms many state-of-the-art methods for 1-shot and 5-shot segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)
Dong, N., Xing, E.P.: Few-shot semantic segmentation with prototype learning. In: BMVC, vol. 3 (2018)
Everingham, M., Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2009)
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)
Hariharan, B., Arbeláez, P., Bourdev, L.D., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: 2011 International Conference on Computer Vision, pp. 991–998 (2011)
Liu, Y., Zhang, X., Zhang, S., He, X.: Part-aware prototype network for few-shot semantic segmentation. ArXiv abs/2007.06309 (2020)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Luo, K., Meng, F., Wu, Q., Li, H.: Weakly supervised semantic segmentation by multiple group cosegmentation. In: 2018 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2018)
Nguyen, K.D.M., Todorovic, S.: Feature weighting and boosting for few-shot segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 622–631 (2019)
Rakelly, K., Shelhamer, E., Darrell, T., Efros, A.A., Levine, S.: Conditional networks for few-shot semantic segmentation. In: ICLR (2018)
Shaban, A., Bansal, S., Liu, Z., Essa, I., Boots, B.: One-shot learning for semantic segmentation. ArXiv abs/1709.03410 (2017). arXiv:1709.03410
Siam, M., Oreshkin, B.N., Jagersand, M.: AMP: adaptive masked proxies for few-shot segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5249–5258 (2019)
Wang, K., Liew, J., Zou, Y., Zhou, D., Feng, J.: PANet: few-shot image semantic segmentation with prototype alignment. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9196–9205 (2019)
Wang, K., Liew, J.H., Zou, Y., Zhou, D., Feng, J.: PANet: few-shot image semantic segmentation with prototype alignment. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9197–9206 (2019)
Xu, X., Meng, F., liang Li, H., Wu, Q., Ngan, K.N., Chen, S.: A new bounding box based pseudo annotation generation method for semantic segmentation. In: 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), pp. 100–103 (2020)
Yang, B., Liu, C., Li, B., Jiao, J., Ye, Q.: Prototype mixture models for few-shot semantic segmentation. ArXiv abs/2008.03898 (2020)
Yang, M., Yu, K., Zhang, C., Li, Z., Yang, K.: DenseASPP for semantic segmentation in street scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3684–3692 (2018)
Yang, Y., Meng, F., Li, H., Ngan, K., Wu, Q.: A new few-shot segmentation network based on class representation. In: 2019 IEEE Visual Communications and Image Processing (VCIP), pp. 1–4 (2019)
Yang, Y., Meng, F., Li, H., Wu, Q., Xu, X., Chen, S.: A new local transformation module for few-shot segmentation. ArXiv abs/1910.05886 (2020)
Zhang, C., Lin, G., Liu, F., Guo, J., Wu, Q., Yao, R.: Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9586–9594 (2019)
Zhang, C., Lin, G., Liu, F., Yao, R., Shen, C.: CANet: class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5217–5226 (2019)
Zhang, X., Wei, Y., Yang, Y., Huang, T.: SG-One: similarity guidance network for one-shot semantic segmentation. IEEE Trans. Cybern. 50(9), 3855–3865 (2020)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)
Acknowledgement
This work was partially supported by National Natural Science Foundation of China (No. 61971095, 61871078, 61831005, and 61871087).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Luo, H. et al. (2021). Few-Shot Segmentation via Complementary Prototype Learning and Cascaded Refinement. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13022. Springer, Cham. https://doi.org/10.1007/978-3-030-88013-2_40
Download citation
DOI: https://doi.org/10.1007/978-3-030-88013-2_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88012-5
Online ISBN: 978-3-030-88013-2
eBook Packages: Computer ScienceComputer Science (R0)