Abstract
Semantic segmentation is a fundamental computer vision task attracting a lot of attention. However, limited works focus on semantic segmentation on fine-grained class scenario, which has more classes and greater inter-class similarity. Due to the lack of data available for this task, we establish two segmentation benchmarks, CUB-seg and FGSCR42-seg, based on CUB and FGSCR42 datasets. To solve the two major problems in this task, spatial inconsistency and extremely similar classes confusion, we propose the Spatial Consistency and Class-level Diversity enhancement Network. First, we build the Spatial Consistency Enhancement Module to take advantage of the low-frequency information in the feature, enhancing the spatial consistency. Second, Fine-grained Regions Contrastive Loss is designed to make the features of different classes more discriminative, promoting the class-level diversity. Extensive experiments show that our method can significantly improve the performance compared to baseline models. Visualization study also prove the effectiveness of our method for enhancing spatial consistency and class-level diversity.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Long, J., Evan, S., Trevor, D.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Badrinarayanan, V., Alex, K., Roberto, C.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017)
Chen, L., George, P., Iasonas, K., Kevin, M., Alan, L.Y.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv preprint http://arxiv.org/abs/1412.7062 (2014)
Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2881–2890 (2017)
Chen, L., George, P., Iasonas, K., Kevin, M., Alan, L.Y.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans. Pattern Anal. Mach. Intell. 40, 834–848 (2017)
Tian, Z., Zhao, H., Shu, M., Yang, Z., Li, R., Jia, J.: Prior guided feature enrichment network for few-shot segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 44(2), 1050–1065 (2020)
Zhao, Q., Liu, B., Lyu, S., Chen, H.: A self-distillation embedded supervised affinity attention model for few-shot segmentation. IEEE Trans. Cogn. Dev. Syst. (2023). https://doi.org/10.1109/TCDS.2023.3251371
Bucher, M., Vu, T., Matthieu, C., Patrick, P.: Zero-shot semantic segmentation. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Xia, F., Wang, P., Chen, X., Alan, L.Y.: Joint multi-person pose estimation and semantic part segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6769–6778 (2017)
Zhang, X., Zhao, W., Luo, H., Peng, J., Fan, J.: Class guided channel weighting network for fine-grained semantic segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, pp. 3344–3352 (2022)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 dataset. In: Computation and Neural Systems Technical Report, 2010–001 (2011)
Di, Y., Jiang, Z., Zhang, H.: A public dataset for fine-grained ship classification in optical remote sensing images. Remote Sens. 13(4), 747 (2021). https://doi.org/10.3390/rs13040747
Sofiiuk, K., Petrov, I., Barinova, O., Konushin, A.: F-BRS: rethinking backpropagating refinement for interactive segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8620–8629 (2020). https://doi.org/10.1109/CVPR42600.2020.00865
Fu, J., et al.: Dual attention network for scene segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3146–3154 (2019)
Chen, L., George P., Florian S., Hartwig A.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint http://arxiv.org/abs/1706.05587 (2017)
Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., Liu, W.: CCNet: criss-cross attention for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 603–612 (2019)
Enze, X., Wenhai, W., Zhiding, Y., Anima A., Jose, A., Ping, L.: SegFormer: simple and efficient design for semantic segmentation with transformers. In: Advances in Neural Information Processing Systems, vol. 34, pp. 12077–12090 (2021)
Everingham, M., Eslami, S., Van, G., Williams, C., Winn, J., Zisserman, A.: The PASCAL visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: International Conference on Learning Representations (2020)
Zhuang, P., Wang, Y., Qiao, Y.: Learning attentive pairwise interaction for fine-grained classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07, pp. 13130–13137 (2020)
He, J., et al.: TransFG: a transformer architecture for fine-grained recognition. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 1, pp. 852–860 (2022)
Khosla, A., Jayadevaprakash, N., Yao, B., Li, F.: Novel dataset for fine-grained image categorization: Stanford dogs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2011)
Kaiming, H., Xiangyu, Z., Shaoqing, R., Jian, S.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 62072021 and 62002005.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhao, Q., Liu, B., Lyu, S., Wang, C., Yang, Y. (2024). Enhancing Spatial Consistency and Class-Level Diversity for Segmenting Fine-Grained Objects. In: Luo, B., Cheng, L., Wu, ZG., Li, H., Li, C. (eds) Neural Information Processing. ICONIP 2023. Communications in Computer and Information Science, vol 1965. Springer, Singapore. https://doi.org/10.1007/978-981-99-8145-8_23
Download citation
DOI: https://doi.org/10.1007/978-981-99-8145-8_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8144-1
Online ISBN: 978-981-99-8145-8
eBook Packages: Computer ScienceComputer Science (R0)