skip to main content
10.1145/3633637.3633694acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiccprConference Proceedingsconference-collections
research-article

Class-Aware Feature Regularization for Semantic Segmentation

Published: 28 February 2024 Publication History

Abstract

In this paper, to address the problem of intra-class consistency and inter-class variation in the deep convolutional neural network (CNN) based methods for semantic segmentation of images, we propose a class-aware feature regularization strategy to revise the features extracted by a deep convolutional neural network, without any change of the original network structure. A pixel-context similarity term is proposed to measure the consistency between feature vectors of pixels and class centers, which guarantees the intra-class consistency of pixels in the interior of an object and is supervised by a One-Hot label to preserve the inter-class variation of different objects. Based on the similarity term, we design a lightweight and efficient plug-in loss term to ensure that the features yielded by a deep CNN possess the quality of intra-class consistency and inter-class variation. As our ideal can be fulfilled effectively by the proposed plug-in loss term, we can simply incorporate it into a CNN-based segmentation model without changing the model structure. The effectiveness of the proposed strategy is proved by incorporating the loss term into some state-of-the-art segmentation models on Cityscapes and ADE20K datasets.

References

[1]
Sonali Bhadoria, Preeti Aggarwal, Chandrashekhar G. Dethe, and Renu Vig. 2012. Comparison of Segmentation Tools for Multiple Modalities in Medical Imaging. Journal of Advances in Information Technology 3 (2012), 197–205. https://api.semanticscholar.org/CorpusID:52262554
[2]
Shubhankar Borse, Hong Cai, Yizhe Zhang, and Fatih Porikli. 2021. Hs3: Learning with proper task complexity in hierarchically supervised semantic segmentation. arXiv preprint arXiv:2111.02333 (2021).
[3]
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3213–3223.
[4]
Jun Fu, Jing Liu, Haijie Tian, Yong Li, Yongjun Bao, Zhiwei Fang, and Hanqing Lu. 2019. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 3146–3154.
[5]
Junjun He, Zhongying Deng, Lei Zhou, Yali Wang, and Yu Qiao. 2019. Adaptive pyramid context network for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7519–7528.
[6]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
[7]
Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132–7141.
[8]
Ye Huang, Di Kang, Liang Chen, Xuefei Zhe, Wenjing Jia, Linchao Bao, and Xiangjian He. 2022. Car: Class-aware regularizations for semantic segmentation. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXVIII. Springer, 518–534.
[9]
Chen-Yu Lee, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu. 2015. Deeply-supervised nets. In Artificial intelligence and statistics. PMLR, 562–570.
[10]
Sun-Ao Liu, Hongtao Xie, Hai Xu, Yongdong Zhang, and Qi Tian. 2022. Partial Class Activation Attention for Semantic Segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16836–16845.
[11]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022.
[12]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440.
[13]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. arxiv:1505.04597 [cs.CV]
[14]
Daniel Seichter, Mona Köhler, Benjamin Lewandowski, Tim Wengefeld, and Horst-Michael Gross. 2021. Efficient rgb-d semantic segmentation for indoor scene analysis. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 13525–13531.
[15]
Ke Sun, Yang Zhao, Borui Jiang, Tianheng Cheng, Bin Xiao, Dong Liu, Yadong Mu, Xinggang Wang, Wenyu Liu, and Jingdong Wang. 2019. High-resolution representations for labeling pixels and regions. arXiv preprint arXiv:1904.04514 (2019).
[16]
Farha Fatina Wahid, Raju G., Shijo Joseph, Drdebabrata Swain, Om Das, and Biswaranjan Acharya. 2023. A Novel Fuzzy-Based Thresholding Approach for Blood Vessel Segmentation from Fundus Image. Journal of Advances in Information Technology 14 (01 2023), 185–192. https://doi.org/10.12720/jait.14.2.185-192
[17]
Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7794–7803.
[18]
Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. 2018. Unified perceptual parsing for scene understanding. In Proceedings of the European conference on computer vision (ECCV). 418–434.
[19]
Minghao Yin, Zhuliang Yao, Yue Cao, Xiu Li, Zheng Zhang, Stephen Lin, and Han Hu. 2020. Disentangled non-local neural networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XV 16. Springer, 191–207.
[20]
Changqian Yu, Jingbo Wang, Changxin Gao, Gang Yu, Chunhua Shen, and Nong Sang. 2020. Context prior for scene segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12416–12425.
[21]
Yuhui Yuan, Xilin Chen, and Jingdong Wang. 2020. Object-contextual representations for semantic segmentation. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. Springer, 173–190.
[22]
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid Scene Parsing Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
[23]
Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. 2017. Scene parsing through ade20k dataset. In Proceedings of the IEEE conference on computer vision and pattern recognition. 633–641.

Index Terms

  1. Class-Aware Feature Regularization for Semantic Segmentation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ICCPR '23: Proceedings of the 2023 12th International Conference on Computing and Pattern Recognition
    October 2023
    589 pages
    ISBN:9798400707988
    DOI:10.1145/3633637
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 28 February 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. class-aware regularization
    2. inter-class variation
    3. intra-class consistency
    4. semantic segmentation

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ICCPR 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 19
      Total Downloads
    • Downloads (Last 12 months)19
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media